2023-10-06T04:38:34.692 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T04:38:34.693 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299 --verbose --owner scheduled_yuriw@teuthology --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/orig.config.yaml 2023-10-06T04:38:34.738 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T04:38:34.823 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} email: yweinste@redhat.com first_in_suite: false job_id: '7412299' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKbo8AaA0SBNBZE0ptrEHo7x74gvdsBIRnPvtAxdvksWWeYJ1dFzKIZy7Vyzh+Fj35g42ByfopqfFPStxzQBIU= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGkRhRxkKOUwHgczTnlg0rpGLrnMsnPV1os71P+MSj+tcnfyIzqBkyl+OIJC6UJ7RsBIak0eR+BLBohXjbpT7s4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T04:38:34.823 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T04:38:34.824 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T04:38:34.824 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T04:38:34.826 INFO:teuthology.task.internal:Checking packages... 2023-10-06T04:38:34.850 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T04:38:34.850 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T04:38:34.850 INFO:teuthology.packaging:ref: None 2023-10-06T04:38:34.850 INFO:teuthology.packaging:tag: None 2023-10-06T04:38:34.850 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:38:34.850 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:38:34.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:38:34.989 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274.g68e53a4a 2023-10-06T04:38:34.991 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T04:38:34.999 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T04:38:34.999 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T04:38:35.051 INFO:teuthology.task.internal:Saving configuration 2023-10-06T04:38:35.068 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T04:38:35.083 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:38:35.108 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:33:21.072999', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKbo8AaA0SBNBZE0ptrEHo7x74gvdsBIRnPvtAxdvksWWeYJ1dFzKIZy7Vyzh+Fj35g42ByfopqfFPStxzQBIU='} 2023-10-06T04:38:35.134 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:33:21.071038', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGkRhRxkKOUwHgczTnlg0rpGLrnMsnPV1os71P+MSj+tcnfyIzqBkyl+OIJC6UJ7RsBIak0eR+BLBohXjbpT7s4='} 2023-10-06T04:38:35.134 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T04:38:35.144 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T04:38:35.145 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T04:38:35.145 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T04:38:35.242 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f803a20cb80>, signals=[15]) 2023-10-06T04:38:35.243 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T04:38:35.251 INFO:teuthology.task.internal:Opening connections... 2023-10-06T04:38:35.251 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:38:35.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:38:35.320 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:38:35.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:38:35.389 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T04:38:35.398 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-10-06T04:38:35.416 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-10-06T04:38:35.416 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-10-06T04:38:35.473 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T04:38:35.473 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T04:38:35.473 INFO:teuthology.orchestra.run.smithi003.stdout:ID="rhel" 2023-10-06T04:38:35.473 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="fedora" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8.4" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T04:38:35.474 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T04:38:35.475 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T04:38:35.475 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-10-06T04:38:35.495 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2023-10-06T04:38:35.512 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2023-10-06T04:38:35.513 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2023-10-06T04:38:35.568 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.4" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T04:38:35.569 INFO:teuthology.orchestra.run.smithi153.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T04:38:35.570 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:38:35.570 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:38:35.570 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:38:35.570 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T04:38:35.570 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T04:38:35.570 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T04:38:35.570 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2023-10-06T04:38:35.594 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T04:38:35.610 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T04:38:35.634 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T04:38:35.634 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:38:35.637 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:38:35.652 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T04:38:35.665 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T04:38:35.665 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:38:35.694 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:38:35.708 INFO:teuthology.orchestra.run.smithi003.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T04:38:35.711 INFO:teuthology.orchestra.run.smithi153.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T04:38:35.712 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T04:38:35.773 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T04:38:35.789 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-06T04:38:35.789 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-06T04:38:35.790 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:38:35.790 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:38:35.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:38:35.806 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2023-10-06T04:38:35.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:38:35.807 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-10-06T04:38:35.862 INFO:teuthology.orchestra.run.smithi153.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T04:38:35.862 INFO:teuthology.task.kernel:Running kernel on smithi153: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T04:38:35.862 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum install -y kernel 2023-10-06T04:38:35.866 INFO:teuthology.orchestra.run.smithi003.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T04:38:35.867 INFO:teuthology.task.kernel:Running kernel on smithi003: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T04:38:35.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2023-10-06T04:38:37.214 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-10-06T04:38:37.214 INFO:teuthology.orchestra.run.smithi153.stdout:Unable to read consumer identity 2023-10-06T04:38:37.327 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-10-06T04:38:37.328 INFO:teuthology.orchestra.run.smithi003.stdout:Unable to read consumer identity 2023-10-06T04:38:38.570 INFO:teuthology.orchestra.run.smithi153.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-10-06T04:38:38.712 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-06T04:38:39.087 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 33 MB/s | 16 MB 00:00 2023-10-06T04:38:40.223 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-10-06T04:38:42.365 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-06T04:38:43.058 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T04:38:43.058 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T04:38:43.059 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T04:38:43.207 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-10-06T04:38:43.208 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2023-10-06T04:38:43.208 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-10-06T04:38:43.309 DEBUG:teuthology.orchestra.run.smithi153:> echo no | sudo yum reinstall kernel || true 2023-10-06T04:38:43.653 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-10-06T04:38:43.653 INFO:teuthology.orchestra.run.smithi153.stdout:Unable to read consumer identity 2023-10-06T04:38:43.690 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-06T04:38:43.944 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:01 ago on Fri 06 Oct 2023 04:38:42 AM UTC. 2023-10-06T04:38:44.103 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T04:38:44.103 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:44.103 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:44.123 INFO:teuthology.orchestra.run.smithi153.stderr:Error: No packages marked for reinstall. 2023-10-06T04:38:44.174 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum reinstall -y kernel || true 2023-10-06T04:38:44.416 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T04:38:44.416 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T04:38:44.416 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T04:38:44.518 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-10-06T04:38:44.518 INFO:teuthology.orchestra.run.smithi153.stdout:Unable to read consumer identity 2023-10-06T04:38:44.564 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-10-06T04:38:44.565 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2023-10-06T04:38:44.565 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-10-06T04:38:44.675 DEBUG:teuthology.orchestra.run.smithi003:> echo no | sudo yum reinstall kernel || true 2023-10-06T04:38:44.798 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 04:38:42 AM UTC. 2023-10-06T04:38:44.935 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T04:38:44.936 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:44.936 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:44.950 INFO:teuthology.orchestra.run.smithi153.stderr:Error: No packages marked for reinstall. 2023-10-06T04:38:45.006 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:38:45.042 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-10-06T04:38:45.042 INFO:teuthology.orchestra.run.smithi003.stdout:Unable to read consumer identity 2023-10-06T04:38:45.131 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:45.131 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:45.131 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:45.131 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T04:38:45.132 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T04:38:45.132 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-06T04:38:45.132 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi153.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T04:38:45.132 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:38:45.256 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:45.257 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:45.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo rpm -qi grub2-tools 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Name : grub2-tools 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Epoch : 1 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Version : 2.02 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Release : 99.el8_4.1 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: x86_64 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Group : System Environment/Base 2023-10-06T04:38:45.284 INFO:teuthology.orchestra.run.smithi153.stdout:Size : 9264631 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:License : GPLv3+ 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Relocations : (not relocatable) 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Packager : Red Hat, Inc. 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Vendor : Red Hat, Inc. 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Summary : Support tools for GRUB. 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout:Description : 2023-10-06T04:38:45.285 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:38:45.286 INFO:teuthology.orchestra.run.smithi153.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T04:38:45.286 INFO:teuthology.orchestra.run.smithi153.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T04:38:45.286 INFO:teuthology.orchestra.run.smithi153.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T04:38:45.286 INFO:teuthology.orchestra.run.smithi153.stdout:hardware devices. 2023-10-06T04:38:45.286 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:38:45.286 INFO:teuthology.orchestra.run.smithi153.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T04:38:45.287 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T04:38:45.287 INFO:teuthology.task.kernel:Updating grub on smithi153 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:45.287 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T04:38:45.349 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 04:38:43 AM UTC. 2023-10-06T04:38:45.478 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T04:38:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:45.498 INFO:teuthology.orchestra.run.smithi003.stderr:Error: No packages marked for reinstall. 2023-10-06T04:38:45.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum reinstall -y kernel || true 2023-10-06T04:38:45.914 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-10-06T04:38:45.914 INFO:teuthology.orchestra.run.smithi003.stdout:Unable to read consumer identity 2023-10-06T04:38:46.207 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:00:03 ago on Fri 06 Oct 2023 04:38:43 AM UTC. 2023-10-06T04:38:46.345 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T04:38:46.346 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:46.346 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T04:38:46.356 INFO:teuthology.orchestra.run.smithi003.stderr:Error: No packages marked for reinstall. 2023-10-06T04:38:46.406 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:38:46.537 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:46.537 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:46.537 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:46.537 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T04:38:46.537 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T04:38:46.537 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-06T04:38:46.537 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T04:38:46.538 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:38:46.667 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:46.668 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:46.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2023-10-06T04:38:46.680 INFO:teuthology.orchestra.run.smithi153.stderr:Generating grub configuration file ... 2023-10-06T04:38:46.696 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2023-10-06T04:38:46.696 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2023-10-06T04:38:46.696 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.02 2023-10-06T04:38:46.696 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 99.el8_4.1 2023-10-06T04:38:46.696 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2023-10-06T04:38:46.696 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T04:38:46.696 INFO:teuthology.orchestra.run.smithi003.stdout:Group : System Environment/Base 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 9264631 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Relocations : (not relocatable) 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : Red Hat, Inc. 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : Red Hat, Inc. 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2023-10-06T04:38:46.697 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2023-10-06T04:38:46.698 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:38:46.698 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T04:38:46.698 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T04:38:46.698 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T04:38:46.698 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2023-10-06T04:38:46.698 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:38:46.698 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T04:38:46.699 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T04:38:46.699 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:46.699 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T04:38:48.039 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2023-10-06T04:38:48.533 INFO:teuthology.orchestra.run.smithi153.stderr:done 2023-10-06T04:38:48.536 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2023-10-06T04:38:48.550 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.l1lVY7SAVP 2023-10-06T04:38:48.550 DEBUG:teuthology.orchestra.run.smithi153:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.l1lVY7SAVP 2023-10-06T04:38:48.621 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.l1lVY7SAVP 2023-10-06T04:38:48.775 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.l1lVY7SAVP is 6KB 2023-10-06T04:38:48.824 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.l1lVY7SAVP 2023-10-06T04:38:48.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T04:38:48.908 INFO:teuthology.orchestra.run.smithi153.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T04:38:48.908 INFO:teuthology.orchestra.run.smithi153.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T04:38:48.908 INFO:teuthology.orchestra.run.smithi153.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T04:38:48.908 INFO:teuthology.orchestra.run.smithi153.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T04:38:48.909 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:49.080 DEBUG:teuthology.orchestra.run.smithi153:> sudo shutdown -r now 2023-10-06T04:38:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:done 2023-10-06T04:38:49.867 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2023-10-06T04:38:49.922 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.k258u6VbAy 2023-10-06T04:38:49.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.k258u6VbAy 2023-10-06T04:38:49.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.k258u6VbAy 2023-10-06T04:38:50.146 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.k258u6VbAy is 6KB 2023-10-06T04:38:50.193 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.k258u6VbAy 2023-10-06T04:38:50.207 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T04:38:50.277 INFO:teuthology.orchestra.run.smithi003.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T04:38:50.277 INFO:teuthology.orchestra.run.smithi003.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T04:38:50.277 INFO:teuthology.orchestra.run.smithi003.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T04:38:50.277 INFO:teuthology.orchestra.run.smithi003.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T04:38:50.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T04:38:50.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2023-10-06T04:39:19.110 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T04:39:19.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-10-06T04:39:19.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:39:20.492 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T04:39:20.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-06T04:39:20.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:39:53.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2023-10-06T04:40:02.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-10-06T04:40:02.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:40:20.510 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T04:40:21.114 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2023-10-06T04:40:29.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-06T04:40:29.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:40:29.935 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-10-06T04:40:30.260 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-06T04:40:30.261 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T04:40:30.261 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-10-06T04:40:30.277 INFO:teuthology.orchestra.run.smithi003.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T04:40:30.278 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:40:30.278 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T04:40:30.278 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T04:40:31.279 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T04:40:31.279 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T04:40:31.407 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2023-10-06T04:40:31.420 DEBUG:teuthology.parallel:result is None 2023-10-06T04:40:33.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-10-06T04:40:33.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:40:33.540 DEBUG:teuthology.orchestra.run.smithi153:> true 2023-10-06T04:40:33.897 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-10-06T04:40:33.897 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T04:40:33.897 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2023-10-06T04:40:33.911 INFO:teuthology.orchestra.run.smithi153.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T04:40:33.912 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T04:40:33.912 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T04:40:33.912 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T04:40:34.912 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T04:40:34.913 DEBUG:teuthology.orchestra.run.smithi153:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T04:40:35.090 INFO:teuthology.orchestra.run.smithi153.stdout:ttyS1 2023-10-06T04:40:35.102 DEBUG:teuthology.parallel:result is None 2023-10-06T04:40:35.102 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T04:40:35.109 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T04:40:35.109 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T04:40:35.112 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T04:40:35.127 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T04:40:35.138 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T04:40:35.170 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T04:40:35.171 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T04:40:35.175 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T04:40:35.214 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T04:40:35.221 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T04:40:35.222 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T04:40:35.241 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T04:40:35.282 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:40:35.294 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:40:35.299 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:40:35.311 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:40:35.312 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T04:40:35.320 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T04:40:35.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T04:40:35.341 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T04:40:35.369 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T04:40:35.378 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T04:40:35.378 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T04:40:35.415 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T04:40:35.429 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:40:35.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:40:35.567 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:40:35.650 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:40:35.696 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:40:35.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T04:40:35.765 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:40:35.806 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:40:35.902 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:40:35.943 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:40:35.986 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:40:35.987 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T04:40:36.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2023-10-06T04:40:36.064 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2023-10-06T04:40:36.122 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:40:36.154 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:40:36.377 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T04:40:36.385 INFO:teuthology.task.internal:Starting timer... 2023-10-06T04:40:36.385 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T04:40:36.460 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T04:40:36.498 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T04:40:36.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2023-10-06T04:40:36.558 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: [ OK ] 2023-10-06T04:40:36.559 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2023-10-06T04:40:36.671 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: [ OK ] 2023-10-06T04:40:36.673 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T04:40:36.673 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2023-10-06T04:40:36.706 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2023-10-06T04:40:36.707 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2023-10-06T04:40:36.731 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2023-10-06T04:40:36.731 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2023-10-06T04:40:36.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:40:36.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:40:36.776 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:40:36.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:40:36.801 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T04:40:36.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2023-10-06T04:40:36.845 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2023-10-06T04:40:36.878 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T04:40:36.887 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T04:40:36.887 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T04:40:36.903 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T04:40:36.904 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T04:49:08.265 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2023-10-06T04:49:08.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-06T04:49:08.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:49:08.336 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-10-06T04:49:08.401 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-06T04:49:08.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-10-06T04:49:08.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:49:08.472 DEBUG:teuthology.orchestra.run.smithi153:> true 2023-10-06T04:49:08.537 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-10-06T04:49:08.537 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T04:49:08.549 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T04:49:08.549 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T04:49:08.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:49:08.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T04:49:08.552 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:49:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T04:49:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T04:49:08.606 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T04:49:08.620 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T04:49:08.629 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2023-10-06T04:49:08.639 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2023-10-06T04:49:08.654 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2023-10-06T04:49:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T04:49:08.664 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2023-10-06T04:49:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T04:49:08.678 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T04:49:08.691 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T04:49:08.742 INFO:teuthology.orchestra.run.smithi003.stderr:bash: ntpq: command not found 2023-10-06T04:49:08.744 INFO:teuthology.orchestra.run.smithi003.stdout:210 Number of sources = 4 2023-10-06T04:49:08.744 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:49:08.745 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2023-10-06T04:49:08.745 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.745 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.745 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.745 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.779 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-10-06T04:49:08.782 INFO:teuthology.orchestra.run.smithi153.stdout:210 Number of sources = 4 2023-10-06T04:49:08.782 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:49:08.782 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-10-06T04:49:08.782 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.782 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.782 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.783 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:49:08.783 INFO:teuthology.run_tasks:Running task install... 2023-10-06T04:49:08.792 DEBUG:teuthology.task.install:project ceph 2023-10-06T04:49:08.792 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T04:49:08.792 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T04:49:08.792 INFO:teuthology.task.install:Using flavor: default 2023-10-06T04:49:08.797 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T04:49:08.798 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T04:49:08.798 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T04:49:08.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:49:08.800 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T04:49:08.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:49:08.944 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T04:49:08.944 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T04:49:08.949 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T04:49:08.949 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T04:49:09.052 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T04:49:09.052 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:49:09.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T04:49:09.079 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T04:49:09.079 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T04:49:09.115 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T04:49:09.115 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:49:09.115 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T04:49:09.141 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T04:49:09.141 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T04:49:09.145 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T04:49:09.206 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T04:49:09.220 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T04:49:09.280 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T04:49:09.285 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2023-10-06T04:49:09.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2023-10-06T04:49:09.344 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2023-10-06T04:49:09.346 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2023-10-06T04:49:09.695 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-10-06T04:49:09.737 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-10-06T04:49:10.017 INFO:teuthology.orchestra.run.smithi003.stdout:44 files removed 2023-10-06T04:49:10.065 INFO:teuthology.orchestra.run.smithi153.stdout:44 files removed 2023-10-06T04:49:10.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T04:49:10.121 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T04:49:10.440 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-10-06T04:49:10.473 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-10-06T04:49:11.224 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 281 kB/s | 78 kB 00:00 2023-10-06T04:49:11.276 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 250 kB/s | 78 kB 00:00 2023-10-06T04:49:11.505 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 49 kB/s | 9.2 kB 00:00 2023-10-06T04:49:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-06T04:49:11.784 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 5.8 kB/s | 1.4 kB 00:00 2023-10-06T04:49:11.901 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-10-06T04:49:11.905 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 38 kB/s | 3.5 kB 00:00 2023-10-06T04:49:12.009 INFO:teuthology.orchestra.run.smithi153.stdout:Copr repo for python3-asyncssh owned by ceph 44 kB/s | 3.5 kB 00:00 2023-10-06T04:49:12.582 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-10-06T04:49:13.609 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-10-06T04:49:15.993 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-10-06T04:49:16.385 INFO:teuthology.orchestra.run.smithi003.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 108 MB/s | 33 MB 00:00 2023-10-06T04:49:16.914 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 3.6 MB/s | 24 kB 00:00 2023-10-06T04:49:17.313 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 120 MB/s | 38 MB 00:00 2023-10-06T04:49:21.864 INFO:teuthology.orchestra.run.smithi003.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 103 MB/s | 38 MB 00:00 2023-10-06T04:49:22.903 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 110 MB/s | 33 MB 00:00 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:49:30.873 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T04:49:30.874 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T04:49:30.875 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T04:49:30.876 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T04:49:30.877 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T04:49:30.880 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T04:49:30.881 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T04:49:31.177 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:49:31.178 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:49:31.189 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T04:49:31.190 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T04:49:31.191 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T04:49:31.191 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T04:49:31.193 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T04:49:31.196 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T04:49:31.197 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T04:49:31.370 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-10-06T04:49:31.381 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================================================== 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================================================== 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T04:49:31.382 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T04:49:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T04:49:31.384 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T04:49:31.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T04:49:31.386 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T04:49:31.387 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T04:49:31.388 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T04:49:31.389 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T04:49:31.390 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T04:49:31.391 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================================================== 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout:Install 118 Packages 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 2 Packages 2023-10-06T04:49:31.392 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:49:31.393 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 201 M 2023-10-06T04:49:31.393 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2023-10-06T04:49:31.626 INFO:teuthology.orchestra.run.smithi153.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 71 kB/s | 6.4 kB 00:00 2023-10-06T04:49:31.718 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-10-06T04:49:31.730 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================================================== 2023-10-06T04:49:31.730 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2023-10-06T04:49:31.730 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================================================== 2023-10-06T04:49:31.730 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2023-10-06T04:49:31.730 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi003.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T04:49:31.735 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T04:49:31.736 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================================================== 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout:Install 118 Packages 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 2 Packages 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 201 M 2023-10-06T04:49:31.742 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-10-06T04:49:31.772 INFO:teuthology.orchestra.run.smithi153.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 5.9 MB/s | 877 kB 00:00 2023-10-06T04:49:31.794 INFO:teuthology.orchestra.run.smithi153.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 7.2 MB/s | 162 kB 00:00 2023-10-06T04:49:31.947 INFO:teuthology.orchestra.run.smithi153.stdout:(4/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 15 MB/s | 2.3 MB 00:00 2023-10-06T04:49:31.956 INFO:teuthology.orchestra.run.smithi003.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 44 kB/s | 6.4 kB 00:00 2023-10-06T04:49:32.073 INFO:teuthology.orchestra.run.smithi153.stdout:(5/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 12 MB/s | 1.6 MB 00:00 2023-10-06T04:49:32.298 INFO:teuthology.orchestra.run.smithi153.stdout:(6/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 6.8 MB/s | 5.2 MB 00:00 2023-10-06T04:49:32.395 INFO:teuthology.orchestra.run.smithi153.stdout:(7/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 12 MB/s | 3.9 MB 00:00 2023-10-06T04:49:32.697 INFO:teuthology.orchestra.run.smithi003.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.2 MB/s | 877 kB 00:00 2023-10-06T04:49:32.825 INFO:teuthology.orchestra.run.smithi003.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 1.2 MB/s | 162 kB 00:00 2023-10-06T04:49:33.906 INFO:teuthology.orchestra.run.smithi003.stdout:(4/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-10-06T04:49:34.177 INFO:teuthology.orchestra.run.smithi003.stdout:(5/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 5.8 MB/s | 1.6 MB 00:00 2023-10-06T04:49:34.191 INFO:teuthology.orchestra.run.smithi153.stdout:(8/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el8 9.3 MB/s | 17 MB 00:01 2023-10-06T04:49:34.307 INFO:teuthology.orchestra.run.smithi153.stdout:(9/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x86 8.3 MB/s | 17 MB 00:02 2023-10-06T04:49:34.307 INFO:teuthology.orchestra.run.smithi153.stdout:(10/120): ceph-selinux-17.2.6-1274.g68e53a4a.el 209 kB/s | 24 kB 00:00 2023-10-06T04:49:34.383 INFO:teuthology.orchestra.run.smithi153.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 3.3 MB/s | 255 kB 00:00 2023-10-06T04:49:34.447 INFO:teuthology.orchestra.run.smithi153.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 395 kB/s | 25 kB 00:00 2023-10-06T04:49:34.583 INFO:teuthology.orchestra.run.smithi153.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 5.7 MB/s | 797 kB 00:00 2023-10-06T04:49:34.634 INFO:teuthology.orchestra.run.smithi003.stdout:(6/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 8.6 MB/s | 3.9 MB 00:00 2023-10-06T04:49:34.647 INFO:teuthology.orchestra.run.smithi153.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 2.7 MB/s | 175 kB 00:00 2023-10-06T04:49:34.713 INFO:teuthology.orchestra.run.smithi153.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 1.9 MB/s | 130 kB 00:00 2023-10-06T04:49:34.794 INFO:teuthology.orchestra.run.smithi153.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 6.2 MB/s | 506 kB 00:00 2023-10-06T04:49:35.762 INFO:teuthology.orchestra.run.smithi153.stdout:(17/120): ceph-common-17.2.6-1274.g68e53a4a.el8 5.5 MB/s | 23 MB 00:04 2023-10-06T04:49:35.858 INFO:teuthology.orchestra.run.smithi153.stdout:(18/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 5.2 MB/s | 5.6 MB 00:01 2023-10-06T04:49:35.860 INFO:teuthology.orchestra.run.smithi153.stdout:(19/120): python3-ceph-argparse-17.2.6-1274.g68 458 kB/s | 45 kB 00:00 2023-10-06T04:49:35.884 INFO:teuthology.orchestra.run.smithi153.stdout:(20/120): python3-ceph-common-17.2.6-1274.g68e5 4.5 MB/s | 115 kB 00:00 2023-10-06T04:49:35.907 INFO:teuthology.orchestra.run.smithi153.stdout:(21/120): python3-cephfs-17.2.6-1274.g68e53a4a. 4.0 MB/s | 190 kB 00:00 2023-10-06T04:49:35.937 INFO:teuthology.orchestra.run.smithi153.stdout:(22/120): python3-rados-17.2.6-1274.g68e53a4a.e 6.9 MB/s | 371 kB 00:00 2023-10-06T04:49:35.977 INFO:teuthology.orchestra.run.smithi153.stdout:(23/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 5.0 MB/s | 358 kB 00:00 2023-10-06T04:49:35.990 INFO:teuthology.orchestra.run.smithi153.stdout:(24/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 2.0 MB/s | 107 kB 00:00 2023-10-06T04:49:36.028 INFO:teuthology.orchestra.run.smithi153.stdout:(25/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.7 MB/s | 91 kB 00:00 2023-10-06T04:49:36.096 INFO:teuthology.orchestra.run.smithi153.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 2.5 MB/s | 173 kB 00:00 2023-10-06T04:49:36.158 INFO:teuthology.orchestra.run.smithi153.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 369 kB/s | 23 kB 00:00 2023-10-06T04:49:36.226 INFO:teuthology.orchestra.run.smithi153.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 1.8 MB/s | 121 kB 00:00 2023-10-06T04:49:36.476 INFO:teuthology.orchestra.run.smithi153.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 6.8 MB/s | 1.7 MB 00:00 2023-10-06T04:49:36.490 INFO:teuthology.orchestra.run.smithi153.stdout:(30/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 6.9 MB/s | 3.5 MB 00:00 2023-10-06T04:49:36.548 INFO:teuthology.orchestra.run.smithi153.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 4.1 MB/s | 238 kB 00:00 2023-10-06T04:49:36.605 INFO:teuthology.orchestra.run.smithi153.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 824 kB/s | 47 kB 00:00 2023-10-06T04:49:36.656 INFO:teuthology.orchestra.run.smithi153.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 289 kB/s | 14 kB 00:00 2023-10-06T04:49:36.714 INFO:teuthology.orchestra.run.smithi153.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 4.5 MB/s | 267 kB 00:00 2023-10-06T04:49:36.775 INFO:teuthology.orchestra.run.smithi153.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-06T04:49:36.827 INFO:teuthology.orchestra.run.smithi153.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 342 kB/s | 18 kB 00:00 2023-10-06T04:49:37.036 INFO:teuthology.orchestra.run.smithi153.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-10-06T04:49:37.071 INFO:teuthology.orchestra.run.smithi153.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-10-06T04:49:37.107 INFO:teuthology.orchestra.run.smithi153.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 2.1 MB/s | 75 kB 00:00 2023-10-06T04:49:37.178 INFO:teuthology.orchestra.run.smithi153.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 6.5 MB/s | 476 kB 00:00 2023-10-06T04:49:37.212 INFO:teuthology.orchestra.run.smithi153.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-10-06T04:49:37.259 INFO:teuthology.orchestra.run.smithi153.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 3.9 MB/s | 185 kB 00:00 2023-10-06T04:49:37.290 INFO:teuthology.orchestra.run.smithi153.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.0 MB/s | 33 kB 00:00 2023-10-06T04:49:37.321 INFO:teuthology.orchestra.run.smithi153.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 526 kB/s | 16 kB 00:00 2023-10-06T04:49:37.354 INFO:teuthology.orchestra.run.smithi153.stdout:(45/120): ceph-mgr-diskprediction-local-17.2.6- 8.4 MB/s | 7.4 MB 00:00 2023-10-06T04:49:37.359 INFO:teuthology.orchestra.run.smithi153.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.6 MB/s | 173 kB 00:00 2023-10-06T04:49:37.393 INFO:teuthology.orchestra.run.smithi153.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 2.3 MB/s | 81 kB 00:00 2023-10-06T04:49:37.424 INFO:teuthology.orchestra.run.smithi153.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 361 kB/s | 11 kB 00:00 2023-10-06T04:49:37.437 INFO:teuthology.orchestra.run.smithi003.stdout:(7/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x86 6.0 MB/s | 17 MB 00:02 2023-10-06T04:49:37.455 INFO:teuthology.orchestra.run.smithi153.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 569 kB/s | 18 kB 00:00 2023-10-06T04:49:37.526 INFO:teuthology.orchestra.run.smithi153.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 17 MB/s | 1.2 MB 00:00 2023-10-06T04:49:37.558 INFO:teuthology.orchestra.run.smithi153.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.5 MB/s | 49 kB 00:00 2023-10-06T04:49:37.574 INFO:teuthology.orchestra.run.smithi153.stdout:(52/120): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2023-10-06T04:49:37.587 INFO:teuthology.orchestra.run.smithi003.stdout:(8/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 915 kB/s | 5.2 MB 00:05 2023-10-06T04:49:37.589 INFO:teuthology.orchestra.run.smithi153.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.8 MB/s | 59 kB 00:00 2023-10-06T04:49:37.608 INFO:teuthology.orchestra.run.smithi153.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.7 MB/s | 60 kB 00:00 2023-10-06T04:49:37.624 INFO:teuthology.orchestra.run.smithi153.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 8.1 MB/s | 283 kB 00:00 2023-10-06T04:49:37.635 INFO:teuthology.orchestra.run.smithi003.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 507 kB/s | 24 kB 00:00 2023-10-06T04:49:37.656 INFO:teuthology.orchestra.run.smithi153.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 327 kB/s | 16 kB 00:00 2023-10-06T04:49:37.658 INFO:teuthology.orchestra.run.smithi153.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 988 kB/s | 34 kB 00:00 2023-10-06T04:49:37.689 INFO:teuthology.orchestra.run.smithi153.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 2.0 MB/s | 64 kB 00:00 2023-10-06T04:49:37.701 INFO:teuthology.orchestra.run.smithi153.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 4.3 MB/s | 196 kB 00:00 2023-10-06T04:49:37.720 INFO:teuthology.orchestra.run.smithi153.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 605 kB/s | 19 kB 00:00 2023-10-06T04:49:37.742 INFO:teuthology.orchestra.run.smithi153.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 596 kB/s | 24 kB 00:00 2023-10-06T04:49:37.752 INFO:teuthology.orchestra.run.smithi153.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 945 kB/s | 29 kB 00:00 2023-10-06T04:49:37.773 INFO:teuthology.orchestra.run.smithi153.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 889 kB/s | 27 kB 00:00 2023-10-06T04:49:37.786 INFO:teuthology.orchestra.run.smithi153.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 7.4 MB/s | 255 kB 00:00 2023-10-06T04:49:37.818 INFO:teuthology.orchestra.run.smithi153.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 5.5 MB/s | 251 kB 00:00 2023-10-06T04:49:37.818 INFO:teuthology.orchestra.run.smithi153.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.8 MB/s | 61 kB 00:00 2023-10-06T04:49:37.849 INFO:teuthology.orchestra.run.smithi153.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 746 kB/s | 23 kB 00:00 2023-10-06T04:49:37.862 INFO:teuthology.orchestra.run.smithi153.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 1.9 MB/s | 85 kB 00:00 2023-10-06T04:49:37.866 INFO:teuthology.orchestra.run.smithi153.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-10-06T04:49:37.894 INFO:teuthology.orchestra.run.smithi153.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 675 kB/s | 19 kB 00:00 2023-10-06T04:49:37.908 INFO:teuthology.orchestra.run.smithi153.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.1 MB/s | 43 kB 00:00 2023-10-06T04:49:37.923 INFO:teuthology.orchestra.run.smithi153.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-10-06T04:49:37.929 INFO:teuthology.orchestra.run.smithi153.stdout:(73/120): thrift-0.13.0-2.el8.x86_64.rpm 22 MB/s | 1.7 MB 00:00 2023-10-06T04:49:37.944 INFO:teuthology.orchestra.run.smithi153.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 10 MB/s | 220 kB 00:00 2023-10-06T04:49:37.956 INFO:teuthology.orchestra.run.smithi153.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 1.7 MB/s | 48 kB 00:00 2023-10-06T04:49:37.959 INFO:teuthology.orchestra.run.smithi153.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-10-06T04:49:37.969 INFO:teuthology.orchestra.run.smithi153.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 3.2 MB/s | 43 kB 00:00 2023-10-06T04:49:37.984 INFO:teuthology.orchestra.run.smithi153.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.2 MB/s | 18 kB 00:00 2023-10-06T04:49:38.001 INFO:teuthology.orchestra.run.smithi153.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 56 MB/s | 2.3 MB 00:00 2023-10-06T04:49:38.008 INFO:teuthology.orchestra.run.smithi153.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 25 MB/s | 605 kB 00:00 2023-10-06T04:49:38.017 INFO:teuthology.orchestra.run.smithi153.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 5.2 MB/s | 83 kB 00:00 2023-10-06T04:49:38.022 INFO:teuthology.orchestra.run.smithi153.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 4.3 MB/s | 59 kB 00:00 2023-10-06T04:49:38.031 INFO:teuthology.orchestra.run.smithi153.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-10-06T04:49:38.035 INFO:teuthology.orchestra.run.smithi153.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.8 MB/s | 38 kB 00:00 2023-10-06T04:49:38.048 INFO:teuthology.orchestra.run.smithi153.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.1 MB/s | 86 kB 00:00 2023-10-06T04:49:38.055 INFO:teuthology.orchestra.run.smithi153.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.9 MB/s | 58 kB 00:00 2023-10-06T04:49:38.064 INFO:teuthology.orchestra.run.smithi153.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.0 MB/s | 47 kB 00:00 2023-10-06T04:49:38.077 INFO:teuthology.orchestra.run.smithi153.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 26 MB/s | 559 kB 00:00 2023-10-06T04:49:38.087 INFO:teuthology.orchestra.run.smithi153.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 27 MB/s | 639 kB 00:00 2023-10-06T04:49:38.091 INFO:teuthology.orchestra.run.smithi153.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.7 MB/s | 70 kB 00:00 2023-10-06T04:49:38.105 INFO:teuthology.orchestra.run.smithi153.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 11 MB/s | 200 kB 00:00 2023-10-06T04:49:38.112 INFO:teuthology.orchestra.run.smithi153.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 4.6 MB/s | 97 kB 00:00 2023-10-06T04:49:38.129 INFO:teuthology.orchestra.run.smithi153.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 2.5 MB/s | 44 kB 00:00 2023-10-06T04:49:38.142 INFO:teuthology.orchestra.run.smithi153.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.0 MB/s | 27 kB 00:00 2023-10-06T04:49:38.158 INFO:teuthology.orchestra.run.smithi153.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-06T04:49:38.178 INFO:teuthology.orchestra.run.smithi153.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 15 MB/s | 320 kB 00:00 2023-10-06T04:49:38.197 INFO:teuthology.orchestra.run.smithi153.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 24 MB/s | 457 kB 00:00 2023-10-06T04:49:38.280 INFO:teuthology.orchestra.run.smithi153.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 58 MB/s | 4.8 MB 00:00 2023-10-06T04:49:38.395 INFO:teuthology.orchestra.run.smithi153.stdout:(99/120): python3-scipy-1.0.0-20.module+el8.1.0 50 MB/s | 15 MB 00:00 2023-10-06T04:49:38.396 INFO:teuthology.orchestra.run.smithi153.stdout:(100/120): python3-markupsafe-0.23-19.el8.x86_6 335 kB/s | 39 kB 00:00 2023-10-06T04:49:38.413 INFO:teuthology.orchestra.run.smithi153.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-06T04:49:38.418 INFO:teuthology.orchestra.run.smithi153.stdout:(102/120): bison-3.0.4-10.el8.x86_64.rpm 29 MB/s | 688 kB 00:00 2023-10-06T04:49:38.429 INFO:teuthology.orchestra.run.smithi153.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 14 MB/s | 225 kB 00:00 2023-10-06T04:49:38.433 INFO:teuthology.orchestra.run.smithi153.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.1 MB/s | 126 kB 00:00 2023-10-06T04:49:38.449 INFO:teuthology.orchestra.run.smithi153.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-10-06T04:49:38.470 INFO:teuthology.orchestra.run.smithi153.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 26 MB/s | 538 kB 00:00 2023-10-06T04:49:38.491 INFO:teuthology.orchestra.run.smithi153.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 60 MB/s | 3.7 MB 00:00 2023-10-06T04:49:38.495 INFO:teuthology.orchestra.run.smithi153.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 12 MB/s | 302 kB 00:00 2023-10-06T04:49:38.509 INFO:teuthology.orchestra.run.smithi153.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 8.7 MB/s | 157 kB 00:00 2023-10-06T04:49:38.515 INFO:teuthology.orchestra.run.smithi153.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 5.4 MB/s | 110 kB 00:00 2023-10-06T04:49:38.527 INFO:teuthology.orchestra.run.smithi153.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-06T04:49:38.529 INFO:teuthology.orchestra.run.smithi153.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.5 MB/s | 20 kB 00:00 2023-10-06T04:49:38.542 INFO:teuthology.orchestra.run.smithi153.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.9 MB/s | 54 kB 00:00 2023-10-06T04:49:38.549 INFO:teuthology.orchestra.run.smithi153.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 32 MB/s | 709 kB 00:00 2023-10-06T04:49:38.624 INFO:teuthology.orchestra.run.smithi153.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 58 MB/s | 4.7 MB 00:00 2023-10-06T04:49:38.642 INFO:teuthology.orchestra.run.smithi153.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 49 MB/s | 4.6 MB 00:00 2023-10-06T04:49:38.650 INFO:teuthology.orchestra.run.smithi153.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 27 MB/s | 713 kB 00:00 2023-10-06T04:49:39.386 INFO:teuthology.orchestra.run.smithi003.stdout:(10/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el 8.6 MB/s | 17 MB 00:01 2023-10-06T04:49:39.412 INFO:teuthology.orchestra.run.smithi003.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 9.7 MB/s | 255 kB 00:00 2023-10-06T04:49:39.433 INFO:teuthology.orchestra.run.smithi003.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 1.2 MB/s | 25 kB 00:00 2023-10-06T04:49:39.473 INFO:teuthology.orchestra.run.smithi153.stdout:(118/120): ceph-test-17.2.6-1274.g68e53a4a.el8. 10 MB/s | 52 MB 00:05 2023-10-06T04:49:39.490 INFO:teuthology.orchestra.run.smithi153.stdout:(119/120): librados2-17.2.6-1274.g68e53a4a.el8. 4.0 MB/s | 3.4 MB 00:00 2023-10-06T04:49:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 12 MB/s | 797 kB 00:00 2023-10-06T04:49:39.504 INFO:teuthology.orchestra.run.smithi153.stdout:(120/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 4.2 MB/s | 3.5 MB 00:00 2023-10-06T04:49:39.507 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:49:39.507 INFO:teuthology.orchestra.run.smithi153.stdout:Total 25 MB/s | 201 MB 00:08 2023-10-06T04:49:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 7.6 MB/s | 175 kB 00:00 2023-10-06T04:49:39.543 INFO:teuthology.orchestra.run.smithi003.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 5.7 MB/s | 130 kB 00:00 2023-10-06T04:49:39.556 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2023-10-06T04:49:39.588 INFO:teuthology.orchestra.run.smithi003.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 11 MB/s | 506 kB 00:00 2023-10-06T04:49:39.669 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2023-10-06T04:49:39.669 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2023-10-06T04:49:39.939 INFO:teuthology.orchestra.run.smithi003.stdout:(17/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 16 MB/s | 5.6 MB 00:00 2023-10-06T04:49:39.960 INFO:teuthology.orchestra.run.smithi003.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 2.0 MB/s | 45 kB 00:00 2023-10-06T04:49:39.983 INFO:teuthology.orchestra.run.smithi003.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 5.1 MB/s | 115 kB 00:00 2023-10-06T04:49:40.005 INFO:teuthology.orchestra.run.smithi003.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 8.4 MB/s | 190 kB 00:00 2023-10-06T04:49:40.048 INFO:teuthology.orchestra.run.smithi003.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 8.5 MB/s | 371 kB 00:00 2023-10-06T04:49:40.090 INFO:teuthology.orchestra.run.smithi003.stdout:(22/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 8.3 MB/s | 358 kB 00:00 2023-10-06T04:49:40.112 INFO:teuthology.orchestra.run.smithi003.stdout:(23/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 4.8 MB/s | 107 kB 00:00 2023-10-06T04:49:40.134 INFO:teuthology.orchestra.run.smithi003.stdout:(24/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 4.1 MB/s | 91 kB 00:00 2023-10-06T04:49:40.408 INFO:teuthology.orchestra.run.smithi003.stdout:(25/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 13 MB/s | 3.5 MB 00:00 2023-10-06T04:49:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 7.4 MB/s | 173 kB 00:00 2023-10-06T04:49:40.453 INFO:teuthology.orchestra.run.smithi003.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 1.0 MB/s | 23 kB 00:00 2023-10-06T04:49:40.475 INFO:teuthology.orchestra.run.smithi003.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 5.4 MB/s | 121 kB 00:00 2023-10-06T04:49:40.781 INFO:teuthology.orchestra.run.smithi003.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 5.5 MB/s | 1.7 MB 00:00 2023-10-06T04:49:41.467 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2023-10-06T04:49:41.471 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2023-10-06T04:49:42.031 INFO:teuthology.orchestra.run.smithi003.stdout:(30/120): ceph-common-17.2.6-1274.g68e53a4a.el8 2.3 MB/s | 23 MB 00:10 2023-10-06T04:49:42.200 INFO:teuthology.orchestra.run.smithi003.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 1.4 MB/s | 238 kB 00:00 2023-10-06T04:49:42.285 INFO:teuthology.orchestra.run.smithi003.stdout:(32/120): ceph-mgr-diskprediction-local-17.2.6- 4.9 MB/s | 7.4 MB 00:01 2023-10-06T04:49:42.286 INFO:teuthology.orchestra.run.smithi003.stdout:(33/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 550 kB/s | 47 kB 00:00 2023-10-06T04:49:42.341 INFO:teuthology.orchestra.run.smithi003.stdout:(34/120): ceph-prometheus-alerts-17.2.6-1274.g6 264 kB/s | 14 kB 00:00 2023-10-06T04:49:42.399 INFO:teuthology.orchestra.run.smithi003.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-06T04:49:42.431 INFO:teuthology.orchestra.run.smithi003.stdout:(36/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 1.8 MB/s | 267 kB 00:00 2023-10-06T04:49:42.443 INFO:teuthology.orchestra.run.smithi003.stdout:(37/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 401 kB/s | 18 kB 00:00 2023-10-06T04:49:42.676 INFO:teuthology.orchestra.run.smithi003.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 253 kB/s | 59 kB 00:00 2023-10-06T04:49:42.731 INFO:teuthology.orchestra.run.smithi003.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-10-06T04:49:42.747 INFO:teuthology.orchestra.run.smithi003.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 970 kB/s | 306 kB 00:00 2023-10-06T04:49:42.805 INFO:teuthology.orchestra.run.smithi003.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 768 kB/s | 44 kB 00:00 2023-10-06T04:49:42.852 INFO:teuthology.orchestra.run.smithi003.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 3.8 MB/s | 185 kB 00:00 2023-10-06T04:49:42.887 INFO:teuthology.orchestra.run.smithi003.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 948 kB/s | 33 kB 00:00 2023-10-06T04:49:42.902 INFO:teuthology.orchestra.run.smithi003.stdout:(44/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-10-06T04:49:42.921 INFO:teuthology.orchestra.run.smithi003.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 483 kB/s | 16 kB 00:00 2023-10-06T04:49:43.092 INFO:teuthology.orchestra.run.smithi003.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 911 kB/s | 173 kB 00:00 2023-10-06T04:49:43.130 INFO:teuthology.orchestra.run.smithi003.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 2.1 MB/s | 81 kB 00:00 2023-10-06T04:49:43.142 INFO:teuthology.orchestra.run.smithi003.stdout:(48/120): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2023-10-06T04:49:43.179 INFO:teuthology.orchestra.run.smithi003.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 232 kB/s | 11 kB 00:00 2023-10-06T04:49:43.182 INFO:teuthology.orchestra.run.smithi003.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 449 kB/s | 18 kB 00:00 2023-10-06T04:49:43.288 INFO:teuthology.orchestra.run.smithi003.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 457 kB/s | 49 kB 00:00 2023-10-06T04:49:43.303 INFO:teuthology.orchestra.run.smithi003.stdout:(52/120): python3-kubernetes-11.0.0-6.el8.noarc 9.4 MB/s | 1.2 MB 00:00 2023-10-06T04:49:43.325 INFO:teuthology.orchestra.run.smithi003.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.6 MB/s | 59 kB 00:00 2023-10-06T04:49:43.338 INFO:teuthology.orchestra.run.smithi003.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.7 MB/s | 60 kB 00:00 2023-10-06T04:49:43.390 INFO:teuthology.orchestra.run.smithi003.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 297 kB/s | 16 kB 00:00 2023-10-06T04:49:43.395 INFO:teuthology.orchestra.run.smithi003.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.9 MB/s | 283 kB 00:00 2023-10-06T04:49:43.424 INFO:teuthology.orchestra.run.smithi003.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-10-06T04:49:43.439 INFO:teuthology.orchestra.run.smithi003.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 4.4 MB/s | 196 kB 00:00 2023-10-06T04:49:43.459 INFO:teuthology.orchestra.run.smithi003.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 1.8 MB/s | 64 kB 00:00 2023-10-06T04:49:43.472 INFO:teuthology.orchestra.run.smithi003.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 562 kB/s | 19 kB 00:00 2023-10-06T04:49:43.497 INFO:teuthology.orchestra.run.smithi003.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 642 kB/s | 24 kB 00:00 2023-10-06T04:49:43.506 INFO:teuthology.orchestra.run.smithi003.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 867 kB/s | 29 kB 00:00 2023-10-06T04:49:43.530 INFO:teuthology.orchestra.run.smithi003.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 823 kB/s | 27 kB 00:00 2023-10-06T04:49:43.554 INFO:teuthology.orchestra.run.smithi003.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 5.3 MB/s | 255 kB 00:00 2023-10-06T04:49:43.572 INFO:teuthology.orchestra.run.smithi003.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 5.9 MB/s | 251 kB 00:00 2023-10-06T04:49:43.589 INFO:teuthology.orchestra.run.smithi003.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.7 MB/s | 61 kB 00:00 2023-10-06T04:49:43.609 INFO:teuthology.orchestra.run.smithi003.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 2.3 MB/s | 85 kB 00:00 2023-10-06T04:49:43.623 INFO:teuthology.orchestra.run.smithi003.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 689 kB/s | 23 kB 00:00 2023-10-06T04:49:43.627 INFO:teuthology.orchestra.run.smithi003.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 16 MB/s | 70 kB 00:00 2023-10-06T04:49:43.751 INFO:teuthology.orchestra.run.smithi003.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-10-06T04:49:43.784 INFO:teuthology.orchestra.run.smithi003.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-06T04:49:43.796 INFO:teuthology.orchestra.run.smithi003.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.2 MB/s | 27 kB 00:00 2023-10-06T04:49:43.811 INFO:teuthology.orchestra.run.smithi003.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-06T04:49:43.828 INFO:teuthology.orchestra.run.smithi003.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 18 MB/s | 320 kB 00:00 2023-10-06T04:49:43.849 INFO:teuthology.orchestra.run.smithi003.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 22 MB/s | 457 kB 00:00 2023-10-06T04:49:43.965 INFO:teuthology.orchestra.run.smithi003.stdout:(76/120): python3-scipy-1.0.0-20.module+el8.1.0 43 MB/s | 15 MB 00:00 2023-10-06T04:49:43.981 INFO:teuthology.orchestra.run.smithi003.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.4 MB/s | 39 kB 00:00 2023-10-06T04:49:44.004 INFO:teuthology.orchestra.run.smithi003.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 30 MB/s | 688 kB 00:00 2023-10-06T04:49:44.021 INFO:teuthology.orchestra.run.smithi003.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-06T04:49:44.035 INFO:teuthology.orchestra.run.smithi003.stdout:(80/120): python3-babel-2.5.1-5.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-10-06T04:49:44.038 INFO:teuthology.orchestra.run.smithi003.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 12 MB/s | 225 kB 00:00 2023-10-06T04:49:44.050 INFO:teuthology.orchestra.run.smithi003.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 8.7 MB/s | 126 kB 00:00 2023-10-06T04:49:44.066 INFO:teuthology.orchestra.run.smithi003.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-10-06T04:49:44.086 INFO:teuthology.orchestra.run.smithi003.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 26 MB/s | 538 kB 00:00 2023-10-06T04:49:44.106 INFO:teuthology.orchestra.run.smithi003.stdout:(85/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 55 MB/s | 3.7 MB 00:00 2023-10-06T04:49:44.110 INFO:teuthology.orchestra.run.smithi003.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 12 MB/s | 302 kB 00:00 2023-10-06T04:49:44.123 INFO:teuthology.orchestra.run.smithi003.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 9.2 MB/s | 157 kB 00:00 2023-10-06T04:49:44.130 INFO:teuthology.orchestra.run.smithi003.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.5 MB/s | 110 kB 00:00 2023-10-06T04:49:44.141 INFO:teuthology.orchestra.run.smithi003.stdout:(89/120): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-10-06T04:49:44.142 INFO:teuthology.orchestra.run.smithi003.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.6 MB/s | 20 kB 00:00 2023-10-06T04:49:44.156 INFO:teuthology.orchestra.run.smithi003.stdout:(91/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.0 MB/s | 54 kB 00:00 2023-10-06T04:49:44.164 INFO:teuthology.orchestra.run.smithi003.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-10-06T04:49:44.244 INFO:teuthology.orchestra.run.smithi003.stdout:(93/120): openblas-threads-0.3.12-1.el8.x86_64. 54 MB/s | 4.7 MB 00:00 2023-10-06T04:49:44.263 INFO:teuthology.orchestra.run.smithi003.stdout:(94/120): openblas-0.3.12-1.el8.x86_64.rpm 46 MB/s | 4.6 MB 00:00 2023-10-06T04:49:44.272 INFO:teuthology.orchestra.run.smithi003.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 25 MB/s | 713 kB 00:00 2023-10-06T04:49:44.278 INFO:teuthology.orchestra.run.smithi003.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.2 MB/s | 19 kB 00:00 2023-10-06T04:49:44.287 INFO:teuthology.orchestra.run.smithi003.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.9 MB/s | 43 kB 00:00 2023-10-06T04:49:44.293 INFO:teuthology.orchestra.run.smithi003.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-10-06T04:49:44.303 INFO:teuthology.orchestra.run.smithi003.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 14 MB/s | 220 kB 00:00 2023-10-06T04:49:44.307 INFO:teuthology.orchestra.run.smithi003.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 3.5 MB/s | 48 kB 00:00 2023-10-06T04:49:44.321 INFO:teuthology.orchestra.run.smithi003.stdout:(101/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 12 MB/s | 229 kB 00:00 2023-10-06T04:49:44.327 INFO:teuthology.orchestra.run.smithi003.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 2.1 MB/s | 43 kB 00:00 2023-10-06T04:49:44.339 INFO:teuthology.orchestra.run.smithi003.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.4 MB/s | 18 kB 00:00 2023-10-06T04:49:44.361 INFO:teuthology.orchestra.run.smithi003.stdout:(104/120): pcre2-devel-10.32-2.el8.x86_64.rpm 28 MB/s | 605 kB 00:00 2023-10-06T04:49:44.368 INFO:teuthology.orchestra.run.smithi003.stdout:(105/120): openssl-devel-1.1.1g-15.el8_3.x86_64 49 MB/s | 2.3 MB 00:00 2023-10-06T04:49:44.375 INFO:teuthology.orchestra.run.smithi003.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 6.0 MB/s | 83 kB 00:00 2023-10-06T04:49:44.382 INFO:teuthology.orchestra.run.smithi003.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 4.2 MB/s | 59 kB 00:00 2023-10-06T04:49:44.389 INFO:teuthology.orchestra.run.smithi003.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 12 MB/s | 169 kB 00:00 2023-10-06T04:49:44.395 INFO:teuthology.orchestra.run.smithi003.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.9 MB/s | 38 kB 00:00 2023-10-06T04:49:44.404 INFO:teuthology.orchestra.run.smithi003.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.8 MB/s | 86 kB 00:00 2023-10-06T04:49:44.412 INFO:teuthology.orchestra.run.smithi003.stdout:(111/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.5 MB/s | 58 kB 00:00 2023-10-06T04:49:44.422 INFO:teuthology.orchestra.run.smithi003.stdout:(112/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.6 MB/s | 47 kB 00:00 2023-10-06T04:49:44.436 INFO:teuthology.orchestra.run.smithi003.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 23 MB/s | 559 kB 00:00 2023-10-06T04:49:44.444 INFO:teuthology.orchestra.run.smithi003.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 28 MB/s | 639 kB 00:00 2023-10-06T04:49:44.452 INFO:teuthology.orchestra.run.smithi003.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.6 MB/s | 70 kB 00:00 2023-10-06T04:49:44.461 INFO:teuthology.orchestra.run.smithi003.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 11 MB/s | 200 kB 00:00 2023-10-06T04:49:44.465 INFO:teuthology.orchestra.run.smithi003.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 6.9 MB/s | 97 kB 00:00 2023-10-06T04:49:45.672 INFO:teuthology.orchestra.run.smithi003.stdout:(118/120): librados2-17.2.6-1274.g68e53a4a.el8. 2.8 MB/s | 3.4 MB 00:01 2023-10-06T04:49:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2023-10-06T04:49:46.461 INFO:teuthology.orchestra.run.smithi003.stdout:(119/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 1.8 MB/s | 3.5 MB 00:01 2023-10-06T04:49:46.527 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T04:49:46.748 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T04:49:46.874 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T04:49:47.098 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T04:49:47.506 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T04:49:47.740 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T04:49:48.136 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T04:49:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T04:49:48.490 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T04:49:48.706 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T04:49:48.808 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T04:49:49.036 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T04:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T04:49:50.107 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T04:49:50.273 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T04:49:50.408 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T04:49:50.663 INFO:teuthology.orchestra.run.smithi003.stdout:(120/120): ceph-test-17.2.6-1274.g68e53a4a.el8. 4.0 MB/s | 52 MB 00:13 2023-10-06T04:49:50.668 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T04:49:50.669 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:49:50.670 INFO:teuthology.orchestra.run.smithi003.stdout:Total 11 MB/s | 201 MB 00:18 2023-10-06T04:49:50.729 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-10-06T04:49:50.856 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-10-06T04:49:50.856 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-10-06T04:49:50.863 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T04:49:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T04:49:51.182 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T04:49:51.339 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T04:49:51.420 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T04:49:51.618 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T04:49:51.911 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T04:49:52.138 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T04:49:52.308 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T04:49:52.431 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T04:49:52.578 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T04:49:52.597 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-10-06T04:49:52.602 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-10-06T04:49:52.730 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T04:49:53.305 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T04:49:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T04:49:54.012 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T04:49:54.189 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T04:49:54.929 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T04:49:55.631 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T04:49:55.913 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T04:49:56.169 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T04:49:56.370 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T04:49:56.498 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T04:49:56.780 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T04:49:57.036 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T04:49:57.364 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T04:49:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-10-06T04:49:57.630 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T04:49:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T04:49:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T04:49:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T04:49:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T04:49:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T04:49:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T04:49:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T04:49:59.715 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T04:49:59.796 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T04:49:59.974 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T04:50:00.078 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T04:50:00.140 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T04:50:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T04:50:00.416 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T04:50:00.798 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T04:50:01.126 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T04:50:01.465 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T04:50:01.509 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T04:50:01.673 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T04:50:01.757 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T04:50:01.818 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T04:50:02.009 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T04:50:02.240 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T04:50:02.254 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T04:50:02.449 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T04:50:02.481 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T04:50:02.681 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T04:50:02.702 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T04:50:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T04:50:02.965 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T04:50:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T04:50:03.018 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T04:50:03.190 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T04:50:03.269 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T04:50:03.357 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T04:50:03.478 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T04:50:03.491 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T04:50:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T04:50:03.708 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T04:50:03.923 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T04:50:04.087 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T04:50:04.199 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T04:50:04.294 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T04:50:04.345 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T04:50:04.456 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T04:50:04.527 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T04:50:04.728 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T04:50:04.933 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T04:50:05.032 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T04:50:05.097 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T04:50:05.256 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T04:50:05.281 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T04:50:05.449 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T04:50:05.665 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T04:50:05.815 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T04:50:06.028 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T04:50:06.146 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T04:50:06.319 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T04:50:06.389 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T04:50:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T04:50:06.683 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T04:50:06.686 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T04:50:06.960 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T04:50:06.977 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T04:50:07.145 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T04:50:07.287 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T04:50:07.308 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T04:50:07.570 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T04:50:07.600 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T04:50:07.763 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T04:50:07.862 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T04:50:07.893 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T04:50:08.094 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T04:50:08.182 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T04:50:08.233 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T04:50:08.431 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T04:50:08.655 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T04:50:08.916 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T04:50:09.174 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T04:50:09.252 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T04:50:09.398 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T04:50:09.585 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T04:50:09.609 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T04:50:09.881 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T04:50:09.902 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T04:50:10.133 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T04:50:10.163 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T04:50:10.359 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T04:50:10.484 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T04:50:10.683 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T04:50:10.755 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T04:50:10.952 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T04:50:11.161 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T04:50:11.481 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T04:50:12.684 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T04:50:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T04:50:12.974 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T04:50:13.237 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T04:50:13.286 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T04:50:13.539 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T04:50:13.548 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T04:50:13.784 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T04:50:13.861 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T04:50:14.002 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T04:50:14.194 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T04:50:14.236 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T04:50:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T04:50:14.498 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T04:50:14.662 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T04:50:14.694 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T04:50:14.861 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T04:50:14.919 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T04:50:15.033 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T04:50:15.224 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T04:50:15.373 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T04:50:15.557 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T04:50:15.752 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T04:50:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T04:50:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T04:50:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T04:50:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T04:50:17.254 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T04:50:17.454 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T04:50:17.629 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T04:50:17.826 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T04:50:18.013 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T04:50:18.084 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T04:50:18.289 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T04:50:18.514 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T04:50:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T04:50:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T04:50:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T04:50:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T04:50:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T04:50:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T04:50:20.102 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T04:50:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T04:50:20.335 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T04:50:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T04:50:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T04:50:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T04:50:20.923 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T04:50:21.000 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T04:50:21.207 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T04:50:21.229 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T04:50:21.348 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T04:50:21.494 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T04:50:21.609 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T04:50:21.770 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T04:50:22.003 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T04:50:22.341 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T04:50:22.629 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T04:50:22.858 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T04:50:23.164 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T04:50:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T04:50:24.879 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T04:50:25.132 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T04:50:25.453 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T04:50:25.718 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T04:50:26.094 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T04:50:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T04:50:26.681 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T04:50:26.920 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T04:50:32.966 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T04:50:33.211 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T04:50:33.839 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T04:50:34.766 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T04:50:34.865 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T04:50:35.043 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T04:50:39.437 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T04:50:39.437 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2023-10-06T04:50:39.437 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2023-10-06T04:50:39.437 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-10-06T04:50:39.437 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /run 2023-10-06T04:50:39.437 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2023-10-06T04:50:39.438 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2023-10-06T04:50:39.438 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2023-10-06T04:50:39.438 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2023-10-06T04:50:39.438 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-10-06T04:50:39.438 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:50:39.823 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T04:50:41.311 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T04:50:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T04:50:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T04:50:41.915 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T04:50:42.323 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T04:50:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T04:50:42.482 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T04:50:45.483 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T04:50:45.670 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T04:50:45.862 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T04:50:46.037 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T04:50:46.188 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T04:50:46.403 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T04:50:46.608 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T04:50:47.577 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T04:50:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T04:50:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T04:50:48.290 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T04:50:48.450 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T04:50:48.486 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T04:50:48.655 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T04:50:48.666 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T04:50:50.297 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T04:50:50.558 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T04:50:50.653 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T04:50:50.813 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T04:50:50.862 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T04:50:51.013 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T04:50:51.361 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T04:50:51.546 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /run 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2023-10-06T04:50:53.497 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-10-06T04:50:53.498 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:50:53.907 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T04:50:55.491 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T04:50:55.721 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T04:50:55.956 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T04:50:56.107 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T04:50:56.494 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T04:50:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T04:50:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T04:50:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T04:50:57.452 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T04:50:57.906 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T04:50:58.113 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 110/122 2023-10-06T04:50:58.372 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T04:50:58.561 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T04:50:58.572 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T04:50:58.733 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T04:50:58.941 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T04:50:59.128 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T04:50:59.336 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T04:50:59.572 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T04:50:59.794 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T04:50:59.852 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T04:50:59.978 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T04:51:00.078 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T04:51:00.086 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T04:51:00.140 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T04:51:00.197 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T04:51:00.197 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T04:51:00.346 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T04:51:00.423 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T04:51:00.423 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T04:51:00.569 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T04:51:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T04:51:00.994 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T04:51:01.170 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T04:51:01.304 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T04:51:01.305 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T04:51:01.306 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T04:51:01.307 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T04:51:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T04:51:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T04:51:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T04:51:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T04:51:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T04:51:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T04:51:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T04:51:01.322 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T04:51:01.322 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T04:51:01.322 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T04:51:01.322 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T04:51:01.322 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T04:51:01.322 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T04:51:01.322 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T04:51:02.199 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T04:51:02.389 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T04:51:02.491 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T04:51:02.884 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T04:51:03.153 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T04:51:03.193 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T04:51:03.380 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T04:51:03.393 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T04:51:03.565 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T04:51:03.565 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:41 ago on Fri 06 Oct 2023 04:49:22 AM UTC. 2023-10-06T04:51:05.067 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T04:51:05.420 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T04:51:05.521 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T04:51:05.580 INFO:teuthology.orchestra.run.smithi153.stdout:Installed products updated. 2023-10-06T04:51:05.706 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T04:51:05.741 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.742 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.743 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T04:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: jq-1.5-12.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T04:51:05.745 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T04:51:05.746 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T04:51:05.747 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T04:51:05.748 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T04:51:05.749 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T04:51:05.750 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:05.751 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T04:51:05.752 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T04:51:05.752 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T04:51:05.752 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T04:51:05.752 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T04:51:05.753 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T04:51:05.753 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:05.753 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-10-06T04:51:05.889 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T04:51:06.059 DEBUG:teuthology.parallel:result is None 2023-10-06T04:51:06.260 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T04:51:06.589 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T04:51:11.856 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T04:51:12.198 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T04:51:12.379 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T04:51:13.136 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T04:51:13.503 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T04:51:13.677 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 111/122 2023-10-06T04:51:13.858 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T04:51:14.028 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T04:51:14.326 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T04:51:14.488 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T04:51:14.705 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T04:51:14.897 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T04:51:15.125 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T04:51:15.378 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T04:51:15.459 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T04:51:15.637 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T04:51:15.647 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T04:51:15.647 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T04:51:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T04:51:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T04:51:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T04:51:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T04:51:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T04:51:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T04:51:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T04:51:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T04:51:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T04:51:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T04:51:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T04:51:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T04:51:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T04:51:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T04:51:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T04:51:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T04:51:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T04:51:19.186 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:58 ago on Fri 06 Oct 2023 04:49:21 AM UTC. 2023-10-06T04:51:21.406 INFO:teuthology.orchestra.run.smithi003.stdout:Installed products updated. 2023-10-06T04:51:21.566 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:21.566 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.567 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.568 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: jq-1.5-12.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T04:51:21.569 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.570 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T04:51:21.571 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T04:51:21.572 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T04:51:21.572 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T04:51:21.572 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T04:51:21.572 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T04:51:21.572 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T04:51:21.572 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T04:51:21.573 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T04:51:21.574 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.575 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T04:51:21.576 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T04:51:21.577 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T04:51:21.578 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:21.578 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-10-06T04:51:21.899 DEBUG:teuthology.parallel:result is None 2023-10-06T04:51:21.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:51:22.257 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T04:51:22.308 INFO:teuthology.orchestra.run.smithi003.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T04:51:22.308 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T04:51:22.308 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T04:51:22.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:51:22.469 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T04:51:22.496 INFO:teuthology.orchestra.run.smithi153.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T04:51:22.496 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T04:51:22.496 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T04:51:22.497 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T04:51:22.497 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:22.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T04:51:22.538 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:22.538 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T04:51:22.583 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T04:51:22.583 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:22.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T04:51:22.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T04:51:22.681 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:22.681 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T04:51:22.711 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T04:51:22.777 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T04:51:22.777 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:22.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T04:51:22.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T04:51:22.867 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:22.867 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T04:51:22.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T04:51:22.956 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T04:51:22.957 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:22.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T04:51:22.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T04:51:23.049 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:23.049 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T04:51:23.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T04:51:23.136 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T04:51:23.199 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T04:51:23.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2023-10-06T04:51:23.202 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /var/log/ceph 2023-10-06T04:51:23.225 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T04:51:23.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T04:51:23.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T04:51:23.291 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T04:51:23.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T04:51:23.331 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T04:51:23.362 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T04:51:23.362 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T04:51:23.362 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T04:51:23.363 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:23.363 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T04:51:23.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:23.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:23.548 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:23.549 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T04:51:23.563 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:23.678 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:23.707 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T04:51:23.707 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T04:51:23.707 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412299', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKbo8AaA0SBNBZE0ptrEHo7x74gvdsBIRnPvtAxdvksWWeYJ1dFzKIZy7Vyzh+Fj35g42ByfopqfFPStxzQBIU=', 'smithi153.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGkRhRxkKOUwHgczTnlg0rpGLrnMsnPV1os71P+MSj+tcnfyIzqBkyl+OIJC6UJ7RsBIak0eR+BLBohXjbpT7s4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T04:51:23.708 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T04:51:23.710 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T04:51:23.756 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T04:51:23.765 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T04:51:23.820 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:23.820 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T04:51:23.843 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T04:51:23.843 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2023-10-06T04:51:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T04:51:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:23.899 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 215028 Links: 1 2023-10-06T04:51:23.899 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:23.899 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:23.899 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-06 04:51:16.367989128 +0000 2023-10-06T04:51:23.899 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-06 04:48:39.371132082 +0000 2023-10-06T04:51:23.899 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-06 04:48:39.371132082 +0000 2023-10-06T04:51:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-06T04:51:23.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T04:51:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-06T04:51:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-06T04:51:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000241858 s, 2.1 MB/s 2023-10-06T04:51:23.963 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T04:51:24.019 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2023-10-06T04:51:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T04:51:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:24.075 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 219290 Links: 1 2023-10-06T04:51:24.075 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:24.076 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:24.076 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-06 04:51:16.367989128 +0000 2023-10-06T04:51:24.076 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-06 04:48:39.887125467 +0000 2023-10-06T04:51:24.076 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-06 04:48:39.887125467 +0000 2023-10-06T04:51:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-06T04:51:24.077 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T04:51:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-06T04:51:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-06T04:51:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000246881 s, 2.1 MB/s 2023-10-06T04:51:24.140 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T04:51:24.196 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2023-10-06T04:51:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 220456 Links: 1 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-06 04:51:16.367989128 +0000 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-06 04:48:40.389119031 +0000 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-06 04:48:40.389119031 +0000 2023-10-06T04:51:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-06T04:51:24.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T04:51:24.315 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-06T04:51:24.315 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-06T04:51:24.315 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000223103 s, 2.3 MB/s 2023-10-06T04:51:24.316 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T04:51:24.372 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 220517 Links: 1 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-06 04:51:16.367989128 +0000 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-06 04:48:40.898112506 +0000 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-06 04:48:40.898112506 +0000 2023-10-06T04:51:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-06T04:51:24.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T04:51:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-06T04:51:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-06T04:51:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000189234 s, 2.7 MB/s 2023-10-06T04:51:24.494 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T04:51:24.550 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:24.550 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:24.551 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T04:51:24.566 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T04:51:24.567 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2023-10-06T04:51:24.622 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T04:51:24.622 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:24.622 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 219335 Links: 1 2023-10-06T04:51:24.622 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:24.622 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:24.622 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-10-06 04:51:00.873203160 +0000 2023-10-06T04:51:24.623 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-10-06 04:48:28.967072724 +0000 2023-10-06T04:51:24.623 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-10-06 04:48:28.967072724 +0000 2023-10-06T04:51:24.623 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-10-06T04:51:24.623 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T04:51:24.685 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-10-06T04:51:24.685 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-10-06T04:51:24.685 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000207836 s, 2.5 MB/s 2023-10-06T04:51:24.686 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T04:51:24.742 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2023-10-06T04:51:24.796 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 218658 Links: 1 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-10-06 04:51:00.873203160 +0000 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-10-06 04:48:29.457064298 +0000 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-10-06 04:48:29.457064298 +0000 2023-10-06T04:51:24.797 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-10-06T04:51:24.797 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T04:51:24.859 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-10-06T04:51:24.860 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-10-06T04:51:24.860 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000203273 s, 2.5 MB/s 2023-10-06T04:51:24.861 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T04:51:24.916 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2023-10-06T04:51:24.971 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 218715 Links: 1 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-10-06 04:51:00.873203160 +0000 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-10-06 04:48:29.951055803 +0000 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-10-06 04:48:29.951055803 +0000 2023-10-06T04:51:24.972 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-10-06T04:51:24.973 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T04:51:25.035 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-10-06T04:51:25.035 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-10-06T04:51:25.035 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000244995 s, 2.1 MB/s 2023-10-06T04:51:25.037 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T04:51:25.092 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2023-10-06T04:51:25.148 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T04:51:25.148 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:51:25.148 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 222512 Links: 1 2023-10-06T04:51:25.148 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:51:25.149 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:51:25.149 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-10-06 04:51:00.873203160 +0000 2023-10-06T04:51:25.149 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-10-06 04:48:30.435047479 +0000 2023-10-06T04:51:25.149 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-10-06 04:48:30.435047479 +0000 2023-10-06T04:51:25.149 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-10-06T04:51:25.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T04:51:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-10-06T04:51:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-10-06T04:51:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000189559 s, 2.7 MB/s 2023-10-06T04:51:25.212 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T04:51:25.268 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:25.268 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T04:51:25.268 INFO:tasks.ceph:Generating config... 2023-10-06T04:51:25.269 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T04:51:25.270 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T04:51:25.270 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T04:51:25.270 INFO:tasks.ceph:[client] pid file = 2023-10-06T04:51:25.270 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T04:51:25.270 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T04:51:25.270 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T04:51:25.270 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T04:51:25.270 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T04:51:25.270 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T04:51:25.270 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T04:51:25.271 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T04:51:25.271 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T04:51:25.271 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T04:51:25.272 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T04:51:25.273 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T04:51:25.273 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T04:51:25.273 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T04:51:25.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T04:51:25.329 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T04:51:25.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T04:51:25.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T04:51:25.439 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '172.21.15.153'), ('mon.c', '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]')] 2023-10-06T04:51:25.440 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T04:51:25.441 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:25.441 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T04:51:25.494 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --add b 172.21.15.153 --addv c '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = octopus 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:fsid fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2023-10-06T04:51:25.577301+0000 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:created 2023-10-06T04:51:25.577301+0000 2023-10-06T04:51:25.578 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 15 (octopus) 2023-10-06T04:51:25.579 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2023-10-06T04:51:25.579 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2023-10-06T04:51:25.579 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2023-10-06T04:51:25.579 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] mon.c 2023-10-06T04:51:25.579 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T04:51:25.580 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T04:51:25.635 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fa7cd5af-16f2-4a73-ab30-42f10cc13e22... 2023-10-06T04:51:25.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:51:25.678 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:51:25.720 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-06T04:51:25.720 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-06T04:51:25.720 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:51:25.720 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-06T04:51:25.720 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.720 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-06T04:51:25.720 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.721 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:51:25.722 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-06T04:51:25.723 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-06T04:51:25.724 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-06T04:51:25.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-06T04:51:25.726 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:51:25.727 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:51:25.728 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:51:25.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-06T04:51:25.730 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-06T04:51:25.731 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:51:25.732 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:51:25.733 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:51:25.733 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-10-06T04:51:25.733 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-10-06T04:51:25.733 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:51:25.733 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-10-06T04:51:25.734 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:51:25.735 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-10-06T04:51:25.736 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-10-06T04:51:25.737 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:51:25.738 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-10-06T04:51:25.739 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:51:25.740 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.741 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.742 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-10-06T04:51:25.743 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:51:25.744 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-10-06T04:51:25.745 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-10-06T04:51:25.746 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-10-06T04:51:25.746 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:51:25.746 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:51:25.746 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:51:25.747 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T04:51:25.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T04:51:25.839 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T04:51:25.840 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:25.840 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T04:51:25.854 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:25.854 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T04:51:25.909 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:51:25.909 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:25.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T04:51:25.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T04:51:25.984 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:25.985 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:26.039 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:51:26.039 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:26.040 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T04:51:26.040 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T04:51:26.076 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:26.076 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:26.146 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T04:51:26.147 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T04:51:26.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T04:51:26.196 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T04:51:26.198 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T04:51:26.379 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T04:51:26.381 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T04:51:26.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T04:51:26.433 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T04:51:26.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T04:51:26.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T04:51:26.549 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T04:51:26.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T04:51:26.576 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T04:51:26.628 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T04:51:26.631 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T04:51:26.656 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T04:51:26.744 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T04:51:26.747 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T04:51:26.771 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T04:51:26.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T04:51:26.810 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T04:51:26.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T04:51:26.902 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T04:51:26.915 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T04:51:26.953 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T04:51:26.965 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T04:51:27.044 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T04:51:27.056 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T04:51:27.056 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T04:51:27.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T04:51:27.082 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:27.082 INFO:tasks.ceph:role: osd.0 2023-10-06T04:51:27.082 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:51:27.083 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T04:51:27.154 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:27.155 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:27.155 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:27.155 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-10-06T04:51:27.155 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:27.155 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:27.156 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:27.156 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:27.156 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:27.156 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:27.595 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-06T04:51:27.596 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-06T04:51:27.596 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T04:51:27.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T04:51:27.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T04:51:27.870 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:27.870 INFO:tasks.ceph:role: osd.1 2023-10-06T04:51:27.870 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:51:27.870 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:27.943 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-06T04:51:28.363 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-06T04:51:28.364 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T04:51:28.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T04:51:28.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T04:51:28.559 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:28.559 INFO:tasks.ceph:role: osd.2 2023-10-06T04:51:28.559 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:51:28.559 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T04:51:28.632 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:28.632 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:28.633 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:28.633 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-10-06T04:51:28.633 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:28.633 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:28.633 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:28.634 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:28.634 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:28.634 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:29.079 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-06T04:51:29.081 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-06T04:51:29.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T04:51:29.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T04:51:29.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T04:51:29.367 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:29.367 INFO:tasks.ceph:role: osd.3 2023-10-06T04:51:29.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:51:29.367 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T04:51:29.441 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:29.442 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:29.865 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-06T04:51:29.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-06T04:51:29.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T04:51:30.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T04:51:30.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:30.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:30.147+0000 7f3d3d8b0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T04:51:30.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:30.147+0000 7f3d3d8b0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T04:51:30.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:30.147+0000 7f3d3d8b0540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T04:51:30.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:30.147+0000 7f3d3d8b0540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T04:51:30.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:30.147+0000 7f3d3d8b0540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T04:51:30.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:30.148+0000 7f3d3d8b0540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T04:51:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:30.677+0000 7f3d3d8b0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:31.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:31.441+0000 7f3d3d8b0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:32.207+0000 7f3d3d8b0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:33.007 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T04:51:33.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:33.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:33.124+0000 7f82a8aba540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T04:51:33.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:33.124+0000 7f82a8aba540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T04:51:33.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:33.124+0000 7f82a8aba540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T04:51:33.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:33.124+0000 7f82a8aba540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T04:51:33.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:33.124+0000 7f82a8aba540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T04:51:33.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:33.125+0000 7f82a8aba540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T04:51:33.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:33.654+0000 7f82a8aba540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:34.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:34.416+0000 7f82a8aba540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:35.180+0000 7f82a8aba540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:35.977 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T04:51:36.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:36.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:36.089+0000 7f9742e90540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T04:51:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:36.090+0000 7f9742e90540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T04:51:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:36.090+0000 7f9742e90540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T04:51:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:36.090+0000 7f9742e90540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T04:51:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:36.090+0000 7f9742e90540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T04:51:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:36.090+0000 7f9742e90540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T04:51:36.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:36.628+0000 7f9742e90540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:37.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:37.392+0000 7f9742e90540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:38.156+0000 7f9742e90540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:38.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T04:51:38.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:39.074+0000 7f2b4a7fc540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T04:51:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:39.074+0000 7f2b4a7fc540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T04:51:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:39.074+0000 7f2b4a7fc540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T04:51:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:39.074+0000 7f2b4a7fc540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T04:51:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:39.074+0000 7f2b4a7fc540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T04:51:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:39.074+0000 7f2b4a7fc540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T04:51:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:39.599+0000 7f2b4a7fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:40.362+0000 7f2b4a7fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:41.125+0000 7f2b4a7fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:41.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T04:51:41.958 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T04:51:41.987 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:41.987 INFO:tasks.ceph:role: osd.4 2023-10-06T04:51:41.988 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:51:41.988 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T04:51:42.062 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:42.062 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:42.062 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:42.062 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 2023-10-06T04:51:42.062 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:42.062 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:42.063 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:42.063 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:42.063 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:42.063 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:42.514 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-10-06T04:51:42.516 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-10-06T04:51:42.516 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T04:51:42.681 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T04:51:42.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T04:51:42.785 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:42.785 INFO:tasks.ceph:role: osd.5 2023-10-06T04:51:42.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:51:42.785 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T04:51:42.858 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:43.302 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-10-06T04:51:43.303 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-10-06T04:51:43.304 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T04:51:43.415 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T04:51:43.447 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T04:51:43.515 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:43.515 INFO:tasks.ceph:role: osd.6 2023-10-06T04:51:43.515 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:51:43.515 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T04:51:43.588 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:43.589 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:44.037 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-10-06T04:51:44.039 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-10-06T04:51:44.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T04:51:44.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T04:51:44.223 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T04:51:44.288 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:51:44.288 INFO:tasks.ceph:role: osd.7 2023-10-06T04:51:44.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:51:44.289 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T04:51:44.361 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:51:44.361 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:51:44.361 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:51:44.361 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 2023-10-06T04:51:44.362 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:51:44.362 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:51:44.362 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:51:44.362 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:51:44.362 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:51:44.362 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:51:44.811 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-10-06T04:51:44.812 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-10-06T04:51:44.812 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T04:51:44.977 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T04:51:45.006 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:45.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:45.098+0000 7f067ed27540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T04:51:45.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:45.098+0000 7f067ed27540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T04:51:45.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:45.098+0000 7f067ed27540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T04:51:45.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:45.098+0000 7f067ed27540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T04:51:45.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:45.098+0000 7f067ed27540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T04:51:45.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:45.099+0000 7f067ed27540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T04:51:45.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:45.624+0000 7f067ed27540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:46.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:46.388+0000 7f067ed27540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:47.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:47.150+0000 7f067ed27540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:47.956 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T04:51:47.985 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:48.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:48.069+0000 7ff942c2c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T04:51:48.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:48.070+0000 7ff942c2c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T04:51:48.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:48.070+0000 7ff942c2c540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T04:51:48.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:48.070+0000 7ff942c2c540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T04:51:48.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:48.070+0000 7ff942c2c540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T04:51:48.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:48.070+0000 7ff942c2c540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T04:51:48.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:48.598+0000 7ff942c2c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:49.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:49.360+0000 7ff942c2c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:50.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:50.122+0000 7ff942c2c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:50.925 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T04:51:50.954 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:51.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:51.036+0000 7fb69f2a1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T04:51:51.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:51.036+0000 7fb69f2a1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T04:51:51.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:51.036+0000 7fb69f2a1540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T04:51:51.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:51.036+0000 7fb69f2a1540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T04:51:51.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:51.036+0000 7fb69f2a1540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T04:51:51.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:51.037+0000 7fb69f2a1540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T04:51:51.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:51.564+0000 7fb69f2a1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:52.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:52.328+0000 7fb69f2a1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:53.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:53.094+0000 7fb69f2a1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:53.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:53.712 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:53.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T04:51:53.922 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:54.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:54.005+0000 7f8b445ca540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T04:51:54.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:54.005+0000 7f8b445ca540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T04:51:54.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:54.005+0000 7f8b445ca540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T04:51:54.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:54.005+0000 7f8b445ca540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T04:51:54.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:54.005+0000 7f8b445ca540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T04:51:54.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:54.006+0000 7f8b445ca540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T04:51:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:54.533+0000 7f8b445ca540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:55.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:55.295+0000 7f8b445ca540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:51:56.057+0000 7f8b445ca540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:51:56.849 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T04:51:56.874 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T04:51:56.875 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:56.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T04:51:56.904 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:56.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T04:51:56.972 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:56.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T04:51:57.039 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:57.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T04:51:57.106 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:57.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T04:51:57.173 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:57.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T04:51:57.239 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:57.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T04:51:57.305 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.306 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T04:51:57.330 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.330 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T04:51:57.394 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T04:51:57.458 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.458 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T04:51:57.522 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.522 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T04:51:57.585 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.586 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T04:51:57.650 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.650 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T04:51:57.713 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:57.714 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T04:51:57.730 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:57.730 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T04:51:57.786 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.786 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T04:51:57.806 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:51:57.806 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T04:51:57.861 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T04:51:57.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T04:51:57.863 DEBUG:teuthology.orchestra.run.smithi153:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T04:51:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2023-10-06T04:51:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA== 2023-10-06T04:51:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T04:51:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ== 2023-10-06T04:51:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g== 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg== 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg== 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug== 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw== 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2023-10-06T04:51:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw== 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg== 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg== 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.4] 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA== 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.5] 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA== 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.6] 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ== 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.7] 2023-10-06T04:51:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg== 2023-10-06T04:51:57.908 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-10-06T04:51:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g== 2023-10-06T04:51:57.908 INFO:teuthology.orchestra.run.smithi003.stdout:[client.2] 2023-10-06T04:51:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw== 2023-10-06T04:51:57.908 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2023-10-06T04:51:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ== 2023-10-06T04:51:57.909 INFO:teuthology.orchestra.run.smithi003.stdout:[client.3] 2023-10-06T04:51:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg== 2023-10-06T04:51:57.924 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.y] 2023-10-06T04:51:57.924 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA== 2023-10-06T04:51:57.924 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ== 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g== 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.0] 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg== 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.1] 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg== 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.2] 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug== 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.3] 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw== 2023-10-06T04:51:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.x] 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw== 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg== 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg== 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.4] 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA== 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.5] 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA== 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.6] 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ== 2023-10-06T04:51:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.7] 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg== 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g== 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:[client.2] 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw== 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:[client.1] 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ== 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:[client.3] 2023-10-06T04:51:57.927 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg== 2023-10-06T04:51:57.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:51:57.953 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:51:58.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.018 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.070 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.122 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.171 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.221 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:51:58.320 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:51:58.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.369 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.422 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:51:58.472 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.474 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.525 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.574 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.623 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:51:58.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.726 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.778 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.828 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.830 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:51:58.880 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T04:51:58.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T04:51:58.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T04:51:58.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T04:51:59.022 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T04:51:59.047 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T04:51:59.139 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T04:51:59.162 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T04:51:59.227 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T04:51:59.318 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T04:51:59.343 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:59.345 DEBUG:teuthology.orchestra.run.smithi153:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:51:59.397 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T04:51:59.398 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T04:51:59.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T04:51:59.400 INFO:tasks.ceph.mon.a:Started 2023-10-06T04:51:59.400 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T04:51:59.401 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T04:51:59.440 INFO:tasks.ceph.mon.b:Started 2023-10-06T04:51:59.440 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T04:51:59.441 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T04:51:59.442 INFO:tasks.ceph.mon.c:Started 2023-10-06T04:51:59.442 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T04:51:59.443 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T04:51:59.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T04:51:59.445 INFO:tasks.ceph.mgr.y:Started 2023-10-06T04:51:59.445 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T04:51:59.445 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T04:51:59.447 INFO:tasks.ceph.mgr.x:Started 2023-10-06T04:51:59.447 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T04:51:59.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T04:51:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.566+0000 7f09842a2700 1 Processor -- start 2023-10-06T04:51:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.566+0000 7f09842a2700 1 -- start start 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.567+0000 7f09842a2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 0x7f097c0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.567+0000 7f09842a2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f097c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.567+0000 7f09842a2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097c06ae30 con 0x7f097c157810 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.567+0000 7f09842a2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097c0b2320 con 0x7f097c157be0 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.567+0000 7f09842a2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c06a220 con 0x7f097c0b1ea0 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098283f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f097c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098183d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 0x7f097c0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098283f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f097c0caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:47662/0 (socket says 172.21.15.3:47662) 2023-10-06T04:51:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098283f700 1 -- 172.21.15.3:0/2282850844 learned_addr learned my addr 172.21.15.3:0/2282850844 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098203e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f097c157810 0x7f097c0c8e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:36580/0 (socket says 172.21.15.3:36580) 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098183d700 1 -- 172.21.15.3:0/2282850844 >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 msgr2=0x7f097c0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098183d700 1 -- 172.21.15.3:0/2282850844 >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 msgr2=0x7f097c0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098283f700 1 -- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 msgr2=0x7f097c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098183d700 1 --2- 172.21.15.3:0/2282850844 >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 0x7f097c0b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098283f700 1 -- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 msgr2=0x7f097c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098283f700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f097c0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098283f700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f097c0caa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:51:59.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.568+0000 7f098183d700 1 --2- 172.21.15.3:0/2282850844 >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 0x7f097c0b1960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972692304 0 0) 0x7f097c06ae30 con 0x7f097c157810 2023-10-06T04:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0968003570 con 0x7f097c157810 2023-10-06T04:51:59.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 494963465 0 0) 0x7f0968003570 con 0x7f097c157810 2023-10-06T04:51:59.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 msgr2=0x7f097c0caa50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:51:59.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f097c0caa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 msgr2=0x7f097c0b1960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 --2- 172.21.15.3:0/2282850844 >> v2:172.21.15.153:3300/0 conn(0x7f097c157be0 0x7f097c0b1960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c06a500 con 0x7f097c157810 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f09780014c0 con 0x7f097c157810 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.767+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 >> v1:172.21.15.153:6789/0 conn(0x7f097c157810 legacy=0x7f097c0c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098103c700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0968003a60 0x7f0968003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098103c700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f0968004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098103c700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968004d50 0x7f09680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0b2320 con 0x7f0968003a60 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c06a220 con 0x7f0968004d50 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096800bb90 con 0x7f097c0b1ea0 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098203e700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0968003a60 0x7f0968003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098183d700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f0968004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098283f700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968004d50 0x7f09680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098203e700 1 -- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 msgr2=0x7f0968004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098203e700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0b1ea0 0x7f0968004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098203e700 1 -- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968004d50 msgr2=0x7f09680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098203e700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968004d50 0x7f09680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098203e700 1 -- 172.21.15.3:0/2282850844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968003570 con 0x7f0968003a60 2023-10-06T04:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.768+0000 7f098203e700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0968003a60 0x7f0968003e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f097800c2d0 tx=0x7f0978001ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5d2504fcdff9d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.769+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097800cba0 con 0x7f0968003a60 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.769+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097800cd20 con 0x7f0968003a60 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.769+0000 7f098103c700 1 -- 172.21.15.3:0/2282850844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978005b00 con 0x7f0968003a60 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.769+0000 7f09842a2700 1 -- 172.21.15.3:0/2282850844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0968003a60 msgr2=0x7f0968003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.769+0000 7f09842a2700 1 --2- 172.21.15.3:0/2282850844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0968003a60 0x7f0968003e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f097800c2d0 tx=0x7f0978001ee0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.769+0000 7f098283f700 1 -- 172.21.15.3:0/2282850844 reap_dead start 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.770+0000 7f09842a2700 1 -- 172.21.15.3:0/2282850844 shutdown_connections 2023-10-06T04:51:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.770+0000 7f09842a2700 1 -- 172.21.15.3:0/2282850844 >> 172.21.15.3:0/2282850844 conn(0x7f097c067a40 msgr2=0x7f097c0ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:51:59.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.770+0000 7f09842a2700 1 -- 172.21.15.3:0/2282850844 shutdown_connections 2023-10-06T04:51:59.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.770+0000 7f09842a2700 1 -- 172.21.15.3:0/2282850844 wait complete. 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.771+0000 7f09842a2700 1 Processor -- start 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.771+0000 7f09842a2700 1 -- start start 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f09842a2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968003a60 0x7f097c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f09842a2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c159500 0x7f097c1498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f09842a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097c151330 0x7f097c14c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f09842a2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c15afc0 con 0x7f097c151330 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f09842a2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c1599e0 con 0x7f0968003a60 2023-10-06T04:51:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f09842a2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c159cc0 con 0x7f097c159500 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f098283f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097c151330 0x7f097c14c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f098203e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968003a60 0x7f097c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f098283f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097c151330 0x7f097c14c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41410/0 (socket says 172.21.15.3:41410) 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f098283f700 1 -- 172.21.15.3:0/364521835 learned_addr learned my addr 172.21.15.3:0/364521835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.772+0000 7f098183d700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c159500 0x7f097c1498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.773+0000 7f098283f700 1 -- 172.21.15.3:0/364521835 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c159500 msgr2=0x7f097c1498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.773+0000 7f098283f700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c159500 0x7f097c1498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.773+0000 7f098283f700 1 -- 172.21.15.3:0/364521835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968003a60 msgr2=0x7f097c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.773+0000 7f098283f700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968003a60 0x7f097c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.773+0000 7f098283f700 1 -- 172.21.15.3:0/364521835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c159fa0 con 0x7f097c151330 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.773+0000 7f098283f700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097c151330 0x7f097c14c5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0974009410 tx=0x7f09740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.774+0000 7f09727fc700 1 -- 172.21.15.3:0/364521835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400f8f0 con 0x7f097c151330 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.774+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097c15a280 con 0x7f097c151330 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.774+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c149de0 con 0x7f097c151330 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.774+0000 7f09727fc700 1 -- 172.21.15.3:0/364521835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097400fc60 con 0x7f097c151330 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.774+0000 7f09727fc700 1 -- 172.21.15.3:0/364521835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400fdd0 con 0x7f097c151330 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.774+0000 7f09727fc700 1 -- 172.21.15.3:0/364521835 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0974003020 con 0x7f097c151330 2023-10-06T04:51:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.774+0000 7f09727fc700 1 -- 172.21.15.3:0/364521835 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f097400a5a0 con 0x7f097c151330 2023-10-06T04:51:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.775+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09640052b0 con 0x7f097c151330 2023-10-06T04:51:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.777+0000 7f09727fc700 1 -- 172.21.15.3:0/364521835 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f097401b070 con 0x7f097c151330 2023-10-06T04:51:59.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.854+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0964005830 con 0x7f097c151330 2023-10-06T04:51:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.861+0000 7f09727fc700 1 -- 172.21.15.3:0/364521835 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0974009bc0 con 0x7f097c151330 2023-10-06T04:51:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097c151330 msgr2=0x7f097c14c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:51:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097c151330 0x7f097c14c5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0974009410 tx=0x7f09740091a0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 shutdown_connections 2023-10-06T04:51:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968003a60 0x7f097c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097c151330 0x7f097c14c5e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 --2- 172.21.15.3:0/364521835 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c159500 0x7f097c1498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:51:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 >> 172.21.15.3:0/364521835 conn(0x7f097c067a40 msgr2=0x7f097c0ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:51:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 shutdown_connections 2023-10-06T04:51:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:51:59.862+0000 7f09842a2700 1 -- 172.21.15.3:0/364521835 wait complete. 2023-10-06T04:51:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2023-10-06T04:51:59.876 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T04:51:59.876 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:59.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T04:51:59.903 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:59.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T04:51:59.969 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:51:59.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T04:52:00.036 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-06T04:52:00.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T04:52:00.102 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:52:00.102 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T04:52:00.126 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:52:00.126 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T04:52:00.193 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:52:00.193 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T04:52:00.257 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-10-06T04:52:00.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T04:52:00.321 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new a13544f3-f143-4c31-b32e-182882aca1bf 0 2023-10-06T04:52:00.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.476+0000 7f614903b700 1 Processor -- start 2023-10-06T04:52:00.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.476+0000 7f614903b700 1 -- start start 2023-10-06T04:52:00.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614903b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6144068190 0x7f614415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614903b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614415d130 con 0x7f6144067dc0 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614903b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440691c0 con 0x7f614415cd40 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614903b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61440694a0 con 0x7f6144068190 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614259c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6144068190 0x7f614415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614359e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f614415cd40 0x7f614406d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:38122/0 (socket says 172.21.15.153:38122) 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614359e700 1 -- 172.21.15.153:0/675915676 learned_addr learned my addr 172.21.15.153:0/675915676 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614259c700 1 -- 172.21.15.153:0/675915676 >> v1:172.21.15.153:6789/0 conn(0x7f614415cd40 legacy=0x7f614406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614259c700 1 -- 172.21.15.153:0/675915676 >> v1:172.21.15.3:6789/0 conn(0x7f6144067dc0 legacy=0x7f61440b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.477+0000 7f614259c700 1 -- 172.21.15.153:0/675915676 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144069780 con 0x7f6144068190 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614259c700 1 --2- 172.21.15.153:0/675915676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6144068190 0x7f614415c800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f61340098e0 tx=0x7f61340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce159a402030ba61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f6141d9b700 1 -- 172.21.15.153:0/675915676 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134009e80 con 0x7f6144068190 2023-10-06T04:52:00.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f6141d9b700 1 -- 172.21.15.153:0/675915676 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613400b860 con 0x7f6144068190 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f6141d9b700 1 -- 172.21.15.153:0/675915676 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613400bbe0 con 0x7f6144068190 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614903b700 1 -- 172.21.15.153:0/675915676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6144068190 msgr2=0x7f614415c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614903b700 1 --2- 172.21.15.153:0/675915676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6144068190 0x7f614415c800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f61340098e0 tx=0x7f61340091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614903b700 1 -- 172.21.15.153:0/675915676 shutdown_connections 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614903b700 1 --2- 172.21.15.153:0/675915676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6144068190 0x7f614415c800 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614903b700 1 -- 172.21.15.153:0/675915676 >> 172.21.15.153:0/675915676 conn(0x7f61440ce270 msgr2=0x7f61440ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614903b700 1 -- 172.21.15.153:0/675915676 shutdown_connections 2023-10-06T04:52:00.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.478+0000 7f614903b700 1 -- 172.21.15.153:0/675915676 wait complete. 2023-10-06T04:52:00.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 Processor -- start 2023-10-06T04:52:00.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 -- start start 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6144067dc0 0x7f6144158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6144068190 0x7f61441595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f614415cd40 0x7f6144159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61441593e0 con 0x7f6144067dc0 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440c8980 con 0x7f614415cd40 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.480+0000 7f614903b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61440c8070 con 0x7f6144068190 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614259c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6144068190 0x7f61441595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614359e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f614415cd40 0x7f6144159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614259c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6144068190 0x7f61441595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33470/0 (socket says 172.21.15.153:33470) 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614259c700 1 -- 172.21.15.153:0/2829920670 learned_addr learned my addr 172.21.15.153:0/2829920670 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:00.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f6142d9d700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6144067dc0 0x7f6144158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614359e700 1 -- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6144068190 msgr2=0x7f61441595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614359e700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6144068190 0x7f61441595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614359e700 1 -- 172.21.15.153:0/2829920670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6144067dc0 msgr2=0x7f6144158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614359e700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6144067dc0 0x7f6144158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614359e700 1 -- 172.21.15.153:0/2829920670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6134009420 con 0x7f614415cd40 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614259c700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6144068190 0x7f61441595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614359e700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f614415cd40 0x7f6144159b00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f613c00cb40 tx=0x7f613c001900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f612affd700 1 -- 172.21.15.153:0/2829920670 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00a7e0 con 0x7f614415cd40 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f612affd700 1 -- 172.21.15.153:0/2829920670 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c00a940 con 0x7f614415cd40 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f612affd700 1 -- 172.21.15.153:0/2829920670 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00ace0 con 0x7f614415cd40 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6144153ef0 con 0x7f614415cd40 2023-10-06T04:52:00.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61440c8330 con 0x7f614415cd40 2023-10-06T04:52:00.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f612affd700 1 -- 172.21.15.153:0/2829920670 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f613c007300 con 0x7f614415cd40 2023-10-06T04:52:00.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.481+0000 7f612affd700 1 -- 172.21.15.153:0/2829920670 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f613c01b750 con 0x7f614415cd40 2023-10-06T04:52:00.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.482+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61440c8af0 con 0x7f614415cd40 2023-10-06T04:52:00.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.484+0000 7f612affd700 1 -- 172.21.15.153:0/2829920670 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f613c00ae40 con 0x7f614415cd40 2023-10-06T04:52:00.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.566+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a13544f3-f143-4c31-b32e-182882aca1bf", "id": 0} v 0) v1 -- 0x7f614404c380 con 0x7f614415cd40 2023-10-06T04:52:00.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.571+0000 7f612affd700 1 -- 172.21.15.153:0/2829920670 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a13544f3-f143-4c31-b32e-182882aca1bf", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f613c007740 con 0x7f614415cd40 2023-10-06T04:52:00.572 INFO:teuthology.orchestra.run.smithi153.stdout:0 2023-10-06T04:52:00.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f614415cd40 msgr2=0x7f6144159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f614415cd40 0x7f6144159b00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f613c00cb40 tx=0x7f613c001900 comp rx=0 tx=0).stop 2023-10-06T04:52:00.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 shutdown_connections 2023-10-06T04:52:00.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f614415cd40 0x7f6144159b00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6144067dc0 0x7f6144158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 --2- 172.21.15.153:0/2829920670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6144068190 0x7f61441595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 >> 172.21.15.153:0/2829920670 conn(0x7f61440ce270 msgr2=0x7f61441584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:00.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 shutdown_connections 2023-10-06T04:52:00.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.572+0000 7f614903b700 1 -- 172.21.15.153:0/2829920670 wait complete. 2023-10-06T04:52:00.585 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new af00dc42-0e73-4607-a038-73b65ceee91e 1 2023-10-06T04:52:00.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.700+0000 7fec78109700 1 Processor -- start 2023-10-06T04:52:00.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.700+0000 7fec78109700 1 -- start start 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec78109700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec7015da10 0x7fec7015de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec78109700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7015e3b0 con 0x7fec700cfb80 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec78109700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec700afca0 con 0x7fec70159eb0 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec78109700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7015d510 con 0x7fec7015da10 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec766a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec7015da10 0x7fec7015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec756a4700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fec70159eb0 0x7fec700ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:38124/0 (socket says 172.21.15.153:38124) 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec766a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec7015da10 0x7fec7015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33474/0 (socket says 172.21.15.153:33474) 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec756a4700 1 -- 172.21.15.153:0/3901170436 learned_addr learned my addr 172.21.15.153:0/3901170436 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-10-06T04:52:00.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec766a6700 1 -- 172.21.15.153:0/3901170436 >> v1:172.21.15.153:6789/0 conn(0x7fec70159eb0 legacy=0x7fec700ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec766a6700 1 -- 172.21.15.153:0/3901170436 >> v1:172.21.15.3:6789/0 conn(0x7fec700cfb80 legacy=0x7fec700af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec74ea3700 1 -- 172.21.15.153:0/3901170436 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398101106 0 0) 0x7fec700afca0 con 0x7fec70159eb0 2023-10-06T04:52:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec766a6700 1 -- 172.21.15.153:0/3901170436 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7015a480 con 0x7fec7015da10 2023-10-06T04:52:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.701+0000 7fec766a6700 1 --2- 172.21.15.153:0/3901170436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec7015da10 0x7fec7015de70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec68009e90 tx=0x7fec68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1055d5ff2551ab1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec74ea3700 1 -- 172.21.15.153:0/3901170436 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6800e070 con 0x7fec7015da10 2023-10-06T04:52:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec74ea3700 1 -- 172.21.15.153:0/3901170436 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6800bc30 con 0x7fec7015da10 2023-10-06T04:52:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec74ea3700 1 -- 172.21.15.153:0/3901170436 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68012430 con 0x7fec7015da10 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec78109700 1 -- 172.21.15.153:0/3901170436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec7015da10 msgr2=0x7fec7015de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec78109700 1 --2- 172.21.15.153:0/3901170436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec7015da10 0x7fec7015de70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec68009e90 tx=0x7fec68009780 comp rx=0 tx=0).stop 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec78109700 1 -- 172.21.15.153:0/3901170436 shutdown_connections 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec78109700 1 --2- 172.21.15.153:0/3901170436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec7015da10 0x7fec7015de70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec78109700 1 -- 172.21.15.153:0/3901170436 >> 172.21.15.153:0/3901170436 conn(0x7fec70070310 msgr2=0x7fec70070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec78109700 1 -- 172.21.15.153:0/3901170436 shutdown_connections 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.702+0000 7fec78109700 1 -- 172.21.15.153:0/3901170436 wait complete. 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 Processor -- start 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 -- start start 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec700cfb80 0x7fec701535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec70159eb0 0x7fec700b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7015da10 0x7fec70155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec700b2ba0 con 0x7fec700cfb80 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec700b1f90 con 0x7fec7015da10 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec78109700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec700b2270 con 0x7fec70159eb0 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec756a4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec70159eb0 0x7fec700b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec766a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7015da10 0x7fec70155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec756a4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec70159eb0 0x7fec700b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33482/0 (socket says 172.21.15.153:33482) 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec756a4700 1 -- 172.21.15.153:0/2452067905 learned_addr learned my addr 172.21.15.153:0/2452067905 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec766a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7015da10 0x7fec70155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57834/0 (socket says 172.21.15.153:57834) 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.704+0000 7fec75ea5700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec700cfb80 0x7fec701535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec756a4700 1 -- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7015da10 msgr2=0x7fec70155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec756a4700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7015da10 0x7fec70155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec756a4700 1 -- 172.21.15.153:0/2452067905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec700cfb80 msgr2=0x7fec701535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec756a4700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec700cfb80 0x7fec701535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec756a4700 1 -- 172.21.15.153:0/2452067905 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec68009a20 con 0x7fec70159eb0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec766a6700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7015da10 0x7fec70155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec756a4700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec70159eb0 0x7fec700b14f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c7e0 tx=0x7fec6c007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec65ffb700 1 -- 172.21.15.153:0/2452067905 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c005b30 con 0x7fec70159eb0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec65ffb700 1 -- 172.21.15.153:0/2452067905 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c005cb0 con 0x7fec70159eb0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec700b2550 con 0x7fec70159eb0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec65ffb700 1 -- 172.21.15.153:0/2452067905 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c007320 con 0x7fec70159eb0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec65ffb700 1 -- 172.21.15.153:0/2452067905 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fec6c012680 con 0x7fec70159eb0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec700b3bf0 con 0x7fec70159eb0 2023-10-06T04:52:00.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.705+0000 7fec65ffb700 1 -- 172.21.15.153:0/2452067905 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fec6c005e20 con 0x7fec70159eb0 2023-10-06T04:52:00.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.706+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec700c7580 con 0x7fec70159eb0 2023-10-06T04:52:00.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.708+0000 7fec65ffb700 1 -- 172.21.15.153:0/2452067905 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fec700c7580 con 0x7fec70159eb0 2023-10-06T04:52:00.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.793+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "af00dc42-0e73-4607-a038-73b65ceee91e", "id": 1} v 0) v1 -- 0x7fec7004c380 con 0x7fec70159eb0 2023-10-06T04:52:00.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.800+0000 7fec65ffb700 1 -- 172.21.15.153:0/2452067905 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af00dc42-0e73-4607-a038-73b65ceee91e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fec6c017070 con 0x7fec70159eb0 2023-10-06T04:52:00.801 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:52:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec70159eb0 msgr2=0x7fec700b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec70159eb0 0x7fec700b14f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c7e0 tx=0x7fec6c007d90 comp rx=0 tx=0).stop 2023-10-06T04:52:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 shutdown_connections 2023-10-06T04:52:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7015da10 0x7fec70155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec700cfb80 0x7fec701535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 --2- 172.21.15.153:0/2452067905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec70159eb0 0x7fec700b14f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 >> 172.21.15.153:0/2452067905 conn(0x7fec70070310 msgr2=0x7fec701545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:00.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 shutdown_connections 2023-10-06T04:52:00.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.801+0000 7fec78109700 1 -- 172.21.15.153:0/2452067905 wait complete. 2023-10-06T04:52:00.815 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde 2 2023-10-06T04:52:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.932+0000 7fa38f242700 1 Processor -- start 2023-10-06T04:52:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.932+0000 7fa38f242700 1 -- start start 2023-10-06T04:52:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38f242700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa38815a820 0x7fa38815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38f242700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38f242700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880ce1b0 con 0x7fa38815a450 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38f242700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa388071200 con 0x7fa38815a820 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38f242700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38815b3b0 con 0x7fa3880c5a80 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38e240700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa38815a450 0x7fa388070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:38126/0 (socket says 172.21.15.153:38126) 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38e240700 1 -- 172.21.15.153:0/2842458088 learned_addr learned my addr 172.21.15.153:0/2842458088 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38ea41700 1 --2- 172.21.15.153:0/2842458088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38da3f700 1 --2- 172.21.15.153:0/2842458088 >> v2:172.21.15.3:3300/0 conn(0x7fa38815a820 0x7fa38815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38d23e700 1 -- 172.21.15.153:0/2842458088 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591985293 0 0) 0x7fa3880ce1b0 con 0x7fa38815a450 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38ea41700 1 -- 172.21.15.153:0/2842458088 >> v2:172.21.15.3:3300/0 conn(0x7fa38815a820 msgr2=0x7fa38815ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38ea41700 1 --2- 172.21.15.153:0/2842458088 >> v2:172.21.15.3:3300/0 conn(0x7fa38815a820 0x7fa38815ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38ea41700 1 -- 172.21.15.153:0/2842458088 >> v1:172.21.15.153:6789/0 conn(0x7fa38815a450 legacy=0x7fa388070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38ea41700 1 -- 172.21.15.153:0/2842458088 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880c6400 con 0x7fa3880c5a80 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.933+0000 7fa38ea41700 1 --2- 172.21.15.153:0/2842458088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880c5ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa380009e60 tx=0x7fa380009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1127cf8e581351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38d23e700 1 -- 172.21.15.153:0/2842458088 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000e070 con 0x7fa3880c5a80 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38d23e700 1 -- 172.21.15.153:0/2842458088 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38000bb60 con 0x7fa3880c5a80 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38d23e700 1 -- 172.21.15.153:0/2842458088 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380012430 con 0x7fa3880c5a80 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 -- 172.21.15.153:0/2842458088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 msgr2=0x7fa3880c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 --2- 172.21.15.153:0/2842458088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880c5ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa380009e60 tx=0x7fa380009780 comp rx=0 tx=0).stop 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 -- 172.21.15.153:0/2842458088 shutdown_connections 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 --2- 172.21.15.153:0/2842458088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880c5ec0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 --2- 172.21.15.153:0/2842458088 >> v2:172.21.15.3:3300/0 conn(0x7fa38815a820 0x7fa38815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 -- 172.21.15.153:0/2842458088 >> 172.21.15.153:0/2842458088 conn(0x7fa388089dd0 msgr2=0x7fa3880996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:00.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 -- 172.21.15.153:0/2842458088 shutdown_connections 2023-10-06T04:52:00.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.934+0000 7fa38f242700 1 -- 172.21.15.153:0/2842458088 wait complete. 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.935+0000 7fa38f242700 1 Processor -- start 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.935+0000 7fa38f242700 1 -- start start 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38f242700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38f242700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa38815a450 0x7fa3881561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38f242700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa38815a820 0x7fa3880cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38f242700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880c7a20 con 0x7fa38815a450 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38f242700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388156700 con 0x7fa38815a820 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38f242700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880c7740 con 0x7fa3880c5a80 2023-10-06T04:52:00.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38ea41700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa38815a820 0x7fa3880cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33506/0 (socket says 172.21.15.153:33506) 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 -- 172.21.15.153:0/2453075175 learned_addr learned my addr 172.21.15.153:0/2453075175 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38da3f700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa38815a450 0x7fa3881561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 -- 172.21.15.153:0/2453075175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa38815a820 msgr2=0x7fa3880cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa38815a820 0x7fa3880cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 -- 172.21.15.153:0/2453075175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa38815a450 msgr2=0x7fa3881561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa38815a450 0x7fa3881561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38e240700 1 -- 172.21.15.153:0/2453075175 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa380009a20 con 0x7fa3880c5a80 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.936+0000 7fa38da3f700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa38815a450 0x7fa3881561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:00.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa38e240700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880afae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa37800c7e0 tx=0x7fa378007df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa37e7fc700 1 -- 172.21.15.153:0/2453075175 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378005b30 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa37e7fc700 1 -- 172.21.15.153:0/2453075175 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378005c90 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3880c9cb0 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa37e7fc700 1 -- 172.21.15.153:0/2453075175 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378007300 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3880b3280 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa37e7fc700 1 -- 172.21.15.153:0/2453075175 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa37801c440 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.937+0000 7fa37e7fc700 1 -- 172.21.15.153:0/2453075175 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fa378005e00 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.938+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3880c9b10 con 0x7fa3880c5a80 2023-10-06T04:52:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:00.940+0000 7fa37e7fc700 1 -- 172.21.15.153:0/2453075175 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa3880c9b10 con 0x7fa3880c5a80 2023-10-06T04:52:00.962 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:00.961+0000 7f7d32bac180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:52:00.982 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:00.981+0000 7f5978c80180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:52:01.022 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.021+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde", "id": 2} v 0) v1 -- 0x7fa388157780 con 0x7fa3880c5a80 2023-10-06T04:52:01.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.025+0000 7fa37e7fc700 1 -- 172.21.15.153:0/2453075175 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa378017070 con 0x7fa3880c5a80 2023-10-06T04:52:01.026 INFO:teuthology.orchestra.run.smithi153.stdout:2 2023-10-06T04:52:01.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 msgr2=0x7fa3880afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880afae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa37800c7e0 tx=0x7fa378007df0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 shutdown_connections 2023-10-06T04:52:01.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa38815a820 0x7fa3880cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3880c5a80 0x7fa3880afae0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 --2- 172.21.15.153:0/2453075175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa38815a450 0x7fa3881561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 >> 172.21.15.153:0/2453075175 conn(0x7fa388089dd0 msgr2=0x7fa38806dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 shutdown_connections 2023-10-06T04:52:01.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.026+0000 7fa38f242700 1 -- 172.21.15.153:0/2453075175 wait complete. 2023-10-06T04:52:01.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 76585c50-61a9-4bff-b04a-4e0f55264fac 3 2023-10-06T04:52:01.100 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:01.099+0000 7f7d32bac180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:52:01.122 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:01.121+0000 7f5978c80180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:52:01.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.154+0000 7f6092fde700 1 Processor -- start 2023-10-06T04:52:01.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.154+0000 7f6092fde700 1 -- start start 2023-10-06T04:52:01.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f6092fde700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f6092fde700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f608c0cef10 0x7f608c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f6092fde700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0afca0 con 0x7f608c0cfb80 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f6092fde700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c15d510 con 0x7f608c0cef10 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f6092fde700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c15a480 con 0x7f608c159eb0 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f6090d7a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f608c0cfb80 0x7f608c0af590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:38130/0 (socket says 172.21.15.153:38130) 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f6090d7a700 1 -- 172.21.15.153:0/3181810771 learned_addr learned my addr 172.21.15.153:0/3181810771 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f609157b700 1 --2- 172.21.15.153:0/3181810771 >> v2:172.21.15.3:3300/0 conn(0x7f608c0cef10 0x7f608c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608b7fe700 1 -- 172.21.15.153:0/3181810771 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789361376 0 0) 0x7f608c0afca0 con 0x7f608c0cfb80 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608b7fe700 1 -- 172.21.15.153:0/3181810771 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c003570 con 0x7f608c0cfb80 2023-10-06T04:52:01.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608bfff700 1 -- 172.21.15.153:0/3181810771 >> v2:172.21.15.3:3300/0 conn(0x7f608c0cef10 msgr2=0x7f608c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608bfff700 1 --2- 172.21.15.153:0/3181810771 >> v2:172.21.15.3:3300/0 conn(0x7f608c0cef10 0x7f608c15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608bfff700 1 -- 172.21.15.153:0/3181810771 >> v1:172.21.15.153:6789/0 conn(0x7f608c0cfb80 legacy=0x7f608c0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608bfff700 1 -- 172.21.15.153:0/3181810771 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c15a760 con 0x7f608c159eb0 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608bfff700 1 --2- 172.21.15.153:0/3181810771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c0ce9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60780098e0 tx=0x7f60780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede6ac644ee908b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608b7fe700 1 -- 172.21.15.153:0/3181810771 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078011630 con 0x7f608c159eb0 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608b7fe700 1 -- 172.21.15.153:0/3181810771 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6078011790 con 0x7f608c159eb0 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.155+0000 7f608b7fe700 1 -- 172.21.15.153:0/3181810771 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078011b30 con 0x7f608c159eb0 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 -- 172.21.15.153:0/3181810771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 msgr2=0x7f608c0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 --2- 172.21.15.153:0/3181810771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c0ce9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60780098e0 tx=0x7f60780091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 -- 172.21.15.153:0/3181810771 shutdown_connections 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 --2- 172.21.15.153:0/3181810771 >> v2:172.21.15.3:3300/0 conn(0x7f608c0cef10 0x7f608c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 --2- 172.21.15.153:0/3181810771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c0ce9d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 -- 172.21.15.153:0/3181810771 >> 172.21.15.153:0/3181810771 conn(0x7f608c070310 msgr2=0x7f608c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 -- 172.21.15.153:0/3181810771 shutdown_connections 2023-10-06T04:52:01.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.156+0000 7f6092fde700 1 -- 172.21.15.153:0/3181810771 wait complete. 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.157+0000 7f6092fde700 1 Processor -- start 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.157+0000 7f6092fde700 1 -- start start 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f6092fde700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c0cef10 0x7f608c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f6092fde700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f608c0cfb80 0x7f608c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f6092fde700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f6092fde700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c15b410 con 0x7f608c0cef10 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f6092fde700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c15c270 con 0x7f608c0cfb80 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f6092fde700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c153550 con 0x7f608c159eb0 2023-10-06T04:52:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f609157b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f608bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f608c0cfb80 0x7f608c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f609157b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c15e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33528/0 (socket says 172.21.15.153:33528) 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f609157b700 1 -- 172.21.15.153:0/1064818672 learned_addr learned my addr 172.21.15.153:0/1064818672 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f608bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f608c0cfb80 0x7f608c15c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57856/0 (socket says 172.21.15.153:57856) 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.158+0000 7f6090d7a700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c0cef10 0x7f608c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f608bfff700 1 -- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 msgr2=0x7f608c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f608bfff700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c15e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f608bfff700 1 -- 172.21.15.153:0/1064818672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c0cef10 msgr2=0x7f608c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f608bfff700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c0cef10 0x7f608c153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f608bfff700 1 -- 172.21.15.153:0/1064818672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6078009420 con 0x7f608c0cfb80 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f609157b700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f608bfff700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f608c0cfb80 0x7f608c15c700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6078016000 tx=0x7f6078017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f6088ff9700 1 -- 172.21.15.153:0/1064818672 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078011900 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f6088ff9700 1 -- 172.21.15.153:0/1064818672 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6078011a60 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f6088ff9700 1 -- 172.21.15.153:0/1064818672 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078025640 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c15b660 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c15bc20 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f6088ff9700 1 -- 172.21.15.153:0/1064818672 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6078025c70 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.159+0000 7f6088ff9700 1 -- 172.21.15.153:0/1064818672 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f607802eca0 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.160+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c04c380 con 0x7f608c0cfb80 2023-10-06T04:52:01.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.162+0000 7f6088ff9700 1 -- 172.21.15.153:0/1064818672 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f6078033060 con 0x7f608c0cfb80 2023-10-06T04:52:01.179 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:01.178+0000 7f7d32bac180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:52:01.201 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:01.200+0000 7f5978c80180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:52:01.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.245+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "76585c50-61a9-4bff-b04a-4e0f55264fac", "id": 3} v 0) v1 -- 0x7f608c15c3b0 con 0x7f608c0cfb80 2023-10-06T04:52:01.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.248+0000 7f6088ff9700 1 -- 172.21.15.153:0/1064818672 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "76585c50-61a9-4bff-b04a-4e0f55264fac", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6078015020 con 0x7f608c0cfb80 2023-10-06T04:52:01.250 INFO:teuthology.orchestra.run.smithi153.stdout:3 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.249+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f608c0cfb80 msgr2=0x7f608c15c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.249+0000 7f6092fde700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f608c0cfb80 0x7f608c15c700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6078016000 tx=0x7f6078017000 comp rx=0 tx=0).stop 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.249+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 shutdown_connections 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.249+0000 7f6092fde700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f608c159eb0 0x7f608c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.249+0000 7f6092fde700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c0cef10 0x7f608c153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.249+0000 7f6092fde700 1 --2- 172.21.15.153:0/1064818672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f608c0cfb80 0x7f608c15c700 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.249+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 >> 172.21.15.153:0/1064818672 conn(0x7f608c070310 msgr2=0x7f608c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.250+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 shutdown_connections 2023-10-06T04:52:01.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.250+0000 7f6092fde700 1 -- 172.21.15.153:0/1064818672 wait complete. 2023-10-06T04:52:01.262 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new b2c6e53d-d13b-4d9a-94c4-d8543698430b 4 2023-10-06T04:52:01.374 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:01.373+0000 7f7d32bac180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:52:01.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.376+0000 7fb960b79700 1 Processor -- start 2023-10-06T04:52:01.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.376+0000 7fb960b79700 1 -- start start 2023-10-06T04:52:01.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.377+0000 7fb960b79700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb95c0b0fc0 0x7fb95c0aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.377+0000 7fb960b79700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c06b720 con 0x7fb95c1593f0 2023-10-06T04:52:01.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.377+0000 7fb960b79700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c06ab50 con 0x7fb95c06ca80 2023-10-06T04:52:01.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.377+0000 7fb960b79700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c06adf0 con 0x7fb95c0b0fc0 2023-10-06T04:52:01.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.377+0000 7fb959d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb95c06ca80 0x7fb95c0a9ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:38132/0 (socket says 172.21.15.153:38132) 2023-10-06T04:52:01.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.377+0000 7fb959d9b700 1 -- 172.21.15.153:0/2070411175 learned_addr learned my addr 172.21.15.153:0/2070411175 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-10-06T04:52:01.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95ad9d700 1 --2- 172.21.15.153:0/2070411175 >> v2:172.21.15.3:3300/0 conn(0x7fb95c0b0fc0 0x7fb95c0aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95959a700 1 -- 172.21.15.153:0/2070411175 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396834187 0 0) 0x7fb95c06ab50 con 0x7fb95c06ca80 2023-10-06T04:52:01.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95959a700 1 -- 172.21.15.153:0/2070411175 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb944003570 con 0x7fb95c06ca80 2023-10-06T04:52:01.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95959a700 1 -- 172.21.15.153:0/2070411175 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647548900 0 0) 0x7fb95c06b720 con 0x7fb95c1593f0 2023-10-06T04:52:01.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95959a700 1 -- 172.21.15.153:0/2070411175 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb95c06ab50 con 0x7fb95c1593f0 2023-10-06T04:52:01.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95ad9d700 1 -- 172.21.15.153:0/2070411175 >> v1:172.21.15.153:6789/0 conn(0x7fb95c06ca80 legacy=0x7fb95c0a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95ad9d700 1 -- 172.21.15.153:0/2070411175 >> v1:172.21.15.3:6789/0 conn(0x7fb95c1593f0 legacy=0x7fb95c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95ad9d700 1 -- 172.21.15.153:0/2070411175 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c06b0d0 con 0x7fb95c0b0fc0 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95ad9d700 1 --2- 172.21.15.153:0/2070411175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c0b0fc0 0x7fb95c0aa1e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb954009e60 tx=0x7fb954009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60492a0945acbb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95959a700 1 -- 172.21.15.153:0/2070411175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400e070 con 0x7fb95c0b0fc0 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95959a700 1 -- 172.21.15.153:0/2070411175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95400bab0 con 0x7fb95c0b0fc0 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.378+0000 7fb95959a700 1 -- 172.21.15.153:0/2070411175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400be70 con 0x7fb95c0b0fc0 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.379+0000 7fb960b79700 1 -- 172.21.15.153:0/2070411175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c0b0fc0 msgr2=0x7fb95c0aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.379+0000 7fb960b79700 1 --2- 172.21.15.153:0/2070411175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c0b0fc0 0x7fb95c0aa1e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb954009e60 tx=0x7fb954009780 comp rx=0 tx=0).stop 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.379+0000 7fb960b79700 1 -- 172.21.15.153:0/2070411175 shutdown_connections 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.379+0000 7fb960b79700 1 --2- 172.21.15.153:0/2070411175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c0b0fc0 0x7fb95c0aa1e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.379+0000 7fb960b79700 1 -- 172.21.15.153:0/2070411175 >> 172.21.15.153:0/2070411175 conn(0x7fb95c06df50 msgr2=0x7fb95c0aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.379+0000 7fb960b79700 1 -- 172.21.15.153:0/2070411175 shutdown_connections 2023-10-06T04:52:01.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.379+0000 7fb960b79700 1 -- 172.21.15.153:0/2070411175 wait complete. 2023-10-06T04:52:01.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.380+0000 7fb960b79700 1 Processor -- start 2023-10-06T04:52:01.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.380+0000 7fb960b79700 1 -- start start 2023-10-06T04:52:01.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb960b79700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb95c06ca80 0x7fb95c156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb960b79700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb95c0b0fc0 0x7fb95c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb960b79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c1593f0 0x7fb95c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb960b79700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c156d40 con 0x7fb95c1593f0 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb960b79700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c16ac80 con 0x7fb95c06ca80 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb960b79700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0cce40 con 0x7fb95c0b0fc0 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb95c06ca80 0x7fb95c156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb959d9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb95c0b0fc0 0x7fb95c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb95c06ca80 0x7fb95c156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57858/0 (socket says 172.21.15.153:57858) 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 -- 172.21.15.153:0/3617137417 learned_addr learned my addr 172.21.15.153:0/3617137417 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95ad9d700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c1593f0 0x7fb95c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 -- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb95c0b0fc0 msgr2=0x7fb95c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb95c0b0fc0 0x7fb95c0cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 -- 172.21.15.153:0/3617137417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c1593f0 msgr2=0x7fb95c152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c1593f0 0x7fb95c152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 -- 172.21.15.153:0/3617137417 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb954009a20 con 0x7fb95c06ca80 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb959d9b700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb95c0b0fc0 0x7fb95c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95ad9d700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c1593f0 0x7fb95c152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb95a59c700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb95c06ca80 0x7fb95c156800 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c5b0 tx=0x7fb950007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb94a7fc700 1 -- 172.21.15.153:0/3617137417 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950010510 con 0x7fb95c06ca80 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb94a7fc700 1 -- 172.21.15.153:0/3617137417 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb950010670 con 0x7fb95c06ca80 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb94a7fc700 1 -- 172.21.15.153:0/3617137417 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950010a30 con 0x7fb95c06ca80 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.381+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c152b50 con 0x7fb95c06ca80 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.382+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c157250 con 0x7fb95c06ca80 2023-10-06T04:52:01.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.382+0000 7fb94a7fc700 1 -- 172.21.15.153:0/3617137417 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb95001b400 con 0x7fb95c06ca80 2023-10-06T04:52:01.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.382+0000 7fb94a7fc700 1 -- 172.21.15.153:0/3617137417 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fb950010e20 con 0x7fb95c06ca80 2023-10-06T04:52:01.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.382+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c16ae30 con 0x7fb95c06ca80 2023-10-06T04:52:01.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.384+0000 7fb94a7fc700 1 -- 172.21.15.153:0/3617137417 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb95001b890 con 0x7fb95c06ca80 2023-10-06T04:52:01.393 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:01.392+0000 7f5978c80180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:52:01.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.466+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b2c6e53d-d13b-4d9a-94c4-d8543698430b", "id": 4} v 0) v1 -- 0x7fb95c04c380 con 0x7fb95c06ca80 2023-10-06T04:52:01.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.470+0000 7fb94a7fc700 1 -- 172.21.15.153:0/3617137417 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b2c6e53d-d13b-4d9a-94c4-d8543698430b", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb95001ba30 con 0x7fb95c06ca80 2023-10-06T04:52:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:4 2023-10-06T04:52:01.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb95c06ca80 msgr2=0x7fb95c156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb95c06ca80 0x7fb95c156800 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c5b0 tx=0x7fb950007e10 comp rx=0 tx=0).stop 2023-10-06T04:52:01.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 shutdown_connections 2023-10-06T04:52:01.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb95c06ca80 0x7fb95c156800 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb95c1593f0 0x7fb95c152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 --2- 172.21.15.153:0/3617137417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb95c0b0fc0 0x7fb95c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 >> 172.21.15.153:0/3617137417 conn(0x7fb95c06df50 msgr2=0x7fb95c153880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 shutdown_connections 2023-10-06T04:52:01.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.471+0000 7fb960b79700 1 -- 172.21.15.153:0/3617137417 wait complete. 2023-10-06T04:52:01.484 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 12e26508-a4e6-4c5a-9a52-a4c718ecf083 5 2023-10-06T04:52:01.555 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:01.554+0000 7f7d32bac180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:52:01.575 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:01.574+0000 7f5978c80180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:52:01.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.598+0000 7f47c14c4700 1 Processor -- start 2023-10-06T04:52:01.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.598+0000 7f47c14c4700 1 -- start start 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47c14c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47bc159eb0 0x7f47bc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47c14c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc0afca0 con 0x7f47bc0cfb80 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47c14c4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc15d510 con 0x7f47bc0cef10 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47c14c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc15a480 con 0x7f47bc159eb0 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47bb7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f47bc0cef10 0x7f47bc15da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:38140/0 (socket says 172.21.15.153:38140) 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47bb7fe700 1 -- 172.21.15.153:0/418051360 learned_addr learned my addr 172.21.15.153:0/418051360 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47ba7fc700 1 --2- 172.21.15.153:0/418051360 >> v2:172.21.15.3:3300/0 conn(0x7f47bc159eb0 0x7f47bc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47b9ffb700 1 -- 172.21.15.153:0/418051360 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087946881 0 0) 0x7f47bc15d510 con 0x7f47bc0cef10 2023-10-06T04:52:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47b9ffb700 1 -- 172.21.15.153:0/418051360 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a4003570 con 0x7f47bc0cef10 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47b9ffb700 1 -- 172.21.15.153:0/418051360 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623471844 0 0) 0x7f47bc0afca0 con 0x7f47bc0cfb80 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47b9ffb700 1 -- 172.21.15.153:0/418051360 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47bc15d510 con 0x7f47bc0cfb80 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47ba7fc700 1 -- 172.21.15.153:0/418051360 >> v1:172.21.15.153:6789/0 conn(0x7f47bc0cef10 legacy=0x7f47bc15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47ba7fc700 1 -- 172.21.15.153:0/418051360 >> v1:172.21.15.3:6789/0 conn(0x7f47bc0cfb80 legacy=0x7f47bc0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.599+0000 7f47ba7fc700 1 -- 172.21.15.153:0/418051360 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc15a760 con 0x7f47bc159eb0 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47ba7fc700 1 --2- 172.21.15.153:0/418051360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 0x7f47bc0ce9d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0098e0 tx=0x7f47ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd7efb04c166efc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47b9ffb700 1 -- 172.21.15.153:0/418051360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac011630 con 0x7f47bc159eb0 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47b9ffb700 1 -- 172.21.15.153:0/418051360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac011790 con 0x7f47bc159eb0 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47b9ffb700 1 -- 172.21.15.153:0/418051360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac011b30 con 0x7f47bc159eb0 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47c14c4700 1 -- 172.21.15.153:0/418051360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 msgr2=0x7f47bc0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47c14c4700 1 --2- 172.21.15.153:0/418051360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 0x7f47bc0ce9d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0098e0 tx=0x7f47ac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47c14c4700 1 -- 172.21.15.153:0/418051360 shutdown_connections 2023-10-06T04:52:01.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47c14c4700 1 --2- 172.21.15.153:0/418051360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 0x7f47bc0ce9d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47c14c4700 1 -- 172.21.15.153:0/418051360 >> 172.21.15.153:0/418051360 conn(0x7f47bc070310 msgr2=0x7f47bc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47c14c4700 1 -- 172.21.15.153:0/418051360 shutdown_connections 2023-10-06T04:52:01.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.600+0000 7f47c14c4700 1 -- 172.21.15.153:0/418051360 wait complete. 2023-10-06T04:52:01.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 Processor -- start 2023-10-06T04:52:01.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 -- start start 2023-10-06T04:52:01.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc0cef10 0x7f47bc0b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47bc0cfb80 0x7f47bc156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 0x7f47bc0c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0b2170 con 0x7f47bc159eb0 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc16d110 con 0x7f47bc0cef10 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47c14c4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc16d3f0 con 0x7f47bc0cfb80 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47baffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc0cef10 0x7f47bc0b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47ba7fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47bc0cfb80 0x7f47bc156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47baffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc0cef10 0x7f47bc0b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57874/0 (socket says 172.21.15.153:57874) 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47baffd700 1 -- 172.21.15.153:0/3745217155 learned_addr learned my addr 172.21.15.153:0/3745217155 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.602+0000 7f47bb7fe700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 0x7f47bc0c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47baffd700 1 -- 172.21.15.153:0/3745217155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47bc0cfb80 msgr2=0x7f47bc156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47baffd700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47bc0cfb80 0x7f47bc156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47baffd700 1 -- 172.21.15.153:0/3745217155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 msgr2=0x7f47bc0c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47baffd700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 0x7f47bc0c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47baffd700 1 -- 172.21.15.153:0/3745217155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac009420 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47baffd700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc0cef10 0x7f47bc0b1c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c790 tx=0x7f47b0007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47a2ffd700 1 -- 172.21.15.153:0/3745217155 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0010510 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47a2ffd700 1 -- 172.21.15.153:0/3745217155 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0010670 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc16d6d0 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47a2ffd700 1 -- 172.21.15.153:0/3745217155 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0010a10 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.603+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc157270 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.604+0000 7f47a2ffd700 1 -- 172.21.15.153:0/3745217155 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f47b001c6c0 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.604+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47bc0ced50 con 0x7f47bc0cef10 2023-10-06T04:52:01.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.604+0000 7f47a2ffd700 1 -- 172.21.15.153:0/3745217155 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f47b0010e00 con 0x7f47bc0cef10 2023-10-06T04:52:01.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.606+0000 7f47a2ffd700 1 -- 172.21.15.153:0/3745217155 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f47b0017070 con 0x7f47bc0cef10 2023-10-06T04:52:01.655 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:01.654+0000 7f7d32bac180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:52:01.656 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:01.655+0000 7f5978c80180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:52:01.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.718+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "12e26508-a4e6-4c5a-9a52-a4c718ecf083", "id": 5} v 0) v1 -- 0x7f47bc04c380 con 0x7f47bc0cef10 2023-10-06T04:52:01.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.722+0000 7f47a2ffd700 1 -- 172.21.15.153:0/3745217155 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12e26508-a4e6-4c5a-9a52-a4c718ecf083", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f47b001cc10 con 0x7f47bc0cef10 2023-10-06T04:52:01.723 INFO:teuthology.orchestra.run.smithi153.stdout:5 2023-10-06T04:52:01.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc0cef10 msgr2=0x7f47bc0b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc0cef10 0x7f47bc0b1c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c790 tx=0x7f47b0007e10 comp rx=0 tx=0).stop 2023-10-06T04:52:01.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 shutdown_connections 2023-10-06T04:52:01.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc0cef10 0x7f47bc0b1c30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47bc159eb0 0x7f47bc0c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 --2- 172.21.15.153:0/3745217155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47bc0cfb80 0x7f47bc156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 >> 172.21.15.153:0/3745217155 conn(0x7f47bc070310 msgr2=0x7f47bc15b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 shutdown_connections 2023-10-06T04:52:01.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.723+0000 7f47c14c4700 1 -- 172.21.15.153:0/3745217155 wait complete. 2023-10-06T04:52:01.736 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new e127d2fe-d41c-4f6a-9936-acac760c6a38 6 2023-10-06T04:52:01.797 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:01.796+0000 7f5978c80180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:52:01.801 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:01.800+0000 7f7d32bac180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:52:01.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.853+0000 7fd911e2d700 1 Processor -- start 2023-10-06T04:52:01.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.853+0000 7fd911e2d700 1 -- start start 2023-10-06T04:52:01.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.854+0000 7fd911e2d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd90c0af660 0x7fd90c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd911e2d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0cd190 0x7fd90c0c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd911e2d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c0cb190 con 0x7fd90c0af290 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd911e2d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90c159950 con 0x7fd90c0af660 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd911e2d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0cd6a0 con 0x7fd90c0cd190 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd90bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd90c0af660 0x7fd90c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd91162c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0cd190 0x7fd90c0c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd910e2b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd90c0af290 0x7fd90c0ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:38142/0 (socket says 172.21.15.153:38142) 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd910e2b700 1 -- 172.21.15.153:0/1043025295 learned_addr learned my addr 172.21.15.153:0/1043025295 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd91162c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0cd190 0x7fd90c0c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33536/0 (socket says 172.21.15.153:33536) 2023-10-06T04:52:01.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd90bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd90c0af660 0x7fd90c0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57884/0 (socket says 172.21.15.153:57884) 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd91162c700 1 -- 172.21.15.153:0/1043025295 >> v2:172.21.15.153:3300/0 conn(0x7fd90c0af660 msgr2=0x7fd90c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd91162c700 1 --2- 172.21.15.153:0/1043025295 >> v2:172.21.15.153:3300/0 conn(0x7fd90c0af660 0x7fd90c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd91162c700 1 -- 172.21.15.153:0/1043025295 >> v1:172.21.15.153:6789/0 conn(0x7fd90c0af290 legacy=0x7fd90c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd90b7fe700 1 -- 172.21.15.153:0/1043025295 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577281256 0 0) 0x7fd90c0cb190 con 0x7fd90c0af290 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.855+0000 7fd91162c700 1 -- 172.21.15.153:0/1043025295 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0cb3f0 con 0x7fd90c0cd190 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd91162c700 1 --2- 172.21.15.153:0/1043025295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0cd190 0x7fd90c0c5c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd904009e60 tx=0x7fd904009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f60b4ebae8a8910 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd90b7fe700 1 -- 172.21.15.153:0/1043025295 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400e070 con 0x7fd90c0cd190 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd90b7fe700 1 -- 172.21.15.153:0/1043025295 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90400bb60 con 0x7fd90c0cd190 2023-10-06T04:52:01.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd90b7fe700 1 -- 172.21.15.153:0/1043025295 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904012430 con 0x7fd90c0cd190 2023-10-06T04:52:01.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd911e2d700 1 -- 172.21.15.153:0/1043025295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0cd190 msgr2=0x7fd90c0c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd911e2d700 1 --2- 172.21.15.153:0/1043025295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0cd190 0x7fd90c0c5c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd904009e60 tx=0x7fd904009780 comp rx=0 tx=0).stop 2023-10-06T04:52:01.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd911e2d700 1 -- 172.21.15.153:0/1043025295 shutdown_connections 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd911e2d700 1 --2- 172.21.15.153:0/1043025295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0cd190 0x7fd90c0c5c20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd911e2d700 1 --2- 172.21.15.153:0/1043025295 >> v2:172.21.15.153:3300/0 conn(0x7fd90c0af660 0x7fd90c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd911e2d700 1 -- 172.21.15.153:0/1043025295 >> 172.21.15.153:0/1043025295 conn(0x7fd90c089dd0 msgr2=0x7fd90c0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.856+0000 7fd911e2d700 1 -- 172.21.15.153:0/1043025295 shutdown_connections 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.857+0000 7fd911e2d700 1 -- 172.21.15.153:0/1043025295 wait complete. 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 Processor -- start 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 -- start start 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd90c0af290 0x7fd90c148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0af660 0x7fd90c1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c0cd190 0x7fd90c157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c149320 con 0x7fd90c0cd190 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0b1600 con 0x7fd90c0af290 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.858+0000 7fd911e2d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0b18a0 con 0x7fd90c0af660 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0af660 0x7fd90c1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd910e2b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd90c0af290 0x7fd90c148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0af660 0x7fd90c1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33548/0 (socket says 172.21.15.153:33548) 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 -- 172.21.15.153:0/1614832011 learned_addr learned my addr 172.21.15.153:0/1614832011 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd910e2b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd90c0af290 0x7fd90c148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57900/0 (socket says 172.21.15.153:57900) 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd91162c700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c0cd190 0x7fd90c157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 -- 172.21.15.153:0/1614832011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd90c0af290 msgr2=0x7fd90c148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd90c0af290 0x7fd90c148de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 -- 172.21.15.153:0/1614832011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c0cd190 msgr2=0x7fd90c157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c0cd190 0x7fd90c157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 -- 172.21.15.153:0/1614832011 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904009a20 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd90bfff700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0af660 0x7fd90c1575a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800ec80 tx=0x7fd8f8009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd908ff9700 1 -- 172.21.15.153:0/1614832011 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800c850 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd908ff9700 1 -- 172.21.15.153:0/1614832011 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f800c9b0 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c0b1b80 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd908ff9700 1 -- 172.21.15.153:0/1614832011 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800cd30 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c0b2140 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd908ff9700 1 -- 172.21.15.153:0/1614832011 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd8f8013900 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.859+0000 7fd908ff9700 1 -- 172.21.15.153:0/1614832011 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fd8f801cbe0 con 0x7fd90c0af660 2023-10-06T04:52:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.860+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c04c380 con 0x7fd90c0af660 2023-10-06T04:52:01.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.862+0000 7fd908ff9700 1 -- 172.21.15.153:0/1614832011 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd8f8021060 con 0x7fd90c0af660 2023-10-06T04:52:01.897 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:01.896+0000 7f5978c80180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:52:01.903 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:01.902+0000 7f7d32bac180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:52:01.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.945+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e127d2fe-d41c-4f6a-9936-acac760c6a38", "id": 6} v 0) v1 -- 0x7fd90c0cc290 con 0x7fd90c0af660 2023-10-06T04:52:01.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.951+0000 7fd908ff9700 1 -- 172.21.15.153:0/1614832011 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e127d2fe-d41c-4f6a-9936-acac760c6a38", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd8f8017070 con 0x7fd90c0af660 2023-10-06T04:52:01.952 INFO:teuthology.orchestra.run.smithi153.stdout:6 2023-10-06T04:52:01.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0af660 msgr2=0x7fd90c1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:01.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0af660 0x7fd90c1575a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800ec80 tx=0x7fd8f8009120 comp rx=0 tx=0).stop 2023-10-06T04:52:01.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 shutdown_connections 2023-10-06T04:52:01.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd90c0af290 0x7fd90c148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c0cd190 0x7fd90c157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 --2- 172.21.15.153:0/1614832011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd90c0af660 0x7fd90c1575a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:01.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 >> 172.21.15.153:0/1614832011 conn(0x7fd90c089dd0 msgr2=0x7fd90c158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:01.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 shutdown_connections 2023-10-06T04:52:01.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:01.952+0000 7fd911e2d700 1 -- 172.21.15.153:0/1614832011 wait complete. 2023-10-06T04:52:01.965 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 09729f85-d60d-44ba-a198-bbfd41694af0 7 2023-10-06T04:52:02.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.079+0000 7fa5049cd700 1 Processor -- start 2023-10-06T04:52:02.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.079+0000 7fa5049cd700 1 -- start start 2023-10-06T04:52:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa5049cd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5000cc140 0x7fa500159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa5049cd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa50015a330 0x7fa5000ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa5049cd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5000ce420 con 0x7fa5000cc510 2023-10-06T04:52:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa5049cd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50015a880 con 0x7fa5000cc140 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa5049cd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50015ab60 con 0x7fa50015a330 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa50015a330 0x7fa5000ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa50015a330 0x7fa5000ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57912/0 (socket says 172.21.15.153:57912) 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fe59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5000cc140 0x7fa500159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fed9d700 1 -- 172.21.15.153:0/909015918 learned_addr learned my addr 172.21.15.153:0/909015918 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4f5d9b700 1 --1- 172.21.15.153:0/909015918 >> v1:172.21.15.3:6789/0 conn(0x7fa5000cc510 0x7fa5000aef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.153:58518/0 (socket says 172.21.15.153:58518) 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fed9d700 1 -- 172.21.15.153:0/909015918 >> v2:172.21.15.3:3300/0 conn(0x7fa5000cc140 msgr2=0x7fa500159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fed9d700 1 --2- 172.21.15.153:0/909015918 >> v2:172.21.15.3:3300/0 conn(0x7fa5000cc140 0x7fa500159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fed9d700 1 -- 172.21.15.153:0/909015918 >> v1:172.21.15.3:6789/0 conn(0x7fa5000cc510 legacy=0x7fa5000aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.080+0000 7fa4fed9d700 1 -- 172.21.15.153:0/909015918 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000ce0b0 con 0x7fa50015a330 2023-10-06T04:52:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa4fed9d700 1 --2- 172.21.15.153:0/909015918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa5000ace50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa500063ea0 tx=0x7fa4f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503714bedf8b1517 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa4fdd9b700 1 -- 172.21.15.153:0/909015918 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800e070 con 0x7fa50015a330 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa4fdd9b700 1 -- 172.21.15.153:0/909015918 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f800ba20 con 0x7fa50015a330 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa4fdd9b700 1 -- 172.21.15.153:0/909015918 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80092e0 con 0x7fa50015a330 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 -- 172.21.15.153:0/909015918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 msgr2=0x7fa5000ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 --2- 172.21.15.153:0/909015918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa5000ace50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa500063ea0 tx=0x7fa4f8009840 comp rx=0 tx=0).stop 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 -- 172.21.15.153:0/909015918 shutdown_connections 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 --2- 172.21.15.153:0/909015918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa5000ace50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 --2- 172.21.15.153:0/909015918 >> v2:172.21.15.3:3300/0 conn(0x7fa5000cc140 0x7fa500159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 -- 172.21.15.153:0/909015918 >> 172.21.15.153:0/909015918 conn(0x7fa5000d03e0 msgr2=0x7fa5000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 -- 172.21.15.153:0/909015918 shutdown_connections 2023-10-06T04:52:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.081+0000 7fa5049cd700 1 -- 172.21.15.153:0/909015918 wait complete. 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 Processor -- start 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 -- start start 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5000cc140 0x7fa500153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5000cc510 0x7fa50016cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa50016d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000c5cb0 con 0x7fa5000cc510 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000c5f90 con 0x7fa50015a330 2023-10-06T04:52:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa5049cd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000c6270 con 0x7fa5000cc140 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa4fe59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5000cc140 0x7fa500153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa4fe59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5000cc140 0x7fa500153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:33558/0 (socket says 172.21.15.153:33558) 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa4fe59c700 1 -- 172.21.15.153:0/2195435657 learned_addr learned my addr 172.21.15.153:0/2195435657 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa4fed9d700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa50016d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.083+0000 7fa4f5d9b700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5000cc510 0x7fa50016cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4fed9d700 1 -- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5000cc140 msgr2=0x7fa500153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4fed9d700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5000cc140 0x7fa500153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4fed9d700 1 -- 172.21.15.153:0/2195435657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5000cc510 msgr2=0x7fa50016cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4fed9d700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5000cc510 0x7fa50016cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4fed9d700 1 -- 172.21.15.153:0/2195435657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8009ae0 con 0x7fa50015a330 2023-10-06T04:52:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4fe59c700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5000cc140 0x7fa500153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4fed9d700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa50016d400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8015000 tx=0x7fa4f8018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4f6ffd700 1 -- 172.21.15.153:0/2195435657 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8008a80 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4f6ffd700 1 -- 172.21.15.153:0/2195435657 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f8008c20 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4f6ffd700 1 -- 172.21.15.153:0/2195435657 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8025460 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5000c6550 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa500153f50 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4f6ffd700 1 -- 172.21.15.153:0/2195435657 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa4f8003040 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.084+0000 7fa4f6ffd700 1 -- 172.21.15.153:0/2195435657 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fa4f8025c20 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.085+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5000c63b0 con 0x7fa50015a330 2023-10-06T04:52:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.087+0000 7fa4f6ffd700 1 -- 172.21.15.153:0/2195435657 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa5000c63b0 con 0x7fa50015a330 2023-10-06T04:52:02.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.169+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "09729f85-d60d-44ba-a198-bbfd41694af0", "id": 7} v 0) v1 -- 0x7fa50004c380 con 0x7fa50015a330 2023-10-06T04:52:02.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.177+0000 7fa4f6ffd700 1 -- 172.21.15.153:0/2195435657 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09729f85-d60d-44ba-a198-bbfd41694af0", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa4f8020070 con 0x7fa50015a330 2023-10-06T04:52:02.178 INFO:teuthology.orchestra.run.smithi153.stdout:7 2023-10-06T04:52:02.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 msgr2=0x7fa50016d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa50016d400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8015000 tx=0x7fa4f8018000 comp rx=0 tx=0).stop 2023-10-06T04:52:02.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 shutdown_connections 2023-10-06T04:52:02.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50015a330 0x7fa50016d400 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5000cc140 0x7fa500153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 --2- 172.21.15.153:0/2195435657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5000cc510 0x7fa50016cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 >> 172.21.15.153:0/2195435657 conn(0x7fa5000d03e0 msgr2=0x7fa50006d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:02.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 shutdown_connections 2023-10-06T04:52:02.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-10-06T04:52:02.178+0000 7fa5049cd700 1 -- 172.21.15.153:0/2195435657 wait complete. 2023-10-06T04:52:02.192 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T04:52:02.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T04:52:02.195 INFO:tasks.ceph.osd.0:Started 2023-10-06T04:52:02.195 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T04:52:02.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T04:52:02.197 INFO:tasks.ceph.osd.1:Started 2023-10-06T04:52:02.197 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T04:52:02.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T04:52:02.199 INFO:tasks.ceph.osd.2:Started 2023-10-06T04:52:02.200 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T04:52:02.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T04:52:02.201 INFO:tasks.ceph.osd.3:Started 2023-10-06T04:52:02.202 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T04:52:02.202 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T04:52:02.204 INFO:tasks.ceph.osd.4:Started 2023-10-06T04:52:02.204 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T04:52:02.204 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T04:52:02.206 INFO:tasks.ceph.osd.5:Started 2023-10-06T04:52:02.206 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T04:52:02.207 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T04:52:02.211 INFO:tasks.ceph.osd.6:Started 2023-10-06T04:52:02.211 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T04:52:02.211 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T04:52:02.213 INFO:tasks.ceph.osd.7:Started 2023-10-06T04:52:02.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:02.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.357+0000 7f63dc019700 1 Processor -- start 2023-10-06T04:52:02.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.358+0000 7f63dc019700 1 -- start start 2023-10-06T04:52:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.358+0000 7f63dc019700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f63d4159460 0x7f63d40a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.358+0000 7f63dc019700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d406ca80 0x7f63d40aff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.358+0000 7f63dc019700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d406aa20 con 0x7f63d4159830 2023-10-06T04:52:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.358+0000 7f63dc019700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d406b2c0 con 0x7f63d4159460 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.358+0000 7f63dc019700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40aa2d0 con 0x7f63d406ca80 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d95b4700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f63d4159830 0x7f63d406c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38668/0 (socket says 172.21.15.3:38668) 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d95b4700 1 -- 172.21.15.3:0/3599138287 learned_addr learned my addr 172.21.15.3:0/3599138287 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63da5b6700 1 --2- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d406ca80 0x7f63d40aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d9db5700 1 --2- 172.21.15.3:0/3599138287 >> v2:172.21.15.153:3300/0 conn(0x7f63d4159460 0x7f63d40a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d8db3700 1 -- 172.21.15.3:0/3599138287 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107042886 0 0) 0x7f63d406aa20 con 0x7f63d4159830 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d8db3700 1 -- 172.21.15.3:0/3599138287 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c0003570 con 0x7f63d4159830 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d9db5700 1 -- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d406ca80 msgr2=0x7f63d40aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d9db5700 1 --2- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d406ca80 0x7f63d40aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d9db5700 1 -- 172.21.15.3:0/3599138287 >> v1:172.21.15.3:6789/0 conn(0x7f63d4159830 legacy=0x7f63d406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d9db5700 1 -- 172.21.15.3:0/3599138287 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d406ac70 con 0x7f63d4159460 2023-10-06T04:52:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63da5b6700 1 --2- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d406ca80 0x7f63d40aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d9db5700 1 --2- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 0x7f63d40a9850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63c40098e0 tx=0x7f63c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ec1205d979b2a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d8db3700 1 -- 172.21.15.3:0/3599138287 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011630 con 0x7f63d4159460 2023-10-06T04:52:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d8db3700 1 -- 172.21.15.3:0/3599138287 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4011790 con 0x7f63d4159460 2023-10-06T04:52:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.359+0000 7f63d8db3700 1 -- 172.21.15.3:0/3599138287 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011b30 con 0x7f63d4159460 2023-10-06T04:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 -- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 msgr2=0x7f63d40a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 --2- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 0x7f63d40a9850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63c40098e0 tx=0x7f63c40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 -- 172.21.15.3:0/3599138287 shutdown_connections 2023-10-06T04:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 --2- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d406ca80 0x7f63d40aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 --2- 172.21.15.3:0/3599138287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 0x7f63d40a9850 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 -- 172.21.15.3:0/3599138287 >> 172.21.15.3:0/3599138287 conn(0x7f63d406df50 msgr2=0x7f63d40aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 -- 172.21.15.3:0/3599138287 shutdown_connections 2023-10-06T04:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.360+0000 7f63dc019700 1 -- 172.21.15.3:0/3599138287 wait complete. 2023-10-06T04:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.361+0000 7f63dc019700 1 Processor -- start 2023-10-06T04:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.361+0000 7f63dc019700 1 -- start start 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63dc019700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d406ca80 0x7f63d40cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63dc019700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 0x7f63d40b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63dc019700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d415e170 0x7f63d415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63dc019700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d415eb50 con 0x7f63d406ca80 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63dc019700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d415bc20 con 0x7f63d4159460 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63dc019700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40b1bd0 con 0x7f63d415e170 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63d9db5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d406ca80 0x7f63d40cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63d9db5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d406ca80 0x7f63d40cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44192/0 (socket says 172.21.15.3:44192) 2023-10-06T04:52:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63d9db5700 1 -- 172.21.15.3:0/1897151094 learned_addr learned my addr 172.21.15.3:0/1897151094 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63d95b4700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 0x7f63d40b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.362+0000 7f63da5b6700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d415e170 0x7f63d415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63d9db5700 1 -- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d415e170 msgr2=0x7f63d415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63d9db5700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d415e170 0x7f63d415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63d9db5700 1 -- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 msgr2=0x7f63d40b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63d9db5700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 0x7f63d40b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63d9db5700 1 -- 172.21.15.3:0/1897151094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc009a20 con 0x7f63d406ca80 2023-10-06T04:52:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63d9db5700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d406ca80 0x7f63d40cbb20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63d40ce290 tx=0x7f63c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1897151094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011b10 con 0x7f63d406ca80 2023-10-06T04:52:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1897151094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4000cc0 con 0x7f63d406ca80 2023-10-06T04:52:02.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c4009420 con 0x7f63d406ca80 2023-10-06T04:52:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1897151094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4025620 con 0x7f63d406ca80 2023-10-06T04:52:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d40b2190 con 0x7f63d406ca80 2023-10-06T04:52:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1897151094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f63c4025c90 con 0x7f63d406ca80 2023-10-06T04:52:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.363+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1897151094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f63c40193b0 con 0x7f63d406ca80 2023-10-06T04:52:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.364+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d4068640 con 0x7f63d406ca80 2023-10-06T04:52:02.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.366+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1897151094 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f63c4033060 con 0x7f63d406ca80 2023-10-06T04:52:02.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.437+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63d4071680 con 0x7f63d406ca80 2023-10-06T04:52:02.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.437+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1897151094 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f63c4015020 con 0x7f63d406ca80 2023-10-06T04:52:02.440 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:02.440 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:02.170377+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d406ca80 msgr2=0x7f63d40cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d406ca80 0x7f63d40cbb20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63d40ce290 tx=0x7f63c4017000 comp rx=0 tx=0).stop 2023-10-06T04:52:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 shutdown_connections 2023-10-06T04:52:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d415e170 0x7f63d415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d406ca80 0x7f63d40cbb20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 --2- 172.21.15.3:0/1897151094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d4159460 0x7f63d40b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 >> 172.21.15.3:0/1897151094 conn(0x7f63d406df50 msgr2=0x7f63d4154980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:02.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 shutdown_connections 2023-10-06T04:52:02.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:02.438+0000 7f63dc019700 1 -- 172.21.15.3:0/1897151094 wait complete. 2023-10-06T04:52:02.450 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:02.449+0000 7f7d32bac180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:52:02.452 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T04:52:02.452 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T04:52:02.467 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:02.466+0000 7f5978c80180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:52:02.527 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:02.526+0000 7f7d32bac180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:52:02.549 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:02.548+0000 7f5978c80180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:52:02.606 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:02.605+0000 7f7d32bac180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:52:02.630 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:02.629+0000 7f5978c80180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:52:02.798 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-06T04:52:02.796+0000 7f095f94f540 -1 Falling back to public interface 2023-10-06T04:52:02.804 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-10-06T04:52:02.803+0000 7f351cadc540 -1 Falling back to public interface 2023-10-06T04:52:02.807 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-06T04:52:02.805+0000 7f6486dc4540 -1 Falling back to public interface 2023-10-06T04:52:02.807 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-10-06T04:52:02.806+0000 7fc170914540 -1 Falling back to public interface 2023-10-06T04:52:02.811 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-06T04:52:02.809+0000 7f8354322540 -1 Falling back to public interface 2023-10-06T04:52:02.832 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-06T04:52:02.830+0000 7fd35458c540 -1 Falling back to public interface 2023-10-06T04:52:02.835 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-10-06T04:52:02.834+0000 7fc722e74540 -1 Falling back to public interface 2023-10-06T04:52:02.836 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-10-06T04:52:02.835+0000 7f878a1ee540 -1 Falling back to public interface 2023-10-06T04:52:02.837 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-10-06T04:52:02.836+0000 7fc71d29d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6823/0: (98) Address already in use 2023-10-06T04:52:03.202 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:03.201+0000 7f7d32bac180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:52:03.243 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:03.242+0000 7f5978c80180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:52:03.258 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:03.340 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:03.339+0000 7f7d32bac180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:52:03.368 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.367+0000 7fb4ef8c3700 1 Processor -- start 2023-10-06T04:52:03.369 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.367+0000 7fb4ef8c3700 1 -- start start 2023-10-06T04:52:03.370 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ef8c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4e80cd820 0x7fb4e8156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:03.370 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ef8c3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e8156e60 con 0x7fb4e80c9410 2023-10-06T04:52:03.370 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ef8c3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e8157140 con 0x7fb4e80c97e0 2023-10-06T04:52:03.370 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ef8c3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e80b1840 con 0x7fb4e80cd820 2023-10-06T04:52:03.370 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ede60700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4e80cd820 0x7fb4e8156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:03.370 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ed65f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb4e80c9410 0x7fb4e80aff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38672/0 (socket says 172.21.15.3:38672) 2023-10-06T04:52:03.371 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ede60700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4e80cd820 0x7fb4e8156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44194/0 (socket says 172.21.15.3:44194) 2023-10-06T04:52:03.371 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ed65f700 1 -- 172.21.15.3:0/2338740871 learned_addr learned my addr 172.21.15.3:0/2338740871 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:03.371 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ede60700 1 -- 172.21.15.3:0/2338740871 >> v1:172.21.15.153:6789/0 conn(0x7fb4e80c97e0 legacy=0x7fb4e80cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:03.371 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ede60700 1 -- 172.21.15.3:0/2338740871 >> v1:172.21.15.3:6789/0 conn(0x7fb4e80c9410 legacy=0x7fb4e80aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:03.372 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4dffff700 1 -- 172.21.15.3:0/2338740871 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414479020 0 0) 0x7fb4e8157140 con 0x7fb4e80c97e0 2023-10-06T04:52:03.372 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.368+0000 7fb4ede60700 1 -- 172.21.15.3:0/2338740871 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e8157420 con 0x7fb4e80cd820 2023-10-06T04:52:03.372 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4ede60700 1 --2- 172.21.15.3:0/2338740871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80cd820 0x7fb4e8156840 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009e60 tx=0x7fb4e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3b482b3f13ed58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:03.372 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4dffff700 1 -- 172.21.15.3:0/2338740871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000e070 con 0x7fb4e80cd820 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4dffff700 1 -- 172.21.15.3:0/2338740871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000bb30 con 0x7fb4e80cd820 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4dffff700 1 -- 172.21.15.3:0/2338740871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0012430 con 0x7fb4e80cd820 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/2338740871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80cd820 msgr2=0x7fb4e8156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4ef8c3700 1 --2- 172.21.15.3:0/2338740871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80cd820 0x7fb4e8156840 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009e60 tx=0x7fb4e0009780 comp rx=0 tx=0).stop 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/2338740871 shutdown_connections 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4ef8c3700 1 --2- 172.21.15.3:0/2338740871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80cd820 0x7fb4e8156840 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.369+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/2338740871 >> 172.21.15.3:0/2338740871 conn(0x7fb4e8068520 msgr2=0x7fb4e8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.370+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/2338740871 shutdown_connections 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.370+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/2338740871 wait complete. 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.371+0000 7fb4ef8c3700 1 Processor -- start 2023-10-06T04:52:03.373 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.371+0000 7fb4ef8c3700 1 -- start start 2023-10-06T04:52:03.375 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e80c9410 0x7fb4e81522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80c97e0 0x7fb4e814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4e80cd820 0x7fb4e806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e8152810 con 0x7fb4e80c97e0 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e80cb310 con 0x7fb4e80c9410 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e80ca460 con 0x7fb4e80cd820 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80c97e0 0x7fb4e814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80c97e0 0x7fb4e814c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44210/0 (socket says 172.21.15.3:44210) 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 -- 172.21.15.3:0/1828447034 learned_addr learned my addr 172.21.15.3:0/1828447034 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ed65f700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e80c9410 0x7fb4e81522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ede60700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4e80cd820 0x7fb4e806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:03.376 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 -- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4e80cd820 msgr2=0x7fb4e806c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4e80cd820 0x7fb4e806c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 -- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e80c9410 msgr2=0x7fb4e81522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e80c9410 0x7fb4e81522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 -- 172.21.15.3:0/1828447034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009a20 con 0x7fb4e80c97e0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ece5e700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80c97e0 0x7fb4e814c0c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c7e0 tx=0x7fb4d8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4dd7fa700 1 -- 172.21.15.3:0/1828447034 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80105c0 con 0x7fb4e80c97e0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4dd7fa700 1 -- 172.21.15.3:0/1828447034 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d8010740 con 0x7fb4e80c97e0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e80ca700 con 0x7fb4e80c97e0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4dd7fa700 1 -- 172.21.15.3:0/1828447034 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8010b00 con 0x7fb4e80c97e0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.372+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e80cacc0 con 0x7fb4e80c97e0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.373+0000 7fb4dd7fa700 1 -- 172.21.15.3:0/1828447034 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb4d801c6c0 con 0x7fb4e80c97e0 2023-10-06T04:52:03.377 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.373+0000 7fb4dd7fa700 1 -- 172.21.15.3:0/1828447034 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fb4d8012b00 con 0x7fb4e80c97e0 2023-10-06T04:52:03.378 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.373+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e8071680 con 0x7fb4e80c97e0 2023-10-06T04:52:03.378 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.375+0000 7fb4dd7fa700 1 -- 172.21.15.3:0/1828447034 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb4d8017070 con 0x7fb4e80c97e0 2023-10-06T04:52:03.391 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:03.389+0000 7f5978c80180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:52:03.448 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.447+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4e80cb450 con 0x7fb4e80c97e0 2023-10-06T04:52:03.449 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.447+0000 7fb4dd7fa700 1 -- 172.21.15.3:0/1828447034 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fb4d801cba0 con 0x7fb4e80c97e0 2023-10-06T04:52:03.449 INFO:teuthology.misc.health.smithi003.stdout: 2023-10-06T04:52:03.449 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:02.170377+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:03.450 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80c97e0 msgr2=0x7fb4e814c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:03.450 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80c97e0 0x7fb4e814c0c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c7e0 tx=0x7fb4d8007df0 comp rx=0 tx=0).stop 2023-10-06T04:52:03.450 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 shutdown_connections 2023-10-06T04:52:03.450 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e80c9410 0x7fb4e81522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:03.451 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4e80cd820 0x7fb4e806c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:03.451 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 --2- 172.21.15.3:0/1828447034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e80c97e0 0x7fb4e814c0c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:03.451 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 >> 172.21.15.3:0/1828447034 conn(0x7fb4e8068520 msgr2=0x7fb4e81582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:03.451 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 shutdown_connections 2023-10-06T04:52:03.451 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:03.448+0000 7fb4ef8c3700 1 -- 172.21.15.3:0/1828447034 wait complete. 2023-10-06T04:52:03.458 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:03.457+0000 7f7d32bac180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:52:03.461 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T04:52:03.487 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:03.485+0000 7f5978c80180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:52:03.709 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:03.708+0000 7f7d32bac180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:52:03.746 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:03.744+0000 7f5978c80180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:52:03.786 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:03.785+0000 7f7d32bac180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:52:03.827 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:03.826+0000 7f5978c80180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:52:04.621 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:04.620+0000 7f7d32bac180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:52:04.691 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:04.690+0000 7f5978c80180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:52:04.695 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:04.694+0000 7f7d32bac180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:52:04.769 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:04.767+0000 7f5978c80180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:52:04.771 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:04.770+0000 7f7d32bac180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:52:04.846 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:04.845+0000 7f5978c80180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:52:04.905 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:04.904+0000 7f7d32bac180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:52:04.995 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:04.994+0000 7f5978c80180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:52:05.005 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:05.004+0000 7f7d32bac180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:52:05.084 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:05.082+0000 7f5978c80180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:52:05.158 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:05.157+0000 7f7d32bac180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:52:05.235 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:05.234+0000 7f7d32bac180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:52:05.246 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:05.245+0000 7f5978c80180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:52:05.319 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:05.317+0000 7f7d32bac180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:52:05.348 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:05.347+0000 7f5978c80180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:52:05.402 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:05.401+0000 7f7d32bac180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:52:05.429 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:05.427+0000 7f5978c80180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:52:05.510 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-10-06T04:52:05.509+0000 7f5978c80180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:52:06.020 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-06T04:52:06.018+0000 7f095f94f540 -1 osd.0 0 log_to_monitors true 2023-10-06T04:52:06.038 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-10-06T04:52:06.037+0000 7f351cadc540 -1 osd.5 0 log_to_monitors true 2023-10-06T04:52:06.265 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-06T04:52:06.263+0000 7f6486dc4540 -1 osd.1 0 log_to_monitors true 2023-10-06T04:52:06.306 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-10-06T04:52:06.304+0000 7fc170914540 -1 osd.6 0 log_to_monitors true 2023-10-06T04:52:06.322 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-06T04:52:06.321+0000 7f8354322540 -1 osd.2 0 log_to_monitors true 2023-10-06T04:52:07.163 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-06T04:52:06.321+0000 7fd35458c540 -1 osd.3 0 log_to_monitors true 2023-10-06T04:52:07.163 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-10-06T04:52:06.379+0000 7fc722e74540 -1 osd.7 0 log_to_monitors true 2023-10-06T04:52:07.163 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-10-06T04:52:06.394+0000 7f878a1ee540 -1 osd.4 0 log_to_monitors true 2023-10-06T04:52:07.164 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:06.427+0000 7f7cebfaf700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T04:52:07.164 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-10-06T04:52:06.427+0000 7f7cebfaf700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T04:52:09.463 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-06T04:52:09.462+0000 7f834af44700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T04:52:09.469 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-06T04:52:09.468+0000 7f8343d3b700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:09.546 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-06T04:52:09.544+0000 7f0956571700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T04:52:09.550 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-06T04:52:09.549+0000 7f094f368700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:09.584 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-10-06T04:52:09.583+0000 7f8780e10700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T04:52:09.585 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-10-06T04:52:09.583+0000 7fc719a96700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T04:52:09.585 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-06T04:52:09.584+0000 7f647f1e9700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T04:52:09.590 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-10-06T04:52:09.589+0000 7fc71288d700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:09.590 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-10-06T04:52:09.588+0000 7f8779c07700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:09.593 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-06T04:52:09.591+0000 7fd34c9b1700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T04:52:09.593 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-06T04:52:09.592+0000 7f64767dd700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:09.600 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-06T04:52:09.599+0000 7fd343fa5700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:09.614 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-10-06T04:52:09.613+0000 7f35136fe700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T04:52:09.620 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-10-06T04:52:09.619+0000 7f350c4f5700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:09.629 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-10-06T04:52:09.628+0000 7fc168d39700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T04:52:09.632 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-10-06T04:52:09.632+0000 7fc16032d700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:52:10.267 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:10.387 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.385+0000 7fb05d1fb700 1 Processor -- start 2023-10-06T04:52:10.387 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.385+0000 7fb05d1fb700 1 -- start start 2023-10-06T04:52:10.389 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05d1fb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb058159540 0x7fb05806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.389 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05d1fb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb0580af6c0 0x7fb0580a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.389 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05d1fb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0580b0840 con 0x7fb058159910 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05d1fb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0580705b0 con 0x7fb0580af6c0 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05d1fb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0580afc30 con 0x7fb058159540 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb0580af6c0 0x7fb0580a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb0580af6c0 0x7fb0580a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44258/0 (socket says 172.21.15.3:44258) 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb05659c700 1 -- 172.21.15.3:0/1811211672 learned_addr learned my addr 172.21.15.3:0/1811211672 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.386+0000 7fb056d9d700 1 --2- 172.21.15.3:0/1811211672 >> v2:172.21.15.153:3300/0 conn(0x7fb058159540 0x7fb05806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb05659c700 1 -- 172.21.15.3:0/1811211672 >> v2:172.21.15.153:3300/0 conn(0x7fb058159540 msgr2=0x7fb05806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb05659c700 1 --2- 172.21.15.3:0/1811211672 >> v2:172.21.15.153:3300/0 conn(0x7fb058159540 0x7fb05806ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb05659c700 1 -- 172.21.15.3:0/1811211672 >> v1:172.21.15.153:6789/0 conn(0x7fb058159910 legacy=0x7fb05806c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.390 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb05659c700 1 -- 172.21.15.3:0/1811211672 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0580aff10 con 0x7fb0580af6c0 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb056d9d700 1 --2- 172.21.15.3:0/1811211672 >> v2:172.21.15.153:3300/0 conn(0x7fb058159540 0x7fb05806ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb05659c700 1 --2- 172.21.15.3:0/1811211672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0580af6c0 0x7fb0580a9850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb05806e490 tx=0x7fb040009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9a422cb275a115 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb055d9b700 1 -- 172.21.15.3:0/1811211672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000b860 con 0x7fb0580af6c0 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb055d9b700 1 -- 172.21.15.3:0/1811211672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04000b9c0 con 0x7fb0580af6c0 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.387+0000 7fb055d9b700 1 -- 172.21.15.3:0/1811211672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000bd60 con 0x7fb0580af6c0 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 -- 172.21.15.3:0/1811211672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0580af6c0 msgr2=0x7fb0580a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/1811211672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0580af6c0 0x7fb0580a9850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb05806e490 tx=0x7fb040009250 comp rx=0 tx=0).stop 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 -- 172.21.15.3:0/1811211672 shutdown_connections 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/1811211672 >> v2:172.21.15.153:3300/0 conn(0x7fb058159540 0x7fb05806ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/1811211672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0580af6c0 0x7fb0580a9850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.391 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 -- 172.21.15.3:0/1811211672 >> 172.21.15.3:0/1811211672 conn(0x7fb058066fb0 msgr2=0x7fb0580673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:10.392 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 -- 172.21.15.3:0/1811211672 shutdown_connections 2023-10-06T04:52:10.392 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.388+0000 7fb05d1fb700 1 -- 172.21.15.3:0/1811211672 wait complete. 2023-10-06T04:52:10.394 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.389+0000 7fb05d1fb700 1 Processor -- start 2023-10-06T04:52:10.394 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.389+0000 7fb05d1fb700 1 -- start start 2023-10-06T04:52:10.394 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05d1fb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0580af6c0 0x7fb0580cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05d1fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb058159540 0x7fb0580c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05d1fb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb058159910 0x7fb05816d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05d1fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb058156a70 con 0x7fb058159540 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05d1fb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0580c8580 con 0x7fb058159910 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05d1fb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0580cc140 con 0x7fb0580af6c0 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb058159540 0x7fb0580c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb058159540 0x7fb0580c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44260/0 (socket says 172.21.15.3:44260) 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 -- 172.21.15.3:0/3020414991 learned_addr learned my addr 172.21.15.3:0/3020414991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb056d9d700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0580af6c0 0x7fb0580cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05759e700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb058159910 0x7fb05816d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.395 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 -- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0580af6c0 msgr2=0x7fb0580cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0580af6c0 0x7fb0580cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 -- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb058159910 msgr2=0x7fb05816d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb058159910 0x7fb05816d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 -- 172.21.15.3:0/3020414991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0400094f0 con 0x7fb058159540 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05659c700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb058159540 0x7fb0580c8040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb040009290 tx=0x7fb040017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb04effd700 1 -- 172.21.15.3:0/3020414991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040026480 con 0x7fb058159540 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb04effd700 1 -- 172.21.15.3:0/3020414991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04001f070 con 0x7fb058159540 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.390+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05816d9b0 con 0x7fb058159540 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.391+0000 7fb04effd700 1 -- 172.21.15.3:0/3020414991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040026850 con 0x7fb058159540 2023-10-06T04:52:10.396 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.391+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0580cb810 con 0x7fb058159540 2023-10-06T04:52:10.397 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.391+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb058071680 con 0x7fb058159540 2023-10-06T04:52:10.397 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.392+0000 7fb04effd700 1 -- 172.21.15.3:0/3020414991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fb04001a430 con 0x7fb058159540 2023-10-06T04:52:10.397 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.392+0000 7fb04effd700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb038066ed0 0x7fb038069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.397 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.392+0000 7fb04effd700 1 -- 172.21.15.3:0/3020414991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fb040021070 con 0x7fb058159540 2023-10-06T04:52:10.397 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.392+0000 7fb056d9d700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb038066ed0 0x7fb038069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.397 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.393+0000 7fb056d9d700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb038066ed0 0x7fb038069380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb048007740 tx=0x7fb0480099d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:10.398 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.397+0000 7fb04effd700 1 -- 172.21.15.3:0/3020414991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0400534a0 con 0x7fb058159540 2023-10-06T04:52:10.600 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.599+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0580cc280 con 0x7fb058159540 2023-10-06T04:52:10.601 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.599+0000 7fb04effd700 1 -- 172.21.15.3:0/3020414991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10348 (secure 0 0 0) 0x7fb040015090 con 0x7fb058159540 2023-10-06T04:52:10.601 INFO:teuthology.misc.health.smithi003.stdout: 2023-10-06T04:52:10.601 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":13,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:10.464542+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:10.603 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb038066ed0 msgr2=0x7fb038069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.603 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb038066ed0 0x7fb038069380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb048007740 tx=0x7fb0480099d0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.603 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb058159540 msgr2=0x7fb0580c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb058159540 0x7fb0580c8040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb040009290 tx=0x7fb040017000 comp rx=0 tx=0).stop 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 shutdown_connections 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb038066ed0 0x7fb038069380 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb058159910 0x7fb05816d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0580af6c0 0x7fb0580cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 --2- 172.21.15.3:0/3020414991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb058159540 0x7fb0580c8040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.601+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 >> 172.21.15.3:0/3020414991 conn(0x7fb058066fb0 msgr2=0x7fb0580c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.602+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 shutdown_connections 2023-10-06T04:52:10.604 INFO:teuthology.misc.health.smithi003.stderr:2023-10-06T04:52:10.602+0000 7fb05d1fb700 1 -- 172.21.15.3:0/3020414991 wait complete. 2023-10-06T04:52:10.614 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T04:52:10.614 INFO:tasks.ceph:Creating RBD pool 2023-10-06T04:52:10.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T04:52:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.731+0000 7fcf4efbd700 1 Processor -- start 2023-10-06T04:52:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.731+0000 7fcf4efbd700 1 -- start start 2023-10-06T04:52:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.731+0000 7fcf4efbd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcf481561c0 0x7fcf4806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4efbd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf4806af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4efbd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf480afed0 con 0x7fcf48156590 2023-10-06T04:52:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4efbd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf480b0770 con 0x7fcf481561c0 2023-10-06T04:52:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4efbd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480c9d60 con 0x7fcf480680d0 2023-10-06T04:52:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4d55a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf4806af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4cd59700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcf481561c0 0x7fcf4806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4d55a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf4806af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:44902/0 (socket says 172.21.15.3:44902) 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4cd59700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcf481561c0 0x7fcf4806aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:56876/0 (socket says 172.21.15.3:56876) 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4d55a700 1 -- 172.21.15.3:0/4159345805 learned_addr learned my addr 172.21.15.3:0/4159345805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4cd59700 1 -- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 msgr2=0x7fcf4806af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4cd59700 1 --2- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf4806af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4cd59700 1 -- 172.21.15.3:0/4159345805 >> v1:172.21.15.153:6789/0 conn(0x7fcf48156590 legacy=0x7fcf480679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4cd59700 1 -- 172.21.15.3:0/4159345805 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf480686b0 con 0x7fcf481561c0 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4d55a700 1 --2- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf4806af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf477fe700 1 -- 172.21.15.3:0/4159345805 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149811484 0 0) 0x7fcf480afed0 con 0x7fcf48156590 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.732+0000 7fcf4cd59700 1 --2- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 0x7fcf4806aa20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000d0d0 tx=0x7fcf4000b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a5f706216195117 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf477fe700 1 -- 172.21.15.3:0/4159345805 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40010070 con 0x7fcf481561c0 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf477fe700 1 -- 172.21.15.3:0/4159345805 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4000b2e0 con 0x7fcf481561c0 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf477fe700 1 -- 172.21.15.3:0/4159345805 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000dd80 con 0x7fcf481561c0 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 -- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 msgr2=0x7fcf4806aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 0x7fcf4806aa20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000d0d0 tx=0x7fcf4000b990 comp rx=0 tx=0).stop 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 -- 172.21.15.3:0/4159345805 shutdown_connections 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf4806af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/4159345805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 0x7fcf4806aa20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 -- 172.21.15.3:0/4159345805 >> 172.21.15.3:0/4159345805 conn(0x7fcf4806df50 msgr2=0x7fcf480aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 -- 172.21.15.3:0/4159345805 shutdown_connections 2023-10-06T04:52:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.733+0000 7fcf4efbd700 1 -- 172.21.15.3:0/4159345805 wait complete. 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.734+0000 7fcf4efbd700 1 Processor -- start 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.734+0000 7fcf4efbd700 1 -- start start 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4efbd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf480c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4efbd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 0x7fcf48144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4efbd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf48156590 0x7fcf480ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4efbd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf481597f0 con 0x7fcf48156590 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4efbd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480c6ea0 con 0x7fcf481561c0 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4efbd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf48158be0 con 0x7fcf480680d0 2023-10-06T04:52:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4d55a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf48156590 0x7fcf480ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4d55a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf48156590 0x7fcf480ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44272/0 (socket says 172.21.15.3:44272) 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4d55a700 1 -- 172.21.15.3:0/2282221648 learned_addr learned my addr 172.21.15.3:0/2282221648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf4cd59700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf480c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.735+0000 7fcf47fff700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 0x7fcf48144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4d55a700 1 -- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 msgr2=0x7fcf480c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4d55a700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf480c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4d55a700 1 -- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 msgr2=0x7fcf48144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4d55a700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 0x7fcf48144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4d55a700 1 -- 172.21.15.3:0/2282221648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf380094f0 con 0x7fcf48156590 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4cd59700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf480c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4d55a700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf48156590 0x7fcf480ccf90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ec50 tx=0x7fcf38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf44ff9700 1 -- 172.21.15.3:0/2282221648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800c850 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf44ff9700 1 -- 172.21.15.3:0/2282221648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3800c9b0 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4000bc30 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf44ff9700 1 -- 172.21.15.3:0/2282221648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800cd70 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.736+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf481591a0 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.737+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf481599a0 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.737+0000 7fcf44ff9700 1 -- 172.21.15.3:0/2282221648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fcf38013680 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.738+0000 7fcf44ff9700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf28066ea0 0x7fcf28069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.738+0000 7fcf44ff9700 1 -- 172.21.15.3:0/2282221648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fcf38084190 con 0x7fcf48156590 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.738+0000 7fcf4cd59700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf28066ea0 0x7fcf28069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.738+0000 7fcf4cd59700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf28066ea0 0x7fcf28069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000b9d0 tx=0x7fcf4001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.742+0000 7fcf44ff9700 1 -- 172.21.15.3:0/2282221648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf481599a0 con 0x7fcf48156590 2023-10-06T04:52:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:10.933+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fcf480b01b0 con 0x7fcf48156590 2023-10-06T04:52:11.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.520+0000 7fcf44ff9700 1 -- 172.21.15.3:0/2282221648 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fcf38051910 con 0x7fcf48156590 2023-10-06T04:52:11.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.522+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf28066ea0 msgr2=0x7fcf28069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:11.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.522+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf28066ea0 0x7fcf28069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000b9d0 tx=0x7fcf4001a000 comp rx=0 tx=0).stop 2023-10-06T04:52:11.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf48156590 msgr2=0x7fcf480ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf48156590 0x7fcf480ccf90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ec50 tx=0x7fcf38009250 comp rx=0 tx=0).stop 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 shutdown_connections 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf28066ea0 0x7fcf28069350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf48156590 0x7fcf480ccf90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf480680d0 0x7fcf480c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 --2- 172.21.15.3:0/2282221648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf481561c0 0x7fcf48144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 >> 172.21.15.3:0/2282221648 conn(0x7fcf4806df50 msgr2=0x7fcf481519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 shutdown_connections 2023-10-06T04:52:11.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.523+0000 7fcf4efbd700 1 -- 172.21.15.3:0/2282221648 wait complete. 2023-10-06T04:52:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'rbd' created 2023-10-06T04:52:11.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T04:52:11.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.659+0000 7f202f3b5700 1 Processor -- start 2023-10-06T04:52:11.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.659+0000 7f202f3b5700 1 -- start start 2023-10-06T04:52:11.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202f3b5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f202809d670 0x7f202809da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202f3b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 0x7f20280ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202f3b5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2028159b70 con 0x7f20280cf6a0 2023-10-06T04:52:11.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202f3b5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280cfb10 con 0x7f202809d670 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202f3b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202815e1a0 con 0x7f202815dd40 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202dbb2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20280cf6a0 0x7f20280b4a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38684/0 (socket says 172.21.15.3:38684) 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202dbb2700 1 -- 172.21.15.3:0/67971920 learned_addr learned my addr 172.21.15.3:0/67971920 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953375474 0 0) 0x7f2028159b70 con 0x7f20280cf6a0 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2010003570 con 0x7f20280cf6a0 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202e3b3700 1 --2- 172.21.15.3:0/67971920 >> v2:172.21.15.153:3300/0 conn(0x7f202809d670 0x7f202809da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.660+0000 7f202ebb4700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 0x7f20280ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 170546977 0 0) 0x7f2010003570 con 0x7f20280cf6a0 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 msgr2=0x7f20280ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 0x7f20280ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 >> v2:172.21.15.153:3300/0 conn(0x7f202809d670 msgr2=0x7f202809da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 --2- 172.21.15.3:0/67971920 >> v2:172.21.15.153:3300/0 conn(0x7f202809d670 0x7f202809da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202815d8b0 con 0x7f20280cf6a0 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f201c001470 con 0x7f20280cf6a0 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 >> v1:172.21.15.3:6789/0 conn(0x7f20280cf6a0 legacy=0x7f20280b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 0x7f2010003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20100040e0 0x7f2010004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2010004d50 0x7f20100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280cfb10 con 0x7f20100040e0 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201000bb10 con 0x7f2010004d50 2023-10-06T04:52:11.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201000bcd0 con 0x7f202815dd40 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202dbb2700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20100040e0 0x7f2010004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202ebb4700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2010004d50 0x7f20100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202e3b3700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 0x7f2010003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202dbb2700 1 -- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 msgr2=0x7f2010003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202dbb2700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 0x7f2010003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202dbb2700 1 -- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2010004d50 msgr2=0x7f20100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202dbb2700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2010004d50 0x7f20100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202dbb2700 1 -- 172.21.15.3:0/67971920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010003570 con 0x7f20100040e0 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202e3b3700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202815dd40 0x7f2010003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:11.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202dbb2700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20100040e0 0x7f2010004810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c2a0 tx=0x7f201c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f7c3bbaec82ee4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00cb70 con 0x7f20100040e0 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.661+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c00ccd0 con 0x7f20100040e0 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202d3b1700 1 -- 172.21.15.3:0/67971920 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c005b70 con 0x7f20100040e0 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202f3b5700 1 -- 172.21.15.3:0/67971920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20100040e0 msgr2=0x7f2010004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202f3b5700 1 --2- 172.21.15.3:0/67971920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20100040e0 0x7f2010004810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c2a0 tx=0x7f201c001d80 comp rx=0 tx=0).stop 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202ebb4700 1 -- 172.21.15.3:0/67971920 reap_dead start 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202f3b5700 1 -- 172.21.15.3:0/67971920 shutdown_connections 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202f3b5700 1 -- 172.21.15.3:0/67971920 >> 172.21.15.3:0/67971920 conn(0x7f2028070310 msgr2=0x7f2028070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202f3b5700 1 -- 172.21.15.3:0/67971920 shutdown_connections 2023-10-06T04:52:11.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.662+0000 7f202f3b5700 1 -- 172.21.15.3:0/67971920 wait complete. 2023-10-06T04:52:11.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.663+0000 7f202f3b5700 1 Processor -- start 2023-10-06T04:52:11.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.663+0000 7f202f3b5700 1 -- start start 2023-10-06T04:52:11.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202f3b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20100040e0 0x7f20281562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202f3b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202815cbe0 0x7f2028154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202f3b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2028154790 0x7f202815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202f3b5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2028154c10 con 0x7f202815cbe0 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202f3b5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2028154fd0 con 0x7f20100040e0 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202f3b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2028155e30 con 0x7f2028154790 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202815cbe0 0x7f2028154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202815cbe0 0x7f2028154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44288/0 (socket says 172.21.15.3:44288) 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 -- 172.21.15.3:0/1527649267 learned_addr learned my addr 172.21.15.3:0/1527649267 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202e3b3700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20100040e0 0x7f20281562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202ebb4700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2028154790 0x7f202815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 -- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2028154790 msgr2=0x7f202815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:11.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2028154790 0x7f202815a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 -- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20100040e0 msgr2=0x7f20281562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20100040e0 0x7f20281562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 -- 172.21.15.3:0/1527649267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028155220 con 0x7f202815cbe0 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202ebb4700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2028154790 0x7f202815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.664+0000 7f202dbb2700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202815cbe0 0x7f2028154250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2010004790 tx=0x7f201c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.665+0000 7f20267fc700 1 -- 172.21.15.3:0/1527649267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00cb70 con 0x7f202815cbe0 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.665+0000 7f20267fc700 1 -- 172.21.15.3:0/1527649267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c001d80 con 0x7f202815cbe0 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.665+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2028155500 con 0x7f202815cbe0 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.665+0000 7f20267fc700 1 -- 172.21.15.3:0/1527649267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c007510 con 0x7f202815cbe0 2023-10-06T04:52:11.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.665+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2028155ac0 con 0x7f202815cbe0 2023-10-06T04:52:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.666+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2028155fa0 con 0x7f202815cbe0 2023-10-06T04:52:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.666+0000 7f20267fc700 1 -- 172.21.15.3:0/1527649267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f201c019070 con 0x7f202815cbe0 2023-10-06T04:52:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.667+0000 7f20267fc700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2014066ea0 0x7f2014069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.667+0000 7f20267fc700 1 -- 172.21.15.3:0/1527649267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5184+0+0 (secure 0 0 0) 0x7f201c0852d0 con 0x7f202815cbe0 2023-10-06T04:52:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.667+0000 7f202e3b3700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2014066ea0 0x7f2014069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.667+0000 7f202e3b3700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2014066ea0 0x7f2014069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2028157330 tx=0x7f202000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.671+0000 7f20267fc700 1 -- 172.21.15.3:0/1527649267 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f201c052830 con 0x7f202815cbe0 2023-10-06T04:52:11.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:11.866+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f202804c380 con 0x7f202815cbe0 2023-10-06T04:52:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.519+0000 7f20267fc700 1 -- 172.21.15.3:0/1527649267 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f201c007e60 con 0x7f202815cbe0 2023-10-06T04:52:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2014066ea0 msgr2=0x7f2014069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2014066ea0 0x7f2014069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2028157330 tx=0x7f202000b040 comp rx=0 tx=0).stop 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202815cbe0 msgr2=0x7f2028154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202815cbe0 0x7f2028154250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2010004790 tx=0x7f201c015000 comp rx=0 tx=0).stop 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 shutdown_connections 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2014066ea0 0x7f2014069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20100040e0 0x7f20281562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2028154790 0x7f202815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 --2- 172.21.15.3:0/1527649267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202815cbe0 0x7f2028154250 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 >> 172.21.15.3:0/1527649267 conn(0x7f2028070310 msgr2=0x7f20280af170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 shutdown_connections 2023-10-06T04:52:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.522+0000 7f202f3b5700 1 -- 172.21.15.3:0/1527649267 wait complete. 2023-10-06T04:52:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T04:52:12.539 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T04:52:12.540 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:52:12.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:52:12.542 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:52:12.542 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:52:12.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:52:12.544 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:52:12.544 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:52:12.545 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:52:12.547 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:52:12.547 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:52:12.547 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:52:12.549 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:52:12.549 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T04:52:12.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:12.630 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:52:12.630 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:52:12.633 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:52:12.633 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:52:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.684+0000 7f7953f14700 1 Processor -- start 2023-10-06T04:52:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.684+0000 7f7953f14700 1 -- start start 2023-10-06T04:52:12.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f7953f14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:12.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f7953f14700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f794c0b0900 0x7f794c0c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:12.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f7953f14700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c0b1c90 con 0x7f794c156410 2023-10-06T04:52:12.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f7953f14700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c0b1080 con 0x7f794c0b0900 2023-10-06T04:52:12.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f7953f14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0b1360 con 0x7f794c06b7d0 2023-10-06T04:52:12.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f79514af700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f79524b1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f794c0b0900 0x7f794c0c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f79514af700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c0af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:37592/0 (socket says 172.21.15.3:37592) 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f79514af700 1 -- 172.21.15.3:0/2496819334 learned_addr learned my addr 172.21.15.3:0/2496819334 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.685+0000 7f7951cb0700 1 --1- 172.21.15.3:0/2496819334 >> v1:172.21.15.153:6789/0 conn(0x7f794c156410 0x7f794c06b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53464/0 (socket says 172.21.15.3:53464) 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f79524b1700 1 -- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 msgr2=0x7f794c0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f79524b1700 1 --2- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f7950cae700 1 -- 172.21.15.3:0/2496819334 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795794805 0 0) 0x7f794c0b1c90 con 0x7f794c156410 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f79524b1700 1 -- 172.21.15.3:0/2496819334 >> v1:172.21.15.153:6789/0 conn(0x7f794c156410 legacy=0x7f794c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f79524b1700 1 -- 172.21.15.3:0/2496819334 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0b1640 con 0x7f794c0b0900 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f79514af700 1 --2- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f79524b1700 1 --2- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0c9ff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7944009e60 tx=0x7f7944009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6477125bf90f5be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f7950cae700 1 -- 172.21.15.3:0/2496819334 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794400e070 con 0x7f794c0b0900 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f7950cae700 1 -- 172.21.15.3:0/2496819334 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794400bb60 con 0x7f794c0b0900 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.686+0000 7f7950cae700 1 -- 172.21.15.3:0/2496819334 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944012430 con 0x7f794c0b0900 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 -- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 msgr2=0x7f794c0c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 --2- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0c9ff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7944009e60 tx=0x7f7944009780 comp rx=0 tx=0).stop 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 -- 172.21.15.3:0/2496819334 shutdown_connections 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 --2- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0c9ff0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 --2- 172.21.15.3:0/2496819334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 -- 172.21.15.3:0/2496819334 >> 172.21.15.3:0/2496819334 conn(0x7f794c0cdc30 msgr2=0x7f794c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 -- 172.21.15.3:0/2496819334 shutdown_connections 2023-10-06T04:52:12.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.687+0000 7f7953f14700 1 -- 172.21.15.3:0/2496819334 wait complete. 2023-10-06T04:52:12.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.688+0000 7f7953f14700 1 Processor -- start 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.688+0000 7f7953f14700 1 -- start start 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.688+0000 7f7953f14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c167950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.688+0000 7f7953f14700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7953f14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794c156410 0x7f794c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7953f14700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c16a830 con 0x7f794c156410 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7953f14700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c16aa80 con 0x7f794c0b0900 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7953f14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c16ad60 con 0x7f794c06b7d0 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794c156410 0x7f794c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794c156410 0x7f794c0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51160/0 (socket says 172.21.15.3:51160) 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 -- 172.21.15.3:0/1661733719 learned_addr learned my addr 172.21.15.3:0/1661733719 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:12.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7951cb0700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79514af700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 -- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 msgr2=0x7f794c167950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c167950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 -- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 msgr2=0x7f794c0cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 -- 172.21.15.3:0/1661733719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c0094f0 con 0x7f794c156410 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79514af700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f79524b1700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794c156410 0x7f794c0cc630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f794400b6c0 tx=0x7f7944019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7941ffb700 1 -- 172.21.15.3:0/1661733719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944027470 con 0x7f794c156410 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7941ffb700 1 -- 172.21.15.3:0/1661733719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79440275d0 con 0x7f794c156410 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7944009a20 con 0x7f794c156410 2023-10-06T04:52:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.689+0000 7f7941ffb700 1 -- 172.21.15.3:0/1661733719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944027990 con 0x7f794c156410 2023-10-06T04:52:12.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.690+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c16b320 con 0x7f794c156410 2023-10-06T04:52:12.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.691+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794c071680 con 0x7f794c156410 2023-10-06T04:52:12.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.691+0000 7f7941ffb700 1 -- 172.21.15.3:0/1661733719 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f7944024070 con 0x7f794c156410 2023-10-06T04:52:12.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.691+0000 7f7941ffb700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7930066ea0 0x7f7930069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:12.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.692+0000 7f7941ffb700 1 -- 172.21.15.3:0/1661733719 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7944086c80 con 0x7f794c156410 2023-10-06T04:52:12.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.692+0000 7f7951cb0700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7930066ea0 0x7f7930069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:12.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.692+0000 7f7951cb0700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7930066ea0 0x7f7930069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7948007fd0 tx=0x7f7948007e10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:12.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.696+0000 7f7941ffb700 1 -- 172.21.15.3:0/1661733719 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79440541c0 con 0x7f794c156410 2023-10-06T04:52:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.899+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f794c06bfa0 con 0x7f794c156410 2023-10-06T04:52:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.900+0000 7f7941ffb700 1 -- 172.21.15.3:0/1661733719 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13626 (secure 0 0 0) 0x7f79440577e0 con 0x7f794c156410 2023-10-06T04:52:12.902 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:12.902 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:12.516820+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:12.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.902+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7930066ea0 msgr2=0x7f7930069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.902+0000 7f7953f14700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7930066ea0 0x7f7930069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7948007fd0 tx=0x7f7948007e10 comp rx=0 tx=0).stop 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.902+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794c156410 msgr2=0x7f794c0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.902+0000 7f7953f14700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794c156410 0x7f794c0cc630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f794400b6c0 tx=0x7f7944019000 comp rx=0 tx=0).stop 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 shutdown_connections 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7930066ea0 0x7f7930069350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794c156410 0x7f794c0cc630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f794c06b7d0 0x7f794c167950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 --2- 172.21.15.3:0/1661733719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794c0b0900 0x7f794c0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 >> 172.21.15.3:0/1661733719 conn(0x7f794c0cdc30 msgr2=0x7f794c066fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 shutdown_connections 2023-10-06T04:52:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:12.903+0000 7f7953f14700 1 -- 172.21.15.3:0/1661733719 wait complete. 2023-10-06T04:52:12.917 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:52:12.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:13.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.041+0000 7f391c0e4700 1 Processor -- start 2023-10-06T04:52:13.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.041+0000 7f391c0e4700 1 -- start start 2023-10-06T04:52:13.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.042+0000 7f391c0e4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 0x7f39140679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.042+0000 7f391c0e4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f391406b130 0x7f3914067f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.042+0000 7f391c0e4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140afed0 con 0x7f39141561c0 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.042+0000 7f391c0e4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39140b0770 con 0x7f391406b130 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.042+0000 7f391c0e4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3914068440 con 0x7f3914156590 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.042+0000 7f3919e80700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f39141561c0 0x7f391406aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44582/0 (socket says 172.21.15.3:44582) 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.042+0000 7f3919e80700 1 -- 172.21.15.3:0/3031738448 learned_addr learned my addr 172.21.15.3:0/3031738448 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391967f700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 0x7f39140679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391a681700 1 --2- 172.21.15.3:0/3031738448 >> v2:172.21.15.153:3300/0 conn(0x7f391406b130 0x7f3914067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299611116 0 0) 0x7f39140afed0 con 0x7f39141561c0 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3900003570 con 0x7f39141561c0 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1329980254 0 0) 0x7f3900003570 con 0x7f39141561c0 2023-10-06T04:52:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 msgr2=0x7f39140679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 0x7f39140679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 >> v2:172.21.15.153:3300/0 conn(0x7f391406b130 msgr2=0x7f3914067f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 --2- 172.21.15.3:0/3031738448 >> v2:172.21.15.153:3300/0 conn(0x7f391406b130 0x7f3914067f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140c9d60 con 0x7f39141561c0 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f3904001470 con 0x7f39141561c0 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 >> v1:172.21.15.3:6789/0 conn(0x7f39141561c0 legacy=0x7f391406aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3900003a60 0x7f3900003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 0x7f3900004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3900004d50 0x7f39000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140b0770 con 0x7f3900004d50 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914068440 con 0x7f3900003a60 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390000ba60 con 0x7f3914156590 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391a681700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3900004d50 0x7f39000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f3919e80700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3900003a60 0x7f3900003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391967f700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 0x7f3900004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391a681700 1 -- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 msgr2=0x7f3900004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391a681700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3914156590 0x7f3900004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391a681700 1 -- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3900003a60 msgr2=0x7f3900003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391a681700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3900003a60 0x7f3900003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.043+0000 7f391a681700 1 -- 172.21.15.3:0/3031738448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900003570 con 0x7f3900004d50 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391a681700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3900004d50 0x7f39000051c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f390c00e210 tx=0x7f390c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bd9590e058d61b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00ea60 con 0x7f3900004d50 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c00f040 con 0x7f3900004d50 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f3918e7e700 1 -- 172.21.15.3:0/3031738448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00ee10 con 0x7f3900004d50 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391c0e4700 1 -- 172.21.15.3:0/3031738448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3900004d50 msgr2=0x7f39000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391c0e4700 1 --2- 172.21.15.3:0/3031738448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3900004d50 0x7f39000051c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f390c00e210 tx=0x7f390c00b040 comp rx=0 tx=0).stop 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391a681700 1 -- 172.21.15.3:0/3031738448 reap_dead start 2023-10-06T04:52:13.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391c0e4700 1 -- 172.21.15.3:0/3031738448 shutdown_connections 2023-10-06T04:52:13.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391c0e4700 1 -- 172.21.15.3:0/3031738448 >> 172.21.15.3:0/3031738448 conn(0x7f391406df50 msgr2=0x7f39140aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:13.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391c0e4700 1 -- 172.21.15.3:0/3031738448 shutdown_connections 2023-10-06T04:52:13.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.044+0000 7f391c0e4700 1 -- 172.21.15.3:0/3031738448 wait complete. 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 Processor -- start 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 -- start start 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3900004d50 0x7f391414c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39141699e0 0x7f3914169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914150850 0x7f3914150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140cb4d0 con 0x7f3914150850 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140cba90 con 0x7f39141699e0 2023-10-06T04:52:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391c0e4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39141511e0 con 0x7f3900004d50 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391a681700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914150850 0x7f3914150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391a681700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914150850 0x7f3914150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51170/0 (socket says 172.21.15.3:51170) 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f391a681700 1 -- 172.21.15.3:0/2793660076 learned_addr learned my addr 172.21.15.3:0/2793660076 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.046+0000 7f3919e80700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3900004d50 0x7f391414c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391967f700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39141699e0 0x7f3914169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391a681700 1 -- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3900004d50 msgr2=0x7f391414c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391a681700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3900004d50 0x7f391414c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391a681700 1 -- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39141699e0 msgr2=0x7f3914169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391a681700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39141699e0 0x7f3914169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391a681700 1 -- 172.21.15.3:0/2793660076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391416a4a0 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391a681700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914150850 0x7f3914150ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f390c016000 tx=0x7f390c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f3909ffb700 1 -- 172.21.15.3:0/2793660076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00eee0 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f3909ffb700 1 -- 172.21.15.3:0/2793660076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c01d630 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f3909ffb700 1 -- 172.21.15.3:0/2793660076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c01d9f0 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391416a780 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.047+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39140c9670 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.049+0000 7f3909ffb700 1 -- 172.21.15.3:0/2793660076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f390c003040 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.049+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391404c380 con 0x7f3914150850 2023-10-06T04:52:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.049+0000 7f3909ffb700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f38f8066ea0 0x7f38f8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.049+0000 7f3909ffb700 1 -- 172.21.15.3:0/2793660076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f390c0854c0 con 0x7f3914150850 2023-10-06T04:52:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.049+0000 7f3919e80700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f38f8066ea0 0x7f38f8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.050+0000 7f3919e80700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f38f8066ea0 0x7f38f8069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39040079e0 tx=0x7f39140b06e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.054+0000 7f3909ffb700 1 -- 172.21.15.3:0/2793660076 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f390c052ac0 con 0x7f3914150850 2023-10-06T04:52:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.253+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f39140686b0 con 0x7f3914150850 2023-10-06T04:52:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.254+0000 7f3909ffb700 1 -- 172.21.15.3:0/2793660076 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f390c0560e0 con 0x7f3914150850 2023-10-06T04:52:13.255 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f38f8066ea0 msgr2=0x7f38f8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f38f8066ea0 0x7f38f8069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39040079e0 tx=0x7f39140b06e0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914150850 msgr2=0x7f3914150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914150850 0x7f3914150ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f390c016000 tx=0x7f390c01a000 comp rx=0 tx=0).stop 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 shutdown_connections 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f38f8066ea0 0x7f38f8069350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914150850 0x7f3914150ca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3900004d50 0x7f391414c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 --2- 172.21.15.3:0/2793660076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39141699e0 0x7f3914169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 >> 172.21.15.3:0/2793660076 conn(0x7f391406df50 msgr2=0x7f391414dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:13.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 shutdown_connections 2023-10-06T04:52:13.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.256+0000 7f391c0e4700 1 -- 172.21.15.3:0/2793660076 wait complete. 2023-10-06T04:52:13.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:13.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.390+0000 7f5a0e90e700 1 Processor -- start 2023-10-06T04:52:13.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.390+0000 7f5a0e90e700 1 -- start start 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.391+0000 7f5a0e90e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a0815c760 0x7f5a0815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.391+0000 7f5a0e90e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a080cb810 con 0x7f5a080cb3c0 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.391+0000 7f5a0e90e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a080cf0f0 con 0x7f5a080afec0 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.391+0000 7f5a0e90e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080ce7c0 con 0x7f5a0815c760 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.391+0000 7f5a07fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a0815c760 0x7f5a0815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.391+0000 7f5a077fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5a080cb3c0 0x7f5a0815db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44598/0 (socket says 172.21.15.3:44598) 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.391+0000 7f5a077fe700 1 -- 172.21.15.3:0/3795768556 learned_addr learned my addr 172.21.15.3:0/3795768556 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a07fff700 1 -- 172.21.15.3:0/3795768556 >> v1:172.21.15.153:6789/0 conn(0x7f5a080afec0 legacy=0x7f5a080cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a07fff700 1 -- 172.21.15.3:0/3795768556 >> v1:172.21.15.3:6789/0 conn(0x7f5a080cb3c0 legacy=0x7f5a0815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a07fff700 1 -- 172.21.15.3:0/3795768556 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080ceaa0 con 0x7f5a0815c760 2023-10-06T04:52:13.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a06ffd700 1 -- 172.21.15.3:0/3795768556 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074727841 0 0) 0x7f59f8001110 con 0x7f5a080cb3c0 2023-10-06T04:52:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a07fff700 1 --2- 172.21.15.3:0/3795768556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0815c760 0x7f5a0815cb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59f00098e0 tx=0x7f59f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14afa703dd70b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a06ffd700 1 -- 172.21.15.3:0/3795768556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0009e80 con 0x7f5a0815c760 2023-10-06T04:52:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a06ffd700 1 -- 172.21.15.3:0/3795768556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f000b860 con 0x7f5a0815c760 2023-10-06T04:52:13.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a06ffd700 1 -- 172.21.15.3:0/3795768556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000bbe0 con 0x7f5a0815c760 2023-10-06T04:52:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a0e90e700 1 -- 172.21.15.3:0/3795768556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0815c760 msgr2=0x7f5a0815cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/3795768556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0815c760 0x7f5a0815cb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59f00098e0 tx=0x7f59f00091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a0e90e700 1 -- 172.21.15.3:0/3795768556 shutdown_connections 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/3795768556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0815c760 0x7f5a0815cb30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a0e90e700 1 -- 172.21.15.3:0/3795768556 >> 172.21.15.3:0/3795768556 conn(0x7f5a0806f940 msgr2=0x7f5a08070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a0e90e700 1 -- 172.21.15.3:0/3795768556 shutdown_connections 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.392+0000 7f5a0e90e700 1 -- 172.21.15.3:0/3795768556 wait complete. 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 Processor -- start 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 -- start start 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a080afec0 0x7f5a080ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a080cb3c0 0x7f5a081694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a0815c760 0x7f5a080b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080b31f0 con 0x7f5a080cb3c0 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080b3d70 con 0x7f5a0815c760 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.394+0000 7f5a0e90e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080b3440 con 0x7f5a080afec0 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a080cb3c0 0x7f5a081694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a080cb3c0 0x7f5a081694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51194/0 (socket says 172.21.15.3:51194) 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 -- 172.21.15.3:0/1432140016 learned_addr learned my addr 172.21.15.3:0/1432140016 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a07fff700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a080afec0 0x7f5a080ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a0ceab700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a0815c760 0x7f5a080b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 -- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a080afec0 msgr2=0x7f5a080ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a080afec0 0x7f5a080ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 -- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a0815c760 msgr2=0x7f5a080b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a0815c760 0x7f5a080b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 -- 172.21.15.3:0/1432140016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0009420 con 0x7f5a080cb3c0 2023-10-06T04:52:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a077fe700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a080cb3c0 0x7f5a081694f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59f800c840 tx=0x7f59f8007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f59fffff700 1 -- 172.21.15.3:0/1432140016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8005b30 con 0x7f5a080cb3c0 2023-10-06T04:52:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f59fffff700 1 -- 172.21.15.3:0/1432140016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8005cd0 con 0x7f5a080cb3c0 2023-10-06T04:52:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a080b3720 con 0x7f5a080cb3c0 2023-10-06T04:52:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f59fffff700 1 -- 172.21.15.3:0/1432140016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8007320 con 0x7f5a080cb3c0 2023-10-06T04:52:13.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.395+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a080aee00 con 0x7f5a080cb3c0 2023-10-06T04:52:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.396+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0804c380 con 0x7f5a080cb3c0 2023-10-06T04:52:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.397+0000 7f59fffff700 1 -- 172.21.15.3:0/1432140016 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f59f8012400 con 0x7f5a080cb3c0 2023-10-06T04:52:13.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.397+0000 7f59fffff700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066ea0 0x7f59e8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.397+0000 7f59fffff700 1 -- 172.21.15.3:0/1432140016 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f59f8083330 con 0x7f5a080cb3c0 2023-10-06T04:52:13.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.397+0000 7f5a07fff700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066ea0 0x7f59e8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.398+0000 7f5a07fff700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066ea0 0x7f59e8069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59f00093f0 tx=0x7f59f0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.401+0000 7f59fffff700 1 -- 172.21.15.3:0/1432140016 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59f804b5a0 con 0x7f5a080cb3c0 2023-10-06T04:52:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.601+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a080cb810 con 0x7f5a080cb3c0 2023-10-06T04:52:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.601+0000 7f59fffff700 1 -- 172.21.15.3:0/1432140016 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a080cb810 con 0x7f5a080cb3c0 2023-10-06T04:52:13.603 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066ea0 msgr2=0x7f59e8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066ea0 0x7f59e8069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59f00093f0 tx=0x7f59f0017000 comp rx=0 tx=0).stop 2023-10-06T04:52:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a080cb3c0 msgr2=0x7f5a081694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a080cb3c0 0x7f5a081694f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59f800c840 tx=0x7f59f8007c10 comp rx=0 tx=0).stop 2023-10-06T04:52:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 shutdown_connections 2023-10-06T04:52:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066ea0 0x7f59e8069350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a0815c760 0x7f5a080b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a080afec0 0x7f5a080ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 --2- 172.21.15.3:0/1432140016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a080cb3c0 0x7f5a081694f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 >> 172.21.15.3:0/1432140016 conn(0x7f5a0806f940 msgr2=0x7f5a080cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:13.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 shutdown_connections 2023-10-06T04:52:13.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.603+0000 7f5a0e90e700 1 -- 172.21.15.3:0/1432140016 wait complete. 2023-10-06T04:52:13.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:13.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.737+0000 7fed55c05700 1 Processor -- start 2023-10-06T04:52:13.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.737+0000 7fed55c05700 1 -- start start 2023-10-06T04:52:13.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed55c05700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fed5015c8f0 0x7fed5015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed55c05700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015ccc0 0x7fed500d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed55c05700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed500cf6c0 con 0x7fed500d06a0 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed55c05700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500cfb50 con 0x7fed5015c8f0 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed55c05700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5015e050 con 0x7fed5015ccc0 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed4f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fed5015c8f0 0x7fed5015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed4effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015ccc0 0x7fed500d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed4ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fed500d06a0 0x7fed5006d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53480/0 (socket says 172.21.15.3:53480) 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.738+0000 7fed4ffff700 1 -- 172.21.15.3:0/1955216064 learned_addr learned my addr 172.21.15.3:0/1955216064 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4effd700 1 -- 172.21.15.3:0/1955216064 >> v2:172.21.15.153:3300/0 conn(0x7fed5015c8f0 msgr2=0x7fed5015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4effd700 1 --2- 172.21.15.3:0/1955216064 >> v2:172.21.15.153:3300/0 conn(0x7fed5015c8f0 0x7fed5015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4effd700 1 -- 172.21.15.3:0/1955216064 >> v1:172.21.15.153:6789/0 conn(0x7fed500d06a0 legacy=0x7fed5006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4effd700 1 -- 172.21.15.3:0/1955216064 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500d0af0 con 0x7fed5015ccc0 2023-10-06T04:52:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4effd700 1 --2- 172.21.15.3:0/1955216064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015ccc0 0x7fed500d0160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fed440098e0 tx=0x7fed440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f9441357e3ba932 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4e7fc700 1 -- 172.21.15.3:0/1955216064 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400b640 con 0x7fed5015ccc0 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4e7fc700 1 -- 172.21.15.3:0/1955216064 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4400b7a0 con 0x7fed5015ccc0 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed4e7fc700 1 -- 172.21.15.3:0/1955216064 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400bb40 con 0x7fed5015ccc0 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.739+0000 7fed55c05700 1 -- 172.21.15.3:0/1955216064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015ccc0 msgr2=0x7fed500d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.740+0000 7fed55c05700 1 --2- 172.21.15.3:0/1955216064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015ccc0 0x7fed500d0160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fed440098e0 tx=0x7fed440091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.740+0000 7fed55c05700 1 -- 172.21.15.3:0/1955216064 shutdown_connections 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.740+0000 7fed55c05700 1 --2- 172.21.15.3:0/1955216064 >> v2:172.21.15.153:3300/0 conn(0x7fed5015c8f0 0x7fed5015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.740+0000 7fed55c05700 1 --2- 172.21.15.3:0/1955216064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015ccc0 0x7fed500d0160 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.740+0000 7fed55c05700 1 -- 172.21.15.3:0/1955216064 >> 172.21.15.3:0/1955216064 conn(0x7fed50070310 msgr2=0x7fed50070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.740+0000 7fed55c05700 1 -- 172.21.15.3:0/1955216064 shutdown_connections 2023-10-06T04:52:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.740+0000 7fed55c05700 1 -- 172.21.15.3:0/1955216064 wait complete. 2023-10-06T04:52:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.741+0000 7fed55c05700 1 Processor -- start 2023-10-06T04:52:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.741+0000 7fed55c05700 1 -- start start 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d06a0 0x7fed500c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015c8f0 0x7fed50155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed5015ccc0 0x7fed5006d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5016ad20 con 0x7fed500d06a0 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed500c65f0 con 0x7fed5015ccc0 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed500c5a70 con 0x7fed5015c8f0 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d06a0 0x7fed500c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d06a0 0x7fed500c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51202/0 (socket says 172.21.15.3:51202) 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 -- 172.21.15.3:0/1477396093 learned_addr learned my addr 172.21.15.3:0/1477396093 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4effd700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015c8f0 0x7fed50155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4ffff700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed5015ccc0 0x7fed5006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 -- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015c8f0 msgr2=0x7fed50155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015c8f0 0x7fed50155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 -- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed5015ccc0 msgr2=0x7fed5006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed5015ccc0 0x7fed5006d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 -- 172.21.15.3:0/1477396093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed44009420 con 0x7fed500d06a0 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed4f7fe700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d06a0 0x7fed500c5220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fed4000ec80 tx=0x7fed400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed377fe700 1 -- 172.21.15.3:0/1477396093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4000c850 con 0x7fed500d06a0 2023-10-06T04:52:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed377fe700 1 -- 172.21.15.3:0/1477396093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4000c9b0 con 0x7fed500d06a0 2023-10-06T04:52:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed500c5d50 con 0x7fed500d06a0 2023-10-06T04:52:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed377fe700 1 -- 172.21.15.3:0/1477396093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4000cd50 con 0x7fed500d06a0 2023-10-06T04:52:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.742+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed500c6310 con 0x7fed500d06a0 2023-10-06T04:52:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.743+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed50071680 con 0x7fed500d06a0 2023-10-06T04:52:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.744+0000 7fed377fe700 1 -- 172.21.15.3:0/1477396093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fed40013680 con 0x7fed500d06a0 2023-10-06T04:52:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.744+0000 7fed377fe700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed3c066ea0 0x7fed3c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.744+0000 7fed377fe700 1 -- 172.21.15.3:0/1477396093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fed400844c0 con 0x7fed500d06a0 2023-10-06T04:52:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.744+0000 7fed4effd700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed3c066ea0 0x7fed3c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.745+0000 7fed4effd700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed3c066ea0 0x7fed3c069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed5015d0b0 tx=0x7fed44017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.749+0000 7fed377fe700 1 -- 172.21.15.3:0/1477396093 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed40051a00 con 0x7fed500d06a0 2023-10-06T04:52:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.989+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed500cdf90 con 0x7fed500d06a0 2023-10-06T04:52:13.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.989+0000 7fed377fe700 1 -- 172.21.15.3:0/1477396093 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3518 (secure 0 0 0) 0x7fed40017dd0 con 0x7fed500d06a0 2023-10-06T04:52:13.991 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:13.991 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-06T04:52:13.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.991+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed3c066ea0 msgr2=0x7fed3c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.991+0000 7fed55c05700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed3c066ea0 0x7fed3c069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed5015d0b0 tx=0x7fed44017000 comp rx=0 tx=0).stop 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d06a0 msgr2=0x7fed500c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d06a0 0x7fed500c5220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fed4000ec80 tx=0x7fed400091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 shutdown_connections 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed3c066ea0 0x7fed3c069350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed5015ccc0 0x7fed5006d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d06a0 0x7fed500c5220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 --2- 172.21.15.3:0/1477396093 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed5015c8f0 0x7fed50155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 >> 172.21.15.3:0/1477396093 conn(0x7fed50070310 msgr2=0x7fed500b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 shutdown_connections 2023-10-06T04:52:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:13.992+0000 7fed55c05700 1 -- 172.21.15.3:0/1477396093 wait complete. 2023-10-06T04:52:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2023-10-06T04:52:14.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:14.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.123+0000 7fb830a11700 1 Processor -- start 2023-10-06T04:52:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.123+0000 7fb830a11700 1 -- start start 2023-10-06T04:52:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb830a11700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb82c15c8f0 0x7fb82c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb830a11700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c15ccc0 0x7fb82c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb830a11700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82c0cf6c0 con 0x7fb82c0d06a0 2023-10-06T04:52:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb830a11700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82c0cfb50 con 0x7fb82c15c8f0 2023-10-06T04:52:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb830a11700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c15e050 con 0x7fb82c15ccc0 2023-10-06T04:52:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb82c15c8f0 0x7fb82c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82ad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb82c0d06a0 0x7fb82c06d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44608/0 (socket says 172.21.15.3:44608) 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82ad9d700 1 -- 172.21.15.3:0/1887638975 learned_addr learned my addr 172.21.15.3:0/1887638975 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb829d9b700 1 --2- 172.21.15.3:0/1887638975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c15ccc0 0x7fb82c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82a59c700 1 -- 172.21.15.3:0/1887638975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c15ccc0 msgr2=0x7fb82c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82a59c700 1 --2- 172.21.15.3:0/1887638975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c15ccc0 0x7fb82c0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82a59c700 1 -- 172.21.15.3:0/1887638975 >> v1:172.21.15.3:6789/0 conn(0x7fb82c0d06a0 legacy=0x7fb82c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82959a700 1 -- 172.21.15.3:0/1887638975 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930340133 0 0) 0x7fb82c0cf6c0 con 0x7fb82c0d06a0 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82a59c700 1 -- 172.21.15.3:0/1887638975 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c0d0af0 con 0x7fb82c15c8f0 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82a59c700 1 --2- 172.21.15.3:0/1887638975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15c8f0 0x7fb82c15db10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098e0 tx=0x7fb81c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed5aa205fd48f6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.124+0000 7fb82959a700 1 -- 172.21.15.3:0/1887638975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c009e80 con 0x7fb82c15c8f0 2023-10-06T04:52:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb82959a700 1 -- 172.21.15.3:0/1887638975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00b860 con 0x7fb82c15c8f0 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb82959a700 1 -- 172.21.15.3:0/1887638975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00bbe0 con 0x7fb82c15c8f0 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 -- 172.21.15.3:0/1887638975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15c8f0 msgr2=0x7fb82c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 --2- 172.21.15.3:0/1887638975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15c8f0 0x7fb82c15db10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098e0 tx=0x7fb81c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 -- 172.21.15.3:0/1887638975 shutdown_connections 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 --2- 172.21.15.3:0/1887638975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15c8f0 0x7fb82c15db10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 --2- 172.21.15.3:0/1887638975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c15ccc0 0x7fb82c0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 -- 172.21.15.3:0/1887638975 >> 172.21.15.3:0/1887638975 conn(0x7fb82c070310 msgr2=0x7fb82c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 -- 172.21.15.3:0/1887638975 shutdown_connections 2023-10-06T04:52:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.125+0000 7fb830a11700 1 -- 172.21.15.3:0/1887638975 wait complete. 2023-10-06T04:52:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.126+0000 7fb830a11700 1 Processor -- start 2023-10-06T04:52:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.126+0000 7fb830a11700 1 -- start start 2023-10-06T04:52:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb830a11700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c0d06a0 0x7fb82c150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb830a11700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c15c8f0 0x7fb82c14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb830a11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15ccc0 0x7fb82c14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb830a11700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c14eaa0 con 0x7fb82c15ccc0 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb830a11700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c14de90 con 0x7fb82c15c8f0 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb830a11700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c14e170 con 0x7fb82c0d06a0 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15ccc0 0x7fb82c14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15ccc0 0x7fb82c14c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51230/0 (socket says 172.21.15.3:51230) 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 -- 172.21.15.3:0/3443575968 learned_addr learned my addr 172.21.15.3:0/3443575968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82a59c700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c0d06a0 0x7fb82c150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb829d9b700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c15c8f0 0x7fb82c14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 -- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c0d06a0 msgr2=0x7fb82c150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c0d06a0 0x7fb82c150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 -- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c15c8f0 msgr2=0x7fb82c14ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c15c8f0 0x7fb82c14ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb82ad9d700 1 -- 172.21.15.3:0/3443575968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009420 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.127+0000 7fb829d9b700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c15c8f0 0x7fb82c14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.128+0000 7fb82ad9d700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15ccc0 0x7fb82c14c990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb82400cbf0 tx=0x7fb8240019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.128+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400a910 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.128+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82400aa70 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.128+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c14e450 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.128+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400ae30 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.128+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c14d1f0 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.129+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c14ec10 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.129+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fb824007300 con 0x7fb82c15ccc0 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.129+0000 7fb81a7fc700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb80c066ea0 0x7fb80c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.130+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb824084560 con 0x7fb82c15ccc0 2023-10-06T04:52:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.130+0000 7fb82a59c700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb80c066ea0 0x7fb80c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.130+0000 7fb82a59c700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb80c066ea0 0x7fb80c069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0091e0 tx=0x7fb81c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.134+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb824051b20 con 0x7fb82c15ccc0 2023-10-06T04:52:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.233+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb824051ef0 con 0x7fb82c15ccc0 2023-10-06T04:52:14.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.330+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb82c071680 con 0x7fb82c15ccc0 2023-10-06T04:52:14.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.331+0000 7fb81a7fc700 1 -- 172.21.15.3:0/3443575968 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13600 (secure 0 0 0) 0x7fb8240555b0 con 0x7fb82c15ccc0 2023-10-06T04:52:14.333 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:14.333 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:13.535789+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:14.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb80c066ea0 msgr2=0x7fb80c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb80c066ea0 0x7fb80c069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0091e0 tx=0x7fb81c017000 comp rx=0 tx=0).stop 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15ccc0 msgr2=0x7fb82c14c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15ccc0 0x7fb82c14c990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb82400cbf0 tx=0x7fb8240019b0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 shutdown_connections 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb80c066ea0 0x7fb80c069350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb82c15ccc0 0x7fb82c14c990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb82c0d06a0 0x7fb82c150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 --2- 172.21.15.3:0/3443575968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c15c8f0 0x7fb82c14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 >> 172.21.15.3:0/3443575968 conn(0x7fb82c070310 msgr2=0x7fb82c15e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 shutdown_connections 2023-10-06T04:52:14.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.333+0000 7fb830a11700 1 -- 172.21.15.3:0/3443575968 wait complete. 2023-10-06T04:52:14.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:52:14.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.468+0000 7f77b0263700 1 Processor -- start 2023-10-06T04:52:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.469+0000 7f77b0263700 1 -- start start 2023-10-06T04:52:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77b0263700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a806a800 0x7f77a8159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77b0263700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77a80ae9d0 0x7f77a80af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77b0263700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a806db30 con 0x7f77a81599c0 2023-10-06T04:52:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77b0263700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a80af220 con 0x7f77a80ae9d0 2023-10-06T04:52:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77b0263700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806d520 con 0x7f77a806a800 2023-10-06T04:52:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77ae800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77a80ae9d0 0x7f77a80af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77adfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f77a81599c0 0x7f77a806a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44624/0 (socket says 172.21.15.3:44624) 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77adfff700 1 -- 172.21.15.3:0/3385519797 learned_addr learned my addr 172.21.15.3:0/3385519797 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77ad7fe700 1 --2- 172.21.15.3:0/3385519797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a806a800 0x7f77a8159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77ae800700 1 -- 172.21.15.3:0/3385519797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a806a800 msgr2=0x7f77a8159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77ae800700 1 --2- 172.21.15.3:0/3385519797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a806a800 0x7f77a8159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77ae800700 1 -- 172.21.15.3:0/3385519797 >> v1:172.21.15.3:6789/0 conn(0x7f77a81599c0 legacy=0x7f77a806a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77acffd700 1 -- 172.21.15.3:0/3385519797 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942162603 0 0) 0x7f77a806db30 con 0x7f77a81599c0 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77ae800700 1 -- 172.21.15.3:0/3385519797 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80aef40 con 0x7f77a80ae9d0 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77ae800700 1 --2- 172.21.15.3:0/3385519797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a80ae9d0 0x7f77a80af650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009e90 tx=0x7f77a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d4ae9466d1e066 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77acffd700 1 -- 172.21.15.3:0/3385519797 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000e070 con 0x7f77a80ae9d0 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77acffd700 1 -- 172.21.15.3:0/3385519797 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a000bc50 con 0x7f77a80ae9d0 2023-10-06T04:52:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.470+0000 7f77acffd700 1 -- 172.21.15.3:0/3385519797 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0012430 con 0x7f77a80ae9d0 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 -- 172.21.15.3:0/3385519797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a80ae9d0 msgr2=0x7f77a80af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 --2- 172.21.15.3:0/3385519797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a80ae9d0 0x7f77a80af650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009e90 tx=0x7f77a0009780 comp rx=0 tx=0).stop 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 -- 172.21.15.3:0/3385519797 shutdown_connections 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 --2- 172.21.15.3:0/3385519797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a80ae9d0 0x7f77a80af650 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 --2- 172.21.15.3:0/3385519797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a806a800 0x7f77a8159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 -- 172.21.15.3:0/3385519797 >> 172.21.15.3:0/3385519797 conn(0x7f77a8067a40 msgr2=0x7f77a80ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 -- 172.21.15.3:0/3385519797 shutdown_connections 2023-10-06T04:52:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.471+0000 7f77b0263700 1 -- 172.21.15.3:0/3385519797 wait complete. 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.472+0000 7f77b0263700 1 Processor -- start 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.472+0000 7f77b0263700 1 -- start start 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77b0263700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a806a800 0x7f77a80c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77b0263700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77a80ae9d0 0x7f77a8169170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77b0263700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a81599c0 0x7f77a816a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77b0263700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80ca700 con 0x7f77a806a800 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77b0263700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a81696b0 con 0x7f77a80ae9d0 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77b0263700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8169970 con 0x7f77a81599c0 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77adfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a806a800 0x7f77a80c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77adfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a806a800 0x7f77a80c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51250/0 (socket says 172.21.15.3:51250) 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77adfff700 1 -- 172.21.15.3:0/2072334410 learned_addr learned my addr 172.21.15.3:0/2072334410 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77ae800700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a81599c0 0x7f77a816a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77ad7fe700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77a80ae9d0 0x7f77a8169170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77adfff700 1 -- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a81599c0 msgr2=0x7f77a816a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77adfff700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a81599c0 0x7f77a816a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77adfff700 1 -- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77a80ae9d0 msgr2=0x7f77a8169170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.473+0000 7f77adfff700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77a80ae9d0 0x7f77a8169170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f77adfff700 1 -- 172.21.15.3:0/2072334410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0009a20 con 0x7f77a806a800 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f77ad7fe700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77a80ae9d0 0x7f77a8169170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f77adfff700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a806a800 0x7f77a80c98c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c5e0 tx=0x7f77a4007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f7799ffb700 1 -- 172.21.15.3:0/2072334410 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40072e0 con 0x7f77a806a800 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f7799ffb700 1 -- 172.21.15.3:0/2072334410 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4007460 con 0x7f77a806a800 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a8169c50 con 0x7f77a806a800 2023-10-06T04:52:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f7799ffb700 1 -- 172.21.15.3:0/2072334410 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40057f0 con 0x7f77a806a800 2023-10-06T04:52:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.474+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a80ca8e0 con 0x7f77a806a800 2023-10-06T04:52:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.475+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a816a170 con 0x7f77a806a800 2023-10-06T04:52:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.475+0000 7f7799ffb700 1 -- 172.21.15.3:0/2072334410 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77a4005c00 con 0x7f77a806a800 2023-10-06T04:52:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.476+0000 7f7799ffb700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f778c066f30 0x7f778c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.476+0000 7f7799ffb700 1 -- 172.21.15.3:0/2072334410 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f77a4083950 con 0x7f77a806a800 2023-10-06T04:52:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.476+0000 7f77ad7fe700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f778c066f30 0x7f778c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.477+0000 7f77ad7fe700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f778c066f30 0x7f778c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77a816c6c0 tx=0x7f779c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.480+0000 7f7799ffb700 1 -- 172.21.15.3:0/2072334410 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77a816a170 con 0x7f77a806a800 2023-10-06T04:52:14.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.680+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f77a816a170 con 0x7f77a806a800 2023-10-06T04:52:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.680+0000 7f7799ffb700 1 -- 172.21.15.3:0/2072334410 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f77a816a170 con 0x7f77a806a800 2023-10-06T04:52:14.682 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f778c066f30 msgr2=0x7f778c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f778c066f30 0x7f778c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77a816c6c0 tx=0x7f779c008040 comp rx=0 tx=0).stop 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a806a800 msgr2=0x7f77a80c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a806a800 0x7f77a80c98c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c5e0 tx=0x7f77a4007e80 comp rx=0 tx=0).stop 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 shutdown_connections 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f778c066f30 0x7f778c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77a81599c0 0x7f77a816a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a806a800 0x7f77a80c98c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 --2- 172.21.15.3:0/2072334410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77a80ae9d0 0x7f77a8169170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 >> 172.21.15.3:0/2072334410 conn(0x7f77a8067a40 msgr2=0x7f77a80acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 shutdown_connections 2023-10-06T04:52:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.682+0000 7f77b0263700 1 -- 172.21.15.3:0/2072334410 wait complete. 2023-10-06T04:52:14.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.816+0000 7f141092c700 1 Processor -- start 2023-10-06T04:52:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.816+0000 7f141092c700 1 -- start start 2023-10-06T04:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f141092c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f140c15cb30 0x7f140c0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f141092c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f140c0afec0 0x7f140c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f141092c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c0cf0f0 con 0x7f140c15c760 2023-10-06T04:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f141092c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c15cf80 con 0x7f140c15cb30 2023-10-06T04:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f141092c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c0ce7c0 con 0x7f140c0afec0 2023-10-06T04:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f140a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f140c15c760 0x7f140c15db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44630/0 (socket says 172.21.15.3:44630) 2023-10-06T04:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f1409d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f140c15cb30 0x7f140c0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f140a59c700 1 -- 172.21.15.3:0/1721245769 learned_addr learned my addr 172.21.15.3:0/1721245769 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f140959a700 1 -- 172.21.15.3:0/1721245769 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947762730 0 0) 0x7f140c0cf0f0 con 0x7f140c15c760 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.817+0000 7f140959a700 1 -- 172.21.15.3:0/1721245769 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f8003570 con 0x7f140c15c760 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f140ad9d700 1 --2- 172.21.15.3:0/1721245769 >> v2:172.21.15.153:3300/0 conn(0x7f140c0afec0 0x7f140c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f1409d9b700 1 -- 172.21.15.3:0/1721245769 >> v2:172.21.15.153:3300/0 conn(0x7f140c0afec0 msgr2=0x7f140c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f1409d9b700 1 --2- 172.21.15.3:0/1721245769 >> v2:172.21.15.153:3300/0 conn(0x7f140c0afec0 0x7f140c0cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f1409d9b700 1 -- 172.21.15.3:0/1721245769 >> v1:172.21.15.3:6789/0 conn(0x7f140c15c760 legacy=0x7f140c15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f1409d9b700 1 -- 172.21.15.3:0/1721245769 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c0ceaa0 con 0x7f140c15cb30 2023-10-06T04:52:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f140ad9d700 1 --2- 172.21.15.3:0/1721245769 >> v2:172.21.15.153:3300/0 conn(0x7f140c0afec0 0x7f140c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f1409d9b700 1 --2- 172.21.15.3:0/1721245769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c0cb3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f140c071040 tx=0x7f13f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ca1bb9b89d905d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f140959a700 1 -- 172.21.15.3:0/1721245769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4011630 con 0x7f140c15cb30 2023-10-06T04:52:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f140959a700 1 -- 172.21.15.3:0/1721245769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f4011790 con 0x7f140c15cb30 2023-10-06T04:52:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f140959a700 1 -- 172.21.15.3:0/1721245769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4011b30 con 0x7f140c15cb30 2023-10-06T04:52:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f141092c700 1 -- 172.21.15.3:0/1721245769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 msgr2=0x7f140c0cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f141092c700 1 --2- 172.21.15.3:0/1721245769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c0cb3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f140c071040 tx=0x7f13f40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f141092c700 1 -- 172.21.15.3:0/1721245769 shutdown_connections 2023-10-06T04:52:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f141092c700 1 --2- 172.21.15.3:0/1721245769 >> v2:172.21.15.153:3300/0 conn(0x7f140c0afec0 0x7f140c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f141092c700 1 --2- 172.21.15.3:0/1721245769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c0cb3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.818+0000 7f141092c700 1 -- 172.21.15.3:0/1721245769 >> 172.21.15.3:0/1721245769 conn(0x7f140c06f940 msgr2=0x7f140c070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.819+0000 7f141092c700 1 -- 172.21.15.3:0/1721245769 shutdown_connections 2023-10-06T04:52:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.819+0000 7f141092c700 1 -- 172.21.15.3:0/1721245769 wait complete. 2023-10-06T04:52:14.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.820+0000 7f141092c700 1 Processor -- start 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.820+0000 7f141092c700 1 -- start start 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f141092c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f140c0afec0 0x7f140c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f141092c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f140c15c760 0x7f140c16da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f141092c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c169ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f141092c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c1590e0 con 0x7f140c15cb30 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f141092c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0b4150 con 0x7f140c0afec0 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f141092c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0b3540 con 0x7f140c15c760 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c169ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51262/0 (socket says 172.21.15.3:51262) 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 -- 172.21.15.3:0/2852826379 learned_addr learned my addr 172.21.15.3:0/2852826379 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140a59c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f140c0afec0 0x7f140c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f1409d9b700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f140c15c760 0x7f140c16da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 -- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f140c15c760 msgr2=0x7f140c16da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f140c15c760 0x7f140c16da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 -- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f140c0afec0 msgr2=0x7f140c158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f140c0afec0 0x7f140c158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 -- 172.21.15.3:0/2852826379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f4009420 con 0x7f140c15cb30 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.821+0000 7f140ad9d700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c169ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f140400cf20 tx=0x7f1404007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.822+0000 7f14027fc700 1 -- 172.21.15.3:0/2852826379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400ace0 con 0x7f140c15cb30 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.822+0000 7f14027fc700 1 -- 172.21.15.3:0/2852826379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140400ae60 con 0x7f140c15cb30 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.822+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c0b3820 con 0x7f140c15cb30 2023-10-06T04:52:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.822+0000 7f14027fc700 1 -- 172.21.15.3:0/2852826379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404016670 con 0x7f140c15cb30 2023-10-06T04:52:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.822+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c0b3de0 con 0x7f140c15cb30 2023-10-06T04:52:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.823+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c04c380 con 0x7f140c15cb30 2023-10-06T04:52:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.823+0000 7f14027fc700 1 -- 172.21.15.3:0/2852826379 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1404016a20 con 0x7f140c15cb30 2023-10-06T04:52:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.823+0000 7f14027fc700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f13ec066f30 0x7f13ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.823+0000 7f14027fc700 1 -- 172.21.15.3:0/2852826379 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1404012070 con 0x7f140c15cb30 2023-10-06T04:52:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.824+0000 7f140a59c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f13ec066f30 0x7f13ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.824+0000 7f140a59c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f13ec066f30 0x7f13ec0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f13fc007fd0 tx=0x7f13fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:14.828+0000 7f14027fc700 1 -- 172.21.15.3:0/2852826379 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f140404e5a0 con 0x7f140c15cb30 2023-10-06T04:52:15.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.022+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f140c0ce120 con 0x7f140c15cb30 2023-10-06T04:52:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.022+0000 7f14027fc700 1 -- 172.21.15.3:0/2852826379 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1404052070 con 0x7f140c15cb30 2023-10-06T04:52:15.024 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:15.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f13ec066f30 msgr2=0x7f13ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f13ec066f30 0x7f13ec0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f13fc007fd0 tx=0x7f13fc009040 comp rx=0 tx=0).stop 2023-10-06T04:52:15.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 msgr2=0x7f140c169ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c169ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f140400cf20 tx=0x7f1404007860 comp rx=0 tx=0).stop 2023-10-06T04:52:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 shutdown_connections 2023-10-06T04:52:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f13ec066f30 0x7f13ec0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f140c0afec0 0x7f140c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140c15cb30 0x7f140c169ac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 --2- 172.21.15.3:0/2852826379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f140c15c760 0x7f140c16da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 >> 172.21.15.3:0/2852826379 conn(0x7f140c06f940 msgr2=0x7f140c15aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 shutdown_connections 2023-10-06T04:52:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.025+0000 7f141092c700 1 -- 172.21.15.3:0/2852826379 wait complete. 2023-10-06T04:52:15.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:52:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.155+0000 7f12d492e700 1 Processor -- start 2023-10-06T04:52:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.155+0000 7f12d492e700 1 -- start start 2023-10-06T04:52:15.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12d492e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12d0157400 0x7f12d00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12d492e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d015aeb0 con 0x7f12d00b3b70 2023-10-06T04:52:15.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12d492e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d015a7a0 con 0x7f12d0157030 2023-10-06T04:52:15.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12d492e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d015aa80 con 0x7f12d0157400 2023-10-06T04:52:15.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12cdd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12d0157400 0x7f12d00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12ced9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12d00b3b70 0x7f12d006e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44640/0 (socket says 172.21.15.3:44640) 2023-10-06T04:52:15.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12cdd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12d0157400 0x7f12d00b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51272/0 (socket says 172.21.15.3:51272) 2023-10-06T04:52:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.156+0000 7f12ced9d700 1 -- 172.21.15.3:0/3466461580 learned_addr learned my addr 172.21.15.3:0/3466461580 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12cdd9b700 1 -- 172.21.15.3:0/3466461580 >> v1:172.21.15.153:6789/0 conn(0x7f12d0157030 legacy=0x7f12d0159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12cdd9b700 1 -- 172.21.15.3:0/3466461580 >> v1:172.21.15.3:6789/0 conn(0x7f12d00b3b70 legacy=0x7f12d006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12cdd9b700 1 -- 172.21.15.3:0/3466461580 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d006dba0 con 0x7f12d0157400 2023-10-06T04:52:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12cdd9b700 1 --2- 172.21.15.3:0/3466461580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00b3630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12b80098e0 tx=0x7f12b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcc4907a66ea1b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12cd59a700 1 -- 172.21.15.3:0/3466461580 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8009e80 con 0x7f12d0157400 2023-10-06T04:52:15.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12cd59a700 1 -- 172.21.15.3:0/3466461580 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b800b820 con 0x7f12d0157400 2023-10-06T04:52:15.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12cd59a700 1 -- 172.21.15.3:0/3466461580 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800bba0 con 0x7f12d0157400 2023-10-06T04:52:15.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12d492e700 1 -- 172.21.15.3:0/3466461580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 msgr2=0x7f12d00b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12d492e700 1 --2- 172.21.15.3:0/3466461580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00b3630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12b80098e0 tx=0x7f12b80091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12d492e700 1 -- 172.21.15.3:0/3466461580 shutdown_connections 2023-10-06T04:52:15.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12d492e700 1 --2- 172.21.15.3:0/3466461580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00b3630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.157+0000 7f12d492e700 1 -- 172.21.15.3:0/3466461580 >> 172.21.15.3:0/3466461580 conn(0x7f12d00ce270 msgr2=0x7f12d00ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:15.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.158+0000 7f12d492e700 1 -- 172.21.15.3:0/3466461580 shutdown_connections 2023-10-06T04:52:15.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.158+0000 7f12d492e700 1 -- 172.21.15.3:0/3466461580 wait complete. 2023-10-06T04:52:15.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 Processor -- start 2023-10-06T04:52:15.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 -- start start 2023-10-06T04:52:15.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12d00b3b70 0x7f12d00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12d0157030 0x7f12d00ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00affe0 con 0x7f12d0157400 2023-10-06T04:52:15.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00af3d0 con 0x7f12d00b3b70 2023-10-06T04:52:15.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12d492e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00af6b0 con 0x7f12d0157030 2023-10-06T04:52:15.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12ced9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12ced9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51284/0 (socket says 172.21.15.3:51284) 2023-10-06T04:52:15.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.159+0000 7f12ced9d700 1 -- 172.21.15.3:0/1528114185 learned_addr learned my addr 172.21.15.3:0/1528114185 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:15.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12cdd9b700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12d0157030 0x7f12d00ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12ce59c700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12d00b3b70 0x7f12d00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12ced9d700 1 -- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12d0157030 msgr2=0x7f12d00ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12ced9d700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12d0157030 0x7f12d00ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12ced9d700 1 -- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12d00b3b70 msgr2=0x7f12d00b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12ced9d700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12d00b3b70 0x7f12d00b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12ced9d700 1 -- 172.21.15.3:0/1528114185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8009420 con 0x7f12d0157400 2023-10-06T04:52:15.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12ced9d700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00ca280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12c000cc40 tx=0x7f12c0001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:15.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12c67fc700 1 -- 172.21.15.3:0/1528114185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000a8b0 con 0x7f12d0157400 2023-10-06T04:52:15.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12c67fc700 1 -- 172.21.15.3:0/1528114185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000aa10 con 0x7f12d0157400 2023-10-06T04:52:15.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d00af990 con 0x7f12d0157400 2023-10-06T04:52:15.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12c67fc700 1 -- 172.21.15.3:0/1528114185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000add0 con 0x7f12d0157400 2023-10-06T04:52:15.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.160+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d00b09b0 con 0x7f12d0157400 2023-10-06T04:52:15.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.161+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d00644c0 con 0x7f12d0157400 2023-10-06T04:52:15.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.161+0000 7f12c67fc700 1 -- 172.21.15.3:0/1528114185 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12c0007300 con 0x7f12d0157400 2023-10-06T04:52:15.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.162+0000 7f12c67fc700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f12b0066f30 0x7f12b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.162+0000 7f12c67fc700 1 -- 172.21.15.3:0/1528114185 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f12c00845e0 con 0x7f12d0157400 2023-10-06T04:52:15.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.162+0000 7f12ce59c700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f12b0066f30 0x7f12b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.162+0000 7f12ce59c700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f12b0066f30 0x7f12b00693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12c8003a30 tx=0x7f12c8007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.166+0000 7f12c67fc700 1 -- 172.21.15.3:0/1528114185 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12c0051ac0 con 0x7f12d0157400 2023-10-06T04:52:15.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.363+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f12d0071680 con 0x7f12d0157400 2023-10-06T04:52:15.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.363+0000 7f12c67fc700 1 -- 172.21.15.3:0/1528114185 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f12c00550e0 con 0x7f12d0157400 2023-10-06T04:52:15.365 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:15.365 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:52:15.365 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:52:15.365 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:52:15.366 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:52:15.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:15.366 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:52:15.366 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:52:15.366 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:52:15.366 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:52:15.366 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:52:15.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f12b0066f30 msgr2=0x7f12b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f12b0066f30 0x7f12b00693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12c8003a30 tx=0x7f12c8007ea0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 msgr2=0x7f12d00ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00ca280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12c000cc40 tx=0x7f12c0001a50 comp rx=0 tx=0).stop 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 shutdown_connections 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f12b0066f30 0x7f12b00693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12d00b3b70 0x7f12d00b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0157400 0x7f12d00ca280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 --2- 172.21.15.3:0/1528114185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12d0157030 0x7f12d00ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 >> 172.21.15.3:0/1528114185 conn(0x7f12d00ce270 msgr2=0x7f12d01578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 shutdown_connections 2023-10-06T04:52:15.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.365+0000 7f12d492e700 1 -- 172.21.15.3:0/1528114185 wait complete. 2023-10-06T04:52:15.379 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:52:15.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:52:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.500+0000 7f2b5de44700 1 Processor -- start 2023-10-06T04:52:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.500+0000 7f2b5de44700 1 -- start start 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.501+0000 7f2b5de44700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b580cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.501+0000 7f2b5de44700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b5815dca0 0x7f2b58159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.501+0000 7f2b5de44700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b580d15c0 con 0x7f2b580d1170 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.501+0000 7f2b5de44700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5815e400 con 0x7f2b5815dca0 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.501+0000 7f2b5de44700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580ccda0 con 0x7f2b581598f0 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.501+0000 7f2b57fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b5815dca0 0x7f2b58159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b577fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b580cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b57fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b5815dca0 0x7f2b58159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51290/0 (socket says 172.21.15.3:51290) 2023-10-06T04:52:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b56ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2b580d1170 0x7f2b5815d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53502/0 (socket says 172.21.15.3:53502) 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b57fff700 1 -- 172.21.15.3:0/2324625164 learned_addr learned my addr 172.21.15.3:0/2324625164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b577fe700 1 -- 172.21.15.3:0/2324625164 >> v2:172.21.15.3:3300/0 conn(0x7f2b5815dca0 msgr2=0x7f2b58159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b577fe700 1 --2- 172.21.15.3:0/2324625164 >> v2:172.21.15.3:3300/0 conn(0x7f2b5815dca0 0x7f2b58159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b577fe700 1 -- 172.21.15.3:0/2324625164 >> v1:172.21.15.153:6789/0 conn(0x7f2b580d1170 legacy=0x7f2b5815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b577fe700 1 -- 172.21.15.3:0/2324625164 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b580cf7d0 con 0x7f2b581598f0 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b567fc700 1 -- 172.21.15.3:0/2324625164 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235314084 0 0) 0x7f2b580d15c0 con 0x7f2b580d1170 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b577fe700 1 --2- 172.21.15.3:0/2324625164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b580cebd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098e0 tx=0x7f2b480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d6b77b11f7bf30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b567fc700 1 -- 172.21.15.3:0/2324625164 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800b6d0 con 0x7f2b581598f0 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b567fc700 1 -- 172.21.15.3:0/2324625164 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4800b830 con 0x7f2b581598f0 2023-10-06T04:52:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.502+0000 7f2b567fc700 1 -- 172.21.15.3:0/2324625164 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800bbd0 con 0x7f2b581598f0 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 -- 172.21.15.3:0/2324625164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 msgr2=0x7f2b580cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 --2- 172.21.15.3:0/2324625164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b580cebd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098e0 tx=0x7f2b480091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 -- 172.21.15.3:0/2324625164 shutdown_connections 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 --2- 172.21.15.3:0/2324625164 >> v2:172.21.15.3:3300/0 conn(0x7f2b5815dca0 0x7f2b58159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 --2- 172.21.15.3:0/2324625164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b580cebd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 -- 172.21.15.3:0/2324625164 >> 172.21.15.3:0/2324625164 conn(0x7f2b58070af0 msgr2=0x7f2b580d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 -- 172.21.15.3:0/2324625164 shutdown_connections 2023-10-06T04:52:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.503+0000 7f2b5de44700 1 -- 172.21.15.3:0/2324625164 wait complete. 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.504+0000 7f2b5de44700 1 Processor -- start 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.504+0000 7f2b5de44700 1 -- start start 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b5de44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b580d1170 0x7f2b580b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b5de44700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b5806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b5de44700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b5815dca0 0x7f2b5815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b5de44700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5806f260 con 0x7f2b580d1170 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b5de44700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5806e650 con 0x7f2b5815dca0 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b5de44700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5806e930 con 0x7f2b581598f0 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b577fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b580d1170 0x7f2b580b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b577fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b580d1170 0x7f2b580b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51304/0 (socket says 172.21.15.3:51304) 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b577fe700 1 -- 172.21.15.3:0/1011139957 learned_addr learned my addr 172.21.15.3:0/1011139957 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b57fff700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b5815dca0 0x7f2b5815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.505+0000 7f2b56ffd700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b5806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b577fe700 1 -- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 msgr2=0x7f2b5806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b577fe700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b5806dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b577fe700 1 -- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b5815dca0 msgr2=0x7f2b5815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b577fe700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b5815dca0 0x7f2b5815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b577fe700 1 -- 172.21.15.3:0/1011139957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48009420 con 0x7f2b580d1170 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b577fe700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b580d1170 0x7f2b580b47d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b580ccee0 tx=0x7f2b48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b3f7fe700 1 -- 172.21.15.3:0/1011139957 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48000d10 con 0x7f2b580d1170 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b3f7fe700 1 -- 172.21.15.3:0/1011139957 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4800b9a0 con 0x7f2b580d1170 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5806ec10 con 0x7f2b580d1170 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b3f7fe700 1 -- 172.21.15.3:0/1011139957 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48025650 con 0x7f2b580d1170 2023-10-06T04:52:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.506+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5815be80 con 0x7f2b580d1170 2023-10-06T04:52:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.507+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5815c070 con 0x7f2b580d1170 2023-10-06T04:52:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.507+0000 7f2b3f7fe700 1 -- 172.21.15.3:0/1011139957 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b48023070 con 0x7f2b580d1170 2023-10-06T04:52:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.508+0000 7f2b3f7fe700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.508+0000 7f2b3f7fe700 1 -- 172.21.15.3:0/1011139957 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2b48085840 con 0x7f2b580d1170 2023-10-06T04:52:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.508+0000 7f2b56ffd700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.509+0000 7f2b56ffd700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b44066f30 0x7f2b440693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b5809e470 tx=0x7f2b4c007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.512+0000 7f2b3f7fe700 1 -- 172.21.15.3:0/1011139957 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b48052ca0 con 0x7f2b580d1170 2023-10-06T04:52:15.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:15.706+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2b5806f540 con 0x7f2b580d1170 2023-10-06T04:52:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.627+0000 7f2b3f7fe700 1 -- 172.21.15.3:0/1011139957 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2b480562c0 con 0x7f2b580d1170 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b44066f30 msgr2=0x7f2b440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b44066f30 0x7f2b440693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b5809e470 tx=0x7f2b4c007ef0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b580d1170 msgr2=0x7f2b580b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b580d1170 0x7f2b580b47d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b580ccee0 tx=0x7f2b48017000 comp rx=0 tx=0).stop 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 shutdown_connections 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b5815dca0 0x7f2b5815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b580d1170 0x7f2b580b47d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 --2- 172.21.15.3:0/1011139957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b581598f0 0x7f2b5806dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.631+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 >> 172.21.15.3:0/1011139957 conn(0x7f2b58070af0 msgr2=0x7f2b581568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.632+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 shutdown_connections 2023-10-06T04:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.632+0000 7f2b5de44700 1 -- 172.21.15.3:0/1011139957 wait complete. 2023-10-06T04:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:52:16.650 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:52:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.781+0000 7f2bf57aa700 1 Processor -- start 2023-10-06T04:52:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.781+0000 7f2bf57aa700 1 -- start start 2023-10-06T04:52:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf57aa700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af660 0x7f2bf00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf57aa700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2bf00cd190 0x7f2bf00c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:16.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf57aa700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf00cb190 con 0x7f2bf00af290 2023-10-06T04:52:16.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf57aa700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf0159950 con 0x7f2bf00cd190 2023-10-06T04:52:16.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf57aa700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf00cd6a0 con 0x7f2bf00af660 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf4fa9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2bf00cd190 0x7f2bf00c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bef7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af660 0x7f2bf00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf4fa9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2bf00cd190 0x7f2bf00c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:55294/0 (socket says 172.21.15.3:55294) 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2beffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2bf00af290 0x7f2bf00ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53510/0 (socket says 172.21.15.3:53510) 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.782+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/3713777871 learned_addr learned my addr 172.21.15.3:0/3713777871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af660 msgr2=0x7f2bf00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2bf4fa9700 1 --2- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af660 0x7f2bf00ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/3713777871 >> v1:172.21.15.153:6789/0 conn(0x7f2bf00af290 legacy=0x7f2bf00ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/3713777871 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf00cb3f0 con 0x7f2bf00cd190 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2beeffd700 1 -- 172.21.15.3:0/3713777871 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101640107 0 0) 0x7f2bf00cb190 con 0x7f2bf00af290 2023-10-06T04:52:16.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af660 0x7f2bf00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2bf4fa9700 1 --2- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009f20 tx=0x7f2be8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e385fa6353f5149 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2beeffd700 1 -- 172.21.15.3:0/3713777871 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800e070 con 0x7f2bf00cd190 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2beeffd700 1 -- 172.21.15.3:0/3713777871 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be800b9d0 con 0x7f2bf00cd190 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.783+0000 7f2beeffd700 1 -- 172.21.15.3:0/3713777871 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80092e0 con 0x7f2bf00cd190 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 -- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00cd190 msgr2=0x7f2bf00c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009f20 tx=0x7f2be8009840 comp rx=0 tx=0).stop 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 -- 172.21.15.3:0/3713777871 shutdown_connections 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5c20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/3713777871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af660 0x7f2bf00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 -- 172.21.15.3:0/3713777871 >> 172.21.15.3:0/3713777871 conn(0x7f2bf0089dd0 msgr2=0x7f2bf00996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 -- 172.21.15.3:0/3713777871 shutdown_connections 2023-10-06T04:52:16.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.784+0000 7f2bf57aa700 1 -- 172.21.15.3:0/3713777871 wait complete. 2023-10-06T04:52:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.785+0000 7f2bf57aa700 1 Processor -- start 2023-10-06T04:52:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.785+0000 7f2bf57aa700 1 -- start start 2023-10-06T04:52:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf57aa700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af290 0x7f2bf00b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf57aa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00af660 0x7f2bf016d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf57aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf57aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf00d1150 con 0x7f2bf00cd190 2023-10-06T04:52:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf57aa700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf00d09c0 con 0x7f2bf00af660 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf57aa700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf016d7e0 con 0x7f2bf00af290 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51310/0 (socket says 172.21.15.3:51310) 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/1573884267 learned_addr learned my addr 172.21.15.3:0/1573884267 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2beffff700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af290 0x7f2bf00b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00af660 0x7f2bf016d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af290 msgr2=0x7f2bf00b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af290 0x7f2bf00b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00af660 msgr2=0x7f2bf016d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00af660 0x7f2bf016d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.786+0000 7f2bf4fa9700 1 -- 172.21.15.3:0/1573884267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00093c0 con 0x7f2bf00cd190 2023-10-06T04:52:16.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.787+0000 7f2bf4fa9700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00cb320 tx=0x7f2be8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.787+0000 7f2be7fff700 1 -- 172.21.15.3:0/1573884267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8008b10 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.787+0000 7f2be7fff700 1 -- 172.21.15.3:0/1573884267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be8008c90 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.787+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be8009ae0 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.787+0000 7f2be7fff700 1 -- 172.21.15.3:0/1573884267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80256b0 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.787+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf00d0e70 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.788+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf004c380 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.788+0000 7f2be7fff700 1 -- 172.21.15.3:0/1573884267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2be8003040 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.788+0000 7f2be7fff700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2bd0066f30 0x7f2bd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.789+0000 7f2be7fff700 1 -- 172.21.15.3:0/1573884267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f2be8085fc0 con 0x7f2bf00cd190 2023-10-06T04:52:16.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.789+0000 7f2beffff700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2bd0066f30 0x7f2bd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:16.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.789+0000 7f2beffff700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2bd0066f30 0x7f2bd00693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00c5c00 tx=0x7f2bd8007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:16.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.793+0000 7f2be7fff700 1 -- 172.21.15.3:0/1573884267 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2be80b75a0 con 0x7f2bf00cd190 2023-10-06T04:52:16.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:16.985+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2bf01591d0 con 0x7f2bf00cd190 2023-10-06T04:52:17.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.636+0000 7f2be7fff700 1 -- 172.21.15.3:0/1573884267 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2be80532b0 con 0x7f2bf00cd190 2023-10-06T04:52:17.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.643+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2bd0066f30 msgr2=0x7f2bd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.643+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2bd0066f30 0x7f2bd00693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00c5c00 tx=0x7f2bd8007d20 comp rx=0 tx=0).stop 2023-10-06T04:52:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.643+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf00cd190 msgr2=0x7f2bf00c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.643+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00cb320 tx=0x7f2be8018000 comp rx=0 tx=0).stop 2023-10-06T04:52:17.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.645+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 shutdown_connections 2023-10-06T04:52:17.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.645+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2bd0066f30 0x7f2bd00693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.645+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf00cd190 0x7f2bf00c5050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.645+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bf00af290 0x7f2bf00b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.645+0000 7f2bf57aa700 1 --2- 172.21.15.3:0/1573884267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bf00af660 0x7f2bf016d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.645+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 >> 172.21.15.3:0/1573884267 conn(0x7f2bf0089dd0 msgr2=0x7f2bf00c60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.645+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 shutdown_connections 2023-10-06T04:52:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.649+0000 7f2bf57aa700 1 -- 172.21.15.3:0/1573884267 wait complete. 2023-10-06T04:52:17.654 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:52:17.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:52:17.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.794+0000 7f1fb1830700 1 Processor -- start 2023-10-06T04:52:17.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.794+0000 7f1fb1830700 1 -- start start 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1fb1830700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1fac0cfb80 0x7f1fac0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1fb1830700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1fac0afad0 0x7f1fac0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1fb1830700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac15d510 con 0x7f1fac0cef10 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1fb1830700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac15a480 con 0x7f1fac0cfb80 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1fb1830700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac15a760 con 0x7f1fac0afad0 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1faaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1fac0cfb80 0x7f1fac0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1fab7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1fac0cef10 0x7f1fac15da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44646/0 (socket says 172.21.15.3:44646) 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1fab7fe700 1 -- 172.21.15.3:0/3854689047 learned_addr learned my addr 172.21.15.3:0/3854689047 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/3854689047 >> v2:172.21.15.153:3300/0 conn(0x7f1fac0afad0 0x7f1fac0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1faaffd700 1 -- 172.21.15.3:0/3854689047 >> v2:172.21.15.153:3300/0 conn(0x7f1fac0afad0 msgr2=0x7f1fac0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1faaffd700 1 --2- 172.21.15.3:0/3854689047 >> v2:172.21.15.153:3300/0 conn(0x7f1fac0afad0 0x7f1fac0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1faaffd700 1 -- 172.21.15.3:0/3854689047 >> v1:172.21.15.3:6789/0 conn(0x7f1fac0cef10 legacy=0x7f1fac15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1faaffd700 1 -- 172.21.15.3:0/3854689047 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac15aa40 con 0x7f1fac0cfb80 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.795+0000 7f1faaffd700 1 --2- 172.21.15.3:0/3854689047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0cfb80 0x7f1fac0af590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0cb2e0 tx=0x7f1f9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913dc0be157d5172 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/3854689047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011630 con 0x7f1fac0cfb80 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/3854689047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c011790 con 0x7f1fac0cfb80 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/3854689047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011b30 con 0x7f1fac0cfb80 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 -- 172.21.15.3:0/3854689047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0cfb80 msgr2=0x7f1fac0af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 --2- 172.21.15.3:0/3854689047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0cfb80 0x7f1fac0af590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0cb2e0 tx=0x7f1f9c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 -- 172.21.15.3:0/3854689047 shutdown_connections 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 --2- 172.21.15.3:0/3854689047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0cfb80 0x7f1fac0af590 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 --2- 172.21.15.3:0/3854689047 >> v2:172.21.15.153:3300/0 conn(0x7f1fac0afad0 0x7f1fac0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 -- 172.21.15.3:0/3854689047 >> 172.21.15.3:0/3854689047 conn(0x7f1fac070310 msgr2=0x7f1fac070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:17.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 -- 172.21.15.3:0/3854689047 shutdown_connections 2023-10-06T04:52:17.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.796+0000 7f1fb1830700 1 -- 172.21.15.3:0/3854689047 wait complete. 2023-10-06T04:52:17.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.797+0000 7f1fb1830700 1 Processor -- start 2023-10-06T04:52:17.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.797+0000 7f1fb1830700 1 -- start start 2023-10-06T04:52:17.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1fb1830700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0afad0 0x7f1fac0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:17.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1fb1830700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fac0cef10 0x7f1fac158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1fb1830700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fac0cfb80 0x7f1fac1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1fb1830700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c74f0 con 0x7f1fac0afad0 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1fb1830700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0b3b50 con 0x7f1fac0cef10 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1fb1830700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0b3e30 con 0x7f1fac0cfb80 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1faaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0afad0 0x7f1fac0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1faaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0afad0 0x7f1fac0b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51326/0 (socket says 172.21.15.3:51326) 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1faaffd700 1 -- 172.21.15.3:0/4264947077 learned_addr learned my addr 172.21.15.3:0/4264947077 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fac0cef10 0x7f1fac158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.798+0000 7f1fab7fe700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fac0cfb80 0x7f1fac1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1faaffd700 1 -- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fac0cfb80 msgr2=0x7f1fac1591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1faaffd700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fac0cfb80 0x7f1fac1591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1faaffd700 1 -- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fac0cef10 msgr2=0x7f1fac158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1faaffd700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fac0cef10 0x7f1fac158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1faaffd700 1 -- 172.21.15.3:0/4264947077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c009420 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1fab7fe700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fac0cfb80 0x7f1fac1591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1faaffd700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0afad0 0x7f1fac0b31e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fac089720 tx=0x7f1f9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1f92ffd700 1 -- 172.21.15.3:0/4264947077 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011b10 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1f92ffd700 1 -- 172.21.15.3:0/4264947077 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c0233f0 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1f92ffd700 1 -- 172.21.15.3:0/4264947077 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c023790 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac0b4110 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.799+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac0b3720 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.800+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac04c380 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.800+0000 7f1f92ffd700 1 -- 172.21.15.3:0/4264947077 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f9c023b80 con 0x7f1fac0afad0 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.801+0000 7f1f92ffd700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f98066f30 0x7f1f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:17.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.801+0000 7f1f92ffd700 1 -- 172.21.15.3:0/4264947077 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f1f9c0858f0 con 0x7f1fac0afad0 2023-10-06T04:52:17.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.801+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f98066f30 0x7f1f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:17.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.801+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f98066f30 0x7f1f980693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fac1575e0 tx=0x7f1fa00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:17.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:17.805+0000 7f1f92ffd700 1 -- 172.21.15.3:0/4264947077 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f9c052ae0 con 0x7f1fac0afad0 2023-10-06T04:52:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.045+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1fac0ce470 con 0x7f1fac0afad0 2023-10-06T04:52:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.639+0000 7f1f92ffd700 1 -- 172.21.15.3:0/4264947077 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1f9c015020 con 0x7f1fac0afad0 2023-10-06T04:52:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.653+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f98066f30 msgr2=0x7f1f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.653+0000 7f1fb1830700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f98066f30 0x7f1f980693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fac1575e0 tx=0x7f1fa00091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.653+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0afad0 msgr2=0x7f1fac0b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.653+0000 7f1fb1830700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0afad0 0x7f1fac0b31e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fac089720 tx=0x7f1f9c017000 comp rx=0 tx=0).stop 2023-10-06T04:52:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.654+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 shutdown_connections 2023-10-06T04:52:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.654+0000 7f1fb1830700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f98066f30 0x7f1f980693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.654+0000 7f1fb1830700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fac0cfb80 0x7f1fac1591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.654+0000 7f1fb1830700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0afad0 0x7f1fac0b31e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.654+0000 7f1fb1830700 1 --2- 172.21.15.3:0/4264947077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fac0cef10 0x7f1fac158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.654+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 >> 172.21.15.3:0/4264947077 conn(0x7f1fac070310 msgr2=0x7f1fac155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.660+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 shutdown_connections 2023-10-06T04:52:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.660+0000 7f1fb1830700 1 -- 172.21.15.3:0/4264947077 wait complete. 2023-10-06T04:52:18.665 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T04:52:18.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:18.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.799+0000 7f21fcd6c700 1 Processor -- start 2023-10-06T04:52:18.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.799+0000 7f21fcd6c700 1 -- start start 2023-10-06T04:52:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.800+0000 7f21fcd6c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 0x7f21f815d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.800+0000 7f21fcd6c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f815e0f0 con 0x7f21f806aaa0 2023-10-06T04:52:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.800+0000 7f21fcd6c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80c93f0 con 0x7f21f8159810 2023-10-06T04:52:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.800+0000 7f21fcd6c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f815d850 con 0x7f21f8159440 2023-10-06T04:52:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.800+0000 7f21f6d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f21f806aaa0 0x7f21f80b1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44662/0 (socket says 172.21.15.3:44662) 2023-10-06T04:52:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f6d9d700 1 -- 172.21.15.3:0/2196499073 learned_addr learned my addr 172.21.15.3:0/2196499073 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f659c700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 0x7f21f815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517983414 0 0) 0x7f21f815e0f0 con 0x7f21f806aaa0 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e4003570 con 0x7f21f806aaa0 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3238543322 0 0) 0x7f21e4003570 con 0x7f21f806aaa0 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 msgr2=0x7f21f815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 0x7f21f815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 >> v1:172.21.15.153:6789/0 conn(0x7f21f8159810 legacy=0x7f21f806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f815db30 con 0x7f21f806aaa0 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f21f0001470 con 0x7f21f806aaa0 2023-10-06T04:52:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 >> v1:172.21.15.3:6789/0 conn(0x7f21f806aaa0 legacy=0x7f21f80b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 0x7f21e4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21e4003fd0 0x7f21e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e4004d50 0x7f21e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f815d850 con 0x7f21e4004d50 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e400bb40 con 0x7f21e4003fd0 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e400bd00 con 0x7f21f8159440 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f6d9d700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e4004d50 0x7f21e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f5d9b700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21e4003fd0 0x7f21e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f659c700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 0x7f21e4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f6d9d700 1 -- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 msgr2=0x7f21e4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f6d9d700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21f8159440 0x7f21e4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f6d9d700 1 -- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21e4003fd0 msgr2=0x7f21e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f6d9d700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21e4003fd0 0x7f21e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.801+0000 7f21f6d9d700 1 -- 172.21.15.3:0/2196499073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4003570 con 0x7f21e4004d50 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21f6d9d700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e4004d50 0x7f21e40051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c3c0 tx=0x7f21f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d5b1cbfdc28c22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f000cc90 con 0x7f21e4004d50 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f000cdf0 con 0x7f21e4004d50 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21f559a700 1 -- 172.21.15.3:0/2196499073 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0005b70 con 0x7f21e4004d50 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21fcd6c700 1 -- 172.21.15.3:0/2196499073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e4004d50 msgr2=0x7f21e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21fcd6c700 1 --2- 172.21.15.3:0/2196499073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e4004d50 0x7f21e40051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c3c0 tx=0x7f21f0001e70 comp rx=0 tx=0).stop 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21f6d9d700 1 -- 172.21.15.3:0/2196499073 reap_dead start 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21fcd6c700 1 -- 172.21.15.3:0/2196499073 shutdown_connections 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21fcd6c700 1 -- 172.21.15.3:0/2196499073 >> 172.21.15.3:0/2196499073 conn(0x7f21f8066fb0 msgr2=0x7f21f80673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21fcd6c700 1 -- 172.21.15.3:0/2196499073 shutdown_connections 2023-10-06T04:52:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.802+0000 7f21fcd6c700 1 -- 172.21.15.3:0/2196499073 wait complete. 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.803+0000 7f21fcd6c700 1 Processor -- start 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21fcd6c700 1 -- start start 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21fcd6c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21e4004d50 0x7f21f815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21fcd6c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f8151960 0x7f21f815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21fcd6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21f8154520 0x7f21f8143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21fcd6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80c8a50 con 0x7f21f8154520 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21fcd6c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f815b590 con 0x7f21f8151960 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21fcd6c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f815b870 con 0x7f21e4004d50 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21f6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21f8154520 0x7f21f8143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21f6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21f8154520 0x7f21f8143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51334/0 (socket says 172.21.15.3:51334) 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21f659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21e4004d50 0x7f21f815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21f6d9d700 1 -- 172.21.15.3:0/475935809 learned_addr learned my addr 172.21.15.3:0/475935809 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.804+0000 7f21f5d9b700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f8151960 0x7f21f815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21f6d9d700 1 -- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21e4004d50 msgr2=0x7f21f815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21f6d9d700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21e4004d50 0x7f21f815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21f6d9d700 1 -- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f8151960 msgr2=0x7f21f815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21f6d9d700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f8151960 0x7f21f815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21f6d9d700 1 -- 172.21.15.3:0/475935809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f815bb50 con 0x7f21f8154520 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21f6d9d700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21f8154520 0x7f21f8143ef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21f0001ed0 tx=0x7f21f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21ee7fc700 1 -- 172.21.15.3:0/475935809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0004a40 con 0x7f21f8154520 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21ee7fc700 1 -- 172.21.15.3:0/475935809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f0004be0 con 0x7f21f8154520 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21ee7fc700 1 -- 172.21.15.3:0/475935809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00070d0 con 0x7f21f8154520 2023-10-06T04:52:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f815be30 con 0x7f21f8154520 2023-10-06T04:52:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.805+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f81549c0 con 0x7f21f8154520 2023-10-06T04:52:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.806+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f8154c10 con 0x7f21f8154520 2023-10-06T04:52:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.806+0000 7f21ee7fc700 1 -- 172.21.15.3:0/475935809 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f21f0019070 con 0x7f21f8154520 2023-10-06T04:52:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.807+0000 7f21ee7fc700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f21d8066f30 0x7f21d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.807+0000 7f21ee7fc700 1 -- 172.21.15.3:0/475935809 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f21f0084ca0 con 0x7f21f8154520 2023-10-06T04:52:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.807+0000 7f21f659c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f21d8066f30 0x7f21d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.807+0000 7f21f659c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f21d8066f30 0x7f21d80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21e80094d0 tx=0x7f21e800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:18.811+0000 7f21ee7fc700 1 -- 172.21.15.3:0/475935809 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21f0051db0 con 0x7f21f8154520 2023-10-06T04:52:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.005+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21f8071680 con 0x7f21f8154520 2023-10-06T04:52:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.006+0000 7f21ee7fc700 1 -- 172.21.15.3:0/475935809 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f21f0017020 con 0x7f21f8154520 2023-10-06T04:52:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:19.009 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":19,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:18.046201+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.008+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f21d8066f30 msgr2=0x7f21d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.008+0000 7f21fcd6c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f21d8066f30 0x7f21d80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21e80094d0 tx=0x7f21e800b040 comp rx=0 tx=0).stop 2023-10-06T04:52:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.008+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21f8154520 msgr2=0x7f21f8143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.008+0000 7f21fcd6c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21f8154520 0x7f21f8143ef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21f0001ed0 tx=0x7f21f0015000 comp rx=0 tx=0).stop 2023-10-06T04:52:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 shutdown_connections 2023-10-06T04:52:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f21d8066f30 0x7f21d80693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21f8154520 0x7f21f8143ef0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21e4004d50 0x7f21f815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 --2- 172.21.15.3:0/475935809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f8151960 0x7f21f815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 >> 172.21.15.3:0/475935809 conn(0x7f21f8066fb0 msgr2=0x7f21f80b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:19.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 shutdown_connections 2023-10-06T04:52:19.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.009+0000 7f21fcd6c700 1 -- 172.21.15.3:0/475935809 wait complete. 2023-10-06T04:52:19.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:19.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.147+0000 7fe529d25700 1 Processor -- start 2023-10-06T04:52:19.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.147+0000 7fe529d25700 1 -- start start 2023-10-06T04:52:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe529d25700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe52406d480 0x7fe524067a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe529d25700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe524067f90 0x7fe5240b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe529d25700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52415a9b0 con 0x7fe524142de0 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe529d25700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5241431b0 con 0x7fe52406d480 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe529d25700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe524068390 con 0x7fe524067f90 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe523fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe52406d480 0x7fe524067a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe523fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe52406d480 0x7fe524067a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51338/0 (socket says 172.21.15.3:51338) 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe523fff700 1 -- 172.21.15.3:0/560544595 learned_addr learned my addr 172.21.15.3:0/560544595 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.148+0000 7fe529524700 1 --2- 172.21.15.3:0/560544595 >> v2:172.21.15.153:3300/0 conn(0x7fe524067f90 0x7fe5240b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe523fff700 1 -- 172.21.15.3:0/560544595 >> v2:172.21.15.153:3300/0 conn(0x7fe524067f90 msgr2=0x7fe5240b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe523fff700 1 --2- 172.21.15.3:0/560544595 >> v2:172.21.15.153:3300/0 conn(0x7fe524067f90 0x7fe5240b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe523fff700 1 -- 172.21.15.3:0/560544595 >> v1:172.21.15.153:6789/0 conn(0x7fe524142de0 legacy=0x7fe5240b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe523fff700 1 -- 172.21.15.3:0/560544595 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524159ae0 con 0x7fe52406d480 2023-10-06T04:52:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe523fff700 1 --2- 172.21.15.3:0/560544595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524067a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5240ad7f0 tx=0x7fe50c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48378ddbcda7a49f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:19.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe5237fe700 1 -- 172.21.15.3:0/560544595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00b820 con 0x7fe52406d480 2023-10-06T04:52:19.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe5237fe700 1 -- 172.21.15.3:0/560544595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c00b980 con 0x7fe52406d480 2023-10-06T04:52:19.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe5237fe700 1 -- 172.21.15.3:0/560544595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00bd20 con 0x7fe52406d480 2023-10-06T04:52:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe529d25700 1 -- 172.21.15.3:0/560544595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 msgr2=0x7fe524067a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe529d25700 1 --2- 172.21.15.3:0/560544595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524067a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5240ad7f0 tx=0x7fe50c009250 comp rx=0 tx=0).stop 2023-10-06T04:52:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe529d25700 1 -- 172.21.15.3:0/560544595 shutdown_connections 2023-10-06T04:52:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe529d25700 1 --2- 172.21.15.3:0/560544595 >> v2:172.21.15.153:3300/0 conn(0x7fe524067f90 0x7fe5240b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe529d25700 1 --2- 172.21.15.3:0/560544595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524067a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe529d25700 1 -- 172.21.15.3:0/560544595 >> 172.21.15.3:0/560544595 conn(0x7fe52406ce40 msgr2=0x7fe5240afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.149+0000 7fe529d25700 1 -- 172.21.15.3:0/560544595 shutdown_connections 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.150+0000 7fe529d25700 1 -- 172.21.15.3:0/560544595 wait complete. 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 Processor -- start 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 -- start start 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524067f90 0x7fe5240c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524131b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe524142de0 0x7fe5240cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240c9190 con 0x7fe52406d480 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe524132050 con 0x7fe524067f90 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.151+0000 7fe529d25700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240c8580 con 0x7fe524142de0 2023-10-06T04:52:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524131b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524131b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51348/0 (socket says 172.21.15.3:51348) 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 -- 172.21.15.3:0/3862139320 learned_addr learned my addr 172.21.15.3:0/3862139320 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe529524700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe524142de0 0x7fe5240cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe528d23700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524067f90 0x7fe5240c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 -- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe524142de0 msgr2=0x7fe5240cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe524142de0 0x7fe5240cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 -- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524067f90 msgr2=0x7fe5240c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524067f90 0x7fe5240c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 -- 172.21.15.3:0/3862139320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0094f0 con 0x7fe52406d480 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe523fff700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524131b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0030c0 tx=0x7fe50c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe520ff9700 1 -- 172.21.15.3:0/3862139320 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c011d00 con 0x7fe52406d480 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe520ff9700 1 -- 172.21.15.3:0/3862139320 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c020070 con 0x7fe52406d480 2023-10-06T04:52:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5240c8860 con 0x7fe52406d480 2023-10-06T04:52:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe520ff9700 1 -- 172.21.15.3:0/3862139320 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c01b6a0 con 0x7fe52406d480 2023-10-06T04:52:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.152+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5240c8e20 con 0x7fe52406d480 2023-10-06T04:52:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.153+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5240c9300 con 0x7fe52406d480 2023-10-06T04:52:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.154+0000 7fe520ff9700 1 -- 172.21.15.3:0/3862139320 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe50c022030 con 0x7fe52406d480 2023-10-06T04:52:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.154+0000 7fe520ff9700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.154+0000 7fe520ff9700 1 -- 172.21.15.3:0/3862139320 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe50c086540 con 0x7fe52406d480 2023-10-06T04:52:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.154+0000 7fe528d23700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.155+0000 7fe528d23700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe514003a30 tx=0x7fe514007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.159+0000 7fe520ff9700 1 -- 172.21.15.3:0/3862139320 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe50c053650 con 0x7fe52406d480 2023-10-06T04:52:19.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.353+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe52404c380 con 0x7fe52406d480 2023-10-06T04:52:19.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.354+0000 7fe520ff9700 1 -- 172.21.15.3:0/3862139320 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16831 (secure 0 0 0) 0x7fe50c011e60 con 0x7fe52406d480 2023-10-06T04:52:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:19.356 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":19,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:18.046201+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:19.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 msgr2=0x7fe5040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe514003a30 tx=0x7fe514007ea0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 msgr2=0x7fe524131b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524131b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0030c0 tx=0x7fe50c016000 comp rx=0 tx=0).stop 2023-10-06T04:52:19.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 shutdown_connections 2023-10-06T04:52:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524067f90 0x7fe5240c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe524142de0 0x7fe5240cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 --2- 172.21.15.3:0/3862139320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe52406d480 0x7fe524131b10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 >> 172.21.15.3:0/3862139320 conn(0x7fe52406ce40 msgr2=0x7fe52415d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 shutdown_connections 2023-10-06T04:52:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.356+0000 7fe529d25700 1 -- 172.21.15.3:0/3862139320 wait complete. 2023-10-06T04:52:19.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:52:19.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.491+0000 7f30b72b5700 1 Processor -- start 2023-10-06T04:52:19.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.491+0000 7f30b72b5700 1 -- start start 2023-10-06T04:52:19.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.492+0000 7f30b72b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b00cbab0 0x7f30b0161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.492+0000 7f30b72b5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b0067ea0 con 0x7f30b0068330 2023-10-06T04:52:19.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.492+0000 7f30b72b5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b006dca0 con 0x7f30b01571c0 2023-10-06T04:52:19.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.492+0000 7f30b72b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b0158760 con 0x7f30b00cbab0 2023-10-06T04:52:19.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.492+0000 7f30b5ab2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f30b0068330 0x7f30b00cb3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44668/0 (socket says 172.21.15.3:44668) 2023-10-06T04:52:19.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.492+0000 7f30b5ab2700 1 -- 172.21.15.3:0/3680452856 learned_addr learned my addr 172.21.15.3:0/3680452856 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:19.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.492+0000 7f30b6ab4700 1 --2- 172.21.15.3:0/3680452856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b00cbab0 0x7f30b0161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.493+0000 7f30b6ab4700 1 -- 172.21.15.3:0/3680452856 >> v1:172.21.15.153:6789/0 conn(0x7f30b01571c0 legacy=0x7f30b006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.493+0000 7f30b6ab4700 1 -- 172.21.15.3:0/3680452856 >> v1:172.21.15.3:6789/0 conn(0x7f30b0068330 legacy=0x7f30b00cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.493+0000 7f30b6ab4700 1 -- 172.21.15.3:0/3680452856 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b0066fb0 con 0x7f30b00cbab0 2023-10-06T04:52:19.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.493+0000 7f30b6ab4700 1 --2- 172.21.15.3:0/3680452856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b00cbab0 0x7f30b0161df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009e90 tx=0x7f30a8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d05688b300fa82ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:19.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.494+0000 7f30b52b1700 1 -- 172.21.15.3:0/3680452856 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a800e070 con 0x7f30b00cbab0 2023-10-06T04:52:19.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.494+0000 7f30b52b1700 1 -- 172.21.15.3:0/3680452856 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a800bc30 con 0x7f30b00cbab0 2023-10-06T04:52:19.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.494+0000 7f30b52b1700 1 -- 172.21.15.3:0/3680452856 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8012430 con 0x7f30b00cbab0 2023-10-06T04:52:19.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.495+0000 7f30b72b5700 1 -- 172.21.15.3:0/3680452856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b00cbab0 msgr2=0x7f30b0161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.495+0000 7f30b72b5700 1 --2- 172.21.15.3:0/3680452856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b00cbab0 0x7f30b0161df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009e90 tx=0x7f30a8009780 comp rx=0 tx=0).stop 2023-10-06T04:52:19.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.495+0000 7f30b72b5700 1 -- 172.21.15.3:0/3680452856 shutdown_connections 2023-10-06T04:52:19.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.495+0000 7f30b72b5700 1 --2- 172.21.15.3:0/3680452856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b00cbab0 0x7f30b0161df0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.495+0000 7f30b72b5700 1 -- 172.21.15.3:0/3680452856 >> 172.21.15.3:0/3680452856 conn(0x7f30b00cd1f0 msgr2=0x7f30b00b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:19.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.495+0000 7f30b72b5700 1 -- 172.21.15.3:0/3680452856 shutdown_connections 2023-10-06T04:52:19.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.495+0000 7f30b72b5700 1 -- 172.21.15.3:0/3680452856 wait complete. 2023-10-06T04:52:19.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.496+0000 7f30b72b5700 1 Processor -- start 2023-10-06T04:52:19.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.496+0000 7f30b72b5700 1 -- start start 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b72b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30b0068330 0x7f30b0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b72b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30b00cbab0 0x7f30b0141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b72b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b01571c0 0x7f30b015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b72b5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b0152260 con 0x7f30b00cbab0 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b72b5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01688a0 con 0x7f30b0068330 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b72b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01518b0 con 0x7f30b01571c0 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30b00cbab0 0x7f30b0141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30b00cbab0 0x7f30b0141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51360/0 (socket says 172.21.15.3:51360) 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 -- 172.21.15.3:0/1653243916 learned_addr learned my addr 172.21.15.3:0/1653243916 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b62b3700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30b0068330 0x7f30b0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b6ab4700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b01571c0 0x7f30b015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 -- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b01571c0 msgr2=0x7f30b015acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b01571c0 0x7f30b015acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 -- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30b0068330 msgr2=0x7f30b0150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30b0068330 0x7f30b0150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.497+0000 7f30b5ab2700 1 -- 172.21.15.3:0/1653243916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a8009a20 con 0x7f30b00cbab0 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.498+0000 7f30b5ab2700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30b00cbab0 0x7f30b0141430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30a000c790 tx=0x7f30a0007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.498+0000 7f30a67fc700 1 -- 172.21.15.3:0/1653243916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0005b30 con 0x7f30b00cbab0 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.498+0000 7f30a67fc700 1 -- 172.21.15.3:0/1653243916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0005cb0 con 0x7f30b00cbab0 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.498+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b0151b90 con 0x7f30b00cbab0 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.498+0000 7f30a67fc700 1 -- 172.21.15.3:0/1653243916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0004a80 con 0x7f30b00cbab0 2023-10-06T04:52:19.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.498+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b0168100 con 0x7f30b00cbab0 2023-10-06T04:52:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.499+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b01523d0 con 0x7f30b00cbab0 2023-10-06T04:52:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.499+0000 7f30a67fc700 1 -- 172.21.15.3:0/1653243916 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30a00070d0 con 0x7f30b00cbab0 2023-10-06T04:52:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.499+0000 7f30a67fc700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3094066f30 0x7f30940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.500+0000 7f30a67fc700 1 -- 172.21.15.3:0/1653243916 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f30a0083980 con 0x7f30b00cbab0 2023-10-06T04:52:19.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.500+0000 7f30b62b3700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3094066f30 0x7f30940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:19.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.500+0000 7f30b62b3700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3094066f30 0x7f30940693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30ac003a10 tx=0x7f30ac007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:19.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.504+0000 7f30a67fc700 1 -- 172.21.15.3:0/1653243916 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30a004b5a0 con 0x7f30b00cbab0 2023-10-06T04:52:19.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:19.745+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f30b004c380 con 0x7f30b00cbab0 2023-10-06T04:52:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.662+0000 7f30a67fc700 1 -- 172.21.15.3:0/1653243916 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f30a0050a90 con 0x7f30b00cbab0 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3094066f30 msgr2=0x7f30940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3094066f30 0x7f30940693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30ac003a10 tx=0x7f30ac007ec0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30b00cbab0 msgr2=0x7f30b0141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30b00cbab0 0x7f30b0141430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30a000c790 tx=0x7f30a0007f40 comp rx=0 tx=0).stop 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 shutdown_connections 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3094066f30 0x7f30940693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30b0068330 0x7f30b0150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30b01571c0 0x7f30b015acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 --2- 172.21.15.3:0/1653243916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30b00cbab0 0x7f30b0141430 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 >> 172.21.15.3:0/1653243916 conn(0x7f30b00cd1f0 msgr2=0x7f30b0154bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:20.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 shutdown_connections 2023-10-06T04:52:20.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.665+0000 7f30b72b5700 1 -- 172.21.15.3:0/1653243916 wait complete. 2023-10-06T04:52:20.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:52:20.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.808+0000 7fd21143c700 1 Processor -- start 2023-10-06T04:52:20.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.809+0000 7fd21143c700 1 -- start start 2023-10-06T04:52:20.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.809+0000 7fd21143c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd20c0cba50 0x7fd20c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.809+0000 7fd21143c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd20c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.809+0000 7fd21143c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20c15a290 con 0x7fd20c150690 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.809+0000 7fd21143c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20c0d0020 con 0x7fd20c0cba50 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.809+0000 7fd21143c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c159f40 con 0x7fd20c15d540 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.809+0000 7fd20affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd20c150690 0x7fd20c15a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44674/0 (socket says 172.21.15.3:44674) 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd20affd700 1 -- 172.21.15.3:0/4203892492 learned_addr learned my addr 172.21.15.3:0/4203892492 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309709763 0 0) 0x7fd20c15a290 con 0x7fd20c150690 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f8003570 con 0x7fd20c150690 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd20b7fe700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd20c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd20a7fc700 1 --2- 172.21.15.3:0/4203892492 >> v2:172.21.15.153:3300/0 conn(0x7fd20c0cba50 0x7fd20c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4199485044 0 0) 0x7fd1f8003570 con 0x7fd20c150690 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 msgr2=0x7fd20c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd20c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 >> v2:172.21.15.153:3300/0 conn(0x7fd20c0cba50 msgr2=0x7fd20c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 --2- 172.21.15.3:0/4203892492 >> v2:172.21.15.153:3300/0 conn(0x7fd20c0cba50 0x7fd20c150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c15dcb0 con 0x7fd20c150690 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fd1fc001470 con 0x7fd20c150690 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd20b7fe700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd20c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:20.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 >> v1:172.21.15.3:6789/0 conn(0x7fd20c150690 legacy=0x7fd20c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8003a60 0x7fd1f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f8004810 0x7fd1f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd1f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c0d0020 con 0x7fd1f8004810 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c159f40 con 0x7fd1f8003a60 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f800ba90 con 0x7fd20c15d540 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd20a7fc700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f8004810 0x7fd1f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd20affd700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8003a60 0x7fd1f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.810+0000 7fd20b7fe700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd1f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20a7fc700 1 -- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 msgr2=0x7fd1f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20a7fc700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd1f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20a7fc700 1 -- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8003a60 msgr2=0x7fd1f8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20a7fc700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8003a60 0x7fd1f8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20a7fc700 1 -- 172.21.15.3:0/4203892492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8003570 con 0x7fd1f8004810 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20b7fe700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c15d540 0x7fd1f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20a7fc700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f8004810 0x7fd1f8004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009c00 tx=0x7fd1f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50f753f1f7ade4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400e710 con 0x7fd1f8004810 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f400e870 con 0x7fd1f8004810 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd209ffb700 1 -- 172.21.15.3:0/4203892492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400ec30 con 0x7fd1f8004810 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd21143c700 1 -- 172.21.15.3:0/4203892492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f8004810 msgr2=0x7fd1f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd21143c700 1 --2- 172.21.15.3:0/4203892492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f8004810 0x7fd1f8004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009c00 tx=0x7fd1f40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd20b7fe700 1 -- 172.21.15.3:0/4203892492 reap_dead start 2023-10-06T04:52:20.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd21143c700 1 -- 172.21.15.3:0/4203892492 shutdown_connections 2023-10-06T04:52:20.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.811+0000 7fd21143c700 1 -- 172.21.15.3:0/4203892492 >> 172.21.15.3:0/4203892492 conn(0x7fd20c070310 msgr2=0x7fd20c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:20.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.812+0000 7fd21143c700 1 -- 172.21.15.3:0/4203892492 shutdown_connections 2023-10-06T04:52:20.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.812+0000 7fd21143c700 1 -- 172.21.15.3:0/4203892492 wait complete. 2023-10-06T04:52:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.813+0000 7fd21143c700 1 Processor -- start 2023-10-06T04:52:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.813+0000 7fd21143c700 1 -- start start 2023-10-06T04:52:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8004810 0x7fd20c143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c0c6360 0x7fd20c0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd20c0b4b70 0x7fd20c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c0c6d10 con 0x7fd20c0b4b70 2023-10-06T04:52:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c169d90 con 0x7fd1f8004810 2023-10-06T04:52:20.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c0b58b0 con 0x7fd20c0c6360 2023-10-06T04:52:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd20c0b4b70 0x7fd20c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd20c0b4b70 0x7fd20c0b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51388/0 (socket says 172.21.15.3:51388) 2023-10-06T04:52:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 -- 172.21.15.3:0/3741556439 learned_addr learned my addr 172.21.15.3:0/3741556439 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20affd700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8004810 0x7fd20c143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20a7fc700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c0c6360 0x7fd20c0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 -- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c0c6360 msgr2=0x7fd20c0c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c0c6360 0x7fd20c0c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 -- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8004810 msgr2=0x7fd20c143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8004810 0x7fd20c143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 -- 172.21.15.3:0/3741556439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c1514c0 con 0x7fd20c0b4b70 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20affd700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8004810 0x7fd20c143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd20b7fe700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd20c0b4b70 0x7fd20c0b4fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0b2f80 tx=0x7fd204009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd202ffd700 1 -- 172.21.15.3:0/3741556439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204002ce0 con 0x7fd20c0b4b70 2023-10-06T04:52:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd202ffd700 1 -- 172.21.15.3:0/3741556439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20400f050 con 0x7fd20c0b4b70 2023-10-06T04:52:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20c150b90 con 0x7fd20c0b4b70 2023-10-06T04:52:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd202ffd700 1 -- 172.21.15.3:0/3741556439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204014660 con 0x7fd20c0b4b70 2023-10-06T04:52:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.814+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20c151150 con 0x7fd20c0b4b70 2023-10-06T04:52:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.815+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20c04c380 con 0x7fd20c0b4b70 2023-10-06T04:52:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.816+0000 7fd202ffd700 1 -- 172.21.15.3:0/3741556439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd204003040 con 0x7fd20c0b4b70 2023-10-06T04:52:20.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.816+0000 7fd202ffd700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ec066f30 0x7fd1ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:20.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.816+0000 7fd202ffd700 1 -- 172.21.15.3:0/3741556439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd204084920 con 0x7fd20c0b4b70 2023-10-06T04:52:20.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.816+0000 7fd20affd700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ec066f30 0x7fd1ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:20.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.817+0000 7fd20affd700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ec066f30 0x7fd1ec0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd20c16d310 tx=0x7fd1fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:20.821+0000 7fd202ffd700 1 -- 172.21.15.3:0/3741556439 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd204051a30 con 0x7fd20c0b4b70 2023-10-06T04:52:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.060+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd20c15df90 con 0x7fd20c0b4b70 2023-10-06T04:52:21.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.664+0000 7fd202ffd700 1 -- 172.21.15.3:0/3741556439 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd204055050 con 0x7fd20c0b4b70 2023-10-06T04:52:21.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ec066f30 msgr2=0x7fd1ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:21.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ec066f30 0x7fd1ec0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd20c16d310 tx=0x7fd1fc009040 comp rx=0 tx=0).stop 2023-10-06T04:52:21.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd20c0b4b70 msgr2=0x7fd20c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:21.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd20c0b4b70 0x7fd20c0b4fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0b2f80 tx=0x7fd204009870 comp rx=0 tx=0).stop 2023-10-06T04:52:21.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 shutdown_connections 2023-10-06T04:52:21.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ec066f30 0x7fd1ec0693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1f8004810 0x7fd20c143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd20c0b4b70 0x7fd20c0b4fc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 --2- 172.21.15.3:0/3741556439 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd20c0c6360 0x7fd20c0c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.667+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 >> 172.21.15.3:0/3741556439 conn(0x7fd20c070310 msgr2=0x7fd20c15c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:21.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.668+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 shutdown_connections 2023-10-06T04:52:21.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.668+0000 7fd21143c700 1 -- 172.21.15.3:0/3741556439 wait complete. 2023-10-06T04:52:21.684 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.809+0000 7f5408876700 1 Processor -- start 2023-10-06T04:52:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.809+0000 7f5408876700 1 -- start start 2023-10-06T04:52:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f5408876700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040b1a50 0x7f540406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f5408876700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5404151b20 0x7f5404157920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f5408876700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540406dae0 con 0x7f54040aedc0 2023-10-06T04:52:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f5408876700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54040cbcd0 con 0x7f5404151b20 2023-10-06T04:52:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f5408876700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540406d800 con 0x7f54040b1a50 2023-10-06T04:52:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f5402d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5404151b20 0x7f5404157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f540259c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040b1a50 0x7f540406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f540259c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040b1a50 0x7f540406ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:37914/0 (socket says 172.21.15.3:37914) 2023-10-06T04:52:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f5402d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5404151b20 0x7f5404157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:55422/0 (socket says 172.21.15.3:55422) 2023-10-06T04:52:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.810+0000 7f540259c700 1 -- 172.21.15.3:0/2117931732 learned_addr learned my addr 172.21.15.3:0/2117931732 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f5402d9d700 1 -- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040b1a50 msgr2=0x7f540406ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f5402d9d700 1 --2- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040b1a50 0x7f540406ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f5402d9d700 1 -- 172.21.15.3:0/2117931732 >> v1:172.21.15.153:6789/0 conn(0x7f54040aedc0 legacy=0x7f5404151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f5402d9d700 1 -- 172.21.15.3:0/2117931732 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040cb680 con 0x7f5404151b20 2023-10-06T04:52:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f540159a700 1 -- 172.21.15.3:0/2117931732 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218450798 0 0) 0x7f540406dae0 con 0x7f54040aedc0 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f540259c700 1 --2- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040b1a50 0x7f540406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f5402d9d700 1 --2- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 0x7f5404157920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009e60 tx=0x7f53fc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b625baf57596737e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f540159a700 1 -- 172.21.15.3:0/2117931732 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00e070 con 0x7f5404151b20 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f540159a700 1 -- 172.21.15.3:0/2117931732 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc00bb60 con 0x7f5404151b20 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f540159a700 1 -- 172.21.15.3:0/2117931732 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc012430 con 0x7f5404151b20 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.811+0000 7f5408876700 1 -- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 msgr2=0x7f5404157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.812+0000 7f5408876700 1 --2- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 0x7f5404157920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009e60 tx=0x7f53fc009780 comp rx=0 tx=0).stop 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.812+0000 7f5408876700 1 -- 172.21.15.3:0/2117931732 shutdown_connections 2023-10-06T04:52:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.812+0000 7f5408876700 1 --2- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040b1a50 0x7f540406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.812+0000 7f5408876700 1 --2- 172.21.15.3:0/2117931732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 0x7f5404157920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.812+0000 7f5408876700 1 -- 172.21.15.3:0/2117931732 >> 172.21.15.3:0/2117931732 conn(0x7f5404066fb0 msgr2=0x7f54040673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:21.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.812+0000 7f5408876700 1 -- 172.21.15.3:0/2117931732 shutdown_connections 2023-10-06T04:52:21.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.812+0000 7f5408876700 1 -- 172.21.15.3:0/2117931732 wait complete. 2023-10-06T04:52:21.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.813+0000 7f5408876700 1 Processor -- start 2023-10-06T04:52:21.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.813+0000 7f5408876700 1 -- start start 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.813+0000 7f5408876700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040aedc0 0x7f54040ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.813+0000 7f5408876700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54040b1a50 0x7f5404146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5408876700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 0x7f540414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5408876700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54040ca2c0 con 0x7f54040b1a50 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5408876700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54040cb370 con 0x7f5404151b20 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5408876700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54040c96b0 con 0x7f54040aedc0 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54040b1a50 0x7f5404146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54040b1a50 0x7f5404146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51396/0 (socket says 172.21.15.3:51396) 2023-10-06T04:52:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 -- 172.21.15.3:0/3126108077 learned_addr learned my addr 172.21.15.3:0/3126108077 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f540259c700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040aedc0 0x7f54040ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5402d9d700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 0x7f540414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 -- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040aedc0 msgr2=0x7f54040ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040aedc0 0x7f54040ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 -- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 msgr2=0x7f540414acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 0x7f540414acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 -- 172.21.15.3:0/3126108077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40094f0 con 0x7f54040b1a50 2023-10-06T04:52:21.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5401d9b700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54040b1a50 0x7f5404146630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f800c7e0 tx=0x7f53f8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:21.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f53f27fc700 1 -- 172.21.15.3:0/3126108077 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8011590 con 0x7f54040b1a50 2023-10-06T04:52:21.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f53f27fc700 1 -- 172.21.15.3:0/3126108077 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f8011710 con 0x7f54040b1a50 2023-10-06T04:52:21.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f53f27fc700 1 -- 172.21.15.3:0/3126108077 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8011ad0 con 0x7f54040b1a50 2023-10-06T04:52:21.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc009a20 con 0x7f54040b1a50 2023-10-06T04:52:21.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.814+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54040c9c70 con 0x7f54040b1a50 2023-10-06T04:52:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.815+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5404071680 con 0x7f54040b1a50 2023-10-06T04:52:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.816+0000 7f53f27fc700 1 -- 172.21.15.3:0/3126108077 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53f801b3b0 con 0x7f54040b1a50 2023-10-06T04:52:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.816+0000 7f53f27fc700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f53e4066f30 0x7f53e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.816+0000 7f53f27fc700 1 -- 172.21.15.3:0/3126108077 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f53f8083b40 con 0x7f54040b1a50 2023-10-06T04:52:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.816+0000 7f540259c700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f53e4066f30 0x7f53e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.817+0000 7f540259c700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f53e4066f30 0x7f53e40693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006d20 tx=0x7f53f4009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:21.821+0000 7f53f27fc700 1 -- 172.21.15.3:0/3126108077 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53f8050c50 con 0x7f54040b1a50 2023-10-06T04:52:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.056+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54040cb960 con 0x7f54040b1a50 2023-10-06T04:52:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.057+0000 7f53f27fc700 1 -- 172.21.15.3:0/3126108077 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7f53f8017070 con 0x7f54040b1a50 2023-10-06T04:52:22.059 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:22.059 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f53e4066f30 msgr2=0x7f53e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f53e4066f30 0x7f53e40693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006d20 tx=0x7f53f4009250 comp rx=0 tx=0).stop 2023-10-06T04:52:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54040b1a50 msgr2=0x7f5404146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54040b1a50 0x7f5404146630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f800c7e0 tx=0x7f53f8007dc0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 shutdown_connections 2023-10-06T04:52:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f53e4066f30 0x7f53e40693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5404151b20 0x7f540414acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54040aedc0 0x7f54040ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 --2- 172.21.15.3:0/3126108077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54040b1a50 0x7f5404146630 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 >> 172.21.15.3:0/3126108077 conn(0x7f5404066fb0 msgr2=0x7f5404067590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 shutdown_connections 2023-10-06T04:52:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.059+0000 7f5408876700 1 -- 172.21.15.3:0/3126108077 wait complete. 2023-10-06T04:52:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:22.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.190+0000 7fec89c9f700 1 Processor -- start 2023-10-06T04:52:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.190+0000 7fec89c9f700 1 -- start start 2023-10-06T04:52:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec89c9f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec840cb3b0 0x7fec840cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec89c9f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fec840cb780 0x7fec84158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec89c9f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec840cf130 con 0x7fec84159500 2023-10-06T04:52:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec89c9f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840ce520 con 0x7fec840cb3b0 2023-10-06T04:52:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec89c9f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840ce800 con 0x7fec840cb780 2023-10-06T04:52:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec88c9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec840cb3b0 0x7fec840cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec8949e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fec84159500 0x7fec840cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44684/0 (socket says 172.21.15.3:44684) 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec8949e700 1 -- 172.21.15.3:0/961538874 learned_addr learned my addr 172.21.15.3:0/961538874 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec83fff700 1 --2- 172.21.15.3:0/961538874 >> v2:172.21.15.153:3300/0 conn(0x7fec840cb780 0x7fec84158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec88c9d700 1 -- 172.21.15.3:0/961538874 >> v2:172.21.15.153:3300/0 conn(0x7fec840cb780 msgr2=0x7fec84158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec88c9d700 1 --2- 172.21.15.3:0/961538874 >> v2:172.21.15.153:3300/0 conn(0x7fec840cb780 0x7fec84158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec88c9d700 1 -- 172.21.15.3:0/961538874 >> v1:172.21.15.3:6789/0 conn(0x7fec84159500 legacy=0x7fec840cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec88c9d700 1 -- 172.21.15.3:0/961538874 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840ceae0 con 0x7fec840cb3b0 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec837fe700 1 -- 172.21.15.3:0/961538874 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122052702 0 0) 0x7fec840cf130 con 0x7fec84159500 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec83fff700 1 --2- 172.21.15.3:0/961538874 >> v2:172.21.15.153:3300/0 conn(0x7fec840cb780 0x7fec84158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.191+0000 7fec88c9d700 1 --2- 172.21.15.3:0/961538874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb3b0 0x7fec840cf5c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec840892b0 tx=0x7fec780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b9dd2323e6a34e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec837fe700 1 -- 172.21.15.3:0/961538874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800b6b0 con 0x7fec840cb3b0 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec837fe700 1 -- 172.21.15.3:0/961538874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7800b810 con 0x7fec840cb3b0 2023-10-06T04:52:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec837fe700 1 -- 172.21.15.3:0/961538874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800bbb0 con 0x7fec840cb3b0 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 -- 172.21.15.3:0/961538874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb3b0 msgr2=0x7fec840cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 --2- 172.21.15.3:0/961538874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb3b0 0x7fec840cf5c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec840892b0 tx=0x7fec780091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 -- 172.21.15.3:0/961538874 shutdown_connections 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 --2- 172.21.15.3:0/961538874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb3b0 0x7fec840cf5c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 --2- 172.21.15.3:0/961538874 >> v2:172.21.15.153:3300/0 conn(0x7fec840cb780 0x7fec84158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 -- 172.21.15.3:0/961538874 >> 172.21.15.3:0/961538874 conn(0x7fec84070310 msgr2=0x7fec84070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 -- 172.21.15.3:0/961538874 shutdown_connections 2023-10-06T04:52:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.192+0000 7fec89c9f700 1 -- 172.21.15.3:0/961538874 wait complete. 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.193+0000 7fec89c9f700 1 Processor -- start 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.193+0000 7fec89c9f700 1 -- start start 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec89c9f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec840cb3b0 0x7fec8414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec89c9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb780 0x7fec840b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec89c9f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec84159500 0x7fec840b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec89c9f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8415cca0 con 0x7fec840cb780 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec89c9f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8415bdb0 con 0x7fec84159500 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec89c9f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8415c090 con 0x7fec840cb3b0 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec83fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb780 0x7fec840b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec83fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb780 0x7fec840b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51410/0 (socket says 172.21.15.3:51410) 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec83fff700 1 -- 172.21.15.3:0/3220910642 learned_addr learned my addr 172.21.15.3:0/3220910642 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec8949e700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec84159500 0x7fec840b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.194+0000 7fec88c9d700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec840cb3b0 0x7fec8414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec83fff700 1 -- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec840cb3b0 msgr2=0x7fec8414bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec83fff700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec840cb3b0 0x7fec8414bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec83fff700 1 -- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec84159500 msgr2=0x7fec840b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec83fff700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec84159500 0x7fec840b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec83fff700 1 -- 172.21.15.3:0/3220910642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec78009420 con 0x7fec840cb780 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec8949e700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec84159500 0x7fec840b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec83fff700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb780 0x7fec840b4ec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c830 tx=0x7fec70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec80ff9700 1 -- 172.21.15.3:0/3220910642 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70011600 con 0x7fec840cb780 2023-10-06T04:52:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec80ff9700 1 -- 172.21.15.3:0/3220910642 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec70011760 con 0x7fec840cb780 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8415c370 con 0x7fec840cb780 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec80ff9700 1 -- 172.21.15.3:0/3220910642 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70011b20 con 0x7fec840cb780 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.195+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8415c930 con 0x7fec840cb780 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.196+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8415ce50 con 0x7fec840cb780 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.196+0000 7fec80ff9700 1 -- 172.21.15.3:0/3220910642 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec7000a3e0 con 0x7fec840cb780 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.197+0000 7fec80ff9700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.197+0000 7fec80ff9700 1 -- 172.21.15.3:0/3220910642 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec70084b20 con 0x7fec840cb780 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.197+0000 7fec88c9d700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.197+0000 7fec88c9d700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec68066f30 0x7fec680693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec780093f0 tx=0x7fec78017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.201+0000 7fec80ff9700 1 -- 172.21.15.3:0/3220910642 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec8415ce50 con 0x7fec840cb780 2023-10-06T04:52:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.395+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec8404c380 con 0x7fec840cb780 2023-10-06T04:52:22.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.396+0000 7fec80ff9700 1 -- 172.21.15.3:0/3220910642 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fec70015090 con 0x7fec840cb780 2023-10-06T04:52:22.397 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:22.398 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec68066f30 msgr2=0x7fec680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec68066f30 0x7fec680693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec780093f0 tx=0x7fec78017000 comp rx=0 tx=0).stop 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb780 msgr2=0x7fec840b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb780 0x7fec840b4ec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c830 tx=0x7fec70008040 comp rx=0 tx=0).stop 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 shutdown_connections 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec84159500 0x7fec840b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec840cb3b0 0x7fec8414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 --2- 172.21.15.3:0/3220910642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840cb780 0x7fec840b4ec0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 >> 172.21.15.3:0/3220910642 conn(0x7fec84070310 msgr2=0x7fec840b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:22.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 shutdown_connections 2023-10-06T04:52:22.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.398+0000 7fec89c9f700 1 -- 172.21.15.3:0/3220910642 wait complete. 2023-10-06T04:52:22.412 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:52:22.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:52:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.529+0000 7fa15999d700 1 Processor -- start 2023-10-06T04:52:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.529+0000 7fa15999d700 1 -- start start 2023-10-06T04:52:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa15999d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa1541596d0 0x7fa1540cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa15999d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1540cf130 con 0x7fa1540cb3b0 2023-10-06T04:52:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa15999d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1540ce520 con 0x7fa1540cb780 2023-10-06T04:52:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa15999d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540ce800 con 0x7fa1541596d0 2023-10-06T04:52:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa15899b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1540cb3b0 0x7fa1540cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35800/0 (socket says 172.21.15.3:35800) 2023-10-06T04:52:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa15899b700 1 -- 172.21.15.3:0/4065675777 learned_addr learned my addr 172.21.15.3:0/4065675777 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156156934 0 0) 0x7fa1540cf130 con 0x7fa1540cb3b0 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa13c003570 con 0x7fa1540cb3b0 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa15919c700 1 --2- 172.21.15.3:0/4065675777 >> v2:172.21.15.153:3300/0 conn(0x7fa1541596d0 0x7fa1540cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.530+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2525973985 0 0) 0x7fa13c003570 con 0x7fa1540cb3b0 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 >> v2:172.21.15.153:3300/0 conn(0x7fa1541596d0 msgr2=0x7fa1540cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 --2- 172.21.15.3:0/4065675777 >> v2:172.21.15.153:3300/0 conn(0x7fa1541596d0 0x7fa1540cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 >> v1:172.21.15.153:6789/0 conn(0x7fa1540cb780 legacy=0x7fa154158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540ceae0 con 0x7fa1540cb3b0 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fa14c0017f0 con 0x7fa1540cb3b0 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 >> v1:172.21.15.3:6789/0 conn(0x7fa1540cb3b0 legacy=0x7fa1540cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa13c003a60 0x7fa13c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c004810 0x7fa13c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c0051c0 0x7fa13c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540ce800 con 0x7fa13c004810 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c00c160 con 0x7fa13c0051c0 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c00c320 con 0x7fa13c003a60 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa153fff700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c004810 0x7fa13c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa15899b700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa13c003a60 0x7fa13c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa15919c700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c0051c0 0x7fa13c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa153fff700 1 -- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa13c003a60 msgr2=0x7fa13c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa153fff700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa13c003a60 0x7fa13c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa153fff700 1 -- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c0051c0 msgr2=0x7fa13c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa153fff700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c0051c0 0x7fa13c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa15919c700 1 -- 172.21.15.3:0/4065675777 reap_dead start 2023-10-06T04:52:22.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa153fff700 1 -- 172.21.15.3:0/4065675777 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c003570 con 0x7fa13c004810 2023-10-06T04:52:22.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa153fff700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c004810 0x7fa13c004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1480018b0 tx=0x7fa148007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57182c6b8bb93982 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148012420 con 0x7fa13c004810 2023-10-06T04:52:22.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1480125a0 con 0x7fa13c004810 2023-10-06T04:52:22.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.531+0000 7fa1537fe700 1 -- 172.21.15.3:0/4065675777 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148012960 con 0x7fa13c004810 2023-10-06T04:52:22.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.532+0000 7fa15999d700 1 -- 172.21.15.3:0/4065675777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c004810 msgr2=0x7fa13c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.532+0000 7fa15999d700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c004810 0x7fa13c004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1480018b0 tx=0x7fa148007d30 comp rx=0 tx=0).stop 2023-10-06T04:52:22.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.532+0000 7fa15999d700 1 -- 172.21.15.3:0/4065675777 shutdown_connections 2023-10-06T04:52:22.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.532+0000 7fa15999d700 1 --2- 172.21.15.3:0/4065675777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c004810 0x7fa13c004c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.532+0000 7fa15999d700 1 -- 172.21.15.3:0/4065675777 >> 172.21.15.3:0/4065675777 conn(0x7fa154070310 msgr2=0x7fa154070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:22.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.532+0000 7fa15999d700 1 -- 172.21.15.3:0/4065675777 shutdown_connections 2023-10-06T04:52:22.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.532+0000 7fa15999d700 1 -- 172.21.15.3:0/4065675777 wait complete. 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.533+0000 7fa15999d700 1 Processor -- start 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.533+0000 7fa15999d700 1 -- start start 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15999d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c004810 0x7fa1540c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15999d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1540b45e0 0x7fa15416aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15999d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1540c7d10 0x7fa15414fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15999d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15414b5e0 con 0x7fa1540b45e0 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15999d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540b4150 con 0x7fa13c004810 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15999d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540c8560 con 0x7fa1540c7d10 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1540b45e0 0x7fa15416aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1540b45e0 0x7fa15416aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32892/0 (socket says 172.21.15.3:32892) 2023-10-06T04:52:22.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 -- 172.21.15.3:0/2906325338 learned_addr learned my addr 172.21.15.3:0/2906325338 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15899b700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c004810 0x7fa1540c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa15919c700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1540c7d10 0x7fa15414fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 -- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1540c7d10 msgr2=0x7fa15414fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1540c7d10 0x7fa15414fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 -- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c004810 msgr2=0x7fa1540c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c004810 0x7fa1540c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 -- 172.21.15.3:0/2906325338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540b3860 con 0x7fa1540b45e0 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.534+0000 7fa153fff700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1540b45e0 0x7fa15416aaa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c3f0 tx=0x7fa148015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.535+0000 7fa150ff9700 1 -- 172.21.15.3:0/2906325338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14801f620 con 0x7fa1540b45e0 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.535+0000 7fa150ff9700 1 -- 172.21.15.3:0/2906325338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14801f780 con 0x7fa1540b45e0 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.535+0000 7fa150ff9700 1 -- 172.21.15.3:0/2906325338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14801fb20 con 0x7fa1540b45e0 2023-10-06T04:52:22.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.535+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1540b3b00 con 0x7fa1540b45e0 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.535+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1540c81b0 con 0x7fa1540b45e0 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.536+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15404c380 con 0x7fa1540b45e0 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.536+0000 7fa150ff9700 1 -- 172.21.15.3:0/2906325338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa14800c420 con 0x7fa1540b45e0 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.536+0000 7fa150ff9700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa140066f30 0x7fa1400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.537+0000 7fa150ff9700 1 -- 172.21.15.3:0/2906325338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa148085d80 con 0x7fa1540b45e0 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.537+0000 7fa15899b700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa140066f30 0x7fa1400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.537+0000 7fa15899b700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa140066f30 0x7fa1400693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa15415d080 tx=0x7fa14c006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.541+0000 7fa150ff9700 1 -- 172.21.15.3:0/2906325338 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa148052e90 con 0x7fa1540b45e0 2023-10-06T04:52:22.648 INFO:tasks.ceph.mds.d.smithi153.stderr:2023-10-06T04:52:22.646+0000 7ff5c55c9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:52:22.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.731+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa15414b5c0 con 0x7fa140066f30 2023-10-06T04:52:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.740+0000 7fa150ff9700 1 -- 172.21.15.3:0/2906325338 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286900 (secure 0 0 0) 0x7fa15414b5c0 con 0x7fa140066f30 2023-10-06T04:52:22.746 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:22.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa140066f30 msgr2=0x7fa1400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa140066f30 0x7fa1400693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa15415d080 tx=0x7fa14c006040 comp rx=0 tx=0).stop 2023-10-06T04:52:22.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1540b45e0 msgr2=0x7fa15416aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1540b45e0 0x7fa15416aaa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c3f0 tx=0x7fa148015000 comp rx=0 tx=0).stop 2023-10-06T04:52:22.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 shutdown_connections 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa140066f30 0x7fa1400693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c004810 0x7fa1540c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1540c7d10 0x7fa15414fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 --2- 172.21.15.3:0/2906325338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1540b45e0 0x7fa15416aaa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.743+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 >> 172.21.15.3:0/2906325338 conn(0x7fa154070310 msgr2=0x7fa1541527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.744+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 shutdown_connections 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.744+0000 7fa15999d700 1 -- 172.21.15.3:0/2906325338 wait complete. 2023-10-06T04:52:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:52:22.758 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-06T04:52:21.423787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381056,"kb_used_data":2752,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593664,"statfs":{"total":773094113280,"available":770655911936,"internally_reserved":0,"allocated":2818048,"data_stored":1741556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.797012"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658025+0000","last_change":"2023-10-06T04:52:18.658025+0000","last_active":"2023-10-06T04:52:18.658025+0000","last_peered":"2023-10-06T04:52:18.658025+0000","last_clean":"2023-10-06T04:52:18.658025+0000","last_became_active":"2023-10-06T04:52:18.657724+0000","last_became_peered":"2023-10-06T04:52:18.657724+0000","last_unstale":"2023-10-06T04:52:18.658025+0000","last_undegraded":"2023-10-06T04:52:18.658025+0000","last_fullsized":"2023-10-06T04:52:18.658025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:01.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658607+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.658607+0000","last_peered":"2023-10-06T04:52:18.658607+0000","last_clean":"2023-10-06T04:52:18.658607+0000","last_became_active":"2023-10-06T04:52:18.658286+0000","last_became_peered":"2023-10-06T04:52:18.658286+0000","last_unstale":"2023-10-06T04:52:18.658607+0000","last_undegraded":"2023-10-06T04:52:18.658607+0000","last_fullsized":"2023-10-06T04:52:18.658607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665708+0000","last_change":"2023-10-06T04:52:18.652791+0000","last_active":"2023-10-06T04:52:18.665708+0000","last_peered":"2023-10-06T04:52:18.665708+0000","last_clean":"2023-10-06T04:52:18.665708+0000","last_became_active":"2023-10-06T04:52:18.652522+0000","last_became_peered":"2023-10-06T04:52:18.652522+0000","last_unstale":"2023-10-06T04:52:18.665708+0000","last_undegraded":"2023-10-06T04:52:18.665708+0000","last_fullsized":"2023-10-06T04:52:18.665708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.649578+0000","last_change":"2023-10-06T04:52:18.649578+0000","last_active":"2023-10-06T04:52:18.649578+0000","last_peered":"2023-10-06T04:52:18.649578+0000","last_clean":"2023-10-06T04:52:18.649578+0000","last_became_active":"2023-10-06T04:52:18.649294+0000","last_became_peered":"2023-10-06T04:52:18.649294+0000","last_unstale":"2023-10-06T04:52:18.649578+0000","last_undegraded":"2023-10-06T04:52:18.649578+0000","last_fullsized":"2023-10-06T04:52:18.649578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:28.571226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658392+0000","last_change":"2023-10-06T04:52:18.653657+0000","last_active":"2023-10-06T04:52:18.658392+0000","last_peered":"2023-10-06T04:52:18.658392+0000","last_clean":"2023-10-06T04:52:18.658392+0000","last_became_active":"2023-10-06T04:52:18.653363+0000","last_became_peered":"2023-10-06T04:52:18.653363+0000","last_unstale":"2023-10-06T04:52:18.658392+0000","last_undegraded":"2023-10-06T04:52:18.658392+0000","last_fullsized":"2023-10-06T04:52:18.658392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665600+0000","last_change":"2023-10-06T04:52:18.654512+0000","last_active":"2023-10-06T04:52:18.665600+0000","last_peered":"2023-10-06T04:52:18.665600+0000","last_clean":"2023-10-06T04:52:18.665600+0000","last_became_active":"2023-10-06T04:52:18.653897+0000","last_became_peered":"2023-10-06T04:52:18.653897+0000","last_unstale":"2023-10-06T04:52:18.665600+0000","last_undegraded":"2023-10-06T04:52:18.665600+0000","last_fullsized":"2023-10-06T04:52:18.665600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:38.713099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655144+0000","last_change":"2023-10-06T04:52:18.655144+0000","last_active":"2023-10-06T04:52:18.655144+0000","last_peered":"2023-10-06T04:52:18.655144+0000","last_clean":"2023-10-06T04:52:18.655144+0000","last_became_active":"2023-10-06T04:52:18.654891+0000","last_became_peered":"2023-10-06T04:52:18.654891+0000","last_unstale":"2023-10-06T04:52:18.655144+0000","last_undegraded":"2023-10-06T04:52:18.655144+0000","last_fullsized":"2023-10-06T04:52:18.655144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666282+0000","last_change":"2023-10-06T04:52:18.666282+0000","last_active":"2023-10-06T04:52:18.666282+0000","last_peered":"2023-10-06T04:52:18.666282+0000","last_clean":"2023-10-06T04:52:18.666282+0000","last_became_active":"2023-10-06T04:52:18.665988+0000","last_became_peered":"2023-10-06T04:52:18.665988+0000","last_unstale":"2023-10-06T04:52:18.666282+0000","last_undegraded":"2023-10-06T04:52:18.666282+0000","last_fullsized":"2023-10-06T04:52:18.666282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665701+0000","last_change":"2023-10-06T04:52:18.661193+0000","last_active":"2023-10-06T04:52:18.665701+0000","last_peered":"2023-10-06T04:52:18.665701+0000","last_clean":"2023-10-06T04:52:18.665701+0000","last_became_active":"2023-10-06T04:52:18.660905+0000","last_became_peered":"2023-10-06T04:52:18.660905+0000","last_unstale":"2023-10-06T04:52:18.665701+0000","last_undegraded":"2023-10-06T04:52:18.665701+0000","last_fullsized":"2023-10-06T04:52:18.665701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.654460+0000","last_change":"2023-10-06T04:52:18.654460+0000","last_active":"2023-10-06T04:52:18.654460+0000","last_peered":"2023-10-06T04:52:18.654460+0000","last_clean":"2023-10-06T04:52:18.654460+0000","last_became_active":"2023-10-06T04:52:18.653370+0000","last_became_peered":"2023-10-06T04:52:18.653370+0000","last_unstale":"2023-10-06T04:52:18.654460+0000","last_undegraded":"2023-10-06T04:52:18.654460+0000","last_fullsized":"2023-10-06T04:52:18.654460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659795+0000","last_change":"2023-10-06T04:52:18.650974+0000","last_active":"2023-10-06T04:52:18.659795+0000","last_peered":"2023-10-06T04:52:18.659795+0000","last_clean":"2023-10-06T04:52:18.659795+0000","last_became_active":"2023-10-06T04:52:18.650708+0000","last_became_peered":"2023-10-06T04:52:18.650708+0000","last_unstale":"2023-10-06T04:52:18.659795+0000","last_undegraded":"2023-10-06T04:52:18.659795+0000","last_fullsized":"2023-10-06T04:52:18.659795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.651167+0000","last_change":"2023-10-06T04:52:18.651167+0000","last_active":"2023-10-06T04:52:18.651167+0000","last_peered":"2023-10-06T04:52:18.651167+0000","last_clean":"2023-10-06T04:52:18.651167+0000","last_became_active":"2023-10-06T04:52:18.650869+0000","last_became_peered":"2023-10-06T04:52:18.650869+0000","last_unstale":"2023-10-06T04:52:18.651167+0000","last_undegraded":"2023-10-06T04:52:18.651167+0000","last_fullsized":"2023-10-06T04:52:18.651167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:55.372045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.653721+0000","last_change":"2023-10-06T04:52:18.653721+0000","last_active":"2023-10-06T04:52:18.653721+0000","last_peered":"2023-10-06T04:52:18.653721+0000","last_clean":"2023-10-06T04:52:18.653721+0000","last_became_active":"2023-10-06T04:52:18.653435+0000","last_became_peered":"2023-10-06T04:52:18.653435+0000","last_unstale":"2023-10-06T04:52:18.653721+0000","last_undegraded":"2023-10-06T04:52:18.653721+0000","last_fullsized":"2023-10-06T04:52:18.653721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:43.201875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655632+0000","last_change":"2023-10-06T04:52:18.655632+0000","last_active":"2023-10-06T04:52:18.655632+0000","last_peered":"2023-10-06T04:52:18.655632+0000","last_clean":"2023-10-06T04:52:18.655632+0000","last_became_active":"2023-10-06T04:52:18.655335+0000","last_became_peered":"2023-10-06T04:52:18.655335+0000","last_unstale":"2023-10-06T04:52:18.655632+0000","last_undegraded":"2023-10-06T04:52:18.655632+0000","last_fullsized":"2023-10-06T04:52:18.655632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663599+0000","last_change":"2023-10-06T04:52:18.663599+0000","last_active":"2023-10-06T04:52:18.663599+0000","last_peered":"2023-10-06T04:52:18.663599+0000","last_clean":"2023-10-06T04:52:18.663599+0000","last_became_active":"2023-10-06T04:52:18.663303+0000","last_became_peered":"2023-10-06T04:52:18.663303+0000","last_unstale":"2023-10-06T04:52:18.663599+0000","last_undegraded":"2023-10-06T04:52:18.663599+0000","last_fullsized":"2023-10-06T04:52:18.663599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665493+0000","last_change":"2023-10-06T04:52:18.652910+0000","last_active":"2023-10-06T04:52:18.665493+0000","last_peered":"2023-10-06T04:52:18.665493+0000","last_clean":"2023-10-06T04:52:18.665493+0000","last_became_active":"2023-10-06T04:52:18.652589+0000","last_became_peered":"2023-10-06T04:52:18.652589+0000","last_unstale":"2023-10-06T04:52:18.665493+0000","last_undegraded":"2023-10-06T04:52:18.665493+0000","last_fullsized":"2023-10-06T04:52:18.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:10.947807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658434+0000","last_change":"2023-10-06T04:52:18.654868+0000","last_active":"2023-10-06T04:52:18.658434+0000","last_peered":"2023-10-06T04:52:18.658434+0000","last_clean":"2023-10-06T04:52:18.658434+0000","last_became_active":"2023-10-06T04:52:18.654590+0000","last_became_peered":"2023-10-06T04:52:18.654590+0000","last_unstale":"2023-10-06T04:52:18.658434+0000","last_undegraded":"2023-10-06T04:52:18.658434+0000","last_fullsized":"2023-10-06T04:52:18.658434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665816+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.665816+0000","last_peered":"2023-10-06T04:52:18.665816+0000","last_clean":"2023-10-06T04:52:18.665816+0000","last_became_active":"2023-10-06T04:52:18.658358+0000","last_became_peered":"2023-10-06T04:52:18.658358+0000","last_unstale":"2023-10-06T04:52:18.665816+0000","last_undegraded":"2023-10-06T04:52:18.665816+0000","last_fullsized":"2023-10-06T04:52:18.665816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662801+0000","last_change":"2023-10-06T04:52:18.662801+0000","last_active":"2023-10-06T04:52:18.662801+0000","last_peered":"2023-10-06T04:52:18.662801+0000","last_clean":"2023-10-06T04:52:18.662801+0000","last_became_active":"2023-10-06T04:52:18.662534+0000","last_became_peered":"2023-10-06T04:52:18.662534+0000","last_unstale":"2023-10-06T04:52:18.662801+0000","last_undegraded":"2023-10-06T04:52:18.662801+0000","last_fullsized":"2023-10-06T04:52:18.662801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:53.970289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666479+0000","last_change":"2023-10-06T04:52:18.664643+0000","last_active":"2023-10-06T04:52:18.666479+0000","last_peered":"2023-10-06T04:52:18.666479+0000","last_clean":"2023-10-06T04:52:18.666479+0000","last_became_active":"2023-10-06T04:52:18.664342+0000","last_became_peered":"2023-10-06T04:52:18.664342+0000","last_unstale":"2023-10-06T04:52:18.666479+0000","last_undegraded":"2023-10-06T04:52:18.666479+0000","last_fullsized":"2023-10-06T04:52:18.666479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:33.202071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663114+0000","last_change":"2023-10-06T04:52:18.663114+0000","last_active":"2023-10-06T04:52:18.663114+0000","last_peered":"2023-10-06T04:52:18.663114+0000","last_clean":"2023-10-06T04:52:18.663114+0000","last_became_active":"2023-10-06T04:52:18.662825+0000","last_became_peered":"2023-10-06T04:52:18.662825+0000","last_unstale":"2023-10-06T04:52:18.663114+0000","last_undegraded":"2023-10-06T04:52:18.663114+0000","last_fullsized":"2023-10-06T04:52:18.663114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650717+0000","last_change":"2023-10-06T04:52:18.648283+0000","last_active":"2023-10-06T04:52:18.650717+0000","last_peered":"2023-10-06T04:52:18.650717+0000","last_clean":"2023-10-06T04:52:18.650717+0000","last_became_active":"2023-10-06T04:52:18.647966+0000","last_became_peered":"2023-10-06T04:52:18.647966+0000","last_unstale":"2023-10-06T04:52:18.650717+0000","last_undegraded":"2023-10-06T04:52:18.650717+0000","last_fullsized":"2023-10-06T04:52:18.650717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661203+0000","last_change":"2023-10-06T04:52:18.655500+0000","last_active":"2023-10-06T04:52:18.661203+0000","last_peered":"2023-10-06T04:52:18.661203+0000","last_clean":"2023-10-06T04:52:18.661203+0000","last_became_active":"2023-10-06T04:52:18.655234+0000","last_became_peered":"2023-10-06T04:52:18.655234+0000","last_unstale":"2023-10-06T04:52:18.661203+0000","last_undegraded":"2023-10-06T04:52:18.661203+0000","last_fullsized":"2023-10-06T04:52:18.661203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660815+0000","last_change":"2023-10-06T04:52:18.660815+0000","last_active":"2023-10-06T04:52:18.660815+0000","last_peered":"2023-10-06T04:52:18.660815+0000","last_clean":"2023-10-06T04:52:18.660815+0000","last_became_active":"2023-10-06T04:52:18.660475+0000","last_became_peered":"2023-10-06T04:52:18.660475+0000","last_unstale":"2023-10-06T04:52:18.660815+0000","last_undegraded":"2023-10-06T04:52:18.660815+0000","last_fullsized":"2023-10-06T04:52:18.660815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663035+0000","last_change":"2023-10-06T04:52:18.663035+0000","last_active":"2023-10-06T04:52:18.663035+0000","last_peered":"2023-10-06T04:52:18.663035+0000","last_clean":"2023-10-06T04:52:18.663035+0000","last_became_active":"2023-10-06T04:52:18.662611+0000","last_became_peered":"2023-10-06T04:52:18.662611+0000","last_unstale":"2023-10-06T04:52:18.663035+0000","last_undegraded":"2023-10-06T04:52:18.663035+0000","last_fullsized":"2023-10-06T04:52:18.663035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658476+0000","last_change":"2023-10-06T04:52:18.658476+0000","last_active":"2023-10-06T04:52:18.658476+0000","last_peered":"2023-10-06T04:52:18.658476+0000","last_clean":"2023-10-06T04:52:18.658476+0000","last_became_active":"2023-10-06T04:52:18.658197+0000","last_became_peered":"2023-10-06T04:52:18.658197+0000","last_unstale":"2023-10-06T04:52:18.658476+0000","last_undegraded":"2023-10-06T04:52:18.658476+0000","last_fullsized":"2023-10-06T04:52:18.658476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658955+0000","last_change":"2023-10-06T04:52:18.658955+0000","last_active":"2023-10-06T04:52:18.658955+0000","last_peered":"2023-10-06T04:52:18.658955+0000","last_clean":"2023-10-06T04:52:18.658955+0000","last_became_active":"2023-10-06T04:52:18.658713+0000","last_became_peered":"2023-10-06T04:52:18.658713+0000","last_unstale":"2023-10-06T04:52:18.658955+0000","last_undegraded":"2023-10-06T04:52:18.658955+0000","last_fullsized":"2023-10-06T04:52:18.658955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.651332+0000","last_change":"2023-10-06T04:52:17.656516+0000","last_active":"2023-10-06T04:52:18.651332+0000","last_peered":"2023-10-06T04:52:18.651332+0000","last_clean":"2023-10-06T04:52:18.651332+0000","last_became_active":"2023-10-06T04:52:17.656032+0000","last_became_peered":"2023-10-06T04:52:17.656032+0000","last_unstale":"2023-10-06T04:52:18.651332+0000","last_undegraded":"2023-10-06T04:52:18.651332+0000","last_fullsized":"2023-10-06T04:52:18.651332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660104+0000","last_change":"2023-10-06T04:52:18.656019+0000","last_active":"2023-10-06T04:52:18.660104+0000","last_peered":"2023-10-06T04:52:18.660104+0000","last_clean":"2023-10-06T04:52:18.660104+0000","last_became_active":"2023-10-06T04:52:18.655673+0000","last_became_peered":"2023-10-06T04:52:18.655673+0000","last_unstale":"2023-10-06T04:52:18.660104+0000","last_undegraded":"2023-10-06T04:52:18.660104+0000","last_fullsized":"2023-10-06T04:52:18.660104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.651598+0000","last_change":"2023-10-06T04:52:17.645852+0000","last_active":"2023-10-06T04:52:17.651598+0000","last_peered":"2023-10-06T04:52:17.651598+0000","last_clean":"2023-10-06T04:52:17.651598+0000","last_became_active":"2023-10-06T04:52:17.645528+0000","last_became_peered":"2023-10-06T04:52:17.645528+0000","last_unstale":"2023-10-06T04:52:17.651598+0000","last_undegraded":"2023-10-06T04:52:17.651598+0000","last_fullsized":"2023-10-06T04:52:17.651598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655629+0000","last_change":"2023-10-06T04:52:18.649691+0000","last_active":"2023-10-06T04:52:18.655629+0000","last_peered":"2023-10-06T04:52:18.655629+0000","last_clean":"2023-10-06T04:52:18.655629+0000","last_became_active":"2023-10-06T04:52:18.649348+0000","last_became_peered":"2023-10-06T04:52:18.649348+0000","last_unstale":"2023-10-06T04:52:18.655629+0000","last_undegraded":"2023-10-06T04:52:18.655629+0000","last_fullsized":"2023-10-06T04:52:18.655629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643218+0000","last_change":"2023-10-06T04:52:17.651301+0000","last_active":"2023-10-06T04:52:18.643218+0000","last_peered":"2023-10-06T04:52:18.643218+0000","last_clean":"2023-10-06T04:52:18.643218+0000","last_became_active":"2023-10-06T04:52:17.650967+0000","last_became_peered":"2023-10-06T04:52:17.650967+0000","last_unstale":"2023-10-06T04:52:18.643218+0000","last_undegraded":"2023-10-06T04:52:18.643218+0000","last_fullsized":"2023-10-06T04:52:18.643218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656399+0000","last_change":"2023-10-06T04:52:18.656399+0000","last_active":"2023-10-06T04:52:18.656399+0000","last_peered":"2023-10-06T04:52:18.656399+0000","last_clean":"2023-10-06T04:52:18.656399+0000","last_became_active":"2023-10-06T04:52:18.656106+0000","last_became_peered":"2023-10-06T04:52:18.656106+0000","last_unstale":"2023-10-06T04:52:18.656399+0000","last_undegraded":"2023-10-06T04:52:18.656399+0000","last_fullsized":"2023-10-06T04:52:18.656399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.640098+0000","last_change":"2023-10-06T04:52:17.654663+0000","last_active":"2023-10-06T04:52:18.640098+0000","last_peered":"2023-10-06T04:52:18.640098+0000","last_clean":"2023-10-06T04:52:18.640098+0000","last_became_active":"2023-10-06T04:52:17.654402+0000","last_became_peered":"2023-10-06T04:52:17.654402+0000","last_unstale":"2023-10-06T04:52:18.640098+0000","last_undegraded":"2023-10-06T04:52:18.640098+0000","last_fullsized":"2023-10-06T04:52:18.640098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660162+0000","last_change":"2023-10-06T04:52:18.655602+0000","last_active":"2023-10-06T04:52:18.660162+0000","last_peered":"2023-10-06T04:52:18.660162+0000","last_clean":"2023-10-06T04:52:18.660162+0000","last_became_active":"2023-10-06T04:52:18.655329+0000","last_became_peered":"2023-10-06T04:52:18.655329+0000","last_unstale":"2023-10-06T04:52:18.660162+0000","last_undegraded":"2023-10-06T04:52:18.660162+0000","last_fullsized":"2023-10-06T04:52:18.660162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.659765+0000","last_change":"2023-10-06T04:52:17.652631+0000","last_active":"2023-10-06T04:52:17.659765+0000","last_peered":"2023-10-06T04:52:17.659765+0000","last_clean":"2023-10-06T04:52:17.659765+0000","last_became_active":"2023-10-06T04:52:17.652340+0000","last_became_peered":"2023-10-06T04:52:17.652340+0000","last_unstale":"2023-10-06T04:52:17.659765+0000","last_undegraded":"2023-10-06T04:52:17.659765+0000","last_fullsized":"2023-10-06T04:52:17.659765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659061+0000","last_change":"2023-10-06T04:52:18.651950+0000","last_active":"2023-10-06T04:52:18.659061+0000","last_peered":"2023-10-06T04:52:18.659061+0000","last_clean":"2023-10-06T04:52:18.659061+0000","last_became_active":"2023-10-06T04:52:18.651587+0000","last_became_peered":"2023-10-06T04:52:18.651587+0000","last_unstale":"2023-10-06T04:52:18.659061+0000","last_undegraded":"2023-10-06T04:52:18.659061+0000","last_fullsized":"2023-10-06T04:52:18.659061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667304+0000","last_change":"2023-10-06T04:52:17.655489+0000","last_active":"2023-10-06T04:52:18.667304+0000","last_peered":"2023-10-06T04:52:18.667304+0000","last_clean":"2023-10-06T04:52:18.667304+0000","last_became_active":"2023-10-06T04:52:17.655131+0000","last_became_peered":"2023-10-06T04:52:17.655131+0000","last_unstale":"2023-10-06T04:52:18.667304+0000","last_undegraded":"2023-10-06T04:52:18.667304+0000","last_fullsized":"2023-10-06T04:52:18.667304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662999+0000","last_change":"2023-10-06T04:52:18.662999+0000","last_active":"2023-10-06T04:52:18.662999+0000","last_peered":"2023-10-06T04:52:18.662999+0000","last_clean":"2023-10-06T04:52:18.662999+0000","last_became_active":"2023-10-06T04:52:18.662603+0000","last_became_peered":"2023-10-06T04:52:18.662603+0000","last_unstale":"2023-10-06T04:52:18.662999+0000","last_undegraded":"2023-10-06T04:52:18.662999+0000","last_fullsized":"2023-10-06T04:52:18.662999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.669196+0000","last_change":"2023-10-06T04:52:17.655293+0000","last_active":"2023-10-06T04:52:18.669196+0000","last_peered":"2023-10-06T04:52:18.669196+0000","last_clean":"2023-10-06T04:52:18.669196+0000","last_became_active":"2023-10-06T04:52:17.654390+0000","last_became_peered":"2023-10-06T04:52:17.654390+0000","last_unstale":"2023-10-06T04:52:18.669196+0000","last_undegraded":"2023-10-06T04:52:18.669196+0000","last_fullsized":"2023-10-06T04:52:18.669196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665827+0000","last_change":"2023-10-06T04:52:18.657322+0000","last_active":"2023-10-06T04:52:18.665827+0000","last_peered":"2023-10-06T04:52:18.665827+0000","last_clean":"2023-10-06T04:52:18.665827+0000","last_became_active":"2023-10-06T04:52:18.657046+0000","last_became_peered":"2023-10-06T04:52:18.657046+0000","last_unstale":"2023-10-06T04:52:18.665827+0000","last_undegraded":"2023-10-06T04:52:18.665827+0000","last_fullsized":"2023-10-06T04:52:18.665827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659746+0000","last_change":"2023-10-06T04:52:17.657213+0000","last_active":"2023-10-06T04:52:18.659746+0000","last_peered":"2023-10-06T04:52:18.659746+0000","last_clean":"2023-10-06T04:52:18.659746+0000","last_became_active":"2023-10-06T04:52:17.656945+0000","last_became_peered":"2023-10-06T04:52:17.656945+0000","last_unstale":"2023-10-06T04:52:18.659746+0000","last_undegraded":"2023-10-06T04:52:18.659746+0000","last_fullsized":"2023-10-06T04:52:18.659746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:17.302823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659796+0000","last_change":"2023-10-06T04:52:18.655013+0000","last_active":"2023-10-06T04:52:18.659796+0000","last_peered":"2023-10-06T04:52:18.659796+0000","last_clean":"2023-10-06T04:52:18.659796+0000","last_became_active":"2023-10-06T04:52:18.654734+0000","last_became_peered":"2023-10-06T04:52:18.654734+0000","last_unstale":"2023-10-06T04:52:18.659796+0000","last_undegraded":"2023-10-06T04:52:18.659796+0000","last_fullsized":"2023-10-06T04:52:18.659796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638374+0000","last_change":"2023-10-06T04:52:17.650696+0000","last_active":"2023-10-06T04:52:18.638374+0000","last_peered":"2023-10-06T04:52:18.638374+0000","last_clean":"2023-10-06T04:52:18.638374+0000","last_became_active":"2023-10-06T04:52:17.650402+0000","last_became_peered":"2023-10-06T04:52:17.650402+0000","last_unstale":"2023-10-06T04:52:18.638374+0000","last_undegraded":"2023-10-06T04:52:18.638374+0000","last_fullsized":"2023-10-06T04:52:18.638374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658995+0000","last_change":"2023-10-06T04:52:18.658995+0000","last_active":"2023-10-06T04:52:18.658995+0000","last_peered":"2023-10-06T04:52:18.658995+0000","last_clean":"2023-10-06T04:52:18.658995+0000","last_became_active":"2023-10-06T04:52:18.658717+0000","last_became_peered":"2023-10-06T04:52:18.658717+0000","last_unstale":"2023-10-06T04:52:18.658995+0000","last_undegraded":"2023-10-06T04:52:18.658995+0000","last_fullsized":"2023-10-06T04:52:18.658995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.661884+0000","last_change":"2023-10-06T04:52:17.657760+0000","last_active":"2023-10-06T04:52:17.661884+0000","last_peered":"2023-10-06T04:52:17.661884+0000","last_clean":"2023-10-06T04:52:17.661884+0000","last_became_active":"2023-10-06T04:52:17.657484+0000","last_became_peered":"2023-10-06T04:52:17.657484+0000","last_unstale":"2023-10-06T04:52:17.661884+0000","last_undegraded":"2023-10-06T04:52:17.661884+0000","last_fullsized":"2023-10-06T04:52:17.661884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665775+0000","last_change":"2023-10-06T04:52:18.661243+0000","last_active":"2023-10-06T04:52:18.665775+0000","last_peered":"2023-10-06T04:52:18.665775+0000","last_clean":"2023-10-06T04:52:18.665775+0000","last_became_active":"2023-10-06T04:52:18.660932+0000","last_became_peered":"2023-10-06T04:52:18.660932+0000","last_unstale":"2023-10-06T04:52:18.665775+0000","last_undegraded":"2023-10-06T04:52:18.665775+0000","last_fullsized":"2023-10-06T04:52:18.665775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666058+0000","last_change":"2023-10-06T04:52:17.665506+0000","last_active":"2023-10-06T04:52:18.666058+0000","last_peered":"2023-10-06T04:52:18.666058+0000","last_clean":"2023-10-06T04:52:18.666058+0000","last_became_active":"2023-10-06T04:52:17.665209+0000","last_became_peered":"2023-10-06T04:52:17.665209+0000","last_unstale":"2023-10-06T04:52:18.666058+0000","last_undegraded":"2023-10-06T04:52:18.666058+0000","last_fullsized":"2023-10-06T04:52:18.666058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660252+0000","last_change":"2023-10-06T04:52:18.654716+0000","last_active":"2023-10-06T04:52:18.660252+0000","last_peered":"2023-10-06T04:52:18.660252+0000","last_clean":"2023-10-06T04:52:18.660252+0000","last_became_active":"2023-10-06T04:52:18.653774+0000","last_became_peered":"2023-10-06T04:52:18.653774+0000","last_unstale":"2023-10-06T04:52:18.660252+0000","last_undegraded":"2023-10-06T04:52:18.660252+0000","last_fullsized":"2023-10-06T04:52:18.660252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.649135+0000","last_change":"2023-10-06T04:52:17.657327+0000","last_active":"2023-10-06T04:52:18.649135+0000","last_peered":"2023-10-06T04:52:18.649135+0000","last_clean":"2023-10-06T04:52:18.649135+0000","last_became_active":"2023-10-06T04:52:17.657054+0000","last_became_peered":"2023-10-06T04:52:17.657054+0000","last_unstale":"2023-10-06T04:52:18.649135+0000","last_undegraded":"2023-10-06T04:52:18.649135+0000","last_fullsized":"2023-10-06T04:52:18.649135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.652991+0000","last_change":"2023-10-06T04:52:18.652991+0000","last_active":"2023-10-06T04:52:18.652991+0000","last_peered":"2023-10-06T04:52:18.652991+0000","last_clean":"2023-10-06T04:52:18.652991+0000","last_became_active":"2023-10-06T04:52:18.652695+0000","last_became_peered":"2023-10-06T04:52:18.652695+0000","last_unstale":"2023-10-06T04:52:18.652991+0000","last_undegraded":"2023-10-06T04:52:18.652991+0000","last_fullsized":"2023-10-06T04:52:18.652991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659840+0000","last_change":"2023-10-06T04:52:17.659228+0000","last_active":"2023-10-06T04:52:18.659840+0000","last_peered":"2023-10-06T04:52:18.659840+0000","last_clean":"2023-10-06T04:52:18.659840+0000","last_became_active":"2023-10-06T04:52:17.658947+0000","last_became_peered":"2023-10-06T04:52:17.658947+0000","last_unstale":"2023-10-06T04:52:18.659840+0000","last_undegraded":"2023-10-06T04:52:18.659840+0000","last_fullsized":"2023-10-06T04:52:18.659840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659272+0000","last_change":"2023-10-06T04:52:18.649861+0000","last_active":"2023-10-06T04:52:18.659272+0000","last_peered":"2023-10-06T04:52:18.659272+0000","last_clean":"2023-10-06T04:52:18.659272+0000","last_became_active":"2023-10-06T04:52:18.649576+0000","last_became_peered":"2023-10-06T04:52:18.649576+0000","last_unstale":"2023-10-06T04:52:18.659272+0000","last_undegraded":"2023-10-06T04:52:18.659272+0000","last_fullsized":"2023-10-06T04:52:18.659272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:14.896487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.649796+0000","last_change":"2023-10-06T04:52:17.653503+0000","last_active":"2023-10-06T04:52:18.649796+0000","last_peered":"2023-10-06T04:52:18.649796+0000","last_clean":"2023-10-06T04:52:18.649796+0000","last_became_active":"2023-10-06T04:52:17.653236+0000","last_became_peered":"2023-10-06T04:52:17.653236+0000","last_unstale":"2023-10-06T04:52:18.649796+0000","last_undegraded":"2023-10-06T04:52:18.649796+0000","last_fullsized":"2023-10-06T04:52:18.649796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:27.560082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661966+0000","last_change":"2023-10-06T04:52:18.661966+0000","last_active":"2023-10-06T04:52:18.661966+0000","last_peered":"2023-10-06T04:52:18.661966+0000","last_clean":"2023-10-06T04:52:18.661966+0000","last_became_active":"2023-10-06T04:52:18.661376+0000","last_became_peered":"2023-10-06T04:52:18.661376+0000","last_unstale":"2023-10-06T04:52:18.661966+0000","last_undegraded":"2023-10-06T04:52:18.661966+0000","last_fullsized":"2023-10-06T04:52:18.661966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652153+0000","last_change":"2023-10-06T04:52:17.650123+0000","last_active":"2023-10-06T04:52:17.652153+0000","last_peered":"2023-10-06T04:52:17.652153+0000","last_clean":"2023-10-06T04:52:17.652153+0000","last_became_active":"2023-10-06T04:52:17.649388+0000","last_became_peered":"2023-10-06T04:52:17.649388+0000","last_unstale":"2023-10-06T04:52:17.652153+0000","last_undegraded":"2023-10-06T04:52:17.652153+0000","last_fullsized":"2023-10-06T04:52:17.652153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650775+0000","last_change":"2023-10-06T04:52:18.647865+0000","last_active":"2023-10-06T04:52:18.650775+0000","last_peered":"2023-10-06T04:52:18.650775+0000","last_clean":"2023-10-06T04:52:18.650775+0000","last_became_active":"2023-10-06T04:52:18.647582+0000","last_became_peered":"2023-10-06T04:52:18.647582+0000","last_unstale":"2023-10-06T04:52:18.650775+0000","last_undegraded":"2023-10-06T04:52:18.650775+0000","last_fullsized":"2023-10-06T04:52:18.650775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665684+0000","last_change":"2023-10-06T04:52:17.651495+0000","last_active":"2023-10-06T04:52:18.665684+0000","last_peered":"2023-10-06T04:52:18.665684+0000","last_clean":"2023-10-06T04:52:18.665684+0000","last_became_active":"2023-10-06T04:52:17.651108+0000","last_became_peered":"2023-10-06T04:52:17.651108+0000","last_unstale":"2023-10-06T04:52:18.665684+0000","last_undegraded":"2023-10-06T04:52:18.665684+0000","last_fullsized":"2023-10-06T04:52:18.665684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666462+0000","last_change":"2023-10-06T04:52:18.662825+0000","last_active":"2023-10-06T04:52:18.666462+0000","last_peered":"2023-10-06T04:52:18.666462+0000","last_clean":"2023-10-06T04:52:18.666462+0000","last_became_active":"2023-10-06T04:52:18.662499+0000","last_became_peered":"2023-10-06T04:52:18.662499+0000","last_unstale":"2023-10-06T04:52:18.666462+0000","last_undegraded":"2023-10-06T04:52:18.666462+0000","last_fullsized":"2023-10-06T04:52:18.666462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:58.798038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.661547+0000","last_change":"2023-10-06T04:52:17.657591+0000","last_active":"2023-10-06T04:52:17.661547+0000","last_peered":"2023-10-06T04:52:17.661547+0000","last_clean":"2023-10-06T04:52:17.661547+0000","last_became_active":"2023-10-06T04:52:17.657291+0000","last_became_peered":"2023-10-06T04:52:17.657291+0000","last_unstale":"2023-10-06T04:52:17.661547+0000","last_undegraded":"2023-10-06T04:52:17.661547+0000","last_fullsized":"2023-10-06T04:52:17.661547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659774+0000","last_change":"2023-10-06T04:52:18.650766+0000","last_active":"2023-10-06T04:52:18.659774+0000","last_peered":"2023-10-06T04:52:18.659774+0000","last_clean":"2023-10-06T04:52:18.659774+0000","last_became_active":"2023-10-06T04:52:18.650514+0000","last_became_peered":"2023-10-06T04:52:18.650514+0000","last_unstale":"2023-10-06T04:52:18.659774+0000","last_undegraded":"2023-10-06T04:52:18.659774+0000","last_fullsized":"2023-10-06T04:52:18.659774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.639506+0000","last_change":"2023-10-06T04:52:17.655486+0000","last_active":"2023-10-06T04:52:18.639506+0000","last_peered":"2023-10-06T04:52:18.639506+0000","last_clean":"2023-10-06T04:52:18.639506+0000","last_became_active":"2023-10-06T04:52:17.654231+0000","last_became_peered":"2023-10-06T04:52:17.654231+0000","last_unstale":"2023-10-06T04:52:18.639506+0000","last_undegraded":"2023-10-06T04:52:18.639506+0000","last_fullsized":"2023-10-06T04:52:18.639506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666321+0000","last_change":"2023-10-06T04:52:18.663641+0000","last_active":"2023-10-06T04:52:18.666321+0000","last_peered":"2023-10-06T04:52:18.666321+0000","last_clean":"2023-10-06T04:52:18.666321+0000","last_became_active":"2023-10-06T04:52:18.663334+0000","last_became_peered":"2023-10-06T04:52:18.663334+0000","last_unstale":"2023-10-06T04:52:18.666321+0000","last_undegraded":"2023-10-06T04:52:18.666321+0000","last_fullsized":"2023-10-06T04:52:18.666321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:18.572482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643070+0000","last_change":"2023-10-06T04:52:17.644385+0000","last_active":"2023-10-06T04:52:18.643070+0000","last_peered":"2023-10-06T04:52:18.643070+0000","last_clean":"2023-10-06T04:52:18.643070+0000","last_became_active":"2023-10-06T04:52:17.643911+0000","last_became_peered":"2023-10-06T04:52:17.643911+0000","last_unstale":"2023-10-06T04:52:18.643070+0000","last_undegraded":"2023-10-06T04:52:18.643070+0000","last_fullsized":"2023-10-06T04:52:18.643070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.657062+0000","last_change":"2023-10-06T04:52:18.657062+0000","last_active":"2023-10-06T04:52:18.657062+0000","last_peered":"2023-10-06T04:52:18.657062+0000","last_clean":"2023-10-06T04:52:18.657062+0000","last_became_active":"2023-10-06T04:52:18.656805+0000","last_became_peered":"2023-10-06T04:52:18.656805+0000","last_unstale":"2023-10-06T04:52:18.657062+0000","last_undegraded":"2023-10-06T04:52:18.657062+0000","last_fullsized":"2023-10-06T04:52:18.657062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664453+0000","last_change":"2023-10-06T04:52:17.648546+0000","last_active":"2023-10-06T04:52:18.664453+0000","last_peered":"2023-10-06T04:52:18.664453+0000","last_clean":"2023-10-06T04:52:18.664453+0000","last_became_active":"2023-10-06T04:52:17.648247+0000","last_became_peered":"2023-10-06T04:52:17.648247+0000","last_unstale":"2023-10-06T04:52:18.664453+0000","last_undegraded":"2023-10-06T04:52:18.664453+0000","last_fullsized":"2023-10-06T04:52:18.664453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642529+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:52:18.642529+0000","last_peered":"2023-10-06T04:52:18.642529+0000","last_clean":"2023-10-06T04:52:18.642529+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:52:18.642529+0000","last_undegraded":"2023-10-06T04:52:18.642529+0000","last_fullsized":"2023-10-06T04:52:18.642529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659829+0000","last_change":"2023-10-06T04:52:18.654743+0000","last_active":"2023-10-06T04:52:18.659829+0000","last_peered":"2023-10-06T04:52:18.659829+0000","last_clean":"2023-10-06T04:52:18.659829+0000","last_became_active":"2023-10-06T04:52:18.653945+0000","last_became_peered":"2023-10-06T04:52:18.653945+0000","last_unstale":"2023-10-06T04:52:18.659829+0000","last_undegraded":"2023-10-06T04:52:18.659829+0000","last_fullsized":"2023-10-06T04:52:18.659829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642776+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:52:18.642776+0000","last_peered":"2023-10-06T04:52:18.642776+0000","last_clean":"2023-10-06T04:52:18.642776+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:52:18.642776+0000","last_undegraded":"2023-10-06T04:52:18.642776+0000","last_fullsized":"2023-10-06T04:52:18.642776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.640876+0000","last_change":"2023-10-06T04:52:17.653308+0000","last_active":"2023-10-06T04:52:18.640876+0000","last_peered":"2023-10-06T04:52:18.640876+0000","last_clean":"2023-10-06T04:52:18.640876+0000","last_became_active":"2023-10-06T04:52:17.653032+0000","last_became_peered":"2023-10-06T04:52:17.653032+0000","last_unstale":"2023-10-06T04:52:18.640876+0000","last_undegraded":"2023-10-06T04:52:18.640876+0000","last_fullsized":"2023-10-06T04:52:18.640876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665740+0000","last_change":"2023-10-06T04:52:18.656477+0000","last_active":"2023-10-06T04:52:18.665740+0000","last_peered":"2023-10-06T04:52:18.665740+0000","last_clean":"2023-10-06T04:52:18.665740+0000","last_became_active":"2023-10-06T04:52:18.656225+0000","last_became_peered":"2023-10-06T04:52:18.656225+0000","last_unstale":"2023-10-06T04:52:18.665740+0000","last_undegraded":"2023-10-06T04:52:18.665740+0000","last_fullsized":"2023-10-06T04:52:18.665740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.634374+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:52:17.634374+0000","last_peered":"2023-10-06T04:52:17.634374+0000","last_clean":"2023-10-06T04:52:17.634374+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:52:17.634374+0000","last_undegraded":"2023-10-06T04:52:17.634374+0000","last_fullsized":"2023-10-06T04:52:17.634374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":162,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.642530+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:52:17.642530+0000","last_peered":"2023-10-06T04:52:17.642530+0000","last_clean":"2023-10-06T04:52:17.642530+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:52:17.642530+0000","last_undegraded":"2023-10-06T04:52:17.642530+0000","last_fullsized":"2023-10-06T04:52:17.642530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638291+0000","last_change":"2023-10-06T04:52:17.656649+0000","last_active":"2023-10-06T04:52:18.638291+0000","last_peered":"2023-10-06T04:52:18.638291+0000","last_clean":"2023-10-06T04:52:18.638291+0000","last_became_active":"2023-10-06T04:52:17.656336+0000","last_became_peered":"2023-10-06T04:52:17.656336+0000","last_unstale":"2023-10-06T04:52:18.638291+0000","last_undegraded":"2023-10-06T04:52:18.638291+0000","last_fullsized":"2023-10-06T04:52:18.638291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662767+0000","last_change":"2023-10-06T04:52:18.659289+0000","last_active":"2023-10-06T04:52:18.662767+0000","last_peered":"2023-10-06T04:52:18.662767+0000","last_clean":"2023-10-06T04:52:18.662767+0000","last_became_active":"2023-10-06T04:52:18.659024+0000","last_became_peered":"2023-10-06T04:52:18.659024+0000","last_unstale":"2023-10-06T04:52:18.662767+0000","last_undegraded":"2023-10-06T04:52:18.662767+0000","last_fullsized":"2023-10-06T04:52:18.662767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.642515+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:52:17.642515+0000","last_peered":"2023-10-06T04:52:17.642515+0000","last_clean":"2023-10-06T04:52:17.642515+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:52:17.642515+0000","last_undegraded":"2023-10-06T04:52:17.642515+0000","last_fullsized":"2023-10-06T04:52:17.642515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652075+0000","last_change":"2023-10-06T04:52:17.648757+0000","last_active":"2023-10-06T04:52:17.652075+0000","last_peered":"2023-10-06T04:52:17.652075+0000","last_clean":"2023-10-06T04:52:17.652075+0000","last_became_active":"2023-10-06T04:52:17.647698+0000","last_became_peered":"2023-10-06T04:52:17.647698+0000","last_unstale":"2023-10-06T04:52:17.652075+0000","last_undegraded":"2023-10-06T04:52:17.652075+0000","last_fullsized":"2023-10-06T04:52:17.652075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.648518+0000","last_change":"2023-10-06T04:52:18.648518+0000","last_active":"2023-10-06T04:52:18.648518+0000","last_peered":"2023-10-06T04:52:18.648518+0000","last_clean":"2023-10-06T04:52:18.648518+0000","last_became_active":"2023-10-06T04:52:18.648172+0000","last_became_peered":"2023-10-06T04:52:18.648172+0000","last_unstale":"2023-10-06T04:52:18.648518+0000","last_undegraded":"2023-10-06T04:52:18.648518+0000","last_fullsized":"2023-10-06T04:52:18.648518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.642418+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:52:17.642418+0000","last_peered":"2023-10-06T04:52:17.642418+0000","last_clean":"2023-10-06T04:52:17.642418+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:52:17.642418+0000","last_undegraded":"2023-10-06T04:52:17.642418+0000","last_fullsized":"2023-10-06T04:52:17.642418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642140+0000","last_change":"2023-10-06T04:52:17.647411+0000","last_active":"2023-10-06T04:52:18.642140+0000","last_peered":"2023-10-06T04:52:18.642140+0000","last_clean":"2023-10-06T04:52:18.642140+0000","last_became_active":"2023-10-06T04:52:17.646893+0000","last_became_peered":"2023-10-06T04:52:17.646893+0000","last_unstale":"2023-10-06T04:52:18.642140+0000","last_undegraded":"2023-10-06T04:52:18.642140+0000","last_fullsized":"2023-10-06T04:52:18.642140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655841+0000","last_change":"2023-10-06T04:52:18.655841+0000","last_active":"2023-10-06T04:52:18.655841+0000","last_peered":"2023-10-06T04:52:18.655841+0000","last_clean":"2023-10-06T04:52:18.655841+0000","last_became_active":"2023-10-06T04:52:18.655528+0000","last_became_peered":"2023-10-06T04:52:18.655528+0000","last_unstale":"2023-10-06T04:52:18.655841+0000","last_undegraded":"2023-10-06T04:52:18.655841+0000","last_fullsized":"2023-10-06T04:52:18.655841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.646208+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:52:17.646208+0000","last_peered":"2023-10-06T04:52:17.646208+0000","last_clean":"2023-10-06T04:52:17.646208+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:52:17.646208+0000","last_undegraded":"2023-10-06T04:52:17.646208+0000","last_fullsized":"2023-10-06T04:52:17.646208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.641848+0000","last_change":"2023-10-06T04:52:17.655012+0000","last_active":"2023-10-06T04:52:18.641848+0000","last_peered":"2023-10-06T04:52:18.641848+0000","last_clean":"2023-10-06T04:52:18.641848+0000","last_became_active":"2023-10-06T04:52:17.654727+0000","last_became_peered":"2023-10-06T04:52:17.654727+0000","last_unstale":"2023-10-06T04:52:18.641848+0000","last_undegraded":"2023-10-06T04:52:18.641848+0000","last_fullsized":"2023-10-06T04:52:18.641848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647619+0000","last_change":"2023-10-06T04:52:18.647619+0000","last_active":"2023-10-06T04:52:18.647619+0000","last_peered":"2023-10-06T04:52:18.647619+0000","last_clean":"2023-10-06T04:52:18.647619+0000","last_became_active":"2023-10-06T04:52:18.647303+0000","last_became_peered":"2023-10-06T04:52:18.647303+0000","last_unstale":"2023-10-06T04:52:18.647619+0000","last_undegraded":"2023-10-06T04:52:18.647619+0000","last_fullsized":"2023-10-06T04:52:18.647619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:35.374135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.648906+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:52:17.648906+0000","last_peered":"2023-10-06T04:52:17.648906+0000","last_clean":"2023-10-06T04:52:17.648906+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:52:17.648906+0000","last_undegraded":"2023-10-06T04:52:17.648906+0000","last_fullsized":"2023-10-06T04:52:17.648906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652504+0000","last_change":"2023-10-06T04:52:17.649887+0000","last_active":"2023-10-06T04:52:17.652504+0000","last_peered":"2023-10-06T04:52:17.652504+0000","last_clean":"2023-10-06T04:52:17.652504+0000","last_became_active":"2023-10-06T04:52:17.649537+0000","last_became_peered":"2023-10-06T04:52:17.649537+0000","last_unstale":"2023-10-06T04:52:17.652504+0000","last_undegraded":"2023-10-06T04:52:17.652504+0000","last_fullsized":"2023-10-06T04:52:17.652504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:57.133434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655515+0000","last_change":"2023-10-06T04:52:18.649286+0000","last_active":"2023-10-06T04:52:18.655515+0000","last_peered":"2023-10-06T04:52:18.655515+0000","last_clean":"2023-10-06T04:52:18.655515+0000","last_became_active":"2023-10-06T04:52:18.648947+0000","last_became_peered":"2023-10-06T04:52:18.648947+0000","last_unstale":"2023-10-06T04:52:18.655515+0000","last_undegraded":"2023-10-06T04:52:18.655515+0000","last_fullsized":"2023-10-06T04:52:18.655515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.648606+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:52:17.648606+0000","last_peered":"2023-10-06T04:52:17.648606+0000","last_clean":"2023-10-06T04:52:17.648606+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:52:17.648606+0000","last_undegraded":"2023-10-06T04:52:17.648606+0000","last_fullsized":"2023-10-06T04:52:17.648606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652549+0000","last_change":"2023-10-06T04:52:17.650178+0000","last_active":"2023-10-06T04:52:17.652549+0000","last_peered":"2023-10-06T04:52:17.652549+0000","last_clean":"2023-10-06T04:52:17.652549+0000","last_became_active":"2023-10-06T04:52:17.649883+0000","last_became_peered":"2023-10-06T04:52:17.649883+0000","last_unstale":"2023-10-06T04:52:17.652549+0000","last_undegraded":"2023-10-06T04:52:17.652549+0000","last_fullsized":"2023-10-06T04:52:17.652549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658639+0000","last_change":"2023-10-06T04:52:18.658639+0000","last_active":"2023-10-06T04:52:18.658639+0000","last_peered":"2023-10-06T04:52:18.658639+0000","last_clean":"2023-10-06T04:52:18.658639+0000","last_became_active":"2023-10-06T04:52:18.658381+0000","last_became_peered":"2023-10-06T04:52:18.658381+0000","last_unstale":"2023-10-06T04:52:18.658639+0000","last_undegraded":"2023-10-06T04:52:18.658639+0000","last_fullsized":"2023-10-06T04:52:18.658639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:51.836228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665805+0000","last_change":"2023-10-06T04:52:17.657299+0000","last_active":"2023-10-06T04:52:18.665805+0000","last_peered":"2023-10-06T04:52:18.665805+0000","last_clean":"2023-10-06T04:52:18.665805+0000","last_became_active":"2023-10-06T04:52:17.656956+0000","last_became_peered":"2023-10-06T04:52:17.656956+0000","last_unstale":"2023-10-06T04:52:18.665805+0000","last_undegraded":"2023-10-06T04:52:18.665805+0000","last_fullsized":"2023-10-06T04:52:18.665805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.652907+0000","last_change":"2023-10-06T04:52:18.652907+0000","last_active":"2023-10-06T04:52:18.652907+0000","last_peered":"2023-10-06T04:52:18.652907+0000","last_clean":"2023-10-06T04:52:18.652907+0000","last_became_active":"2023-10-06T04:52:18.652600+0000","last_became_peered":"2023-10-06T04:52:18.652600+0000","last_unstale":"2023-10-06T04:52:18.652907+0000","last_undegraded":"2023-10-06T04:52:18.652907+0000","last_fullsized":"2023-10-06T04:52:18.652907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:13.950333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667707+0000","last_change":"2023-10-06T04:52:17.656487+0000","last_active":"2023-10-06T04:52:18.667707+0000","last_peered":"2023-10-06T04:52:18.667707+0000","last_clean":"2023-10-06T04:52:18.667707+0000","last_became_active":"2023-10-06T04:52:17.656205+0000","last_became_peered":"2023-10-06T04:52:17.656205+0000","last_unstale":"2023-10-06T04:52:18.667707+0000","last_undegraded":"2023-10-06T04:52:18.667707+0000","last_fullsized":"2023-10-06T04:52:18.667707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665639+0000","last_change":"2023-10-06T04:52:18.654103+0000","last_active":"2023-10-06T04:52:18.665639+0000","last_peered":"2023-10-06T04:52:18.665639+0000","last_clean":"2023-10-06T04:52:18.665639+0000","last_became_active":"2023-10-06T04:52:18.653791+0000","last_became_peered":"2023-10-06T04:52:18.653791+0000","last_unstale":"2023-10-06T04:52:18.665639+0000","last_undegraded":"2023-10-06T04:52:18.665639+0000","last_fullsized":"2023-10-06T04:52:18.665639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:18.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667657+0000","last_change":"2023-10-06T04:52:17.656962+0000","last_active":"2023-10-06T04:52:18.667657+0000","last_peered":"2023-10-06T04:52:18.667657+0000","last_clean":"2023-10-06T04:52:18.667657+0000","last_became_active":"2023-10-06T04:52:17.656699+0000","last_became_peered":"2023-10-06T04:52:17.656699+0000","last_unstale":"2023-10-06T04:52:18.667657+0000","last_undegraded":"2023-10-06T04:52:18.667657+0000","last_fullsized":"2023-10-06T04:52:18.667657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665510+0000","last_change":"2023-10-06T04:52:18.653480+0000","last_active":"2023-10-06T04:52:18.665510+0000","last_peered":"2023-10-06T04:52:18.665510+0000","last_clean":"2023-10-06T04:52:18.665510+0000","last_became_active":"2023-10-06T04:52:18.653200+0000","last_became_peered":"2023-10-06T04:52:18.653200+0000","last_unstale":"2023-10-06T04:52:18.665510+0000","last_undegraded":"2023-10-06T04:52:18.665510+0000","last_fullsized":"2023-10-06T04:52:18.665510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:51.240487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663382+0000","last_change":"2023-10-06T04:52:17.651435+0000","last_active":"2023-10-06T04:52:18.663382+0000","last_peered":"2023-10-06T04:52:18.663382+0000","last_clean":"2023-10-06T04:52:18.663382+0000","last_became_active":"2023-10-06T04:52:17.651063+0000","last_became_peered":"2023-10-06T04:52:17.651063+0000","last_unstale":"2023-10-06T04:52:18.663382+0000","last_undegraded":"2023-10-06T04:52:18.663382+0000","last_fullsized":"2023-10-06T04:52:18.663382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664028+0000","last_change":"2023-10-06T04:52:18.658488+0000","last_active":"2023-10-06T04:52:18.664028+0000","last_peered":"2023-10-06T04:52:18.664028+0000","last_clean":"2023-10-06T04:52:18.664028+0000","last_became_active":"2023-10-06T04:52:18.657977+0000","last_became_peered":"2023-10-06T04:52:18.657977+0000","last_unstale":"2023-10-06T04:52:18.664028+0000","last_undegraded":"2023-10-06T04:52:18.664028+0000","last_fullsized":"2023-10-06T04:52:18.664028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661377+0000","last_change":"2023-10-06T04:52:17.652321+0000","last_active":"2023-10-06T04:52:18.661377+0000","last_peered":"2023-10-06T04:52:18.661377+0000","last_clean":"2023-10-06T04:52:18.661377+0000","last_became_active":"2023-10-06T04:52:17.651998+0000","last_became_peered":"2023-10-06T04:52:17.651998+0000","last_unstale":"2023-10-06T04:52:18.661377+0000","last_undegraded":"2023-10-06T04:52:18.661377+0000","last_fullsized":"2023-10-06T04:52:18.661377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666345+0000","last_change":"2023-10-06T04:52:18.664061+0000","last_active":"2023-10-06T04:52:18.666345+0000","last_peered":"2023-10-06T04:52:18.666345+0000","last_clean":"2023-10-06T04:52:18.666345+0000","last_became_active":"2023-10-06T04:52:18.663769+0000","last_became_peered":"2023-10-06T04:52:18.663769+0000","last_unstale":"2023-10-06T04:52:18.666345+0000","last_undegraded":"2023-10-06T04:52:18.666345+0000","last_fullsized":"2023-10-06T04:52:18.666345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662900+0000","last_change":"2023-10-06T04:52:18.662900+0000","last_active":"2023-10-06T04:52:18.662900+0000","last_peered":"2023-10-06T04:52:18.662900+0000","last_clean":"2023-10-06T04:52:18.662900+0000","last_became_active":"2023-10-06T04:52:18.662558+0000","last_became_peered":"2023-10-06T04:52:18.662558+0000","last_unstale":"2023-10-06T04:52:18.662900+0000","last_undegraded":"2023-10-06T04:52:18.662900+0000","last_fullsized":"2023-10-06T04:52:18.662900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:18.313966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664001+0000","last_change":"2023-10-06T04:52:18.658296+0000","last_active":"2023-10-06T04:52:18.664001+0000","last_peered":"2023-10-06T04:52:18.664001+0000","last_clean":"2023-10-06T04:52:18.664001+0000","last_became_active":"2023-10-06T04:52:18.657589+0000","last_became_peered":"2023-10-06T04:52:18.657589+0000","last_unstale":"2023-10-06T04:52:18.664001+0000","last_undegraded":"2023-10-06T04:52:18.664001+0000","last_fullsized":"2023-10-06T04:52:18.664001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666380+0000","last_change":"2023-10-06T04:52:18.663851+0000","last_active":"2023-10-06T04:52:18.666380+0000","last_peered":"2023-10-06T04:52:18.666380+0000","last_clean":"2023-10-06T04:52:18.666380+0000","last_became_active":"2023-10-06T04:52:18.663574+0000","last_became_peered":"2023-10-06T04:52:18.663574+0000","last_unstale":"2023-10-06T04:52:18.666380+0000","last_undegraded":"2023-10-06T04:52:18.666380+0000","last_fullsized":"2023-10-06T04:52:18.666380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:54.373475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656769+0000","last_change":"2023-10-06T04:52:18.652292+0000","last_active":"2023-10-06T04:52:18.656769+0000","last_peered":"2023-10-06T04:52:18.656769+0000","last_clean":"2023-10-06T04:52:18.656769+0000","last_became_active":"2023-10-06T04:52:18.651919+0000","last_became_peered":"2023-10-06T04:52:18.651919+0000","last_unstale":"2023-10-06T04:52:18.656769+0000","last_undegraded":"2023-10-06T04:52:18.656769+0000","last_fullsized":"2023-10-06T04:52:18.656769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650895+0000","last_change":"2023-10-06T04:52:18.650895+0000","last_active":"2023-10-06T04:52:18.650895+0000","last_peered":"2023-10-06T04:52:18.650895+0000","last_clean":"2023-10-06T04:52:18.650895+0000","last_became_active":"2023-10-06T04:52:18.650623+0000","last_became_peered":"2023-10-06T04:52:18.650623+0000","last_unstale":"2023-10-06T04:52:18.650895+0000","last_undegraded":"2023-10-06T04:52:18.650895+0000","last_fullsized":"2023-10-06T04:52:18.650895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":658310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":782336,"data_stored":656558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":196608,"data_stored":66814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:52:22.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:52:22.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.916+0000 7f0e0d537700 1 Processor -- start 2023-10-06T04:52:22.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.916+0000 7f0e0d537700 1 -- start start 2023-10-06T04:52:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e0d537700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e080af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e0d537700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e0806b7d0 0x7f0e080c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e0d537700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e080b1c90 con 0x7f0e080b0900 2023-10-06T04:52:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e0d537700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e080b1080 con 0x7f0e0806b7d0 2023-10-06T04:52:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e0d537700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080b1360 con 0x7f0e08156410 2023-10-06T04:52:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e080af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e077fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e0806b7d0 0x7f0e080c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e077fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e0806b7d0 0x7f0e080c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:37724/0 (socket says 172.21.15.3:37724) 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e080af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59306/0 (socket says 172.21.15.3:59306) 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.917+0000 7f0e077fe700 1 -- 172.21.15.3:0/1485588639 learned_addr learned my addr 172.21.15.3:0/1485588639 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e06ffd700 1 -- 172.21.15.3:0/1485588639 >> v2:172.21.15.153:3300/0 conn(0x7f0e0806b7d0 msgr2=0x7f0e080c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/1485588639 >> v2:172.21.15.153:3300/0 conn(0x7f0e0806b7d0 0x7f0e080c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e06ffd700 1 -- 172.21.15.3:0/1485588639 >> v1:172.21.15.153:6789/0 conn(0x7f0e080b0900 legacy=0x7f0e0806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e06ffd700 1 -- 172.21.15.3:0/1485588639 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e080b1640 con 0x7f0e08156410 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e077fe700 1 --2- 172.21.15.3:0/1485588639 >> v2:172.21.15.153:3300/0 conn(0x7f0e0806b7d0 0x7f0e080c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e05ffb700 1 -- 172.21.15.3:0/1485588639 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287292341 0 0) 0x7f0e080b1c90 con 0x7f0e080b0900 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/1485588639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e080af480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009e60 tx=0x7f0e00009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592d908c1824ce8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e05ffb700 1 -- 172.21.15.3:0/1485588639 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000e070 con 0x7f0e08156410 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e05ffb700 1 -- 172.21.15.3:0/1485588639 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000bb60 con 0x7f0e08156410 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.918+0000 7f0e05ffb700 1 -- 172.21.15.3:0/1485588639 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00012430 con 0x7f0e08156410 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 -- 172.21.15.3:0/1485588639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 msgr2=0x7f0e080af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 --2- 172.21.15.3:0/1485588639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e080af480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009e60 tx=0x7f0e00009780 comp rx=0 tx=0).stop 2023-10-06T04:52:22.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 -- 172.21.15.3:0/1485588639 shutdown_connections 2023-10-06T04:52:22.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 --2- 172.21.15.3:0/1485588639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e080af480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 --2- 172.21.15.3:0/1485588639 >> v2:172.21.15.153:3300/0 conn(0x7f0e0806b7d0 0x7f0e080c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 -- 172.21.15.3:0/1485588639 >> 172.21.15.3:0/1485588639 conn(0x7f0e080cdc30 msgr2=0x7f0e0806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:22.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 -- 172.21.15.3:0/1485588639 shutdown_connections 2023-10-06T04:52:22.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.919+0000 7f0e0d537700 1 -- 172.21.15.3:0/1485588639 wait complete. 2023-10-06T04:52:22.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.920+0000 7f0e0d537700 1 Processor -- start 2023-10-06T04:52:22.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.920+0000 7f0e0d537700 1 -- start start 2023-10-06T04:52:22.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e0d537700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e0806b7d0 0x7f0e0816d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e0d537700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e080b0900 0x7f0e0806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e0d537700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e08158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e0d537700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0816dea0 con 0x7f0e080b0900 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e0d537700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08158a70 con 0x7f0e0806b7d0 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e0d537700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080c8b70 con 0x7f0e08156410 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e080b0900 0x7f0e0806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e077fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e08158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e0806b7d0 0x7f0e0816d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e080b0900 0x7f0e0806ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32896/0 (socket says 172.21.15.3:32896) 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.921+0000 7f0e067fc700 1 -- 172.21.15.3:0/3623328852 learned_addr learned my addr 172.21.15.3:0/3623328852 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e077fe700 1 -- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e0806b7d0 msgr2=0x7f0e0816d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e077fe700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e0806b7d0 0x7f0e0816d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e077fe700 1 -- 172.21.15.3:0/3623328852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e080b0900 msgr2=0x7f0e0806ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:22.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e077fe700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e080b0900 0x7f0e0806ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:22.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e077fe700 1 -- 172.21.15.3:0/3623328852 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df80094f0 con 0x7f0e08156410 2023-10-06T04:52:22.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e0806b7d0 0x7f0e0816d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:22.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e077fe700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e08158f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0df8006fd0 tx=0x7f0df8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0dfeffd700 1 -- 172.21.15.3:0/3623328852 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8013680 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0dfeffd700 1 -- 172.21.15.3:0/3623328852 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df80137e0 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e00009a20 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0dfeffd700 1 -- 172.21.15.3:0/3623328852 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8013ba0 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.922+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e08158700 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.923+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e08071680 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.924+0000 7f0dfeffd700 1 -- 172.21.15.3:0/3623328852 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0df800c3e0 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.924+0000 7f0dfeffd700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.924+0000 7f0dfeffd700 1 -- 172.21.15.3:0/3623328852 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0df80847d0 con 0x7f0e08156410 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.925+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:22.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.925+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0de8066f30 0x7f0de80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e000097c0 tx=0x7f0e00019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:22.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:22.929+0000 7f0dfeffd700 1 -- 172.21.15.3:0/3623328852 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0df80518e0 con 0x7f0e08156410 2023-10-06T04:52:23.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.117+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e08158bb0 con 0x7f0de8066f30 2023-10-06T04:52:23.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.126+0000 7f0dfeffd700 1 -- 172.21.15.3:0/3623328852 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286900 (secure 0 0 0) 0x7f0e08158bb0 con 0x7f0de8066f30 2023-10-06T04:52:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0de8066f30 msgr2=0x7f0de80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0de8066f30 0x7f0de80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e000097c0 tx=0x7f0e00019000 comp rx=0 tx=0).stop 2023-10-06T04:52:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 msgr2=0x7f0e08158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e08158f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0df8006fd0 tx=0x7f0df8009250 comp rx=0 tx=0).stop 2023-10-06T04:52:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 shutdown_connections 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e0806b7d0 0x7f0e0816d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e08156410 0x7f0e08158f00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 --2- 172.21.15.3:0/3623328852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e080b0900 0x7f0e0806ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 >> 172.21.15.3:0/3623328852 conn(0x7f0e080cdc30 msgr2=0x7f0e080c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 shutdown_connections 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.129+0000 7f0e0d537700 1 -- 172.21.15.3:0/3623328852 wait complete. 2023-10-06T04:52:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:52:23.143 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-06T04:52:21.423787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381056,"kb_used_data":2752,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593664,"statfs":{"total":773094113280,"available":770655911936,"internally_reserved":0,"allocated":2818048,"data_stored":1741556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.797012"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658025+0000","last_change":"2023-10-06T04:52:18.658025+0000","last_active":"2023-10-06T04:52:18.658025+0000","last_peered":"2023-10-06T04:52:18.658025+0000","last_clean":"2023-10-06T04:52:18.658025+0000","last_became_active":"2023-10-06T04:52:18.657724+0000","last_became_peered":"2023-10-06T04:52:18.657724+0000","last_unstale":"2023-10-06T04:52:18.658025+0000","last_undegraded":"2023-10-06T04:52:18.658025+0000","last_fullsized":"2023-10-06T04:52:18.658025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:01.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658607+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.658607+0000","last_peered":"2023-10-06T04:52:18.658607+0000","last_clean":"2023-10-06T04:52:18.658607+0000","last_became_active":"2023-10-06T04:52:18.658286+0000","last_became_peered":"2023-10-06T04:52:18.658286+0000","last_unstale":"2023-10-06T04:52:18.658607+0000","last_undegraded":"2023-10-06T04:52:18.658607+0000","last_fullsized":"2023-10-06T04:52:18.658607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665708+0000","last_change":"2023-10-06T04:52:18.652791+0000","last_active":"2023-10-06T04:52:18.665708+0000","last_peered":"2023-10-06T04:52:18.665708+0000","last_clean":"2023-10-06T04:52:18.665708+0000","last_became_active":"2023-10-06T04:52:18.652522+0000","last_became_peered":"2023-10-06T04:52:18.652522+0000","last_unstale":"2023-10-06T04:52:18.665708+0000","last_undegraded":"2023-10-06T04:52:18.665708+0000","last_fullsized":"2023-10-06T04:52:18.665708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.649578+0000","last_change":"2023-10-06T04:52:18.649578+0000","last_active":"2023-10-06T04:52:18.649578+0000","last_peered":"2023-10-06T04:52:18.649578+0000","last_clean":"2023-10-06T04:52:18.649578+0000","last_became_active":"2023-10-06T04:52:18.649294+0000","last_became_peered":"2023-10-06T04:52:18.649294+0000","last_unstale":"2023-10-06T04:52:18.649578+0000","last_undegraded":"2023-10-06T04:52:18.649578+0000","last_fullsized":"2023-10-06T04:52:18.649578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:28.571226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658392+0000","last_change":"2023-10-06T04:52:18.653657+0000","last_active":"2023-10-06T04:52:18.658392+0000","last_peered":"2023-10-06T04:52:18.658392+0000","last_clean":"2023-10-06T04:52:18.658392+0000","last_became_active":"2023-10-06T04:52:18.653363+0000","last_became_peered":"2023-10-06T04:52:18.653363+0000","last_unstale":"2023-10-06T04:52:18.658392+0000","last_undegraded":"2023-10-06T04:52:18.658392+0000","last_fullsized":"2023-10-06T04:52:18.658392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665600+0000","last_change":"2023-10-06T04:52:18.654512+0000","last_active":"2023-10-06T04:52:18.665600+0000","last_peered":"2023-10-06T04:52:18.665600+0000","last_clean":"2023-10-06T04:52:18.665600+0000","last_became_active":"2023-10-06T04:52:18.653897+0000","last_became_peered":"2023-10-06T04:52:18.653897+0000","last_unstale":"2023-10-06T04:52:18.665600+0000","last_undegraded":"2023-10-06T04:52:18.665600+0000","last_fullsized":"2023-10-06T04:52:18.665600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:38.713099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655144+0000","last_change":"2023-10-06T04:52:18.655144+0000","last_active":"2023-10-06T04:52:18.655144+0000","last_peered":"2023-10-06T04:52:18.655144+0000","last_clean":"2023-10-06T04:52:18.655144+0000","last_became_active":"2023-10-06T04:52:18.654891+0000","last_became_peered":"2023-10-06T04:52:18.654891+0000","last_unstale":"2023-10-06T04:52:18.655144+0000","last_undegraded":"2023-10-06T04:52:18.655144+0000","last_fullsized":"2023-10-06T04:52:18.655144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666282+0000","last_change":"2023-10-06T04:52:18.666282+0000","last_active":"2023-10-06T04:52:18.666282+0000","last_peered":"2023-10-06T04:52:18.666282+0000","last_clean":"2023-10-06T04:52:18.666282+0000","last_became_active":"2023-10-06T04:52:18.665988+0000","last_became_peered":"2023-10-06T04:52:18.665988+0000","last_unstale":"2023-10-06T04:52:18.666282+0000","last_undegraded":"2023-10-06T04:52:18.666282+0000","last_fullsized":"2023-10-06T04:52:18.666282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665701+0000","last_change":"2023-10-06T04:52:18.661193+0000","last_active":"2023-10-06T04:52:18.665701+0000","last_peered":"2023-10-06T04:52:18.665701+0000","last_clean":"2023-10-06T04:52:18.665701+0000","last_became_active":"2023-10-06T04:52:18.660905+0000","last_became_peered":"2023-10-06T04:52:18.660905+0000","last_unstale":"2023-10-06T04:52:18.665701+0000","last_undegraded":"2023-10-06T04:52:18.665701+0000","last_fullsized":"2023-10-06T04:52:18.665701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.654460+0000","last_change":"2023-10-06T04:52:18.654460+0000","last_active":"2023-10-06T04:52:18.654460+0000","last_peered":"2023-10-06T04:52:18.654460+0000","last_clean":"2023-10-06T04:52:18.654460+0000","last_became_active":"2023-10-06T04:52:18.653370+0000","last_became_peered":"2023-10-06T04:52:18.653370+0000","last_unstale":"2023-10-06T04:52:18.654460+0000","last_undegraded":"2023-10-06T04:52:18.654460+0000","last_fullsized":"2023-10-06T04:52:18.654460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659795+0000","last_change":"2023-10-06T04:52:18.650974+0000","last_active":"2023-10-06T04:52:18.659795+0000","last_peered":"2023-10-06T04:52:18.659795+0000","last_clean":"2023-10-06T04:52:18.659795+0000","last_became_active":"2023-10-06T04:52:18.650708+0000","last_became_peered":"2023-10-06T04:52:18.650708+0000","last_unstale":"2023-10-06T04:52:18.659795+0000","last_undegraded":"2023-10-06T04:52:18.659795+0000","last_fullsized":"2023-10-06T04:52:18.659795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.651167+0000","last_change":"2023-10-06T04:52:18.651167+0000","last_active":"2023-10-06T04:52:18.651167+0000","last_peered":"2023-10-06T04:52:18.651167+0000","last_clean":"2023-10-06T04:52:18.651167+0000","last_became_active":"2023-10-06T04:52:18.650869+0000","last_became_peered":"2023-10-06T04:52:18.650869+0000","last_unstale":"2023-10-06T04:52:18.651167+0000","last_undegraded":"2023-10-06T04:52:18.651167+0000","last_fullsized":"2023-10-06T04:52:18.651167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:55.372045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.653721+0000","last_change":"2023-10-06T04:52:18.653721+0000","last_active":"2023-10-06T04:52:18.653721+0000","last_peered":"2023-10-06T04:52:18.653721+0000","last_clean":"2023-10-06T04:52:18.653721+0000","last_became_active":"2023-10-06T04:52:18.653435+0000","last_became_peered":"2023-10-06T04:52:18.653435+0000","last_unstale":"2023-10-06T04:52:18.653721+0000","last_undegraded":"2023-10-06T04:52:18.653721+0000","last_fullsized":"2023-10-06T04:52:18.653721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:43.201875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655632+0000","last_change":"2023-10-06T04:52:18.655632+0000","last_active":"2023-10-06T04:52:18.655632+0000","last_peered":"2023-10-06T04:52:18.655632+0000","last_clean":"2023-10-06T04:52:18.655632+0000","last_became_active":"2023-10-06T04:52:18.655335+0000","last_became_peered":"2023-10-06T04:52:18.655335+0000","last_unstale":"2023-10-06T04:52:18.655632+0000","last_undegraded":"2023-10-06T04:52:18.655632+0000","last_fullsized":"2023-10-06T04:52:18.655632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663599+0000","last_change":"2023-10-06T04:52:18.663599+0000","last_active":"2023-10-06T04:52:18.663599+0000","last_peered":"2023-10-06T04:52:18.663599+0000","last_clean":"2023-10-06T04:52:18.663599+0000","last_became_active":"2023-10-06T04:52:18.663303+0000","last_became_peered":"2023-10-06T04:52:18.663303+0000","last_unstale":"2023-10-06T04:52:18.663599+0000","last_undegraded":"2023-10-06T04:52:18.663599+0000","last_fullsized":"2023-10-06T04:52:18.663599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665493+0000","last_change":"2023-10-06T04:52:18.652910+0000","last_active":"2023-10-06T04:52:18.665493+0000","last_peered":"2023-10-06T04:52:18.665493+0000","last_clean":"2023-10-06T04:52:18.665493+0000","last_became_active":"2023-10-06T04:52:18.652589+0000","last_became_peered":"2023-10-06T04:52:18.652589+0000","last_unstale":"2023-10-06T04:52:18.665493+0000","last_undegraded":"2023-10-06T04:52:18.665493+0000","last_fullsized":"2023-10-06T04:52:18.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:10.947807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658434+0000","last_change":"2023-10-06T04:52:18.654868+0000","last_active":"2023-10-06T04:52:18.658434+0000","last_peered":"2023-10-06T04:52:18.658434+0000","last_clean":"2023-10-06T04:52:18.658434+0000","last_became_active":"2023-10-06T04:52:18.654590+0000","last_became_peered":"2023-10-06T04:52:18.654590+0000","last_unstale":"2023-10-06T04:52:18.658434+0000","last_undegraded":"2023-10-06T04:52:18.658434+0000","last_fullsized":"2023-10-06T04:52:18.658434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665816+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.665816+0000","last_peered":"2023-10-06T04:52:18.665816+0000","last_clean":"2023-10-06T04:52:18.665816+0000","last_became_active":"2023-10-06T04:52:18.658358+0000","last_became_peered":"2023-10-06T04:52:18.658358+0000","last_unstale":"2023-10-06T04:52:18.665816+0000","last_undegraded":"2023-10-06T04:52:18.665816+0000","last_fullsized":"2023-10-06T04:52:18.665816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662801+0000","last_change":"2023-10-06T04:52:18.662801+0000","last_active":"2023-10-06T04:52:18.662801+0000","last_peered":"2023-10-06T04:52:18.662801+0000","last_clean":"2023-10-06T04:52:18.662801+0000","last_became_active":"2023-10-06T04:52:18.662534+0000","last_became_peered":"2023-10-06T04:52:18.662534+0000","last_unstale":"2023-10-06T04:52:18.662801+0000","last_undegraded":"2023-10-06T04:52:18.662801+0000","last_fullsized":"2023-10-06T04:52:18.662801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:53.970289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666479+0000","last_change":"2023-10-06T04:52:18.664643+0000","last_active":"2023-10-06T04:52:18.666479+0000","last_peered":"2023-10-06T04:52:18.666479+0000","last_clean":"2023-10-06T04:52:18.666479+0000","last_became_active":"2023-10-06T04:52:18.664342+0000","last_became_peered":"2023-10-06T04:52:18.664342+0000","last_unstale":"2023-10-06T04:52:18.666479+0000","last_undegraded":"2023-10-06T04:52:18.666479+0000","last_fullsized":"2023-10-06T04:52:18.666479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:33.202071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663114+0000","last_change":"2023-10-06T04:52:18.663114+0000","last_active":"2023-10-06T04:52:18.663114+0000","last_peered":"2023-10-06T04:52:18.663114+0000","last_clean":"2023-10-06T04:52:18.663114+0000","last_became_active":"2023-10-06T04:52:18.662825+0000","last_became_peered":"2023-10-06T04:52:18.662825+0000","last_unstale":"2023-10-06T04:52:18.663114+0000","last_undegraded":"2023-10-06T04:52:18.663114+0000","last_fullsized":"2023-10-06T04:52:18.663114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650717+0000","last_change":"2023-10-06T04:52:18.648283+0000","last_active":"2023-10-06T04:52:18.650717+0000","last_peered":"2023-10-06T04:52:18.650717+0000","last_clean":"2023-10-06T04:52:18.650717+0000","last_became_active":"2023-10-06T04:52:18.647966+0000","last_became_peered":"2023-10-06T04:52:18.647966+0000","last_unstale":"2023-10-06T04:52:18.650717+0000","last_undegraded":"2023-10-06T04:52:18.650717+0000","last_fullsized":"2023-10-06T04:52:18.650717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661203+0000","last_change":"2023-10-06T04:52:18.655500+0000","last_active":"2023-10-06T04:52:18.661203+0000","last_peered":"2023-10-06T04:52:18.661203+0000","last_clean":"2023-10-06T04:52:18.661203+0000","last_became_active":"2023-10-06T04:52:18.655234+0000","last_became_peered":"2023-10-06T04:52:18.655234+0000","last_unstale":"2023-10-06T04:52:18.661203+0000","last_undegraded":"2023-10-06T04:52:18.661203+0000","last_fullsized":"2023-10-06T04:52:18.661203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660815+0000","last_change":"2023-10-06T04:52:18.660815+0000","last_active":"2023-10-06T04:52:18.660815+0000","last_peered":"2023-10-06T04:52:18.660815+0000","last_clean":"2023-10-06T04:52:18.660815+0000","last_became_active":"2023-10-06T04:52:18.660475+0000","last_became_peered":"2023-10-06T04:52:18.660475+0000","last_unstale":"2023-10-06T04:52:18.660815+0000","last_undegraded":"2023-10-06T04:52:18.660815+0000","last_fullsized":"2023-10-06T04:52:18.660815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663035+0000","last_change":"2023-10-06T04:52:18.663035+0000","last_active":"2023-10-06T04:52:18.663035+0000","last_peered":"2023-10-06T04:52:18.663035+0000","last_clean":"2023-10-06T04:52:18.663035+0000","last_became_active":"2023-10-06T04:52:18.662611+0000","last_became_peered":"2023-10-06T04:52:18.662611+0000","last_unstale":"2023-10-06T04:52:18.663035+0000","last_undegraded":"2023-10-06T04:52:18.663035+0000","last_fullsized":"2023-10-06T04:52:18.663035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658476+0000","last_change":"2023-10-06T04:52:18.658476+0000","last_active":"2023-10-06T04:52:18.658476+0000","last_peered":"2023-10-06T04:52:18.658476+0000","last_clean":"2023-10-06T04:52:18.658476+0000","last_became_active":"2023-10-06T04:52:18.658197+0000","last_became_peered":"2023-10-06T04:52:18.658197+0000","last_unstale":"2023-10-06T04:52:18.658476+0000","last_undegraded":"2023-10-06T04:52:18.658476+0000","last_fullsized":"2023-10-06T04:52:18.658476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658955+0000","last_change":"2023-10-06T04:52:18.658955+0000","last_active":"2023-10-06T04:52:18.658955+0000","last_peered":"2023-10-06T04:52:18.658955+0000","last_clean":"2023-10-06T04:52:18.658955+0000","last_became_active":"2023-10-06T04:52:18.658713+0000","last_became_peered":"2023-10-06T04:52:18.658713+0000","last_unstale":"2023-10-06T04:52:18.658955+0000","last_undegraded":"2023-10-06T04:52:18.658955+0000","last_fullsized":"2023-10-06T04:52:18.658955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.651332+0000","last_change":"2023-10-06T04:52:17.656516+0000","last_active":"2023-10-06T04:52:18.651332+0000","last_peered":"2023-10-06T04:52:18.651332+0000","last_clean":"2023-10-06T04:52:18.651332+0000","last_became_active":"2023-10-06T04:52:17.656032+0000","last_became_peered":"2023-10-06T04:52:17.656032+0000","last_unstale":"2023-10-06T04:52:18.651332+0000","last_undegraded":"2023-10-06T04:52:18.651332+0000","last_fullsized":"2023-10-06T04:52:18.651332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660104+0000","last_change":"2023-10-06T04:52:18.656019+0000","last_active":"2023-10-06T04:52:18.660104+0000","last_peered":"2023-10-06T04:52:18.660104+0000","last_clean":"2023-10-06T04:52:18.660104+0000","last_became_active":"2023-10-06T04:52:18.655673+0000","last_became_peered":"2023-10-06T04:52:18.655673+0000","last_unstale":"2023-10-06T04:52:18.660104+0000","last_undegraded":"2023-10-06T04:52:18.660104+0000","last_fullsized":"2023-10-06T04:52:18.660104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.651598+0000","last_change":"2023-10-06T04:52:17.645852+0000","last_active":"2023-10-06T04:52:17.651598+0000","last_peered":"2023-10-06T04:52:17.651598+0000","last_clean":"2023-10-06T04:52:17.651598+0000","last_became_active":"2023-10-06T04:52:17.645528+0000","last_became_peered":"2023-10-06T04:52:17.645528+0000","last_unstale":"2023-10-06T04:52:17.651598+0000","last_undegraded":"2023-10-06T04:52:17.651598+0000","last_fullsized":"2023-10-06T04:52:17.651598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655629+0000","last_change":"2023-10-06T04:52:18.649691+0000","last_active":"2023-10-06T04:52:18.655629+0000","last_peered":"2023-10-06T04:52:18.655629+0000","last_clean":"2023-10-06T04:52:18.655629+0000","last_became_active":"2023-10-06T04:52:18.649348+0000","last_became_peered":"2023-10-06T04:52:18.649348+0000","last_unstale":"2023-10-06T04:52:18.655629+0000","last_undegraded":"2023-10-06T04:52:18.655629+0000","last_fullsized":"2023-10-06T04:52:18.655629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643218+0000","last_change":"2023-10-06T04:52:17.651301+0000","last_active":"2023-10-06T04:52:18.643218+0000","last_peered":"2023-10-06T04:52:18.643218+0000","last_clean":"2023-10-06T04:52:18.643218+0000","last_became_active":"2023-10-06T04:52:17.650967+0000","last_became_peered":"2023-10-06T04:52:17.650967+0000","last_unstale":"2023-10-06T04:52:18.643218+0000","last_undegraded":"2023-10-06T04:52:18.643218+0000","last_fullsized":"2023-10-06T04:52:18.643218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656399+0000","last_change":"2023-10-06T04:52:18.656399+0000","last_active":"2023-10-06T04:52:18.656399+0000","last_peered":"2023-10-06T04:52:18.656399+0000","last_clean":"2023-10-06T04:52:18.656399+0000","last_became_active":"2023-10-06T04:52:18.656106+0000","last_became_peered":"2023-10-06T04:52:18.656106+0000","last_unstale":"2023-10-06T04:52:18.656399+0000","last_undegraded":"2023-10-06T04:52:18.656399+0000","last_fullsized":"2023-10-06T04:52:18.656399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.640098+0000","last_change":"2023-10-06T04:52:17.654663+0000","last_active":"2023-10-06T04:52:18.640098+0000","last_peered":"2023-10-06T04:52:18.640098+0000","last_clean":"2023-10-06T04:52:18.640098+0000","last_became_active":"2023-10-06T04:52:17.654402+0000","last_became_peered":"2023-10-06T04:52:17.654402+0000","last_unstale":"2023-10-06T04:52:18.640098+0000","last_undegraded":"2023-10-06T04:52:18.640098+0000","last_fullsized":"2023-10-06T04:52:18.640098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660162+0000","last_change":"2023-10-06T04:52:18.655602+0000","last_active":"2023-10-06T04:52:18.660162+0000","last_peered":"2023-10-06T04:52:18.660162+0000","last_clean":"2023-10-06T04:52:18.660162+0000","last_became_active":"2023-10-06T04:52:18.655329+0000","last_became_peered":"2023-10-06T04:52:18.655329+0000","last_unstale":"2023-10-06T04:52:18.660162+0000","last_undegraded":"2023-10-06T04:52:18.660162+0000","last_fullsized":"2023-10-06T04:52:18.660162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.659765+0000","last_change":"2023-10-06T04:52:17.652631+0000","last_active":"2023-10-06T04:52:17.659765+0000","last_peered":"2023-10-06T04:52:17.659765+0000","last_clean":"2023-10-06T04:52:17.659765+0000","last_became_active":"2023-10-06T04:52:17.652340+0000","last_became_peered":"2023-10-06T04:52:17.652340+0000","last_unstale":"2023-10-06T04:52:17.659765+0000","last_undegraded":"2023-10-06T04:52:17.659765+0000","last_fullsized":"2023-10-06T04:52:17.659765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659061+0000","last_change":"2023-10-06T04:52:18.651950+0000","last_active":"2023-10-06T04:52:18.659061+0000","last_peered":"2023-10-06T04:52:18.659061+0000","last_clean":"2023-10-06T04:52:18.659061+0000","last_became_active":"2023-10-06T04:52:18.651587+0000","last_became_peered":"2023-10-06T04:52:18.651587+0000","last_unstale":"2023-10-06T04:52:18.659061+0000","last_undegraded":"2023-10-06T04:52:18.659061+0000","last_fullsized":"2023-10-06T04:52:18.659061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667304+0000","last_change":"2023-10-06T04:52:17.655489+0000","last_active":"2023-10-06T04:52:18.667304+0000","last_peered":"2023-10-06T04:52:18.667304+0000","last_clean":"2023-10-06T04:52:18.667304+0000","last_became_active":"2023-10-06T04:52:17.655131+0000","last_became_peered":"2023-10-06T04:52:17.655131+0000","last_unstale":"2023-10-06T04:52:18.667304+0000","last_undegraded":"2023-10-06T04:52:18.667304+0000","last_fullsized":"2023-10-06T04:52:18.667304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662999+0000","last_change":"2023-10-06T04:52:18.662999+0000","last_active":"2023-10-06T04:52:18.662999+0000","last_peered":"2023-10-06T04:52:18.662999+0000","last_clean":"2023-10-06T04:52:18.662999+0000","last_became_active":"2023-10-06T04:52:18.662603+0000","last_became_peered":"2023-10-06T04:52:18.662603+0000","last_unstale":"2023-10-06T04:52:18.662999+0000","last_undegraded":"2023-10-06T04:52:18.662999+0000","last_fullsized":"2023-10-06T04:52:18.662999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.669196+0000","last_change":"2023-10-06T04:52:17.655293+0000","last_active":"2023-10-06T04:52:18.669196+0000","last_peered":"2023-10-06T04:52:18.669196+0000","last_clean":"2023-10-06T04:52:18.669196+0000","last_became_active":"2023-10-06T04:52:17.654390+0000","last_became_peered":"2023-10-06T04:52:17.654390+0000","last_unstale":"2023-10-06T04:52:18.669196+0000","last_undegraded":"2023-10-06T04:52:18.669196+0000","last_fullsized":"2023-10-06T04:52:18.669196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665827+0000","last_change":"2023-10-06T04:52:18.657322+0000","last_active":"2023-10-06T04:52:18.665827+0000","last_peered":"2023-10-06T04:52:18.665827+0000","last_clean":"2023-10-06T04:52:18.665827+0000","last_became_active":"2023-10-06T04:52:18.657046+0000","last_became_peered":"2023-10-06T04:52:18.657046+0000","last_unstale":"2023-10-06T04:52:18.665827+0000","last_undegraded":"2023-10-06T04:52:18.665827+0000","last_fullsized":"2023-10-06T04:52:18.665827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659746+0000","last_change":"2023-10-06T04:52:17.657213+0000","last_active":"2023-10-06T04:52:18.659746+0000","last_peered":"2023-10-06T04:52:18.659746+0000","last_clean":"2023-10-06T04:52:18.659746+0000","last_became_active":"2023-10-06T04:52:17.656945+0000","last_became_peered":"2023-10-06T04:52:17.656945+0000","last_unstale":"2023-10-06T04:52:18.659746+0000","last_undegraded":"2023-10-06T04:52:18.659746+0000","last_fullsized":"2023-10-06T04:52:18.659746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:17.302823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659796+0000","last_change":"2023-10-06T04:52:18.655013+0000","last_active":"2023-10-06T04:52:18.659796+0000","last_peered":"2023-10-06T04:52:18.659796+0000","last_clean":"2023-10-06T04:52:18.659796+0000","last_became_active":"2023-10-06T04:52:18.654734+0000","last_became_peered":"2023-10-06T04:52:18.654734+0000","last_unstale":"2023-10-06T04:52:18.659796+0000","last_undegraded":"2023-10-06T04:52:18.659796+0000","last_fullsized":"2023-10-06T04:52:18.659796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638374+0000","last_change":"2023-10-06T04:52:17.650696+0000","last_active":"2023-10-06T04:52:18.638374+0000","last_peered":"2023-10-06T04:52:18.638374+0000","last_clean":"2023-10-06T04:52:18.638374+0000","last_became_active":"2023-10-06T04:52:17.650402+0000","last_became_peered":"2023-10-06T04:52:17.650402+0000","last_unstale":"2023-10-06T04:52:18.638374+0000","last_undegraded":"2023-10-06T04:52:18.638374+0000","last_fullsized":"2023-10-06T04:52:18.638374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658995+0000","last_change":"2023-10-06T04:52:18.658995+0000","last_active":"2023-10-06T04:52:18.658995+0000","last_peered":"2023-10-06T04:52:18.658995+0000","last_clean":"2023-10-06T04:52:18.658995+0000","last_became_active":"2023-10-06T04:52:18.658717+0000","last_became_peered":"2023-10-06T04:52:18.658717+0000","last_unstale":"2023-10-06T04:52:18.658995+0000","last_undegraded":"2023-10-06T04:52:18.658995+0000","last_fullsized":"2023-10-06T04:52:18.658995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.661884+0000","last_change":"2023-10-06T04:52:17.657760+0000","last_active":"2023-10-06T04:52:17.661884+0000","last_peered":"2023-10-06T04:52:17.661884+0000","last_clean":"2023-10-06T04:52:17.661884+0000","last_became_active":"2023-10-06T04:52:17.657484+0000","last_became_peered":"2023-10-06T04:52:17.657484+0000","last_unstale":"2023-10-06T04:52:17.661884+0000","last_undegraded":"2023-10-06T04:52:17.661884+0000","last_fullsized":"2023-10-06T04:52:17.661884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665775+0000","last_change":"2023-10-06T04:52:18.661243+0000","last_active":"2023-10-06T04:52:18.665775+0000","last_peered":"2023-10-06T04:52:18.665775+0000","last_clean":"2023-10-06T04:52:18.665775+0000","last_became_active":"2023-10-06T04:52:18.660932+0000","last_became_peered":"2023-10-06T04:52:18.660932+0000","last_unstale":"2023-10-06T04:52:18.665775+0000","last_undegraded":"2023-10-06T04:52:18.665775+0000","last_fullsized":"2023-10-06T04:52:18.665775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666058+0000","last_change":"2023-10-06T04:52:17.665506+0000","last_active":"2023-10-06T04:52:18.666058+0000","last_peered":"2023-10-06T04:52:18.666058+0000","last_clean":"2023-10-06T04:52:18.666058+0000","last_became_active":"2023-10-06T04:52:17.665209+0000","last_became_peered":"2023-10-06T04:52:17.665209+0000","last_unstale":"2023-10-06T04:52:18.666058+0000","last_undegraded":"2023-10-06T04:52:18.666058+0000","last_fullsized":"2023-10-06T04:52:18.666058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660252+0000","last_change":"2023-10-06T04:52:18.654716+0000","last_active":"2023-10-06T04:52:18.660252+0000","last_peered":"2023-10-06T04:52:18.660252+0000","last_clean":"2023-10-06T04:52:18.660252+0000","last_became_active":"2023-10-06T04:52:18.653774+0000","last_became_peered":"2023-10-06T04:52:18.653774+0000","last_unstale":"2023-10-06T04:52:18.660252+0000","last_undegraded":"2023-10-06T04:52:18.660252+0000","last_fullsized":"2023-10-06T04:52:18.660252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.649135+0000","last_change":"2023-10-06T04:52:17.657327+0000","last_active":"2023-10-06T04:52:18.649135+0000","last_peered":"2023-10-06T04:52:18.649135+0000","last_clean":"2023-10-06T04:52:18.649135+0000","last_became_active":"2023-10-06T04:52:17.657054+0000","last_became_peered":"2023-10-06T04:52:17.657054+0000","last_unstale":"2023-10-06T04:52:18.649135+0000","last_undegraded":"2023-10-06T04:52:18.649135+0000","last_fullsized":"2023-10-06T04:52:18.649135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.652991+0000","last_change":"2023-10-06T04:52:18.652991+0000","last_active":"2023-10-06T04:52:18.652991+0000","last_peered":"2023-10-06T04:52:18.652991+0000","last_clean":"2023-10-06T04:52:18.652991+0000","last_became_active":"2023-10-06T04:52:18.652695+0000","last_became_peered":"2023-10-06T04:52:18.652695+0000","last_unstale":"2023-10-06T04:52:18.652991+0000","last_undegraded":"2023-10-06T04:52:18.652991+0000","last_fullsized":"2023-10-06T04:52:18.652991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659840+0000","last_change":"2023-10-06T04:52:17.659228+0000","last_active":"2023-10-06T04:52:18.659840+0000","last_peered":"2023-10-06T04:52:18.659840+0000","last_clean":"2023-10-06T04:52:18.659840+0000","last_became_active":"2023-10-06T04:52:17.658947+0000","last_became_peered":"2023-10-06T04:52:17.658947+0000","last_unstale":"2023-10-06T04:52:18.659840+0000","last_undegraded":"2023-10-06T04:52:18.659840+0000","last_fullsized":"2023-10-06T04:52:18.659840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659272+0000","last_change":"2023-10-06T04:52:18.649861+0000","last_active":"2023-10-06T04:52:18.659272+0000","last_peered":"2023-10-06T04:52:18.659272+0000","last_clean":"2023-10-06T04:52:18.659272+0000","last_became_active":"2023-10-06T04:52:18.649576+0000","last_became_peered":"2023-10-06T04:52:18.649576+0000","last_unstale":"2023-10-06T04:52:18.659272+0000","last_undegraded":"2023-10-06T04:52:18.659272+0000","last_fullsized":"2023-10-06T04:52:18.659272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:14.896487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.649796+0000","last_change":"2023-10-06T04:52:17.653503+0000","last_active":"2023-10-06T04:52:18.649796+0000","last_peered":"2023-10-06T04:52:18.649796+0000","last_clean":"2023-10-06T04:52:18.649796+0000","last_became_active":"2023-10-06T04:52:17.653236+0000","last_became_peered":"2023-10-06T04:52:17.653236+0000","last_unstale":"2023-10-06T04:52:18.649796+0000","last_undegraded":"2023-10-06T04:52:18.649796+0000","last_fullsized":"2023-10-06T04:52:18.649796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:27.560082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661966+0000","last_change":"2023-10-06T04:52:18.661966+0000","last_active":"2023-10-06T04:52:18.661966+0000","last_peered":"2023-10-06T04:52:18.661966+0000","last_clean":"2023-10-06T04:52:18.661966+0000","last_became_active":"2023-10-06T04:52:18.661376+0000","last_became_peered":"2023-10-06T04:52:18.661376+0000","last_unstale":"2023-10-06T04:52:18.661966+0000","last_undegraded":"2023-10-06T04:52:18.661966+0000","last_fullsized":"2023-10-06T04:52:18.661966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652153+0000","last_change":"2023-10-06T04:52:17.650123+0000","last_active":"2023-10-06T04:52:17.652153+0000","last_peered":"2023-10-06T04:52:17.652153+0000","last_clean":"2023-10-06T04:52:17.652153+0000","last_became_active":"2023-10-06T04:52:17.649388+0000","last_became_peered":"2023-10-06T04:52:17.649388+0000","last_unstale":"2023-10-06T04:52:17.652153+0000","last_undegraded":"2023-10-06T04:52:17.652153+0000","last_fullsized":"2023-10-06T04:52:17.652153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650775+0000","last_change":"2023-10-06T04:52:18.647865+0000","last_active":"2023-10-06T04:52:18.650775+0000","last_peered":"2023-10-06T04:52:18.650775+0000","last_clean":"2023-10-06T04:52:18.650775+0000","last_became_active":"2023-10-06T04:52:18.647582+0000","last_became_peered":"2023-10-06T04:52:18.647582+0000","last_unstale":"2023-10-06T04:52:18.650775+0000","last_undegraded":"2023-10-06T04:52:18.650775+0000","last_fullsized":"2023-10-06T04:52:18.650775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665684+0000","last_change":"2023-10-06T04:52:17.651495+0000","last_active":"2023-10-06T04:52:18.665684+0000","last_peered":"2023-10-06T04:52:18.665684+0000","last_clean":"2023-10-06T04:52:18.665684+0000","last_became_active":"2023-10-06T04:52:17.651108+0000","last_became_peered":"2023-10-06T04:52:17.651108+0000","last_unstale":"2023-10-06T04:52:18.665684+0000","last_undegraded":"2023-10-06T04:52:18.665684+0000","last_fullsized":"2023-10-06T04:52:18.665684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666462+0000","last_change":"2023-10-06T04:52:18.662825+0000","last_active":"2023-10-06T04:52:18.666462+0000","last_peered":"2023-10-06T04:52:18.666462+0000","last_clean":"2023-10-06T04:52:18.666462+0000","last_became_active":"2023-10-06T04:52:18.662499+0000","last_became_peered":"2023-10-06T04:52:18.662499+0000","last_unstale":"2023-10-06T04:52:18.666462+0000","last_undegraded":"2023-10-06T04:52:18.666462+0000","last_fullsized":"2023-10-06T04:52:18.666462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:58.798038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.661547+0000","last_change":"2023-10-06T04:52:17.657591+0000","last_active":"2023-10-06T04:52:17.661547+0000","last_peered":"2023-10-06T04:52:17.661547+0000","last_clean":"2023-10-06T04:52:17.661547+0000","last_became_active":"2023-10-06T04:52:17.657291+0000","last_became_peered":"2023-10-06T04:52:17.657291+0000","last_unstale":"2023-10-06T04:52:17.661547+0000","last_undegraded":"2023-10-06T04:52:17.661547+0000","last_fullsized":"2023-10-06T04:52:17.661547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659774+0000","last_change":"2023-10-06T04:52:18.650766+0000","last_active":"2023-10-06T04:52:18.659774+0000","last_peered":"2023-10-06T04:52:18.659774+0000","last_clean":"2023-10-06T04:52:18.659774+0000","last_became_active":"2023-10-06T04:52:18.650514+0000","last_became_peered":"2023-10-06T04:52:18.650514+0000","last_unstale":"2023-10-06T04:52:18.659774+0000","last_undegraded":"2023-10-06T04:52:18.659774+0000","last_fullsized":"2023-10-06T04:52:18.659774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.639506+0000","last_change":"2023-10-06T04:52:17.655486+0000","last_active":"2023-10-06T04:52:18.639506+0000","last_peered":"2023-10-06T04:52:18.639506+0000","last_clean":"2023-10-06T04:52:18.639506+0000","last_became_active":"2023-10-06T04:52:17.654231+0000","last_became_peered":"2023-10-06T04:52:17.654231+0000","last_unstale":"2023-10-06T04:52:18.639506+0000","last_undegraded":"2023-10-06T04:52:18.639506+0000","last_fullsized":"2023-10-06T04:52:18.639506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666321+0000","last_change":"2023-10-06T04:52:18.663641+0000","last_active":"2023-10-06T04:52:18.666321+0000","last_peered":"2023-10-06T04:52:18.666321+0000","last_clean":"2023-10-06T04:52:18.666321+0000","last_became_active":"2023-10-06T04:52:18.663334+0000","last_became_peered":"2023-10-06T04:52:18.663334+0000","last_unstale":"2023-10-06T04:52:18.666321+0000","last_undegraded":"2023-10-06T04:52:18.666321+0000","last_fullsized":"2023-10-06T04:52:18.666321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:18.572482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643070+0000","last_change":"2023-10-06T04:52:17.644385+0000","last_active":"2023-10-06T04:52:18.643070+0000","last_peered":"2023-10-06T04:52:18.643070+0000","last_clean":"2023-10-06T04:52:18.643070+0000","last_became_active":"2023-10-06T04:52:17.643911+0000","last_became_peered":"2023-10-06T04:52:17.643911+0000","last_unstale":"2023-10-06T04:52:18.643070+0000","last_undegraded":"2023-10-06T04:52:18.643070+0000","last_fullsized":"2023-10-06T04:52:18.643070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.657062+0000","last_change":"2023-10-06T04:52:18.657062+0000","last_active":"2023-10-06T04:52:18.657062+0000","last_peered":"2023-10-06T04:52:18.657062+0000","last_clean":"2023-10-06T04:52:18.657062+0000","last_became_active":"2023-10-06T04:52:18.656805+0000","last_became_peered":"2023-10-06T04:52:18.656805+0000","last_unstale":"2023-10-06T04:52:18.657062+0000","last_undegraded":"2023-10-06T04:52:18.657062+0000","last_fullsized":"2023-10-06T04:52:18.657062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664453+0000","last_change":"2023-10-06T04:52:17.648546+0000","last_active":"2023-10-06T04:52:18.664453+0000","last_peered":"2023-10-06T04:52:18.664453+0000","last_clean":"2023-10-06T04:52:18.664453+0000","last_became_active":"2023-10-06T04:52:17.648247+0000","last_became_peered":"2023-10-06T04:52:17.648247+0000","last_unstale":"2023-10-06T04:52:18.664453+0000","last_undegraded":"2023-10-06T04:52:18.664453+0000","last_fullsized":"2023-10-06T04:52:18.664453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642529+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:52:18.642529+0000","last_peered":"2023-10-06T04:52:18.642529+0000","last_clean":"2023-10-06T04:52:18.642529+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:52:18.642529+0000","last_undegraded":"2023-10-06T04:52:18.642529+0000","last_fullsized":"2023-10-06T04:52:18.642529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659829+0000","last_change":"2023-10-06T04:52:18.654743+0000","last_active":"2023-10-06T04:52:18.659829+0000","last_peered":"2023-10-06T04:52:18.659829+0000","last_clean":"2023-10-06T04:52:18.659829+0000","last_became_active":"2023-10-06T04:52:18.653945+0000","last_became_peered":"2023-10-06T04:52:18.653945+0000","last_unstale":"2023-10-06T04:52:18.659829+0000","last_undegraded":"2023-10-06T04:52:18.659829+0000","last_fullsized":"2023-10-06T04:52:18.659829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642776+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:52:18.642776+0000","last_peered":"2023-10-06T04:52:18.642776+0000","last_clean":"2023-10-06T04:52:18.642776+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:52:18.642776+0000","last_undegraded":"2023-10-06T04:52:18.642776+0000","last_fullsized":"2023-10-06T04:52:18.642776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.640876+0000","last_change":"2023-10-06T04:52:17.653308+0000","last_active":"2023-10-06T04:52:18.640876+0000","last_peered":"2023-10-06T04:52:18.640876+0000","last_clean":"2023-10-06T04:52:18.640876+0000","last_became_active":"2023-10-06T04:52:17.653032+0000","last_became_peered":"2023-10-06T04:52:17.653032+0000","last_unstale":"2023-10-06T04:52:18.640876+0000","last_undegraded":"2023-10-06T04:52:18.640876+0000","last_fullsized":"2023-10-06T04:52:18.640876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665740+0000","last_change":"2023-10-06T04:52:18.656477+0000","last_active":"2023-10-06T04:52:18.665740+0000","last_peered":"2023-10-06T04:52:18.665740+0000","last_clean":"2023-10-06T04:52:18.665740+0000","last_became_active":"2023-10-06T04:52:18.656225+0000","last_became_peered":"2023-10-06T04:52:18.656225+0000","last_unstale":"2023-10-06T04:52:18.665740+0000","last_undegraded":"2023-10-06T04:52:18.665740+0000","last_fullsized":"2023-10-06T04:52:18.665740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.634374+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:52:17.634374+0000","last_peered":"2023-10-06T04:52:17.634374+0000","last_clean":"2023-10-06T04:52:17.634374+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:52:17.634374+0000","last_undegraded":"2023-10-06T04:52:17.634374+0000","last_fullsized":"2023-10-06T04:52:17.634374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":162,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.642530+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:52:17.642530+0000","last_peered":"2023-10-06T04:52:17.642530+0000","last_clean":"2023-10-06T04:52:17.642530+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:52:17.642530+0000","last_undegraded":"2023-10-06T04:52:17.642530+0000","last_fullsized":"2023-10-06T04:52:17.642530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638291+0000","last_change":"2023-10-06T04:52:17.656649+0000","last_active":"2023-10-06T04:52:18.638291+0000","last_peered":"2023-10-06T04:52:18.638291+0000","last_clean":"2023-10-06T04:52:18.638291+0000","last_became_active":"2023-10-06T04:52:17.656336+0000","last_became_peered":"2023-10-06T04:52:17.656336+0000","last_unstale":"2023-10-06T04:52:18.638291+0000","last_undegraded":"2023-10-06T04:52:18.638291+0000","last_fullsized":"2023-10-06T04:52:18.638291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662767+0000","last_change":"2023-10-06T04:52:18.659289+0000","last_active":"2023-10-06T04:52:18.662767+0000","last_peered":"2023-10-06T04:52:18.662767+0000","last_clean":"2023-10-06T04:52:18.662767+0000","last_became_active":"2023-10-06T04:52:18.659024+0000","last_became_peered":"2023-10-06T04:52:18.659024+0000","last_unstale":"2023-10-06T04:52:18.662767+0000","last_undegraded":"2023-10-06T04:52:18.662767+0000","last_fullsized":"2023-10-06T04:52:18.662767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.642515+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:52:17.642515+0000","last_peered":"2023-10-06T04:52:17.642515+0000","last_clean":"2023-10-06T04:52:17.642515+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:52:17.642515+0000","last_undegraded":"2023-10-06T04:52:17.642515+0000","last_fullsized":"2023-10-06T04:52:17.642515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652075+0000","last_change":"2023-10-06T04:52:17.648757+0000","last_active":"2023-10-06T04:52:17.652075+0000","last_peered":"2023-10-06T04:52:17.652075+0000","last_clean":"2023-10-06T04:52:17.652075+0000","last_became_active":"2023-10-06T04:52:17.647698+0000","last_became_peered":"2023-10-06T04:52:17.647698+0000","last_unstale":"2023-10-06T04:52:17.652075+0000","last_undegraded":"2023-10-06T04:52:17.652075+0000","last_fullsized":"2023-10-06T04:52:17.652075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.648518+0000","last_change":"2023-10-06T04:52:18.648518+0000","last_active":"2023-10-06T04:52:18.648518+0000","last_peered":"2023-10-06T04:52:18.648518+0000","last_clean":"2023-10-06T04:52:18.648518+0000","last_became_active":"2023-10-06T04:52:18.648172+0000","last_became_peered":"2023-10-06T04:52:18.648172+0000","last_unstale":"2023-10-06T04:52:18.648518+0000","last_undegraded":"2023-10-06T04:52:18.648518+0000","last_fullsized":"2023-10-06T04:52:18.648518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.642418+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:52:17.642418+0000","last_peered":"2023-10-06T04:52:17.642418+0000","last_clean":"2023-10-06T04:52:17.642418+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:52:17.642418+0000","last_undegraded":"2023-10-06T04:52:17.642418+0000","last_fullsized":"2023-10-06T04:52:17.642418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642140+0000","last_change":"2023-10-06T04:52:17.647411+0000","last_active":"2023-10-06T04:52:18.642140+0000","last_peered":"2023-10-06T04:52:18.642140+0000","last_clean":"2023-10-06T04:52:18.642140+0000","last_became_active":"2023-10-06T04:52:17.646893+0000","last_became_peered":"2023-10-06T04:52:17.646893+0000","last_unstale":"2023-10-06T04:52:18.642140+0000","last_undegraded":"2023-10-06T04:52:18.642140+0000","last_fullsized":"2023-10-06T04:52:18.642140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655841+0000","last_change":"2023-10-06T04:52:18.655841+0000","last_active":"2023-10-06T04:52:18.655841+0000","last_peered":"2023-10-06T04:52:18.655841+0000","last_clean":"2023-10-06T04:52:18.655841+0000","last_became_active":"2023-10-06T04:52:18.655528+0000","last_became_peered":"2023-10-06T04:52:18.655528+0000","last_unstale":"2023-10-06T04:52:18.655841+0000","last_undegraded":"2023-10-06T04:52:18.655841+0000","last_fullsized":"2023-10-06T04:52:18.655841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.646208+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:52:17.646208+0000","last_peered":"2023-10-06T04:52:17.646208+0000","last_clean":"2023-10-06T04:52:17.646208+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:52:17.646208+0000","last_undegraded":"2023-10-06T04:52:17.646208+0000","last_fullsized":"2023-10-06T04:52:17.646208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.641848+0000","last_change":"2023-10-06T04:52:17.655012+0000","last_active":"2023-10-06T04:52:18.641848+0000","last_peered":"2023-10-06T04:52:18.641848+0000","last_clean":"2023-10-06T04:52:18.641848+0000","last_became_active":"2023-10-06T04:52:17.654727+0000","last_became_peered":"2023-10-06T04:52:17.654727+0000","last_unstale":"2023-10-06T04:52:18.641848+0000","last_undegraded":"2023-10-06T04:52:18.641848+0000","last_fullsized":"2023-10-06T04:52:18.641848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647619+0000","last_change":"2023-10-06T04:52:18.647619+0000","last_active":"2023-10-06T04:52:18.647619+0000","last_peered":"2023-10-06T04:52:18.647619+0000","last_clean":"2023-10-06T04:52:18.647619+0000","last_became_active":"2023-10-06T04:52:18.647303+0000","last_became_peered":"2023-10-06T04:52:18.647303+0000","last_unstale":"2023-10-06T04:52:18.647619+0000","last_undegraded":"2023-10-06T04:52:18.647619+0000","last_fullsized":"2023-10-06T04:52:18.647619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:35.374135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.648906+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:52:17.648906+0000","last_peered":"2023-10-06T04:52:17.648906+0000","last_clean":"2023-10-06T04:52:17.648906+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:52:17.648906+0000","last_undegraded":"2023-10-06T04:52:17.648906+0000","last_fullsized":"2023-10-06T04:52:17.648906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652504+0000","last_change":"2023-10-06T04:52:17.649887+0000","last_active":"2023-10-06T04:52:17.652504+0000","last_peered":"2023-10-06T04:52:17.652504+0000","last_clean":"2023-10-06T04:52:17.652504+0000","last_became_active":"2023-10-06T04:52:17.649537+0000","last_became_peered":"2023-10-06T04:52:17.649537+0000","last_unstale":"2023-10-06T04:52:17.652504+0000","last_undegraded":"2023-10-06T04:52:17.652504+0000","last_fullsized":"2023-10-06T04:52:17.652504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:57.133434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655515+0000","last_change":"2023-10-06T04:52:18.649286+0000","last_active":"2023-10-06T04:52:18.655515+0000","last_peered":"2023-10-06T04:52:18.655515+0000","last_clean":"2023-10-06T04:52:18.655515+0000","last_became_active":"2023-10-06T04:52:18.648947+0000","last_became_peered":"2023-10-06T04:52:18.648947+0000","last_unstale":"2023-10-06T04:52:18.655515+0000","last_undegraded":"2023-10-06T04:52:18.655515+0000","last_fullsized":"2023-10-06T04:52:18.655515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.648606+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:52:17.648606+0000","last_peered":"2023-10-06T04:52:17.648606+0000","last_clean":"2023-10-06T04:52:17.648606+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:52:17.648606+0000","last_undegraded":"2023-10-06T04:52:17.648606+0000","last_fullsized":"2023-10-06T04:52:17.648606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-06T04:52:17.652549+0000","last_change":"2023-10-06T04:52:17.650178+0000","last_active":"2023-10-06T04:52:17.652549+0000","last_peered":"2023-10-06T04:52:17.652549+0000","last_clean":"2023-10-06T04:52:17.652549+0000","last_became_active":"2023-10-06T04:52:17.649883+0000","last_became_peered":"2023-10-06T04:52:17.649883+0000","last_unstale":"2023-10-06T04:52:17.652549+0000","last_undegraded":"2023-10-06T04:52:17.652549+0000","last_fullsized":"2023-10-06T04:52:17.652549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658639+0000","last_change":"2023-10-06T04:52:18.658639+0000","last_active":"2023-10-06T04:52:18.658639+0000","last_peered":"2023-10-06T04:52:18.658639+0000","last_clean":"2023-10-06T04:52:18.658639+0000","last_became_active":"2023-10-06T04:52:18.658381+0000","last_became_peered":"2023-10-06T04:52:18.658381+0000","last_unstale":"2023-10-06T04:52:18.658639+0000","last_undegraded":"2023-10-06T04:52:18.658639+0000","last_fullsized":"2023-10-06T04:52:18.658639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:51.836228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665805+0000","last_change":"2023-10-06T04:52:17.657299+0000","last_active":"2023-10-06T04:52:18.665805+0000","last_peered":"2023-10-06T04:52:18.665805+0000","last_clean":"2023-10-06T04:52:18.665805+0000","last_became_active":"2023-10-06T04:52:17.656956+0000","last_became_peered":"2023-10-06T04:52:17.656956+0000","last_unstale":"2023-10-06T04:52:18.665805+0000","last_undegraded":"2023-10-06T04:52:18.665805+0000","last_fullsized":"2023-10-06T04:52:18.665805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.652907+0000","last_change":"2023-10-06T04:52:18.652907+0000","last_active":"2023-10-06T04:52:18.652907+0000","last_peered":"2023-10-06T04:52:18.652907+0000","last_clean":"2023-10-06T04:52:18.652907+0000","last_became_active":"2023-10-06T04:52:18.652600+0000","last_became_peered":"2023-10-06T04:52:18.652600+0000","last_unstale":"2023-10-06T04:52:18.652907+0000","last_undegraded":"2023-10-06T04:52:18.652907+0000","last_fullsized":"2023-10-06T04:52:18.652907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:13.950333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667707+0000","last_change":"2023-10-06T04:52:17.656487+0000","last_active":"2023-10-06T04:52:18.667707+0000","last_peered":"2023-10-06T04:52:18.667707+0000","last_clean":"2023-10-06T04:52:18.667707+0000","last_became_active":"2023-10-06T04:52:17.656205+0000","last_became_peered":"2023-10-06T04:52:17.656205+0000","last_unstale":"2023-10-06T04:52:18.667707+0000","last_undegraded":"2023-10-06T04:52:18.667707+0000","last_fullsized":"2023-10-06T04:52:18.667707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665639+0000","last_change":"2023-10-06T04:52:18.654103+0000","last_active":"2023-10-06T04:52:18.665639+0000","last_peered":"2023-10-06T04:52:18.665639+0000","last_clean":"2023-10-06T04:52:18.665639+0000","last_became_active":"2023-10-06T04:52:18.653791+0000","last_became_peered":"2023-10-06T04:52:18.653791+0000","last_unstale":"2023-10-06T04:52:18.665639+0000","last_undegraded":"2023-10-06T04:52:18.665639+0000","last_fullsized":"2023-10-06T04:52:18.665639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:18.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667657+0000","last_change":"2023-10-06T04:52:17.656962+0000","last_active":"2023-10-06T04:52:18.667657+0000","last_peered":"2023-10-06T04:52:18.667657+0000","last_clean":"2023-10-06T04:52:18.667657+0000","last_became_active":"2023-10-06T04:52:17.656699+0000","last_became_peered":"2023-10-06T04:52:17.656699+0000","last_unstale":"2023-10-06T04:52:18.667657+0000","last_undegraded":"2023-10-06T04:52:18.667657+0000","last_fullsized":"2023-10-06T04:52:18.667657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665510+0000","last_change":"2023-10-06T04:52:18.653480+0000","last_active":"2023-10-06T04:52:18.665510+0000","last_peered":"2023-10-06T04:52:18.665510+0000","last_clean":"2023-10-06T04:52:18.665510+0000","last_became_active":"2023-10-06T04:52:18.653200+0000","last_became_peered":"2023-10-06T04:52:18.653200+0000","last_unstale":"2023-10-06T04:52:18.665510+0000","last_undegraded":"2023-10-06T04:52:18.665510+0000","last_fullsized":"2023-10-06T04:52:18.665510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:51.240487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663382+0000","last_change":"2023-10-06T04:52:17.651435+0000","last_active":"2023-10-06T04:52:18.663382+0000","last_peered":"2023-10-06T04:52:18.663382+0000","last_clean":"2023-10-06T04:52:18.663382+0000","last_became_active":"2023-10-06T04:52:17.651063+0000","last_became_peered":"2023-10-06T04:52:17.651063+0000","last_unstale":"2023-10-06T04:52:18.663382+0000","last_undegraded":"2023-10-06T04:52:18.663382+0000","last_fullsized":"2023-10-06T04:52:18.663382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664028+0000","last_change":"2023-10-06T04:52:18.658488+0000","last_active":"2023-10-06T04:52:18.664028+0000","last_peered":"2023-10-06T04:52:18.664028+0000","last_clean":"2023-10-06T04:52:18.664028+0000","last_became_active":"2023-10-06T04:52:18.657977+0000","last_became_peered":"2023-10-06T04:52:18.657977+0000","last_unstale":"2023-10-06T04:52:18.664028+0000","last_undegraded":"2023-10-06T04:52:18.664028+0000","last_fullsized":"2023-10-06T04:52:18.664028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661377+0000","last_change":"2023-10-06T04:52:17.652321+0000","last_active":"2023-10-06T04:52:18.661377+0000","last_peered":"2023-10-06T04:52:18.661377+0000","last_clean":"2023-10-06T04:52:18.661377+0000","last_became_active":"2023-10-06T04:52:17.651998+0000","last_became_peered":"2023-10-06T04:52:17.651998+0000","last_unstale":"2023-10-06T04:52:18.661377+0000","last_undegraded":"2023-10-06T04:52:18.661377+0000","last_fullsized":"2023-10-06T04:52:18.661377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666345+0000","last_change":"2023-10-06T04:52:18.664061+0000","last_active":"2023-10-06T04:52:18.666345+0000","last_peered":"2023-10-06T04:52:18.666345+0000","last_clean":"2023-10-06T04:52:18.666345+0000","last_became_active":"2023-10-06T04:52:18.663769+0000","last_became_peered":"2023-10-06T04:52:18.663769+0000","last_unstale":"2023-10-06T04:52:18.666345+0000","last_undegraded":"2023-10-06T04:52:18.666345+0000","last_fullsized":"2023-10-06T04:52:18.666345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662900+0000","last_change":"2023-10-06T04:52:18.662900+0000","last_active":"2023-10-06T04:52:18.662900+0000","last_peered":"2023-10-06T04:52:18.662900+0000","last_clean":"2023-10-06T04:52:18.662900+0000","last_became_active":"2023-10-06T04:52:18.662558+0000","last_became_peered":"2023-10-06T04:52:18.662558+0000","last_unstale":"2023-10-06T04:52:18.662900+0000","last_undegraded":"2023-10-06T04:52:18.662900+0000","last_fullsized":"2023-10-06T04:52:18.662900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:18.313966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664001+0000","last_change":"2023-10-06T04:52:18.658296+0000","last_active":"2023-10-06T04:52:18.664001+0000","last_peered":"2023-10-06T04:52:18.664001+0000","last_clean":"2023-10-06T04:52:18.664001+0000","last_became_active":"2023-10-06T04:52:18.657589+0000","last_became_peered":"2023-10-06T04:52:18.657589+0000","last_unstale":"2023-10-06T04:52:18.664001+0000","last_undegraded":"2023-10-06T04:52:18.664001+0000","last_fullsized":"2023-10-06T04:52:18.664001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666380+0000","last_change":"2023-10-06T04:52:18.663851+0000","last_active":"2023-10-06T04:52:18.666380+0000","last_peered":"2023-10-06T04:52:18.666380+0000","last_clean":"2023-10-06T04:52:18.666380+0000","last_became_active":"2023-10-06T04:52:18.663574+0000","last_became_peered":"2023-10-06T04:52:18.663574+0000","last_unstale":"2023-10-06T04:52:18.666380+0000","last_undegraded":"2023-10-06T04:52:18.666380+0000","last_fullsized":"2023-10-06T04:52:18.666380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:54.373475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656769+0000","last_change":"2023-10-06T04:52:18.652292+0000","last_active":"2023-10-06T04:52:18.656769+0000","last_peered":"2023-10-06T04:52:18.656769+0000","last_clean":"2023-10-06T04:52:18.656769+0000","last_became_active":"2023-10-06T04:52:18.651919+0000","last_became_peered":"2023-10-06T04:52:18.651919+0000","last_unstale":"2023-10-06T04:52:18.656769+0000","last_undegraded":"2023-10-06T04:52:18.656769+0000","last_fullsized":"2023-10-06T04:52:18.656769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650895+0000","last_change":"2023-10-06T04:52:18.650895+0000","last_active":"2023-10-06T04:52:18.650895+0000","last_peered":"2023-10-06T04:52:18.650895+0000","last_clean":"2023-10-06T04:52:18.650895+0000","last_became_active":"2023-10-06T04:52:18.650623+0000","last_became_peered":"2023-10-06T04:52:18.650623+0000","last_unstale":"2023-10-06T04:52:18.650895+0000","last_undegraded":"2023-10-06T04:52:18.650895+0000","last_fullsized":"2023-10-06T04:52:18.650895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":658310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":782336,"data_stored":656558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":196608,"data_stored":66814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:52:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:52:23.150 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T04:52:23.150 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T04:52:23.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T04:52:23.151 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T04:52:23.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.301+0000 7f79d2318700 1 Processor -- start 2023-10-06T04:52:23.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.301+0000 7f79d2318700 1 -- start start 2023-10-06T04:52:23.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79d2318700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f79cc15c8f0 0x7f79cc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79d2318700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc0cf6c0 con 0x7f79cc0d0870 2023-10-06T04:52:23.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79d2318700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc0cfb50 con 0x7f79cc15ccc0 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79d2318700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79cc15e050 con 0x7f79cc15c8f0 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79d08b5700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79cc0d0870 0x7f79cc0b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35806/0 (socket says 172.21.15.3:35806) 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79d08b5700 1 -- 172.21.15.3:0/1907907339 learned_addr learned my addr 172.21.15.3:0/1907907339 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79cbfff700 1 --2- 172.21.15.3:0/1907907339 >> v2:172.21.15.153:3300/0 conn(0x7f79cc15c8f0 0x7f79cc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79caffd700 1 -- 172.21.15.3:0/1907907339 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262157546 0 0) 0x7f79cc0cf6c0 con 0x7f79cc0d0870 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79caffd700 1 -- 172.21.15.3:0/1907907339 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b4003570 con 0x7f79cc0d0870 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79cbfff700 1 -- 172.21.15.3:0/1907907339 >> v1:172.21.15.153:6789/0 conn(0x7f79cc15ccc0 legacy=0x7f79cc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79cbfff700 1 -- 172.21.15.3:0/1907907339 >> v1:172.21.15.3:6789/0 conn(0x7f79cc0d0870 legacy=0x7f79cc0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.302+0000 7f79cbfff700 1 -- 172.21.15.3:0/1907907339 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0ceab0 con 0x7f79cc15c8f0 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79cbfff700 1 --2- 172.21.15.3:0/1907907339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc15db10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009910 tx=0x7f79c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d8cc4e573a52f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79caffd700 1 -- 172.21.15.3:0/1907907339 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000b640 con 0x7f79cc15c8f0 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79caffd700 1 -- 172.21.15.3:0/1907907339 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c000b7a0 con 0x7f79cc15c8f0 2023-10-06T04:52:23.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79caffd700 1 -- 172.21.15.3:0/1907907339 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000bb60 con 0x7f79cc15c8f0 2023-10-06T04:52:23.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79d2318700 1 -- 172.21.15.3:0/1907907339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 msgr2=0x7f79cc15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79d2318700 1 --2- 172.21.15.3:0/1907907339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc15db10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009910 tx=0x7f79c00091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79d2318700 1 -- 172.21.15.3:0/1907907339 shutdown_connections 2023-10-06T04:52:23.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79d2318700 1 --2- 172.21.15.3:0/1907907339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc15db10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79d2318700 1 -- 172.21.15.3:0/1907907339 >> 172.21.15.3:0/1907907339 conn(0x7f79cc070310 msgr2=0x7f79cc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79d2318700 1 -- 172.21.15.3:0/1907907339 shutdown_connections 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.303+0000 7f79d2318700 1 -- 172.21.15.3:0/1907907339 wait complete. 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.304+0000 7f79d2318700 1 Processor -- start 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79d2318700 1 -- start start 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79d2318700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79cc0d0870 0x7f79cc16de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79d2318700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79d2318700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79cc15ccc0 0x7f79cc1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79d2318700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0b6370 con 0x7f79cc0d0870 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79d2318700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc14ef50 con 0x7f79cc15c8f0 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79d2318700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc16e380 con 0x7f79cc15ccc0 2023-10-06T04:52:23.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79cbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79cc0d0870 0x7f79cc16de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79cbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79cc0d0870 0x7f79cc16de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32900/0 (socket says 172.21.15.3:32900) 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.305+0000 7f79cbfff700 1 -- 172.21.15.3:0/2725765886 learned_addr learned my addr 172.21.15.3:0/2725765886 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cb7fe700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79d08b5700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79cc15ccc0 0x7f79cc1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cbfff700 1 -- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79cc15ccc0 msgr2=0x7f79cc1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cbfff700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79cc15ccc0 0x7f79cc1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cbfff700 1 -- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 msgr2=0x7f79cc14a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cbfff700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc14a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cbfff700 1 -- 172.21.15.3:0/2725765886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0009420 con 0x7f79cc0d0870 2023-10-06T04:52:23.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79d08b5700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79cc15ccc0 0x7f79cc1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cb7fe700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79cbfff700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79cc0d0870 0x7f79cc16de40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f79cc06eaa0 tx=0x7f79c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79b3fff700 1 -- 172.21.15.3:0/2725765886 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c001f070 con 0x7f79cc0d0870 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79b3fff700 1 -- 172.21.15.3:0/2725765886 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c000bb20 con 0x7f79cc0d0870 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79cc14a580 con 0x7f79cc0d0870 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79b3fff700 1 -- 172.21.15.3:0/2725765886 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0011670 con 0x7f79cc0d0870 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.306+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc0b4870 con 0x7f79cc0d0870 2023-10-06T04:52:23.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.307+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc14f100 con 0x7f79cc0d0870 2023-10-06T04:52:23.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.308+0000 7f79b3fff700 1 -- 172.21.15.3:0/2725765886 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f79c00119e0 con 0x7f79cc0d0870 2023-10-06T04:52:23.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.308+0000 7f79b3fff700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f79b8066f30 0x7f79b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.308+0000 7f79b3fff700 1 -- 172.21.15.3:0/2725765886 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f79c0085ab0 con 0x7f79cc0d0870 2023-10-06T04:52:23.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.308+0000 7f79cb7fe700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f79b8066f30 0x7f79b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.309+0000 7f79cb7fe700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f79b8066f30 0x7f79b80693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f79cc15be80 tx=0x7f79bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:23.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.312+0000 7f79b3fff700 1 -- 172.21.15.3:0/2725765886 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79cc14f100 con 0x7f79cc0d0870 2023-10-06T04:52:23.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.565+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f79cc14f0c0 con 0x7f79cc0d0870 2023-10-06T04:52:23.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.568+0000 7f79b3fff700 1 -- 172.21.15.3:0/2725765886 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163705 (secure 0 0 0) 0x7f79c0015090 con 0x7f79cc0d0870 2023-10-06T04:52:23.571 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:23.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f79b8066f30 msgr2=0x7f79b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f79b8066f30 0x7f79b80693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f79cc15be80 tx=0x7f79bc009040 comp rx=0 tx=0).stop 2023-10-06T04:52:23.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79cc0d0870 msgr2=0x7f79cc16de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79cc0d0870 0x7f79cc16de40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f79cc06eaa0 tx=0x7f79c0017000 comp rx=0 tx=0).stop 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 shutdown_connections 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f79b8066f30 0x7f79b80693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79cc15ccc0 0x7f79cc1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79cc0d0870 0x7f79cc16de40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 --2- 172.21.15.3:0/2725765886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79cc15c8f0 0x7f79cc14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 >> 172.21.15.3:0/2725765886 conn(0x7f79cc070310 msgr2=0x7f79cc155dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 shutdown_connections 2023-10-06T04:52:23.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.570+0000 7f79d2318700 1 -- 172.21.15.3:0/2725765886 wait complete. 2023-10-06T04:52:23.584 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":90310},{"type":"v1","addr":"172.21.15.153:6833","nonce":90310}]},"active_addr":"172.21.15.153:6833/90310","active_change":"2023-10-06T04:52:05.416190+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2262830232}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":1201469391}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":336446932}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":3493597100}]}]} 2023-10-06T04:52:23.588 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T04:52:23.588 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T04:52:23.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.707+0000 7f4d6bce0700 1 Processor -- start 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.707+0000 7f4d6bce0700 1 -- start start 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6bce0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d6415c8f0 0x7f4d6415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6bce0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d640d0870 0x7f4d640b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6bce0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d640cf6c0 con 0x7f4d6415ccc0 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6bce0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640cfb50 con 0x7f4d640d0870 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6bce0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6415e050 con 0x7f4d6415c8f0 2023-10-06T04:52:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6a27d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d640d0870 0x7f4d640b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6927b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d6415ccc0 0x7f4d640d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35818/0 (socket says 172.21.15.3:35818) 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6927b700 1 -- 172.21.15.3:0/1182145000 learned_addr learned my addr 172.21.15.3:0/1182145000 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6a27d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d640d0870 0x7f4d640b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32906/0 (socket says 172.21.15.3:32906) 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d69a7c700 1 --2- 172.21.15.3:0/1182145000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d6415c8f0 0x7f4d6415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6a27d700 1 -- 172.21.15.3:0/1182145000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d6415c8f0 msgr2=0x7f4d6415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6a27d700 1 --2- 172.21.15.3:0/1182145000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d6415c8f0 0x7f4d6415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6a27d700 1 -- 172.21.15.3:0/1182145000 >> v1:172.21.15.3:6789/0 conn(0x7f4d6415ccc0 legacy=0x7f4d640d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d68a7a700 1 -- 172.21.15.3:0/1182145000 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239703523 0 0) 0x7f4d640cf6c0 con 0x7f4d6415ccc0 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6a27d700 1 -- 172.21.15.3:0/1182145000 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640ceab0 con 0x7f4d640d0870 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d6a27d700 1 --2- 172.21.15.3:0/1182145000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d640d0870 0x7f4d640b3ba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0098e0 tx=0x7f4d5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c281fdd33e79afff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.708+0000 7f4d68a7a700 1 -- 172.21.15.3:0/1182145000 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00b640 con 0x7f4d640d0870 2023-10-06T04:52:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d68a7a700 1 -- 172.21.15.3:0/1182145000 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00b7a0 con 0x7f4d640d0870 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d68a7a700 1 -- 172.21.15.3:0/1182145000 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00bb40 con 0x7f4d640d0870 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 -- 172.21.15.3:0/1182145000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d640d0870 msgr2=0x7f4d640b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/1182145000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d640d0870 0x7f4d640b3ba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0098e0 tx=0x7f4d5c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 -- 172.21.15.3:0/1182145000 shutdown_connections 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/1182145000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d640d0870 0x7f4d640b3ba0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/1182145000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d6415c8f0 0x7f4d6415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 -- 172.21.15.3:0/1182145000 >> 172.21.15.3:0/1182145000 conn(0x7f4d64070310 msgr2=0x7f4d64070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 -- 172.21.15.3:0/1182145000 shutdown_connections 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.709+0000 7f4d6bce0700 1 -- 172.21.15.3:0/1182145000 wait complete. 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.710+0000 7f4d6bce0700 1 Processor -- start 2023-10-06T04:52:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.710+0000 7f4d6bce0700 1 -- start start 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6bce0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d640d0870 0x7f4d640b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6bce0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6415c8f0 0x7f4d6416a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6bce0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d6415ccc0 0x7f4d64153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6bce0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640b4f80 con 0x7f4d6415c8f0 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6bce0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640b5b00 con 0x7f4d6415ccc0 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6bce0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640b51d0 con 0x7f4d640d0870 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6415c8f0 0x7f4d6416a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6415c8f0 0x7f4d6416a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32920/0 (socket says 172.21.15.3:32920) 2023-10-06T04:52:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 -- 172.21.15.3:0/59642159 learned_addr learned my addr 172.21.15.3:0/59642159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d69a7c700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d640d0870 0x7f4d640b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6a27d700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d6415ccc0 0x7f4d64153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 -- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d640d0870 msgr2=0x7f4d640b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d640d0870 0x7f4d640b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 -- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d6415ccc0 msgr2=0x7f4d64153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d6415ccc0 0x7f4d64153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.711+0000 7f4d6927b700 1 -- 172.21.15.3:0/59642159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c009420 con 0x7f4d6415c8f0 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.712+0000 7f4d6927b700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6415c8f0 0x7f4d6416a390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c810 tx=0x7f4d58007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.712+0000 7f4d61ffb700 1 -- 172.21.15.3:0/59642159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58005c00 con 0x7f4d6415c8f0 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.712+0000 7f4d61ffb700 1 -- 172.21.15.3:0/59642159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58005d60 con 0x7f4d6415c8f0 2023-10-06T04:52:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.712+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d640b54b0 con 0x7f4d6415c8f0 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.712+0000 7f4d61ffb700 1 -- 172.21.15.3:0/59642159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58007300 con 0x7f4d6415c8f0 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.712+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6416a8d0 con 0x7f4d6415c8f0 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.713+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d640b5c70 con 0x7f4d6415c8f0 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.713+0000 7f4d61ffb700 1 -- 172.21.15.3:0/59642159 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d5801b3b0 con 0x7f4d6415c8f0 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.713+0000 7f4d61ffb700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.714+0000 7f4d61ffb700 1 -- 172.21.15.3:0/59642159 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4d58083b50 con 0x7f4d6415c8f0 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.714+0000 7f4d69a7c700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.714+0000 7f4d69a7c700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400f570 tx=0x7f4d5400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.718+0000 7f4d61ffb700 1 -- 172.21.15.3:0/59642159 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d58050c60 con 0x7f4d6415c8f0 2023-10-06T04:52:23.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:23.818 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:23.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.916+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d64071680 con 0x7f4d6415c8f0 2023-10-06T04:52:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.917+0000 7f4d61ffb700 1 -- 172.21.15.3:0/59642159 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f4d58017020 con 0x7f4d6415c8f0 2023-10-06T04:52:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:23.919 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d50066f30 msgr2=0x7f4d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400f570 tx=0x7f4d5400b040 comp rx=0 tx=0).stop 2023-10-06T04:52:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6415c8f0 msgr2=0x7f4d6416a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6415c8f0 0x7f4d6416a390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c810 tx=0x7f4d58007db0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 shutdown_connections 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d6415ccc0 0x7f4d64153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d640d0870 0x7f4d640b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 --2- 172.21.15.3:0/59642159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6415c8f0 0x7f4d6416a390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.919+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 >> 172.21.15.3:0/59642159 conn(0x7f4d64070310 msgr2=0x7f4d640b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.920+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 shutdown_connections 2023-10-06T04:52:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:23.920+0000 7f4d6bce0700 1 -- 172.21.15.3:0/59642159 wait complete. 2023-10-06T04:52:23.933 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T04:52:23.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:24.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.050+0000 7fd34f59e700 1 Processor -- start 2023-10-06T04:52:24.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.050+0000 7fd34f59e700 1 -- start start 2023-10-06T04:52:24.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3500cb780 0x7fd350158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd350159500 0x7fd3500cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3500cf130 con 0x7fd3500cb3b0 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34f59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3500ce520 con 0x7fd3500cb780 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3500ce800 con 0x7fd350159500 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3500cb780 0x7fd350158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3500cb3b0 0x7fd3500cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35824/0 (socket says 172.21.15.3:35824) 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3500cb780 0x7fd350158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32936/0 (socket says 172.21.15.3:32936) 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34e59c700 1 -- 172.21.15.3:0/3830317154 learned_addr learned my addr 172.21.15.3:0/3830317154 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.051+0000 7fd34ed9d700 1 --2- 172.21.15.3:0/3830317154 >> v2:172.21.15.153:3300/0 conn(0x7fd350159500 0x7fd3500cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34dd9b700 1 -- 172.21.15.3:0/3830317154 >> v2:172.21.15.153:3300/0 conn(0x7fd350159500 msgr2=0x7fd3500cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34dd9b700 1 --2- 172.21.15.3:0/3830317154 >> v2:172.21.15.153:3300/0 conn(0x7fd350159500 0x7fd3500cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34dd9b700 1 -- 172.21.15.3:0/3830317154 >> v1:172.21.15.3:6789/0 conn(0x7fd3500cb3b0 legacy=0x7fd3500cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34dd9b700 1 -- 172.21.15.3:0/3830317154 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3500ceae0 con 0x7fd3500cb780 2023-10-06T04:52:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34d59a700 1 -- 172.21.15.3:0/3830317154 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022742073 0 0) 0x7fd3500cf130 con 0x7fd3500cb3b0 2023-10-06T04:52:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34dd9b700 1 --2- 172.21.15.3:0/3830317154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb780 0x7fd350158fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3500cbfd0 tx=0x7fd3380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b64f95916ee6d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34d59a700 1 -- 172.21.15.3:0/3830317154 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800b6b0 con 0x7fd3500cb780 2023-10-06T04:52:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34d59a700 1 -- 172.21.15.3:0/3830317154 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800b810 con 0x7fd3500cb780 2023-10-06T04:52:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34d59a700 1 -- 172.21.15.3:0/3830317154 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800bbb0 con 0x7fd3500cb780 2023-10-06T04:52:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34f59e700 1 -- 172.21.15.3:0/3830317154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb780 msgr2=0x7fd350158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3830317154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb780 0x7fd350158fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3500cbfd0 tx=0x7fd3380091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34f59e700 1 -- 172.21.15.3:0/3830317154 shutdown_connections 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3830317154 >> v2:172.21.15.153:3300/0 conn(0x7fd350159500 0x7fd3500cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3830317154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb780 0x7fd350158fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34f59e700 1 -- 172.21.15.3:0/3830317154 >> 172.21.15.3:0/3830317154 conn(0x7fd350070310 msgr2=0x7fd350070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.052+0000 7fd34f59e700 1 -- 172.21.15.3:0/3830317154 shutdown_connections 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.053+0000 7fd34f59e700 1 -- 172.21.15.3:0/3830317154 wait complete. 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 Processor -- start 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 -- start start 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb3b0 0x7fd350157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cb780 0x7fd350143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350159500 0x7fd350152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500c8c70 con 0x7fd3500cb3b0 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500c8060 con 0x7fd3500cb780 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500c8340 con 0x7fd350159500 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb3b0 0x7fd350157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb3b0 0x7fd350157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32938/0 (socket says 172.21.15.3:32938) 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34e59c700 1 -- 172.21.15.3:0/3701619223 learned_addr learned my addr 172.21.15.3:0/3701619223 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.054+0000 7fd34dd9b700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cb780 0x7fd350143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34ed9d700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350159500 0x7fd350152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34e59c700 1 -- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350159500 msgr2=0x7fd350152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34e59c700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350159500 0x7fd350152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34e59c700 1 -- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cb780 msgr2=0x7fd350143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34e59c700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cb780 0x7fd350143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34e59c700 1 -- 172.21.15.3:0/3701619223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd338009420 con 0x7fd3500cb3b0 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34ed9d700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350159500 0x7fd350152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34e59c700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb3b0 0x7fd350157bf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd34800c7e0 tx=0x7fd348007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd3467fc700 1 -- 172.21.15.3:0/3701619223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348005b30 con 0x7fd3500cb3b0 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd3467fc700 1 -- 172.21.15.3:0/3701619223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348005c90 con 0x7fd3500cb3b0 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3500c8620 con 0x7fd3500cb3b0 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd3467fc700 1 -- 172.21.15.3:0/3701619223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348007300 con 0x7fd3500cb3b0 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.055+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3500b3050 con 0x7fd3500cb3b0 2023-10-06T04:52:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.056+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35004c380 con 0x7fd3500cb3b0 2023-10-06T04:52:24.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.056+0000 7fd3467fc700 1 -- 172.21.15.3:0/3701619223 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3480076b0 con 0x7fd3500cb3b0 2023-10-06T04:52:24.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.057+0000 7fd3467fc700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd330066f30 0x7fd3300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.057+0000 7fd3467fc700 1 -- 172.21.15.3:0/3701619223 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd348083c00 con 0x7fd3500cb3b0 2023-10-06T04:52:24.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.057+0000 7fd34dd9b700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd330066f30 0x7fd3300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.058+0000 7fd34dd9b700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd330066f30 0x7fd3300693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3380093f0 tx=0x7fd338017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.061+0000 7fd3467fc700 1 -- 172.21.15.3:0/3701619223 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd34808a080 con 0x7fd3500cb3b0 2023-10-06T04:52:24.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.254+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3500cedc0 con 0x7fd3500cb3b0 2023-10-06T04:52:24.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.255+0000 7fd3467fc700 1 -- 172.21.15.3:0/3701619223 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fd348017090 con 0x7fd3500cb3b0 2023-10-06T04:52:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:24.257 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd330066f30 msgr2=0x7fd3300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd330066f30 0x7fd3300693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3380093f0 tx=0x7fd338017000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb3b0 msgr2=0x7fd350157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb3b0 0x7fd350157bf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd34800c7e0 tx=0x7fd348007df0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 shutdown_connections 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd330066f30 0x7fd3300693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350159500 0x7fd350152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cb3b0 0x7fd350157bf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 --2- 172.21.15.3:0/3701619223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cb780 0x7fd350143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 >> 172.21.15.3:0/3701619223 conn(0x7fd350070310 msgr2=0x7fd3500c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 shutdown_connections 2023-10-06T04:52:24.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.257+0000 7fd34f59e700 1 -- 172.21.15.3:0/3701619223 wait complete. 2023-10-06T04:52:24.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T04:52:24.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T04:52:24.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T04:52:24.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T04:52:24.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T04:52:24.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T04:52:24.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T04:52:24.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T04:52:24.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.476+0000 7ff17d5c4700 1 Processor -- start 2023-10-06T04:52:24.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.476+0000 7f53ab87e700 1 Processor -- start 2023-10-06T04:52:24.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.477+0000 7ff17d5c4700 1 -- start start 2023-10-06T04:52:24.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.477+0000 7f53ab87e700 1 -- start start 2023-10-06T04:52:24.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7f53ab87e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f53a41622e0 0x7f53a4146e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7f53ab87e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a40add80 con 0x7f53a40b1a50 2023-10-06T04:52:24.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7f53ab87e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a406ca80 con 0x7f53a40b1e20 2023-10-06T04:52:24.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7f53ab87e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a40af220 con 0x7f53a41622e0 2023-10-06T04:52:24.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7f53a961a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f53a40b1a50 0x7f53a406c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35848/0 (socket says 172.21.15.3:35848) 2023-10-06T04:52:24.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7f53a961a700 1 -- 172.21.15.3:0/1036559400 learned_addr learned my addr 172.21.15.3:0/1036559400 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7ff17d5c4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bf6d0 0x7ff1780af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7ff17d5c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff178161ed0 con 0x7ff178161960 2023-10-06T04:52:24.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7ff17d5c4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1781621b0 con 0x7ff1780bfaa0 2023-10-06T04:52:24.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.478+0000 7ff17d5c4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff178162490 con 0x7ff1780bf6d0 2023-10-06T04:52:24.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff1767fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff1780bfaa0 0x7ff17806f0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49482/0 (socket says 172.21.15.3:49482) 2023-10-06T04:52:24.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff1767fc700 1 -- 172.21.15.3:0/2318484607 learned_addr learned my addr 172.21.15.3:0/2318484607 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff176ffd700 1 --2- 172.21.15.3:0/2318484607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bf6d0 0x7ff1780af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500464462 0 0) 0x7f53a40add80 con 0x7f53a40b1a50 2023-10-06T04:52:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5390003570 con 0x7f53a40b1a50 2023-10-06T04:52:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580763967 0 0) 0x7f53a406ca80 con 0x7f53a40b1e20 2023-10-06T04:52:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a40add80 con 0x7f53a40b1e20 2023-10-06T04:52:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090533353 0 0) 0x7f5390003570 con 0x7f53a40b1a50 2023-10-06T04:52:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 >> v2:172.21.15.153:3300/0 conn(0x7f53a41622e0 msgr2=0x7f53a4146e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:52:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff175ffb700 1 -- 172.21.15.3:0/2318484607 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316703430 0 0) 0x7ff1781621b0 con 0x7ff1780bfaa0 2023-10-06T04:52:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff175ffb700 1 -- 172.21.15.3:0/2318484607 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff160003570 con 0x7ff1780bfaa0 2023-10-06T04:52:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff176ffd700 1 -- 172.21.15.3:0/2318484607 >> v1:172.21.15.153:6789/0 conn(0x7ff1780bfaa0 legacy=0x7ff17806f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff176ffd700 1 -- 172.21.15.3:0/2318484607 >> v1:172.21.15.3:6789/0 conn(0x7ff178161960 legacy=0x7ff1780c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff176ffd700 1 -- 172.21.15.3:0/2318484607 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff178162770 con 0x7ff1780bf6d0 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 --2- 172.21.15.3:0/1036559400 >> v2:172.21.15.153:3300/0 conn(0x7f53a41622e0 0x7f53a4146e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 >> v1:172.21.15.153:6789/0 conn(0x7f53a40b1e20 legacy=0x7f53a4147360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a406cd10 con 0x7f53a40b1a50 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f53a0001470 con 0x7f53a40b1a50 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 >> v1:172.21.15.3:6789/0 conn(0x7f53a40b1a50 legacy=0x7f53a406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5390004d20 0x7f5390005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53900037b0 0x7f5390003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7f539bfff700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53900043b0 0x7f539000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40af220 con 0x7f5390004d20 2023-10-06T04:52:24.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539000c3a0 con 0x7f53900037b0 2023-10-06T04:52:24.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539000c560 con 0x7f53900043b0 2023-10-06T04:52:24.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a8e19700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53900037b0 0x7f5390003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff21149e700 1 Processor -- start 2023-10-06T04:52:24.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.479+0000 7ff21149e700 1 -- start start 2023-10-06T04:52:24.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a961a700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5390004d20 0x7f5390005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7ff176ffd700 1 --2- 172.21.15.3:0/2318484607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bf6d0 0x7ff1780af3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009910 tx=0x7ff16c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14cab02d3abe8ca2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a9e1b700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53900043b0 0x7f539000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7ff175ffb700 1 -- 172.21.15.3:0/2318484607 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c011630 con 0x7ff1780bf6d0 2023-10-06T04:52:24.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7ff175ffb700 1 -- 172.21.15.3:0/2318484607 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c011790 con 0x7ff1780bf6d0 2023-10-06T04:52:24.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7ff175ffb700 1 -- 172.21.15.3:0/2318484607 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c011b50 con 0x7ff1780bf6d0 2023-10-06T04:52:24.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a961a700 1 -- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53900043b0 msgr2=0x7f539000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a961a700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53900043b0 0x7f539000bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a961a700 1 -- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53900037b0 msgr2=0x7f5390003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a961a700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53900037b0 0x7f5390003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a961a700 1 -- 172.21.15.3:0/1036559400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5390003570 con 0x7f5390004d20 2023-10-06T04:52:24.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a9e1b700 1 -- 172.21.15.3:0/1036559400 reap_dead start 2023-10-06T04:52:24.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f44d26eb700 1 Processor -- start 2023-10-06T04:52:24.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f44d26eb700 1 -- start start 2023-10-06T04:52:24.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f50d3268700 1 Processor -- start 2023-10-06T04:52:24.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f61f803e700 1 Processor -- start 2023-10-06T04:52:24.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.480+0000 7f53a961a700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5390004d20 0x7f5390005110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c210 tx=0x7f53a0001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e849986ace5dd93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff21149e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0699e0 0x7ff20c0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff21149e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c0aee10 con 0x7ff20c15db90 2023-10-06T04:52:24.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff21149e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c0c7fe0 con 0x7ff20c0c6e30 2023-10-06T04:52:24.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff21149e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0c73d0 con 0x7ff20c0699e0 2023-10-06T04:52:24.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff17d5c4700 1 -- 172.21.15.3:0/2318484607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bf6d0 msgr2=0x7ff1780af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff17d5c4700 1 --2- 172.21.15.3:0/2318484607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bf6d0 0x7ff1780af3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009910 tx=0x7ff16c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff20a7fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff20c0c6e30 0x7ff20c15d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49500/0 (socket says 172.21.15.3:49500) 2023-10-06T04:52:24.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff20a7fc700 1 -- 172.21.15.3:0/1648529999 learned_addr learned my addr 172.21.15.3:0/1648529999 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7ff20affd700 1 --2- 172.21.15.3:0/1648529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0699e0 0x7ff20c0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7f61f803e700 1 -- start start 2023-10-06T04:52:24.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7f50d3268700 1 -- start start 2023-10-06T04:52:24.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff20affd700 1 -- 172.21.15.3:0/1648529999 >> v1:172.21.15.153:6789/0 conn(0x7ff20c0c6e30 legacy=0x7ff20c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff20affd700 1 -- 172.21.15.3:0/1648529999 >> v1:172.21.15.3:6789/0 conn(0x7ff20c15db90 legacy=0x7ff20c0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff20affd700 1 -- 172.21.15.3:0/1648529999 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c0c76b0 con 0x7ff20c0699e0 2023-10-06T04:52:24.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff17d5c4700 1 -- 172.21.15.3:0/2318484607 shutdown_connections 2023-10-06T04:52:24.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff17d5c4700 1 --2- 172.21.15.3:0/2318484607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bf6d0 0x7ff1780af3f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff17d5c4700 1 -- 172.21.15.3:0/2318484607 >> 172.21.15.3:0/2318484607 conn(0x7ff178070310 msgr2=0x7ff178070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff20affd700 1 --2- 172.21.15.3:0/1648529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0699e0 0x7ff20c0c68f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009910 tx=0x7ff1fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7869ea4d01bc8ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a000cc10 con 0x7f5390004d20 2023-10-06T04:52:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a000cd70 con 0x7f5390004d20 2023-10-06T04:52:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.481+0000 7f539bfff700 1 -- 172.21.15.3:0/1036559400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0005b70 con 0x7f5390004d20 2023-10-06T04:52:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f53ab87e700 1 -- 172.21.15.3:0/1036559400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5390004d20 msgr2=0x7f5390005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f53ab87e700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5390004d20 0x7f5390005110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c210 tx=0x7f53a0001d40 comp rx=0 tx=0).stop 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44d26eb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44d26eb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44cc1406b0 0x7f44cc140af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44d26eb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc0cd380 con 0x7f44cc0b3a40 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44d26eb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44cc0cd660 con 0x7f44cc1406b0 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44d26eb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0cd940 con 0x7f44cc0b3e10 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59378/0 (socket says 172.21.15.3:59378) 2023-10-06T04:52:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44cb7fe700 1 -- 172.21.15.3:0/3327986195 learned_addr learned my addr 172.21.15.3:0/3327986195 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f803e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f006ecc0 0x7f61f01557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f803e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61f006a110 0x7f61f00c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f803e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f00c8ba0 con 0x7f61f0155400 2023-10-06T04:52:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f803e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f00ab9e0 con 0x7f61f006a110 2023-10-06T04:52:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f803e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00c8e00 con 0x7f61f006ecc0 2023-10-06T04:52:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff209ffb700 1 -- 172.21.15.3:0/1648529999 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00b820 con 0x7ff20c0699e0 2023-10-06T04:52:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff209ffb700 1 -- 172.21.15.3:0/1648529999 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc00b9a0 con 0x7ff20c0699e0 2023-10-06T04:52:24.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff209ffb700 1 -- 172.21.15.3:0/1648529999 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00bd60 con 0x7ff20c0699e0 2023-10-06T04:52:24.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7ff21149e700 1 -- 172.21.15.3:0/1648529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0699e0 msgr2=0x7ff20c0c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7ff21149e700 1 --2- 172.21.15.3:0/1648529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0699e0 0x7ff20c0c68f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009910 tx=0x7ff1fc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff17d5c4700 1 -- 172.21.15.3:0/2318484607 shutdown_connections 2023-10-06T04:52:24.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7ff17d5c4700 1 -- 172.21.15.3:0/2318484607 wait complete. 2023-10-06T04:52:24.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d3268700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f50cc155400 0x7f50cc1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d3268700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50cc06ecc0 0x7f50cc0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d3268700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50cc0c8ba0 con 0x7f50cc0cbb60 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d3268700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50cc0ab9e0 con 0x7f50cc06ecc0 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d3268700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50cc0c8e00 con 0x7f50cc155400 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d1004700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f50cc155400 0x7f50cc1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d0803700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f50cc0cbb60 0x7f50cc06e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35868/0 (socket says 172.21.15.3:35868) 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d1004700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f50cc155400 0x7f50cc1557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:37812/0 (socket says 172.21.15.3:37812) 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d0803700 1 -- 172.21.15.3:0/2981550461 learned_addr learned my addr 172.21.15.3:0/2981550461 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f53ab87e700 1 -- 172.21.15.3:0/1036559400 shutdown_connections 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f53ab87e700 1 --2- 172.21.15.3:0/1036559400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5390004d20 0x7f5390005110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f53ab87e700 1 -- 172.21.15.3:0/1036559400 >> 172.21.15.3:0/1036559400 conn(0x7f53a4066fb0 msgr2=0x7f53a40673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f53ab87e700 1 -- 172.21.15.3:0/1036559400 shutdown_connections 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f53ab87e700 1 -- 172.21.15.3:0/1036559400 wait complete. 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44cb7fe700 1 -- 172.21.15.3:0/3327986195 >> v2:172.21.15.153:3300/0 conn(0x7f44cc1406b0 msgr2=0x7f44cc140af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44cb7fe700 1 --2- 172.21.15.3:0/3327986195 >> v2:172.21.15.153:3300/0 conn(0x7f44cc1406b0 0x7f44cc140af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44cb7fe700 1 -- 172.21.15.3:0/3327986195 >> v1:172.21.15.153:6789/0 conn(0x7f44cc0b3a40 legacy=0x7f44cc0680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f44cb7fe700 1 -- 172.21.15.3:0/3327986195 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0cdc20 con 0x7f44cc0b3e10 2023-10-06T04:52:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f65db700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61f006a110 0x7f61f00c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f65db700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61f006a110 0x7f61f00c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:37810/0 (socket says 172.21.15.3:37810) 2023-10-06T04:52:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f65db700 1 -- 172.21.15.3:0/3604237369 learned_addr learned my addr 172.21.15.3:0/3604237369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.482+0000 7f61f55d9700 1 --2- 172.21.15.3:0/3604237369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f006ecc0 0x7f61f01557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f50d1805700 1 --2- 172.21.15.3:0/2981550461 >> v2:172.21.15.3:3300/0 conn(0x7f50cc06ecc0 0x7f50cc0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f50c3fff700 1 -- 172.21.15.3:0/2981550461 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376986572 0 0) 0x7f50cc0c8ba0 con 0x7f50cc0cbb60 2023-10-06T04:52:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f50c3fff700 1 -- 172.21.15.3:0/2981550461 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b8003570 con 0x7f50cc0cbb60 2023-10-06T04:52:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7ff21149e700 1 -- 172.21.15.3:0/1648529999 shutdown_connections 2023-10-06T04:52:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7ff21149e700 1 --2- 172.21.15.3:0/1648529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0699e0 0x7ff20c0c68f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7ff21149e700 1 -- 172.21.15.3:0/1648529999 >> 172.21.15.3:0/1648529999 conn(0x7ff20c06c370 msgr2=0x7ff20c06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f50d1004700 1 -- 172.21.15.3:0/2981550461 >> v2:172.21.15.3:3300/0 conn(0x7f50cc06ecc0 msgr2=0x7f50cc0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f50d1004700 1 --2- 172.21.15.3:0/2981550461 >> v2:172.21.15.3:3300/0 conn(0x7f50cc06ecc0 0x7f50cc0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f50d1004700 1 -- 172.21.15.3:0/2981550461 >> v1:172.21.15.3:6789/0 conn(0x7f50cc0cbb60 legacy=0x7f50cc06e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f50d1004700 1 -- 172.21.15.3:0/2981550461 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc0c90e0 con 0x7f50cc155400 2023-10-06T04:52:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f61f55d9700 1 -- 172.21.15.3:0/3604237369 >> v2:172.21.15.153:3300/0 conn(0x7f61f006a110 msgr2=0x7f61f00c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f61f55d9700 1 --2- 172.21.15.3:0/3604237369 >> v2:172.21.15.153:3300/0 conn(0x7f61f006a110 0x7f61f00c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f61f55d9700 1 -- 172.21.15.3:0/3604237369 >> v1:172.21.15.153:6789/0 conn(0x7f61f0155400 legacy=0x7f61f006e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f61f55d9700 1 -- 172.21.15.3:0/3604237369 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00c90e0 con 0x7f61f006ecc0 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f61f55d9700 1 --2- 172.21.15.3:0/3604237369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f006ecc0 0x7f61f01557d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0098e0 tx=0x7f61ec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338d72b848984b03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.483+0000 7f61f4dd8700 1 -- 172.21.15.3:0/3604237369 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00b640 con 0x7f61f006ecc0 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f61f4dd8700 1 -- 172.21.15.3:0/3604237369 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec00b7a0 con 0x7f61f006ecc0 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f61f4dd8700 1 -- 172.21.15.3:0/3604237369 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00bb40 con 0x7f61f006ecc0 2023-10-06T04:52:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7ff17d5c4700 1 Processor -- start 2023-10-06T04:52:24.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f53ab87e700 1 Processor -- start 2023-10-06T04:52:24.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7ff17d5c4700 1 -- start start 2023-10-06T04:52:24.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f50d1004700 1 --2- 172.21.15.3:0/2981550461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc155400 0x7f50cc1557d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f50cc069580 tx=0x7f50bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8182ff27787106a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f61f803e700 1 -- 172.21.15.3:0/3604237369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f006ecc0 msgr2=0x7f61f01557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f61f803e700 1 --2- 172.21.15.3:0/3604237369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f006ecc0 0x7f61f01557d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0098e0 tx=0x7f61ec0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f44cb7fe700 1 --2- 172.21.15.3:0/3327986195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0ccd30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0099e0 tx=0x7f44bc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d9b96188cd3017 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff21149e700 1 -- 172.21.15.3:0/1648529999 shutdown_connections 2023-10-06T04:52:24.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.484+0000 7f53ab87e700 1 -- start start 2023-10-06T04:52:24.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f61f803e700 1 -- 172.21.15.3:0/3604237369 shutdown_connections 2023-10-06T04:52:24.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f61f803e700 1 --2- 172.21.15.3:0/3604237369 >> v2:172.21.15.153:3300/0 conn(0x7f61f006a110 0x7f61f00c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f61f803e700 1 --2- 172.21.15.3:0/3604237369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f006ecc0 0x7f61f01557d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f61f803e700 1 -- 172.21.15.3:0/3604237369 >> 172.21.15.3:0/3604237369 conn(0x7f61f006c370 msgr2=0x7f61f006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f44caffd700 1 -- 172.21.15.3:0/3327986195 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00b7e0 con 0x7f44cc0b3e10 2023-10-06T04:52:24.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f44caffd700 1 -- 172.21.15.3:0/3327986195 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc00b940 con 0x7f44cc0b3e10 2023-10-06T04:52:24.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f44caffd700 1 -- 172.21.15.3:0/3327986195 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00bce0 con 0x7f44cc0b3e10 2023-10-06T04:52:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff17d5c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1780bf6d0 0x7ff17815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff17d5c4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bfaa0 0x7ff1780c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff17d5c4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff178161960 0x7ff1780cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff17d5c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17815e0e0 con 0x7ff1780bf6d0 2023-10-06T04:52:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff17d5c4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17816e130 con 0x7ff178161960 2023-10-06T04:52:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff17d5c4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17816de40 con 0x7ff1780bfaa0 2023-10-06T04:52:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7f61f803e700 1 -- 172.21.15.3:0/3604237369 shutdown_connections 2023-10-06T04:52:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1780bf6d0 0x7ff17815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff1767fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bfaa0 0x7ff1780c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1780bf6d0 0x7ff17815dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32958/0 (socket says 172.21.15.3:32958) 2023-10-06T04:52:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 -- 172.21.15.3:0/305759058 learned_addr learned my addr 172.21.15.3:0/305759058 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff1767fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bfaa0 0x7ff1780c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59386/0 (socket says 172.21.15.3:59386) 2023-10-06T04:52:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53ab87e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5390004d20 0x7f53a4140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53ab87e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a4145b90 0x7f53a4146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53ab87e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a4141dd0 0x7f53a4142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53ab87e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41427b0 con 0x7f53a4145b90 2023-10-06T04:52:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53ab87e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4146710 con 0x7f53a4141dd0 2023-10-06T04:52:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53ab87e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a415f2a0 con 0x7f5390004d20 2023-10-06T04:52:24.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f85505aa700 1 Processor -- start 2023-10-06T04:52:24.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a8e19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a4145b90 0x7f53a4146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a961a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5390004d20 0x7f53a4140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a961a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5390004d20 0x7f53a4140210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59396/0 (socket says 172.21.15.3:59396) 2023-10-06T04:52:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a8e19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a4145b90 0x7f53a4146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32974/0 (socket says 172.21.15.3:32974) 2023-10-06T04:52:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a961a700 1 -- 172.21.15.3:0/1359923477 learned_addr learned my addr 172.21.15.3:0/1359923477 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f44d26eb700 1 -- 172.21.15.3:0/3327986195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 msgr2=0x7f44cc0ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f44d26eb700 1 --2- 172.21.15.3:0/3327986195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0ccd30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0099e0 tx=0x7f44bc009250 comp rx=0 tx=0).stop 2023-10-06T04:52:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f61f803e700 1 -- 172.21.15.3:0/3604237369 wait complete. 2023-10-06T04:52:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 -- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bfaa0 msgr2=0x7ff1780c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1780bfaa0 0x7ff1780c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 -- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff178161960 msgr2=0x7ff1780cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff178161960 0x7ff1780cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 -- 172.21.15.3:0/305759058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c009420 con 0x7ff1780bf6d0 2023-10-06T04:52:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7ff176ffd700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1780bf6d0 0x7ff17815dba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0030c0 tx=0x7ff16c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.485+0000 7ff21149e700 1 -- 172.21.15.3:0/1648529999 wait complete. 2023-10-06T04:52:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7ff21149e700 1 Processor -- start 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7ff166ffd700 1 -- 172.21.15.3:0/305759058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c027480 con 0x7ff1780bf6d0 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7ff166ffd700 1 -- 172.21.15.3:0/305759058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c020070 con 0x7ff1780bf6d0 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1780c6790 con 0x7ff1780bf6d0 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7ff166ffd700 1 -- 172.21.15.3:0/305759058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c027850 con 0x7ff1780bf6d0 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1780c61d0 con 0x7ff1780bf6d0 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f85505aa700 1 -- start start 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7f85505aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f854815c8f0 0x7f854815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f85505aa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85480cf6c0 con 0x7f85480d0870 2023-10-06T04:52:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f85505aa700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85480cfb50 con 0x7f854815ccc0 2023-10-06T04:52:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f85505aa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854815e050 con 0x7f854815c8f0 2023-10-06T04:52:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a8e19700 1 -- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5390004d20 msgr2=0x7f53a4140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a8e19700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5390004d20 0x7f53a4140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a8e19700 1 -- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a4141dd0 msgr2=0x7f53a4142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a8e19700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a4141dd0 0x7f53a4142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a8e19700 1 -- 172.21.15.3:0/1359923477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a415f580 con 0x7f53a4145b90 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.486+0000 7f53a961a700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5390004d20 0x7f53a4140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7f53a8e19700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a4145b90 0x7f53a4146000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f53a4061120 tx=0x7f5394006f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394012600 con 0x7f53a4145b90 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5394012760 con 0x7f53a4145b90 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394012b20 con 0x7f53a4145b90 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a415f860 con 0x7f53a4145b90 2023-10-06T04:52:24.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a4140750 con 0x7f53a4145b90 2023-10-06T04:52:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f61f803e700 1 Processor -- start 2023-10-06T04:52:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f61f803e700 1 -- start start 2023-10-06T04:52:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.487+0000 7ff21149e700 1 -- start start 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff21149e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff20c0699e0 0x7ff20c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff21149e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0c6e30 0x7ff20c16dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff21149e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff20c15db90 0x7ff20c15f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff21149e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0c4000 con 0x7ff20c0699e0 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff21149e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0c42e0 con 0x7ff20c15db90 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff21149e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c16e590 con 0x7ff20c0c6e30 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff20b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff20c15db90 0x7ff20c15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff20b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff20c15db90 0x7ff20c15f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:37846/0 (socket says 172.21.15.3:37846) 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff20b7fe700 1 -- 172.21.15.3:0/996643509 learned_addr learned my addr 172.21.15.3:0/996643509 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7ff20a7fc700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0c6e30 0x7ff20c16dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f44d26eb700 1 -- 172.21.15.3:0/3327986195 shutdown_connections 2023-10-06T04:52:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f44d26eb700 1 --2- 172.21.15.3:0/3327986195 >> v2:172.21.15.153:3300/0 conn(0x7f44cc1406b0 0x7f44cc140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f44d26eb700 1 --2- 172.21.15.3:0/3327986195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0ccd30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f44d26eb700 1 -- 172.21.15.3:0/3327986195 >> 172.21.15.3:0/3327986195 conn(0x7f44cc0ce270 msgr2=0x7f44cc0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff20affd700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff20c0699e0 0x7ff20c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f44d26eb700 1 -- 172.21.15.3:0/3327986195 shutdown_connections 2023-10-06T04:52:24.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff20affd700 1 -- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0c6e30 msgr2=0x7ff20c16dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff20affd700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff20c0c6e30 0x7ff20c16dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff20affd700 1 -- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff20c15db90 msgr2=0x7ff20c15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff20affd700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff20c15db90 0x7ff20c15f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff20affd700 1 -- 172.21.15.3:0/996643509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc009420 con 0x7ff20c0699e0 2023-10-06T04:52:24.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff20affd700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff20c0699e0 0x7ff20c0c5a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff20c162690 tx=0x7ff1fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f803e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61f006a110 0x7f61f00c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f803e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61f006ecc0 0x7f61f0176d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f803e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f0155400 0x7f61f0173790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f803e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00c3030 con 0x7f61f006a110 2023-10-06T04:52:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f803e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0177250 con 0x7f61f006ecc0 2023-10-06T04:52:24.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f803e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0173cd0 con 0x7f61f0155400 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f854e346700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f854815c8f0 0x7f854815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f854e346700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f854815c8f0 0x7f854815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32982/0 (socket says 172.21.15.3:32982) 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f854e346700 1 -- 172.21.15.3:0/3551444445 learned_addr learned my addr 172.21.15.3:0/3551444445 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f854e346700 1 -- 172.21.15.3:0/3551444445 >> v1:172.21.15.153:6789/0 conn(0x7f854815ccc0 legacy=0x7f85480d0160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f854e346700 1 -- 172.21.15.3:0/3551444445 >> v1:172.21.15.3:6789/0 conn(0x7f85480d0870 legacy=0x7f85480b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.488+0000 7f854e346700 1 -- 172.21.15.3:0/3551444445 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480ceab0 con 0x7f854815c8f0 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f854e346700 1 --2- 172.21.15.3:0/3551444445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854815c8f0 0x7f854815db10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85440099e0 tx=0x7f8544009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463f09358ff0589e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f55d9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61f006ecc0 0x7f61f0176d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f61f65db700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f0155400 0x7f61f0173790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61f006ecc0 0x7f61f0176d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:37858/0 (socket says 172.21.15.3:37858) 2023-10-06T04:52:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 -- 172.21.15.3:0/962507702 learned_addr learned my addr 172.21.15.3:0/962507702 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f65db700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f0155400 0x7f61f0173790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59408/0 (socket says 172.21.15.3:59408) 2023-10-06T04:52:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f854d344700 1 -- 172.21.15.3:0/3551444445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544014b70 con 0x7f854815c8f0 2023-10-06T04:52:24.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7ff166ffd700 1 -- 172.21.15.3:0/305759058 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff16c01c450 con 0x7ff1780bf6d0 2023-10-06T04:52:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7ff166ffd700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff15c066f00 0x7ff15c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7ff166ffd700 1 -- 172.21.15.3:0/305759058 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff16c022070 con 0x7ff1780bf6d0 2023-10-06T04:52:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f854d344700 1 -- 172.21.15.3:0/3551444445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8544014cd0 con 0x7f854815c8f0 2023-10-06T04:52:24.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f854d344700 1 -- 172.21.15.3:0/3551444445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854400b800 con 0x7f854815c8f0 2023-10-06T04:52:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 -- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f0155400 msgr2=0x7f61f0173790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61f0155400 0x7f61f0173790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 -- 172.21.15.3:0/962507702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61f006a110 msgr2=0x7f61f00c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61f006a110 0x7f61f00c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 -- 172.21.15.3:0/962507702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec009420 con 0x7f61f006ecc0 2023-10-06T04:52:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61f55d9700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61f006ecc0 0x7f61f0176d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f6b0 tx=0x7f61ec017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7ff1767fc700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff15c066f00 0x7ff15c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7ff17d5c4700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7ff1540015f0 0x7ff154003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 --> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff154006b90 con 0x7ff1540015f0 2023-10-06T04:52:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.489+0000 7f44d26eb700 1 -- 172.21.15.3:0/3327986195 wait complete. 2023-10-06T04:52:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f44d26eb700 1 Processor -- start 2023-10-06T04:52:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7ff1777fe700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7ff1540015f0 0x7ff154003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f85505aa700 1 -- 172.21.15.3:0/3551444445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854815c8f0 msgr2=0x7f854815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f85505aa700 1 --2- 172.21.15.3:0/3551444445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854815c8f0 0x7f854815db10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85440099e0 tx=0x7f8544009250 comp rx=0 tx=0).stop 2023-10-06T04:52:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f50c3fff700 1 -- 172.21.15.3:0/2981550461 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc011630 con 0x7f50cc155400 2023-10-06T04:52:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f50c3fff700 1 -- 172.21.15.3:0/2981550461 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc011790 con 0x7f50cc155400 2023-10-06T04:52:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f50c3fff700 1 -- 172.21.15.3:0/2981550461 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc011b30 con 0x7f50cc155400 2023-10-06T04:52:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7ff1767fc700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff15c066f00 0x7ff15c0693b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1781610e0 tx=0x7ff168009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc011e40 con 0x7ff20c0699e0 2023-10-06T04:52:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc01c450 con 0x7ff20c0699e0 2023-10-06T04:52:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc01c7d0 con 0x7ff20c0699e0 2023-10-06T04:52:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c16e870 con 0x7ff20c0699e0 2023-10-06T04:52:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c16e280 con 0x7ff20c0699e0 2023-10-06T04:52:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7ff1777fe700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7ff1540015f0 0x7ff154003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53a41408e0 con 0x7f53a4145b90 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f44d26eb700 1 -- start start 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d26eb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44cc0b3a40 0x7f44cc1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d26eb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d26eb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44cc1406b0 0x7f44cc1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d26eb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc1559c0 con 0x7f44cc1406b0 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d26eb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0c3780 con 0x7f44cc0b3a40 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d26eb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc154db0 con 0x7f44cc0b3e10 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d0c88700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44cc1406b0 0x7f44cc1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d0c88700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44cc1406b0 0x7f44cc1523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33008/0 (socket says 172.21.15.3:33008) 2023-10-06T04:52:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44d0c88700 1 -- 172.21.15.3:0/3864435151 learned_addr learned my addr 172.21.15.3:0/3864435151 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f85505aa700 1 -- 172.21.15.3:0/3551444445 shutdown_connections 2023-10-06T04:52:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f85505aa700 1 --2- 172.21.15.3:0/3551444445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854815c8f0 0x7f854815db10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f85505aa700 1 -- 172.21.15.3:0/3551444445 >> 172.21.15.3:0/3551444445 conn(0x7f8548070310 msgr2=0x7f8548070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f50d3268700 1 -- 172.21.15.3:0/2981550461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc155400 msgr2=0x7f50cc1557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f50d3268700 1 --2- 172.21.15.3:0/2981550461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc155400 0x7f50cc1557d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f50cc069580 tx=0x7f50bc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f50d3268700 1 -- 172.21.15.3:0/2981550461 shutdown_connections 2023-10-06T04:52:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f50d3268700 1 --2- 172.21.15.3:0/2981550461 >> v2:172.21.15.3:3300/0 conn(0x7f50cc06ecc0 0x7f50cc0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f50d3268700 1 --2- 172.21.15.3:0/2981550461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc155400 0x7f50cc1557d0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f50cc069580 tx=0x7f50bc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f50d3268700 1 -- 172.21.15.3:0/2981550461 >> 172.21.15.3:0/2981550461 conn(0x7f50cc06c370 msgr2=0x7f50cc06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44cbfff700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44cc0b3a40 0x7f44cc1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f50d3268700 1 -- 172.21.15.3:0/2981550461 shutdown_connections 2023-10-06T04:52:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f50d3268700 1 -- 172.21.15.3:0/2981550461 wait complete. 2023-10-06T04:52:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f539401b400 con 0x7f53a4145b90 2023-10-06T04:52:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f53997fa700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5388066fb0 0x7f5388069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5394084400 con 0x7f53a4145b90 2023-10-06T04:52:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f53997fa700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f538806f7e0 0x7f5388071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 --> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53880722a0 con 0x7f538806f7e0 2023-10-06T04:52:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f53a9e1b700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f538806f7e0 0x7f5388071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44cb7fe700 1 -- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44cc0b3a40 msgr2=0x7f44cc1546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44cb7fe700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44cc0b3a40 0x7f44cc1546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f44cb7fe700 1 -- 172.21.15.3:0/3864435151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44cc1406b0 msgr2=0x7f44cc1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f44cb7fe700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44cc1406b0 0x7f44cc1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f44cb7fe700 1 -- 172.21.15.3:0/3864435151 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0094f0 con 0x7f44cc0b3e10 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f44cbfff700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44cc0b3a40 0x7f44cc1546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f44cb7fe700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0c3240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44cc06b6e0 tx=0x7f44bc016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff20c071680 con 0x7ff20c0699e0 2023-10-06T04:52:24.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f53a9e1b700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f538806f7e0 0x7f5388071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.493+0000 7f53a961a700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5388066fb0 0x7f5388069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec01bcb0 con 0x7f61f006ecc0 2023-10-06T04:52:24.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec01be30 con 0x7f61f006ecc0 2023-10-06T04:52:24.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.490+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec011670 con 0x7f61f006ecc0 2023-10-06T04:52:24.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f00ca1e0 con 0x7f61f006ecc0 2023-10-06T04:52:24.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.491+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f00ca7a0 con 0x7f61f006ecc0 2023-10-06T04:52:24.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.492+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61f0071680 con 0x7f61f006ecc0 2023-10-06T04:52:24.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5394051470 con 0x7f53a4145b90 2023-10-06T04:52:24.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7f50d3268700 1 Processor -- start 2023-10-06T04:52:24.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== osd.4 v2:172.21.15.153:6817/90907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f53880722a0 con 0x7f538806f7e0 2023-10-06T04:52:24.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7f85505aa700 1 -- 172.21.15.3:0/3551444445 shutdown_connections 2023-10-06T04:52:24.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7ff166ffd700 1 -- 172.21.15.3:0/305759058 <== osd.1 v2:172.21.15.3:6808/90655 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff154006b90 con 0x7ff1540015f0 2023-10-06T04:52:24.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f85505aa700 1 -- 172.21.15.3:0/3551444445 wait complete. 2023-10-06T04:52:24.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7f50d3268700 1 -- start start 2023-10-06T04:52:24.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f50d3268700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc06ecc0 0x7f50cc0af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f50d3268700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50cc0cbb60 0x7f50cc0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f50d3268700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50cc16e050 0x7f50cc16e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f50d3268700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc0aee10 con 0x7f50cc0cbb60 2023-10-06T04:52:24.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f50d3268700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc16eb60 con 0x7f50cc06ecc0 2023-10-06T04:52:24.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f50d3268700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc16ee40 con 0x7f50cc16e050 2023-10-06T04:52:24.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1004700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc06ecc0 0x7f50cc0af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1004700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc06ecc0 0x7f50cc0af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:37876/0 (socket says 172.21.15.3:37876) 2023-10-06T04:52:24.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1004700 1 -- 172.21.15.3:0/1251078724 learned_addr learned my addr 172.21.15.3:0/1251078724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1805700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50cc16e050 0x7f50cc16e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7f53a961a700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5388066fb0 0x7f5388069460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f53a0007ec0 tx=0x7f53a0015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f44b3fff700 1 -- 172.21.15.3:0/3864435151 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc01b470 con 0x7f44cc0b3e10 2023-10-06T04:52:24.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f44b3fff700 1 -- 172.21.15.3:0/3864435151 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc01b5d0 con 0x7f44cc0b3e10 2023-10-06T04:52:24.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc155090 con 0x7f44cc0b3e10 2023-10-06T04:52:24.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc155650 con 0x7f44cc0b3e10 2023-10-06T04:52:24.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.494+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61ec0119e0 con 0x7f61f006ecc0 2023-10-06T04:52:24.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f61e5ffb700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61d4066f30 0x7f61d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d0803700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50cc0cbb60 0x7f50cc0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1fc01cb40 con 0x7ff20c0699e0 2023-10-06T04:52:24.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7ff1f2ffd700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1f8067080 0x7ff1f8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1fc0868a0 con 0x7ff20c0699e0 2023-10-06T04:52:24.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7ff1f2ffd700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7ff1f806f930 0x7ff1f8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 --> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1f80723f0 con 0x7ff1f806f930 2023-10-06T04:52:24.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1fc086bd0 con 0x7ff20c0699e0 2023-10-06T04:52:24.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f85505aa700 1 Processor -- start 2023-10-06T04:52:24.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1805700 1 -- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc06ecc0 msgr2=0x7f50cc0af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1805700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc06ecc0 0x7f50cc0af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1805700 1 -- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50cc0cbb60 msgr2=0x7f50cc0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1805700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50cc0cbb60 0x7f50cc0c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1805700 1 -- 172.21.15.3:0/1251078724 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc009420 con 0x7f50cc16e050 2023-10-06T04:52:24.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1004700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50cc06ecc0 0x7f50cc0af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:24.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.495+0000 7f44b3fff700 1 -- 172.21.15.3:0/3864435151 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc01b950 con 0x7f44cc0b3e10 2023-10-06T04:52:24.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f85505aa700 1 -- start start 2023-10-06T04:52:24.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50d1805700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50cc16e050 0x7f50cc16e4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50c4007ea0 tx=0x7f50c4009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f44b3fff700 1 -- 172.21.15.3:0/3864435151 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44bc01c450 con 0x7f44cc0b3e10 2023-10-06T04:52:24.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4018070 con 0x7f50cc16e050 2023-10-06T04:52:24.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.496+0000 7feac80d5700 1 Processor -- start 2023-10-06T04:52:24.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c400eac0 con 0x7f50cc16e050 2023-10-06T04:52:24.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c400ee60 con 0x7f50cc16e050 2023-10-06T04:52:24.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f44b3fff700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44b8067000 0x7f44b80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f44b3fff700 1 -- 172.21.15.3:0/3864435151 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44bc086600 con 0x7f44cc0b3e10 2023-10-06T04:52:24.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f44cbfff700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44b8067000 0x7f44b80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f85505aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85480d0870 0x7f8548177490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f85505aa700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f854815c8f0 0x7f85481699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f85505aa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854817a010 0x7f8548152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f85505aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8548068830 con 0x7f85480d0870 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f85505aa700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854817a4a0 con 0x7f854817a010 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7f85505aa700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854817a780 con 0x7f854815c8f0 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f854815c8f0 0x7f85481699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854eb47700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854817a010 0x7f8548152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f854815c8f0 0x7f85481699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59442/0 (socket says 172.21.15.3:59442) 2023-10-06T04:52:24.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 -- 172.21.15.3:0/2989807719 learned_addr learned my addr 172.21.15.3:0/2989807719 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f61ec056d40 con 0x7f61f006ecc0 2023-10-06T04:52:24.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7ff20a7fc700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1f8067080 0x7ff1f8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f61e5ffb700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f61d406f7e0 0x7f61d4071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 --> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61d40722a0 con 0x7f61d406f7e0 2023-10-06T04:52:24.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61ec053080 con 0x7f61f006ecc0 2023-10-06T04:52:24.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 -- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854817a010 msgr2=0x7f8548152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854817a010 0x7f8548152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 -- 172.21.15.3:0/2989807719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85480d0870 msgr2=0x7f8548177490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85480d0870 0x7f8548177490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f61f65db700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f61d406f7e0 0x7f61d4071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 -- 172.21.15.3:0/2989807719 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85440094f0 con 0x7f854815c8f0 2023-10-06T04:52:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7ff20a7fc700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1f8067080 0x7ff1f8069530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff200007130 tx=0x7ff200007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f854db45700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f854815c8f0 0x7f85481699c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f853800b5e0 tx=0x7f85380090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853800f510 con 0x7f854815c8f0 2023-10-06T04:52:24.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853800f6b0 con 0x7f854815c8f0 2023-10-06T04:52:24.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853800fa70 con 0x7f854815c8f0 2023-10-06T04:52:24.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854817aa60 con 0x7f854815c8f0 2023-10-06T04:52:24.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8548151e30 con 0x7f854815c8f0 2023-10-06T04:52:24.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f61f5dda700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61d4066f30 0x7f61d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc0af060 con 0x7f50cc16e050 2023-10-06T04:52:24.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc0afda0 con 0x7f50cc16e050 2023-10-06T04:52:24.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50cc16f060 con 0x7f50cc16e050 2023-10-06T04:52:24.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8548071680 con 0x7f854815c8f0 2023-10-06T04:52:24.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.497+0000 7feac80d5700 1 -- start start 2023-10-06T04:52:24.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac80d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac0155400 0x7feac01557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac80d5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feac006a110 0x7feac00c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac80d5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac00c8ba0 con 0x7feac006e5b0 2023-10-06T04:52:24.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac80d5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac00ab9e0 con 0x7feac006a110 2023-10-06T04:52:24.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac80d5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c8e00 con 0x7feac0155400 2023-10-06T04:52:24.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac5e71700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac0155400 0x7feac01557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac5e71700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac0155400 0x7feac01557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59452/0 (socket says 172.21.15.3:59452) 2023-10-06T04:52:24.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac5e71700 1 -- 172.21.15.3:0/727400708 learned_addr learned my addr 172.21.15.3:0/727400708 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac5670700 1 --2- 172.21.15.3:0/727400708 >> v2:172.21.15.3:3300/0 conn(0x7feac006a110 0x7feac00c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac5e71700 1 -- 172.21.15.3:0/727400708 >> v2:172.21.15.3:3300/0 conn(0x7feac006a110 msgr2=0x7feac00c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac5e71700 1 --2- 172.21.15.3:0/727400708 >> v2:172.21.15.3:3300/0 conn(0x7feac006a110 0x7feac00c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.498+0000 7feac5e71700 1 -- 172.21.15.3:0/727400708 >> v1:172.21.15.3:6789/0 conn(0x7feac006e5b0 legacy=0x7feac00c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7feac5e71700 1 -- 172.21.15.3:0/727400708 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00c90e0 con 0x7feac0155400 2023-10-06T04:52:24.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7feac5e71700 1 --2- 172.21.15.3:0/727400708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac0155400 0x7feac01557d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feabc0099b0 tx=0x7feabc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cde6fa94b8a59b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7feac4e6f700 1 -- 172.21.15.3:0/727400708 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00b820 con 0x7feac0155400 2023-10-06T04:52:24.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7feac4e6f700 1 -- 172.21.15.3:0/727400708 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc00b980 con 0x7feac0155400 2023-10-06T04:52:24.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7feac4e6f700 1 -- 172.21.15.3:0/727400708 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00bd20 con 0x7feac0155400 2023-10-06T04:52:24.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50c4003020 con 0x7f50cc16e050 2023-10-06T04:52:24.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.499+0000 7f61f5dda700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61d4066f30 0x7f61d40693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61f00cc1b0 tx=0x7f61e8007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7f50c17fa700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7feac80d5700 1 -- 172.21.15.3:0/727400708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac0155400 msgr2=0x7feac01557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7feac80d5700 1 --2- 172.21.15.3:0/727400708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac0155400 0x7feac01557d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feabc0099b0 tx=0x7feabc009250 comp rx=0 tx=0).stop 2023-10-06T04:52:24.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.500+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f50c4084790 con 0x7f50cc16e050 2023-10-06T04:52:24.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f50d1004700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f50c17fa700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7f50b006f7e0 0x7f50b0071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 --> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50b00722a0 con 0x7f50b006f7e0 2023-10-06T04:52:24.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c4086dd0 con 0x7f50cc16e050 2023-10-06T04:52:24.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f50d0803700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7f50b006f7e0 0x7f50b0071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f50d1004700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f50b0066f30 0x7f50b00693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0091e0 tx=0x7f50bc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f50d0803700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7f50b006f7e0 0x7f50b0071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.501+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8538024430 con 0x7f854815c8f0 2023-10-06T04:52:24.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f853e7fc700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f852c066f30 0x7f852c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f44cbfff700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44b8067000 0x7f44b80694b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009a80 tx=0x7f44c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7ff20b7fe700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7ff1f806f930 0x7ff1f8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8538084a50 con 0x7f854815c8f0 2023-10-06T04:52:24.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f44d26eb700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f44ac0015f0 0x7f44ac003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 --> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44ac006b90 con 0x7f44ac0015f0 2023-10-06T04:52:24.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f853e7fc700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f852c06f7e0 0x7f852c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 --> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f852c0722a0 con 0x7f852c06f7e0 2023-10-06T04:52:24.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8538084e20 con 0x7f854815c8f0 2023-10-06T04:52:24.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7feac80d5700 1 -- 172.21.15.3:0/727400708 shutdown_connections 2023-10-06T04:52:24.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7feac80d5700 1 --2- 172.21.15.3:0/727400708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac0155400 0x7feac01557d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7feac80d5700 1 --2- 172.21.15.3:0/727400708 >> v2:172.21.15.3:3300/0 conn(0x7feac006a110 0x7feac00c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7feac80d5700 1 -- 172.21.15.3:0/727400708 >> 172.21.15.3:0/727400708 conn(0x7feac006c370 msgr2=0x7feac006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== osd.3 v2:172.21.15.3:6824/90685 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f50b00722a0 con 0x7f50b006f7e0 2023-10-06T04:52:24.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f44d0c88700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f44ac0015f0 0x7f44ac003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f854e346700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f852c066f30 0x7f852c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f854eb47700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f852c06f7e0 0x7f852c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7ff20b7fe700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7ff1f806f930 0x7ff1f8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7f44d0c88700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f44ac0015f0 0x7f44ac003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.502+0000 7feac80d5700 1 -- 172.21.15.3:0/727400708 shutdown_connections 2023-10-06T04:52:24.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.503+0000 7f854eb47700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f852c06f7e0 0x7f852c071bf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.503+0000 7feac80d5700 1 -- 172.21.15.3:0/727400708 wait complete. 2023-10-06T04:52:24.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.503+0000 7f44b3fff700 1 -- 172.21.15.3:0/3864435151 <== osd.5 v2:172.21.15.153:6800/90897 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f44ac006b90 con 0x7f44ac0015f0 2023-10-06T04:52:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.503+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== osd.2 v2:172.21.15.3:6816/90656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f852c0722a0 con 0x7f852c06f7e0 2023-10-06T04:52:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.503+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== osd.0 v2:172.21.15.3:6800/90654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff1f80723f0 con 0x7ff1f806f930 2023-10-06T04:52:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.504+0000 7feac80d5700 1 Processor -- start 2023-10-06T04:52:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.504+0000 7feac80d5700 1 -- start start 2023-10-06T04:52:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac80d5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac006a110 0x7feac00ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac80d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac006e5b0 0x7feac00cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac80d5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0155400 0x7feac00cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac80d5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac013de30 con 0x7feac006a110 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac80d5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00cc9e0 con 0x7feac0155400 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac80d5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac00cccc0 con 0x7feac006e5b0 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac006a110 0x7feac00ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac006a110 0x7feac00ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33018/0 (socket says 172.21.15.3:33018) 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 -- 172.21.15.3:0/62233638 learned_addr learned my addr 172.21.15.3:0/62233638 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5670700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac006e5b0 0x7feac00cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac6672700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0155400 0x7feac00cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 -- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac006e5b0 msgr2=0x7feac00cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feac006e5b0 0x7feac00cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 -- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0155400 msgr2=0x7feac00cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac0155400 0x7feac00cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.505+0000 7feac5e71700 1 -- 172.21.15.3:0/62233638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0094f0 con 0x7feac006a110 2023-10-06T04:52:24.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7feac5e71700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac006a110 0x7feac00ac2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feabc0094c0 tx=0x7feabc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7f61f65db700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f61d406f7e0 0x7f61d4071bf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7f854e346700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f852c066f30 0x7f852c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85440099e0 tx=0x7f854401b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc020070 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc011d70 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc01a790 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac013e080 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.506+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac0151740 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.507+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feac0071680 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.508+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feabc01abc0 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.508+0000 7feab5ffb700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feaa4066fd0 0x7feaa4069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.508+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feabc087780 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.508+0000 7feab5ffb700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7feaa406f800 0x7feaa4071c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.508+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 --> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feaa40722c0 con 0x7feaa406f800 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.508+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7feabc01ae70 con 0x7feac006a110 2023-10-06T04:52:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.509+0000 7feac6672700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7feaa406f800 0x7feaa4071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.509+0000 7feac6672700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7feaa406f800 0x7feaa4071c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.510+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== osd.6 v2:172.21.15.153:6808/90898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7feaa40722c0 con 0x7feaa406f800 2023-10-06T04:52:24.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.511+0000 7feac5670700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feaa4066fd0 0x7feaa4069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.511+0000 7feac5670700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feaa4066fd0 0x7feaa4069480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feac00bf510 tx=0x7feab0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.511+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== osd.7 v2:172.21.15.153:6816/90928 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f61d40722a0 con 0x7f61d406f7e0 2023-10-06T04:52:24.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.529+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 --> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44ac005c80 con 0x7f44ac0015f0 2023-10-06T04:52:24.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.529+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 --> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f53a4071680 con 0x7f538806f7e0 2023-10-06T04:52:24.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.529+0000 7f44b3fff700 1 -- 172.21.15.3:0/3864435151 <== osd.5 v2:172.21.15.153:6800/90897 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f44ac005c80 con 0x7f44ac0015f0 2023-10-06T04:52:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53997fa700 1 -- 172.21.15.3:0/1359923477 <== osd.4 v2:172.21.15.153:6817/90907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f53a4071680 con 0x7f538806f7e0 2023-10-06T04:52:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f44ac0015f0 msgr2=0x7f44ac003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f44d26eb700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f44ac0015f0 0x7f44ac003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f538806f7e0 msgr2=0x7f5388071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53ab87e700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f538806f7e0 0x7f5388071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44b8067000 msgr2=0x7f44b80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f44d26eb700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44b8067000 0x7f44b80694b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009a80 tx=0x7f44c0008040 comp rx=0 tx=0).stop 2023-10-06T04:52:24.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 msgr2=0x7f44cc0c3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f44d26eb700 1 --2- 172.21.15.3:0/3864435151 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44cc0b3e10 0x7f44cc0c3240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44cc06b6e0 tx=0x7f44bc016000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f44d0c88700 1 -- 172.21.15.3:0/3864435151 reap_dead start 2023-10-06T04:52:24.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5388066fb0 msgr2=0x7f5388069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53ab87e700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5388066fb0 0x7f5388069460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f53a0007ec0 tx=0x7f53a0015000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a4145b90 msgr2=0x7f53a4146000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53ab87e700 1 --2- 172.21.15.3:0/1359923477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a4145b90 0x7f53a4146000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f53a4061120 tx=0x7f5394006f10 comp rx=0 tx=0).stop 2023-10-06T04:52:24.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.530+0000 7f53a9e1b700 1 -- 172.21.15.3:0/1359923477 reap_dead start 2023-10-06T04:52:24.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.531+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 shutdown_connections 2023-10-06T04:52:24.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.531+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 >> 172.21.15.3:0/3864435151 conn(0x7f44cc0ce270 msgr2=0x7f44cc06df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.531+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 shutdown_connections 2023-10-06T04:52:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.531+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 >> 172.21.15.3:0/1359923477 conn(0x7f53a4066fb0 msgr2=0x7f53a40cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.531+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 shutdown_connections 2023-10-06T04:52:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.531+0000 7f44d26eb700 1 -- 172.21.15.3:0/3864435151 wait complete. 2023-10-06T04:52:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.532+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 shutdown_connections 2023-10-06T04:52:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.532+0000 7f53ab87e700 1 -- 172.21.15.3:0/1359923477 wait complete. 2023-10-06T04:52:24.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.532+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 --> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f50cc071680 con 0x7f50b006f7e0 2023-10-06T04:52:24.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.534+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 --> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff20c0c7990 con 0x7ff1f806f930 2023-10-06T04:52:24.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.534+0000 7f50c17fa700 1 -- 172.21.15.3:0/1251078724 <== osd.3 v2:172.21.15.3:6824/90685 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f50cc071680 con 0x7f50b006f7e0 2023-10-06T04:52:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7f50b006f7e0 msgr2=0x7f50b0071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7f50b006f7e0 0x7f50b0071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f50b0066f30 msgr2=0x7f50b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f50b0066f30 0x7f50b00693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0091e0 tx=0x7f50bc017000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50cc16e050 msgr2=0x7f50cc16e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 --2- 172.21.15.3:0/1251078724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50cc16e050 0x7f50cc16e4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50c4007ea0 tx=0x7f50c4009850 comp rx=0 tx=0).stop 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d1805700 1 -- 172.21.15.3:0/1251078724 reap_dead start 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 shutdown_connections 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 >> 172.21.15.3:0/1251078724 conn(0x7f50cc06c370 msgr2=0x7f50cc14ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 shutdown_connections 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.535+0000 7f50d3268700 1 -- 172.21.15.3:0/1251078724 wait complete. 2023-10-06T04:52:24.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.536+0000 7ff1f2ffd700 1 -- 172.21.15.3:0/996643509 <== osd.0 v2:172.21.15.3:6800/90654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff204009e90 con 0x7ff1f806f930 2023-10-06T04:52:24.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.537+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 --> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff154005c80 con 0x7ff1540015f0 2023-10-06T04:52:24.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.537+0000 7ff166ffd700 1 -- 172.21.15.3:0/305759058 <== osd.1 v2:172.21.15.3:6808/90655 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff154005c80 con 0x7ff1540015f0 2023-10-06T04:52:24.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.538+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7ff1540015f0 msgr2=0x7ff154003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.538+0000 7ff17d5c4700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7ff1540015f0 0x7ff154003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.538+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7ff1f806f930 msgr2=0x7ff1f8071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.538+0000 7ff21149e700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7ff1f806f930 0x7ff1f8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.538+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 --> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feac00abc80 con 0x7feaa406f800 2023-10-06T04:52:24.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.538+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1f8067080 msgr2=0x7ff1f8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7ff21149e700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1f8067080 0x7ff1f8069530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff200007130 tx=0x7ff200007eb0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff20c0699e0 msgr2=0x7ff20c0c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7ff21149e700 1 --2- 172.21.15.3:0/996643509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff20c0699e0 0x7ff20c0c5a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff20c162690 tx=0x7ff1fc016000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7ff20b7fe700 1 -- 172.21.15.3:0/996643509 reap_dead start 2023-10-06T04:52:24.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 shutdown_connections 2023-10-06T04:52:24.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 >> 172.21.15.3:0/996643509 conn(0x7ff20c06c370 msgr2=0x7ff20c0cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.541+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff15c066f00 msgr2=0x7ff15c0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.541+0000 7ff17d5c4700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff15c066f00 0x7ff15c0693b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1781610e0 tx=0x7ff168009040 comp rx=0 tx=0).stop 2023-10-06T04:52:24.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.541+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1780bf6d0 msgr2=0x7ff17815dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.541+0000 7ff17d5c4700 1 --2- 172.21.15.3:0/305759058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1780bf6d0 0x7ff17815dba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0030c0 tx=0x7ff16c016000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.541+0000 7ff1777fe700 1 -- 172.21.15.3:0/305759058 reap_dead start 2023-10-06T04:52:24.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 --> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85480cd510 con 0x7f852c06f7e0 2023-10-06T04:52:24.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7f853e7fc700 1 -- 172.21.15.3:0/2989807719 <== osd.2 v2:172.21.15.3:6816/90656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85480cd510 con 0x7f852c06f7e0 2023-10-06T04:52:24.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f852c06f7e0 msgr2=0x7f852c071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.539+0000 7f85505aa700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f852c06f7e0 0x7f852c071bf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.542+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 shutdown_connections 2023-10-06T04:52:24.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f852c066f30 msgr2=0x7f852c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f852c066f30 0x7f852c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85440099e0 tx=0x7f854401b000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f854815c8f0 msgr2=0x7f85481699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 --2- 172.21.15.3:0/2989807719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f854815c8f0 0x7f85481699c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f853800b5e0 tx=0x7f85380090f0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f854eb47700 1 -- 172.21.15.3:0/2989807719 reap_dead start 2023-10-06T04:52:24.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 shutdown_connections 2023-10-06T04:52:24.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.542+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 >> 172.21.15.3:0/305759058 conn(0x7ff178070310 msgr2=0x7ff17806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 >> 172.21.15.3:0/2989807719 conn(0x7f8548070310 msgr2=0x7f8548156560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 shutdown_connections 2023-10-06T04:52:24.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.540+0000 7f85505aa700 1 -- 172.21.15.3:0/2989807719 wait complete. 2023-10-06T04:52:24.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.542+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 shutdown_connections 2023-10-06T04:52:24.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.542+0000 7ff17d5c4700 1 -- 172.21.15.3:0/305759058 wait complete. 2023-10-06T04:52:24.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.543+0000 7feab5ffb700 1 -- 172.21.15.3:0/62233638 <== osd.6 v2:172.21.15.153:6808/90898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feac00abc80 con 0x7feaa406f800 2023-10-06T04:52:24.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.543+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7feaa406f800 msgr2=0x7feaa4071c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.543+0000 7feac80d5700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7feaa406f800 0x7feaa4071c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.543+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feaa4066fd0 msgr2=0x7feaa4069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.543+0000 7feac80d5700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feaa4066fd0 0x7feaa4069480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feac00bf510 tx=0x7feab0008040 comp rx=0 tx=0).stop 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.544+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac006a110 msgr2=0x7feac00ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.544+0000 7feac80d5700 1 --2- 172.21.15.3:0/62233638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac006a110 0x7feac00ac2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feabc0094c0 tx=0x7feabc016000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.544+0000 7feac6672700 1 -- 172.21.15.3:0/62233638 reap_dead start 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.544+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 shutdown_connections 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.544+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 >> 172.21.15.3:0/62233638 conn(0x7feac006c370 msgr2=0x7feac00ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.547+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 --> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61f00abc80 con 0x7f61d406f7e0 2023-10-06T04:52:24.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.547+0000 7f61e5ffb700 1 -- 172.21.15.3:0/962507702 <== osd.7 v2:172.21.15.153:6816/90928 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61f00abc80 con 0x7f61d406f7e0 2023-10-06T04:52:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.548+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f61d406f7e0 msgr2=0x7f61d4071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.548+0000 7f61f803e700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f61d406f7e0 0x7f61d4071bf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 shutdown_connections 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61d4066f30 msgr2=0x7f61d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61d4066f30 0x7f61d40693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61f00cc1b0 tx=0x7f61e8007ec0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61f006ecc0 msgr2=0x7f61f0176d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 --2- 172.21.15.3:0/962507702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61f006ecc0 0x7f61f0176d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f6b0 tx=0x7f61ec017000 comp rx=0 tx=0).stop 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f65db700 1 -- 172.21.15.3:0/962507702 reap_dead start 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 shutdown_connections 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 >> 172.21.15.3:0/962507702 conn(0x7f61f006c370 msgr2=0x7f61f00ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 shutdown_connections 2023-10-06T04:52:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.549+0000 7f61f803e700 1 -- 172.21.15.3:0/962507702 wait complete. 2023-10-06T04:52:24.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.551+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 shutdown_connections 2023-10-06T04:52:24.599 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:24.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T04:52:24.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.554+0000 7feac80d5700 1 -- 172.21.15.3:0/62233638 wait complete. 2023-10-06T04:52:24.601 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:24.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T04:52:24.602 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:24.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T04:52:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.556+0000 7ff21149e700 1 -- 172.21.15.3:0/996643509 wait complete. 2023-10-06T04:52:24.603 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:24.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T04:52:24.604 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:24.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T04:52:24.605 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:24.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T04:52:24.606 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:24.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T04:52:24.607 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:24.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T04:52:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.838+0000 7f6b612be700 1 Processor -- start 2023-10-06T04:52:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.838+0000 7f6b612be700 1 -- start start 2023-10-06T04:52:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b612be700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b5c155750 0x7f6b5c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b612be700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c15a520 con 0x7f6b5c06c620 2023-10-06T04:52:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b612be700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c06cb00 con 0x7f6b5c0b1f10 2023-10-06T04:52:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b612be700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5c06d710 con 0x7f6b5c155750 2023-10-06T04:52:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b5a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6b5c06c620 0x7f6b5c0b1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35888/0 (socket says 172.21.15.3:35888) 2023-10-06T04:52:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b5a7fc700 1 -- 172.21.15.3:0/3810140607 learned_addr learned my addr 172.21.15.3:0/3810140607 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849249395 0 0) 0x7f6b5c15a520 con 0x7f6b5c06c620 2023-10-06T04:52:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b48003570 con 0x7f6b5c06c620 2023-10-06T04:52:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.840+0000 7f6b5affd700 1 --2- 172.21.15.3:0/3810140607 >> v2:172.21.15.3:3300/0 conn(0x7f6b5c155750 0x7f6b5c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2845433565 0 0) 0x7f6b48003570 con 0x7f6b5c06c620 2023-10-06T04:52:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 >> v2:172.21.15.3:3300/0 conn(0x7f6b5c155750 msgr2=0x7f6b5c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 --2- 172.21.15.3:0/3810140607 >> v2:172.21.15.3:3300/0 conn(0x7f6b5c155750 0x7f6b5c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 >> v1:172.21.15.153:6789/0 conn(0x7f6b5c0b1f10 legacy=0x7f6b5c0c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c06cde0 con 0x7f6b5c06c620 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6b54001470 con 0x7f6b5c06c620 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 >> v1:172.21.15.3:6789/0 conn(0x7f6b5c06c620 legacy=0x7f6b5c0b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b48003ae0 0x7f6b48003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48004810 0x7f6b48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b480051c0 0x7f6b48014660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c06d710 con 0x7f6b48003ae0 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48009c80 con 0x7f6b480051c0 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48014ba0 con 0x7f6b48004810 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.841+0000 7f6b5affd700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b48003ae0 0x7f6b48003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b5a7fc700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48004810 0x7f6b48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b5affd700 1 -- 172.21.15.3:0/3810140607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48004810 msgr2=0x7f6b48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b5affd700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48004810 0x7f6b48004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b5affd700 1 -- 172.21.15.3:0/3810140607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b480051c0 msgr2=0x7f6b48014660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:52:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b5affd700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b480051c0 0x7f6b48014660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b5affd700 1 -- 172.21.15.3:0/3810140607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c06cde0 con 0x7f6b48003ae0 2023-10-06T04:52:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b5affd700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b48003ae0 0x7f6b48003ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b44006f00 tx=0x7f6b440080e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1541be7649780936 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44012070 con 0x7f6b48003ae0 2023-10-06T04:52:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4400a820 con 0x7f6b48003ae0 2023-10-06T04:52:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.842+0000 7f6b59ffb700 1 -- 172.21.15.3:0/3810140607 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400a9d0 con 0x7f6b48003ae0 2023-10-06T04:52:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.844+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/3810140607 reap_dead start 2023-10-06T04:52:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.844+0000 7f6b612be700 1 -- 172.21.15.3:0/3810140607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b48003ae0 msgr2=0x7f6b48003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.844+0000 7f6b612be700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b48003ae0 0x7f6b48003ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b44006f00 tx=0x7f6b440080e0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.846+0000 7f6b612be700 1 -- 172.21.15.3:0/3810140607 shutdown_connections 2023-10-06T04:52:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.846+0000 7f6b612be700 1 --2- 172.21.15.3:0/3810140607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b48003ae0 0x7f6b48003ed0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.846+0000 7f6b612be700 1 -- 172.21.15.3:0/3810140607 >> 172.21.15.3:0/3810140607 conn(0x7f6b5c066fb0 msgr2=0x7f6b5c0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.846+0000 7f6b612be700 1 -- 172.21.15.3:0/3810140607 shutdown_connections 2023-10-06T04:52:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.846+0000 7f6b612be700 1 -- 172.21.15.3:0/3810140607 wait complete. 2023-10-06T04:52:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.848+0000 7f6b612be700 1 Processor -- start 2023-10-06T04:52:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.848+0000 7f6b612be700 1 -- start start 2023-10-06T04:52:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b612be700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48003ae0 0x7f6b5c165950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b612be700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b5c1463d0 0x7f6b5c148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b612be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5c149dd0 0x7f6b5c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b612be700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c14c210 con 0x7f6b5c149dd0 2023-10-06T04:52:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b612be700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c14bf30 con 0x7f6b5c1463d0 2023-10-06T04:52:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b612be700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c14cd30 con 0x7f6b48003ae0 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b5affd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48003ae0 0x7f6b5c165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b5a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b5c1463d0 0x7f6b5c148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b5affd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48003ae0 0x7f6b5c165950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59474/0 (socket says 172.21.15.3:59474) 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.849+0000 7f6b5affd700 1 -- 172.21.15.3:0/455796095 learned_addr learned my addr 172.21.15.3:0/455796095 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b5b7fe700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5c149dd0 0x7f6b5c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b5affd700 1 -- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b5c1463d0 msgr2=0x7f6b5c148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b5affd700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b5c1463d0 0x7f6b5c148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b5affd700 1 -- 172.21.15.3:0/455796095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5c149dd0 msgr2=0x7f6b5c14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b5affd700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5c149dd0 0x7f6b5c14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b5affd700 1 -- 172.21.15.3:0/455796095 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c14d010 con 0x7f6b48003ae0 2023-10-06T04:52:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b5affd700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48003ae0 0x7f6b5c165950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400efd0 tx=0x7f6b44019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f6b52ffd700 1 -- 172.21.15.3:0/455796095 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400a990 con 0x7f6b48003ae0 2023-10-06T04:52:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.851+0000 7f6b52ffd700 1 -- 172.21.15.3:0/455796095 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b44006740 con 0x7f6b48003ae0 2023-10-06T04:52:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.851+0000 7f6b52ffd700 1 -- 172.21.15.3:0/455796095 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44017470 con 0x7f6b48003ae0 2023-10-06T04:52:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.851+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c14d2f0 con 0x7f6b48003ae0 2023-10-06T04:52:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.851+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c14a240 con 0x7f6b48003ae0 2023-10-06T04:52:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.850+0000 7f020a06d700 1 Processor -- start 2023-10-06T04:52:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.851+0000 7f020a06d700 1 -- start start 2023-10-06T04:52:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7fd0d12e5700 1 Processor -- start 2023-10-06T04:52:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7f07e76ce700 1 Processor -- start 2023-10-06T04:52:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7f07e76ce700 1 -- start start 2023-10-06T04:52:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7f020a06d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f02040c9040 0x7f0204155370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7f020a06d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02041558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7f020a06d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02040c7630 con 0x7f0204159d90 2023-10-06T04:52:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7f020a06d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0204155df0 con 0x7f02040c9040 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7f020a06d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020406acc0 con 0x7f020406f270 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f0203fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0204159d90 0x7f020406c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49542/0 (socket says 172.21.15.3:49542) 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f0203fff700 1 -- 172.21.15.3:0/3566738104 learned_addr learned my addr 172.21.15.3:0/3566738104 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f02037fe700 1 --2- 172.21.15.3:0/3566738104 >> v2:172.21.15.153:3300/0 conn(0x7f02040c9040 0x7f0204155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f0202ffd700 1 --2- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02041558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.852+0000 7fd0d12e5700 1 -- start start 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7fd0d12e5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc0c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7fd0d12e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0cc0c6910 0x7fd0cc0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7fd0d12e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0c8bb0 con 0x7fd0cc15da00 2023-10-06T04:52:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7fd0d12e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0cc06bce0 con 0x7fd0cc0c6910 2023-10-06T04:52:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7fd0d12e5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c7d40 con 0x7fd0cc069ea0 2023-10-06T04:52:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7efee5d51700 1 Processor -- start 2023-10-06T04:52:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7efee5d51700 1 -- start start 2023-10-06T04:52:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f07e76ce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07e015d870 0x7f07e00b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f07e76ce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07e015dc40 0x7f07e006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f07e76ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 0x7f07e00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f07e76ce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e00b47a0 con 0x7f07e015dc40 2023-10-06T04:52:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f07e76ce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e00b4d20 con 0x7f07e015d870 2023-10-06T04:52:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f07e76ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e006e2d0 con 0x7f07e006de90 2023-10-06T04:52:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.853+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c14a520 con 0x7f6b48003ae0 2023-10-06T04:52:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0ca7fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc0c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0ca7fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc0c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59494/0 (socket says 172.21.15.3:59494) 2023-10-06T04:52:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0ca7fc700 1 -- 172.21.15.3:0/288056978 learned_addr learned my addr 172.21.15.3:0/288056978 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f02037fe700 1 -- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 msgr2=0x7f02041558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f02037fe700 1 --2- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02041558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f02037fe700 1 -- 172.21.15.3:0/3566738104 >> v1:172.21.15.153:6789/0 conn(0x7f0204159d90 legacy=0x7f020406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f02037fe700 1 -- 172.21.15.3:0/3566738104 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040c6d40 con 0x7f02040c9040 2023-10-06T04:52:24.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f6b52ffd700 1 -- 172.21.15.3:0/455796095 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b440178a0 con 0x7f6b48003ae0 2023-10-06T04:52:24.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0cb7fe700 1 --2- 172.21.15.3:0/288056978 >> v2:172.21.15.3:3300/0 conn(0x7fd0cc0c6910 0x7fd0cc0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f07e546a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07e015d870 0x7f07e00b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f02037fe700 1 --2- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f0204155370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01f400aa90 tx=0x7f01f400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a9586f193501e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0ca7fc700 1 -- 172.21.15.3:0/288056978 >> v2:172.21.15.3:3300/0 conn(0x7fd0cc0c6910 msgr2=0x7fd0cc0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0ca7fc700 1 --2- 172.21.15.3:0/288056978 >> v2:172.21.15.3:3300/0 conn(0x7fd0cc0c6910 0x7fd0cc0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0ca7fc700 1 -- 172.21.15.3:0/288056978 >> v1:172.21.15.3:6789/0 conn(0x7fd0cc15da00 legacy=0x7fd0cc069790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:52:24.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7fd0ca7fc700 1 -- 172.21.15.3:0/288056978 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0c7fa0 con 0x7fd0cc069ea0 2023-10-06T04:52:24.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efee5d51700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efee5d51700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efee0069950 0x7efee0166040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efee5d51700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efee0166580 0x7efee006e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efee5d51700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00c65c0 con 0x7efee0166580 2023-10-06T04:52:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efee5d51700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00c6e60 con 0x7efee0069950 2023-10-06T04:52:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efee5d51700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00c9080 con 0x7efee00c8cb0 2023-10-06T04:52:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f6b52ffd700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b3c066fb0 0x7f6b3c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f07e546a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07e015d870 0x7f07e00b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:37932/0 (socket says 172.21.15.3:37932) 2023-10-06T04:52:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f07e546a700 1 -- 172.21.15.3:0/530795465 learned_addr learned my addr 172.21.15.3:0/530795465 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efedeffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efee0069950 0x7efee0166040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f02027fc700 1 -- 172.21.15.3:0/3566738104 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400c940 con 0x7f02040c9040 2023-10-06T04:52:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f02027fc700 1 -- 172.21.15.3:0/3566738104 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f400caa0 con 0x7f02040c9040 2023-10-06T04:52:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f02027fc700 1 -- 172.21.15.3:0/3566738104 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400ce20 con 0x7f02040c9040 2023-10-06T04:52:24.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f6b52ffd700 1 -- 172.21.15.3:0/455796095 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6b44086240 con 0x7f6b48003ae0 2023-10-06T04:52:24.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f6b5a7fc700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b3c066fb0 0x7f6b3c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7f07e4c69700 1 --2- 172.21.15.3:0/530795465 >> v2:172.21.15.3:3300/0 conn(0x7f07e015dc40 0x7f07e006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7fd0ca7fc700 1 --2- 172.21.15.3:0/288056978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc0c9040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009f20 tx=0x7fd0c4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f747edf1c9916dbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f24bec8a700 1 Processor -- start 2023-10-06T04:52:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efedf7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efedffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efee0166580 0x7efee006e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efedf7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59498/0 (socket says 172.21.15.3:59498) 2023-10-06T04:52:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.854+0000 7efedf7fe700 1 -- 172.21.15.3:0/3552256551 learned_addr learned my addr 172.21.15.3:0/3552256551 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7efedffff700 1 -- 172.21.15.3:0/3552256551 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 msgr2=0x7efee00c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7efedffff700 1 --2- 172.21.15.3:0/3552256551 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7efedffff700 1 -- 172.21.15.3:0/3552256551 >> v2:172.21.15.153:3300/0 conn(0x7efee0069950 msgr2=0x7efee0166040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7efedffff700 1 --2- 172.21.15.3:0/3552256551 >> v2:172.21.15.153:3300/0 conn(0x7efee0069950 0x7efee0166040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7efedffff700 1 -- 172.21.15.3:0/3552256551 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00c6810 con 0x7efee0166580 2023-10-06T04:52:24.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07e4c69700 1 -- 172.21.15.3:0/530795465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 msgr2=0x7f07e00b51b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:52:24.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07e4c69700 1 --2- 172.21.15.3:0/530795465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 0x7f07e00b51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07e4c69700 1 -- 172.21.15.3:0/530795465 >> v2:172.21.15.153:3300/0 conn(0x7f07e015d870 msgr2=0x7f07e00b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:24.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07e4c69700 1 --2- 172.21.15.3:0/530795465 >> v2:172.21.15.153:3300/0 conn(0x7f07e015d870 0x7f07e00b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07e4c69700 1 -- 172.21.15.3:0/530795465 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00b3f00 con 0x7f07e015dc40 2023-10-06T04:52:24.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07e4c69700 1 --2- 172.21.15.3:0/530795465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015dc40 0x7f07e006d950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07e006be70 tx=0x7f07d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fe1eadcd28a715 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f6b5a7fc700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b3c066fb0 0x7f6b3c069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c149350 tx=0x7f6b54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7efedffff700 1 --2- 172.21.15.3:0/3552256551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0166580 0x7efee006e5b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efee00c9b00 tx=0x7efed8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad4de1a2f322d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f24bec8a700 1 -- start start 2023-10-06T04:52:24.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07d7fff700 1 -- 172.21.15.3:0/530795465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800e070 con 0x7f07e015dc40 2023-10-06T04:52:24.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07d7fff700 1 -- 172.21.15.3:0/530795465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d800bc30 con 0x7f07e015dc40 2023-10-06T04:52:24.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.855+0000 7f07d7fff700 1 -- 172.21.15.3:0/530795465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8012430 con 0x7f07e015dc40 2023-10-06T04:52:24.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 -- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 msgr2=0x7f0204155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 --2- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f0204155370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01f400aa90 tx=0x7f01f400a330 comp rx=0 tx=0).stop 2023-10-06T04:52:24.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7efede7fc700 1 -- 172.21.15.3:0/3552256551 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800e070 con 0x7efee0166580 2023-10-06T04:52:24.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7efede7fc700 1 -- 172.21.15.3:0/3552256551 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800bbb0 con 0x7efee0166580 2023-10-06T04:52:24.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7efede7fc700 1 -- 172.21.15.3:0/3552256551 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8012430 con 0x7efee0166580 2023-10-06T04:52:24.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 -- 172.21.15.3:0/3566738104 shutdown_connections 2023-10-06T04:52:24.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 --2- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f0204155370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 --2- 172.21.15.3:0/3566738104 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02041558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 -- 172.21.15.3:0/3566738104 >> 172.21.15.3:0/3566738104 conn(0x7f0204070310 msgr2=0x7f0204070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 -- 172.21.15.3:0/3566738104 shutdown_connections 2023-10-06T04:52:24.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f020a06d700 1 -- 172.21.15.3:0/3566738104 wait complete. 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24bec8a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f24b80cbb60 0x7f24b80c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24bec8a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24bec8a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b80c8ba0 con 0x7f24b806a110 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24bec8a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80c8e00 con 0x7f24b80cbb60 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24bec8a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80c90e0 con 0x7f24b806ecc0 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24b7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f24b806a110 0x7f24b806e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35892/0 (socket says 172.21.15.3:35892) 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24b7fff700 1 -- 172.21.15.3:0/3018069618 learned_addr learned my addr 172.21.15.3:0/3018069618 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f24bd227700 1 --2- 172.21.15.3:0/3018069618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f07e76ce700 1 -- 172.21.15.3:0/530795465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015dc40 msgr2=0x7f07e006d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f07e76ce700 1 --2- 172.21.15.3:0/530795465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015dc40 0x7f07e006d950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07e006be70 tx=0x7f07d8009780 comp rx=0 tx=0).stop 2023-10-06T04:52:24.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f07e76ce700 1 -- 172.21.15.3:0/530795465 shutdown_connections 2023-10-06T04:52:24.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f07e76ce700 1 --2- 172.21.15.3:0/530795465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 0x7f07e00b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f07e76ce700 1 --2- 172.21.15.3:0/530795465 >> v2:172.21.15.153:3300/0 conn(0x7f07e015d870 0x7f07e00b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f07e76ce700 1 --2- 172.21.15.3:0/530795465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015dc40 0x7f07e006d950 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.856+0000 7f07e76ce700 1 -- 172.21.15.3:0/530795465 >> 172.21.15.3:0/530795465 conn(0x7f07e0070310 msgr2=0x7f07e0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 -- 172.21.15.3:0/3552256551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0166580 msgr2=0x7efee006e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 --2- 172.21.15.3:0/3552256551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0166580 0x7efee006e5b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efee00c9b00 tx=0x7efed8009780 comp rx=0 tx=0).stop 2023-10-06T04:52:24.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f24b77fe700 1 -- 172.21.15.3:0/3018069618 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444969443 0 0) 0x7f24b80c8ba0 con 0x7f24b806a110 2023-10-06T04:52:24.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f24b77fe700 1 -- 172.21.15.3:0/3018069618 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a0003570 con 0x7f24b806a110 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f24bd227700 1 -- 172.21.15.3:0/3018069618 >> v2:172.21.15.153:3300/0 conn(0x7f24b80cbb60 msgr2=0x7f24b80c8660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f24bd227700 1 --2- 172.21.15.3:0/3018069618 >> v2:172.21.15.153:3300/0 conn(0x7f24b80cbb60 0x7f24b80c8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f24bd227700 1 -- 172.21.15.3:0/3018069618 >> v1:172.21.15.3:6789/0 conn(0x7f24b806a110 legacy=0x7f24b806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f24bd227700 1 -- 172.21.15.3:0/3018069618 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80abc80 con 0x7f24b806ecc0 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 -- 172.21.15.3:0/3552256551 shutdown_connections 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 --2- 172.21.15.3:0/3552256551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0166580 0x7efee006e5b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 --2- 172.21.15.3:0/3552256551 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 --2- 172.21.15.3:0/3552256551 >> v2:172.21.15.153:3300/0 conn(0x7efee0069950 0x7efee0166040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f07e76ce700 1 -- 172.21.15.3:0/530795465 shutdown_connections 2023-10-06T04:52:24.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 -- 172.21.15.3:0/3552256551 >> 172.21.15.3:0/3552256551 conn(0x7efee006c370 msgr2=0x7efee006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 -- 172.21.15.3:0/3552256551 shutdown_connections 2023-10-06T04:52:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f07e76ce700 1 -- 172.21.15.3:0/530795465 wait complete. 2023-10-06T04:52:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7f24bd227700 1 --2- 172.21.15.3:0/3018069618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b81551b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009e60 tx=0x7f24b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3b350f3f47b866 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.857+0000 7efee5d51700 1 -- 172.21.15.3:0/3552256551 wait complete. 2023-10-06T04:52:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 Processor -- start 2023-10-06T04:52:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 -- start start 2023-10-06T04:52:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02040bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f02040bc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0204159d90 0x7f02041526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02040bc500 con 0x7f0204159d90 2023-10-06T04:52:24.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02041550c0 con 0x7f02040c9040 2023-10-06T04:52:24.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.858+0000 7f020a06d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02040bb8f0 con 0x7f020406f270 2023-10-06T04:52:24.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f07e76ce700 1 Processor -- start 2023-10-06T04:52:24.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0204159d90 0x7f02041526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f02037fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02040bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0204159d90 0x7f02041526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33098/0 (socket says 172.21.15.3:33098) 2023-10-06T04:52:24.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f02037fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02040bb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59514/0 (socket says 172.21.15.3:59514) 2023-10-06T04:52:24.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 -- 172.21.15.3:0/2080878752 learned_addr learned my addr 172.21.15.3:0/2080878752 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0202ffd700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f02040bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7efee5d51700 1 Processor -- start 2023-10-06T04:52:24.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7efee5d51700 1 -- start start 2023-10-06T04:52:24.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f07e76ce700 1 -- start start 2023-10-06T04:52:24.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 -- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 msgr2=0x7f02040bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02040bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 -- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 msgr2=0x7f02040bc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f02040bc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 -- 172.21.15.3:0/2080878752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f400a5d0 con 0x7f0204159d90 2023-10-06T04:52:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0202ffd700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f02040bc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f6b52ffd700 1 -- 172.21.15.3:0/455796095 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b5c14a520 con 0x7f6b48003ae0 2023-10-06T04:52:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f0203fff700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0204159d90 0x7f02041526f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01fc001c80 tx=0x7f01fc0061b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7f24b77fe700 1 -- 172.21.15.3:0/3018069618 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b000e070 con 0x7f24b806ecc0 2023-10-06T04:52:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f24b77fe700 1 -- 172.21.15.3:0/3018069618 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b000bbf0 con 0x7f24b806ecc0 2023-10-06T04:52:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f24b77fe700 1 -- 172.21.15.3:0/3018069618 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0012430 con 0x7f24b806ecc0 2023-10-06T04:52:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f07e76ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 0x7f07e01577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f07e76ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015d870 0x7f07e0066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f07e76ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07e00674f0 0x7f07e015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f07e76ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e006a700 con 0x7f07e015d870 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f07e76ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e0068680 con 0x7f07e00674f0 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f07e76ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e0067a70 con 0x7f07e006de90 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7fd0c9ffb700 1 -- 172.21.15.3:0/288056978 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400e070 con 0x7fd0cc069ea0 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7fd0c9ffb700 1 -- 172.21.15.3:0/288056978 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c400b990 con 0x7fd0cc069ea0 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.859+0000 7fd0c9ffb700 1 -- 172.21.15.3:0/288056978 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40092e0 con 0x7fd0cc069ea0 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7fd0d12e5700 1 -- 172.21.15.3:0/288056978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 msgr2=0x7fd0cc0c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/288056978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc0c9040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009f20 tx=0x7fd0c4009840 comp rx=0 tx=0).stop 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efee5d51700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0069950 0x7efee00c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efee5d51700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efee5d51700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0166580 0x7efee0169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efee5d51700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0165530 con 0x7efee0069950 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efee5d51700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00b1c20 con 0x7efee0166580 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efee5d51700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee0162230 con 0x7efee00c8cb0 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efedf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0069950 0x7efee00c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efedf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0069950 0x7efee00c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33108/0 (socket says 172.21.15.3:33108) 2023-10-06T04:52:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7efedf7fe700 1 -- 172.21.15.3:0/3587389939 learned_addr learned my addr 172.21.15.3:0/3587389939 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f24bec8a700 1 -- 172.21.15.3:0/3018069618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 msgr2=0x7f24b81551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f24bec8a700 1 --2- 172.21.15.3:0/3018069618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b81551b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009e60 tx=0x7f24b0009780 comp rx=0 tx=0).stop 2023-10-06T04:52:24.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015d870 0x7f07e0066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7fd0d12e5700 1 -- 172.21.15.3:0/288056978 shutdown_connections 2023-10-06T04:52:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/288056978 >> v2:172.21.15.3:3300/0 conn(0x7fd0cc0c6910 0x7fd0cc0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/288056978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc0c9040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7fd0d12e5700 1 -- 172.21.15.3:0/288056978 >> 172.21.15.3:0/288056978 conn(0x7fd0cc06c370 msgr2=0x7fd0cc06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7efedffff700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0166580 0x7efee0169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015d870 0x7f07e0066fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33110/0 (socket says 172.21.15.3:33110) 2023-10-06T04:52:24.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e5c6b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07e00674f0 0x7f07e015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 -- 172.21.15.3:0/3420906396 learned_addr learned my addr 172.21.15.3:0/3420906396 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7fd0d12e5700 1 -- 172.21.15.3:0/288056978 shutdown_connections 2023-10-06T04:52:24.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7fd0d12e5700 1 -- 172.21.15.3:0/288056978 wait complete. 2023-10-06T04:52:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 -- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 msgr2=0x7f07e01577a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:52:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 0x7f07e01577a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 -- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07e00674f0 msgr2=0x7f07e015c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07e00674f0 0x7f07e015c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 -- 172.21.15.3:0/3420906396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8009a20 con 0x7f07e015d870 2023-10-06T04:52:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f07e4c69700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015d870 0x7f07e0066fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07d80097c0 tx=0x7f07d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f24bec8a700 1 -- 172.21.15.3:0/3018069618 shutdown_connections 2023-10-06T04:52:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f24bec8a700 1 --2- 172.21.15.3:0/3018069618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b81551b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f24bec8a700 1 --2- 172.21.15.3:0/3018069618 >> v2:172.21.15.153:3300/0 conn(0x7f24b80cbb60 0x7f24b80c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f24bec8a700 1 -- 172.21.15.3:0/3018069618 >> 172.21.15.3:0/3018069618 conn(0x7f24b806c370 msgr2=0x7f24b806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f07d57fa700 1 -- 172.21.15.3:0/3420906396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8012c50 con 0x7f07e015d870 2023-10-06T04:52:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f07d57fa700 1 -- 172.21.15.3:0/3420906396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8021070 con 0x7f07e015d870 2023-10-06T04:52:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e0067d50 con 0x7f07e015d870 2023-10-06T04:52:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e0068310 con 0x7f07e015d870 2023-10-06T04:52:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7efedf7fe700 1 -- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 msgr2=0x7efee00afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7efedf7fe700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7efedf7fe700 1 -- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0166580 msgr2=0x7efee0169d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7efedf7fe700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0166580 0x7efee0169d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7efedf7fe700 1 -- 172.21.15.3:0/3587389939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8009a20 con 0x7efee0069950 2023-10-06T04:52:24.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7efedffff700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0166580 0x7efee0169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:24.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7efedf7fe700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0069950 0x7efee00c9d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efed400ea50 tx=0x7efed4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.863+0000 7fd0d12e5700 1 Processor -- start 2023-10-06T04:52:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.863+0000 7f07d57fa700 1 -- 172.21.15.3:0/3420906396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8026480 con 0x7f07e015d870 2023-10-06T04:52:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7efec77fe700 1 -- 172.21.15.3:0/3587389939 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400c850 con 0x7efee0069950 2023-10-06T04:52:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7efec77fe700 1 -- 172.21.15.3:0/3587389939 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400c9b0 con 0x7efee0069950 2023-10-06T04:52:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee0162480 con 0x7efee0069950 2023-10-06T04:52:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7efec77fe700 1 -- 172.21.15.3:0/3587389939 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400cd30 con 0x7efee0069950 2023-10-06T04:52:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee0162d20 con 0x7efee0069950 2023-10-06T04:52:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.863+0000 7fd0d12e5700 1 -- start start 2023-10-06T04:52:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0d12e5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc14a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0d12e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0c6910 0x7fd0cc0c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0d12e5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc15da00 0x7fd0cc0c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0d12e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c4a90 con 0x7fd0cc0c6910 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0d12e5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0acc50 con 0x7fd0cc15da00 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0d12e5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c41f0 con 0x7fd0cc069ea0 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0caffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc14a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0c6910 0x7fd0cc0c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0caffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc14a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59550/0 (socket says 172.21.15.3:59550) 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0caffd700 1 -- 172.21.15.3:0/4019594830 learned_addr learned my addr 172.21.15.3:0/4019594830 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7fd0cb7fe700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc15da00 0x7fd0cc0c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7fd0ca7fc700 1 -- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 msgr2=0x7fd0cc14a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7fd0ca7fc700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc14a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7fd0ca7fc700 1 -- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc15da00 msgr2=0x7fd0cc0c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7fd0ca7fc700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc15da00 0x7fd0cc0c1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7fd0ca7fc700 1 -- 172.21.15.3:0/4019594830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4009ae0 con 0x7fd0cc0c6910 2023-10-06T04:52:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7fd0cb7fe700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc15da00 0x7fd0cc0c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7f07d57fa700 1 -- 172.21.15.3:0/3420906396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07d8023030 con 0x7f07e015d870 2023-10-06T04:52:24.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7f07d57fa700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07c4066fb0 0x7f07c4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7f07d57fa700 1 -- 172.21.15.3:0/3420906396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f07d80865b0 con 0x7f07e015d870 2023-10-06T04:52:24.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7f07e546a700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07c4066fb0 0x7f07c4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7fd0ca7fc700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0c6910 0x7fd0cc0c1320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4014000 tx=0x7fd0c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee0071680 con 0x7efee0069950 2023-10-06T04:52:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.865+0000 7f07e546a700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07c4066fb0 0x7f07c4069460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07d0006d40 tx=0x7f07d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.866+0000 7f039f1f9700 1 Processor -- start 2023-10-06T04:52:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f01eb7fe700 1 -- 172.21.15.3:0/2080878752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0080d0 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f01eb7fe700 1 -- 172.21.15.3:0/2080878752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc008250 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.860+0000 7f01eb7fe700 1 -- 172.21.15.3:0/2080878752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00d8a0 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02040bbbd0 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.861+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02040bc190 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.862+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02040bc670 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.863+0000 7f01eb7fe700 1 -- 172.21.15.3:0/2080878752 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01fc00dc50 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.863+0000 7f01eb7fe700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f01f0066f30 0x7f01f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.863+0000 7f01eb7fe700 1 -- 172.21.15.3:0/2080878752 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f01fc083e10 con 0x7f0204159d90 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7f02037fe700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f01f0066f30 0x7f01f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.864+0000 7f02037fe700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f01f0066f30 0x7f01f00693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f400a370 tx=0x7f01f4019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.866+0000 7f039f1f9700 1 -- start start 2023-10-06T04:52:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.866+0000 7f24bec8a700 1 -- 172.21.15.3:0/3018069618 shutdown_connections 2023-10-06T04:52:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.866+0000 7f24bec8a700 1 -- 172.21.15.3:0/3018069618 wait complete. 2023-10-06T04:52:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.866+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c00052b0 con 0x7f07e015d870 2023-10-06T04:52:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039f1f9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039f1f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f039806c620 0x7f03980b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039f1f9700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039815a520 con 0x7f0398155b20 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039f1f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039806d710 con 0x7f039806c620 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039f1f9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039806cb00 con 0x7f0398155750 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039cf95700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039d796700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0398155b20 0x7f039806a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49556/0 (socket says 172.21.15.3:49556) 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039d796700 1 -- 172.21.15.3:0/2527191753 learned_addr learned my addr 172.21.15.3:0/2527191753 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.867+0000 7f039cf95700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59562/0 (socket says 172.21.15.3:59562) 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f039cf95700 1 -- 172.21.15.3:0/2527191753 >> v2:172.21.15.3:3300/0 conn(0x7f039806c620 msgr2=0x7f03980b1800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f03977fe700 1 -- 172.21.15.3:0/2527191753 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347560008 0 0) 0x7f039815a520 con 0x7f0398155b20 2023-10-06T04:52:24.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f039cf95700 1 --2- 172.21.15.3:0/2527191753 >> v2:172.21.15.3:3300/0 conn(0x7f039806c620 0x7f03980b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f039cf95700 1 -- 172.21.15.3:0/2527191753 >> v1:172.21.15.153:6789/0 conn(0x7f0398155b20 legacy=0x7f039806a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f039cf95700 1 -- 172.21.15.3:0/2527191753 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039806cde0 con 0x7f0398155750 2023-10-06T04:52:24.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f039cf95700 1 --2- 172.21.15.3:0/2527191753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398159fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f038400aa90 tx=0x7f038400a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd84b825b2a4b7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f24bec8a700 1 Processor -- start 2023-10-06T04:52:24.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f03977fe700 1 -- 172.21.15.3:0/2527191753 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038400ca40 con 0x7f0398155750 2023-10-06T04:52:24.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f03977fe700 1 -- 172.21.15.3:0/2527191753 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038400cba0 con 0x7f0398155750 2023-10-06T04:52:24.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f03977fe700 1 -- 172.21.15.3:0/2527191753 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384018400 con 0x7f0398155750 2023-10-06T04:52:24.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f24bec8a700 1 -- start start 2023-10-06T04:52:24.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f352ece1700 1 Processor -- start 2023-10-06T04:52:24.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.868+0000 7f352ece1700 1 -- start start 2023-10-06T04:52:24.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bec8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b806a110 0x7f24b81474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bec8a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f039f1f9700 1 -- 172.21.15.3:0/2527191753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 msgr2=0x7f0398159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f039f1f9700 1 --2- 172.21.15.3:0/2527191753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398159fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f038400aa90 tx=0x7f038400a330 comp rx=0 tx=0).stop 2023-10-06T04:52:24.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bec8a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24b80cbb60 0x7f24b80b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bec8a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80bf890 con 0x7f24b806a110 2023-10-06T04:52:24.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bec8a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80bfb70 con 0x7f24b80cbb60 2023-10-06T04:52:24.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bec8a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80bfe50 con 0x7f24b806ecc0 2023-10-06T04:52:24.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bca26700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b806a110 0x7f24b81474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f039f1f9700 1 -- 172.21.15.3:0/2527191753 shutdown_connections 2023-10-06T04:52:24.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f039f1f9700 1 --2- 172.21.15.3:0/2527191753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398159fe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bca26700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b806a110 0x7f24b81474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33132/0 (socket says 172.21.15.3:33132) 2023-10-06T04:52:24.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bca26700 1 -- 172.21.15.3:0/657675499 learned_addr learned my addr 172.21.15.3:0/657675499 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f039f1f9700 1 --2- 172.21.15.3:0/2527191753 >> v2:172.21.15.3:3300/0 conn(0x7f039806c620 0x7f03980b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f039f1f9700 1 -- 172.21.15.3:0/2527191753 >> 172.21.15.3:0/2527191753 conn(0x7f0398066fb0 msgr2=0x7f03980673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24bd227700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24b80cbb60 0x7f24b80b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f24b7fff700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f352ece1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35280b1f10 0x7f352815d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f352ece1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352815e0f0 con 0x7f3528159810 2023-10-06T04:52:24.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f352ece1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35280c93f0 con 0x7f3528159440 2023-10-06T04:52:24.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f352ece1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352815d850 con 0x7f35280b1f10 2023-10-06T04:52:24.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f352ca7d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3528159440 0x7f352806c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49562/0 (socket says 172.21.15.3:49562) 2023-10-06T04:52:24.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.869+0000 7f352ca7d700 1 -- 172.21.15.3:0/1688754258 learned_addr learned my addr 172.21.15.3:0/1688754258 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:24.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7fd0b2ffd700 1 -- 172.21.15.3:0/4019594830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40128b0 con 0x7fd0cc0c6910 2023-10-06T04:52:24.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f24bca26700 1 -- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 msgr2=0x7f24b80b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f24bca26700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b80b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f24bca26700 1 -- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24b80cbb60 msgr2=0x7f24b80b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f24bca26700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24b80cbb60 0x7f24b80b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f24bca26700 1 -- 172.21.15.3:0/657675499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0009a20 con 0x7f24b806a110 2023-10-06T04:52:24.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f24bd227700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24b80cbb60 0x7f24b80b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:24.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f01eb7fe700 1 -- 172.21.15.3:0/2080878752 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01fc050f20 con 0x7f0204159d90 2023-10-06T04:52:24.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc0c44d0 con 0x7fd0cc0c6910 2023-10-06T04:52:24.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc0ace60 con 0x7fd0cc0c6910 2023-10-06T04:52:24.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f24bca26700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b806a110 0x7f24b81474f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b5e0 tx=0x7f24a80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f352d27e700 1 --2- 172.21.15.3:0/1688754258 >> v2:172.21.15.3:3300/0 conn(0x7f35280b1f10 0x7f352815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f352d27e700 1 -- 172.21.15.3:0/1688754258 >> v1:172.21.15.153:6789/0 conn(0x7f3528159440 legacy=0x7f352806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f352d27e700 1 -- 172.21.15.3:0/1688754258 >> v1:172.21.15.3:6789/0 conn(0x7f3528159810 legacy=0x7f35280b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f352d27e700 1 -- 172.21.15.3:0/1688754258 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352815db30 con 0x7f35280b1f10 2023-10-06T04:52:24.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f039f1f9700 1 -- 172.21.15.3:0/2527191753 shutdown_connections 2023-10-06T04:52:24.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.870+0000 7f352d27e700 1 --2- 172.21.15.3:0/1688754258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3520009e60 tx=0x7f3520009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f0c9768eb44519 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.871+0000 7f039f1f9700 1 -- 172.21.15.3:0/2527191753 wait complete. 2023-10-06T04:52:24.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.871+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc0ad050 con 0x7fd0cc0c6910 2023-10-06T04:52:24.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.873+0000 7efec77fe700 1 -- 172.21.15.3:0/3587389939 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efed4013680 con 0x7efee0069950 2023-10-06T04:52:24.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.873+0000 7f039f1f9700 1 Processor -- start 2023-10-06T04:52:24.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.873+0000 7f039f1f9700 1 -- start start 2023-10-06T04:52:24.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.873+0000 7efec77fe700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efecc067000 0x7efecc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7efedeffd700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efecc067000 0x7efecc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7efedeffd700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efecc067000 0x7efecc0694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efee0087a10 tx=0x7efed0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7f039f1f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f039806c620 0x7f039814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7f039f1f9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398165290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7f039f1f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0398155b20 0x7f03980b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7f039f1f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0398166270 con 0x7f0398155b20 2023-10-06T04:52:24.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7f039f1f9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039814d060 con 0x7f039806c620 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.874+0000 7f039f1f9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0398166910 con 0x7f0398155750 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0398155b20 0x7f03980b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0398155b20 0x7f03980b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33142/0 (socket says 172.21.15.3:33142) 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 -- 172.21.15.3:0/516607046 learned_addr learned my addr 172.21.15.3:0/516607046 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f0397fff700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039cf95700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f039806c620 0x7f039814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 -- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 msgr2=0x7f0398165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 -- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f039806c620 msgr2=0x7f039814cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f039806c620 0x7f039814cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 -- 172.21.15.3:0/516607046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038400a5d0 con 0x7f0398155b20 2023-10-06T04:52:24.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039cf95700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f039806c620 0x7f039814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:24.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f0397fff700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:24.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.875+0000 7f039d796700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0398155b20 0x7f03980b0c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f039000ccc0 tx=0x7f0390001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.876+0000 7fd0b2ffd700 1 -- 172.21.15.3:0/4019594830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c4012c20 con 0x7fd0cc0c6910 2023-10-06T04:52:24.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.876+0000 7fd0b2ffd700 1 -- 172.21.15.3:0/4019594830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4012db0 con 0x7fd0cc0c6910 2023-10-06T04:52:24.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.876+0000 7f24b4ff9700 1 -- 172.21.15.3:0/657675499 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800f510 con 0x7f24b806a110 2023-10-06T04:52:24.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.876+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b80c0130 con 0x7f24b806a110 2023-10-06T04:52:24.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.876+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b80b0ea0 con 0x7f24b806a110 2023-10-06T04:52:24.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.876+0000 7f07d57fa700 1 -- 172.21.15.3:0/3420906396 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07d8053660 con 0x7f07e015d870 2023-10-06T04:52:24.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.876+0000 7f35277fe700 1 -- 172.21.15.3:0/1688754258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352000e070 con 0x7f35280b1f10 2023-10-06T04:52:24.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.877+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b8071680 con 0x7f24b806a110 2023-10-06T04:52:24.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f35277fe700 1 -- 172.21.15.3:0/1688754258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352000bc30 con 0x7f35280b1f10 2023-10-06T04:52:24.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f35277fe700 1 -- 172.21.15.3:0/1688754258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520012430 con 0x7f35280b1f10 2023-10-06T04:52:24.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f352ece1700 1 -- 172.21.15.3:0/1688754258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 msgr2=0x7f352815d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f352ece1700 1 --2- 172.21.15.3:0/1688754258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3520009e60 tx=0x7f3520009780 comp rx=0 tx=0).stop 2023-10-06T04:52:24.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7f24b4ff9700 1 -- 172.21.15.3:0/657675499 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8009210 con 0x7f24b806a110 2023-10-06T04:52:24.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7f24b4ff9700 1 -- 172.21.15.3:0/657675499 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800f880 con 0x7f24b806a110 2023-10-06T04:52:24.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7f352ece1700 1 -- 172.21.15.3:0/1688754258 shutdown_connections 2023-10-06T04:52:24.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7f352ece1700 1 --2- 172.21.15.3:0/1688754258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7f352ece1700 1 -- 172.21.15.3:0/1688754258 >> 172.21.15.3:0/1688754258 conn(0x7f3528066fb0 msgr2=0x7f35280673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:24.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7f352ece1700 1 -- 172.21.15.3:0/1688754258 shutdown_connections 2023-10-06T04:52:24.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7f352ece1700 1 -- 172.21.15.3:0/1688754258 wait complete. 2023-10-06T04:52:24.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.880+0000 7fd0b2ffd700 1 -- 172.21.15.3:0/4019594830 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0c4028460 con 0x7fd0cc0c6910 2023-10-06T04:52:24.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.881+0000 7fd0b2ffd700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0b8066f30 0x7fd0b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.881+0000 7fd0caffd700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0b8066f30 0x7fd0b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.882+0000 7fd0caffd700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0b8066f30 0x7fd0b80693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009b80 tx=0x7fd0bc000f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.882+0000 7f352ece1700 1 Processor -- start 2023-10-06T04:52:24.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.882+0000 7f352ece1700 1 -- start start 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ece1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ece1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3528159440 0x7f35281571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ece1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3528159810 0x7f352815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ece1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528156d10 con 0x7f35280b1f10 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ece1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528158400 con 0x7f3528159810 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ece1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352815c6e0 con 0x7f3528159440 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ca7d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.883+0000 7f352ca7d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33146/0 (socket says 172.21.15.3:33146) 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3527fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3528159440 0x7f35281571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f352ca7d700 1 -- 172.21.15.3:0/3154049931 learned_addr learned my addr 172.21.15.3:0/3154049931 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f352d27e700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3528159810 0x7f352815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3527fff700 1 -- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3528159810 msgr2=0x7f352815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3527fff700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3528159810 0x7f352815ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3527fff700 1 -- 172.21.15.3:0/3154049931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 msgr2=0x7f352815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3527fff700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3527fff700 1 -- 172.21.15.3:0/3154049931 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3520009a20 con 0x7f3528159440 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f352ca7d700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3527fff700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3528159440 0x7f35281571a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f351400c490 tx=0x7f3514007ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3524ff9700 1 -- 172.21.15.3:0/3154049931 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400ce40 con 0x7f3528159440 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3524ff9700 1 -- 172.21.15.3:0/3154049931 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3514011590 con 0x7f3528159440 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f3524ff9700 1 -- 172.21.15.3:0/3154049931 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514011950 con 0x7f3528159440 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.884+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352815b890 con 0x7f3528159440 2023-10-06T04:52:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.885+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35281563e0 con 0x7f3528159440 2023-10-06T04:52:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.886+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3528156e80 con 0x7f3528159440 2023-10-06T04:52:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.886+0000 7fd0b2ffd700 1 -- 172.21.15.3:0/4019594830 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd0c4057c70 con 0x7fd0cc0c6910 2023-10-06T04:52:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.886+0000 7f3524ff9700 1 -- 172.21.15.3:0/3154049931 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3514011cc0 con 0x7f3528159440 2023-10-06T04:52:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.886+0000 7f3524ff9700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f350c066fb0 0x7f350c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.887+0000 7f3524ff9700 1 -- 172.21.15.3:0/3154049931 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f35140839a0 con 0x7f3528159440 2023-10-06T04:52:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.887+0000 7f352ca7d700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f350c066fb0 0x7f350c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.887+0000 7f352ca7d700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f350c066fb0 0x7f350c069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f0394ff9700 1 -- 172.21.15.3:0/516607046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000a8a0 con 0x7f0398155b20 2023-10-06T04:52:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f0394ff9700 1 -- 172.21.15.3:0/516607046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000aa00 con 0x7f0398155b20 2023-10-06T04:52:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f0394ff9700 1 -- 172.21.15.3:0/516607046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000adc0 con 0x7f0398155b20 2023-10-06T04:52:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0398165cb0 con 0x7f0398155b20 2023-10-06T04:52:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.879+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03981665a0 con 0x7f0398155b20 2023-10-06T04:52:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.886+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03981663e0 con 0x7f0398155b20 2023-10-06T04:52:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.888+0000 7efec77fe700 1 -- 172.21.15.3:0/3587389939 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efed4056630 con 0x7efee0069950 2023-10-06T04:52:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.888+0000 7efec77fe700 1 -- 172.21.15.3:0/3587389939 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efed4052270 con 0x7efee0069950 2023-10-06T04:52:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.888+0000 7f24b4ff9700 1 -- 172.21.15.3:0/657675499 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24a800f9e0 con 0x7f24b806a110 2023-10-06T04:52:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.888+0000 7f24b4ff9700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24a4066f30 0x7f24a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.889+0000 7f24b7fff700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24a4066f30 0x7f24a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.889+0000 7f0394ff9700 1 -- 172.21.15.3:0/516607046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0390007300 con 0x7f0398155b20 2023-10-06T04:52:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.889+0000 7f24b7fff700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24a4066f30 0x7f24a40693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f24b80bf710 tx=0x7f24ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.890+0000 7f0394ff9700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f037c067000 0x7f037c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:24.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.890+0000 7f039cf95700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f037c067000 0x7f037c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:24.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.891+0000 7f039cf95700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f037c067000 0x7f037c0694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f038400a5a0 tx=0x7f038401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:24.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.891+0000 7f3524ff9700 1 -- 172.21.15.3:0/3154049931 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3514050a10 con 0x7f3528159440 2023-10-06T04:52:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.896+0000 7fd0b2ffd700 1 -- 172.21.15.3:0/4019594830 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0c40538b0 con 0x7fd0cc0c6910 2023-10-06T04:52:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.902+0000 7f0394ff9700 1 -- 172.21.15.3:0/516607046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0390056a20 con 0x7f0398155b20 2023-10-06T04:52:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.902+0000 7f24b4ff9700 1 -- 172.21.15.3:0/657675499 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f24a8019360 con 0x7f24b806a110 2023-10-06T04:52:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.906+0000 7f24b4ff9700 1 -- 172.21.15.3:0/657675499 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24a8055300 con 0x7f24b806a110 2023-10-06T04:52:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:24.913+0000 7f0394ff9700 1 -- 172.21.15.3:0/516607046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0390052660 con 0x7f0398155b20 2023-10-06T04:52:25.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.255+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6b5c14a520 con 0x7f6b48003ae0 2023-10-06T04:52:25.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.256+0000 7f6b52ffd700 1 -- 172.21.15.3:0/455796095 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b5c14a520 con 0x7f6b48003ae0 2023-10-06T04:52:25.263 INFO:teuthology.orchestra.run.smithi003.stdout:55834574852 2023-10-06T04:52:25.267 INFO:teuthology.orchestra.run.smithi003.stdout:55834574852 2023-10-06T04:52:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.261+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0204071680 con 0x7f0204159d90 2023-10-06T04:52:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.262+0000 7f01eb7fe700 1 -- 172.21.15.3:0/2080878752 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01fc018090 con 0x7f0204159d90 2023-10-06T04:52:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b3c066fb0 msgr2=0x7f6b3c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b3c066fb0 0x7f6b3c069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c149350 tx=0x7f6b54009040 comp rx=0 tx=0).stop 2023-10-06T04:52:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48003ae0 msgr2=0x7f6b5c165950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48003ae0 0x7f6b5c165950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400efd0 tx=0x7f6b44019000 comp rx=0 tx=0).stop 2023-10-06T04:52:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 shutdown_connections 2023-10-06T04:52:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b3c066fb0 0x7f6b3c069460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5c149dd0 0x7f6b5c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b48003ae0 0x7f6b5c165950 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 --2- 172.21.15.3:0/455796095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b5c1463d0 0x7f6b5c148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 >> 172.21.15.3:0/455796095 conn(0x7f6b5c066fb0 msgr2=0x7f6b5c1535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f01f0066f30 msgr2=0x7f01f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f01f0066f30 0x7f01f00693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f400a370 tx=0x7f01f4019000 comp rx=0 tx=0).stop 2023-10-06T04:52:25.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0204159d90 msgr2=0x7f02041526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0204159d90 0x7f02041526f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01fc001c80 tx=0x7f01fc0061b0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 shutdown_connections 2023-10-06T04:52:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f01f0066f30 0x7f01f00693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0204159d90 0x7f02041526f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f020406f270 0x7f02040bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 --2- 172.21.15.3:0/2080878752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02040c9040 0x7f02040bc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.271+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 >> 172.21.15.3:0/2080878752 conn(0x7f0204070310 msgr2=0x7f02040afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.272+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 shutdown_connections 2023-10-06T04:52:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.272+0000 7f020a06d700 1 -- 172.21.15.3:0/2080878752 wait complete. 2023-10-06T04:52:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.272+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7efee00c6af0 con 0x7efee0069950 2023-10-06T04:52:25.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.272+0000 7efec77fe700 1 -- 172.21.15.3:0/3587389939 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efed4017090 con 0x7efee0069950 2023-10-06T04:52:25.277 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efecc067000 msgr2=0x7efecc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efecc067000 0x7efecc0694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efee0087a10 tx=0x7efed0008040 comp rx=0 tx=0).stop 2023-10-06T04:52:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0069950 msgr2=0x7efee00c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0069950 0x7efee00c9d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efed400ea50 tx=0x7efed4009250 comp rx=0 tx=0).stop 2023-10-06T04:52:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 shutdown_connections 2023-10-06T04:52:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efecc067000 0x7efecc0694b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0166580 0x7efee0169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee0069950 0x7efee00c9d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 --2- 172.21.15.3:0/3587389939 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00c8cb0 0x7efee00afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 >> 172.21.15.3:0/3587389939 conn(0x7efee006c370 msgr2=0x7efee00cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 shutdown_connections 2023-10-06T04:52:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.276+0000 7efee5d51700 1 -- 172.21.15.3:0/3587389939 wait complete. 2023-10-06T04:52:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.278+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3528071680 con 0x7f3528159440 2023-10-06T04:52:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.278+0000 7f3524ff9700 1 -- 172.21.15.3:0/3154049931 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3514017020 con 0x7f3528159440 2023-10-06T04:52:25.281 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 shutdown_connections 2023-10-06T04:52:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f6b612be700 1 -- 172.21.15.3:0/455796095 wait complete. 2023-10-06T04:52:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f350c066fb0 msgr2=0x7f350c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f350c066fb0 0x7f350c069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c009040 comp rx=0 tx=0).stop 2023-10-06T04:52:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3528159440 msgr2=0x7f35281571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3528159440 0x7f35281571a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f351400c490 tx=0x7f3514007ea0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 shutdown_connections 2023-10-06T04:52:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f350c066fb0 0x7f350c069460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3528159810 0x7f352815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35280b1f10 0x7f352815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 --2- 172.21.15.3:0/3154049931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3528159440 0x7f35281571a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 >> 172.21.15.3:0/3154049931 conn(0x7f3528066fb0 msgr2=0x7f3528152860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 shutdown_connections 2023-10-06T04:52:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.281+0000 7f352ece1700 1 -- 172.21.15.3:0/3154049931 wait complete. 2023-10-06T04:52:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.285+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd0cc071680 con 0x7fd0cc0c6910 2023-10-06T04:52:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.286+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f07c0005560 con 0x7f07e015d870 2023-10-06T04:52:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.286+0000 7fd0b2ffd700 1 -- 172.21.15.3:0/4019594830 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd0c4056ed0 con 0x7fd0cc0c6910 2023-10-06T04:52:25.288 INFO:teuthology.orchestra.run.smithi003.stdout:55834574852 2023-10-06T04:52:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.286+0000 7f07d57fa700 1 -- 172.21.15.3:0/3420906396 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f07d8056c80 con 0x7f07e015d870 2023-10-06T04:52:25.288 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0b8066f30 msgr2=0x7fd0b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0b8066f30 0x7fd0b80693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009b80 tx=0x7fd0bc000f90 comp rx=0 tx=0).stop 2023-10-06T04:52:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0c6910 msgr2=0x7fd0cc0c1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0c6910 0x7fd0cc0c1320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4014000 tx=0x7fd0c4017000 comp rx=0 tx=0).stop 2023-10-06T04:52:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 shutdown_connections 2023-10-06T04:52:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0b8066f30 0x7fd0b80693e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc15da00 0x7fd0cc0c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0cc069ea0 0x7fd0cc14a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 --2- 172.21.15.3:0/4019594830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0cc0c6910 0x7fd0cc0c1320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 >> 172.21.15.3:0/4019594830 conn(0x7fd0cc06c370 msgr2=0x7fd0cc0b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.289+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 shutdown_connections 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.290+0000 7fd0d12e5700 1 -- 172.21.15.3:0/4019594830 wait complete. 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.290+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07c4066fb0 msgr2=0x7f07c4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.290+0000 7f07e76ce700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07c4066fb0 0x7f07c4069460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07d0006d40 tx=0x7f07d0008040 comp rx=0 tx=0).stop 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.290+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015d870 msgr2=0x7f07e0066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.290+0000 7f07e76ce700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015d870 0x7f07e0066fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07d80097c0 tx=0x7f07d8019000 comp rx=0 tx=0).stop 2023-10-06T04:52:25.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.293+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 shutdown_connections 2023-10-06T04:52:25.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.293+0000 7f07e76ce700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07c4066fb0 0x7f07c4069460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.293+0000 7f07e76ce700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07e00674f0 0x7f07e015c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.293+0000 7f07e76ce700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07e006de90 0x7f07e01577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.293+0000 7f07e76ce700 1 --2- 172.21.15.3:0/3420906396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e015d870 0x7f07e0066fb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.293+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 >> 172.21.15.3:0/3420906396 conn(0x7f07e0070310 msgr2=0x7f07e0160bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.294+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 shutdown_connections 2023-10-06T04:52:25.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.294+0000 7f07e76ce700 1 -- 172.21.15.3:0/3420906396 wait complete. 2023-10-06T04:52:25.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-06T04:52:25.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-10-06T04:52:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.306+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f24b80c93c0 con 0x7f24b806a110 2023-10-06T04:52:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.307+0000 7f24b4ff9700 1 -- 172.21.15.3:0/657675499 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24a8055120 con 0x7f24b806a110 2023-10-06T04:52:25.308 INFO:teuthology.orchestra.run.smithi003.stdout:55834574852 2023-10-06T04:52:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.310+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24a4066f30 msgr2=0x7f24a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.310+0000 7f24bec8a700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24a4066f30 0x7f24a40693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f24b80bf710 tx=0x7f24ac009040 comp rx=0 tx=0).stop 2023-10-06T04:52:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.310+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b806a110 msgr2=0x7f24b81474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.310+0000 7f24bec8a700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b806a110 0x7f24b81474f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b5e0 tx=0x7f24a80090f0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0398071680 con 0x7f0398155b20 2023-10-06T04:52:25.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-06T04:52:25.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2023-10-06T04:52:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 shutdown_connections 2023-10-06T04:52:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f24bec8a700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24a4066f30 0x7f24a40693e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f24bec8a700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24b80cbb60 0x7f24b80b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f24bec8a700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b806a110 0x7f24b81474f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f24bec8a700 1 --2- 172.21.15.3:0/657675499 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24b806ecc0 0x7f24b80b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 >> 172.21.15.3:0/657675499 conn(0x7f24b806c370 msgr2=0x7f24b80ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.311+0000 7f0394ff9700 1 -- 172.21.15.3:0/516607046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0390055c80 con 0x7f0398155b20 2023-10-06T04:52:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.312+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 shutdown_connections 2023-10-06T04:52:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.312+0000 7f24bec8a700 1 -- 172.21.15.3:0/657675499 wait complete. 2023-10-06T04:52:25.316 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:25.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-06T04:52:25.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.314+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f037c067000 msgr2=0x7f037c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.314+0000 7f039f1f9700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f037c067000 0x7f037c0694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f038400a5a0 tx=0x7f038401a000 comp rx=0 tx=0).stop 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.314+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0398155b20 msgr2=0x7f03980b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.314+0000 7f039f1f9700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0398155b20 0x7f03980b0c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f039000ccc0 tx=0x7f0390001a80 comp rx=0 tx=0).stop 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 shutdown_connections 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f037c067000 0x7f037c0694b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f039806c620 0x7f039814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0398155b20 0x7f03980b0c10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 --2- 172.21.15.3:0/516607046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0398155750 0x7f0398165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 >> 172.21.15.3:0/516607046 conn(0x7f0398066fb0 msgr2=0x7f03981514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 shutdown_connections 2023-10-06T04:52:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:25.315+0000 7f039f1f9700 1 -- 172.21.15.3:0/516607046 wait complete. 2023-10-06T04:52:25.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-10-06T04:52:25.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-10-06T04:52:25.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-10-06T04:52:26.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T04:52:26.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T04:52:26.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T04:52:26.315 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T04:52:26.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T04:52:26.323 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T04:52:26.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T04:52:26.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T04:52:26.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.502+0000 7f3951d32700 1 Processor -- start 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.502+0000 7f3951d32700 1 -- start start 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.503+0000 7f3951d32700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f394c155400 0x7f394c1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f3951d32700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06a110 0x7f394c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f3951d32700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c0c8ba0 con 0x7f394c06e5b0 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f3951d32700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c0ab9e0 con 0x7f394c155400 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f3951d32700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0c8e00 con 0x7f394c06a110 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f394bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f394c06e5b0 0x7f394c06e9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49566/0 (socket says 172.21.15.3:49566) 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f394bfff700 1 -- 172.21.15.3:0/886844929 learned_addr learned my addr 172.21.15.3:0/886844929 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f394affd700 1 --2- 172.21.15.3:0/886844929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06a110 0x7f394c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f394a7fc700 1 -- 172.21.15.3:0/886844929 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30811942 0 0) 0x7f394c0c8ba0 con 0x7f394c06e5b0 2023-10-06T04:52:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.504+0000 7f394a7fc700 1 -- 172.21.15.3:0/886844929 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3934003570 con 0x7f394c06e5b0 2023-10-06T04:52:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394affd700 1 -- 172.21.15.3:0/886844929 >> v2:172.21.15.3:3300/0 conn(0x7f394c155400 msgr2=0x7f394c1557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394affd700 1 --2- 172.21.15.3:0/886844929 >> v2:172.21.15.3:3300/0 conn(0x7f394c155400 0x7f394c1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394affd700 1 -- 172.21.15.3:0/886844929 >> v1:172.21.15.153:6789/0 conn(0x7f394c06e5b0 legacy=0x7f394c06e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394affd700 1 -- 172.21.15.3:0/886844929 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c0c90e0 con 0x7f394c06a110 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394affd700 1 --2- 172.21.15.3:0/886844929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06a110 0x7f394c0c8660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f393c0098e0 tx=0x7f393c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b310c93d59dabea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394a7fc700 1 -- 172.21.15.3:0/886844929 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00f750 con 0x7f394c06a110 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394a7fc700 1 -- 172.21.15.3:0/886844929 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c00f8b0 con 0x7f394c06a110 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.505+0000 7f394a7fc700 1 -- 172.21.15.3:0/886844929 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00fc30 con 0x7f394c06a110 2023-10-06T04:52:26.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.506+0000 7f29d6d8c700 1 Processor -- start 2023-10-06T04:52:26.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.506+0000 7f3951d32700 1 -- 172.21.15.3:0/886844929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06a110 msgr2=0x7f394c0c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.506+0000 7f3951d32700 1 --2- 172.21.15.3:0/886844929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06a110 0x7f394c0c8660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f393c0098e0 tx=0x7f393c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.506+0000 7f29d6d8c700 1 -- start start 2023-10-06T04:52:26.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29d6d8c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d0155400 0x7f29d01557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29d6d8c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f29d006ecc0 0x7f29d00c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29d6d8c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d00c8ba0 con 0x7f29d006a110 2023-10-06T04:52:26.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29d6d8c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d00ab9e0 con 0x7f29d006ecc0 2023-10-06T04:52:26.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29d6d8c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00c8e00 con 0x7f29d0155400 2023-10-06T04:52:26.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29cffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f29d006a110 0x7f29d006e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35920/0 (socket says 172.21.15.3:35920) 2023-10-06T04:52:26.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29cffff700 1 -- 172.21.15.3:0/3149755110 learned_addr learned my addr 172.21.15.3:0/3149755110 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:26.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29d5329700 1 --2- 172.21.15.3:0/3149755110 >> v2:172.21.15.153:3300/0 conn(0x7f29d006ecc0 0x7f29d00c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.508+0000 7f29d4b28700 1 --2- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d0155400 0x7f29d01557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f3951d32700 1 -- 172.21.15.3:0/886844929 shutdown_connections 2023-10-06T04:52:26.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f3951d32700 1 --2- 172.21.15.3:0/886844929 >> v2:172.21.15.3:3300/0 conn(0x7f394c155400 0x7f394c1557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f3951d32700 1 --2- 172.21.15.3:0/886844929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06a110 0x7f394c0c8660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f3951d32700 1 -- 172.21.15.3:0/886844929 >> 172.21.15.3:0/886844929 conn(0x7f394c06c370 msgr2=0x7f394c06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f29d5329700 1 -- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d0155400 msgr2=0x7f29d01557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f29d5329700 1 --2- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d0155400 0x7f29d01557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f29d5329700 1 -- 172.21.15.3:0/3149755110 >> v1:172.21.15.3:6789/0 conn(0x7f29d006a110 legacy=0x7f29d006e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f29d5329700 1 -- 172.21.15.3:0/3149755110 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d00c90e0 con 0x7f29d006ecc0 2023-10-06T04:52:26.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f3951d32700 1 -- 172.21.15.3:0/886844929 shutdown_connections 2023-10-06T04:52:26.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.509+0000 7f3951d32700 1 -- 172.21.15.3:0/886844929 wait complete. 2023-10-06T04:52:26.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.510+0000 7f29d5329700 1 --2- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006ecc0 0x7f29d00c8660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29c40098e0 tx=0x7f29c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54e85058ec46226b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.510+0000 7f29cf7fe700 1 -- 172.21.15.3:0/3149755110 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400b640 con 0x7f29d006ecc0 2023-10-06T04:52:26.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.510+0000 7f29cf7fe700 1 -- 172.21.15.3:0/3149755110 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c400b7a0 con 0x7f29d006ecc0 2023-10-06T04:52:26.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.510+0000 7f29cf7fe700 1 -- 172.21.15.3:0/3149755110 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400bb40 con 0x7f29d006ecc0 2023-10-06T04:52:26.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.511+0000 7f3951d32700 1 Processor -- start 2023-10-06T04:52:26.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.511+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006ecc0 msgr2=0x7f29d00c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.511+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006ecc0 0x7f29d00c8660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29c40098e0 tx=0x7f29c40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3149755110 shutdown_connections 2023-10-06T04:52:26.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006ecc0 0x7f29d00c8660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3149755110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d0155400 0x7f29d01557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3149755110 >> 172.21.15.3:0/3149755110 conn(0x7f29d006c370 msgr2=0x7f29d006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.511+0000 7f3951d32700 1 -- start start 2023-10-06T04:52:26.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3149755110 shutdown_connections 2023-10-06T04:52:26.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3149755110 wait complete. 2023-10-06T04:52:26.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.514+0000 7f29d6d8c700 1 Processor -- start 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f3951d32700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394c06a110 0x7f394c0c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f3951d32700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06e5b0 0x7f394c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f3951d32700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c155400 0x7f394c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f3951d32700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0c4a40 con 0x7f394c06a110 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f3951d32700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0c39a0 con 0x7f394c155400 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.512+0000 7f3951d32700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0c3c80 con 0x7f394c06e5b0 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394c06a110 0x7f394c0c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394c06a110 0x7f394c0c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33152/0 (socket says 172.21.15.3:33152) 2023-10-06T04:52:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 -- 172.21.15.3:0/1373783811 learned_addr learned my addr 172.21.15.3:0/1373783811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 -- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06e5b0 msgr2=0x7f394c1531e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06e5b0 0x7f394c1531e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 -- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c155400 msgr2=0x7f394c0c9b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c155400 0x7f394c0c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.513+0000 7f394b7fe700 1 -- 172.21.15.3:0/1373783811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c009420 con 0x7f394c06a110 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.514+0000 7f394b7fe700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394c06a110 0x7f394c0c3030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f394000b810 tx=0x7f39400091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.514+0000 7f29d6d8c700 1 -- start start 2023-10-06T04:52:26.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f29d6d8c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006a110 0x7f29d014a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f29d6d8c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d006ecc0 0x7f29d0149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f29d6d8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d0155400 0x7f29d00ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f29d6d8c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0149e90 con 0x7f29d0155400 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f29d6d8c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d014b310 con 0x7f29d006a110 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f29d6d8c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0162280 con 0x7f29d006ecc0 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f393b7fe700 1 -- 172.21.15.3:0/1373783811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000f570 con 0x7f394c06a110 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f393b7fe700 1 -- 172.21.15.3:0/1373783811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000f6d0 con 0x7f394c06a110 2023-10-06T04:52:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f393b7fe700 1 -- 172.21.15.3:0/1373783811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000fa90 con 0x7f394c06a110 2023-10-06T04:52:26.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.516+0000 7f29d5329700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d0155400 0x7f29d00ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.516+0000 7f29d5329700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d0155400 0x7f29d00ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33162/0 (socket says 172.21.15.3:33162) 2023-10-06T04:52:26.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.516+0000 7f29d5329700 1 -- 172.21.15.3:0/3039427634 learned_addr learned my addr 172.21.15.3:0/3039427634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.515+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c0c3f60 con 0x7f394c06a110 2023-10-06T04:52:26.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.516+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c0c4520 con 0x7f394c06a110 2023-10-06T04:52:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.516+0000 7f29d5329700 1 -- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d006ecc0 msgr2=0x7f29d0149950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:52:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.516+0000 7f29d5329700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d006ecc0 0x7f29d0149950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29d5329700 1 -- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006a110 msgr2=0x7f29d014a090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:52:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29d5329700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006a110 0x7f29d014a090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29d5329700 1 -- 172.21.15.3:0/3039427634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4009420 con 0x7f29d0155400 2023-10-06T04:52:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29d5329700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d0155400 0x7f29d00ca1c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29c40093f0 tx=0x7f29c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29ccff9700 1 -- 172.21.15.3:0/3039427634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4011420 con 0x7f29d0155400 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29ccff9700 1 -- 172.21.15.3:0/3039427634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c40115c0 con 0x7f29d0155400 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29ccff9700 1 -- 172.21.15.3:0/3039427634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4011980 con 0x7f29d0155400 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.517+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d014a700 con 0x7f29d0155400 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.518+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d014acc0 con 0x7f29d0155400 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.518+0000 7f393b7fe700 1 -- 172.21.15.3:0/1373783811 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f394001a460 con 0x7f394c06a110 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.519+0000 7f393b7fe700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3930067050 0x7f3930069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.519+0000 7f393b7fe700 1 -- 172.21.15.3:0/1373783811 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3940084310 con 0x7f394c06a110 2023-10-06T04:52:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.520+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b00052b0 con 0x7f29d0155400 2023-10-06T04:52:26.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.520+0000 7f394affd700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3930067050 0x7f3930069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.521+0000 7f29ccff9700 1 -- 172.21.15.3:0/3039427634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29c4011d30 con 0x7f29d0155400 2023-10-06T04:52:26.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.522+0000 7f29ccff9700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29bc067000 0x7f29bc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.522+0000 7f29ccff9700 1 -- 172.21.15.3:0/3039427634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f29c4086000 con 0x7f29d0155400 2023-10-06T04:52:26.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.522+0000 7f29d4b28700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29bc067000 0x7f29bc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.522+0000 7f394affd700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3930067050 0x7f3930069500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f394c0b1b20 tx=0x7f393c019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.523+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c0c4740 con 0x7f394c06a110 2023-10-06T04:52:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.523+0000 7f29d4b28700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29bc067000 0x7f29bc0694b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009b10 tx=0x7f29c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.530+0000 7f29ccff9700 1 -- 172.21.15.3:0/3039427634 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29c4053110 con 0x7f29d0155400 2023-10-06T04:52:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.538+0000 7f393b7fe700 1 -- 172.21.15.3:0/1373783811 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3940054f10 con 0x7f394c06a110 2023-10-06T04:52:26.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.551+0000 7f7948788700 1 Processor -- start 2023-10-06T04:52:26.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.552+0000 7f7948788700 1 -- start start 2023-10-06T04:52:26.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.553+0000 7f7948788700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7940157e10 0x7f794015af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.553+0000 7f7948788700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79401581e0 0x7f7940158d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.553+0000 7f7948788700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794015b790 con 0x7f7940159260 2023-10-06T04:52:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.553+0000 7f7948788700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794015ba70 con 0x7f7940157e10 2023-10-06T04:52:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.553+0000 7f7948788700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794015bd50 con 0x7f79401581e0 2023-10-06T04:52:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7946d25700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7940159260 0x7f79400c8a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49572/0 (socket says 172.21.15.3:49572) 2023-10-06T04:52:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7946d25700 1 -- 172.21.15.3:0/1164685896 learned_addr learned my addr 172.21.15.3:0/1164685896 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945d23700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79401581e0 0x7f7940158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272522567 0 0) 0x7f794015b790 con 0x7f7940159260 2023-10-06T04:52:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f792c003570 con 0x7f7940159260 2023-10-06T04:52:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7946524700 1 --2- 172.21.15.3:0/1164685896 >> v2:172.21.15.3:3300/0 conn(0x7f7940157e10 0x7f794015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1011525847 0 0) 0x7f792c003570 con 0x7f7940159260 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79401581e0 msgr2=0x7f7940158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79401581e0 0x7f7940158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 >> v2:172.21.15.3:3300/0 conn(0x7f7940157e10 msgr2=0x7f794015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 --2- 172.21.15.3:0/1164685896 >> v2:172.21.15.3:3300/0 conn(0x7f7940157e10 0x7f794015af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400adb50 con 0x7f7940159260 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f79380017f0 con 0x7f7940159260 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 >> v1:172.21.15.153:6789/0 conn(0x7f7940159260 legacy=0x7f79400c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c003ae0 0x7f792c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c004810 0x7f792c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.554+0000 7f7945522700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f792c0051c0 0x7f792c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794015ba70 con 0x7f792c0051c0 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794015bd50 con 0x7f792c003ae0 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c00bfd0 con 0x7f792c004810 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7946524700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c003ae0 0x7f792c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7946d25700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f792c0051c0 0x7f792c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7946d25700 1 -- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c004810 msgr2=0x7f792c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7946d25700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c004810 0x7f792c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7946d25700 1 -- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c003ae0 msgr2=0x7f792c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7946d25700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c003ae0 0x7f792c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.555+0000 7f7946d25700 1 -- 172.21.15.3:0/1164685896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400adb50 con 0x7f792c0051c0 2023-10-06T04:52:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.556+0000 7f7946d25700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f792c0051c0 0x7f792c00ba60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f794006d150 tx=0x7f79380021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513a7fc56a4c5982 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.556+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938011690 con 0x7f792c0051c0 2023-10-06T04:52:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.556+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380117f0 con 0x7f792c0051c0 2023-10-06T04:52:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.556+0000 7f7945522700 1 -- 172.21.15.3:0/1164685896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938011bb0 con 0x7f792c0051c0 2023-10-06T04:52:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.557+0000 7f7948788700 1 -- 172.21.15.3:0/1164685896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f792c0051c0 msgr2=0x7f792c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.557+0000 7f7948788700 1 --2- 172.21.15.3:0/1164685896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f792c0051c0 0x7f792c00ba60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f794006d150 tx=0x7f79380021e0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.557+0000 7f7946d25700 1 -- 172.21.15.3:0/1164685896 reap_dead start 2023-10-06T04:52:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.557+0000 7f7948788700 1 -- 172.21.15.3:0/1164685896 shutdown_connections 2023-10-06T04:52:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.557+0000 7f7948788700 1 -- 172.21.15.3:0/1164685896 >> 172.21.15.3:0/1164685896 conn(0x7f794006c370 msgr2=0x7f794006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.562+0000 7f7014d99700 1 Processor -- start 2023-10-06T04:52:26.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.562+0000 7f7014d99700 1 -- start start 2023-10-06T04:52:26.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.563+0000 7ff1f6356700 1 Processor -- start 2023-10-06T04:52:26.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.563+0000 7ff1f6356700 1 -- start start 2023-10-06T04:52:26.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f7014d99700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f70100b1f10 0x7f701015d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f7014d99700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70100c93f0 con 0x7f701015c930 2023-10-06T04:52:26.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f7014d99700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701015e0d0 con 0x7f701015cd00 2023-10-06T04:52:26.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f7014d99700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70100caf50 con 0x7f70100b1f10 2023-10-06T04:52:26.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f700ed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f70100b1f10 0x7f701015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f700dd9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f701015cd00 0x7f70100b1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49580/0 (socket says 172.21.15.3:49580) 2023-10-06T04:52:26.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f700dd9b700 1 -- 172.21.15.3:0/1076759242 learned_addr learned my addr 172.21.15.3:0/1076759242 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:26.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f7fedbdb700 1 Processor -- start 2023-10-06T04:52:26.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f7fedbdb700 1 -- start start 2023-10-06T04:52:26.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f700ed9d700 1 -- 172.21.15.3:0/1076759242 >> v1:172.21.15.153:6789/0 conn(0x7f701015cd00 legacy=0x7f70100b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f700ed9d700 1 -- 172.21.15.3:0/1076759242 >> v1:172.21.15.3:6789/0 conn(0x7f701015c930 legacy=0x7f701006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f700ed9d700 1 -- 172.21.15.3:0/1076759242 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701015d830 con 0x7f70100b1f10 2023-10-06T04:52:26.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.564+0000 7f7948788700 1 -- 172.21.15.3:0/1164685896 shutdown_connections 2023-10-06T04:52:26.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7f700ed9d700 1 --2- 172.21.15.3:0/1076759242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70100b1f10 0x7f701015d150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7008009e60 tx=0x7f7008009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36fbcf7a3ff6649c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7f700d59a700 1 -- 172.21.15.3:0/1076759242 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700800e070 con 0x7f70100b1f10 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1f6356700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f015cd00 0x7ff1f015d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1f6356700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1f006aad0 0x7ff1f00b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1f6356700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f00c93f0 con 0x7ff1f015c930 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1f6356700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f00caf50 con 0x7ff1f006aad0 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1f6356700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f015e0d0 con 0x7ff1f015cd00 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1effff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff1f015c930 0x7ff1f006c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49594/0 (socket says 172.21.15.3:49594) 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1effff700 1 -- 172.21.15.3:0/3974649995 learned_addr learned my addr 172.21.15.3:0/3974649995 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1ef7fe700 1 --2- 172.21.15.3:0/3974649995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f015cd00 0x7ff1f015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7ff1f48f3700 1 --2- 172.21.15.3:0/3974649995 >> v2:172.21.15.3:3300/0 conn(0x7ff1f006aad0 0x7ff1f00b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1ef7fe700 1 -- 172.21.15.3:0/3974649995 >> v2:172.21.15.3:3300/0 conn(0x7ff1f006aad0 msgr2=0x7ff1f00b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1ef7fe700 1 --2- 172.21.15.3:0/3974649995 >> v2:172.21.15.3:3300/0 conn(0x7ff1f006aad0 0x7ff1f00b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1ef7fe700 1 -- 172.21.15.3:0/3974649995 >> v1:172.21.15.153:6789/0 conn(0x7ff1f015c930 legacy=0x7ff1f006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1ef7fe700 1 -- 172.21.15.3:0/3974649995 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f015d830 con 0x7ff1f015cd00 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1f48f3700 1 --2- 172.21.15.3:0/3974649995 >> v2:172.21.15.3:3300/0 conn(0x7ff1f006aad0 0x7ff1f00b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fedbdb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe806b7d0 0x7f7fe80af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fedbdb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe80b1c90 con 0x7f7fe8156410 2023-10-06T04:52:26.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fedbdb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe80b1080 con 0x7f7fe80b0900 2023-10-06T04:52:26.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fedbdb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80b1360 con 0x7f7fe806b7d0 2023-10-06T04:52:26.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7fe8156410 0x7f7fe806b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35934/0 (socket says 172.21.15.3:35934) 2023-10-06T04:52:26.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe77fe700 1 -- 172.21.15.3:0/704026292 learned_addr learned my addr 172.21.15.3:0/704026292 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:26.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1ef7fe700 1 --2- 172.21.15.3:0/3974649995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f015cd00 0x7ff1f015d150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40098e0 tx=0x7ff1e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615b960602401bd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7f700d59a700 1 -- 172.21.15.3:0/1076759242 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700800bbf0 con 0x7f70100b1f10 2023-10-06T04:52:26.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.565+0000 7f700d59a700 1 -- 172.21.15.3:0/1076759242 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008012430 con 0x7f70100b1f10 2023-10-06T04:52:26.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7014d99700 1 -- 172.21.15.3:0/1076759242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70100b1f10 msgr2=0x7f701015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7014d99700 1 --2- 172.21.15.3:0/1076759242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70100b1f10 0x7f701015d150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7008009e60 tx=0x7f7008009780 comp rx=0 tx=0).stop 2023-10-06T04:52:26.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1eeffd700 1 -- 172.21.15.3:0/3974649995 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4009e80 con 0x7ff1f015cd00 2023-10-06T04:52:26.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe6ffd700 1 --2- 172.21.15.3:0/704026292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe806b7d0 0x7f7fe80af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1eeffd700 1 -- 172.21.15.3:0/3974649995 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400b760 con 0x7ff1f015cd00 2023-10-06T04:52:26.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7ff1eeffd700 1 -- 172.21.15.3:0/3974649995 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400bae0 con 0x7ff1f015cd00 2023-10-06T04:52:26.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7014d99700 1 -- 172.21.15.3:0/1076759242 shutdown_connections 2023-10-06T04:52:26.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7014d99700 1 --2- 172.21.15.3:0/1076759242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70100b1f10 0x7f701015d150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7014d99700 1 -- 172.21.15.3:0/1076759242 >> 172.21.15.3:0/1076759242 conn(0x7f7010066fb0 msgr2=0x7f70100673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe67fc700 1 -- 172.21.15.3:0/704026292 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244815134 0 0) 0x7f7fe80b1c90 con 0x7f7fe8156410 2023-10-06T04:52:26.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe67fc700 1 -- 172.21.15.3:0/704026292 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd0003570 con 0x7f7fe8156410 2023-10-06T04:52:26.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7014d99700 1 -- 172.21.15.3:0/1076759242 shutdown_connections 2023-10-06T04:52:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe6ffd700 1 -- 172.21.15.3:0/704026292 >> v1:172.21.15.153:6789/0 conn(0x7f7fe80b0900 legacy=0x7f7fe80c9ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:52:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe6ffd700 1 -- 172.21.15.3:0/704026292 >> v1:172.21.15.3:6789/0 conn(0x7f7fe8156410 legacy=0x7f7fe806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7fe6ffd700 1 -- 172.21.15.3:0/704026292 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe80b1640 con 0x7f7fe806b7d0 2023-10-06T04:52:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7f7fe6ffd700 1 --2- 172.21.15.3:0/704026292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe806b7d0 0x7f7fe80af480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009910 tx=0x7f7fd80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5500586cbf5ab44c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7f7fe67fc700 1 -- 172.21.15.3:0/704026292 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800b640 con 0x7f7fe806b7d0 2023-10-06T04:52:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7f7fe67fc700 1 -- 172.21.15.3:0/704026292 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd800b7a0 con 0x7f7fe806b7d0 2023-10-06T04:52:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7f7fe67fc700 1 -- 172.21.15.3:0/704026292 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800bb60 con 0x7f7fe806b7d0 2023-10-06T04:52:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7ff1f6356700 1 -- 172.21.15.3:0/3974649995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f015cd00 msgr2=0x7ff1f015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7ff1f6356700 1 --2- 172.21.15.3:0/3974649995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f015cd00 0x7ff1f015d150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40098e0 tx=0x7ff1e40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7ff1f6356700 1 -- 172.21.15.3:0/3974649995 shutdown_connections 2023-10-06T04:52:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7ff1f6356700 1 --2- 172.21.15.3:0/3974649995 >> v2:172.21.15.3:3300/0 conn(0x7ff1f006aad0 0x7ff1f00b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7ff1f6356700 1 --2- 172.21.15.3:0/3974649995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f015cd00 0x7ff1f015d150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.567+0000 7ff1f6356700 1 -- 172.21.15.3:0/3974649995 >> 172.21.15.3:0/3974649995 conn(0x7ff1f0066fb0 msgr2=0x7ff1f00673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7f3420fc4700 1 Processor -- start 2023-10-06T04:52:26.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7f3420fc4700 1 -- start start 2023-10-06T04:52:26.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7f7fedbdb700 1 -- 172.21.15.3:0/704026292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe806b7d0 msgr2=0x7f7fe80af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/704026292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe806b7d0 0x7f7fe80af480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009910 tx=0x7f7fd80091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.566+0000 7f7014d99700 1 -- 172.21.15.3:0/1076759242 wait complete. 2023-10-06T04:52:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7f7014d99700 1 Processor -- start 2023-10-06T04:52:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7ff1f6356700 1 -- 172.21.15.3:0/3974649995 shutdown_connections 2023-10-06T04:52:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7ff1f6356700 1 -- 172.21.15.3:0/3974649995 wait complete. 2023-10-06T04:52:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7ff1f6356700 1 Processor -- start 2023-10-06T04:52:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.568+0000 7f7014d99700 1 -- start start 2023-10-06T04:52:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7ff1f6356700 1 -- start start 2023-10-06T04:52:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7f3420fc4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f341c155750 0x7f341c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7f3420fc4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c0b1f10 0x7f341c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7f3420fc4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c15a520 con 0x7f341c06c620 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7f3420fc4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c06d710 con 0x7f341c155750 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7f3420fc4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c06cb00 con 0x7f341c0b1f10 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f341c155750 0x7f341c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f3419d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f341c06c620 0x7f341c0b1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35940/0 (socket says 172.21.15.3:35940) 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f3419d9b700 1 -- 172.21.15.3:0/900031152 learned_addr learned my addr 172.21.15.3:0/900031152 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f7fedbdb700 1 -- 172.21.15.3:0/704026292 shutdown_connections 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/704026292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe806b7d0 0x7f7fe80af480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f7fedbdb700 1 -- 172.21.15.3:0/704026292 >> 172.21.15.3:0/704026292 conn(0x7f7fe80cdc30 msgr2=0x7f7fe806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341a59c700 1 -- 172.21.15.3:0/900031152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c0b1f10 msgr2=0x7f341c0c95a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:52:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341a59c700 1 --2- 172.21.15.3:0/900031152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c0b1f10 0x7f341c0c95a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341a59c700 1 -- 172.21.15.3:0/900031152 >> v1:172.21.15.3:6789/0 conn(0x7f341c06c620 legacy=0x7f341c0b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341959a700 1 -- 172.21.15.3:0/900031152 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709603256 0 0) 0x7f341c15a520 con 0x7f341c06c620 2023-10-06T04:52:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341a59c700 1 -- 172.21.15.3:0/900031152 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c06cde0 con 0x7f341c155750 2023-10-06T04:52:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341a59c700 1 --2- 172.21.15.3:0/900031152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 0x7f341c159fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f340c00bb90 tx=0x7f340c00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6deb423d291472d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.569+0000 7f7948788700 1 -- 172.21.15.3:0/1164685896 wait complete. 2023-10-06T04:52:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f7948788700 1 Processor -- start 2023-10-06T04:52:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7ff1f6356700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f006aad0 0x7ff1f0159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7ff1f6356700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f015c930 0x7ff1f015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7ff1f6356700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f0157fb0 0x7ff1f016a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7ff1f6356700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f01590c0 con 0x7ff1f006aad0 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7ff1f6356700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f015be50 con 0x7ff1f015c930 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7ff1f6356700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00a5760 con 0x7ff1f0157fb0 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f006aad0 0x7ff1f0159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f006aad0 0x7ff1f0159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33202/0 (socket says 172.21.15.3:33202) 2023-10-06T04:52:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1effff700 1 -- 172.21.15.3:0/3903365581 learned_addr learned my addr 172.21.15.3:0/3903365581 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f7014d99700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70100b1f10 0x7f701016ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f7014d99700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701015c930 0x7f701014c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f7014d99700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7010171510 0x7f7010171990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f7014d99700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701015ba20 con 0x7f70100b1f10 2023-10-06T04:52:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f7014d99700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7010172080 con 0x7f701015c930 2023-10-06T04:52:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f7fedbdb700 1 -- 172.21.15.3:0/704026292 shutdown_connections 2023-10-06T04:52:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f7014d99700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7010172360 con 0x7f7010171510 2023-10-06T04:52:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1ef7fe700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f015c930 0x7ff1f015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1f48f3700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f0157fb0 0x7ff1f016a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f700e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70100b1f10 0x7f701016ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f700dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701015c930 0x7f701014c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f700e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70100b1f10 0x7f701016ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33208/0 (socket says 172.21.15.3:33208) 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f700e59c700 1 -- 172.21.15.3:0/2528537138 learned_addr learned my addr 172.21.15.3:0/2528537138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f700dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701015c930 0x7f701014c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:38072/0 (socket says 172.21.15.3:38072) 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7f700ed9d700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7010171510 0x7f7010171990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1ef7fe700 1 -- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f0157fb0 msgr2=0x7ff1f016a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1ef7fe700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f0157fb0 0x7ff1f016a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1ef7fe700 1 -- 172.21.15.3:0/3903365581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f006aad0 msgr2=0x7ff1f0159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1ef7fe700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f006aad0 0x7ff1f0159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1ef7fe700 1 -- 172.21.15.3:0/3903365581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e4009420 con 0x7ff1f015c930 2023-10-06T04:52:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.571+0000 7ff1f48f3700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f0157fb0 0x7ff1f016a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7ff1ef7fe700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f015c930 0x7ff1f015b910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4015000 tx=0x7ff1e4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f700ed9d700 1 -- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701015c930 msgr2=0x7f701014c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f700ed9d700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701015c930 0x7f701014c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f700ed9d700 1 -- 172.21.15.3:0/2528537138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70100b1f10 msgr2=0x7f701016ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f700ed9d700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70100b1f10 0x7f701016ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f700ed9d700 1 -- 172.21.15.3:0/2528537138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7008009a20 con 0x7f7010171510 2023-10-06T04:52:26.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f700e59c700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70100b1f10 0x7f701016ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:26.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7ff1d7fff700 1 -- 172.21.15.3:0/3903365581 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4006e50 con 0x7ff1f015c930 2023-10-06T04:52:26.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f700ed9d700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7010171510 0x7f7010171990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7008015000 tx=0x7f7008018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7ff1d7fff700 1 -- 172.21.15.3:0/3903365581 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400b8d0 con 0x7ff1f015c930 2023-10-06T04:52:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7ff1f6356700 1 -- 172.21.15.3:0/3903365581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f016acd0 con 0x7ff1f015c930 2023-10-06T04:52:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f7948788700 1 -- start start 2023-10-06T04:52:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7948788700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c0051c0 0x7f794014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7948788700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794014a620 0x7f794014b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7948788700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794014d8a0 0x7f794014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7948788700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794014e280 con 0x7f794014a620 2023-10-06T04:52:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7948788700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794014aa90 con 0x7f794014d8a0 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7948788700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794014c110 con 0x7f792c0051c0 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c0051c0 0x7f794014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946d25700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794014d8a0 0x7f794014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c0051c0 0x7f794014f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59698/0 (socket says 172.21.15.3:59698) 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 -- 172.21.15.3:0/1431373710 learned_addr learned my addr 172.21.15.3:0/1431373710 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341959a700 1 -- 172.21.15.3:0/900031152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c00dba0 con 0x7f341c155750 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341959a700 1 -- 172.21.15.3:0/900031152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c00dd00 con 0x7f341c155750 2023-10-06T04:52:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.570+0000 7f341959a700 1 -- 172.21.15.3:0/900031152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0196b0 con 0x7f341c155750 2023-10-06T04:52:26.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7fedbdb700 1 -- 172.21.15.3:0/704026292 wait complete. 2023-10-06T04:52:26.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 -- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794014d8a0 msgr2=0x7f794014dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794014d8a0 0x7f794014dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 -- 172.21.15.3:0/1431373710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794014a620 msgr2=0x7f794014b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794014a620 0x7f794014b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7f7946524700 1 -- 172.21.15.3:0/1431373710 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794014c3f0 con 0x7f792c0051c0 2023-10-06T04:52:26.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.573+0000 7f7946524700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c0051c0 0x7f794014f230 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f793c011b10 tx=0x7f793c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7ff1f6356700 1 -- 172.21.15.3:0/3903365581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f0158790 con 0x7ff1f015c930 2023-10-06T04:52:26.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.572+0000 7ff1d7fff700 1 -- 172.21.15.3:0/3903365581 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e401c460 con 0x7ff1f015c930 2023-10-06T04:52:26.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.573+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c80052b0 con 0x7ff1f015c930 2023-10-06T04:52:26.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.574+0000 7ff1d7fff700 1 -- 172.21.15.3:0/3903365581 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1e401c870 con 0x7ff1f015c930 2023-10-06T04:52:26.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.574+0000 7f7fedbdb700 1 Processor -- start 2023-10-06T04:52:26.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.574+0000 7f7fedbdb700 1 -- start start 2023-10-06T04:52:26.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.574+0000 7ff1d7fff700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1dc066f30 0x7ff1dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.574+0000 7ff1d7fff700 1 -- 172.21.15.3:0/3903365581 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1e40867d0 con 0x7ff1f015c930 2023-10-06T04:52:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.574+0000 7ff1effff700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1dc066f30 0x7ff1dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.575+0000 7ff1effff700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1dc066f30 0x7ff1dc0693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001c90 tx=0x7ff1e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.575+0000 7f7fedbdb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fe806b7d0 0x7f7fe8170800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.575+0000 7f7fedbdb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe80b0900 0x7f7fe81579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.575+0000 7f7fedbdb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe8156410 0x7f7fe8157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.575+0000 7f7fedbdb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8159170 con 0x7f7fe8156410 2023-10-06T04:52:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.575+0000 7f7fedbdb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe816dd00 con 0x7f7fe806b7d0 2023-10-06T04:52:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.575+0000 7f7fedbdb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8170d40 con 0x7f7fe80b0900 2023-10-06T04:52:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fe806b7d0 0x7f7fe8170800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe8156410 0x7f7fe8157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fe806b7d0 0x7f7fe8170800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:38098/0 (socket says 172.21.15.3:38098) 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 -- 172.21.15.3:0/1431947707 learned_addr learned my addr 172.21.15.3:0/1431947707 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 -- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe80b0900 msgr2=0x7f7fe81579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe80b0900 0x7f7fe81579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 -- 172.21.15.3:0/1431947707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe8156410 msgr2=0x7f7fe8157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe8156410 0x7f7fe8157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 -- 172.21.15.3:0/1431947707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd8009420 con 0x7f7fe806b7d0 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe7fff700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe8156410 0x7f7fe8157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7fe77fe700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fe806b7d0 0x7f7fe8170800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c7c0 tx=0x7f7fdc007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f6ffe7fc700 1 -- 172.21.15.3:0/2528537138 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008012e20 con 0x7f7010171510 2023-10-06T04:52:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f6ffe7fc700 1 -- 172.21.15.3:0/2528537138 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700801e640 con 0x7f7010171510 2023-10-06T04:52:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701016e260 con 0x7f7010171510 2023-10-06T04:52:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701016ded0 con 0x7f7010171510 2023-10-06T04:52:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f3420fc4700 1 -- 172.21.15.3:0/900031152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 msgr2=0x7f341c159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.576+0000 7f3420fc4700 1 --2- 172.21.15.3:0/900031152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 0x7f341c159fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f340c00bb90 tx=0x7f340c00b430 comp rx=0 tx=0).stop 2023-10-06T04:52:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f6ffe7fc700 1 -- 172.21.15.3:0/2528537138 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700801ea00 con 0x7f7010171510 2023-10-06T04:52:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f79367fc700 1 -- 172.21.15.3:0/1431373710 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c00f930 con 0x7f792c0051c0 2023-10-06T04:52:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f79367fc700 1 -- 172.21.15.3:0/1431373710 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c00fa90 con 0x7f792c0051c0 2023-10-06T04:52:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794014c6d0 con 0x7f792c0051c0 2023-10-06T04:52:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794014cc90 con 0x7f792c0051c0 2023-10-06T04:52:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f3420fc4700 1 -- 172.21.15.3:0/900031152 shutdown_connections 2023-10-06T04:52:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f3420fc4700 1 --2- 172.21.15.3:0/900031152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c0b1f10 0x7f341c0c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f3420fc4700 1 --2- 172.21.15.3:0/900031152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 0x7f341c159fe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f3420fc4700 1 -- 172.21.15.3:0/900031152 >> 172.21.15.3:0/900031152 conn(0x7f341c066fb0 msgr2=0x7f341c0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f79367fc700 1 -- 172.21.15.3:0/1431373710 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c00fe10 con 0x7f792c0051c0 2023-10-06T04:52:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.577+0000 7f3420fc4700 1 -- 172.21.15.3:0/900031152 shutdown_connections 2023-10-06T04:52:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.578+0000 7f3420fc4700 1 -- 172.21.15.3:0/900031152 wait complete. 2023-10-06T04:52:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.578+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff40052b0 con 0x7f7010171510 2023-10-06T04:52:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.578+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79280052b0 con 0x7f792c0051c0 2023-10-06T04:52:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.578+0000 7f79367fc700 1 -- 172.21.15.3:0/1431373710 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f793c01d690 con 0x7f792c0051c0 2023-10-06T04:52:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.579+0000 7f79367fc700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7924067000 0x7f79240694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.579+0000 7f79367fc700 1 -- 172.21.15.3:0/1431373710 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f793c018070 con 0x7f792c0051c0 2023-10-06T04:52:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.579+0000 7f7945d23700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7924067000 0x7f79240694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.579+0000 7f7fcf7fe700 1 -- 172.21.15.3:0/1431947707 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc005b30 con 0x7f7fe806b7d0 2023-10-06T04:52:26.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.579+0000 7f7fcf7fe700 1 -- 172.21.15.3:0/1431947707 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc005c90 con 0x7f7fe806b7d0 2023-10-06T04:52:26.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.579+0000 7f7fcf7fe700 1 -- 172.21.15.3:0/1431947707 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc007320 con 0x7f7fe806b7d0 2023-10-06T04:52:26.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.580+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe8158560 con 0x7f7fe806b7d0 2023-10-06T04:52:26.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.580+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8158b20 con 0x7f7fe806b7d0 2023-10-06T04:52:26.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.580+0000 7f7945d23700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7924067000 0x7f79240694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79400bef90 tx=0x7f79300091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.580+0000 7f3420fc4700 1 Processor -- start 2023-10-06T04:52:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.580+0000 7f3420fc4700 1 -- start start 2023-10-06T04:52:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.580+0000 7ff1d7fff700 1 -- 172.21.15.3:0/3903365581 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1e4057370 con 0x7ff1f015c930 2023-10-06T04:52:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3420fc4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c06c620 0x7f341c1479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3420fc4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f341c0b1f10 0x7f341c14b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3420fc4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 0x7f341c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3420fc4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c146950 con 0x7f341c155750 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3420fc4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c146c30 con 0x7f341c0b1f10 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3420fc4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c146f10 con 0x7f341c06c620 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3419d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f341c0b1f10 0x7f341c14b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3419d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f341c0b1f10 0x7f341c14b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:38108/0 (socket says 172.21.15.3:38108) 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f341a59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c06c620 0x7f341c1479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f3419d9b700 1 -- 172.21.15.3:0/78326692 learned_addr learned my addr 172.21.15.3:0/78326692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f341ad9d700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 0x7f341c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f341a59c700 1 -- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f341c0b1f10 msgr2=0x7f341c14b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f341a59c700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f341c0b1f10 0x7f341c14b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f341a59c700 1 -- 172.21.15.3:0/78326692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 msgr2=0x7f341c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f341a59c700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 0x7f341c153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f341a59c700 1 -- 172.21.15.3:0/78326692 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c00b6d0 con 0x7f341c06c620 2023-10-06T04:52:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f341a59c700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c06c620 0x7f341c1479f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f340c018000 tx=0x7f340c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe8071680 con 0x7f7fe806b7d0 2023-10-06T04:52:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.581+0000 7f7fcf7fe700 1 -- 172.21.15.3:0/1431947707 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fdc012400 con 0x7f7fe806b7d0 2023-10-06T04:52:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f7fcf7fe700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7fd4066f30 0x7f7fd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f7fcf7fe700 1 -- 172.21.15.3:0/1431947707 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7fdc083b30 con 0x7f7fe806b7d0 2023-10-06T04:52:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.582+0000 7f7fe6ffd700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7fd4066f30 0x7f7fd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.583+0000 7f7fe6ffd700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7fd4066f30 0x7f7fd40693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8159790 tx=0x7f7fd8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.584+0000 7f340a7fc700 1 -- 172.21.15.3:0/78326692 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c010780 con 0x7f341c06c620 2023-10-06T04:52:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.586+0000 7f6ffe7fc700 1 -- 172.21.15.3:0/2528537138 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7008026400 con 0x7f7010171510 2023-10-06T04:52:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.586+0000 7f6ffe7fc700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ff0067000 0x7f6ff00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.586+0000 7f6ffe7fc700 1 -- 172.21.15.3:0/2528537138 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f70080873c0 con 0x7f7010171510 2023-10-06T04:52:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.588+0000 7f700e59c700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ff0067000 0x7f6ff00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.589+0000 7f6ffe7fc700 1 -- 172.21.15.3:0/2528537138 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7008054530 con 0x7f7010171510 2023-10-06T04:52:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.586+0000 7f7fcf7fe700 1 -- 172.21.15.3:0/1431947707 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fdc050c50 con 0x7f7fe806b7d0 2023-10-06T04:52:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.585+0000 7f79367fc700 1 -- 172.21.15.3:0/1431373710 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f793c052550 con 0x7f792c0051c0 2023-10-06T04:52:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.584+0000 7f340a7fc700 1 -- 172.21.15.3:0/78326692 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c01e040 con 0x7f341c06c620 2023-10-06T04:52:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.584+0000 7f340a7fc700 1 -- 172.21.15.3:0/78326692 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c010af0 con 0x7f341c06c620 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.584+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c1471f0 con 0x7f341c06c620 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.584+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c14bd50 con 0x7f341c06c620 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.585+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c147710 con 0x7f341c06c620 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.590+0000 7f340a7fc700 1 -- 172.21.15.3:0/78326692 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f340c02c430 con 0x7f341c06c620 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.590+0000 7f340a7fc700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33fc066f30 0x7f33fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.591+0000 7f340a7fc700 1 -- 172.21.15.3:0/78326692 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f340c014070 con 0x7f341c06c620 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.591+0000 7f3419d9b700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33fc066f30 0x7f33fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.591+0000 7f3419d9b700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33fc066f30 0x7f33fc0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f341c147f30 tx=0x7f3410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.593+0000 7f340a7fc700 1 -- 172.21.15.3:0/78326692 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f340c053870 con 0x7f341c06c620 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.596+0000 7f930830f700 1 Processor -- start 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.596+0000 7f930830f700 1 -- start start 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.597+0000 7f930830f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f930006b7d0 0x7f93000af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.597+0000 7f930830f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93000b1c90 con 0x7f9300156410 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.597+0000 7f930830f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93000b1080 con 0x7f93000b0900 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.597+0000 7f930830f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93000b1360 con 0x7f930006b7d0 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93058aa700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f930006b7d0 0x7f93000af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93058aa700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f930006b7d0 0x7f93000af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:38110/0 (socket says 172.21.15.3:38110) 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93058aa700 1 -- 172.21.15.3:0/2099522943 learned_addr learned my addr 172.21.15.3:0/2099522943 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93068ac700 1 --1- 172.21.15.3:0/2099522943 >> v1:172.21.15.153:6789/0 conn(0x7f93000b0900 0x7f93000c9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49610/0 (socket says 172.21.15.3:49610) 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93058aa700 1 -- 172.21.15.3:0/2099522943 >> v1:172.21.15.153:6789/0 conn(0x7f93000b0900 legacy=0x7f93000c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93050a9700 1 -- 172.21.15.3:0/2099522943 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389996848 0 0) 0x7f93000b1080 con 0x7f93000b0900 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93058aa700 1 -- 172.21.15.3:0/2099522943 >> v1:172.21.15.3:6789/0 conn(0x7f9300156410 legacy=0x7f930006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.598+0000 7f93058aa700 1 -- 172.21.15.3:0/2099522943 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93000b1640 con 0x7f930006b7d0 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.599+0000 7f93058aa700 1 --2- 172.21.15.3:0/2099522943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93000af480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009f20 tx=0x7f92f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927b8cb810bb36fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.599+0000 7f93050a9700 1 -- 172.21.15.3:0/2099522943 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f80092e0 con 0x7f930006b7d0 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.599+0000 7f93050a9700 1 -- 172.21.15.3:0/2099522943 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f800ba40 con 0x7f930006b7d0 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.599+0000 7f93050a9700 1 -- 172.21.15.3:0/2099522943 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f800bde0 con 0x7f930006b7d0 2023-10-06T04:52:26.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.600+0000 7f930830f700 1 -- 172.21.15.3:0/2099522943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 msgr2=0x7f93000af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.600+0000 7f930830f700 1 --2- 172.21.15.3:0/2099522943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93000af480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009f20 tx=0x7f92f8009840 comp rx=0 tx=0).stop 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.600+0000 7f930830f700 1 -- 172.21.15.3:0/2099522943 shutdown_connections 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.600+0000 7f930830f700 1 --2- 172.21.15.3:0/2099522943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93000af480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.600+0000 7f930830f700 1 -- 172.21.15.3:0/2099522943 >> 172.21.15.3:0/2099522943 conn(0x7f93000cdc30 msgr2=0x7f930006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.600+0000 7f930830f700 1 -- 172.21.15.3:0/2099522943 shutdown_connections 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.600+0000 7f930830f700 1 -- 172.21.15.3:0/2099522943 wait complete. 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.602+0000 7f930830f700 1 Processor -- start 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.602+0000 7f930830f700 1 -- start start 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f930830f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93001455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f930830f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93000b0900 0x7f9300159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f930830f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9300156410 0x7f930014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f930830f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9300157290 con 0x7f93000b0900 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f930830f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9300156960 con 0x7f930006b7d0 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f930830f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9300156c40 con 0x7f9300156410 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f93060ab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93001455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f93060ab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93001455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:38122/0 (socket says 172.21.15.3:38122) 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.603+0000 7f93060ab700 1 -- 172.21.15.3:0/3199151150 learned_addr learned my addr 172.21.15.3:0/3199151150 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f93058aa700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93000b0900 0x7f9300159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f93060ab700 1 -- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9300156410 msgr2=0x7f930014af90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f93060ab700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9300156410 0x7f930014af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f93060ab700 1 -- 172.21.15.3:0/3199151150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93000b0900 msgr2=0x7f9300159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f93060ab700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93000b0900 0x7f9300159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f93060ab700 1 -- 172.21.15.3:0/3199151150 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f8009ae0 con 0x7f930006b7d0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f93060ab700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93001455e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f92f000b610 tx=0x7f92f00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f92f67fc700 1 -- 172.21.15.3:0/3199151150 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f000f510 con 0x7f930006b7d0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f92f67fc700 1 -- 172.21.15.3:0/3199151150 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f000f690 con 0x7f930006b7d0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.604+0000 7f92f67fc700 1 -- 172.21.15.3:0/3199151150 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f000fa50 con 0x7f930006b7d0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.605+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9300156f20 con 0x7f930006b7d0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.605+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930014b4d0 con 0x7f930006b7d0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.607+0000 7f92f67fc700 1 -- 172.21.15.3:0/3199151150 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92f001a400 con 0x7f930006b7d0 2023-10-06T04:52:26.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.608+0000 7f700e59c700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ff0067000 0x7f6ff00694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7000007fd0 tx=0x7f7000007f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.608+0000 7f92f67fc700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92e4066f30 0x7f92e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.608+0000 7f92f67fc700 1 -- 172.21.15.3:0/3199151150 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f92f00840f0 con 0x7f930006b7d0 2023-10-06T04:52:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.608+0000 7f93058aa700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92e4066f30 0x7f92e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.609+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9300157570 con 0x7f930006b7d0 2023-10-06T04:52:26.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.611+0000 7f93058aa700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92e4066f30 0x7f92e40693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009540 tx=0x7f92f8019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:26.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.623+0000 7f92f67fc700 1 -- 172.21.15.3:0/3199151150 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92f0051280 con 0x7f930006b7d0 2023-10-06T04:52:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.899+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f29b0005560 con 0x7f29d0155400 2023-10-06T04:52:26.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.905+0000 7f29ccff9700 1 -- 172.21.15.3:0/3039427634 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29c4056730 con 0x7f29d0155400 2023-10-06T04:52:26.910 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:26.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29bc067000 msgr2=0x7f29bc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29bc067000 0x7f29bc0694b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009b10 tx=0x7f29c0008040 comp rx=0 tx=0).stop 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d0155400 msgr2=0x7f29d00ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d0155400 0x7f29d00ca1c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29c40093f0 tx=0x7f29c4017000 comp rx=0 tx=0).stop 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 shutdown_connections 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29bc067000 0x7f29bc0694b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29d006a110 0x7f29d014a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d0155400 0x7f29d00ca1c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 --2- 172.21.15.3:0/3039427634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29d006ecc0 0x7f29d0149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 >> 172.21.15.3:0/3039427634 conn(0x7f29d006c370 msgr2=0x7f29d00c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 shutdown_connections 2023-10-06T04:52:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.913+0000 7f29d6d8c700 1 -- 172.21.15.3:0/3039427634 wait complete. 2023-10-06T04:52:26.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-06T04:52:26.933 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:26.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.941+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f394c071680 con 0x7f394c06a110 2023-10-06T04:52:26.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.942+0000 7f393b7fe700 1 -- 172.21.15.3:0/1373783811 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3940054aa0 con 0x7f394c06a110 2023-10-06T04:52:26.943 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:26.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.946+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3930067050 msgr2=0x7f3930069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.946+0000 7f3951d32700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3930067050 0x7f3930069500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f394c0b1b20 tx=0x7f393c019000 comp rx=0 tx=0).stop 2023-10-06T04:52:26.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.946+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394c06a110 msgr2=0x7f394c0c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.946+0000 7f3951d32700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394c06a110 0x7f394c0c3030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f394000b810 tx=0x7f39400091d0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.949+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 shutdown_connections 2023-10-06T04:52:26.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.949+0000 7f3951d32700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3930067050 0x7f3930069500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.949+0000 7f3951d32700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c155400 0x7f394c0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.949+0000 7f3951d32700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394c06a110 0x7f394c0c3030 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.949+0000 7f3951d32700 1 --2- 172.21.15.3:0/1373783811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f394c06e5b0 0x7f394c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.949+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 >> 172.21.15.3:0/1373783811 conn(0x7f394c06c370 msgr2=0x7f394c0c9320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.950+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 shutdown_connections 2023-10-06T04:52:26.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.950+0000 7f3951d32700 1 -- 172.21.15.3:0/1373783811 wait complete. 2023-10-06T04:52:26.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-10-06T04:52:26.969 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:26.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.970+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7fe8159450 con 0x7f7fe806b7d0 2023-10-06T04:52:26.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.971+0000 7f7fcf7fe700 1 -- 172.21.15.3:0/1431947707 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7fdc054270 con 0x7f7fe806b7d0 2023-10-06T04:52:26.973 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:26.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.973+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7fd4066f30 msgr2=0x7f7fd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.973+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7fd4066f30 0x7f7fd40693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8159790 tx=0x7f7fd8016000 comp rx=0 tx=0).stop 2023-10-06T04:52:26.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.973+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fe806b7d0 msgr2=0x7f7fe8170800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.973+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fe806b7d0 0x7f7fe8170800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c7c0 tx=0x7f7fdc007e00 comp rx=0 tx=0).stop 2023-10-06T04:52:26.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 shutdown_connections 2023-10-06T04:52:26.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7fd4066f30 0x7f7fd40693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fe806b7d0 0x7f7fe8170800 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe8156410 0x7f7fe8157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 --2- 172.21.15.3:0/1431947707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fe80b0900 0x7f7fe81579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 >> 172.21.15.3:0/1431947707 conn(0x7f7fe80cdc30 msgr2=0x7f7fe806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 shutdown_connections 2023-10-06T04:52:26.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.974+0000 7f7fedbdb700 1 -- 172.21.15.3:0/1431947707 wait complete. 2023-10-06T04:52:26.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.985+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9300156aa0 con 0x7f930006b7d0 2023-10-06T04:52:26.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.985+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6ff4005830 con 0x7f7010171510 2023-10-06T04:52:26.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.985+0000 7f92f67fc700 1 -- 172.21.15.3:0/3199151150 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9300156aa0 con 0x7f930006b7d0 2023-10-06T04:52:26.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.986+0000 7f6ffe7fc700 1 -- 172.21.15.3:0/2528537138 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7008057b50 con 0x7f7010171510 2023-10-06T04:52:26.988 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:26.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2023-10-06T04:52:26.989 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:26.990 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92e4066f30 msgr2=0x7f92e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92e4066f30 0x7f92e40693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009540 tx=0x7f92f8019000 comp rx=0 tx=0).stop 2023-10-06T04:52:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 msgr2=0x7f93001455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93001455e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f92f000b610 tx=0x7f92f00090f0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 shutdown_connections 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92e4066f30 0x7f92e40693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930006b7d0 0x7f93001455e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9300156410 0x7f930014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 --2- 172.21.15.3:0/3199151150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93000b0900 0x7f9300159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 >> 172.21.15.3:0/3199151150 conn(0x7f93000cdc30 msgr2=0x7f930006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 shutdown_connections 2023-10-06T04:52:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.988+0000 7f930830f700 1 -- 172.21.15.3:0/3199151150 wait complete. 2023-10-06T04:52:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.989+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ff0067000 msgr2=0x7f6ff00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.989+0000 7f7014d99700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ff0067000 0x7f6ff00694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7000007fd0 tx=0x7f7000007f60 comp rx=0 tx=0).stop 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.989+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7010171510 msgr2=0x7f7010171990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.989+0000 7f7014d99700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7010171510 0x7f7010171990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7008015000 tx=0x7f7008018000 comp rx=0 tx=0).stop 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 shutdown_connections 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ff0067000 0x7f6ff00694b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7010171510 0x7f7010171990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70100b1f10 0x7f701016ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 --2- 172.21.15.3:0/2528537138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701015c930 0x7f701014c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 >> 172.21.15.3:0/2528537138 conn(0x7f7010066fb0 msgr2=0x7f70100b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 shutdown_connections 2023-10-06T04:52:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.990+0000 7f7014d99700 1 -- 172.21.15.3:0/2528537138 wait complete. 2023-10-06T04:52:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.993+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1c8005830 con 0x7ff1f015c930 2023-10-06T04:52:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.993+0000 7ff1d7fff700 1 -- 172.21.15.3:0/3903365581 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1e4056f00 con 0x7ff1f015c930 2023-10-06T04:52:26.995 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.997+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1dc066f30 msgr2=0x7ff1dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.997+0000 7ff1d5ffb700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1dc066f30 0x7ff1dc0693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001c90 tx=0x7ff1e0009040 comp rx=0 tx=0).stop 2023-10-06T04:52:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.997+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f015c930 msgr2=0x7ff1f015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.997+0000 7ff1d5ffb700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f015c930 0x7ff1f015b910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4015000 tx=0x7ff1e4016000 comp rx=0 tx=0).stop 2023-10-06T04:52:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.998+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7928005830 con 0x7f792c0051c0 2023-10-06T04:52:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:26.998+0000 7f79367fc700 1 -- 172.21.15.3:0/1431373710 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f793c055b70 con 0x7f792c0051c0 2023-10-06T04:52:27.000 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 shutdown_connections 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff1dc066f30 0x7ff1dc0693e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1f0157fb0 0x7ff1f016a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f006aad0 0x7ff1f0159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 --2- 172.21.15.3:0/3903365581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f015c930 0x7ff1f015b910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 >> 172.21.15.3:0/3903365581 conn(0x7ff1f0066fb0 msgr2=0x7ff1f0156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 shutdown_connections 2023-10-06T04:52:27.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7ff1d5ffb700 1 -- 172.21.15.3:0/3903365581 wait complete. 2023-10-06T04:52:27.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7924067000 msgr2=0x7f79240694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7f7948788700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7924067000 0x7f79240694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79400bef90 tx=0x7f79300091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c0051c0 msgr2=0x7f794014f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7f7948788700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c0051c0 0x7f794014f230 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f793c011b10 tx=0x7f793c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.000+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 shutdown_connections 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.001+0000 7f7948788700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7924067000 0x7f79240694b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.001+0000 7f7948788700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f794014d8a0 0x7f794014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.001+0000 7f7948788700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f792c0051c0 0x7f794014f230 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.001+0000 7f7948788700 1 --2- 172.21.15.3:0/1431373710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794014a620 0x7f794014b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.001+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 >> 172.21.15.3:0/1431373710 conn(0x7f794006c370 msgr2=0x7f7940159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.001+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 shutdown_connections 2023-10-06T04:52:27.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.001+0000 7f7948788700 1 -- 172.21.15.3:0/1431373710 wait complete. 2023-10-06T04:52:27.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-06T04:52:27.005 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:27.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-06T04:52:27.012 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.012+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f341c14bf40 con 0x7f341c06c620 2023-10-06T04:52:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.013+0000 7f340a7fc700 1 -- 172.21.15.3:0/78326692 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f340c056e90 con 0x7f341c06c620 2023-10-06T04:52:27.015 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33fc066f30 msgr2=0x7f33fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33fc066f30 0x7f33fc0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f341c147f30 tx=0x7f3410009040 comp rx=0 tx=0).stop 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c06c620 msgr2=0x7f341c1479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c06c620 0x7f341c1479f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f340c018000 tx=0x7f340c01b000 comp rx=0 tx=0).stop 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 shutdown_connections 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33fc066f30 0x7f33fc0693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c155750 0x7f341c153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f341c06c620 0x7f341c1479f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 --2- 172.21.15.3:0/78326692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f341c0b1f10 0x7f341c14b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 >> 172.21.15.3:0/78326692 conn(0x7f341c066fb0 msgr2=0x7f341c067380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 shutdown_connections 2023-10-06T04:52:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.015+0000 7f3420fc4700 1 -- 172.21.15.3:0/78326692 wait complete. 2023-10-06T04:52:27.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2023-10-06T04:52:27.023 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:27.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2023-10-06T04:52:27.023 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:27.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-06T04:52:27.030 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:27.030 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T04:52:27.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:52:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.186+0000 7fc6ae4f0700 1 Processor -- start 2023-10-06T04:52:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.186+0000 7fc6ae4f0700 1 -- start start 2023-10-06T04:52:27.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6ae4f0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a815c8f0 0x7fc6a815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6ae4f0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc6a815ccc0 0x7fc6a80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6ae4f0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6a80d06a0 0x7fc6a806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6ae4f0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80cf6c0 con 0x7fc6a80d06a0 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6ae4f0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80cfb50 con 0x7fc6a815ccc0 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6ae4f0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a815e050 con 0x7fc6a815c8f0 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6aca8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6a80d06a0 0x7fc6a806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6aca8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6a80d06a0 0x7fc6a806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33250/0 (socket says 172.21.15.3:33250) 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6aca8d700 1 -- 172.21.15.3:0/3862878210 learned_addr learned my addr 172.21.15.3:0/3862878210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6a77fe700 1 --2- 172.21.15.3:0/3862878210 >> v2:172.21.15.153:3300/0 conn(0x7fc6a815ccc0 0x7fc6a80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6aca8d700 1 -- 172.21.15.3:0/3862878210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a815c8f0 msgr2=0x7fc6a815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6a7fff700 1 --2- 172.21.15.3:0/3862878210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a815c8f0 0x7fc6a815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6aca8d700 1 --2- 172.21.15.3:0/3862878210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a815c8f0 0x7fc6a815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6aca8d700 1 -- 172.21.15.3:0/3862878210 >> v2:172.21.15.153:3300/0 conn(0x7fc6a815ccc0 msgr2=0x7fc6a80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.187+0000 7fc6aca8d700 1 --2- 172.21.15.3:0/3862878210 >> v2:172.21.15.153:3300/0 conn(0x7fc6a815ccc0 0x7fc6a80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6aca8d700 1 -- 172.21.15.3:0/3862878210 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a80d0af0 con 0x7fc6a80d06a0 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6a77fe700 1 --2- 172.21.15.3:0/3862878210 >> v2:172.21.15.153:3300/0 conn(0x7fc6a815ccc0 0x7fc6a80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6a7fff700 1 --2- 172.21.15.3:0/3862878210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a815c8f0 0x7fc6a815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6aca8d700 1 --2- 172.21.15.3:0/3862878210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a80d06a0 0x7fc6a806d950 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6a806bed0 tx=0x7fc6a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf754b225d63170 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6a6ffd700 1 -- 172.21.15.3:0/3862878210 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000e070 con 0x7fc6a80d06a0 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6a6ffd700 1 -- 172.21.15.3:0/3862878210 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a000ba50 con 0x7fc6a80d06a0 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6a6ffd700 1 -- 172.21.15.3:0/3862878210 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00092e0 con 0x7fc6a80d06a0 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/3862878210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a80d06a0 msgr2=0x7fc6a806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/3862878210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a80d06a0 0x7fc6a806d950 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6a806bed0 tx=0x7fc6a0009840 comp rx=0 tx=0).stop 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/3862878210 shutdown_connections 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/3862878210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a80d06a0 0x7fc6a806d950 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/3862878210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a815c8f0 0x7fc6a815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/3862878210 >> v2:172.21.15.153:3300/0 conn(0x7fc6a815ccc0 0x7fc6a80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/3862878210 >> 172.21.15.3:0/3862878210 conn(0x7fc6a8070310 msgr2=0x7fc6a8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/3862878210 shutdown_connections 2023-10-06T04:52:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.188+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/3862878210 wait complete. 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.189+0000 7fc6ae4f0700 1 Processor -- start 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6ae4f0700 1 -- start start 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6ae4f0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a80d06a0 0x7fc6a8145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6ae4f0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a815c8f0 0x7fc6a815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6ae4f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a815ccc0 0x7fc6a80c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6ae4f0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a8146440 con 0x7fc6a815ccc0 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6ae4f0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80c5220 con 0x7fc6a815c8f0 2023-10-06T04:52:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6ae4f0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80c5500 con 0x7fc6a80d06a0 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.190+0000 7fc6aca8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a815ccc0 0x7fc6a80c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a815ccc0 0x7fc6a80c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33264/0 (socket says 172.21.15.3:33264) 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 -- 172.21.15.3:0/858906185 learned_addr learned my addr 172.21.15.3:0/858906185 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6a77fe700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a815c8f0 0x7fc6a815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6a7fff700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a80d06a0 0x7fc6a8145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 -- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a80d06a0 msgr2=0x7fc6a8145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a80d06a0 0x7fc6a8145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 -- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a815c8f0 msgr2=0x7fc6a815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a815c8f0 0x7fc6a815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 -- 172.21.15.3:0/858906185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a0009ae0 con 0x7fc6a815ccc0 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6a7fff700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a80d06a0 0x7fc6a8145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6aca8d700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a815ccc0 0x7fc6a80c6c50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6a806eaa0 tx=0x7fc6a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc68ffff700 1 -- 172.21.15.3:0/858906185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0008dd0 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc68ffff700 1 -- 172.21.15.3:0/858906185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a001c410 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a80c57e0 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc68ffff700 1 -- 172.21.15.3:0/858906185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001c7d0 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.191+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a80c5da0 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.192+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a8071680 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.193+0000 7fc68ffff700 1 -- 172.21.15.3:0/858906185 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6a0003040 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.193+0000 7fc68ffff700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc694066f30 0x7fc6940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.193+0000 7fc68ffff700 1 -- 172.21.15.3:0/858906185 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc6a0086060 con 0x7fc6a815ccc0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.193+0000 7fc6a7fff700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc694066f30 0x7fc6940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.194+0000 7fc6a7fff700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc694066f30 0x7fc6940693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc69800ab50 tx=0x7fc698008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.197+0000 7fc68ffff700 1 -- 172.21.15.3:0/858906185 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6a0053170 con 0x7fc6a815ccc0 2023-10-06T04:52:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.387+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6a80ccd70 con 0x7fc694066f30 2023-10-06T04:52:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.396+0000 7fc68ffff700 1 -- 172.21.15.3:0/858906185 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286785 (secure 0 0 0) 0x7fc6a80ccd70 con 0x7fc694066f30 2023-10-06T04:52:27.402 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:27.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc694066f30 msgr2=0x7fc6940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc694066f30 0x7fc6940693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc69800ab50 tx=0x7fc698008040 comp rx=0 tx=0).stop 2023-10-06T04:52:27.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a815ccc0 msgr2=0x7fc6a80c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a815ccc0 0x7fc6a80c6c50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6a806eaa0 tx=0x7fc6a0018000 comp rx=0 tx=0).stop 2023-10-06T04:52:27.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 shutdown_connections 2023-10-06T04:52:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc694066f30 0x7fc6940693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a815ccc0 0x7fc6a80c6c50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6a80d06a0 0x7fc6a8145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 --2- 172.21.15.3:0/858906185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a815c8f0 0x7fc6a815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 >> 172.21.15.3:0/858906185 conn(0x7fc6a8070310 msgr2=0x7fc6a80b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 shutdown_connections 2023-10-06T04:52:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.399+0000 7fc6ae4f0700 1 -- 172.21.15.3:0/858906185 wait complete. 2023-10-06T04:52:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:52:27.413 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T04:52:25.424819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381116,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593604,"statfs":{"total":773094113280,"available":770655850496,"internally_reserved":0,"allocated":2879488,"data_stored":1774761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.793045"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663223+0000","last_change":"2023-10-06T04:52:18.658025+0000","last_active":"2023-10-06T04:52:18.663223+0000","last_peered":"2023-10-06T04:52:18.663223+0000","last_clean":"2023-10-06T04:52:18.663223+0000","last_became_active":"2023-10-06T04:52:18.657724+0000","last_became_peered":"2023-10-06T04:52:18.657724+0000","last_unstale":"2023-10-06T04:52:18.663223+0000","last_undegraded":"2023-10-06T04:52:18.663223+0000","last_fullsized":"2023-10-06T04:52:18.663223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:01.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661205+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.661205+0000","last_peered":"2023-10-06T04:52:18.661205+0000","last_clean":"2023-10-06T04:52:18.661205+0000","last_became_active":"2023-10-06T04:52:18.658286+0000","last_became_peered":"2023-10-06T04:52:18.658286+0000","last_unstale":"2023-10-06T04:52:18.661205+0000","last_undegraded":"2023-10-06T04:52:18.661205+0000","last_fullsized":"2023-10-06T04:52:18.661205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665708+0000","last_change":"2023-10-06T04:52:18.652791+0000","last_active":"2023-10-06T04:52:18.665708+0000","last_peered":"2023-10-06T04:52:18.665708+0000","last_clean":"2023-10-06T04:52:18.665708+0000","last_became_active":"2023-10-06T04:52:18.652522+0000","last_became_peered":"2023-10-06T04:52:18.652522+0000","last_unstale":"2023-10-06T04:52:18.665708+0000","last_undegraded":"2023-10-06T04:52:18.665708+0000","last_fullsized":"2023-10-06T04:52:18.665708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662069+0000","last_change":"2023-10-06T04:52:18.649578+0000","last_active":"2023-10-06T04:52:18.662069+0000","last_peered":"2023-10-06T04:52:18.662069+0000","last_clean":"2023-10-06T04:52:18.662069+0000","last_became_active":"2023-10-06T04:52:18.649294+0000","last_became_peered":"2023-10-06T04:52:18.649294+0000","last_unstale":"2023-10-06T04:52:18.662069+0000","last_undegraded":"2023-10-06T04:52:18.662069+0000","last_fullsized":"2023-10-06T04:52:18.662069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:28.571226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658392+0000","last_change":"2023-10-06T04:52:18.653657+0000","last_active":"2023-10-06T04:52:18.658392+0000","last_peered":"2023-10-06T04:52:18.658392+0000","last_clean":"2023-10-06T04:52:18.658392+0000","last_became_active":"2023-10-06T04:52:18.653363+0000","last_became_peered":"2023-10-06T04:52:18.653363+0000","last_unstale":"2023-10-06T04:52:18.658392+0000","last_undegraded":"2023-10-06T04:52:18.658392+0000","last_fullsized":"2023-10-06T04:52:18.658392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665600+0000","last_change":"2023-10-06T04:52:18.654512+0000","last_active":"2023-10-06T04:52:18.665600+0000","last_peered":"2023-10-06T04:52:18.665600+0000","last_clean":"2023-10-06T04:52:18.665600+0000","last_became_active":"2023-10-06T04:52:18.653897+0000","last_became_peered":"2023-10-06T04:52:18.653897+0000","last_unstale":"2023-10-06T04:52:18.665600+0000","last_undegraded":"2023-10-06T04:52:18.665600+0000","last_fullsized":"2023-10-06T04:52:18.665600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:38.713099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663473+0000","last_change":"2023-10-06T04:52:18.655144+0000","last_active":"2023-10-06T04:52:18.663473+0000","last_peered":"2023-10-06T04:52:18.663473+0000","last_clean":"2023-10-06T04:52:18.663473+0000","last_became_active":"2023-10-06T04:52:18.654891+0000","last_became_peered":"2023-10-06T04:52:18.654891+0000","last_unstale":"2023-10-06T04:52:18.663473+0000","last_undegraded":"2023-10-06T04:52:18.663473+0000","last_fullsized":"2023-10-06T04:52:18.663473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667181+0000","last_change":"2023-10-06T04:52:18.666282+0000","last_active":"2023-10-06T04:52:18.667181+0000","last_peered":"2023-10-06T04:52:18.667181+0000","last_clean":"2023-10-06T04:52:18.667181+0000","last_became_active":"2023-10-06T04:52:18.665988+0000","last_became_peered":"2023-10-06T04:52:18.665988+0000","last_unstale":"2023-10-06T04:52:18.667181+0000","last_undegraded":"2023-10-06T04:52:18.667181+0000","last_fullsized":"2023-10-06T04:52:18.667181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665701+0000","last_change":"2023-10-06T04:52:18.661193+0000","last_active":"2023-10-06T04:52:18.665701+0000","last_peered":"2023-10-06T04:52:18.665701+0000","last_clean":"2023-10-06T04:52:18.665701+0000","last_became_active":"2023-10-06T04:52:18.660905+0000","last_became_peered":"2023-10-06T04:52:18.660905+0000","last_unstale":"2023-10-06T04:52:18.665701+0000","last_undegraded":"2023-10-06T04:52:18.665701+0000","last_fullsized":"2023-10-06T04:52:18.665701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656410+0000","last_change":"2023-10-06T04:52:18.654460+0000","last_active":"2023-10-06T04:52:18.656410+0000","last_peered":"2023-10-06T04:52:18.656410+0000","last_clean":"2023-10-06T04:52:18.656410+0000","last_became_active":"2023-10-06T04:52:18.653370+0000","last_became_peered":"2023-10-06T04:52:18.653370+0000","last_unstale":"2023-10-06T04:52:18.656410+0000","last_undegraded":"2023-10-06T04:52:18.656410+0000","last_fullsized":"2023-10-06T04:52:18.656410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659795+0000","last_change":"2023-10-06T04:52:18.650974+0000","last_active":"2023-10-06T04:52:18.659795+0000","last_peered":"2023-10-06T04:52:18.659795+0000","last_clean":"2023-10-06T04:52:18.659795+0000","last_became_active":"2023-10-06T04:52:18.650708+0000","last_became_peered":"2023-10-06T04:52:18.650708+0000","last_unstale":"2023-10-06T04:52:18.659795+0000","last_undegraded":"2023-10-06T04:52:18.659795+0000","last_fullsized":"2023-10-06T04:52:18.659795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659746+0000","last_change":"2023-10-06T04:52:18.651167+0000","last_active":"2023-10-06T04:52:18.659746+0000","last_peered":"2023-10-06T04:52:18.659746+0000","last_clean":"2023-10-06T04:52:18.659746+0000","last_became_active":"2023-10-06T04:52:18.650869+0000","last_became_peered":"2023-10-06T04:52:18.650869+0000","last_unstale":"2023-10-06T04:52:18.659746+0000","last_undegraded":"2023-10-06T04:52:18.659746+0000","last_fullsized":"2023-10-06T04:52:18.659746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:55.372045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655957+0000","last_change":"2023-10-06T04:52:18.653721+0000","last_active":"2023-10-06T04:52:18.655957+0000","last_peered":"2023-10-06T04:52:18.655957+0000","last_clean":"2023-10-06T04:52:18.655957+0000","last_became_active":"2023-10-06T04:52:18.653435+0000","last_became_peered":"2023-10-06T04:52:18.653435+0000","last_unstale":"2023-10-06T04:52:18.655957+0000","last_undegraded":"2023-10-06T04:52:18.655957+0000","last_fullsized":"2023-10-06T04:52:18.655957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:43.201875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656748+0000","last_change":"2023-10-06T04:52:18.655632+0000","last_active":"2023-10-06T04:52:18.656748+0000","last_peered":"2023-10-06T04:52:18.656748+0000","last_clean":"2023-10-06T04:52:18.656748+0000","last_became_active":"2023-10-06T04:52:18.655335+0000","last_became_peered":"2023-10-06T04:52:18.655335+0000","last_unstale":"2023-10-06T04:52:18.656748+0000","last_undegraded":"2023-10-06T04:52:18.656748+0000","last_fullsized":"2023-10-06T04:52:18.656748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665013+0000","last_change":"2023-10-06T04:52:18.663599+0000","last_active":"2023-10-06T04:52:18.665013+0000","last_peered":"2023-10-06T04:52:18.665013+0000","last_clean":"2023-10-06T04:52:18.665013+0000","last_became_active":"2023-10-06T04:52:18.663303+0000","last_became_peered":"2023-10-06T04:52:18.663303+0000","last_unstale":"2023-10-06T04:52:18.665013+0000","last_undegraded":"2023-10-06T04:52:18.665013+0000","last_fullsized":"2023-10-06T04:52:18.665013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665493+0000","last_change":"2023-10-06T04:52:18.652910+0000","last_active":"2023-10-06T04:52:18.665493+0000","last_peered":"2023-10-06T04:52:18.665493+0000","last_clean":"2023-10-06T04:52:18.665493+0000","last_became_active":"2023-10-06T04:52:18.652589+0000","last_became_peered":"2023-10-06T04:52:18.652589+0000","last_unstale":"2023-10-06T04:52:18.665493+0000","last_undegraded":"2023-10-06T04:52:18.665493+0000","last_fullsized":"2023-10-06T04:52:18.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:10.947807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658434+0000","last_change":"2023-10-06T04:52:18.654868+0000","last_active":"2023-10-06T04:52:18.658434+0000","last_peered":"2023-10-06T04:52:18.658434+0000","last_clean":"2023-10-06T04:52:18.658434+0000","last_became_active":"2023-10-06T04:52:18.654590+0000","last_became_peered":"2023-10-06T04:52:18.654590+0000","last_unstale":"2023-10-06T04:52:18.658434+0000","last_undegraded":"2023-10-06T04:52:18.658434+0000","last_fullsized":"2023-10-06T04:52:18.658434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665816+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.665816+0000","last_peered":"2023-10-06T04:52:18.665816+0000","last_clean":"2023-10-06T04:52:18.665816+0000","last_became_active":"2023-10-06T04:52:18.658358+0000","last_became_peered":"2023-10-06T04:52:18.658358+0000","last_unstale":"2023-10-06T04:52:18.665816+0000","last_undegraded":"2023-10-06T04:52:18.665816+0000","last_fullsized":"2023-10-06T04:52:18.665816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665491+0000","last_change":"2023-10-06T04:52:18.662801+0000","last_active":"2023-10-06T04:52:18.665491+0000","last_peered":"2023-10-06T04:52:18.665491+0000","last_clean":"2023-10-06T04:52:18.665491+0000","last_became_active":"2023-10-06T04:52:18.662534+0000","last_became_peered":"2023-10-06T04:52:18.662534+0000","last_unstale":"2023-10-06T04:52:18.665491+0000","last_undegraded":"2023-10-06T04:52:18.665491+0000","last_fullsized":"2023-10-06T04:52:18.665491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:53.970289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666479+0000","last_change":"2023-10-06T04:52:18.664643+0000","last_active":"2023-10-06T04:52:18.666479+0000","last_peered":"2023-10-06T04:52:18.666479+0000","last_clean":"2023-10-06T04:52:18.666479+0000","last_became_active":"2023-10-06T04:52:18.664342+0000","last_became_peered":"2023-10-06T04:52:18.664342+0000","last_unstale":"2023-10-06T04:52:18.666479+0000","last_undegraded":"2023-10-06T04:52:18.666479+0000","last_fullsized":"2023-10-06T04:52:18.666479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:33.202071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664983+0000","last_change":"2023-10-06T04:52:18.663114+0000","last_active":"2023-10-06T04:52:18.664983+0000","last_peered":"2023-10-06T04:52:18.664983+0000","last_clean":"2023-10-06T04:52:18.664983+0000","last_became_active":"2023-10-06T04:52:18.662825+0000","last_became_peered":"2023-10-06T04:52:18.662825+0000","last_unstale":"2023-10-06T04:52:18.664983+0000","last_undegraded":"2023-10-06T04:52:18.664983+0000","last_fullsized":"2023-10-06T04:52:18.664983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650717+0000","last_change":"2023-10-06T04:52:18.648283+0000","last_active":"2023-10-06T04:52:18.650717+0000","last_peered":"2023-10-06T04:52:18.650717+0000","last_clean":"2023-10-06T04:52:18.650717+0000","last_became_active":"2023-10-06T04:52:18.647966+0000","last_became_peered":"2023-10-06T04:52:18.647966+0000","last_unstale":"2023-10-06T04:52:18.650717+0000","last_undegraded":"2023-10-06T04:52:18.650717+0000","last_fullsized":"2023-10-06T04:52:18.650717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661203+0000","last_change":"2023-10-06T04:52:18.655500+0000","last_active":"2023-10-06T04:52:18.661203+0000","last_peered":"2023-10-06T04:52:18.661203+0000","last_clean":"2023-10-06T04:52:18.661203+0000","last_became_active":"2023-10-06T04:52:18.655234+0000","last_became_peered":"2023-10-06T04:52:18.655234+0000","last_unstale":"2023-10-06T04:52:18.661203+0000","last_undegraded":"2023-10-06T04:52:18.661203+0000","last_fullsized":"2023-10-06T04:52:18.661203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663966+0000","last_change":"2023-10-06T04:52:18.660815+0000","last_active":"2023-10-06T04:52:18.663966+0000","last_peered":"2023-10-06T04:52:18.663966+0000","last_clean":"2023-10-06T04:52:18.663966+0000","last_became_active":"2023-10-06T04:52:18.660475+0000","last_became_peered":"2023-10-06T04:52:18.660475+0000","last_unstale":"2023-10-06T04:52:18.663966+0000","last_undegraded":"2023-10-06T04:52:18.663966+0000","last_fullsized":"2023-10-06T04:52:18.663966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667741+0000","last_change":"2023-10-06T04:52:18.663035+0000","last_active":"2023-10-06T04:52:18.667741+0000","last_peered":"2023-10-06T04:52:18.667741+0000","last_clean":"2023-10-06T04:52:18.667741+0000","last_became_active":"2023-10-06T04:52:18.662611+0000","last_became_peered":"2023-10-06T04:52:18.662611+0000","last_unstale":"2023-10-06T04:52:18.667741+0000","last_undegraded":"2023-10-06T04:52:18.667741+0000","last_fullsized":"2023-10-06T04:52:18.667741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661242+0000","last_change":"2023-10-06T04:52:18.658476+0000","last_active":"2023-10-06T04:52:18.661242+0000","last_peered":"2023-10-06T04:52:18.661242+0000","last_clean":"2023-10-06T04:52:18.661242+0000","last_became_active":"2023-10-06T04:52:18.658197+0000","last_became_peered":"2023-10-06T04:52:18.658197+0000","last_unstale":"2023-10-06T04:52:18.661242+0000","last_undegraded":"2023-10-06T04:52:18.661242+0000","last_fullsized":"2023-10-06T04:52:18.661242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661137+0000","last_change":"2023-10-06T04:52:18.658955+0000","last_active":"2023-10-06T04:52:18.661137+0000","last_peered":"2023-10-06T04:52:18.661137+0000","last_clean":"2023-10-06T04:52:18.661137+0000","last_became_active":"2023-10-06T04:52:18.658713+0000","last_became_peered":"2023-10-06T04:52:18.658713+0000","last_unstale":"2023-10-06T04:52:18.661137+0000","last_undegraded":"2023-10-06T04:52:18.661137+0000","last_fullsized":"2023-10-06T04:52:18.661137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.651332+0000","last_change":"2023-10-06T04:52:17.656516+0000","last_active":"2023-10-06T04:52:18.651332+0000","last_peered":"2023-10-06T04:52:18.651332+0000","last_clean":"2023-10-06T04:52:18.651332+0000","last_became_active":"2023-10-06T04:52:17.656032+0000","last_became_peered":"2023-10-06T04:52:17.656032+0000","last_unstale":"2023-10-06T04:52:18.651332+0000","last_undegraded":"2023-10-06T04:52:18.651332+0000","last_fullsized":"2023-10-06T04:52:18.651332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660104+0000","last_change":"2023-10-06T04:52:18.656019+0000","last_active":"2023-10-06T04:52:18.660104+0000","last_peered":"2023-10-06T04:52:18.660104+0000","last_clean":"2023-10-06T04:52:18.660104+0000","last_became_active":"2023-10-06T04:52:18.655673+0000","last_became_peered":"2023-10-06T04:52:18.655673+0000","last_unstale":"2023-10-06T04:52:18.660104+0000","last_undegraded":"2023-10-06T04:52:18.660104+0000","last_fullsized":"2023-10-06T04:52:18.660104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643759+0000","last_change":"2023-10-06T04:52:17.645852+0000","last_active":"2023-10-06T04:52:18.643759+0000","last_peered":"2023-10-06T04:52:18.643759+0000","last_clean":"2023-10-06T04:52:18.643759+0000","last_became_active":"2023-10-06T04:52:17.645528+0000","last_became_peered":"2023-10-06T04:52:17.645528+0000","last_unstale":"2023-10-06T04:52:18.643759+0000","last_undegraded":"2023-10-06T04:52:18.643759+0000","last_fullsized":"2023-10-06T04:52:18.643759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655629+0000","last_change":"2023-10-06T04:52:18.649691+0000","last_active":"2023-10-06T04:52:18.655629+0000","last_peered":"2023-10-06T04:52:18.655629+0000","last_clean":"2023-10-06T04:52:18.655629+0000","last_became_active":"2023-10-06T04:52:18.649348+0000","last_became_peered":"2023-10-06T04:52:18.649348+0000","last_unstale":"2023-10-06T04:52:18.655629+0000","last_undegraded":"2023-10-06T04:52:18.655629+0000","last_fullsized":"2023-10-06T04:52:18.655629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643218+0000","last_change":"2023-10-06T04:52:17.651301+0000","last_active":"2023-10-06T04:52:18.643218+0000","last_peered":"2023-10-06T04:52:18.643218+0000","last_clean":"2023-10-06T04:52:18.643218+0000","last_became_active":"2023-10-06T04:52:17.650967+0000","last_became_peered":"2023-10-06T04:52:17.650967+0000","last_unstale":"2023-10-06T04:52:18.643218+0000","last_undegraded":"2023-10-06T04:52:18.643218+0000","last_fullsized":"2023-10-06T04:52:18.643218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662097+0000","last_change":"2023-10-06T04:52:18.656399+0000","last_active":"2023-10-06T04:52:18.662097+0000","last_peered":"2023-10-06T04:52:18.662097+0000","last_clean":"2023-10-06T04:52:18.662097+0000","last_became_active":"2023-10-06T04:52:18.656106+0000","last_became_peered":"2023-10-06T04:52:18.656106+0000","last_unstale":"2023-10-06T04:52:18.662097+0000","last_undegraded":"2023-10-06T04:52:18.662097+0000","last_fullsized":"2023-10-06T04:52:18.662097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.640098+0000","last_change":"2023-10-06T04:52:17.654663+0000","last_active":"2023-10-06T04:52:18.640098+0000","last_peered":"2023-10-06T04:52:18.640098+0000","last_clean":"2023-10-06T04:52:18.640098+0000","last_became_active":"2023-10-06T04:52:17.654402+0000","last_became_peered":"2023-10-06T04:52:17.654402+0000","last_unstale":"2023-10-06T04:52:18.640098+0000","last_undegraded":"2023-10-06T04:52:18.640098+0000","last_fullsized":"2023-10-06T04:52:18.640098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660162+0000","last_change":"2023-10-06T04:52:18.655602+0000","last_active":"2023-10-06T04:52:18.660162+0000","last_peered":"2023-10-06T04:52:18.660162+0000","last_clean":"2023-10-06T04:52:18.660162+0000","last_became_active":"2023-10-06T04:52:18.655329+0000","last_became_peered":"2023-10-06T04:52:18.655329+0000","last_unstale":"2023-10-06T04:52:18.660162+0000","last_undegraded":"2023-10-06T04:52:18.660162+0000","last_fullsized":"2023-10-06T04:52:18.660162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647360+0000","last_change":"2023-10-06T04:52:17.652631+0000","last_active":"2023-10-06T04:52:18.647360+0000","last_peered":"2023-10-06T04:52:18.647360+0000","last_clean":"2023-10-06T04:52:18.647360+0000","last_became_active":"2023-10-06T04:52:17.652340+0000","last_became_peered":"2023-10-06T04:52:17.652340+0000","last_unstale":"2023-10-06T04:52:18.647360+0000","last_undegraded":"2023-10-06T04:52:18.647360+0000","last_fullsized":"2023-10-06T04:52:18.647360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659061+0000","last_change":"2023-10-06T04:52:18.651950+0000","last_active":"2023-10-06T04:52:18.659061+0000","last_peered":"2023-10-06T04:52:18.659061+0000","last_clean":"2023-10-06T04:52:18.659061+0000","last_became_active":"2023-10-06T04:52:18.651587+0000","last_became_peered":"2023-10-06T04:52:18.651587+0000","last_unstale":"2023-10-06T04:52:18.659061+0000","last_undegraded":"2023-10-06T04:52:18.659061+0000","last_fullsized":"2023-10-06T04:52:18.659061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667446+0000","last_change":"2023-10-06T04:52:17.655489+0000","last_active":"2023-10-06T04:52:18.667446+0000","last_peered":"2023-10-06T04:52:18.667446+0000","last_clean":"2023-10-06T04:52:18.667446+0000","last_became_active":"2023-10-06T04:52:17.655131+0000","last_became_peered":"2023-10-06T04:52:17.655131+0000","last_unstale":"2023-10-06T04:52:18.667446+0000","last_undegraded":"2023-10-06T04:52:18.667446+0000","last_fullsized":"2023-10-06T04:52:18.667446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667757+0000","last_change":"2023-10-06T04:52:18.662999+0000","last_active":"2023-10-06T04:52:18.667757+0000","last_peered":"2023-10-06T04:52:18.667757+0000","last_clean":"2023-10-06T04:52:18.667757+0000","last_became_active":"2023-10-06T04:52:18.662603+0000","last_became_peered":"2023-10-06T04:52:18.662603+0000","last_unstale":"2023-10-06T04:52:18.667757+0000","last_undegraded":"2023-10-06T04:52:18.667757+0000","last_fullsized":"2023-10-06T04:52:18.667757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.669196+0000","last_change":"2023-10-06T04:52:17.655293+0000","last_active":"2023-10-06T04:52:18.669196+0000","last_peered":"2023-10-06T04:52:18.669196+0000","last_clean":"2023-10-06T04:52:18.669196+0000","last_became_active":"2023-10-06T04:52:17.654390+0000","last_became_peered":"2023-10-06T04:52:17.654390+0000","last_unstale":"2023-10-06T04:52:18.669196+0000","last_undegraded":"2023-10-06T04:52:18.669196+0000","last_fullsized":"2023-10-06T04:52:18.669196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665827+0000","last_change":"2023-10-06T04:52:18.657322+0000","last_active":"2023-10-06T04:52:18.665827+0000","last_peered":"2023-10-06T04:52:18.665827+0000","last_clean":"2023-10-06T04:52:18.665827+0000","last_became_active":"2023-10-06T04:52:18.657046+0000","last_became_peered":"2023-10-06T04:52:18.657046+0000","last_unstale":"2023-10-06T04:52:18.665827+0000","last_undegraded":"2023-10-06T04:52:18.665827+0000","last_fullsized":"2023-10-06T04:52:18.665827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659855+0000","last_change":"2023-10-06T04:52:17.657213+0000","last_active":"2023-10-06T04:52:18.659855+0000","last_peered":"2023-10-06T04:52:18.659855+0000","last_clean":"2023-10-06T04:52:18.659855+0000","last_became_active":"2023-10-06T04:52:17.656945+0000","last_became_peered":"2023-10-06T04:52:17.656945+0000","last_unstale":"2023-10-06T04:52:18.659855+0000","last_undegraded":"2023-10-06T04:52:18.659855+0000","last_fullsized":"2023-10-06T04:52:18.659855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:17.302823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659796+0000","last_change":"2023-10-06T04:52:18.655013+0000","last_active":"2023-10-06T04:52:18.659796+0000","last_peered":"2023-10-06T04:52:18.659796+0000","last_clean":"2023-10-06T04:52:18.659796+0000","last_became_active":"2023-10-06T04:52:18.654734+0000","last_became_peered":"2023-10-06T04:52:18.654734+0000","last_unstale":"2023-10-06T04:52:18.659796+0000","last_undegraded":"2023-10-06T04:52:18.659796+0000","last_fullsized":"2023-10-06T04:52:18.659796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638374+0000","last_change":"2023-10-06T04:52:17.650696+0000","last_active":"2023-10-06T04:52:18.638374+0000","last_peered":"2023-10-06T04:52:18.638374+0000","last_clean":"2023-10-06T04:52:18.638374+0000","last_became_active":"2023-10-06T04:52:17.650402+0000","last_became_peered":"2023-10-06T04:52:17.650402+0000","last_unstale":"2023-10-06T04:52:18.638374+0000","last_undegraded":"2023-10-06T04:52:18.638374+0000","last_fullsized":"2023-10-06T04:52:18.638374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661155+0000","last_change":"2023-10-06T04:52:18.658995+0000","last_active":"2023-10-06T04:52:18.661155+0000","last_peered":"2023-10-06T04:52:18.661155+0000","last_clean":"2023-10-06T04:52:18.661155+0000","last_became_active":"2023-10-06T04:52:18.658717+0000","last_became_peered":"2023-10-06T04:52:18.658717+0000","last_unstale":"2023-10-06T04:52:18.661155+0000","last_undegraded":"2023-10-06T04:52:18.661155+0000","last_fullsized":"2023-10-06T04:52:18.661155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646904+0000","last_change":"2023-10-06T04:52:17.657760+0000","last_active":"2023-10-06T04:52:18.646904+0000","last_peered":"2023-10-06T04:52:18.646904+0000","last_clean":"2023-10-06T04:52:18.646904+0000","last_became_active":"2023-10-06T04:52:17.657484+0000","last_became_peered":"2023-10-06T04:52:17.657484+0000","last_unstale":"2023-10-06T04:52:18.646904+0000","last_undegraded":"2023-10-06T04:52:18.646904+0000","last_fullsized":"2023-10-06T04:52:18.646904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665775+0000","last_change":"2023-10-06T04:52:18.661243+0000","last_active":"2023-10-06T04:52:18.665775+0000","last_peered":"2023-10-06T04:52:18.665775+0000","last_clean":"2023-10-06T04:52:18.665775+0000","last_became_active":"2023-10-06T04:52:18.660932+0000","last_became_peered":"2023-10-06T04:52:18.660932+0000","last_unstale":"2023-10-06T04:52:18.665775+0000","last_undegraded":"2023-10-06T04:52:18.665775+0000","last_fullsized":"2023-10-06T04:52:18.665775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666172+0000","last_change":"2023-10-06T04:52:17.665506+0000","last_active":"2023-10-06T04:52:18.666172+0000","last_peered":"2023-10-06T04:52:18.666172+0000","last_clean":"2023-10-06T04:52:18.666172+0000","last_became_active":"2023-10-06T04:52:17.665209+0000","last_became_peered":"2023-10-06T04:52:17.665209+0000","last_unstale":"2023-10-06T04:52:18.666172+0000","last_undegraded":"2023-10-06T04:52:18.666172+0000","last_fullsized":"2023-10-06T04:52:18.666172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660252+0000","last_change":"2023-10-06T04:52:18.654716+0000","last_active":"2023-10-06T04:52:18.660252+0000","last_peered":"2023-10-06T04:52:18.660252+0000","last_clean":"2023-10-06T04:52:18.660252+0000","last_became_active":"2023-10-06T04:52:18.653774+0000","last_became_peered":"2023-10-06T04:52:18.653774+0000","last_unstale":"2023-10-06T04:52:18.660252+0000","last_undegraded":"2023-10-06T04:52:18.660252+0000","last_fullsized":"2023-10-06T04:52:18.660252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659771+0000","last_change":"2023-10-06T04:52:17.657327+0000","last_active":"2023-10-06T04:52:18.659771+0000","last_peered":"2023-10-06T04:52:18.659771+0000","last_clean":"2023-10-06T04:52:18.659771+0000","last_became_active":"2023-10-06T04:52:17.657054+0000","last_became_peered":"2023-10-06T04:52:17.657054+0000","last_unstale":"2023-10-06T04:52:18.659771+0000","last_undegraded":"2023-10-06T04:52:18.659771+0000","last_fullsized":"2023-10-06T04:52:18.659771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655497+0000","last_change":"2023-10-06T04:52:18.652991+0000","last_active":"2023-10-06T04:52:18.655497+0000","last_peered":"2023-10-06T04:52:18.655497+0000","last_clean":"2023-10-06T04:52:18.655497+0000","last_became_active":"2023-10-06T04:52:18.652695+0000","last_became_peered":"2023-10-06T04:52:18.652695+0000","last_unstale":"2023-10-06T04:52:18.655497+0000","last_undegraded":"2023-10-06T04:52:18.655497+0000","last_fullsized":"2023-10-06T04:52:18.655497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665605+0000","last_change":"2023-10-06T04:52:17.659228+0000","last_active":"2023-10-06T04:52:18.665605+0000","last_peered":"2023-10-06T04:52:18.665605+0000","last_clean":"2023-10-06T04:52:18.665605+0000","last_became_active":"2023-10-06T04:52:17.658947+0000","last_became_peered":"2023-10-06T04:52:17.658947+0000","last_unstale":"2023-10-06T04:52:18.665605+0000","last_undegraded":"2023-10-06T04:52:18.665605+0000","last_fullsized":"2023-10-06T04:52:18.665605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659272+0000","last_change":"2023-10-06T04:52:18.649861+0000","last_active":"2023-10-06T04:52:18.659272+0000","last_peered":"2023-10-06T04:52:18.659272+0000","last_clean":"2023-10-06T04:52:18.659272+0000","last_became_active":"2023-10-06T04:52:18.649576+0000","last_became_peered":"2023-10-06T04:52:18.649576+0000","last_unstale":"2023-10-06T04:52:18.659272+0000","last_undegraded":"2023-10-06T04:52:18.659272+0000","last_fullsized":"2023-10-06T04:52:18.659272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:14.896487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.657811+0000","last_change":"2023-10-06T04:52:17.653503+0000","last_active":"2023-10-06T04:52:18.657811+0000","last_peered":"2023-10-06T04:52:18.657811+0000","last_clean":"2023-10-06T04:52:18.657811+0000","last_became_active":"2023-10-06T04:52:17.653236+0000","last_became_peered":"2023-10-06T04:52:17.653236+0000","last_unstale":"2023-10-06T04:52:18.657811+0000","last_undegraded":"2023-10-06T04:52:18.657811+0000","last_fullsized":"2023-10-06T04:52:18.657811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:27.560082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664792+0000","last_change":"2023-10-06T04:52:18.661966+0000","last_active":"2023-10-06T04:52:18.664792+0000","last_peered":"2023-10-06T04:52:18.664792+0000","last_clean":"2023-10-06T04:52:18.664792+0000","last_became_active":"2023-10-06T04:52:18.661376+0000","last_became_peered":"2023-10-06T04:52:18.661376+0000","last_unstale":"2023-10-06T04:52:18.664792+0000","last_undegraded":"2023-10-06T04:52:18.664792+0000","last_fullsized":"2023-10-06T04:52:18.664792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643420+0000","last_change":"2023-10-06T04:52:17.650123+0000","last_active":"2023-10-06T04:52:18.643420+0000","last_peered":"2023-10-06T04:52:18.643420+0000","last_clean":"2023-10-06T04:52:18.643420+0000","last_became_active":"2023-10-06T04:52:17.649388+0000","last_became_peered":"2023-10-06T04:52:17.649388+0000","last_unstale":"2023-10-06T04:52:18.643420+0000","last_undegraded":"2023-10-06T04:52:18.643420+0000","last_fullsized":"2023-10-06T04:52:18.643420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650775+0000","last_change":"2023-10-06T04:52:18.647865+0000","last_active":"2023-10-06T04:52:18.650775+0000","last_peered":"2023-10-06T04:52:18.650775+0000","last_clean":"2023-10-06T04:52:18.650775+0000","last_became_active":"2023-10-06T04:52:18.647582+0000","last_became_peered":"2023-10-06T04:52:18.647582+0000","last_unstale":"2023-10-06T04:52:18.650775+0000","last_undegraded":"2023-10-06T04:52:18.650775+0000","last_fullsized":"2023-10-06T04:52:18.650775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665684+0000","last_change":"2023-10-06T04:52:17.651495+0000","last_active":"2023-10-06T04:52:18.665684+0000","last_peered":"2023-10-06T04:52:18.665684+0000","last_clean":"2023-10-06T04:52:18.665684+0000","last_became_active":"2023-10-06T04:52:17.651108+0000","last_became_peered":"2023-10-06T04:52:17.651108+0000","last_unstale":"2023-10-06T04:52:18.665684+0000","last_undegraded":"2023-10-06T04:52:18.665684+0000","last_fullsized":"2023-10-06T04:52:18.665684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666462+0000","last_change":"2023-10-06T04:52:18.662825+0000","last_active":"2023-10-06T04:52:18.666462+0000","last_peered":"2023-10-06T04:52:18.666462+0000","last_clean":"2023-10-06T04:52:18.666462+0000","last_became_active":"2023-10-06T04:52:18.662499+0000","last_became_peered":"2023-10-06T04:52:18.662499+0000","last_unstale":"2023-10-06T04:52:18.666462+0000","last_undegraded":"2023-10-06T04:52:18.666462+0000","last_fullsized":"2023-10-06T04:52:18.666462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:58.798038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646929+0000","last_change":"2023-10-06T04:52:17.657591+0000","last_active":"2023-10-06T04:52:18.646929+0000","last_peered":"2023-10-06T04:52:18.646929+0000","last_clean":"2023-10-06T04:52:18.646929+0000","last_became_active":"2023-10-06T04:52:17.657291+0000","last_became_peered":"2023-10-06T04:52:17.657291+0000","last_unstale":"2023-10-06T04:52:18.646929+0000","last_undegraded":"2023-10-06T04:52:18.646929+0000","last_fullsized":"2023-10-06T04:52:18.646929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659774+0000","last_change":"2023-10-06T04:52:18.650766+0000","last_active":"2023-10-06T04:52:18.659774+0000","last_peered":"2023-10-06T04:52:18.659774+0000","last_clean":"2023-10-06T04:52:18.659774+0000","last_became_active":"2023-10-06T04:52:18.650514+0000","last_became_peered":"2023-10-06T04:52:18.650514+0000","last_unstale":"2023-10-06T04:52:18.659774+0000","last_undegraded":"2023-10-06T04:52:18.659774+0000","last_fullsized":"2023-10-06T04:52:18.659774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.639506+0000","last_change":"2023-10-06T04:52:17.655486+0000","last_active":"2023-10-06T04:52:18.639506+0000","last_peered":"2023-10-06T04:52:18.639506+0000","last_clean":"2023-10-06T04:52:18.639506+0000","last_became_active":"2023-10-06T04:52:17.654231+0000","last_became_peered":"2023-10-06T04:52:17.654231+0000","last_unstale":"2023-10-06T04:52:18.639506+0000","last_undegraded":"2023-10-06T04:52:18.639506+0000","last_fullsized":"2023-10-06T04:52:18.639506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666321+0000","last_change":"2023-10-06T04:52:18.663641+0000","last_active":"2023-10-06T04:52:18.666321+0000","last_peered":"2023-10-06T04:52:18.666321+0000","last_clean":"2023-10-06T04:52:18.666321+0000","last_became_active":"2023-10-06T04:52:18.663334+0000","last_became_peered":"2023-10-06T04:52:18.663334+0000","last_unstale":"2023-10-06T04:52:18.666321+0000","last_undegraded":"2023-10-06T04:52:18.666321+0000","last_fullsized":"2023-10-06T04:52:18.666321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:18.572482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643070+0000","last_change":"2023-10-06T04:52:17.644385+0000","last_active":"2023-10-06T04:52:18.643070+0000","last_peered":"2023-10-06T04:52:18.643070+0000","last_clean":"2023-10-06T04:52:18.643070+0000","last_became_active":"2023-10-06T04:52:17.643911+0000","last_became_peered":"2023-10-06T04:52:17.643911+0000","last_unstale":"2023-10-06T04:52:18.643070+0000","last_undegraded":"2023-10-06T04:52:18.643070+0000","last_fullsized":"2023-10-06T04:52:18.643070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661372+0000","last_change":"2023-10-06T04:52:18.657062+0000","last_active":"2023-10-06T04:52:18.661372+0000","last_peered":"2023-10-06T04:52:18.661372+0000","last_clean":"2023-10-06T04:52:18.661372+0000","last_became_active":"2023-10-06T04:52:18.656805+0000","last_became_peered":"2023-10-06T04:52:18.656805+0000","last_unstale":"2023-10-06T04:52:18.661372+0000","last_undegraded":"2023-10-06T04:52:18.661372+0000","last_fullsized":"2023-10-06T04:52:18.661372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664453+0000","last_change":"2023-10-06T04:52:17.648546+0000","last_active":"2023-10-06T04:52:18.664453+0000","last_peered":"2023-10-06T04:52:18.664453+0000","last_clean":"2023-10-06T04:52:18.664453+0000","last_became_active":"2023-10-06T04:52:17.648247+0000","last_became_peered":"2023-10-06T04:52:17.648247+0000","last_unstale":"2023-10-06T04:52:18.664453+0000","last_undegraded":"2023-10-06T04:52:18.664453+0000","last_fullsized":"2023-10-06T04:52:18.664453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642529+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:52:18.642529+0000","last_peered":"2023-10-06T04:52:18.642529+0000","last_clean":"2023-10-06T04:52:18.642529+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:52:18.642529+0000","last_undegraded":"2023-10-06T04:52:18.642529+0000","last_fullsized":"2023-10-06T04:52:18.642529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659829+0000","last_change":"2023-10-06T04:52:18.654743+0000","last_active":"2023-10-06T04:52:18.659829+0000","last_peered":"2023-10-06T04:52:18.659829+0000","last_clean":"2023-10-06T04:52:18.659829+0000","last_became_active":"2023-10-06T04:52:18.653945+0000","last_became_peered":"2023-10-06T04:52:18.653945+0000","last_unstale":"2023-10-06T04:52:18.659829+0000","last_undegraded":"2023-10-06T04:52:18.659829+0000","last_fullsized":"2023-10-06T04:52:18.659829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642776+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:52:18.642776+0000","last_peered":"2023-10-06T04:52:18.642776+0000","last_clean":"2023-10-06T04:52:18.642776+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:52:18.642776+0000","last_undegraded":"2023-10-06T04:52:18.642776+0000","last_fullsized":"2023-10-06T04:52:18.642776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T04:52:19.637862+0000","last_change":"2023-10-06T04:52:17.653308+0000","last_active":"2023-10-06T04:52:19.637862+0000","last_peered":"2023-10-06T04:52:19.637862+0000","last_clean":"2023-10-06T04:52:19.637862+0000","last_became_active":"2023-10-06T04:52:17.653032+0000","last_became_peered":"2023-10-06T04:52:17.653032+0000","last_unstale":"2023-10-06T04:52:19.637862+0000","last_undegraded":"2023-10-06T04:52:19.637862+0000","last_fullsized":"2023-10-06T04:52:19.637862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665740+0000","last_change":"2023-10-06T04:52:18.656477+0000","last_active":"2023-10-06T04:52:18.665740+0000","last_peered":"2023-10-06T04:52:18.665740+0000","last_clean":"2023-10-06T04:52:18.665740+0000","last_became_active":"2023-10-06T04:52:18.656225+0000","last_became_peered":"2023-10-06T04:52:18.656225+0000","last_unstale":"2023-10-06T04:52:18.665740+0000","last_undegraded":"2023-10-06T04:52:18.665740+0000","last_fullsized":"2023-10-06T04:52:18.665740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643280+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:52:18.643280+0000","last_peered":"2023-10-06T04:52:18.643280+0000","last_clean":"2023-10-06T04:52:18.643280+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:52:18.643280+0000","last_undegraded":"2023-10-06T04:52:18.643280+0000","last_fullsized":"2023-10-06T04:52:18.643280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647019+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:52:18.647019+0000","last_peered":"2023-10-06T04:52:18.647019+0000","last_clean":"2023-10-06T04:52:18.647019+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:52:18.647019+0000","last_undegraded":"2023-10-06T04:52:18.647019+0000","last_fullsized":"2023-10-06T04:52:18.647019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638291+0000","last_change":"2023-10-06T04:52:17.656649+0000","last_active":"2023-10-06T04:52:18.638291+0000","last_peered":"2023-10-06T04:52:18.638291+0000","last_clean":"2023-10-06T04:52:18.638291+0000","last_became_active":"2023-10-06T04:52:17.656336+0000","last_became_peered":"2023-10-06T04:52:17.656336+0000","last_unstale":"2023-10-06T04:52:18.638291+0000","last_undegraded":"2023-10-06T04:52:18.638291+0000","last_fullsized":"2023-10-06T04:52:18.638291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662767+0000","last_change":"2023-10-06T04:52:18.659289+0000","last_active":"2023-10-06T04:52:18.662767+0000","last_peered":"2023-10-06T04:52:18.662767+0000","last_clean":"2023-10-06T04:52:18.662767+0000","last_became_active":"2023-10-06T04:52:18.659024+0000","last_became_peered":"2023-10-06T04:52:18.659024+0000","last_unstale":"2023-10-06T04:52:18.662767+0000","last_undegraded":"2023-10-06T04:52:18.662767+0000","last_fullsized":"2023-10-06T04:52:18.662767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647410+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:52:18.647410+0000","last_peered":"2023-10-06T04:52:18.647410+0000","last_clean":"2023-10-06T04:52:18.647410+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:52:18.647410+0000","last_undegraded":"2023-10-06T04:52:18.647410+0000","last_fullsized":"2023-10-06T04:52:18.647410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646114+0000","last_change":"2023-10-06T04:52:17.648757+0000","last_active":"2023-10-06T04:52:18.646114+0000","last_peered":"2023-10-06T04:52:18.646114+0000","last_clean":"2023-10-06T04:52:18.646114+0000","last_became_active":"2023-10-06T04:52:17.647698+0000","last_became_peered":"2023-10-06T04:52:17.647698+0000","last_unstale":"2023-10-06T04:52:18.646114+0000","last_undegraded":"2023-10-06T04:52:18.646114+0000","last_fullsized":"2023-10-06T04:52:18.646114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.653635+0000","last_change":"2023-10-06T04:52:18.648518+0000","last_active":"2023-10-06T04:52:18.653635+0000","last_peered":"2023-10-06T04:52:18.653635+0000","last_clean":"2023-10-06T04:52:18.653635+0000","last_became_active":"2023-10-06T04:52:18.648172+0000","last_became_peered":"2023-10-06T04:52:18.648172+0000","last_unstale":"2023-10-06T04:52:18.653635+0000","last_undegraded":"2023-10-06T04:52:18.653635+0000","last_fullsized":"2023-10-06T04:52:18.653635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646590+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:52:18.646590+0000","last_peered":"2023-10-06T04:52:18.646590+0000","last_clean":"2023-10-06T04:52:18.646590+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:52:18.646590+0000","last_undegraded":"2023-10-06T04:52:18.646590+0000","last_fullsized":"2023-10-06T04:52:18.646590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642140+0000","last_change":"2023-10-06T04:52:17.647411+0000","last_active":"2023-10-06T04:52:18.642140+0000","last_peered":"2023-10-06T04:52:18.642140+0000","last_clean":"2023-10-06T04:52:18.642140+0000","last_became_active":"2023-10-06T04:52:17.646893+0000","last_became_peered":"2023-10-06T04:52:17.646893+0000","last_unstale":"2023-10-06T04:52:18.642140+0000","last_undegraded":"2023-10-06T04:52:18.642140+0000","last_fullsized":"2023-10-06T04:52:18.642140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662059+0000","last_change":"2023-10-06T04:52:18.655841+0000","last_active":"2023-10-06T04:52:18.662059+0000","last_peered":"2023-10-06T04:52:18.662059+0000","last_clean":"2023-10-06T04:52:18.662059+0000","last_became_active":"2023-10-06T04:52:18.655528+0000","last_became_peered":"2023-10-06T04:52:18.655528+0000","last_unstale":"2023-10-06T04:52:18.662059+0000","last_undegraded":"2023-10-06T04:52:18.662059+0000","last_fullsized":"2023-10-06T04:52:18.662059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646567+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:52:18.646567+0000","last_peered":"2023-10-06T04:52:18.646567+0000","last_clean":"2023-10-06T04:52:18.646567+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:52:18.646567+0000","last_undegraded":"2023-10-06T04:52:18.646567+0000","last_fullsized":"2023-10-06T04:52:18.646567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.641848+0000","last_change":"2023-10-06T04:52:17.655012+0000","last_active":"2023-10-06T04:52:18.641848+0000","last_peered":"2023-10-06T04:52:18.641848+0000","last_clean":"2023-10-06T04:52:18.641848+0000","last_became_active":"2023-10-06T04:52:17.654727+0000","last_became_peered":"2023-10-06T04:52:17.654727+0000","last_unstale":"2023-10-06T04:52:18.641848+0000","last_undegraded":"2023-10-06T04:52:18.641848+0000","last_fullsized":"2023-10-06T04:52:18.641848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.653697+0000","last_change":"2023-10-06T04:52:18.647619+0000","last_active":"2023-10-06T04:52:18.653697+0000","last_peered":"2023-10-06T04:52:18.653697+0000","last_clean":"2023-10-06T04:52:18.653697+0000","last_became_active":"2023-10-06T04:52:18.647303+0000","last_became_peered":"2023-10-06T04:52:18.647303+0000","last_unstale":"2023-10-06T04:52:18.653697+0000","last_undegraded":"2023-10-06T04:52:18.653697+0000","last_fullsized":"2023-10-06T04:52:18.653697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:35.374135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.645972+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:52:18.645972+0000","last_peered":"2023-10-06T04:52:18.645972+0000","last_clean":"2023-10-06T04:52:18.645972+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:52:18.645972+0000","last_undegraded":"2023-10-06T04:52:18.645972+0000","last_fullsized":"2023-10-06T04:52:18.645972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647129+0000","last_change":"2023-10-06T04:52:17.649887+0000","last_active":"2023-10-06T04:52:18.647129+0000","last_peered":"2023-10-06T04:52:18.647129+0000","last_clean":"2023-10-06T04:52:18.647129+0000","last_became_active":"2023-10-06T04:52:17.649537+0000","last_became_peered":"2023-10-06T04:52:17.649537+0000","last_unstale":"2023-10-06T04:52:18.647129+0000","last_undegraded":"2023-10-06T04:52:18.647129+0000","last_fullsized":"2023-10-06T04:52:18.647129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:57.133434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655515+0000","last_change":"2023-10-06T04:52:18.649286+0000","last_active":"2023-10-06T04:52:18.655515+0000","last_peered":"2023-10-06T04:52:18.655515+0000","last_clean":"2023-10-06T04:52:18.655515+0000","last_became_active":"2023-10-06T04:52:18.648947+0000","last_became_peered":"2023-10-06T04:52:18.648947+0000","last_unstale":"2023-10-06T04:52:18.655515+0000","last_undegraded":"2023-10-06T04:52:18.655515+0000","last_fullsized":"2023-10-06T04:52:18.655515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646165+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:52:18.646165+0000","last_peered":"2023-10-06T04:52:18.646165+0000","last_clean":"2023-10-06T04:52:18.646165+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:52:18.646165+0000","last_undegraded":"2023-10-06T04:52:18.646165+0000","last_fullsized":"2023-10-06T04:52:18.646165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647419+0000","last_change":"2023-10-06T04:52:17.650178+0000","last_active":"2023-10-06T04:52:18.647419+0000","last_peered":"2023-10-06T04:52:18.647419+0000","last_clean":"2023-10-06T04:52:18.647419+0000","last_became_active":"2023-10-06T04:52:17.649883+0000","last_became_peered":"2023-10-06T04:52:17.649883+0000","last_unstale":"2023-10-06T04:52:18.647419+0000","last_undegraded":"2023-10-06T04:52:18.647419+0000","last_fullsized":"2023-10-06T04:52:18.647419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661164+0000","last_change":"2023-10-06T04:52:18.658639+0000","last_active":"2023-10-06T04:52:18.661164+0000","last_peered":"2023-10-06T04:52:18.661164+0000","last_clean":"2023-10-06T04:52:18.661164+0000","last_became_active":"2023-10-06T04:52:18.658381+0000","last_became_peered":"2023-10-06T04:52:18.658381+0000","last_unstale":"2023-10-06T04:52:18.661164+0000","last_undegraded":"2023-10-06T04:52:18.661164+0000","last_fullsized":"2023-10-06T04:52:18.661164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:51.836228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665805+0000","last_change":"2023-10-06T04:52:17.657299+0000","last_active":"2023-10-06T04:52:18.665805+0000","last_peered":"2023-10-06T04:52:18.665805+0000","last_clean":"2023-10-06T04:52:18.665805+0000","last_became_active":"2023-10-06T04:52:17.656956+0000","last_became_peered":"2023-10-06T04:52:17.656956+0000","last_unstale":"2023-10-06T04:52:18.665805+0000","last_undegraded":"2023-10-06T04:52:18.665805+0000","last_fullsized":"2023-10-06T04:52:18.665805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659769+0000","last_change":"2023-10-06T04:52:18.652907+0000","last_active":"2023-10-06T04:52:18.659769+0000","last_peered":"2023-10-06T04:52:18.659769+0000","last_clean":"2023-10-06T04:52:18.659769+0000","last_became_active":"2023-10-06T04:52:18.652600+0000","last_became_peered":"2023-10-06T04:52:18.652600+0000","last_unstale":"2023-10-06T04:52:18.659769+0000","last_undegraded":"2023-10-06T04:52:18.659769+0000","last_fullsized":"2023-10-06T04:52:18.659769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:13.950333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667832+0000","last_change":"2023-10-06T04:52:17.656487+0000","last_active":"2023-10-06T04:52:18.667832+0000","last_peered":"2023-10-06T04:52:18.667832+0000","last_clean":"2023-10-06T04:52:18.667832+0000","last_became_active":"2023-10-06T04:52:17.656205+0000","last_became_peered":"2023-10-06T04:52:17.656205+0000","last_unstale":"2023-10-06T04:52:18.667832+0000","last_undegraded":"2023-10-06T04:52:18.667832+0000","last_fullsized":"2023-10-06T04:52:18.667832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665639+0000","last_change":"2023-10-06T04:52:18.654103+0000","last_active":"2023-10-06T04:52:18.665639+0000","last_peered":"2023-10-06T04:52:18.665639+0000","last_clean":"2023-10-06T04:52:18.665639+0000","last_became_active":"2023-10-06T04:52:18.653791+0000","last_became_peered":"2023-10-06T04:52:18.653791+0000","last_unstale":"2023-10-06T04:52:18.665639+0000","last_undegraded":"2023-10-06T04:52:18.665639+0000","last_fullsized":"2023-10-06T04:52:18.665639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:18.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667826+0000","last_change":"2023-10-06T04:52:17.656962+0000","last_active":"2023-10-06T04:52:18.667826+0000","last_peered":"2023-10-06T04:52:18.667826+0000","last_clean":"2023-10-06T04:52:18.667826+0000","last_became_active":"2023-10-06T04:52:17.656699+0000","last_became_peered":"2023-10-06T04:52:17.656699+0000","last_unstale":"2023-10-06T04:52:18.667826+0000","last_undegraded":"2023-10-06T04:52:18.667826+0000","last_fullsized":"2023-10-06T04:52:18.667826+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665510+0000","last_change":"2023-10-06T04:52:18.653480+0000","last_active":"2023-10-06T04:52:18.665510+0000","last_peered":"2023-10-06T04:52:18.665510+0000","last_clean":"2023-10-06T04:52:18.665510+0000","last_became_active":"2023-10-06T04:52:18.653200+0000","last_became_peered":"2023-10-06T04:52:18.653200+0000","last_unstale":"2023-10-06T04:52:18.665510+0000","last_undegraded":"2023-10-06T04:52:18.665510+0000","last_fullsized":"2023-10-06T04:52:18.665510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:51.240487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663382+0000","last_change":"2023-10-06T04:52:17.651435+0000","last_active":"2023-10-06T04:52:18.663382+0000","last_peered":"2023-10-06T04:52:18.663382+0000","last_clean":"2023-10-06T04:52:18.663382+0000","last_became_active":"2023-10-06T04:52:17.651063+0000","last_became_peered":"2023-10-06T04:52:17.651063+0000","last_unstale":"2023-10-06T04:52:18.663382+0000","last_undegraded":"2023-10-06T04:52:18.663382+0000","last_fullsized":"2023-10-06T04:52:18.663382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664028+0000","last_change":"2023-10-06T04:52:18.658488+0000","last_active":"2023-10-06T04:52:18.664028+0000","last_peered":"2023-10-06T04:52:18.664028+0000","last_clean":"2023-10-06T04:52:18.664028+0000","last_became_active":"2023-10-06T04:52:18.657977+0000","last_became_peered":"2023-10-06T04:52:18.657977+0000","last_unstale":"2023-10-06T04:52:18.664028+0000","last_undegraded":"2023-10-06T04:52:18.664028+0000","last_fullsized":"2023-10-06T04:52:18.664028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661377+0000","last_change":"2023-10-06T04:52:17.652321+0000","last_active":"2023-10-06T04:52:18.661377+0000","last_peered":"2023-10-06T04:52:18.661377+0000","last_clean":"2023-10-06T04:52:18.661377+0000","last_became_active":"2023-10-06T04:52:17.651998+0000","last_became_peered":"2023-10-06T04:52:17.651998+0000","last_unstale":"2023-10-06T04:52:18.661377+0000","last_undegraded":"2023-10-06T04:52:18.661377+0000","last_fullsized":"2023-10-06T04:52:18.661377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666345+0000","last_change":"2023-10-06T04:52:18.664061+0000","last_active":"2023-10-06T04:52:18.666345+0000","last_peered":"2023-10-06T04:52:18.666345+0000","last_clean":"2023-10-06T04:52:18.666345+0000","last_became_active":"2023-10-06T04:52:18.663769+0000","last_became_peered":"2023-10-06T04:52:18.663769+0000","last_unstale":"2023-10-06T04:52:18.666345+0000","last_undegraded":"2023-10-06T04:52:18.666345+0000","last_fullsized":"2023-10-06T04:52:18.666345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664651+0000","last_change":"2023-10-06T04:52:18.662900+0000","last_active":"2023-10-06T04:52:18.664651+0000","last_peered":"2023-10-06T04:52:18.664651+0000","last_clean":"2023-10-06T04:52:18.664651+0000","last_became_active":"2023-10-06T04:52:18.662558+0000","last_became_peered":"2023-10-06T04:52:18.662558+0000","last_unstale":"2023-10-06T04:52:18.664651+0000","last_undegraded":"2023-10-06T04:52:18.664651+0000","last_fullsized":"2023-10-06T04:52:18.664651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:18.313966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664001+0000","last_change":"2023-10-06T04:52:18.658296+0000","last_active":"2023-10-06T04:52:18.664001+0000","last_peered":"2023-10-06T04:52:18.664001+0000","last_clean":"2023-10-06T04:52:18.664001+0000","last_became_active":"2023-10-06T04:52:18.657589+0000","last_became_peered":"2023-10-06T04:52:18.657589+0000","last_unstale":"2023-10-06T04:52:18.664001+0000","last_undegraded":"2023-10-06T04:52:18.664001+0000","last_fullsized":"2023-10-06T04:52:18.664001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666380+0000","last_change":"2023-10-06T04:52:18.663851+0000","last_active":"2023-10-06T04:52:18.666380+0000","last_peered":"2023-10-06T04:52:18.666380+0000","last_clean":"2023-10-06T04:52:18.666380+0000","last_became_active":"2023-10-06T04:52:18.663574+0000","last_became_peered":"2023-10-06T04:52:18.663574+0000","last_unstale":"2023-10-06T04:52:18.666380+0000","last_undegraded":"2023-10-06T04:52:18.666380+0000","last_fullsized":"2023-10-06T04:52:18.666380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:54.373475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656769+0000","last_change":"2023-10-06T04:52:18.652292+0000","last_active":"2023-10-06T04:52:18.656769+0000","last_peered":"2023-10-06T04:52:18.656769+0000","last_clean":"2023-10-06T04:52:18.656769+0000","last_became_active":"2023-10-06T04:52:18.651919+0000","last_became_peered":"2023-10-06T04:52:18.651919+0000","last_unstale":"2023-10-06T04:52:18.656769+0000","last_undegraded":"2023-10-06T04:52:18.656769+0000","last_fullsized":"2023-10-06T04:52:18.656769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.657972+0000","last_change":"2023-10-06T04:52:18.650895+0000","last_active":"2023-10-06T04:52:18.657972+0000","last_peered":"2023-10-06T04:52:18.657972+0000","last_clean":"2023-10-06T04:52:18.657972+0000","last_became_active":"2023-10-06T04:52:18.650623+0000","last_became_peered":"2023-10-06T04:52:18.650623+0000","last_unstale":"2023-10-06T04:52:18.657972+0000","last_undegraded":"2023-10-06T04:52:18.657972+0000","last_fullsized":"2023-10-06T04:52:18.657972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":664951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298064,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073776,"statfs":{"total":96636764160,"available":96331546624,"internally_reserved":0,"allocated":794624,"data_stored":663199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":73455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:52:27.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:52:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.570+0000 7f6102c98700 1 Processor -- start 2023-10-06T04:52:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.570+0000 7f6102c98700 1 -- start start 2023-10-06T04:52:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f6102c98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60fc0cd950 0x7f60fc0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f6102c98700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0b2740 0x7f60fc0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f6102c98700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60fc0b0ce0 con 0x7f60fc15cb50 2023-10-06T04:52:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f6102c98700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc0cb820 con 0x7f60fc0cd950 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f6102c98700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b0760 con 0x7f60fc0b2740 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f60fbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60fc0cd950 0x7f60fc0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f60fbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60fc0cd950 0x7f60fc0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33270/0 (socket says 172.21.15.3:33270) 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.571+0000 7f60fbfff700 1 -- 172.21.15.3:0/193706135 learned_addr learned my addr 172.21.15.3:0/193706135 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6101235700 1 --2- 172.21.15.3:0/193706135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0b2740 0x7f60fc0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fbfff700 1 -- 172.21.15.3:0/193706135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0b2740 msgr2=0x7f60fc0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fbfff700 1 --2- 172.21.15.3:0/193706135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0b2740 0x7f60fc0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fbfff700 1 -- 172.21.15.3:0/193706135 >> v1:172.21.15.153:6789/0 conn(0x7f60fc15cb50 legacy=0x7f60fc0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fbfff700 1 -- 172.21.15.3:0/193706135 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc0b0a00 con 0x7f60fc0cd950 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6101235700 1 --2- 172.21.15.3:0/193706135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0b2740 0x7f60fc0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fbfff700 1 --2- 172.21.15.3:0/193706135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0cd950 0x7f60fc0b2200 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0099b0 tx=0x7f60ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445583b88320c986 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fb7fe700 1 -- 172.21.15.3:0/193706135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00b820 con 0x7f60fc0cd950 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fb7fe700 1 -- 172.21.15.3:0/193706135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec00b980 con 0x7f60fc0cd950 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f60fb7fe700 1 -- 172.21.15.3:0/193706135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00bd20 con 0x7f60fc0cd950 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6102c98700 1 -- 172.21.15.3:0/193706135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0cd950 msgr2=0x7f60fc0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6102c98700 1 --2- 172.21.15.3:0/193706135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0cd950 0x7f60fc0b2200 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0099b0 tx=0x7f60ec009250 comp rx=0 tx=0).stop 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6102c98700 1 -- 172.21.15.3:0/193706135 shutdown_connections 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6102c98700 1 --2- 172.21.15.3:0/193706135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0b2740 0x7f60fc0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6102c98700 1 --2- 172.21.15.3:0/193706135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0cd950 0x7f60fc0b2200 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6102c98700 1 -- 172.21.15.3:0/193706135 >> 172.21.15.3:0/193706135 conn(0x7f60fc070310 msgr2=0x7f60fc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.572+0000 7f6102c98700 1 -- 172.21.15.3:0/193706135 shutdown_connections 2023-10-06T04:52:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.573+0000 7f6102c98700 1 -- 172.21.15.3:0/193706135 wait complete. 2023-10-06T04:52:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 Processor -- start 2023-10-06T04:52:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 -- start start 2023-10-06T04:52:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0b2740 0x7f60fc14ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0cd950 0x7f60fc0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60fc15cb50 0x7f60fc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0c5850 con 0x7f60fc0b2740 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0c5b30 con 0x7f60fc15cb50 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6102c98700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0c5e10 con 0x7f60fc0cd950 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.574+0000 7f6100a34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0b2740 0x7f60fc14ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0b2740 0x7f60fc14ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33280/0 (socket says 172.21.15.3:33280) 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 -- 172.21.15.3:0/1653139342 learned_addr learned my addr 172.21.15.3:0/1653139342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f60fbfff700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0cd950 0x7f60fc0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6101235700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60fc15cb50 0x7f60fc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 -- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0cd950 msgr2=0x7f60fc0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0cd950 0x7f60fc0c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 -- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60fc15cb50 msgr2=0x7f60fc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60fc15cb50 0x7f60fc0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 -- 172.21.15.3:0/1653139342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0094f0 con 0x7f60fc0b2740 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6100a34700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0b2740 0x7f60fc14ca50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60f400bc30 tx=0x7f60f4009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f60f8ff9700 1 -- 172.21.15.3:0/1653139342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f400c9b0 con 0x7f60fc0b2740 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f60f8ff9700 1 -- 172.21.15.3:0/1653139342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f400cb50 con 0x7f60fc0b2740 2023-10-06T04:52:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f60f8ff9700 1 -- 172.21.15.3:0/1653139342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f4014410 con 0x7f60fc0b2740 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc0c60f0 con 0x7f60fc0b2740 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.575+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc0c51c0 con 0x7f60fc0b2740 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.576+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc071680 con 0x7f60fc0b2740 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.577+0000 7f60f8ff9700 1 -- 172.21.15.3:0/1653139342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60f4014800 con 0x7f60fc0b2740 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.577+0000 7f60f8ff9700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60dc066f30 0x7f60dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.577+0000 7f60f8ff9700 1 -- 172.21.15.3:0/1653139342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f60f4084820 con 0x7f60fc0b2740 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.577+0000 7f60fbfff700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60dc066f30 0x7f60dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.578+0000 7f60fbfff700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60dc066f30 0x7f60dc0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f60fc089980 tx=0x7f60ec016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.581+0000 7f60f8ff9700 1 -- 172.21.15.3:0/1653139342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60f4051930 con 0x7f60fc0b2740 2023-10-06T04:52:27.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.772+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60fc0c5ca0 con 0x7f60dc066f30 2023-10-06T04:52:27.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.780+0000 7f60f8ff9700 1 -- 172.21.15.3:0/1653139342 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286708 (secure 0 0 0) 0x7f60fc0c5ca0 con 0x7f60dc066f30 2023-10-06T04:52:27.785 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.783+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60dc066f30 msgr2=0x7f60dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.783+0000 7f6102c98700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60dc066f30 0x7f60dc0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f60fc089980 tx=0x7f60ec016000 comp rx=0 tx=0).stop 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.783+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0b2740 msgr2=0x7f60fc14ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.783+0000 7f6102c98700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0b2740 0x7f60fc14ca50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60f400bc30 tx=0x7f60f4009820 comp rx=0 tx=0).stop 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 shutdown_connections 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60dc066f30 0x7f60dc0693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60fc15cb50 0x7f60fc0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60fc0b2740 0x7f60fc14ca50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 --2- 172.21.15.3:0/1653139342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60fc0cd950 0x7f60fc0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 >> 172.21.15.3:0/1653139342 conn(0x7f60fc070310 msgr2=0x7f60fc15e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 shutdown_connections 2023-10-06T04:52:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.784+0000 7f6102c98700 1 -- 172.21.15.3:0/1653139342 wait complete. 2023-10-06T04:52:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:52:27.797 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T04:52:27.425410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381116,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593604,"statfs":{"total":773094113280,"available":770655850496,"internally_reserved":0,"allocated":2879488,"data_stored":1774761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.788005"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663223+0000","last_change":"2023-10-06T04:52:18.658025+0000","last_active":"2023-10-06T04:52:18.663223+0000","last_peered":"2023-10-06T04:52:18.663223+0000","last_clean":"2023-10-06T04:52:18.663223+0000","last_became_active":"2023-10-06T04:52:18.657724+0000","last_became_peered":"2023-10-06T04:52:18.657724+0000","last_unstale":"2023-10-06T04:52:18.663223+0000","last_undegraded":"2023-10-06T04:52:18.663223+0000","last_fullsized":"2023-10-06T04:52:18.663223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:01.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661205+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.661205+0000","last_peered":"2023-10-06T04:52:18.661205+0000","last_clean":"2023-10-06T04:52:18.661205+0000","last_became_active":"2023-10-06T04:52:18.658286+0000","last_became_peered":"2023-10-06T04:52:18.658286+0000","last_unstale":"2023-10-06T04:52:18.661205+0000","last_undegraded":"2023-10-06T04:52:18.661205+0000","last_fullsized":"2023-10-06T04:52:18.661205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665708+0000","last_change":"2023-10-06T04:52:18.652791+0000","last_active":"2023-10-06T04:52:18.665708+0000","last_peered":"2023-10-06T04:52:18.665708+0000","last_clean":"2023-10-06T04:52:18.665708+0000","last_became_active":"2023-10-06T04:52:18.652522+0000","last_became_peered":"2023-10-06T04:52:18.652522+0000","last_unstale":"2023-10-06T04:52:18.665708+0000","last_undegraded":"2023-10-06T04:52:18.665708+0000","last_fullsized":"2023-10-06T04:52:18.665708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662069+0000","last_change":"2023-10-06T04:52:18.649578+0000","last_active":"2023-10-06T04:52:18.662069+0000","last_peered":"2023-10-06T04:52:18.662069+0000","last_clean":"2023-10-06T04:52:18.662069+0000","last_became_active":"2023-10-06T04:52:18.649294+0000","last_became_peered":"2023-10-06T04:52:18.649294+0000","last_unstale":"2023-10-06T04:52:18.662069+0000","last_undegraded":"2023-10-06T04:52:18.662069+0000","last_fullsized":"2023-10-06T04:52:18.662069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:28.571226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658392+0000","last_change":"2023-10-06T04:52:18.653657+0000","last_active":"2023-10-06T04:52:18.658392+0000","last_peered":"2023-10-06T04:52:18.658392+0000","last_clean":"2023-10-06T04:52:18.658392+0000","last_became_active":"2023-10-06T04:52:18.653363+0000","last_became_peered":"2023-10-06T04:52:18.653363+0000","last_unstale":"2023-10-06T04:52:18.658392+0000","last_undegraded":"2023-10-06T04:52:18.658392+0000","last_fullsized":"2023-10-06T04:52:18.658392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665600+0000","last_change":"2023-10-06T04:52:18.654512+0000","last_active":"2023-10-06T04:52:18.665600+0000","last_peered":"2023-10-06T04:52:18.665600+0000","last_clean":"2023-10-06T04:52:18.665600+0000","last_became_active":"2023-10-06T04:52:18.653897+0000","last_became_peered":"2023-10-06T04:52:18.653897+0000","last_unstale":"2023-10-06T04:52:18.665600+0000","last_undegraded":"2023-10-06T04:52:18.665600+0000","last_fullsized":"2023-10-06T04:52:18.665600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:38.713099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663473+0000","last_change":"2023-10-06T04:52:18.655144+0000","last_active":"2023-10-06T04:52:18.663473+0000","last_peered":"2023-10-06T04:52:18.663473+0000","last_clean":"2023-10-06T04:52:18.663473+0000","last_became_active":"2023-10-06T04:52:18.654891+0000","last_became_peered":"2023-10-06T04:52:18.654891+0000","last_unstale":"2023-10-06T04:52:18.663473+0000","last_undegraded":"2023-10-06T04:52:18.663473+0000","last_fullsized":"2023-10-06T04:52:18.663473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667181+0000","last_change":"2023-10-06T04:52:18.666282+0000","last_active":"2023-10-06T04:52:18.667181+0000","last_peered":"2023-10-06T04:52:18.667181+0000","last_clean":"2023-10-06T04:52:18.667181+0000","last_became_active":"2023-10-06T04:52:18.665988+0000","last_became_peered":"2023-10-06T04:52:18.665988+0000","last_unstale":"2023-10-06T04:52:18.667181+0000","last_undegraded":"2023-10-06T04:52:18.667181+0000","last_fullsized":"2023-10-06T04:52:18.667181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665701+0000","last_change":"2023-10-06T04:52:18.661193+0000","last_active":"2023-10-06T04:52:18.665701+0000","last_peered":"2023-10-06T04:52:18.665701+0000","last_clean":"2023-10-06T04:52:18.665701+0000","last_became_active":"2023-10-06T04:52:18.660905+0000","last_became_peered":"2023-10-06T04:52:18.660905+0000","last_unstale":"2023-10-06T04:52:18.665701+0000","last_undegraded":"2023-10-06T04:52:18.665701+0000","last_fullsized":"2023-10-06T04:52:18.665701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656410+0000","last_change":"2023-10-06T04:52:18.654460+0000","last_active":"2023-10-06T04:52:18.656410+0000","last_peered":"2023-10-06T04:52:18.656410+0000","last_clean":"2023-10-06T04:52:18.656410+0000","last_became_active":"2023-10-06T04:52:18.653370+0000","last_became_peered":"2023-10-06T04:52:18.653370+0000","last_unstale":"2023-10-06T04:52:18.656410+0000","last_undegraded":"2023-10-06T04:52:18.656410+0000","last_fullsized":"2023-10-06T04:52:18.656410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659795+0000","last_change":"2023-10-06T04:52:18.650974+0000","last_active":"2023-10-06T04:52:18.659795+0000","last_peered":"2023-10-06T04:52:18.659795+0000","last_clean":"2023-10-06T04:52:18.659795+0000","last_became_active":"2023-10-06T04:52:18.650708+0000","last_became_peered":"2023-10-06T04:52:18.650708+0000","last_unstale":"2023-10-06T04:52:18.659795+0000","last_undegraded":"2023-10-06T04:52:18.659795+0000","last_fullsized":"2023-10-06T04:52:18.659795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659746+0000","last_change":"2023-10-06T04:52:18.651167+0000","last_active":"2023-10-06T04:52:18.659746+0000","last_peered":"2023-10-06T04:52:18.659746+0000","last_clean":"2023-10-06T04:52:18.659746+0000","last_became_active":"2023-10-06T04:52:18.650869+0000","last_became_peered":"2023-10-06T04:52:18.650869+0000","last_unstale":"2023-10-06T04:52:18.659746+0000","last_undegraded":"2023-10-06T04:52:18.659746+0000","last_fullsized":"2023-10-06T04:52:18.659746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:55.372045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655957+0000","last_change":"2023-10-06T04:52:18.653721+0000","last_active":"2023-10-06T04:52:18.655957+0000","last_peered":"2023-10-06T04:52:18.655957+0000","last_clean":"2023-10-06T04:52:18.655957+0000","last_became_active":"2023-10-06T04:52:18.653435+0000","last_became_peered":"2023-10-06T04:52:18.653435+0000","last_unstale":"2023-10-06T04:52:18.655957+0000","last_undegraded":"2023-10-06T04:52:18.655957+0000","last_fullsized":"2023-10-06T04:52:18.655957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:43.201875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656748+0000","last_change":"2023-10-06T04:52:18.655632+0000","last_active":"2023-10-06T04:52:18.656748+0000","last_peered":"2023-10-06T04:52:18.656748+0000","last_clean":"2023-10-06T04:52:18.656748+0000","last_became_active":"2023-10-06T04:52:18.655335+0000","last_became_peered":"2023-10-06T04:52:18.655335+0000","last_unstale":"2023-10-06T04:52:18.656748+0000","last_undegraded":"2023-10-06T04:52:18.656748+0000","last_fullsized":"2023-10-06T04:52:18.656748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665013+0000","last_change":"2023-10-06T04:52:18.663599+0000","last_active":"2023-10-06T04:52:18.665013+0000","last_peered":"2023-10-06T04:52:18.665013+0000","last_clean":"2023-10-06T04:52:18.665013+0000","last_became_active":"2023-10-06T04:52:18.663303+0000","last_became_peered":"2023-10-06T04:52:18.663303+0000","last_unstale":"2023-10-06T04:52:18.665013+0000","last_undegraded":"2023-10-06T04:52:18.665013+0000","last_fullsized":"2023-10-06T04:52:18.665013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665493+0000","last_change":"2023-10-06T04:52:18.652910+0000","last_active":"2023-10-06T04:52:18.665493+0000","last_peered":"2023-10-06T04:52:18.665493+0000","last_clean":"2023-10-06T04:52:18.665493+0000","last_became_active":"2023-10-06T04:52:18.652589+0000","last_became_peered":"2023-10-06T04:52:18.652589+0000","last_unstale":"2023-10-06T04:52:18.665493+0000","last_undegraded":"2023-10-06T04:52:18.665493+0000","last_fullsized":"2023-10-06T04:52:18.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:10.947807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.658434+0000","last_change":"2023-10-06T04:52:18.654868+0000","last_active":"2023-10-06T04:52:18.658434+0000","last_peered":"2023-10-06T04:52:18.658434+0000","last_clean":"2023-10-06T04:52:18.658434+0000","last_became_active":"2023-10-06T04:52:18.654590+0000","last_became_peered":"2023-10-06T04:52:18.654590+0000","last_unstale":"2023-10-06T04:52:18.658434+0000","last_undegraded":"2023-10-06T04:52:18.658434+0000","last_fullsized":"2023-10-06T04:52:18.658434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665816+0000","last_change":"2023-10-06T04:52:18.658607+0000","last_active":"2023-10-06T04:52:18.665816+0000","last_peered":"2023-10-06T04:52:18.665816+0000","last_clean":"2023-10-06T04:52:18.665816+0000","last_became_active":"2023-10-06T04:52:18.658358+0000","last_became_peered":"2023-10-06T04:52:18.658358+0000","last_unstale":"2023-10-06T04:52:18.665816+0000","last_undegraded":"2023-10-06T04:52:18.665816+0000","last_fullsized":"2023-10-06T04:52:18.665816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665491+0000","last_change":"2023-10-06T04:52:18.662801+0000","last_active":"2023-10-06T04:52:18.665491+0000","last_peered":"2023-10-06T04:52:18.665491+0000","last_clean":"2023-10-06T04:52:18.665491+0000","last_became_active":"2023-10-06T04:52:18.662534+0000","last_became_peered":"2023-10-06T04:52:18.662534+0000","last_unstale":"2023-10-06T04:52:18.665491+0000","last_undegraded":"2023-10-06T04:52:18.665491+0000","last_fullsized":"2023-10-06T04:52:18.665491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:53.970289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666479+0000","last_change":"2023-10-06T04:52:18.664643+0000","last_active":"2023-10-06T04:52:18.666479+0000","last_peered":"2023-10-06T04:52:18.666479+0000","last_clean":"2023-10-06T04:52:18.666479+0000","last_became_active":"2023-10-06T04:52:18.664342+0000","last_became_peered":"2023-10-06T04:52:18.664342+0000","last_unstale":"2023-10-06T04:52:18.666479+0000","last_undegraded":"2023-10-06T04:52:18.666479+0000","last_fullsized":"2023-10-06T04:52:18.666479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:33.202071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664983+0000","last_change":"2023-10-06T04:52:18.663114+0000","last_active":"2023-10-06T04:52:18.664983+0000","last_peered":"2023-10-06T04:52:18.664983+0000","last_clean":"2023-10-06T04:52:18.664983+0000","last_became_active":"2023-10-06T04:52:18.662825+0000","last_became_peered":"2023-10-06T04:52:18.662825+0000","last_unstale":"2023-10-06T04:52:18.664983+0000","last_undegraded":"2023-10-06T04:52:18.664983+0000","last_fullsized":"2023-10-06T04:52:18.664983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650717+0000","last_change":"2023-10-06T04:52:18.648283+0000","last_active":"2023-10-06T04:52:18.650717+0000","last_peered":"2023-10-06T04:52:18.650717+0000","last_clean":"2023-10-06T04:52:18.650717+0000","last_became_active":"2023-10-06T04:52:18.647966+0000","last_became_peered":"2023-10-06T04:52:18.647966+0000","last_unstale":"2023-10-06T04:52:18.650717+0000","last_undegraded":"2023-10-06T04:52:18.650717+0000","last_fullsized":"2023-10-06T04:52:18.650717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661203+0000","last_change":"2023-10-06T04:52:18.655500+0000","last_active":"2023-10-06T04:52:18.661203+0000","last_peered":"2023-10-06T04:52:18.661203+0000","last_clean":"2023-10-06T04:52:18.661203+0000","last_became_active":"2023-10-06T04:52:18.655234+0000","last_became_peered":"2023-10-06T04:52:18.655234+0000","last_unstale":"2023-10-06T04:52:18.661203+0000","last_undegraded":"2023-10-06T04:52:18.661203+0000","last_fullsized":"2023-10-06T04:52:18.661203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663966+0000","last_change":"2023-10-06T04:52:18.660815+0000","last_active":"2023-10-06T04:52:18.663966+0000","last_peered":"2023-10-06T04:52:18.663966+0000","last_clean":"2023-10-06T04:52:18.663966+0000","last_became_active":"2023-10-06T04:52:18.660475+0000","last_became_peered":"2023-10-06T04:52:18.660475+0000","last_unstale":"2023-10-06T04:52:18.663966+0000","last_undegraded":"2023-10-06T04:52:18.663966+0000","last_fullsized":"2023-10-06T04:52:18.663966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667741+0000","last_change":"2023-10-06T04:52:18.663035+0000","last_active":"2023-10-06T04:52:18.667741+0000","last_peered":"2023-10-06T04:52:18.667741+0000","last_clean":"2023-10-06T04:52:18.667741+0000","last_became_active":"2023-10-06T04:52:18.662611+0000","last_became_peered":"2023-10-06T04:52:18.662611+0000","last_unstale":"2023-10-06T04:52:18.667741+0000","last_undegraded":"2023-10-06T04:52:18.667741+0000","last_fullsized":"2023-10-06T04:52:18.667741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661242+0000","last_change":"2023-10-06T04:52:18.658476+0000","last_active":"2023-10-06T04:52:18.661242+0000","last_peered":"2023-10-06T04:52:18.661242+0000","last_clean":"2023-10-06T04:52:18.661242+0000","last_became_active":"2023-10-06T04:52:18.658197+0000","last_became_peered":"2023-10-06T04:52:18.658197+0000","last_unstale":"2023-10-06T04:52:18.661242+0000","last_undegraded":"2023-10-06T04:52:18.661242+0000","last_fullsized":"2023-10-06T04:52:18.661242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:47.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661137+0000","last_change":"2023-10-06T04:52:18.658955+0000","last_active":"2023-10-06T04:52:18.661137+0000","last_peered":"2023-10-06T04:52:18.661137+0000","last_clean":"2023-10-06T04:52:18.661137+0000","last_became_active":"2023-10-06T04:52:18.658713+0000","last_became_peered":"2023-10-06T04:52:18.658713+0000","last_unstale":"2023-10-06T04:52:18.661137+0000","last_undegraded":"2023-10-06T04:52:18.661137+0000","last_fullsized":"2023-10-06T04:52:18.661137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:59.556355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.651332+0000","last_change":"2023-10-06T04:52:17.656516+0000","last_active":"2023-10-06T04:52:18.651332+0000","last_peered":"2023-10-06T04:52:18.651332+0000","last_clean":"2023-10-06T04:52:18.651332+0000","last_became_active":"2023-10-06T04:52:17.656032+0000","last_became_peered":"2023-10-06T04:52:17.656032+0000","last_unstale":"2023-10-06T04:52:18.651332+0000","last_undegraded":"2023-10-06T04:52:18.651332+0000","last_fullsized":"2023-10-06T04:52:18.651332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660104+0000","last_change":"2023-10-06T04:52:18.656019+0000","last_active":"2023-10-06T04:52:18.660104+0000","last_peered":"2023-10-06T04:52:18.660104+0000","last_clean":"2023-10-06T04:52:18.660104+0000","last_became_active":"2023-10-06T04:52:18.655673+0000","last_became_peered":"2023-10-06T04:52:18.655673+0000","last_unstale":"2023-10-06T04:52:18.660104+0000","last_undegraded":"2023-10-06T04:52:18.660104+0000","last_fullsized":"2023-10-06T04:52:18.660104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643759+0000","last_change":"2023-10-06T04:52:17.645852+0000","last_active":"2023-10-06T04:52:18.643759+0000","last_peered":"2023-10-06T04:52:18.643759+0000","last_clean":"2023-10-06T04:52:18.643759+0000","last_became_active":"2023-10-06T04:52:17.645528+0000","last_became_peered":"2023-10-06T04:52:17.645528+0000","last_unstale":"2023-10-06T04:52:18.643759+0000","last_undegraded":"2023-10-06T04:52:18.643759+0000","last_fullsized":"2023-10-06T04:52:18.643759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655629+0000","last_change":"2023-10-06T04:52:18.649691+0000","last_active":"2023-10-06T04:52:18.655629+0000","last_peered":"2023-10-06T04:52:18.655629+0000","last_clean":"2023-10-06T04:52:18.655629+0000","last_became_active":"2023-10-06T04:52:18.649348+0000","last_became_peered":"2023-10-06T04:52:18.649348+0000","last_unstale":"2023-10-06T04:52:18.655629+0000","last_undegraded":"2023-10-06T04:52:18.655629+0000","last_fullsized":"2023-10-06T04:52:18.655629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643218+0000","last_change":"2023-10-06T04:52:17.651301+0000","last_active":"2023-10-06T04:52:18.643218+0000","last_peered":"2023-10-06T04:52:18.643218+0000","last_clean":"2023-10-06T04:52:18.643218+0000","last_became_active":"2023-10-06T04:52:17.650967+0000","last_became_peered":"2023-10-06T04:52:17.650967+0000","last_unstale":"2023-10-06T04:52:18.643218+0000","last_undegraded":"2023-10-06T04:52:18.643218+0000","last_fullsized":"2023-10-06T04:52:18.643218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662097+0000","last_change":"2023-10-06T04:52:18.656399+0000","last_active":"2023-10-06T04:52:18.662097+0000","last_peered":"2023-10-06T04:52:18.662097+0000","last_clean":"2023-10-06T04:52:18.662097+0000","last_became_active":"2023-10-06T04:52:18.656106+0000","last_became_peered":"2023-10-06T04:52:18.656106+0000","last_unstale":"2023-10-06T04:52:18.662097+0000","last_undegraded":"2023-10-06T04:52:18.662097+0000","last_fullsized":"2023-10-06T04:52:18.662097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.640098+0000","last_change":"2023-10-06T04:52:17.654663+0000","last_active":"2023-10-06T04:52:18.640098+0000","last_peered":"2023-10-06T04:52:18.640098+0000","last_clean":"2023-10-06T04:52:18.640098+0000","last_became_active":"2023-10-06T04:52:17.654402+0000","last_became_peered":"2023-10-06T04:52:17.654402+0000","last_unstale":"2023-10-06T04:52:18.640098+0000","last_undegraded":"2023-10-06T04:52:18.640098+0000","last_fullsized":"2023-10-06T04:52:18.640098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660162+0000","last_change":"2023-10-06T04:52:18.655602+0000","last_active":"2023-10-06T04:52:18.660162+0000","last_peered":"2023-10-06T04:52:18.660162+0000","last_clean":"2023-10-06T04:52:18.660162+0000","last_became_active":"2023-10-06T04:52:18.655329+0000","last_became_peered":"2023-10-06T04:52:18.655329+0000","last_unstale":"2023-10-06T04:52:18.660162+0000","last_undegraded":"2023-10-06T04:52:18.660162+0000","last_fullsized":"2023-10-06T04:52:18.660162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647360+0000","last_change":"2023-10-06T04:52:17.652631+0000","last_active":"2023-10-06T04:52:18.647360+0000","last_peered":"2023-10-06T04:52:18.647360+0000","last_clean":"2023-10-06T04:52:18.647360+0000","last_became_active":"2023-10-06T04:52:17.652340+0000","last_became_peered":"2023-10-06T04:52:17.652340+0000","last_unstale":"2023-10-06T04:52:18.647360+0000","last_undegraded":"2023-10-06T04:52:18.647360+0000","last_fullsized":"2023-10-06T04:52:18.647360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659061+0000","last_change":"2023-10-06T04:52:18.651950+0000","last_active":"2023-10-06T04:52:18.659061+0000","last_peered":"2023-10-06T04:52:18.659061+0000","last_clean":"2023-10-06T04:52:18.659061+0000","last_became_active":"2023-10-06T04:52:18.651587+0000","last_became_peered":"2023-10-06T04:52:18.651587+0000","last_unstale":"2023-10-06T04:52:18.659061+0000","last_undegraded":"2023-10-06T04:52:18.659061+0000","last_fullsized":"2023-10-06T04:52:18.659061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667446+0000","last_change":"2023-10-06T04:52:17.655489+0000","last_active":"2023-10-06T04:52:18.667446+0000","last_peered":"2023-10-06T04:52:18.667446+0000","last_clean":"2023-10-06T04:52:18.667446+0000","last_became_active":"2023-10-06T04:52:17.655131+0000","last_became_peered":"2023-10-06T04:52:17.655131+0000","last_unstale":"2023-10-06T04:52:18.667446+0000","last_undegraded":"2023-10-06T04:52:18.667446+0000","last_fullsized":"2023-10-06T04:52:18.667446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667757+0000","last_change":"2023-10-06T04:52:18.662999+0000","last_active":"2023-10-06T04:52:18.667757+0000","last_peered":"2023-10-06T04:52:18.667757+0000","last_clean":"2023-10-06T04:52:18.667757+0000","last_became_active":"2023-10-06T04:52:18.662603+0000","last_became_peered":"2023-10-06T04:52:18.662603+0000","last_unstale":"2023-10-06T04:52:18.667757+0000","last_undegraded":"2023-10-06T04:52:18.667757+0000","last_fullsized":"2023-10-06T04:52:18.667757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.669196+0000","last_change":"2023-10-06T04:52:17.655293+0000","last_active":"2023-10-06T04:52:18.669196+0000","last_peered":"2023-10-06T04:52:18.669196+0000","last_clean":"2023-10-06T04:52:18.669196+0000","last_became_active":"2023-10-06T04:52:17.654390+0000","last_became_peered":"2023-10-06T04:52:17.654390+0000","last_unstale":"2023-10-06T04:52:18.669196+0000","last_undegraded":"2023-10-06T04:52:18.669196+0000","last_fullsized":"2023-10-06T04:52:18.669196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665827+0000","last_change":"2023-10-06T04:52:18.657322+0000","last_active":"2023-10-06T04:52:18.665827+0000","last_peered":"2023-10-06T04:52:18.665827+0000","last_clean":"2023-10-06T04:52:18.665827+0000","last_became_active":"2023-10-06T04:52:18.657046+0000","last_became_peered":"2023-10-06T04:52:18.657046+0000","last_unstale":"2023-10-06T04:52:18.665827+0000","last_undegraded":"2023-10-06T04:52:18.665827+0000","last_fullsized":"2023-10-06T04:52:18.665827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:28.916531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659855+0000","last_change":"2023-10-06T04:52:17.657213+0000","last_active":"2023-10-06T04:52:18.659855+0000","last_peered":"2023-10-06T04:52:18.659855+0000","last_clean":"2023-10-06T04:52:18.659855+0000","last_became_active":"2023-10-06T04:52:17.656945+0000","last_became_peered":"2023-10-06T04:52:17.656945+0000","last_unstale":"2023-10-06T04:52:18.659855+0000","last_undegraded":"2023-10-06T04:52:18.659855+0000","last_fullsized":"2023-10-06T04:52:18.659855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:17.302823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659796+0000","last_change":"2023-10-06T04:52:18.655013+0000","last_active":"2023-10-06T04:52:18.659796+0000","last_peered":"2023-10-06T04:52:18.659796+0000","last_clean":"2023-10-06T04:52:18.659796+0000","last_became_active":"2023-10-06T04:52:18.654734+0000","last_became_peered":"2023-10-06T04:52:18.654734+0000","last_unstale":"2023-10-06T04:52:18.659796+0000","last_undegraded":"2023-10-06T04:52:18.659796+0000","last_fullsized":"2023-10-06T04:52:18.659796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638374+0000","last_change":"2023-10-06T04:52:17.650696+0000","last_active":"2023-10-06T04:52:18.638374+0000","last_peered":"2023-10-06T04:52:18.638374+0000","last_clean":"2023-10-06T04:52:18.638374+0000","last_became_active":"2023-10-06T04:52:17.650402+0000","last_became_peered":"2023-10-06T04:52:17.650402+0000","last_unstale":"2023-10-06T04:52:18.638374+0000","last_undegraded":"2023-10-06T04:52:18.638374+0000","last_fullsized":"2023-10-06T04:52:18.638374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661155+0000","last_change":"2023-10-06T04:52:18.658995+0000","last_active":"2023-10-06T04:52:18.661155+0000","last_peered":"2023-10-06T04:52:18.661155+0000","last_clean":"2023-10-06T04:52:18.661155+0000","last_became_active":"2023-10-06T04:52:18.658717+0000","last_became_peered":"2023-10-06T04:52:18.658717+0000","last_unstale":"2023-10-06T04:52:18.661155+0000","last_undegraded":"2023-10-06T04:52:18.661155+0000","last_fullsized":"2023-10-06T04:52:18.661155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.364450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646904+0000","last_change":"2023-10-06T04:52:17.657760+0000","last_active":"2023-10-06T04:52:18.646904+0000","last_peered":"2023-10-06T04:52:18.646904+0000","last_clean":"2023-10-06T04:52:18.646904+0000","last_became_active":"2023-10-06T04:52:17.657484+0000","last_became_peered":"2023-10-06T04:52:17.657484+0000","last_unstale":"2023-10-06T04:52:18.646904+0000","last_undegraded":"2023-10-06T04:52:18.646904+0000","last_fullsized":"2023-10-06T04:52:18.646904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665775+0000","last_change":"2023-10-06T04:52:18.661243+0000","last_active":"2023-10-06T04:52:18.665775+0000","last_peered":"2023-10-06T04:52:18.665775+0000","last_clean":"2023-10-06T04:52:18.665775+0000","last_became_active":"2023-10-06T04:52:18.660932+0000","last_became_peered":"2023-10-06T04:52:18.660932+0000","last_unstale":"2023-10-06T04:52:18.665775+0000","last_undegraded":"2023-10-06T04:52:18.665775+0000","last_fullsized":"2023-10-06T04:52:18.665775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666172+0000","last_change":"2023-10-06T04:52:17.665506+0000","last_active":"2023-10-06T04:52:18.666172+0000","last_peered":"2023-10-06T04:52:18.666172+0000","last_clean":"2023-10-06T04:52:18.666172+0000","last_became_active":"2023-10-06T04:52:17.665209+0000","last_became_peered":"2023-10-06T04:52:17.665209+0000","last_unstale":"2023-10-06T04:52:18.666172+0000","last_undegraded":"2023-10-06T04:52:18.666172+0000","last_fullsized":"2023-10-06T04:52:18.666172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.660252+0000","last_change":"2023-10-06T04:52:18.654716+0000","last_active":"2023-10-06T04:52:18.660252+0000","last_peered":"2023-10-06T04:52:18.660252+0000","last_clean":"2023-10-06T04:52:18.660252+0000","last_became_active":"2023-10-06T04:52:18.653774+0000","last_became_peered":"2023-10-06T04:52:18.653774+0000","last_unstale":"2023-10-06T04:52:18.660252+0000","last_undegraded":"2023-10-06T04:52:18.660252+0000","last_fullsized":"2023-10-06T04:52:18.660252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659771+0000","last_change":"2023-10-06T04:52:17.657327+0000","last_active":"2023-10-06T04:52:18.659771+0000","last_peered":"2023-10-06T04:52:18.659771+0000","last_clean":"2023-10-06T04:52:18.659771+0000","last_became_active":"2023-10-06T04:52:17.657054+0000","last_became_peered":"2023-10-06T04:52:17.657054+0000","last_unstale":"2023-10-06T04:52:18.659771+0000","last_undegraded":"2023-10-06T04:52:18.659771+0000","last_fullsized":"2023-10-06T04:52:18.659771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655497+0000","last_change":"2023-10-06T04:52:18.652991+0000","last_active":"2023-10-06T04:52:18.655497+0000","last_peered":"2023-10-06T04:52:18.655497+0000","last_clean":"2023-10-06T04:52:18.655497+0000","last_became_active":"2023-10-06T04:52:18.652695+0000","last_became_peered":"2023-10-06T04:52:18.652695+0000","last_unstale":"2023-10-06T04:52:18.655497+0000","last_undegraded":"2023-10-06T04:52:18.655497+0000","last_fullsized":"2023-10-06T04:52:18.655497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:14.469459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665605+0000","last_change":"2023-10-06T04:52:17.659228+0000","last_active":"2023-10-06T04:52:18.665605+0000","last_peered":"2023-10-06T04:52:18.665605+0000","last_clean":"2023-10-06T04:52:18.665605+0000","last_became_active":"2023-10-06T04:52:17.658947+0000","last_became_peered":"2023-10-06T04:52:17.658947+0000","last_unstale":"2023-10-06T04:52:18.665605+0000","last_undegraded":"2023-10-06T04:52:18.665605+0000","last_fullsized":"2023-10-06T04:52:18.665605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659272+0000","last_change":"2023-10-06T04:52:18.649861+0000","last_active":"2023-10-06T04:52:18.659272+0000","last_peered":"2023-10-06T04:52:18.659272+0000","last_clean":"2023-10-06T04:52:18.659272+0000","last_became_active":"2023-10-06T04:52:18.649576+0000","last_became_peered":"2023-10-06T04:52:18.649576+0000","last_unstale":"2023-10-06T04:52:18.659272+0000","last_undegraded":"2023-10-06T04:52:18.659272+0000","last_fullsized":"2023-10-06T04:52:18.659272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:14.896487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.657811+0000","last_change":"2023-10-06T04:52:17.653503+0000","last_active":"2023-10-06T04:52:18.657811+0000","last_peered":"2023-10-06T04:52:18.657811+0000","last_clean":"2023-10-06T04:52:18.657811+0000","last_became_active":"2023-10-06T04:52:17.653236+0000","last_became_peered":"2023-10-06T04:52:17.653236+0000","last_unstale":"2023-10-06T04:52:18.657811+0000","last_undegraded":"2023-10-06T04:52:18.657811+0000","last_fullsized":"2023-10-06T04:52:18.657811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:27.560082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664792+0000","last_change":"2023-10-06T04:52:18.661966+0000","last_active":"2023-10-06T04:52:18.664792+0000","last_peered":"2023-10-06T04:52:18.664792+0000","last_clean":"2023-10-06T04:52:18.664792+0000","last_became_active":"2023-10-06T04:52:18.661376+0000","last_became_peered":"2023-10-06T04:52:18.661376+0000","last_unstale":"2023-10-06T04:52:18.664792+0000","last_undegraded":"2023-10-06T04:52:18.664792+0000","last_fullsized":"2023-10-06T04:52:18.664792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643420+0000","last_change":"2023-10-06T04:52:17.650123+0000","last_active":"2023-10-06T04:52:18.643420+0000","last_peered":"2023-10-06T04:52:18.643420+0000","last_clean":"2023-10-06T04:52:18.643420+0000","last_became_active":"2023-10-06T04:52:17.649388+0000","last_became_peered":"2023-10-06T04:52:17.649388+0000","last_unstale":"2023-10-06T04:52:18.643420+0000","last_undegraded":"2023-10-06T04:52:18.643420+0000","last_fullsized":"2023-10-06T04:52:18.643420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.650775+0000","last_change":"2023-10-06T04:52:18.647865+0000","last_active":"2023-10-06T04:52:18.650775+0000","last_peered":"2023-10-06T04:52:18.650775+0000","last_clean":"2023-10-06T04:52:18.650775+0000","last_became_active":"2023-10-06T04:52:18.647582+0000","last_became_peered":"2023-10-06T04:52:18.647582+0000","last_unstale":"2023-10-06T04:52:18.650775+0000","last_undegraded":"2023-10-06T04:52:18.650775+0000","last_fullsized":"2023-10-06T04:52:18.650775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:24.184823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665684+0000","last_change":"2023-10-06T04:52:17.651495+0000","last_active":"2023-10-06T04:52:18.665684+0000","last_peered":"2023-10-06T04:52:18.665684+0000","last_clean":"2023-10-06T04:52:18.665684+0000","last_became_active":"2023-10-06T04:52:17.651108+0000","last_became_peered":"2023-10-06T04:52:17.651108+0000","last_unstale":"2023-10-06T04:52:18.665684+0000","last_undegraded":"2023-10-06T04:52:18.665684+0000","last_fullsized":"2023-10-06T04:52:18.665684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:27.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666462+0000","last_change":"2023-10-06T04:52:18.662825+0000","last_active":"2023-10-06T04:52:18.666462+0000","last_peered":"2023-10-06T04:52:18.666462+0000","last_clean":"2023-10-06T04:52:18.666462+0000","last_became_active":"2023-10-06T04:52:18.662499+0000","last_became_peered":"2023-10-06T04:52:18.662499+0000","last_unstale":"2023-10-06T04:52:18.666462+0000","last_undegraded":"2023-10-06T04:52:18.666462+0000","last_fullsized":"2023-10-06T04:52:18.666462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:58.798038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646929+0000","last_change":"2023-10-06T04:52:17.657591+0000","last_active":"2023-10-06T04:52:18.646929+0000","last_peered":"2023-10-06T04:52:18.646929+0000","last_clean":"2023-10-06T04:52:18.646929+0000","last_became_active":"2023-10-06T04:52:17.657291+0000","last_became_peered":"2023-10-06T04:52:17.657291+0000","last_unstale":"2023-10-06T04:52:18.646929+0000","last_undegraded":"2023-10-06T04:52:18.646929+0000","last_fullsized":"2023-10-06T04:52:18.646929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659774+0000","last_change":"2023-10-06T04:52:18.650766+0000","last_active":"2023-10-06T04:52:18.659774+0000","last_peered":"2023-10-06T04:52:18.659774+0000","last_clean":"2023-10-06T04:52:18.659774+0000","last_became_active":"2023-10-06T04:52:18.650514+0000","last_became_peered":"2023-10-06T04:52:18.650514+0000","last_unstale":"2023-10-06T04:52:18.659774+0000","last_undegraded":"2023-10-06T04:52:18.659774+0000","last_fullsized":"2023-10-06T04:52:18.659774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.639506+0000","last_change":"2023-10-06T04:52:17.655486+0000","last_active":"2023-10-06T04:52:18.639506+0000","last_peered":"2023-10-06T04:52:18.639506+0000","last_clean":"2023-10-06T04:52:18.639506+0000","last_became_active":"2023-10-06T04:52:17.654231+0000","last_became_peered":"2023-10-06T04:52:17.654231+0000","last_unstale":"2023-10-06T04:52:18.639506+0000","last_undegraded":"2023-10-06T04:52:18.639506+0000","last_fullsized":"2023-10-06T04:52:18.639506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666321+0000","last_change":"2023-10-06T04:52:18.663641+0000","last_active":"2023-10-06T04:52:18.666321+0000","last_peered":"2023-10-06T04:52:18.666321+0000","last_clean":"2023-10-06T04:52:18.666321+0000","last_became_active":"2023-10-06T04:52:18.663334+0000","last_became_peered":"2023-10-06T04:52:18.663334+0000","last_unstale":"2023-10-06T04:52:18.666321+0000","last_undegraded":"2023-10-06T04:52:18.666321+0000","last_fullsized":"2023-10-06T04:52:18.666321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:18.572482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643070+0000","last_change":"2023-10-06T04:52:17.644385+0000","last_active":"2023-10-06T04:52:18.643070+0000","last_peered":"2023-10-06T04:52:18.643070+0000","last_clean":"2023-10-06T04:52:18.643070+0000","last_became_active":"2023-10-06T04:52:17.643911+0000","last_became_peered":"2023-10-06T04:52:17.643911+0000","last_unstale":"2023-10-06T04:52:18.643070+0000","last_undegraded":"2023-10-06T04:52:18.643070+0000","last_fullsized":"2023-10-06T04:52:18.643070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661372+0000","last_change":"2023-10-06T04:52:18.657062+0000","last_active":"2023-10-06T04:52:18.661372+0000","last_peered":"2023-10-06T04:52:18.661372+0000","last_clean":"2023-10-06T04:52:18.661372+0000","last_became_active":"2023-10-06T04:52:18.656805+0000","last_became_peered":"2023-10-06T04:52:18.656805+0000","last_unstale":"2023-10-06T04:52:18.661372+0000","last_undegraded":"2023-10-06T04:52:18.661372+0000","last_fullsized":"2023-10-06T04:52:18.661372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:11.654664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664453+0000","last_change":"2023-10-06T04:52:17.648546+0000","last_active":"2023-10-06T04:52:18.664453+0000","last_peered":"2023-10-06T04:52:18.664453+0000","last_clean":"2023-10-06T04:52:18.664453+0000","last_became_active":"2023-10-06T04:52:17.648247+0000","last_became_peered":"2023-10-06T04:52:17.648247+0000","last_unstale":"2023-10-06T04:52:18.664453+0000","last_undegraded":"2023-10-06T04:52:18.664453+0000","last_fullsized":"2023-10-06T04:52:18.664453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642529+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:52:18.642529+0000","last_peered":"2023-10-06T04:52:18.642529+0000","last_clean":"2023-10-06T04:52:18.642529+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:52:18.642529+0000","last_undegraded":"2023-10-06T04:52:18.642529+0000","last_fullsized":"2023-10-06T04:52:18.642529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659829+0000","last_change":"2023-10-06T04:52:18.654743+0000","last_active":"2023-10-06T04:52:18.659829+0000","last_peered":"2023-10-06T04:52:18.659829+0000","last_clean":"2023-10-06T04:52:18.659829+0000","last_became_active":"2023-10-06T04:52:18.653945+0000","last_became_peered":"2023-10-06T04:52:18.653945+0000","last_unstale":"2023-10-06T04:52:18.659829+0000","last_undegraded":"2023-10-06T04:52:18.659829+0000","last_fullsized":"2023-10-06T04:52:18.659829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642776+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:52:18.642776+0000","last_peered":"2023-10-06T04:52:18.642776+0000","last_clean":"2023-10-06T04:52:18.642776+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:52:18.642776+0000","last_undegraded":"2023-10-06T04:52:18.642776+0000","last_fullsized":"2023-10-06T04:52:18.642776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T04:52:19.637862+0000","last_change":"2023-10-06T04:52:17.653308+0000","last_active":"2023-10-06T04:52:19.637862+0000","last_peered":"2023-10-06T04:52:19.637862+0000","last_clean":"2023-10-06T04:52:19.637862+0000","last_became_active":"2023-10-06T04:52:17.653032+0000","last_became_peered":"2023-10-06T04:52:17.653032+0000","last_unstale":"2023-10-06T04:52:19.637862+0000","last_undegraded":"2023-10-06T04:52:19.637862+0000","last_fullsized":"2023-10-06T04:52:19.637862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:12.939190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665740+0000","last_change":"2023-10-06T04:52:18.656477+0000","last_active":"2023-10-06T04:52:18.665740+0000","last_peered":"2023-10-06T04:52:18.665740+0000","last_clean":"2023-10-06T04:52:18.665740+0000","last_became_active":"2023-10-06T04:52:18.656225+0000","last_became_peered":"2023-10-06T04:52:18.656225+0000","last_unstale":"2023-10-06T04:52:18.665740+0000","last_undegraded":"2023-10-06T04:52:18.665740+0000","last_fullsized":"2023-10-06T04:52:18.665740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:01.923170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.643280+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:52:18.643280+0000","last_peered":"2023-10-06T04:52:18.643280+0000","last_clean":"2023-10-06T04:52:18.643280+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:52:18.643280+0000","last_undegraded":"2023-10-06T04:52:18.643280+0000","last_fullsized":"2023-10-06T04:52:18.643280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647019+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:52:18.647019+0000","last_peered":"2023-10-06T04:52:18.647019+0000","last_clean":"2023-10-06T04:52:18.647019+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:52:18.647019+0000","last_undegraded":"2023-10-06T04:52:18.647019+0000","last_fullsized":"2023-10-06T04:52:18.647019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.638291+0000","last_change":"2023-10-06T04:52:17.656649+0000","last_active":"2023-10-06T04:52:18.638291+0000","last_peered":"2023-10-06T04:52:18.638291+0000","last_clean":"2023-10-06T04:52:18.638291+0000","last_became_active":"2023-10-06T04:52:17.656336+0000","last_became_peered":"2023-10-06T04:52:17.656336+0000","last_unstale":"2023-10-06T04:52:18.638291+0000","last_undegraded":"2023-10-06T04:52:18.638291+0000","last_fullsized":"2023-10-06T04:52:18.638291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:00.954790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662767+0000","last_change":"2023-10-06T04:52:18.659289+0000","last_active":"2023-10-06T04:52:18.662767+0000","last_peered":"2023-10-06T04:52:18.662767+0000","last_clean":"2023-10-06T04:52:18.662767+0000","last_became_active":"2023-10-06T04:52:18.659024+0000","last_became_peered":"2023-10-06T04:52:18.659024+0000","last_unstale":"2023-10-06T04:52:18.662767+0000","last_undegraded":"2023-10-06T04:52:18.662767+0000","last_fullsized":"2023-10-06T04:52:18.662767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:03.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647410+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:52:18.647410+0000","last_peered":"2023-10-06T04:52:18.647410+0000","last_clean":"2023-10-06T04:52:18.647410+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:52:18.647410+0000","last_undegraded":"2023-10-06T04:52:18.647410+0000","last_fullsized":"2023-10-06T04:52:18.647410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646114+0000","last_change":"2023-10-06T04:52:17.648757+0000","last_active":"2023-10-06T04:52:18.646114+0000","last_peered":"2023-10-06T04:52:18.646114+0000","last_clean":"2023-10-06T04:52:18.646114+0000","last_became_active":"2023-10-06T04:52:17.647698+0000","last_became_peered":"2023-10-06T04:52:17.647698+0000","last_unstale":"2023-10-06T04:52:18.646114+0000","last_undegraded":"2023-10-06T04:52:18.646114+0000","last_fullsized":"2023-10-06T04:52:18.646114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:42.190732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.653635+0000","last_change":"2023-10-06T04:52:18.648518+0000","last_active":"2023-10-06T04:52:18.653635+0000","last_peered":"2023-10-06T04:52:18.653635+0000","last_clean":"2023-10-06T04:52:18.653635+0000","last_became_active":"2023-10-06T04:52:18.648172+0000","last_became_peered":"2023-10-06T04:52:18.648172+0000","last_unstale":"2023-10-06T04:52:18.653635+0000","last_undegraded":"2023-10-06T04:52:18.653635+0000","last_fullsized":"2023-10-06T04:52:18.653635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:43.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646590+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:52:18.646590+0000","last_peered":"2023-10-06T04:52:18.646590+0000","last_clean":"2023-10-06T04:52:18.646590+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:52:18.646590+0000","last_undegraded":"2023-10-06T04:52:18.646590+0000","last_fullsized":"2023-10-06T04:52:18.646590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.642140+0000","last_change":"2023-10-06T04:52:17.647411+0000","last_active":"2023-10-06T04:52:18.642140+0000","last_peered":"2023-10-06T04:52:18.642140+0000","last_clean":"2023-10-06T04:52:18.642140+0000","last_became_active":"2023-10-06T04:52:17.646893+0000","last_became_peered":"2023-10-06T04:52:17.646893+0000","last_unstale":"2023-10-06T04:52:18.642140+0000","last_undegraded":"2023-10-06T04:52:18.642140+0000","last_fullsized":"2023-10-06T04:52:18.642140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.662059+0000","last_change":"2023-10-06T04:52:18.655841+0000","last_active":"2023-10-06T04:52:18.662059+0000","last_peered":"2023-10-06T04:52:18.662059+0000","last_clean":"2023-10-06T04:52:18.662059+0000","last_became_active":"2023-10-06T04:52:18.655528+0000","last_became_peered":"2023-10-06T04:52:18.655528+0000","last_unstale":"2023-10-06T04:52:18.662059+0000","last_undegraded":"2023-10-06T04:52:18.662059+0000","last_fullsized":"2023-10-06T04:52:18.662059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646567+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:52:18.646567+0000","last_peered":"2023-10-06T04:52:18.646567+0000","last_clean":"2023-10-06T04:52:18.646567+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:52:18.646567+0000","last_undegraded":"2023-10-06T04:52:18.646567+0000","last_fullsized":"2023-10-06T04:52:18.646567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.641848+0000","last_change":"2023-10-06T04:52:17.655012+0000","last_active":"2023-10-06T04:52:18.641848+0000","last_peered":"2023-10-06T04:52:18.641848+0000","last_clean":"2023-10-06T04:52:18.641848+0000","last_became_active":"2023-10-06T04:52:17.654727+0000","last_became_peered":"2023-10-06T04:52:17.654727+0000","last_unstale":"2023-10-06T04:52:18.641848+0000","last_undegraded":"2023-10-06T04:52:18.641848+0000","last_fullsized":"2023-10-06T04:52:18.641848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.653697+0000","last_change":"2023-10-06T04:52:18.647619+0000","last_active":"2023-10-06T04:52:18.653697+0000","last_peered":"2023-10-06T04:52:18.653697+0000","last_clean":"2023-10-06T04:52:18.653697+0000","last_became_active":"2023-10-06T04:52:18.647303+0000","last_became_peered":"2023-10-06T04:52:18.647303+0000","last_unstale":"2023-10-06T04:52:18.653697+0000","last_undegraded":"2023-10-06T04:52:18.653697+0000","last_fullsized":"2023-10-06T04:52:18.653697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:35.374135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.645972+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:52:18.645972+0000","last_peered":"2023-10-06T04:52:18.645972+0000","last_clean":"2023-10-06T04:52:18.645972+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:52:18.645972+0000","last_undegraded":"2023-10-06T04:52:18.645972+0000","last_fullsized":"2023-10-06T04:52:18.645972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647129+0000","last_change":"2023-10-06T04:52:17.649887+0000","last_active":"2023-10-06T04:52:18.647129+0000","last_peered":"2023-10-06T04:52:18.647129+0000","last_clean":"2023-10-06T04:52:18.647129+0000","last_became_active":"2023-10-06T04:52:17.649537+0000","last_became_peered":"2023-10-06T04:52:17.649537+0000","last_unstale":"2023-10-06T04:52:18.647129+0000","last_undegraded":"2023-10-06T04:52:18.647129+0000","last_fullsized":"2023-10-06T04:52:18.647129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:57.133434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.655515+0000","last_change":"2023-10-06T04:52:18.649286+0000","last_active":"2023-10-06T04:52:18.655515+0000","last_peered":"2023-10-06T04:52:18.655515+0000","last_clean":"2023-10-06T04:52:18.655515+0000","last_became_active":"2023-10-06T04:52:18.648947+0000","last_became_peered":"2023-10-06T04:52:18.648947+0000","last_unstale":"2023-10-06T04:52:18.655515+0000","last_undegraded":"2023-10-06T04:52:18.655515+0000","last_fullsized":"2023-10-06T04:52:18.655515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.605535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.646165+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:52:18.646165+0000","last_peered":"2023-10-06T04:52:18.646165+0000","last_clean":"2023-10-06T04:52:18.646165+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:52:18.646165+0000","last_undegraded":"2023-10-06T04:52:18.646165+0000","last_fullsized":"2023-10-06T04:52:18.646165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.647419+0000","last_change":"2023-10-06T04:52:17.650178+0000","last_active":"2023-10-06T04:52:18.647419+0000","last_peered":"2023-10-06T04:52:18.647419+0000","last_clean":"2023-10-06T04:52:18.647419+0000","last_became_active":"2023-10-06T04:52:17.649883+0000","last_became_peered":"2023-10-06T04:52:17.649883+0000","last_unstale":"2023-10-06T04:52:18.647419+0000","last_undegraded":"2023-10-06T04:52:18.647419+0000","last_fullsized":"2023-10-06T04:52:18.647419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661164+0000","last_change":"2023-10-06T04:52:18.658639+0000","last_active":"2023-10-06T04:52:18.661164+0000","last_peered":"2023-10-06T04:52:18.661164+0000","last_clean":"2023-10-06T04:52:18.661164+0000","last_became_active":"2023-10-06T04:52:18.658381+0000","last_became_peered":"2023-10-06T04:52:18.658381+0000","last_unstale":"2023-10-06T04:52:18.661164+0000","last_undegraded":"2023-10-06T04:52:18.661164+0000","last_fullsized":"2023-10-06T04:52:18.661164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:51.836228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665805+0000","last_change":"2023-10-06T04:52:17.657299+0000","last_active":"2023-10-06T04:52:18.665805+0000","last_peered":"2023-10-06T04:52:18.665805+0000","last_clean":"2023-10-06T04:52:18.665805+0000","last_became_active":"2023-10-06T04:52:17.656956+0000","last_became_peered":"2023-10-06T04:52:17.656956+0000","last_unstale":"2023-10-06T04:52:18.665805+0000","last_undegraded":"2023-10-06T04:52:18.665805+0000","last_fullsized":"2023-10-06T04:52:18.665805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.659769+0000","last_change":"2023-10-06T04:52:18.652907+0000","last_active":"2023-10-06T04:52:18.659769+0000","last_peered":"2023-10-06T04:52:18.659769+0000","last_clean":"2023-10-06T04:52:18.659769+0000","last_became_active":"2023-10-06T04:52:18.652600+0000","last_became_peered":"2023-10-06T04:52:18.652600+0000","last_unstale":"2023-10-06T04:52:18.659769+0000","last_undegraded":"2023-10-06T04:52:18.659769+0000","last_fullsized":"2023-10-06T04:52:18.659769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:13.950333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667832+0000","last_change":"2023-10-06T04:52:17.656487+0000","last_active":"2023-10-06T04:52:18.667832+0000","last_peered":"2023-10-06T04:52:18.667832+0000","last_clean":"2023-10-06T04:52:18.667832+0000","last_became_active":"2023-10-06T04:52:17.656205+0000","last_became_peered":"2023-10-06T04:52:17.656205+0000","last_unstale":"2023-10-06T04:52:18.667832+0000","last_undegraded":"2023-10-06T04:52:18.667832+0000","last_fullsized":"2023-10-06T04:52:18.667832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665639+0000","last_change":"2023-10-06T04:52:18.654103+0000","last_active":"2023-10-06T04:52:18.665639+0000","last_peered":"2023-10-06T04:52:18.665639+0000","last_clean":"2023-10-06T04:52:18.665639+0000","last_became_active":"2023-10-06T04:52:18.653791+0000","last_became_peered":"2023-10-06T04:52:18.653791+0000","last_unstale":"2023-10-06T04:52:18.665639+0000","last_undegraded":"2023-10-06T04:52:18.665639+0000","last_fullsized":"2023-10-06T04:52:18.665639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:18.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.667826+0000","last_change":"2023-10-06T04:52:17.656962+0000","last_active":"2023-10-06T04:52:18.667826+0000","last_peered":"2023-10-06T04:52:18.667826+0000","last_clean":"2023-10-06T04:52:18.667826+0000","last_became_active":"2023-10-06T04:52:17.656699+0000","last_became_peered":"2023-10-06T04:52:17.656699+0000","last_unstale":"2023-10-06T04:52:18.667826+0000","last_undegraded":"2023-10-06T04:52:18.667826+0000","last_fullsized":"2023-10-06T04:52:18.667826+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:54.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.665510+0000","last_change":"2023-10-06T04:52:18.653480+0000","last_active":"2023-10-06T04:52:18.665510+0000","last_peered":"2023-10-06T04:52:18.665510+0000","last_clean":"2023-10-06T04:52:18.665510+0000","last_became_active":"2023-10-06T04:52:18.653200+0000","last_became_peered":"2023-10-06T04:52:18.653200+0000","last_unstale":"2023-10-06T04:52:18.665510+0000","last_undegraded":"2023-10-06T04:52:18.665510+0000","last_fullsized":"2023-10-06T04:52:18.665510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:51.240487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.663382+0000","last_change":"2023-10-06T04:52:17.651435+0000","last_active":"2023-10-06T04:52:18.663382+0000","last_peered":"2023-10-06T04:52:18.663382+0000","last_clean":"2023-10-06T04:52:18.663382+0000","last_became_active":"2023-10-06T04:52:17.651063+0000","last_became_peered":"2023-10-06T04:52:17.651063+0000","last_unstale":"2023-10-06T04:52:18.663382+0000","last_undegraded":"2023-10-06T04:52:18.663382+0000","last_fullsized":"2023-10-06T04:52:18.663382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:36.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664028+0000","last_change":"2023-10-06T04:52:18.658488+0000","last_active":"2023-10-06T04:52:18.664028+0000","last_peered":"2023-10-06T04:52:18.664028+0000","last_clean":"2023-10-06T04:52:18.664028+0000","last_became_active":"2023-10-06T04:52:18.657977+0000","last_became_peered":"2023-10-06T04:52:18.657977+0000","last_unstale":"2023-10-06T04:52:18.664028+0000","last_undegraded":"2023-10-06T04:52:18.664028+0000","last_fullsized":"2023-10-06T04:52:18.664028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:07.626504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.661377+0000","last_change":"2023-10-06T04:52:17.652321+0000","last_active":"2023-10-06T04:52:18.661377+0000","last_peered":"2023-10-06T04:52:18.661377+0000","last_clean":"2023-10-06T04:52:18.661377+0000","last_became_active":"2023-10-06T04:52:17.651998+0000","last_became_peered":"2023-10-06T04:52:17.651998+0000","last_unstale":"2023-10-06T04:52:18.661377+0000","last_undegraded":"2023-10-06T04:52:18.661377+0000","last_fullsized":"2023-10-06T04:52:18.661377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:16.615165+0000","last_clean_scrub_stamp":"2023-10-06T04:52:16.615165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:52.959145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666345+0000","last_change":"2023-10-06T04:52:18.664061+0000","last_active":"2023-10-06T04:52:18.666345+0000","last_peered":"2023-10-06T04:52:18.666345+0000","last_clean":"2023-10-06T04:52:18.666345+0000","last_became_active":"2023-10-06T04:52:18.663769+0000","last_became_peered":"2023-10-06T04:52:18.663769+0000","last_unstale":"2023-10-06T04:52:18.666345+0000","last_undegraded":"2023-10-06T04:52:18.666345+0000","last_fullsized":"2023-10-06T04:52:18.666345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:13.855227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664651+0000","last_change":"2023-10-06T04:52:18.662900+0000","last_active":"2023-10-06T04:52:18.664651+0000","last_peered":"2023-10-06T04:52:18.664651+0000","last_clean":"2023-10-06T04:52:18.664651+0000","last_became_active":"2023-10-06T04:52:18.662558+0000","last_became_peered":"2023-10-06T04:52:18.662558+0000","last_unstale":"2023-10-06T04:52:18.664651+0000","last_undegraded":"2023-10-06T04:52:18.664651+0000","last_fullsized":"2023-10-06T04:52:18.664651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:18.313966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.664001+0000","last_change":"2023-10-06T04:52:18.658296+0000","last_active":"2023-10-06T04:52:18.664001+0000","last_peered":"2023-10-06T04:52:18.664001+0000","last_clean":"2023-10-06T04:52:18.664001+0000","last_became_active":"2023-10-06T04:52:18.657589+0000","last_became_peered":"2023-10-06T04:52:18.657589+0000","last_unstale":"2023-10-06T04:52:18.664001+0000","last_undegraded":"2023-10-06T04:52:18.664001+0000","last_fullsized":"2023-10-06T04:52:18.664001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.044051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.666380+0000","last_change":"2023-10-06T04:52:18.663851+0000","last_active":"2023-10-06T04:52:18.666380+0000","last_peered":"2023-10-06T04:52:18.666380+0000","last_clean":"2023-10-06T04:52:18.666380+0000","last_became_active":"2023-10-06T04:52:18.663574+0000","last_became_peered":"2023-10-06T04:52:18.663574+0000","last_unstale":"2023-10-06T04:52:18.666380+0000","last_undegraded":"2023-10-06T04:52:18.666380+0000","last_fullsized":"2023-10-06T04:52:18.666380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:54.373475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.656769+0000","last_change":"2023-10-06T04:52:18.652292+0000","last_active":"2023-10-06T04:52:18.656769+0000","last_peered":"2023-10-06T04:52:18.656769+0000","last_clean":"2023-10-06T04:52:18.656769+0000","last_became_active":"2023-10-06T04:52:18.651919+0000","last_became_peered":"2023-10-06T04:52:18.651919+0000","last_unstale":"2023-10-06T04:52:18.656769+0000","last_undegraded":"2023-10-06T04:52:18.656769+0000","last_fullsized":"2023-10-06T04:52:18.656769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:41.164934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T04:52:18.657972+0000","last_change":"2023-10-06T04:52:18.650895+0000","last_active":"2023-10-06T04:52:18.657972+0000","last_peered":"2023-10-06T04:52:18.657972+0000","last_clean":"2023-10-06T04:52:18.657972+0000","last_became_active":"2023-10-06T04:52:18.650623+0000","last_became_peered":"2023-10-06T04:52:18.650623+0000","last_unstale":"2023-10-06T04:52:18.657972+0000","last_undegraded":"2023-10-06T04:52:18.657972+0000","last_fullsized":"2023-10-06T04:52:18.657972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:17.626309+0000","last_clean_scrub_stamp":"2023-10-06T04:52:17.626309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:09.323344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":664951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1779999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298064,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073776,"statfs":{"total":96636764160,"available":96331546624,"internally_reserved":0,"allocated":794624,"data_stored":663199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.069}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4210000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0309999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.367}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5510000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":73455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9039999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7949999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7210000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:52:27.803 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T04:52:27.803 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T04:52:27.804 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T04:52:27.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:52:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.953+0000 7f72806e3700 1 Processor -- start 2023-10-06T04:52:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.953+0000 7f72806e3700 1 -- start start 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.954+0000 7f72806e3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f72780b27b0 0x7f727815a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.954+0000 7f72806e3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727815d6d0 con 0x7f727815d240 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.954+0000 7f72806e3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727815d9b0 con 0x7f72780cd320 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.954+0000 7f72806e3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727815aba0 con 0x7f72780b27b0 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.954+0000 7f727dc7e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f727815d240 0x7f72780ccc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35958/0 (socket says 172.21.15.3:35958) 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.954+0000 7f727dc7e700 1 -- 172.21.15.3:0/3229676741 learned_addr learned my addr 172.21.15.3:0/3229676741 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727e47f700 1 --2- 172.21.15.3:0/3229676741 >> v2:172.21.15.153:3300/0 conn(0x7f72780b27b0 0x7f727815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727d47d700 1 -- 172.21.15.3:0/3229676741 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860708999 0 0) 0x7f727815d6d0 con 0x7f727815d240 2023-10-06T04:52:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727d47d700 1 -- 172.21.15.3:0/3229676741 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7260003570 con 0x7f727815d240 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727e47f700 1 -- 172.21.15.3:0/3229676741 >> v1:172.21.15.153:6789/0 conn(0x7f72780cd320 legacy=0x7f72780aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727e47f700 1 -- 172.21.15.3:0/3229676741 >> v1:172.21.15.3:6789/0 conn(0x7f727815d240 legacy=0x7f72780ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727d47d700 1 -- 172.21.15.3:0/3229676741 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2135551630 0 0) 0x7f7260003570 con 0x7f727815d240 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727e47f700 1 -- 172.21.15.3:0/3229676741 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727815dc90 con 0x7f72780b27b0 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727e47f700 1 --2- 172.21.15.3:0/3229676741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780b27b0 0x7f727815a660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f72700098e0 tx=0x7f72700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9b4cbaaf27ed03b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727d47d700 1 -- 172.21.15.3:0/3229676741 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c001470 con 0x7f72780b27b0 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727d47d700 1 -- 172.21.15.3:0/3229676741 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7270011420 con 0x7f72780b27b0 2023-10-06T04:52:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.955+0000 7f727d47d700 1 -- 172.21.15.3:0/3229676741 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72700117a0 con 0x7f72780b27b0 2023-10-06T04:52:27.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.956+0000 7f72806e3700 1 -- 172.21.15.3:0/3229676741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780b27b0 msgr2=0x7f727815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.956+0000 7f72806e3700 1 --2- 172.21.15.3:0/3229676741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780b27b0 0x7f727815a660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f72700098e0 tx=0x7f72700091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.956+0000 7f72806e3700 1 -- 172.21.15.3:0/3229676741 shutdown_connections 2023-10-06T04:52:27.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.956+0000 7f72806e3700 1 --2- 172.21.15.3:0/3229676741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780b27b0 0x7f727815a660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.956+0000 7f72806e3700 1 -- 172.21.15.3:0/3229676741 >> 172.21.15.3:0/3229676741 conn(0x7f7278070310 msgr2=0x7f7278070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:27.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.956+0000 7f72806e3700 1 -- 172.21.15.3:0/3229676741 shutdown_connections 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.956+0000 7f72806e3700 1 -- 172.21.15.3:0/3229676741 wait complete. 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.957+0000 7f72806e3700 1 Processor -- start 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.957+0000 7f72806e3700 1 -- start start 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72806e3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72780b27b0 0x7f72780c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72806e3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780cd320 0x7f727816dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72806e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727815d240 0x7f72780b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72806e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727816e100 con 0x7f727815d240 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72806e3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278150d10 con 0x7f72780cd320 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72806e3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7278150fb0 con 0x7f72780b27b0 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727815d240 0x7f72780b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727815d240 0x7f72780b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33286/0 (socket says 172.21.15.3:33286) 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 -- 172.21.15.3:0/4156407242 learned_addr learned my addr 172.21.15.3:0/4156407242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727e47f700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72780b27b0 0x7f72780c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727dc7e700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780cd320 0x7f727816dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 -- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72780b27b0 msgr2=0x7f72780c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72780b27b0 0x7f72780c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 -- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780cd320 msgr2=0x7f727816dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780cd320 0x7f727816dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 -- 172.21.15.3:0/4156407242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270009420 con 0x7f727815d240 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f727ec80700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727815d240 0x7f72780b12b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f726800c8e0 tx=0x7f7268001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72767fc700 1 -- 172.21.15.3:0/4156407242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800ce80 con 0x7f727815d240 2023-10-06T04:52:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72767fc700 1 -- 172.21.15.3:0/4156407242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726800a7e0 con 0x7f727815d240 2023-10-06T04:52:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7278151290 con 0x7f727815d240 2023-10-06T04:52:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.958+0000 7f72767fc700 1 -- 172.21.15.3:0/4156407242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800ab80 con 0x7f727815d240 2023-10-06T04:52:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.959+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72780ca300 con 0x7f727815d240 2023-10-06T04:52:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.959+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72781517b0 con 0x7f727815d240 2023-10-06T04:52:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.960+0000 7f72767fc700 1 -- 172.21.15.3:0/4156407242 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7268012420 con 0x7f727815d240 2023-10-06T04:52:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.960+0000 7f72767fc700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7264066f30 0x7f72640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.960+0000 7f72767fc700 1 -- 172.21.15.3:0/4156407242 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7268084940 con 0x7f727815d240 2023-10-06T04:52:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.960+0000 7f727e47f700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7264066f30 0x7f72640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.961+0000 7f727e47f700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7264066f30 0x7f72640693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7270009fd0 tx=0x7f7270016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:27.965+0000 7f72767fc700 1 -- 172.21.15.3:0/4156407242 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72781517b0 con 0x7f727815d240 2023-10-06T04:52:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.200+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f72781517b0 con 0x7f727815d240 2023-10-06T04:52:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.201+0000 7f72767fc700 1 -- 172.21.15.3:0/4156407242 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f72781517b0 con 0x7f727815d240 2023-10-06T04:52:28.202 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:28.203 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7264066f30 msgr2=0x7f72640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7264066f30 0x7f72640693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7270009fd0 tx=0x7f7270016000 comp rx=0 tx=0).stop 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727815d240 msgr2=0x7f72780b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727815d240 0x7f72780b12b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f726800c8e0 tx=0x7f7268001710 comp rx=0 tx=0).stop 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 shutdown_connections 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7264066f30 0x7f72640693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f727815d240 0x7f72780b12b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72780b27b0 0x7f72780c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 --2- 172.21.15.3:0/4156407242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72780cd320 0x7f727816dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 >> 172.21.15.3:0/4156407242 conn(0x7f7278070310 msgr2=0x7f727814fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:28.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 shutdown_connections 2023-10-06T04:52:28.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.203+0000 7f72806e3700 1 -- 172.21.15.3:0/4156407242 wait complete. 2023-10-06T04:52:28.216 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T04:52:28.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.334+0000 7fa526534700 1 Processor -- start 2023-10-06T04:52:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.334+0000 7fa526534700 1 -- start start 2023-10-06T04:52:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa526534700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa52015c8f0 0x7fa52015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa526534700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5200d0870 0x7fa5200b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa526534700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5200cf6c0 con 0x7fa52015ccc0 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa526534700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5200cfb50 con 0x7fa52015c8f0 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa526534700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52015e050 con 0x7fa5200d0870 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa524ad1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5200d0870 0x7fa5200b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa51ffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa52015c8f0 0x7fa52015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa524ad1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5200d0870 0x7fa5200b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59766/0 (socket says 172.21.15.3:59766) 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa524ad1700 1 -- 172.21.15.3:0/1039360450 learned_addr learned my addr 172.21.15.3:0/1039360450 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa51ffff700 1 --2- 172.21.15.3:0/1039360450 >> v2:172.21.15.153:3300/0 conn(0x7fa52015c8f0 0x7fa52015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:38180/0 (socket says 172.21.15.3:38180) 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.335+0000 7fa51f7fe700 1 --1- 172.21.15.3:0/1039360450 >> v1:172.21.15.153:6789/0 conn(0x7fa52015ccc0 0x7fa5200d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49640/0 (socket says 172.21.15.3:49640) 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa524ad1700 1 -- 172.21.15.3:0/1039360450 >> v2:172.21.15.153:3300/0 conn(0x7fa52015c8f0 msgr2=0x7fa52015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa524ad1700 1 --2- 172.21.15.3:0/1039360450 >> v2:172.21.15.153:3300/0 conn(0x7fa52015c8f0 0x7fa52015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa524ad1700 1 -- 172.21.15.3:0/1039360450 >> v1:172.21.15.153:6789/0 conn(0x7fa52015ccc0 legacy=0x7fa5200d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa524ad1700 1 -- 172.21.15.3:0/1039360450 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200ceab0 con 0x7fa5200d0870 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa524ad1700 1 --2- 172.21.15.3:0/1039360450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5200d0870 0x7fa5200b3ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa518009f20 tx=0x7fa518009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb8554cfef0780a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa51effd700 1 -- 172.21.15.3:0/1039360450 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800e070 con 0x7fa5200d0870 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa51effd700 1 -- 172.21.15.3:0/1039360450 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51800b9d0 con 0x7fa5200d0870 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa51effd700 1 -- 172.21.15.3:0/1039360450 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5180092e0 con 0x7fa5200d0870 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa526534700 1 -- 172.21.15.3:0/1039360450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5200d0870 msgr2=0x7fa5200b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.336+0000 7fa526534700 1 --2- 172.21.15.3:0/1039360450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5200d0870 0x7fa5200b3ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa518009f20 tx=0x7fa518009840 comp rx=0 tx=0).stop 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.337+0000 7fa526534700 1 -- 172.21.15.3:0/1039360450 shutdown_connections 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.337+0000 7fa526534700 1 --2- 172.21.15.3:0/1039360450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5200d0870 0x7fa5200b3ba0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.337+0000 7fa526534700 1 --2- 172.21.15.3:0/1039360450 >> v2:172.21.15.153:3300/0 conn(0x7fa52015c8f0 0x7fa52015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.337+0000 7fa526534700 1 -- 172.21.15.3:0/1039360450 >> 172.21.15.3:0/1039360450 conn(0x7fa520070310 msgr2=0x7fa520070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.337+0000 7fa526534700 1 -- 172.21.15.3:0/1039360450 shutdown_connections 2023-10-06T04:52:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.337+0000 7fa526534700 1 -- 172.21.15.3:0/1039360450 wait complete. 2023-10-06T04:52:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.338+0000 7fa526534700 1 Processor -- start 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.338+0000 7fa526534700 1 -- start start 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5200d0870 0x7fa52015b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52015c8f0 0x7fa5200c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa52015ccc0 0x7fa520154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200c6960 con 0x7fa5200d0870 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200c60c0 con 0x7fa52015ccc0 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200c63a0 con 0x7fa52015c8f0 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5200d0870 0x7fa52015b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5200d0870 0x7fa52015b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33290/0 (socket says 172.21.15.3:33290) 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 -- 172.21.15.3:0/2779480928 learned_addr learned my addr 172.21.15.3:0/2779480928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51f7fe700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52015c8f0 0x7fa5200c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa524ad1700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa52015ccc0 0x7fa520154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 -- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52015c8f0 msgr2=0x7fa5200c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52015c8f0 0x7fa5200c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 -- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa52015ccc0 msgr2=0x7fa520154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa52015ccc0 0x7fa520154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 -- 172.21.15.3:0/2779480928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa518009ae0 con 0x7fa5200d0870 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51f7fe700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52015c8f0 0x7fa5200c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa524ad1700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa52015ccc0 0x7fa520154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa51ffff700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5200d0870 0x7fa52015b8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa51000ec80 tx=0x7fa5100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa507fff700 1 -- 172.21.15.3:0/2779480928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000c9d0 con 0x7fa5200d0870 2023-10-06T04:52:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa507fff700 1 -- 172.21.15.3:0/2779480928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51000cb30 con 0x7fa5200d0870 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5200c6680 con 0x7fa5200d0870 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa507fff700 1 -- 172.21.15.3:0/2779480928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510013470 con 0x7fa5200d0870 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.339+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa520159020 con 0x7fa5200d0870 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.340+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa520070e40 con 0x7fa5200d0870 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.341+0000 7fa507fff700 1 -- 172.21.15.3:0/2779480928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa510013820 con 0x7fa5200d0870 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.341+0000 7fa507fff700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa50c066f30 0x7fa50c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.341+0000 7fa507fff700 1 -- 172.21.15.3:0/2779480928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa510084a20 con 0x7fa5200d0870 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.341+0000 7fa51f7fe700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa50c066f30 0x7fa50c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.342+0000 7fa51f7fe700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa50c066f30 0x7fa50c0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa514007fd0 tx=0x7fa514007e10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.345+0000 7fa507fff700 1 -- 172.21.15.3:0/2779480928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa510051b30 con 0x7fa5200d0870 2023-10-06T04:52:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.540+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa520071680 con 0x7fa5200d0870 2023-10-06T04:52:28.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.541+0000 7fa507fff700 1 -- 172.21.15.3:0/2779480928 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fa510055150 con 0x7fa5200d0870 2023-10-06T04:52:28.542 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:28.543 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:28.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa50c066f30 msgr2=0x7fa50c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa50c066f30 0x7fa50c0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa514007fd0 tx=0x7fa514007e10 comp rx=0 tx=0).stop 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5200d0870 msgr2=0x7fa52015b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5200d0870 0x7fa52015b8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa51000ec80 tx=0x7fa5100091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 shutdown_connections 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa50c066f30 0x7fa50c0693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa52015ccc0 0x7fa520154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5200d0870 0x7fa52015b8f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 --2- 172.21.15.3:0/2779480928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52015c8f0 0x7fa5200c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 >> 172.21.15.3:0/2779480928 conn(0x7fa520070310 msgr2=0x7fa5200b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 shutdown_connections 2023-10-06T04:52:28.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.543+0000 7fa526534700 1 -- 172.21.15.3:0/2779480928 wait complete. 2023-10-06T04:52:28.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:52:15.735846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:52:17.015278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:52:28.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.677+0000 7febfe7bf700 1 Processor -- start 2023-10-06T04:52:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.677+0000 7febfe7bf700 1 -- start start 2023-10-06T04:52:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfe7bf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7febf80af660 0x7febf80ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfe7bf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfe7bf700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf80cb190 con 0x7febf80af290 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfe7bf700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf8159950 con 0x7febf80af660 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfe7bf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf80cd6a0 con 0x7febf80cd190 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfcfbc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7febf80af660 0x7febf80ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfdfbe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfd7bd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7febf80af290 0x7febf80ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49652/0 (socket says 172.21.15.3:49652) 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfdfbe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59778/0 (socket says 172.21.15.3:59778) 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfd7bd700 1 -- 172.21.15.3:0/2191177435 learned_addr learned my addr 172.21.15.3:0/2191177435 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.678+0000 7febfcfbc700 1 --2- 172.21.15.3:0/2191177435 >> v2:172.21.15.153:3300/0 conn(0x7febf80af660 0x7febf80ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:38194/0 (socket says 172.21.15.3:38194) 2023-10-06T04:52:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febf7fff700 1 -- 172.21.15.3:0/2191177435 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174975298 0 0) 0x7febf80cb190 con 0x7febf80af290 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febf7fff700 1 -- 172.21.15.3:0/2191177435 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febe0003570 con 0x7febf80af290 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfcfbc700 1 -- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 msgr2=0x7febf80c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfcfbc700 1 --2- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfcfbc700 1 -- 172.21.15.3:0/2191177435 >> v1:172.21.15.153:6789/0 conn(0x7febf80af290 legacy=0x7febf80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfcfbc700 1 -- 172.21.15.3:0/2191177435 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf80cb3f0 con 0x7febf80af660 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfdfbe700 1 --2- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfcfbc700 1 --2- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 0x7febf80ccc50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febec0098d0 tx=0x7febec009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dff01cca9a4d9de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febf7fff700 1 -- 172.21.15.3:0/2191177435 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec005c10 con 0x7febf80af660 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febf7fff700 1 -- 172.21.15.3:0/2191177435 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec005d70 con 0x7febf80af660 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febf7fff700 1 -- 172.21.15.3:0/2191177435 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec006260 con 0x7febf80af660 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfe7bf700 1 -- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 msgr2=0x7febf80ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 0x7febf80ccc50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febec0098d0 tx=0x7febec009120 comp rx=0 tx=0).stop 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfe7bf700 1 -- 172.21.15.3:0/2191177435 shutdown_connections 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2191177435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 0x7febf80ccc50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.679+0000 7febfe7bf700 1 -- 172.21.15.3:0/2191177435 >> 172.21.15.3:0/2191177435 conn(0x7febf8089dd0 msgr2=0x7febf80996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.680+0000 7febfe7bf700 1 -- 172.21.15.3:0/2191177435 shutdown_connections 2023-10-06T04:52:28.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.680+0000 7febfe7bf700 1 -- 172.21.15.3:0/2191177435 wait complete. 2023-10-06T04:52:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.681+0000 7febfe7bf700 1 Processor -- start 2023-10-06T04:52:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.681+0000 7febfe7bf700 1 -- start start 2023-10-06T04:52:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfe7bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febf80af290 0x7febf80c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfe7bf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 0x7febf8157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfe7bf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfe7bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf80c7b90 con 0x7febf80af290 2023-10-06T04:52:28.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfe7bf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf80c7e70 con 0x7febf80af660 2023-10-06T04:52:28.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfe7bf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf80c8150 con 0x7febf80cd190 2023-10-06T04:52:28.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febf80af290 0x7febf80c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febf80af290 0x7febf80c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33304/0 (socket says 172.21.15.3:33304) 2023-10-06T04:52:28.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 -- 172.21.15.3:0/2536579708 learned_addr learned my addr 172.21.15.3:0/2536579708 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfdfbe700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfcfbc700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 0x7febf8157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 -- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 msgr2=0x7febf80c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 -- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 msgr2=0x7febf8157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 0x7febf8157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 -- 172.21.15.3:0/2536579708 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009420 con 0x7febf80af290 2023-10-06T04:52:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfd7bd700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febf80af290 0x7febf80c7360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7febf000cd20 tx=0x7febf0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febf57fa700 1 -- 172.21.15.3:0/2536579708 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf000ce20 con 0x7febf80af290 2023-10-06T04:52:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febf57fa700 1 -- 172.21.15.3:0/2536579708 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf000aac0 con 0x7febf80af290 2023-10-06T04:52:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.682+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec0093c0 con 0x7febf80af290 2023-10-06T04:52:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.683+0000 7febf57fa700 1 -- 172.21.15.3:0/2536579708 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf000ae40 con 0x7febf80af290 2023-10-06T04:52:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.683+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf80c78a0 con 0x7febf80af290 2023-10-06T04:52:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.684+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf804c380 con 0x7febf80af290 2023-10-06T04:52:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.684+0000 7febf57fa700 1 -- 172.21.15.3:0/2536579708 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febf0012610 con 0x7febf80af290 2023-10-06T04:52:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.684+0000 7febf57fa700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7febe4066f30 0x7febe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.684+0000 7febf57fa700 1 -- 172.21.15.3:0/2536579708 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7febf0084990 con 0x7febf80af290 2023-10-06T04:52:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.684+0000 7febfcfbc700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7febe4066f30 0x7febe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.685+0000 7febfcfbc700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7febe4066f30 0x7febe40693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febec0098d0 tx=0x7febec016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.688+0000 7febf57fa700 1 -- 172.21.15.3:0/2536579708 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febf0051b20 con 0x7febf80af290 2023-10-06T04:52:28.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.883+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7febf80ca6d0 con 0x7febf80af290 2023-10-06T04:52:28.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.883+0000 7febf57fa700 1 -- 172.21.15.3:0/2536579708 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7febf0084070 con 0x7febf80af290 2023-10-06T04:52:28.885 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7febe4066f30 msgr2=0x7febe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7febe4066f30 0x7febe40693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febec0098d0 tx=0x7febec016000 comp rx=0 tx=0).stop 2023-10-06T04:52:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febf80af290 msgr2=0x7febf80c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febf80af290 0x7febf80c7360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7febf000cd20 tx=0x7febf0001b10 comp rx=0 tx=0).stop 2023-10-06T04:52:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 shutdown_connections 2023-10-06T04:52:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7febe4066f30 0x7febe40693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febf80cd190 0x7febf80c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febf80af290 0x7febf80c7360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 --2- 172.21.15.3:0/2536579708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf80af660 0x7febf8157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:28.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 >> 172.21.15.3:0/2536579708 conn(0x7febf8089dd0 msgr2=0x7febf806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:28.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 shutdown_connections 2023-10-06T04:52:28.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:28.885+0000 7febfe7bf700 1 -- 172.21.15.3:0/2536579708 wait complete. 2023-10-06T04:52:28.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.018+0000 7f2033ca2700 1 Processor -- start 2023-10-06T04:52:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.018+0000 7f2033ca2700 1 -- start start 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f2033ca2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f202c06a340 0x7f202c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f2033ca2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f202c06adb0 0x7f202c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f2033ca2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c0ae9d0 0x7f202c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f2033ca2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c06dac0 con 0x7f202c06adb0 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f2033ca2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c15a160 con 0x7f202c06a340 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f2033ca2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0af1b0 con 0x7f202c0ae9d0 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f203123d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f202c06adb0 0x7f202c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f203123d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f202c06adb0 0x7f202c159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33318/0 (socket says 172.21.15.3:33318) 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f203123d700 1 -- 172.21.15.3:0/1329456257 learned_addr learned my addr 172.21.15.3:0/1329456257 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.019+0000 7f203223f700 1 --2- 172.21.15.3:0/1329456257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c0ae9d0 0x7f202c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2031a3e700 1 --2- 172.21.15.3:0/1329456257 >> v2:172.21.15.153:3300/0 conn(0x7f202c06a340 0x7f202c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f203123d700 1 -- 172.21.15.3:0/1329456257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c0ae9d0 msgr2=0x7f202c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f203123d700 1 --2- 172.21.15.3:0/1329456257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c0ae9d0 0x7f202c06c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f203123d700 1 -- 172.21.15.3:0/1329456257 >> v2:172.21.15.153:3300/0 conn(0x7f202c06a340 msgr2=0x7f202c1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f203123d700 1 --2- 172.21.15.3:0/1329456257 >> v2:172.21.15.153:3300/0 conn(0x7f202c06a340 0x7f202c1596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f203123d700 1 -- 172.21.15.3:0/1329456257 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c06d4b0 con 0x7f202c06adb0 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f203123d700 1 --2- 172.21.15.3:0/1329456257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06adb0 0x7f202c159c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f202c06b590 tx=0x7f201c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fe32afd76b75a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2030a3c700 1 -- 172.21.15.3:0/1329456257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00b8f0 con 0x7f202c06adb0 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2030a3c700 1 -- 172.21.15.3:0/1329456257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c00ba50 con 0x7f202c06adb0 2023-10-06T04:52:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2030a3c700 1 -- 172.21.15.3:0/1329456257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00bdf0 con 0x7f202c06adb0 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 -- 172.21.15.3:0/1329456257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06adb0 msgr2=0x7f202c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 --2- 172.21.15.3:0/1329456257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06adb0 0x7f202c159c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f202c06b590 tx=0x7f201c009250 comp rx=0 tx=0).stop 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 -- 172.21.15.3:0/1329456257 shutdown_connections 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 --2- 172.21.15.3:0/1329456257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c0ae9d0 0x7f202c06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 --2- 172.21.15.3:0/1329456257 >> v2:172.21.15.153:3300/0 conn(0x7f202c06a340 0x7f202c1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 --2- 172.21.15.3:0/1329456257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06adb0 0x7f202c159c20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 -- 172.21.15.3:0/1329456257 >> 172.21.15.3:0/1329456257 conn(0x7f202c067a40 msgr2=0x7f202c0ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 -- 172.21.15.3:0/1329456257 shutdown_connections 2023-10-06T04:52:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.020+0000 7f2033ca2700 1 -- 172.21.15.3:0/1329456257 wait complete. 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 Processor -- start 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 -- start start 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c06a340 0x7f202c15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0ae9d0 0x7f202c14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f202c15af30 0x7f202c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c15be20 con 0x7f202c0ae9d0 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c15b8f0 con 0x7f202c15af30 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f2033ca2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0b1a50 con 0x7f202c06a340 2023-10-06T04:52:29.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f203123d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0ae9d0 0x7f202c14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f203123d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0ae9d0 0x7f202c14b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33328/0 (socket says 172.21.15.3:33328) 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.022+0000 7f203123d700 1 -- 172.21.15.3:0/2320588152 learned_addr learned my addr 172.21.15.3:0/2320588152 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f2031a3e700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c06a340 0x7f202c15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203223f700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f202c15af30 0x7f202c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203123d700 1 -- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c06a340 msgr2=0x7f202c15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203123d700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c06a340 0x7f202c15a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203123d700 1 -- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f202c15af30 msgr2=0x7f202c15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203123d700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f202c15af30 0x7f202c15b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203123d700 1 -- 172.21.15.3:0/2320588152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c0094f0 con 0x7f202c0ae9d0 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203223f700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f202c15af30 0x7f202c15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f203123d700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0ae9d0 0x7f202c14b9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f201c006fd0 tx=0x7f201c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f2021ffb700 1 -- 172.21.15.3:0/2320588152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c011e20 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f2021ffb700 1 -- 172.21.15.3:0/2320588152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c01a630 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c15ad20 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f2021ffb700 1 -- 172.21.15.3:0/2320588152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c01a9f0 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.023+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c14bf20 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.024+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c06c000 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.025+0000 7f2021ffb700 1 -- 172.21.15.3:0/2320588152 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f201c02d440 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.025+0000 7f2021ffb700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2010066f30 0x7f20100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.025+0000 7f2021ffb700 1 -- 172.21.15.3:0/2320588152 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f201c0861f0 con 0x7f202c0ae9d0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.025+0000 7f2031a3e700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2010066f30 0x7f20100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.026+0000 7f2031a3e700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2010066f30 0x7f20100693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f202400e5f0 tx=0x7f20240076d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.029+0000 7f2021ffb700 1 -- 172.21.15.3:0/2320588152 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f201c053300 con 0x7f202c0ae9d0 2023-10-06T04:52:29.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.223+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f202c0b1b90 con 0x7f202c0ae9d0 2023-10-06T04:52:29.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.223+0000 7f2021ffb700 1 -- 172.21.15.3:0/2320588152 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f201c056920 con 0x7f202c0ae9d0 2023-10-06T04:52:29.225 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2010066f30 msgr2=0x7f20100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2010066f30 0x7f20100693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f202400e5f0 tx=0x7f20240076d0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0ae9d0 msgr2=0x7f202c14b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0ae9d0 0x7f202c14b9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f201c006fd0 tx=0x7f201c017000 comp rx=0 tx=0).stop 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 shutdown_connections 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2010066f30 0x7f20100693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f202c15af30 0x7f202c15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f202c06a340 0x7f202c15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 --2- 172.21.15.3:0/2320588152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c0ae9d0 0x7f202c14b9e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.225+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 >> 172.21.15.3:0/2320588152 conn(0x7f202c067a40 msgr2=0x7f202c0a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:29.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.226+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 shutdown_connections 2023-10-06T04:52:29.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.226+0000 7f2033ca2700 1 -- 172.21.15.3:0/2320588152 wait complete. 2023-10-06T04:52:29.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:52:29.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.365+0000 7faba3531700 1 Processor -- start 2023-10-06T04:52:29.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.366+0000 7faba3531700 1 -- start start 2023-10-06T04:52:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.366+0000 7faba3531700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.366+0000 7faba3531700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c15a290 con 0x7fab9c0cba50 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.366+0000 7faba3531700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c0d0020 con 0x7fab9c15d540 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.366+0000 7faba3531700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c159f40 con 0x7fab9c150690 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7faba0acc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fab9c0cba50 0x7fab9c15a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35962/0 (socket says 172.21.15.3:35962) 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7faba0acc700 1 -- 172.21.15.3:0/805334263 learned_addr learned my addr 172.21.15.3:0/805334263 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7faba12cd700 1 --2- 172.21.15.3:0/805334263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7fab9bfff700 1 -- 172.21.15.3:0/805334263 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482625349 0 0) 0x7fab9c15a290 con 0x7fab9c0cba50 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7fab9bfff700 1 -- 172.21.15.3:0/805334263 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab84003570 con 0x7fab9c0cba50 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7fab9bfff700 1 -- 172.21.15.3:0/805334263 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187792266 0 0) 0x7fab9c0d0020 con 0x7fab9c15d540 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7fab9bfff700 1 -- 172.21.15.3:0/805334263 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9c15a290 con 0x7fab9c15d540 2023-10-06T04:52:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7faba12cd700 1 -- 172.21.15.3:0/805334263 >> v1:172.21.15.153:6789/0 conn(0x7fab9c15d540 legacy=0x7fab9c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7faba12cd700 1 -- 172.21.15.3:0/805334263 >> v1:172.21.15.3:6789/0 conn(0x7fab9c0cba50 legacy=0x7fab9c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7faba12cd700 1 -- 172.21.15.3:0/805334263 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c15dcb0 con 0x7fab9c150690 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7faba12cd700 1 --2- 172.21.15.3:0/805334263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c150a60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab94009910 tx=0x7fab940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2028a62c18c93101 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7fab9bfff700 1 -- 172.21.15.3:0/805334263 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94011630 con 0x7fab9c150690 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7fab9bfff700 1 -- 172.21.15.3:0/805334263 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab94011790 con 0x7fab9c150690 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.367+0000 7fab9bfff700 1 -- 172.21.15.3:0/805334263 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94011b30 con 0x7fab9c150690 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.368+0000 7faba3531700 1 -- 172.21.15.3:0/805334263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 msgr2=0x7fab9c150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.368+0000 7faba3531700 1 --2- 172.21.15.3:0/805334263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c150a60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab94009910 tx=0x7fab940091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.368+0000 7faba3531700 1 -- 172.21.15.3:0/805334263 shutdown_connections 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.368+0000 7faba3531700 1 --2- 172.21.15.3:0/805334263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c150a60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.368+0000 7faba3531700 1 -- 172.21.15.3:0/805334263 >> 172.21.15.3:0/805334263 conn(0x7fab9c070310 msgr2=0x7fab9c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.368+0000 7faba3531700 1 -- 172.21.15.3:0/805334263 shutdown_connections 2023-10-06T04:52:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.368+0000 7faba3531700 1 -- 172.21.15.3:0/805334263 wait complete. 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.369+0000 7faba3531700 1 Processor -- start 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.369+0000 7faba3531700 1 -- start start 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba3531700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c0cba50 0x7fab9c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba3531700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba3531700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab9c15d540 0x7fab9c151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba3531700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0c9fb0 con 0x7fab9c0cba50 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba3531700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0c9400 con 0x7fab9c15d540 2023-10-06T04:52:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba3531700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0c9680 con 0x7fab9c150690 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c0cba50 0x7fab9c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c0cba50 0x7fab9c0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33344/0 (socket says 172.21.15.3:33344) 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 -- 172.21.15.3:0/3629977763 learned_addr learned my addr 172.21.15.3:0/3629977763 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba0acc700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba1ace700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab9c15d540 0x7fab9c151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 -- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 msgr2=0x7fab9c144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 -- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab9c15d540 msgr2=0x7fab9c151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab9c15d540 0x7fab9c151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 -- 172.21.15.3:0/3629977763 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab94009420 con 0x7fab9c0cba50 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.370+0000 7faba12cd700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c0cba50 0x7fab9c0b3890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab94009fd0 tx=0x7fab94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.371+0000 7fab997fa700 1 -- 172.21.15.3:0/3629977763 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9401f070 con 0x7fab9c0cba50 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.371+0000 7fab997fa700 1 -- 172.21.15.3:0/3629977763 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab94002f70 con 0x7fab9c0cba50 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.371+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c0c9960 con 0x7fab9c0cba50 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.371+0000 7fab997fa700 1 -- 172.21.15.3:0/3629977763 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94023410 con 0x7fab9c0cba50 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.371+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c0b3dd0 con 0x7fab9c0cba50 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.372+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c04c380 con 0x7fab9c0cba50 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.372+0000 7fab997fa700 1 -- 172.21.15.3:0/3629977763 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab940193b0 con 0x7fab9c0cba50 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.372+0000 7fab997fa700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fab88066f30 0x7fab880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.372+0000 7fab997fa700 1 -- 172.21.15.3:0/3629977763 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab94085870 con 0x7fab9c0cba50 2023-10-06T04:52:29.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.373+0000 7faba0acc700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fab88066f30 0x7fab880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.373+0000 7faba0acc700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fab88066f30 0x7fab880693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab9c153a60 tx=0x7fab90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.377+0000 7fab997fa700 1 -- 172.21.15.3:0/3629977763 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab94052980 con 0x7fab9c0cba50 2023-10-06T04:52:29.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.575+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab9c15df90 con 0x7fab9c0cba50 2023-10-06T04:52:29.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.576+0000 7fab997fa700 1 -- 172.21.15.3:0/3629977763 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab94028020 con 0x7fab9c0cba50 2023-10-06T04:52:29.578 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:52:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fab88066f30 msgr2=0x7fab880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fab88066f30 0x7fab880693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab9c153a60 tx=0x7fab90009040 comp rx=0 tx=0).stop 2023-10-06T04:52:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c0cba50 msgr2=0x7fab9c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c0cba50 0x7fab9c0b3890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab94009fd0 tx=0x7fab94017000 comp rx=0 tx=0).stop 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 shutdown_connections 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fab88066f30 0x7fab880693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab9c15d540 0x7fab9c151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c0cba50 0x7fab9c0b3890 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 --2- 172.21.15.3:0/3629977763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab9c150690 0x7fab9c144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 >> 172.21.15.3:0/3629977763 conn(0x7fab9c070310 msgr2=0x7fab9c0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 shutdown_connections 2023-10-06T04:52:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.578+0000 7faba3531700 1 -- 172.21.15.3:0/3629977763 wait complete. 2023-10-06T04:52:29.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:52:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.714+0000 7fecb4084700 1 Processor -- start 2023-10-06T04:52:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb4084700 1 -- start start 2023-10-06T04:52:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb4084700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fecac0cfb80 0x7fecac0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb4084700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb4084700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac15d510 con 0x7fecac0cef10 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb4084700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecac15a480 con 0x7fecac0cfb80 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb4084700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac15a760 con 0x7fecac0afad0 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb1e20700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fecac0cfb80 0x7fecac0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb2621700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fecac0cef10 0x7fecac15da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35972/0 (socket says 172.21.15.3:35972) 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.715+0000 7fecb2621700 1 -- 172.21.15.3:0/2146058062 learned_addr learned my addr 172.21.15.3:0/2146058062 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb161f700 1 --2- 172.21.15.3:0/2146058062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb1e20700 1 -- 172.21.15.3:0/2146058062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 msgr2=0x7fecac0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb1e20700 1 --2- 172.21.15.3:0/2146058062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb1e20700 1 -- 172.21.15.3:0/2146058062 >> v1:172.21.15.3:6789/0 conn(0x7fecac0cef10 legacy=0x7fecac15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb1e20700 1 -- 172.21.15.3:0/2146058062 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac15aa40 con 0x7fecac0cfb80 2023-10-06T04:52:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb0e1e700 1 -- 172.21.15.3:0/2146058062 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417480861 0 0) 0x7fecac15d510 con 0x7fecac0cef10 2023-10-06T04:52:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb1e20700 1 --2- 172.21.15.3:0/2146058062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cfb80 0x7fecac0af590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0098e0 tx=0x7fec9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c70aded219f9df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb0e1e700 1 -- 172.21.15.3:0/2146058062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00b640 con 0x7fecac0cfb80 2023-10-06T04:52:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb0e1e700 1 -- 172.21.15.3:0/2146058062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c00b7a0 con 0x7fecac0cfb80 2023-10-06T04:52:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.716+0000 7fecb0e1e700 1 -- 172.21.15.3:0/2146058062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00bb40 con 0x7fecac0cfb80 2023-10-06T04:52:29.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 -- 172.21.15.3:0/2146058062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cfb80 msgr2=0x7fecac0af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 --2- 172.21.15.3:0/2146058062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cfb80 0x7fecac0af590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0098e0 tx=0x7fec9c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 -- 172.21.15.3:0/2146058062 shutdown_connections 2023-10-06T04:52:29.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 --2- 172.21.15.3:0/2146058062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cfb80 0x7fecac0af590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 --2- 172.21.15.3:0/2146058062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 -- 172.21.15.3:0/2146058062 >> 172.21.15.3:0/2146058062 conn(0x7fecac070310 msgr2=0x7fecac070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 -- 172.21.15.3:0/2146058062 shutdown_connections 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.717+0000 7fecb4084700 1 -- 172.21.15.3:0/2146058062 wait complete. 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.718+0000 7fecb4084700 1 Processor -- start 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.718+0000 7fecb4084700 1 -- start start 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.718+0000 7fecb4084700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb4084700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cef10 0x7fecac0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb4084700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac0cfb80 0x7fecac0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb4084700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0b1d30 con 0x7fecac0cef10 2023-10-06T04:52:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb4084700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0b2010 con 0x7fecac0cfb80 2023-10-06T04:52:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb4084700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0b22f0 con 0x7fecac0afad0 2023-10-06T04:52:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cef10 0x7fecac0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cef10 0x7fecac0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33362/0 (socket says 172.21.15.3:33362) 2023-10-06T04:52:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 -- 172.21.15.3:0/113126790 learned_addr learned my addr 172.21.15.3:0/113126790 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb2621700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac0cfb80 0x7fecac0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb1e20700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 -- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 msgr2=0x7fecac0b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 -- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac0cfb80 msgr2=0x7fecac0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac0cfb80 0x7fecac0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 -- 172.21.15.3:0/113126790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c009420 con 0x7fecac0cef10 2023-10-06T04:52:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb161f700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cef10 0x7fecac0c7e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feca800ec80 tx=0x7feca80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7feca1ffb700 1 -- 172.21.15.3:0/113126790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca800c9d0 con 0x7fecac0cef10 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7feca1ffb700 1 -- 172.21.15.3:0/113126790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca800cb30 con 0x7fecac0cef10 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac0b25d0 con 0x7fecac0cef10 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7feca1ffb700 1 -- 172.21.15.3:0/113126790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8013470 con 0x7fecac0cef10 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.719+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac0b5e90 con 0x7fecac0cef10 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.720+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac04c380 con 0x7fecac0cef10 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.721+0000 7feca1ffb700 1 -- 172.21.15.3:0/113126790 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feca8013820 con 0x7fecac0cef10 2023-10-06T04:52:29.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.721+0000 7feca1ffb700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec90066f30 0x7fec900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:29.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.721+0000 7feca1ffb700 1 -- 172.21.15.3:0/113126790 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feca8084a00 con 0x7fecac0cef10 2023-10-06T04:52:29.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.721+0000 7fecb1e20700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec90066f30 0x7fec900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:29.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.722+0000 7fecb1e20700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec90066f30 0x7fec900693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0093f0 tx=0x7fec9c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:29.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.726+0000 7feca1ffb700 1 -- 172.21.15.3:0/113126790 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feca8051b10 con 0x7fecac0cef10 2023-10-06T04:52:29.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.918+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fecac0b6210 con 0x7fecac0cef10 2023-10-06T04:52:29.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.918+0000 7feca1ffb700 1 -- 172.21.15.3:0/113126790 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7feca8055130 con 0x7fecac0cef10 2023-10-06T04:52:29.920 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:52:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec90066f30 msgr2=0x7fec900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec90066f30 0x7fec900693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0093f0 tx=0x7fec9c017000 comp rx=0 tx=0).stop 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cef10 msgr2=0x7fecac0c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cef10 0x7fecac0c7e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feca800ec80 tx=0x7feca80091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 shutdown_connections 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec90066f30 0x7fec900693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac0cfb80 0x7fecac0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fecac0afad0 0x7fecac0b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 --2- 172.21.15.3:0/113126790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecac0cef10 0x7fecac0c7e90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 >> 172.21.15.3:0/113126790 conn(0x7fecac070310 msgr2=0x7fecac1569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.920+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 shutdown_connections 2023-10-06T04:52:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:29.921+0000 7fecb4084700 1 -- 172.21.15.3:0/113126790 wait complete. 2023-10-06T04:52:29.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:30.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.051+0000 7f5d68050700 1 Processor -- start 2023-10-06T04:52:30.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.051+0000 7f5d68050700 1 -- start start 2023-10-06T04:52:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d68050700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d6015cb50 0x7f5d600b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d68050700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5d600cd950 0x7f5d600cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d68050700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600b0ce0 con 0x7f5d600b0590 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d68050700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d600cb820 con 0x7f5d600cd950 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d68050700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b0a00 con 0x7f5d6015cb50 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d655eb700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5d600b0590 0x7f5d600b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35988/0 (socket says 172.21.15.3:35988) 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d655eb700 1 -- 172.21.15.3:0/929756165 learned_addr learned my addr 172.21.15.3:0/929756165 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d665ed700 1 --2- 172.21.15.3:0/929756165 >> v2:172.21.15.153:3300/0 conn(0x7f5d600cd950 0x7f5d600cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d65dec700 1 --2- 172.21.15.3:0/929756165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d6015cb50 0x7f5d600b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.052+0000 7f5d64dea700 1 -- 172.21.15.3:0/929756165 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579081275 0 0) 0x7f5d600b0ce0 con 0x7f5d600b0590 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d64dea700 1 -- 172.21.15.3:0/929756165 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d48003570 con 0x7f5d600b0590 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d65dec700 1 -- 172.21.15.3:0/929756165 >> v2:172.21.15.153:3300/0 conn(0x7f5d600cd950 msgr2=0x7f5d600cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d65dec700 1 --2- 172.21.15.3:0/929756165 >> v2:172.21.15.153:3300/0 conn(0x7f5d600cd950 0x7f5d600cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d65dec700 1 -- 172.21.15.3:0/929756165 >> v1:172.21.15.3:6789/0 conn(0x7f5d600b0590 legacy=0x7f5d600b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d64dea700 1 -- 172.21.15.3:0/929756165 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2155639571 0 0) 0x7f5d48003570 con 0x7f5d600b0590 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d65dec700 1 -- 172.21.15.3:0/929756165 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600cb1d0 con 0x7f5d6015cb50 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d665ed700 1 --2- 172.21.15.3:0/929756165 >> v2:172.21.15.153:3300/0 conn(0x7f5d600cd950 0x7f5d600cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d65dec700 1 --2- 172.21.15.3:0/929756165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d6015cb50 0x7f5d600b0050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098d0 tx=0x7f5d58009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c7313ae7c8e31c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d64dea700 1 -- 172.21.15.3:0/929756165 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54001470 con 0x7f5d6015cb50 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d64dea700 1 -- 172.21.15.3:0/929756165 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58011420 con 0x7f5d6015cb50 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.053+0000 7f5d64dea700 1 -- 172.21.15.3:0/929756165 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580117a0 con 0x7f5d6015cb50 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 -- 172.21.15.3:0/929756165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d6015cb50 msgr2=0x7f5d600b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 --2- 172.21.15.3:0/929756165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d6015cb50 0x7f5d600b0050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098d0 tx=0x7f5d58009120 comp rx=0 tx=0).stop 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 -- 172.21.15.3:0/929756165 shutdown_connections 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 --2- 172.21.15.3:0/929756165 >> v2:172.21.15.153:3300/0 conn(0x7f5d600cd950 0x7f5d600cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 --2- 172.21.15.3:0/929756165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d6015cb50 0x7f5d600b0050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 -- 172.21.15.3:0/929756165 >> 172.21.15.3:0/929756165 conn(0x7f5d60070310 msgr2=0x7f5d60070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 -- 172.21.15.3:0/929756165 shutdown_connections 2023-10-06T04:52:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.054+0000 7f5d68050700 1 -- 172.21.15.3:0/929756165 wait complete. 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.055+0000 7f5d68050700 1 Processor -- start 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.055+0000 7f5d68050700 1 -- start start 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d68050700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600b0590 0x7f5d60146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d68050700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d600cd950 0x7f5d601514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d68050700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015cb50 0x7f5d600b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d68050700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d601694b0 con 0x7f5d600cd950 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d68050700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600c6d60 con 0x7f5d6015cb50 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d68050700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60146d90 con 0x7f5d600b0590 2023-10-06T04:52:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d600cd950 0x7f5d601514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d600cd950 0x7f5d601514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33374/0 (socket says 172.21.15.3:33374) 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 -- 172.21.15.3:0/608948298 learned_addr learned my addr 172.21.15.3:0/608948298 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d665ed700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015cb50 0x7f5d600b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d65dec700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600b0590 0x7f5d60146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 -- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600b0590 msgr2=0x7f5d60146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600b0590 0x7f5d60146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 -- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015cb50 msgr2=0x7f5d600b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015cb50 0x7f5d600b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 -- 172.21.15.3:0/608948298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d50009a20 con 0x7f5d600cd950 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d665ed700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015cb50 0x7f5d600b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d655eb700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d600cd950 0x7f5d601514a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400cb90 tx=0x7f5d54001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d5dffb700 1 -- 172.21.15.3:0/608948298 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54007150 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d5dffb700 1 -- 172.21.15.3:0/608948298 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540072b0 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d580093c0 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.056+0000 7f5d5dffb700 1 -- 172.21.15.3:0/608948298 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54007670 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.057+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d600c6430 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.057+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d60071680 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.058+0000 7f5d5dffb700 1 -- 172.21.15.3:0/608948298 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d54004a40 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.058+0000 7f5d5dffb700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5d4c066f30 0x7f5d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.058+0000 7f5d5dffb700 1 -- 172.21.15.3:0/608948298 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d54083f10 con 0x7f5d600cd950 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.059+0000 7f5d65dec700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5d4c066f30 0x7f5d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.059+0000 7f5d65dec700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5d4c066f30 0x7f5d4c0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5d580030a0 tx=0x7f5d58016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.063+0000 7f5d5dffb700 1 -- 172.21.15.3:0/608948298 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d540510a0 con 0x7f5d600cd950 2023-10-06T04:52:30.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.300+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d600cb4b0 con 0x7f5d600cd950 2023-10-06T04:52:30.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.301+0000 7f5d5dffb700 1 -- 172.21.15.3:0/608948298 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7f5d54018020 con 0x7f5d600cd950 2023-10-06T04:52:30.303 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:30.303 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5d4c066f30 msgr2=0x7f5d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5d4c066f30 0x7f5d4c0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5d580030a0 tx=0x7f5d58016000 comp rx=0 tx=0).stop 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d600cd950 msgr2=0x7f5d601514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d600cd950 0x7f5d601514a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400cb90 tx=0x7f5d54001aa0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 shutdown_connections 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5d4c066f30 0x7f5d4c0693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015cb50 0x7f5d600b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600b0590 0x7f5d60146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 --2- 172.21.15.3:0/608948298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d600cd950 0x7f5d601514a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 >> 172.21.15.3:0/608948298 conn(0x7f5d60070310 msgr2=0x7f5d600b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:30.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 shutdown_connections 2023-10-06T04:52:30.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.303+0000 7f5d68050700 1 -- 172.21.15.3:0/608948298 wait complete. 2023-10-06T04:52:30.306 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:30.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:30.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.434+0000 7f07afb10700 1 Processor -- start 2023-10-06T04:52:30.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.434+0000 7f07afb10700 1 -- start start 2023-10-06T04:52:30.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07afb10700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07a8157700 0x7f07a80cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07afb10700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a8067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07afb10700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a80b2e50 con 0x7f07a8157ad0 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07afb10700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a8158290 con 0x7f07a8157700 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07afb10700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80b1fc0 con 0x7f07a80b1b10 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07ae0ad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a8067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07ad8ac700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07a8157700 0x7f07a80cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07ae0ad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a8067b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59848/0 (socket says 172.21.15.3:59848) 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.435+0000 7f07ae0ad700 1 -- 172.21.15.3:0/3091514292 learned_addr learned my addr 172.21.15.3:0/3091514292 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ae0ad700 1 -- 172.21.15.3:0/3091514292 >> v2:172.21.15.153:3300/0 conn(0x7f07a8157700 msgr2=0x7f07a80cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ae0ad700 1 --2- 172.21.15.3:0/3091514292 >> v2:172.21.15.153:3300/0 conn(0x7f07a8157700 0x7f07a80cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ae0ad700 1 -- 172.21.15.3:0/3091514292 >> v1:172.21.15.153:6789/0 conn(0x7f07a8157ad0 legacy=0x7f07a80b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ae0ad700 1 -- 172.21.15.3:0/3091514292 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a80b22a0 con 0x7f07a80b1b10 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ae0ad700 1 --2- 172.21.15.3:0/3091514292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a8067b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0798009f20 tx=0x7f0798009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572080ded15c549 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ac8aa700 1 -- 172.21.15.3:0/3091514292 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800e070 con 0x7f07a80b1b10 2023-10-06T04:52:30.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ac8aa700 1 -- 172.21.15.3:0/3091514292 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079800b9d0 con 0x7f07a80b1b10 2023-10-06T04:52:30.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07ac8aa700 1 -- 172.21.15.3:0/3091514292 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980092e0 con 0x7f07a80b1b10 2023-10-06T04:52:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07afb10700 1 -- 172.21.15.3:0/3091514292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 msgr2=0x7f07a8067b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.436+0000 7f07afb10700 1 --2- 172.21.15.3:0/3091514292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a8067b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0798009f20 tx=0x7f0798009840 comp rx=0 tx=0).stop 2023-10-06T04:52:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.437+0000 7f07afb10700 1 -- 172.21.15.3:0/3091514292 shutdown_connections 2023-10-06T04:52:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.437+0000 7f07afb10700 1 --2- 172.21.15.3:0/3091514292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a8067b30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.437+0000 7f07afb10700 1 --2- 172.21.15.3:0/3091514292 >> v2:172.21.15.153:3300/0 conn(0x7f07a8157700 0x7f07a80cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.437+0000 7f07afb10700 1 -- 172.21.15.3:0/3091514292 >> 172.21.15.3:0/3091514292 conn(0x7f07a80ce2c0 msgr2=0x7f07a80ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.437+0000 7f07afb10700 1 -- 172.21.15.3:0/3091514292 shutdown_connections 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.437+0000 7f07afb10700 1 -- 172.21.15.3:0/3091514292 wait complete. 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.438+0000 7f07afb10700 1 Processor -- start 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.438+0000 7f07afb10700 1 -- start start 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07afb10700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a816dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07afb10700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a8157700 0x7f07a80c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07afb10700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a8157ad0 0x7f07a8146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07afb10700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80c8550 con 0x7f07a8157700 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07afb10700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80c9f20 con 0x7f07a8157ad0 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07afb10700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a81525e0 con 0x7f07a80b1b10 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a8157700 0x7f07a80c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a8157700 0x7f07a80c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33382/0 (socket says 172.21.15.3:33382) 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 -- 172.21.15.3:0/1309849434 learned_addr learned my addr 172.21.15.3:0/1309849434 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ae0ad700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a8157ad0 0x7f07a8146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad8ac700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a816dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 -- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 msgr2=0x7f07a816dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a816dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 -- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a8157ad0 msgr2=0x7f07a8146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a8157ad0 0x7f07a8146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 -- 172.21.15.3:0/1309849434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798009ae0 con 0x7f07a8157700 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07ad0ab700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a8157700 0x7f07a80c99e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f079c00c510 tx=0x7f079c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07a5ffb700 1 -- 172.21.15.3:0/1309849434 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c011590 con 0x7f07a8157700 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07a5ffb700 1 -- 172.21.15.3:0/1309849434 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c011710 con 0x7f07a8157700 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a80c7f90 con 0x7f07a8157700 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.439+0000 7f07a5ffb700 1 -- 172.21.15.3:0/1309849434 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c011ad0 con 0x7f07a8157700 2023-10-06T04:52:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.440+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a806d620 con 0x7f07a8157700 2023-10-06T04:52:30.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.441+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a80644c0 con 0x7f07a8157700 2023-10-06T04:52:30.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.441+0000 7f07a5ffb700 1 -- 172.21.15.3:0/1309849434 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f079c01b3b0 con 0x7f07a8157700 2023-10-06T04:52:30.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.441+0000 7f07a5ffb700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0794066f30 0x7f07940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.441+0000 7f07a5ffb700 1 -- 172.21.15.3:0/1309849434 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f079c083a50 con 0x7f07a8157700 2023-10-06T04:52:30.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.442+0000 7f07ad8ac700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0794066f30 0x7f07940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.442+0000 7f07ad8ac700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0794066f30 0x7f07940693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009db0 tx=0x7f07a00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.446+0000 7f07a5ffb700 1 -- 172.21.15.3:0/1309849434 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f079c050b60 con 0x7f07a8157700 2023-10-06T04:52:30.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.639+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07a806d460 con 0x7f07a8157700 2023-10-06T04:52:30.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.640+0000 7f07a5ffb700 1 -- 172.21.15.3:0/1309849434 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f079c017070 con 0x7f07a8157700 2023-10-06T04:52:30.641 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:30.641 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:30.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0794066f30 msgr2=0x7f07940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0794066f30 0x7f07940693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009db0 tx=0x7f07a00091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a8157700 msgr2=0x7f07a80c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a8157700 0x7f07a80c99e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f079c00c510 tx=0x7f079c00c820 comp rx=0 tx=0).stop 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 shutdown_connections 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0794066f30 0x7f07940693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a8157ad0 0x7f07a8146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07a80b1b10 0x7f07a816dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 --2- 172.21.15.3:0/1309849434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a8157700 0x7f07a80c99e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 >> 172.21.15.3:0/1309849434 conn(0x7f07a80ce2c0 msgr2=0x7f07a8152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 shutdown_connections 2023-10-06T04:52:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.642+0000 7f07afb10700 1 -- 172.21.15.3:0/1309849434 wait complete. 2023-10-06T04:52:30.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:52:15.735846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:52:17.015278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:52:30.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:30.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb8962c3700 1 Processor -- start 2023-10-06T04:52:30.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb8962c3700 1 -- start start 2023-10-06T04:52:30.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb8962c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb890150690 0x7fb890150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb8962c3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8900cba50 0x7fb89015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb8962c3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89015da80 con 0x7fb89015a720 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb8962c3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89015a290 con 0x7fb890150690 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb8962c3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900d0020 con 0x7fb8900cba50 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.777+0000 7fb88ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb890150690 0x7fb890150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb894860700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb89015a720 0x7fb8900af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35994/0 (socket says 172.21.15.3:35994) 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb894860700 1 -- 172.21.15.3:0/1065008123 learned_addr learned my addr 172.21.15.3:0/1065008123 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88f7fe700 1 --2- 172.21.15.3:0/1065008123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8900cba50 0x7fb89015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88ffff700 1 -- 172.21.15.3:0/1065008123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8900cba50 msgr2=0x7fb89015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88ffff700 1 --2- 172.21.15.3:0/1065008123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8900cba50 0x7fb89015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88ffff700 1 -- 172.21.15.3:0/1065008123 >> v1:172.21.15.3:6789/0 conn(0x7fb89015a720 legacy=0x7fb8900af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88effd700 1 -- 172.21.15.3:0/1065008123 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741457834 0 0) 0x7fb89015da80 con 0x7fb89015a720 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88ffff700 1 -- 172.21.15.3:0/1065008123 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890159f40 con 0x7fb890150690 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88f7fe700 1 --2- 172.21.15.3:0/1065008123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8900cba50 0x7fb89015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88ffff700 1 --2- 172.21.15.3:0/1065008123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb890150690 0x7fb890150a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb8800098e0 tx=0x7fb8800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0c5ac4ac072893 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88effd700 1 -- 172.21.15.3:0/1065008123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009e80 con 0x7fb890150690 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88effd700 1 -- 172.21.15.3:0/1065008123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000b860 con 0x7fb890150690 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.778+0000 7fb88effd700 1 -- 172.21.15.3:0/1065008123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000bbe0 con 0x7fb890150690 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 -- 172.21.15.3:0/1065008123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb890150690 msgr2=0x7fb890150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 --2- 172.21.15.3:0/1065008123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb890150690 0x7fb890150a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb8800098e0 tx=0x7fb8800091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 -- 172.21.15.3:0/1065008123 shutdown_connections 2023-10-06T04:52:30.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 --2- 172.21.15.3:0/1065008123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb890150690 0x7fb890150a60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 --2- 172.21.15.3:0/1065008123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8900cba50 0x7fb89015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 -- 172.21.15.3:0/1065008123 >> 172.21.15.3:0/1065008123 conn(0x7fb890070310 msgr2=0x7fb890070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:30.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 -- 172.21.15.3:0/1065008123 shutdown_connections 2023-10-06T04:52:30.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.779+0000 7fb8962c3700 1 -- 172.21.15.3:0/1065008123 wait complete. 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.780+0000 7fb8962c3700 1 Processor -- start 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.780+0000 7fb8962c3700 1 -- start start 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb8962c3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8900cba50 0x7fb8901536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb8962c3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb890150690 0x7fb8901572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb8962c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89015a720 0x7fb8900d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb8962c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015d9a0 con 0x7fb89015a720 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb8962c3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890151790 con 0x7fb8900cba50 2023-10-06T04:52:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb8962c3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901546f0 con 0x7fb890150690 2023-10-06T04:52:30.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89015a720 0x7fb8900d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89015a720 0x7fb8900d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33406/0 (socket says 172.21.15.3:33406) 2023-10-06T04:52:30.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 -- 172.21.15.3:0/3549632870 learned_addr learned my addr 172.21.15.3:0/3549632870 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:30.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb88f7fe700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb890150690 0x7fb8901572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb88ffff700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8900cba50 0x7fb8901536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 -- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb890150690 msgr2=0x7fb8901572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb890150690 0x7fb8901572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 -- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8900cba50 msgr2=0x7fb8901536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8900cba50 0x7fb8901536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 -- 172.21.15.3:0/3549632870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880009420 con 0x7fb89015a720 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb88ffff700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8900cba50 0x7fb8901536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb894860700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89015a720 0x7fb8900d0560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb88800cc40 tx=0x7fb888001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb877fff700 1 -- 172.21.15.3:0/3549632870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88800a920 con 0x7fb89015a720 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb877fff700 1 -- 172.21.15.3:0/3549632870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88800aaa0 con 0x7fb89015a720 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb890150ef0 con 0x7fb89015a720 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.781+0000 7fb877fff700 1 -- 172.21.15.3:0/3549632870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88800ae60 con 0x7fb89015a720 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.782+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8901514b0 con 0x7fb89015a720 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.782+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89004c380 con 0x7fb89015a720 2023-10-06T04:52:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.783+0000 7fb877fff700 1 -- 172.21.15.3:0/3549632870 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb888007300 con 0x7fb89015a720 2023-10-06T04:52:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.783+0000 7fb877fff700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb87c066f30 0x7fb87c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.783+0000 7fb877fff700 1 -- 172.21.15.3:0/3549632870 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb888084a00 con 0x7fb89015a720 2023-10-06T04:52:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.783+0000 7fb88ffff700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb87c066f30 0x7fb87c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.784+0000 7fb88ffff700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb87c066f30 0x7fb87c0693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb880003080 tx=0x7fb880017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.787+0000 7fb877fff700 1 -- 172.21.15.3:0/3549632870 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb888051b90 con 0x7fb89015a720 2023-10-06T04:52:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.981+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb89015dcb0 con 0x7fb89015a720 2023-10-06T04:52:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.982+0000 7fb877fff700 1 -- 172.21.15.3:0/3549632870 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb8880551b0 con 0x7fb89015a720 2023-10-06T04:52:30.983 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb87c066f30 msgr2=0x7fb87c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb87c066f30 0x7fb87c0693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb880003080 tx=0x7fb880017000 comp rx=0 tx=0).stop 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89015a720 msgr2=0x7fb8900d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89015a720 0x7fb8900d0560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb88800cc40 tx=0x7fb888001970 comp rx=0 tx=0).stop 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 shutdown_connections 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb87c066f30 0x7fb87c0693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8900cba50 0x7fb8901536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89015a720 0x7fb8900d0560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 --2- 172.21.15.3:0/3549632870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb890150690 0x7fb8901572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 >> 172.21.15.3:0/3549632870 conn(0x7fb890070310 msgr2=0x7fb8900b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 shutdown_connections 2023-10-06T04:52:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:30.984+0000 7fb8962c3700 1 -- 172.21.15.3:0/3549632870 wait complete. 2023-10-06T04:52:30.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:31.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.117+0000 7f0706b6f700 1 Processor -- start 2023-10-06T04:52:31.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.117+0000 7f0706b6f700 1 -- start start 2023-10-06T04:52:31.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.117+0000 7f0706b6f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f070015ac90 0x7f07000b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.117+0000 7f0706b6f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f07000cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.117+0000 7f0706b6f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07000c9520 con 0x7f0700070310 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.117+0000 7f0706b6f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07000c9fa0 con 0x7f070015ac90 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.117+0000 7f0706b6f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070006aea0 con 0x7f07000ae9d0 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f070015ac90 0x7f07000b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f070015ac90 0x7f07000b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33416/0 (socket says 172.21.15.3:33416) 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 -- 172.21.15.3:0/4097122004 learned_addr learned my addr 172.21.15.3:0/4097122004 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070510c700 1 --2- 172.21.15.3:0/4097122004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f07000cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 -- 172.21.15.3:0/4097122004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 msgr2=0x7f07000cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 --2- 172.21.15.3:0/4097122004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f07000cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 -- 172.21.15.3:0/4097122004 >> v1:172.21.15.153:6789/0 conn(0x7f0700070310 legacy=0x7f070006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:31.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 -- 172.21.15.3:0/4097122004 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070006a290 con 0x7f070015ac90 2023-10-06T04:52:31.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070510c700 1 --2- 172.21.15.3:0/4097122004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f07000cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:31.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f070490b700 1 --2- 172.21.15.3:0/4097122004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f070015ac90 0x7f07000b1960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009f70 tx=0x7f06f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7f62bb985c916e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f06ff7fe700 1 -- 172.21.15.3:0/4097122004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800e070 con 0x7f070015ac90 2023-10-06T04:52:31.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f06ff7fe700 1 -- 172.21.15.3:0/4097122004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f800bb20 con 0x7f070015ac90 2023-10-06T04:52:31.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.118+0000 7f06ff7fe700 1 -- 172.21.15.3:0/4097122004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f80092e0 con 0x7f070015ac90 2023-10-06T04:52:31.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 -- 172.21.15.3:0/4097122004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f070015ac90 msgr2=0x7f07000b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 --2- 172.21.15.3:0/4097122004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f070015ac90 0x7f07000b1960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009f70 tx=0x7f06f8009840 comp rx=0 tx=0).stop 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 -- 172.21.15.3:0/4097122004 shutdown_connections 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 --2- 172.21.15.3:0/4097122004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f07000cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 --2- 172.21.15.3:0/4097122004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f070015ac90 0x7f07000b1960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 -- 172.21.15.3:0/4097122004 >> 172.21.15.3:0/4097122004 conn(0x7f0700067a40 msgr2=0x7f07000ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 -- 172.21.15.3:0/4097122004 shutdown_connections 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.119+0000 7f0706b6f700 1 -- 172.21.15.3:0/4097122004 wait complete. 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.120+0000 7f0706b6f700 1 Processor -- start 2023-10-06T04:52:31.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.120+0000 7f0706b6f700 1 -- start start 2023-10-06T04:52:31.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f0706b6f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0700070310 0x7f07000c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f0706b6f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f070016d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f0706b6f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f070015ac90 0x7f07000ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f0706b6f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000a9eb0 con 0x7f0700070310 2023-10-06T04:52:31.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f0706b6f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b0320 con 0x7f070015ac90 2023-10-06T04:52:31.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f0706b6f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000c61f0 con 0x7f07000ae9d0 2023-10-06T04:52:31.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0700070310 0x7f07000c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0700070310 0x7f07000c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33418/0 (socket says 172.21.15.3:33418) 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 -- 172.21.15.3:0/1370017801 learned_addr learned my addr 172.21.15.3:0/1370017801 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 -- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 msgr2=0x7f070016d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f06fffff700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f070016d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070510c700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f070015ac90 0x7f07000ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f070016d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 -- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f070015ac90 msgr2=0x7f07000ab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f070015ac90 0x7f07000ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 -- 172.21.15.3:0/1370017801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f8009ae0 con 0x7f0700070310 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f070490b700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0700070310 0x7f07000c5cb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b770 tx=0x7f06f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f06fcff9700 1 -- 172.21.15.3:0/1370017801 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8012b80 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f06fcff9700 1 -- 172.21.15.3:0/1370017801 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f8012ce0 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07000c8790 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.121+0000 7f06fcff9700 1 -- 172.21.15.3:0/1370017801 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f801d670 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.122+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07000a9bd0 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.122+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070006c000 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.123+0000 7f06fcff9700 1 -- 172.21.15.3:0/1370017801 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06f801daa0 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.123+0000 7f06fcff9700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.123+0000 7f06fcff9700 1 -- 172.21.15.3:0/1370017801 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f06f8085f10 con 0x7f0700070310 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.124+0000 7f06fffff700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.124+0000 7f06fffff700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06e0066f30 0x7f06e00693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f070016d290 tx=0x7f06f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.128+0000 7f06fcff9700 1 -- 172.21.15.3:0/1370017801 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06f8053020 con 0x7f0700070310 2023-10-06T04:52:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.320+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f070004c380 con 0x7f0700070310 2023-10-06T04:52:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.320+0000 7f06fcff9700 1 -- 172.21.15.3:0/1370017801 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06f8056640 con 0x7f0700070310 2023-10-06T04:52:31.322 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06e0066f30 msgr2=0x7f06e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06e0066f30 0x7f06e00693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f070016d290 tx=0x7f06f0009040 comp rx=0 tx=0).stop 2023-10-06T04:52:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0700070310 msgr2=0x7f07000c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0700070310 0x7f07000c5cb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b770 tx=0x7f06f8017000 comp rx=0 tx=0).stop 2023-10-06T04:52:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 shutdown_connections 2023-10-06T04:52:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f070015ac90 0x7f07000ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0700070310 0x7f07000c5cb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 --2- 172.21.15.3:0/1370017801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07000ae9d0 0x7f070016d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 >> 172.21.15.3:0/1370017801 conn(0x7f0700067a40 msgr2=0x7f07000b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 shutdown_connections 2023-10-06T04:52:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.322+0000 7f0706b6f700 1 -- 172.21.15.3:0/1370017801 wait complete. 2023-10-06T04:52:31.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:52:31.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.456+0000 7ff5c7c76700 1 Processor -- start 2023-10-06T04:52:31.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.456+0000 7ff5c7c76700 1 -- start start 2023-10-06T04:52:31.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.456+0000 7ff5c7c76700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff5c0150690 0x7ff5c0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.456+0000 7ff5c7c76700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff5c00cba50 0x7ff5c015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c7c76700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c015da80 con 0x7ff5c015a720 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c7c76700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c015a290 con 0x7ff5c00cba50 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c7c76700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00d0020 con 0x7ff5c0150690 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c5211700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff5c00cba50 0x7ff5c015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c6213700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff5c015a720 0x7ff5c00af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36006/0 (socket says 172.21.15.3:36006) 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c6213700 1 -- 172.21.15.3:0/866390876 learned_addr learned my addr 172.21.15.3:0/866390876 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c5a12700 1 --2- 172.21.15.3:0/866390876 >> v2:172.21.15.153:3300/0 conn(0x7ff5c0150690 0x7ff5c0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c5211700 1 -- 172.21.15.3:0/866390876 >> v2:172.21.15.153:3300/0 conn(0x7ff5c0150690 msgr2=0x7ff5c0150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c5211700 1 --2- 172.21.15.3:0/866390876 >> v2:172.21.15.153:3300/0 conn(0x7ff5c0150690 0x7ff5c0150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c5211700 1 -- 172.21.15.3:0/866390876 >> v1:172.21.15.3:6789/0 conn(0x7ff5c015a720 legacy=0x7ff5c00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c5211700 1 -- 172.21.15.3:0/866390876 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0159f40 con 0x7ff5c00cba50 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c4a10700 1 -- 172.21.15.3:0/866390876 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973873230 0 0) 0x7ff5c015da80 con 0x7ff5c015a720 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c5211700 1 --2- 172.21.15.3:0/866390876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c015d540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00cdf80 tx=0x7ff5b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d04239de1a93b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c4a10700 1 -- 172.21.15.3:0/866390876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400b6b0 con 0x7ff5c00cba50 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c4a10700 1 -- 172.21.15.3:0/866390876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400b810 con 0x7ff5c00cba50 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.457+0000 7ff5c4a10700 1 -- 172.21.15.3:0/866390876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400bbb0 con 0x7ff5c00cba50 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 -- 172.21.15.3:0/866390876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 msgr2=0x7ff5c015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/866390876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c015d540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00cdf80 tx=0x7ff5b40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 -- 172.21.15.3:0/866390876 shutdown_connections 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/866390876 >> v2:172.21.15.153:3300/0 conn(0x7ff5c0150690 0x7ff5c0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/866390876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c015d540 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 -- 172.21.15.3:0/866390876 >> 172.21.15.3:0/866390876 conn(0x7ff5c0070310 msgr2=0x7ff5c0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:31.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 -- 172.21.15.3:0/866390876 shutdown_connections 2023-10-06T04:52:31.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.458+0000 7ff5c7c76700 1 -- 172.21.15.3:0/866390876 wait complete. 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.459+0000 7ff5c7c76700 1 Processor -- start 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.459+0000 7ff5c7c76700 1 -- start start 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c7c76700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c7c76700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5c0150690 0x7ff5c016d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c7c76700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5c015a720 0x7ff5c0143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c7c76700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c016e2c0 con 0x7ff5c00cba50 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c7c76700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c016d830 con 0x7ff5c0150690 2023-10-06T04:52:31.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c7c76700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c016da20 con 0x7ff5c015a720 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c00b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33432/0 (socket says 172.21.15.3:33432) 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 -- 172.21.15.3:0/443174455 learned_addr learned my addr 172.21.15.3:0/443174455 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c6213700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5c015a720 0x7ff5c0143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5211700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5c0150690 0x7ff5c016d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 -- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5c015a720 msgr2=0x7ff5c0143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5c015a720 0x7ff5c0143d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 -- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5c0150690 msgr2=0x7ff5c016d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5c0150690 0x7ff5c016d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 -- 172.21.15.3:0/443174455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b4009420 con 0x7ff5c00cba50 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5211700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5c0150690 0x7ff5c016d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:31.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c5a12700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c00b3ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800e190 tx=0x7ff5b8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5bdffb700 1 -- 172.21.15.3:0/443174455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800ee30 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5bdffb700 1 -- 172.21.15.3:0/443174455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b8014420 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5bdffb700 1 -- 172.21.15.3:0/443174455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80147e0 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.460+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c016dd00 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.461+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c00c7690 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.462+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c00c74d0 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.462+0000 7ff5bdffb700 1 -- 172.21.15.3:0/443174455 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5b8014b90 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.462+0000 7ff5bdffb700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff5ac066f30 0x7ff5ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.462+0000 7ff5bdffb700 1 -- 172.21.15.3:0/443174455 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff5b8084ef0 con 0x7ff5c00cba50 2023-10-06T04:52:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.463+0000 7ff5c5211700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff5ac066f30 0x7ff5ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.463+0000 7ff5c5211700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff5ac066f30 0x7ff5ac0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40093f0 tx=0x7ff5b4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.467+0000 7ff5bdffb700 1 -- 172.21.15.3:0/443174455 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5b8052000 con 0x7ff5c00cba50 2023-10-06T04:52:31.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.660+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5c00c7820 con 0x7ff5c00cba50 2023-10-06T04:52:31.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.660+0000 7ff5bdffb700 1 -- 172.21.15.3:0/443174455 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5b8009a60 con 0x7ff5c00cba50 2023-10-06T04:52:31.661 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:52:31.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff5ac066f30 msgr2=0x7ff5ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff5ac066f30 0x7ff5ac0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40093f0 tx=0x7ff5b4017000 comp rx=0 tx=0).stop 2023-10-06T04:52:31.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 msgr2=0x7ff5c00b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c00b3ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800e190 tx=0x7ff5b8009920 comp rx=0 tx=0).stop 2023-10-06T04:52:31.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 shutdown_connections 2023-10-06T04:52:31.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff5ac066f30 0x7ff5ac0693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5c015a720 0x7ff5c0143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5c00cba50 0x7ff5c00b3ca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 --2- 172.21.15.3:0/443174455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5c0150690 0x7ff5c016d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 >> 172.21.15.3:0/443174455 conn(0x7ff5c0070310 msgr2=0x7ff5c00c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:31.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 shutdown_connections 2023-10-06T04:52:31.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.662+0000 7ff5c7c76700 1 -- 172.21.15.3:0/443174455 wait complete. 2023-10-06T04:52:31.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:52:31.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.797+0000 7fc1fd548700 1 Processor -- start 2023-10-06T04:52:31.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.797+0000 7fc1fd548700 1 -- start start 2023-10-06T04:52:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1fd548700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80b2450 0x7fc1f80b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1fd548700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1f80cb2b0 0x7fc1f80cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1fd548700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f815a000 con 0x7fc1f80caa10 2023-10-06T04:52:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1fd548700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f8159a80 con 0x7fc1f80cb2b0 2023-10-06T04:52:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1fd548700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80cfbb0 con 0x7fc1f80b2450 2023-10-06T04:52:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1f6ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80b2450 0x7fc1f80b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1f77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1f80cb2b0 0x7fc1f80cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1f6ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80b2450 0x7fc1f80b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59912/0 (socket says 172.21.15.3:59912) 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/1463437471 learned_addr learned my addr 172.21.15.3:0/1463437471 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1f5ffb700 1 -- 172.21.15.3:0/1463437471 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522965448 0 0) 0x7fc1f815a000 con 0x7fc1f80caa10 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.798+0000 7fc1f5ffb700 1 -- 172.21.15.3:0/1463437471 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e0003570 con 0x7fc1f80caa10 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/1463437471 >> v2:172.21.15.153:3300/0 conn(0x7fc1f80cb2b0 msgr2=0x7fc1f80cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f6ffd700 1 --2- 172.21.15.3:0/1463437471 >> v2:172.21.15.153:3300/0 conn(0x7fc1f80cb2b0 0x7fc1f80cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/1463437471 >> v1:172.21.15.153:6789/0 conn(0x7fc1f80caa10 legacy=0x7fc1f815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/1463437471 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80cfe90 con 0x7fc1f80b2450 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f77fe700 1 --2- 172.21.15.3:0/1463437471 >> v2:172.21.15.153:3300/0 conn(0x7fc1f80cb2b0 0x7fc1f80cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f6ffd700 1 --2- 172.21.15.3:0/1463437471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80b2450 0x7fc1f80b2820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009990 tx=0x7fc1e8009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d2c91e15de48d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f5ffb700 1 -- 172.21.15.3:0/1463437471 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8005c10 con 0x7fc1f80b2450 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f5ffb700 1 -- 172.21.15.3:0/1463437471 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e8005d70 con 0x7fc1f80b2450 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1f5ffb700 1 -- 172.21.15.3:0/1463437471 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8006240 con 0x7fc1f80b2450 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1fd548700 1 -- 172.21.15.3:0/1463437471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80b2450 msgr2=0x7fc1f80b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1fd548700 1 --2- 172.21.15.3:0/1463437471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80b2450 0x7fc1f80b2820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009990 tx=0x7fc1e8009220 comp rx=0 tx=0).stop 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1fd548700 1 -- 172.21.15.3:0/1463437471 shutdown_connections 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1fd548700 1 --2- 172.21.15.3:0/1463437471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80b2450 0x7fc1f80b2820 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1fd548700 1 --2- 172.21.15.3:0/1463437471 >> v2:172.21.15.153:3300/0 conn(0x7fc1f80cb2b0 0x7fc1f80cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.799+0000 7fc1fd548700 1 -- 172.21.15.3:0/1463437471 >> 172.21.15.3:0/1463437471 conn(0x7fc1f806ecc0 msgr2=0x7fc1f806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.800+0000 7fc1fd548700 1 -- 172.21.15.3:0/1463437471 shutdown_connections 2023-10-06T04:52:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.800+0000 7fc1fd548700 1 -- 172.21.15.3:0/1463437471 wait complete. 2023-10-06T04:52:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 Processor -- start 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 -- start start 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f80b2450 0x7fc1f814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80cb2b0 0x7fc1f8152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1f81532a0 0x7fc1f8150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f8153740 con 0x7fc1f80b2450 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81524a0 con 0x7fc1f81532a0 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.801+0000 7fc1fd548700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f814f5c0 con 0x7fc1f80cb2b0 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f80b2450 0x7fc1f814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f80b2450 0x7fc1f814f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33448/0 (socket says 172.21.15.3:33448) 2023-10-06T04:52:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/544618312 learned_addr learned my addr 172.21.15.3:0/544618312 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f67fc700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80cb2b0 0x7fc1f8152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f77fe700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1f81532a0 0x7fc1f8150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80cb2b0 msgr2=0x7fc1f8152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80cb2b0 0x7fc1f8152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1f81532a0 msgr2=0x7fc1f8150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1f81532a0 0x7fc1f8150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 -- 172.21.15.3:0/544618312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0009a20 con 0x7fc1f80b2450 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1f6ffd700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f80b2450 0x7fc1f814f080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009f80 tx=0x7fc1e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1deffd700 1 -- 172.21.15.3:0/544618312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8011a10 con 0x7fc1f80b2450 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1deffd700 1 -- 172.21.15.3:0/544618312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e8011b90 con 0x7fc1f80b2450 2023-10-06T04:52:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e8009480 con 0x7fc1f80b2450 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1deffd700 1 -- 172.21.15.3:0/544618312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e801b430 con 0x7fc1f80b2450 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.802+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f8151b70 con 0x7fc1f80b2450 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.803+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f81694b0 con 0x7fc1f80b2450 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.804+0000 7fc1deffd700 1 -- 172.21.15.3:0/544618312 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1e801b820 con 0x7fc1f80b2450 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.804+0000 7fc1deffd700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1e4066f30 0x7fc1e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.804+0000 7fc1deffd700 1 -- 172.21.15.3:0/544618312 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1e8086000 con 0x7fc1f80b2450 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.804+0000 7fc1f67fc700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1e4066f30 0x7fc1e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.805+0000 7fc1f67fc700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1e4066f30 0x7fc1e40693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8152cb0 tx=0x7fc1ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:31.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:31.808+0000 7fc1deffd700 1 -- 172.21.15.3:0/544618312 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1e8053110 con 0x7fc1f80b2450 2023-10-06T04:52:32.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.001+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1f804c380 con 0x7fc1f80b2450 2023-10-06T04:52:32.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.001+0000 7fc1deffd700 1 -- 172.21.15.3:0/544618312 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc1e802a020 con 0x7fc1f80b2450 2023-10-06T04:52:32.003 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:52:32.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1e4066f30 msgr2=0x7fc1e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1e4066f30 0x7fc1e40693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8152cb0 tx=0x7fc1ec009040 comp rx=0 tx=0).stop 2023-10-06T04:52:32.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f80b2450 msgr2=0x7fc1f814f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f80b2450 0x7fc1f814f080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009f80 tx=0x7fc1e8016000 comp rx=0 tx=0).stop 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 shutdown_connections 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1e4066f30 0x7fc1e40693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1f81532a0 0x7fc1f8150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f80b2450 0x7fc1f814f080 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 --2- 172.21.15.3:0/544618312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1f80cb2b0 0x7fc1f8152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 >> 172.21.15.3:0/544618312 conn(0x7fc1f806ecc0 msgr2=0x7fc1f80b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 shutdown_connections 2023-10-06T04:52:32.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.003+0000 7fc1fd548700 1 -- 172.21.15.3:0/544618312 wait complete. 2023-10-06T04:52:32.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:32.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.134+0000 7f24eb1c8700 1 Processor -- start 2023-10-06T04:52:32.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24eb1c8700 1 -- start start 2023-10-06T04:52:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24eb1c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f24e4068240 0x7f24e40cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24eb1c8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e40b2e50 con 0x7f24e40b1b10 2023-10-06T04:52:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24eb1c8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e4158290 con 0x7f24e4157700 2023-10-06T04:52:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24eb1c8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e40b1fc0 con 0x7f24e4068240 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24e9765700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f24e4068240 0x7f24e40cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24e3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f24e40b1b10 0x7f24e4067b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36012/0 (socket says 172.21.15.3:36012) 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.135+0000 7f24e3fff700 1 -- 172.21.15.3:0/1239360756 learned_addr learned my addr 172.21.15.3:0/1239360756 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e9765700 1 -- 172.21.15.3:0/1239360756 >> v1:172.21.15.153:6789/0 conn(0x7f24e4157700 legacy=0x7f24e40b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e9765700 1 -- 172.21.15.3:0/1239360756 >> v1:172.21.15.3:6789/0 conn(0x7f24e40b1b10 legacy=0x7f24e4067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e37fe700 1 -- 172.21.15.3:0/1239360756 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133773732 0 0) 0x7f24e40b2e50 con 0x7f24e40b1b10 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e9765700 1 -- 172.21.15.3:0/1239360756 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e40b22a0 con 0x7f24e4068240 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e9765700 1 --2- 172.21.15.3:0/1239360756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4068240 0x7f24e40cc270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009910 tx=0x7f24cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bc3149699f9837 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e37fe700 1 -- 172.21.15.3:0/1239360756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00b7e0 con 0x7f24e4068240 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e37fe700 1 -- 172.21.15.3:0/1239360756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc00b940 con 0x7f24e4068240 2023-10-06T04:52:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.136+0000 7f24e37fe700 1 -- 172.21.15.3:0/1239360756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00bce0 con 0x7f24e4068240 2023-10-06T04:52:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.137+0000 7f24eb1c8700 1 -- 172.21.15.3:0/1239360756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4068240 msgr2=0x7f24e40cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.137+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/1239360756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4068240 0x7f24e40cc270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009910 tx=0x7f24cc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.137+0000 7f24eb1c8700 1 -- 172.21.15.3:0/1239360756 shutdown_connections 2023-10-06T04:52:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.137+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/1239360756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4068240 0x7f24e40cc270 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.137+0000 7f24eb1c8700 1 -- 172.21.15.3:0/1239360756 >> 172.21.15.3:0/1239360756 conn(0x7f24e40ce2c0 msgr2=0x7f24e40ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.137+0000 7f24eb1c8700 1 -- 172.21.15.3:0/1239360756 shutdown_connections 2023-10-06T04:52:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.137+0000 7f24eb1c8700 1 -- 172.21.15.3:0/1239360756 wait complete. 2023-10-06T04:52:32.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.138+0000 7f24eb1c8700 1 Processor -- start 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.138+0000 7f24eb1c8700 1 -- start start 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e4068240 0x7f24e4159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e40b1b10 0x7f24e416a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e4157700 0x7f24e4168c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4141ab0 con 0x7f24e40b1b10 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e416a2d0 con 0x7f24e4157700 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e41696c0 con 0x7f24e4068240 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e40b1b10 0x7f24e416a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e40b1b10 0x7f24e416a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33468/0 (socket says 172.21.15.3:33468) 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 -- 172.21.15.3:0/140967392 learned_addr learned my addr 172.21.15.3:0/140967392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e8f64700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e4068240 0x7f24e4159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e9765700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e4157700 0x7f24e4168c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 -- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e4068240 msgr2=0x7f24e4159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e4068240 0x7f24e4159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 -- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e4157700 msgr2=0x7f24e4168c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e4157700 0x7f24e4168c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 -- 172.21.15.3:0/140967392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc009420 con 0x7f24e40b1b10 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e8f64700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e4068240 0x7f24e4159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e3fff700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e40b1b10 0x7f24e416a760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c810 tx=0x7f24d4007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e0ff9700 1 -- 172.21.15.3:0/140967392 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4005c00 con 0x7f24e40b1b10 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e0ff9700 1 -- 172.21.15.3:0/140967392 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d4005d60 con 0x7f24e40b1b10 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e41699a0 con 0x7f24e40b1b10 2023-10-06T04:52:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24e0ff9700 1 -- 172.21.15.3:0/140967392 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4007320 con 0x7f24e40b1b10 2023-10-06T04:52:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.139+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e4169f60 con 0x7f24e40b1b10 2023-10-06T04:52:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.140+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40644c0 con 0x7f24e40b1b10 2023-10-06T04:52:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.141+0000 7f24e0ff9700 1 -- 172.21.15.3:0/140967392 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24d401b400 con 0x7f24e40b1b10 2023-10-06T04:52:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.141+0000 7f24e0ff9700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24c4066f60 0x7f24c4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.141+0000 7f24e0ff9700 1 -- 172.21.15.3:0/140967392 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f24d4084140 con 0x7f24e40b1b10 2023-10-06T04:52:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.141+0000 7f24e8f64700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24c4066f60 0x7f24c4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.142+0000 7f24e8f64700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24c4066f60 0x7f24c4069410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24dc007fd0 tx=0x7f24dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.145+0000 7f24e0ff9700 1 -- 172.21.15.3:0/140967392 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24d40512d0 con 0x7f24e40b1b10 2023-10-06T04:52:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.383+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24e4071680 con 0x7f24e40b1b10 2023-10-06T04:52:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.383+0000 7f24e0ff9700 1 -- 172.21.15.3:0/140967392 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7f24d40548f0 con 0x7f24e40b1b10 2023-10-06T04:52:32.385 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:32.385 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24c4066f60 msgr2=0x7f24c4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24c4066f60 0x7f24c4069410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24dc007fd0 tx=0x7f24dc009040 comp rx=0 tx=0).stop 2023-10-06T04:52:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e40b1b10 msgr2=0x7f24e416a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e40b1b10 0x7f24e416a760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c810 tx=0x7f24d4007db0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 shutdown_connections 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f24c4066f60 0x7f24c4069410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e4157700 0x7f24e4168c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e4068240 0x7f24e4159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 --2- 172.21.15.3:0/140967392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e40b1b10 0x7f24e416a760 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 >> 172.21.15.3:0/140967392 conn(0x7f24e40ce2c0 msgr2=0x7f24e40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 shutdown_connections 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.385+0000 7f24eb1c8700 1 -- 172.21.15.3:0/140967392 wait complete. 2023-10-06T04:52:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:32.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:32.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.516+0000 7f0d2c853700 1 Processor -- start 2023-10-06T04:52:32.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.516+0000 7f0d2c853700 1 -- start start 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2c853700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d280cb3b0 0x7f0d280cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2c853700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0d281596d0 0x7f0d280cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2c853700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d280cf130 con 0x7f0d280cb780 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2c853700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d280ce520 con 0x7f0d281596d0 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2c853700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d280ce800 con 0x7f0d280cb3b0 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2759e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0d281596d0 0x7f0d280cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d26d9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d280cb3b0 0x7f0d280cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2759e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0d281596d0 0x7f0d280cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:52962/0 (socket says 172.21.15.3:52962) 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2759e700 1 -- 172.21.15.3:0/445525332 learned_addr learned my addr 172.21.15.3:0/445525332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2659c700 1 --1- 172.21.15.3:0/445525332 >> v1:172.21.15.153:6789/0 conn(0x7f0d280cb780 0x7f0d28158fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49894/0 (socket says 172.21.15.3:49894) 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d25d9b700 1 -- 172.21.15.3:0/445525332 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287601012 0 0) 0x7f0d280cf130 con 0x7f0d280cb780 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d25d9b700 1 -- 172.21.15.3:0/445525332 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d14003570 con 0x7f0d280cb780 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d26d9d700 1 -- 172.21.15.3:0/445525332 >> v2:172.21.15.153:3300/0 conn(0x7f0d281596d0 msgr2=0x7f0d280cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d26d9d700 1 --2- 172.21.15.3:0/445525332 >> v2:172.21.15.153:3300/0 conn(0x7f0d281596d0 0x7f0d280cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d26d9d700 1 -- 172.21.15.3:0/445525332 >> v1:172.21.15.153:6789/0 conn(0x7f0d280cb780 legacy=0x7f0d28158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d26d9d700 1 -- 172.21.15.3:0/445525332 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280ceae0 con 0x7f0d280cb3b0 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.517+0000 7f0d2759e700 1 --2- 172.21.15.3:0/445525332 >> v2:172.21.15.153:3300/0 conn(0x7f0d281596d0 0x7f0d280cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d26d9d700 1 --2- 172.21.15.3:0/445525332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d280cb3b0 0x7f0d280cf5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d180098e0 tx=0x7f0d180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8685f753f04bcfd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d25d9b700 1 -- 172.21.15.3:0/445525332 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18011630 con 0x7f0d280cb3b0 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d25d9b700 1 -- 172.21.15.3:0/445525332 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18011790 con 0x7f0d280cb3b0 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d25d9b700 1 -- 172.21.15.3:0/445525332 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18011b30 con 0x7f0d280cb3b0 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 -- 172.21.15.3:0/445525332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d280cb3b0 msgr2=0x7f0d280cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 --2- 172.21.15.3:0/445525332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d280cb3b0 0x7f0d280cf5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d180098e0 tx=0x7f0d180091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 -- 172.21.15.3:0/445525332 shutdown_connections 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 --2- 172.21.15.3:0/445525332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d280cb3b0 0x7f0d280cf5c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 --2- 172.21.15.3:0/445525332 >> v2:172.21.15.153:3300/0 conn(0x7f0d281596d0 0x7f0d280cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 -- 172.21.15.3:0/445525332 >> 172.21.15.3:0/445525332 conn(0x7f0d28070310 msgr2=0x7f0d28070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 -- 172.21.15.3:0/445525332 shutdown_connections 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.518+0000 7f0d2c853700 1 -- 172.21.15.3:0/445525332 wait complete. 2023-10-06T04:52:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 Processor -- start 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 -- start start 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d280cb3b0 0x7f0d28149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d280cb780 0x7f0d2815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d281596d0 0x7f0d280b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2814a3b0 con 0x7f0d280cb780 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2815bd10 con 0x7f0d280cb3b0 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2c853700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2815bff0 con 0x7f0d281596d0 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d280cb780 0x7f0d2815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d26d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d280cb3b0 0x7f0d28149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d281596d0 0x7f0d280b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d280cb780 0x7f0d2815b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57668/0 (socket says 172.21.15.3:57668) 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.520+0000 7f0d2659c700 1 -- 172.21.15.3:0/1118191522 learned_addr learned my addr 172.21.15.3:0/1118191522 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2759e700 1 -- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d280cb3b0 msgr2=0x7f0d28149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2759e700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d280cb3b0 0x7f0d28149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2759e700 1 -- 172.21.15.3:0/1118191522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d280cb780 msgr2=0x7f0d2815b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2759e700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d280cb780 0x7f0d2815b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2759e700 1 -- 172.21.15.3:0/1118191522 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20009ae0 con 0x7f0d281596d0 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2659c700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d280cb780 0x7f0d2815b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d26d9d700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d280cb3b0 0x7f0d28149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2759e700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d281596d0 0x7f0d280b49c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009ea0 tx=0x7f0d2000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d1effd700 1 -- 172.21.15.3:0/1118191522 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20018070 con 0x7f0d281596d0 2023-10-06T04:52:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d1effd700 1 -- 172.21.15.3:0/1118191522 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2000f040 con 0x7f0d281596d0 2023-10-06T04:52:32.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d1effd700 1 -- 172.21.15.3:0/1118191522 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2000cac0 con 0x7f0d281596d0 2023-10-06T04:52:32.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d18009420 con 0x7f0d281596d0 2023-10-06T04:52:32.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.521+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2815c5b0 con 0x7f0d281596d0 2023-10-06T04:52:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.522+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2804c380 con 0x7f0d281596d0 2023-10-06T04:52:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.523+0000 7f0d1effd700 1 -- 172.21.15.3:0/1118191522 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d20003020 con 0x7f0d281596d0 2023-10-06T04:52:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.523+0000 7f0d1effd700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d08066f30 0x7f0d080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.523+0000 7f0d1effd700 1 -- 172.21.15.3:0/1118191522 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d20084940 con 0x7f0d281596d0 2023-10-06T04:52:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.523+0000 7f0d26d9d700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d08066f30 0x7f0d080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.524+0000 7f0d26d9d700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d08066f30 0x7f0d080693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d180091e0 tx=0x7f0d18017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.528+0000 7f0d1effd700 1 -- 172.21.15.3:0/1118191522 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d20051a50 con 0x7f0d281596d0 2023-10-06T04:52:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.720+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d2815c130 con 0x7f0d281596d0 2023-10-06T04:52:32.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.722+0000 7f0d1effd700 1 -- 172.21.15.3:0/1118191522 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f0d2815c130 con 0x7f0d281596d0 2023-10-06T04:52:32.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:32.725 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d08066f30 msgr2=0x7f0d080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d08066f30 0x7f0d080693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d180091e0 tx=0x7f0d18017000 comp rx=0 tx=0).stop 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d281596d0 msgr2=0x7f0d280b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d281596d0 0x7f0d280b49c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009ea0 tx=0x7f0d2000b040 comp rx=0 tx=0).stop 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 shutdown_connections 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d08066f30 0x7f0d080693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d280cb3b0 0x7f0d28149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d281596d0 0x7f0d280b49c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 --2- 172.21.15.3:0/1118191522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d280cb780 0x7f0d2815b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.723+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 >> 172.21.15.3:0/1118191522 conn(0x7f0d28070310 msgr2=0x7f0d280b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.724+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 shutdown_connections 2023-10-06T04:52:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.724+0000 7f0d2c853700 1 -- 172.21.15.3:0/1118191522 wait complete. 2023-10-06T04:52:32.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:32.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.854+0000 7fa3958a4700 1 Processor -- start 2023-10-06T04:52:32.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.854+0000 7fa3958a4700 1 -- start start 2023-10-06T04:52:32.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa3958a4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa39015af90 0x7fa3900cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa3958a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa390157540 0x7fa3900b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa3958a4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39006aa20 con 0x7fa390157170 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa3958a4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa390067880 con 0x7fa390157540 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa3958a4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39015b450 con 0x7fa39015af90 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa38f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa390157540 0x7fa3900b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa38f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa390157540 0x7fa3900b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57682/0 (socket says 172.21.15.3:57682) 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.855+0000 7fa38f7fe700 1 -- 172.21.15.3:0/3480253413 learned_addr learned my addr 172.21.15.3:0/3480253413 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38e7fc700 1 --2- 172.21.15.3:0/3480253413 >> v2:172.21.15.153:3300/0 conn(0x7fa39015af90 0x7fa3900cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38f7fe700 1 -- 172.21.15.3:0/3480253413 >> v2:172.21.15.153:3300/0 conn(0x7fa39015af90 msgr2=0x7fa3900cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38f7fe700 1 --2- 172.21.15.3:0/3480253413 >> v2:172.21.15.153:3300/0 conn(0x7fa39015af90 0x7fa3900cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38f7fe700 1 -- 172.21.15.3:0/3480253413 >> v1:172.21.15.153:6789/0 conn(0x7fa390157170 legacy=0x7fa390067170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38f7fe700 1 -- 172.21.15.3:0/3480253413 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39015b730 con 0x7fa390157540 2023-10-06T04:52:32.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38f7fe700 1 --2- 172.21.15.3:0/3480253413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3900b2250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa390068a70 tx=0x7fa380009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4987bde3b797041 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38dffb700 1 -- 172.21.15.3:0/3480253413 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000e070 con 0x7fa390157540 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38dffb700 1 -- 172.21.15.3:0/3480253413 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38000ba20 con 0x7fa390157540 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa38dffb700 1 -- 172.21.15.3:0/3480253413 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800092e0 con 0x7fa390157540 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa3958a4700 1 -- 172.21.15.3:0/3480253413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 msgr2=0x7fa3900b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.856+0000 7fa3958a4700 1 --2- 172.21.15.3:0/3480253413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3900b2250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa390068a70 tx=0x7fa380009840 comp rx=0 tx=0).stop 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.857+0000 7fa3958a4700 1 -- 172.21.15.3:0/3480253413 shutdown_connections 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.857+0000 7fa3958a4700 1 --2- 172.21.15.3:0/3480253413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3900b2250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.857+0000 7fa3958a4700 1 --2- 172.21.15.3:0/3480253413 >> v2:172.21.15.153:3300/0 conn(0x7fa39015af90 0x7fa3900cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.857+0000 7fa3958a4700 1 -- 172.21.15.3:0/3480253413 >> 172.21.15.3:0/3480253413 conn(0x7fa39006df50 msgr2=0x7fa3900aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.857+0000 7fa3958a4700 1 -- 172.21.15.3:0/3480253413 shutdown_connections 2023-10-06T04:52:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.857+0000 7fa3958a4700 1 -- 172.21.15.3:0/3480253413 wait complete. 2023-10-06T04:52:32.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.858+0000 7fa3958a4700 1 Processor -- start 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.858+0000 7fa3958a4700 1 -- start start 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.858+0000 7fa3958a4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa390157170 0x7fa3900acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa3958a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3901469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa3958a4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa39015af90 0x7fa3901685b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa3958a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa390168d50 con 0x7fa390157540 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa3958a4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ad350 con 0x7fa390157170 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa3958a4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ad630 con 0x7fa39015af90 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3901469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3901469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57692/0 (socket says 172.21.15.3:57692) 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 -- 172.21.15.3:0/822411462 learned_addr learned my addr 172.21.15.3:0/822411462 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 -- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa39015af90 msgr2=0x7fa3901685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38f7fe700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa39015af90 0x7fa3901685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38effd700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa390157170 0x7fa3900acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa39015af90 0x7fa3901685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 -- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa390157170 msgr2=0x7fa3900acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa390157170 0x7fa3900acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 -- 172.21.15.3:0/822411462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa380009ae0 con 0x7fa390157540 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38f7fe700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa39015af90 0x7fa3901685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38effd700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa390157170 0x7fa3900acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa38e7fc700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3901469f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa37800e900 tx=0x7fa3780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa386ffd700 1 -- 172.21.15.3:0/822411462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800eea0 con 0x7fa390157540 2023-10-06T04:52:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa386ffd700 1 -- 172.21.15.3:0/822411462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37800c850 con 0x7fa390157540 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3900ad910 con 0x7fa390157540 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa386ffd700 1 -- 172.21.15.3:0/822411462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800cc10 con 0x7fa390157540 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.859+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa390168af0 con 0x7fa390157540 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.860+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390158fa0 con 0x7fa390157540 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.861+0000 7fa386ffd700 1 -- 172.21.15.3:0/822411462 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa378013470 con 0x7fa390157540 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.861+0000 7fa386ffd700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa370066f30 0x7fa3700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.861+0000 7fa386ffd700 1 -- 172.21.15.3:0/822411462 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa378084830 con 0x7fa390157540 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.861+0000 7fa38effd700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa370066f30 0x7fa3700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.862+0000 7fa38effd700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa370066f30 0x7fa3700693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa388001840 tx=0x7fa388009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:32.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:32.865+0000 7fa386ffd700 1 -- 172.21.15.3:0/822411462 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa390158fa0 con 0x7fa390157540 2023-10-06T04:52:33.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.106+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa390158fa0 con 0x7fa390157540 2023-10-06T04:52:33.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.107+0000 7fa386ffd700 1 -- 172.21.15.3:0/822411462 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7fa390158fa0 con 0x7fa390157540 2023-10-06T04:52:33.109 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:33.109 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:33.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa370066f30 msgr2=0x7fa3700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa370066f30 0x7fa3700693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa388001840 tx=0x7fa388009040 comp rx=0 tx=0).stop 2023-10-06T04:52:33.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 msgr2=0x7fa3901469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3901469f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa37800e900 tx=0x7fa3780091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 shutdown_connections 2023-10-06T04:52:33.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa370066f30 0x7fa3700693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa390157170 0x7fa3900acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa39015af90 0x7fa3901685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 --2- 172.21.15.3:0/822411462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa390157540 0x7fa3901469f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 >> 172.21.15.3:0/822411462 conn(0x7fa39006df50 msgr2=0x7fa3901476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:33.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 shutdown_connections 2023-10-06T04:52:33.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.109+0000 7fa3958a4700 1 -- 172.21.15.3:0/822411462 wait complete. 2023-10-06T04:52:33.112 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:33.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:52:18.046513+0000', 'modified': '2023-10-06T04:52:21.661968+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6837/3245320636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 3245320636}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3245320636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:52:33.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:52:33.525 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T04:52:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.649+0000 7fa92aca6700 1 Processor -- start 2023-10-06T04:52:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.650+0000 7fa92aca6700 1 -- start start 2023-10-06T04:52:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.650+0000 7fa92aca6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa92415c930 0x7fa92415d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.650+0000 7fa92aca6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa92406aad0 0x7fa9240b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.650+0000 7fa92aca6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9240c93f0 con 0x7fa92415cd00 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.650+0000 7fa92aca6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9240caf50 con 0x7fa92415c930 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.650+0000 7fa92aca6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92415e0d0 con 0x7fa92406aad0 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa92415c930 0x7fa92415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa92415c930 0x7fa92415d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57696/0 (socket says 172.21.15.3:57696) 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 -- 172.21.15.3:0/4008120069 learned_addr learned my addr 172.21.15.3:0/4008120069 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa929243700 1 --2- 172.21.15.3:0/4008120069 >> v2:172.21.15.153:3300/0 conn(0x7fa92406aad0 0x7fa9240b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 -- 172.21.15.3:0/4008120069 >> v2:172.21.15.153:3300/0 conn(0x7fa92406aad0 msgr2=0x7fa9240b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 --2- 172.21.15.3:0/4008120069 >> v2:172.21.15.153:3300/0 conn(0x7fa92406aad0 0x7fa9240b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 -- 172.21.15.3:0/4008120069 >> v1:172.21.15.153:6789/0 conn(0x7fa92415cd00 legacy=0x7fa92406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 -- 172.21.15.3:0/4008120069 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92415d830 con 0x7fa92415c930 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa928a42700 1 --2- 172.21.15.3:0/4008120069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa92415d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b2620 tx=0x7fa918009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4cb86f3090267a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa9237fe700 1 -- 172.21.15.3:0/4008120069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918009e70 con 0x7fa92415c930 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa9237fe700 1 -- 172.21.15.3:0/4008120069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91800b860 con 0x7fa92415c930 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.651+0000 7fa9237fe700 1 -- 172.21.15.3:0/4008120069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91800bc00 con 0x7fa92415c930 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 -- 172.21.15.3:0/4008120069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 msgr2=0x7fa92415d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 --2- 172.21.15.3:0/4008120069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa92415d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b2620 tx=0x7fa918009250 comp rx=0 tx=0).stop 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 -- 172.21.15.3:0/4008120069 shutdown_connections 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 --2- 172.21.15.3:0/4008120069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa92415d150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 --2- 172.21.15.3:0/4008120069 >> v2:172.21.15.153:3300/0 conn(0x7fa92406aad0 0x7fa9240b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 -- 172.21.15.3:0/4008120069 >> 172.21.15.3:0/4008120069 conn(0x7fa924066fb0 msgr2=0x7fa9240673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 -- 172.21.15.3:0/4008120069 shutdown_connections 2023-10-06T04:52:33.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.652+0000 7fa92aca6700 1 -- 172.21.15.3:0/4008120069 wait complete. 2023-10-06T04:52:33.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.653+0000 7fa92aca6700 1 Processor -- start 2023-10-06T04:52:33.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.653+0000 7fa92aca6700 1 -- start start 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa92aca6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92406aad0 0x7fa92416cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa92aca6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa9240b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa92aca6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92415ede0 0x7fa924169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa92aca6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92416d7f0 con 0x7fa92415c930 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa92aca6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92416dad0 con 0x7fa92415ede0 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa92aca6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92416ddb0 con 0x7fa92406aad0 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa923fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa9240b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa923fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa9240b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57706/0 (socket says 172.21.15.3:57706) 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa923fff700 1 -- 172.21.15.3:0/1715933885 learned_addr learned my addr 172.21.15.3:0/1715933885 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa928a42700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92406aad0 0x7fa92416cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa929243700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92415ede0 0x7fa924169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.654+0000 7fa923fff700 1 -- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92406aad0 msgr2=0x7fa92416cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa923fff700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92406aad0 0x7fa92416cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa923fff700 1 -- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92415ede0 msgr2=0x7fa924169a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa923fff700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92415ede0 0x7fa924169a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa923fff700 1 -- 172.21.15.3:0/1715933885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9180094f0 con 0x7fa92415c930 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa923fff700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa9240b0af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c4e0 tx=0x7fa91000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa920ff9700 1 -- 172.21.15.3:0/1715933885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011590 con 0x7fa92415c930 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa920ff9700 1 -- 172.21.15.3:0/1715933885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910011710 con 0x7fa92415c930 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa924159100 con 0x7fa92415c930 2023-10-06T04:52:33.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa920ff9700 1 -- 172.21.15.3:0/1715933885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011ad0 con 0x7fa92415c930 2023-10-06T04:52:33.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.655+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92416d4e0 con 0x7fa92415c930 2023-10-06T04:52:33.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.656+0000 7fa920ff9700 1 -- 172.21.15.3:0/1715933885 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa91001b3b0 con 0x7fa92415c930 2023-10-06T04:52:33.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.657+0000 7fa920ff9700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa908066f30 0x7fa9080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.657+0000 7fa920ff9700 1 -- 172.21.15.3:0/1715933885 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa910083a20 con 0x7fa92415c930 2023-10-06T04:52:33.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.657+0000 7fa928a42700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa908066f30 0x7fa9080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.658+0000 7fa928a42700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa908066f30 0x7fa9080693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b2620 tx=0x7fa918017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:33.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.659+0000 7fa933f9eb80 1 Processor -- start 2023-10-06T04:52:33.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.659+0000 7fa933f9eb80 1 -- start start 2023-10-06T04:52:33.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.659+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e89a90 0x564de3e89e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e42ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3dc2c80 con 0x564de3e43bd0 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3d99a70 con 0x564de3e89a90 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x564de3d97270 con 0x564de3e459a0 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e42ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e42ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57718/0 (socket says 172.21.15.3:57718) 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 -- 172.21.15.3:0/825676230 learned_addr learned my addr 172.21.15.3:0/825676230 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa923fff700 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa928a42700 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e89a90 0x564de3e89e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 -- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 msgr2=0x564de3e42580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e42580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 -- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e89a90 msgr2=0x564de3e89e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e89a90 0x564de3e89e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 -- 172.21.15.3:0/825676230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564de3c2d270 con 0x564de3e43bd0 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa929243700 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e42ac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa91c011530 tx=0x7fa91c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2559281f6b523a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa90dffb700 1 -- 172.21.15.3:0/825676230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00e040 con 0x564de3e43bd0 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa90dffb700 1 -- 172.21.15.3:0/825676230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c00f040 con 0x564de3e43bd0 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.660+0000 7fa90dffb700 1 -- 172.21.15.3:0/825676230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c015870 con 0x564de3e43bd0 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 -- 172.21.15.3:0/825676230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 msgr2=0x564de3e42ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e42ac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa91c011530 tx=0x7fa91c00b040 comp rx=0 tx=0).stop 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 -- 172.21.15.3:0/825676230 shutdown_connections 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e89a90 0x564de3e89e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e42ac0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/825676230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e42580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 -- 172.21.15.3:0/825676230 >> 172.21.15.3:0/825676230 conn(0x564de3e44070 msgr2=0x564de3e44440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:33.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 -- 172.21.15.3:0/825676230 shutdown_connections 2023-10-06T04:52:33.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 -- 172.21.15.3:0/825676230 wait complete. 2023-10-06T04:52:33.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.661+0000 7fa933f9eb80 1 -- start start 2023-10-06T04:52:33.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.662+0000 7fa933f9eb80 1 Processor -- start 2023-10-06T04:52:33.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.662+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3e6ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.662+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e6b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.662+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e6d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.662+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3d93060 con 0x564de3e89a90 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.662+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3d3e890 con 0x564de3e43bd0 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.662+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x564de3d59ea0 con 0x564de3e459a0 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa933f9eb80 10 client.0 _ll_register_callbacks cb 0x564de3c2ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e6d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e6d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57720/0 (socket says 172.21.15.3:57720) 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 -- 172.21.15.3:0/90892332 learned_addr learned my addr 172.21.15.3:0/90892332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa928a42700 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3e6ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa923fff700 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e6b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 -- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 msgr2=0x564de3e6b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e6b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 -- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 msgr2=0x564de3e6ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3e6ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 -- 172.21.15.3:0/90892332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c011230 con 0x564de3e89a90 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa929243700 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e6d1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0114d0 tx=0x7fa91c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba601fac1000f815 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa902ffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa902ffd700 1 -- 172.21.15.3:0/90892332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00e070 con 0x564de3e89a90 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa902ffd700 1 -- 172.21.15.3:0/90892332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa91c01d610 con 0x564de3e89a90 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564de3e37690 con 0x564de3e89a90 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa902ffd700 1 -- 172.21.15.3:0/90892332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91c0158c0 con 0x564de3e89a90 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa933f9eb80 10 client.4650 fetch_fsmap learned FSMap version 7 2023-10-06T04:52:33.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.663+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x564de3bcffe0 con 0x564de3e89a90 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa902ffd700 1 -- 172.21.15.3:0/90892332 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564de3e37690 con 0x564de3e89a90 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa902ffd700 1 -- 172.21.15.3:0/90892332 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa91c01b020 con 0x564de3e89a90 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa933f9eb80 10 client.4650 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa933f9eb80 10 client.4650 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa933f9eb80 20 client.4650 populate_metadata read hostname 'smithi003' 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e6d730 0x564de3e6db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa933f9eb80 4 client.4650 mds_command: new command op to 4303 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 --> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564de3e145c0 con 0x564de3e6d730 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa928a42700 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e6d730 0x564de3e6db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa902ffd700 10 client.4650 ms_handle_connect on v2:172.21.15.153:6836/3245320636 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.664+0000 7fa928a42700 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e6d730 0x564de3e6db00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9296e74a17fc1b4c server_cookie=2a17af99c9b4fd58 in_seq=0 out_seq=0 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa902ffd700 1 -- 172.21.15.3:0/90892332 <== mds.0 v2:172.21.15.153:6836/3245320636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564de3e145c0 con 0x564de3e6d730 2023-10-06T04:52:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa902ffd700 10 client.4650 handle_command_reply: tid=0 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 client.4650 shutdown 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 msgr2=0x564de3e6d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e6d1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0114d0 tx=0x7fa91c01d000 comp rx=0 tx=0).stop 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 shutdown_connections 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3e6ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e6d730 0x564de3e6db00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e6d1f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/90892332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e6b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa902ffd700 0 client.4650 ms_handle_reset on v2:172.21.15.153:6836/3245320636 2023-10-06T04:52:33.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 >> 172.21.15.3:0/90892332 conn(0x564de3e44070 msgr2=0x564de3e3d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:33.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 shutdown_connections 2023-10-06T04:52:33.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.665+0000 7fa933f9eb80 1 -- 172.21.15.3:0/90892332 wait complete. 2023-10-06T04:52:33.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.666+0000 7fa933f9eb80 20 client.4650 trim_cache size 0 max 16384 2023-10-06T04:52:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.682+0000 7fa933f9eb80 1 Processor -- start 2023-10-06T04:52:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.682+0000 7fa933f9eb80 1 -- start start 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e44d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e459a0 0x564de3e878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e89a90 0x564de3e6bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3c15e30 con 0x564de3e43bd0 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3e3f640 con 0x564de3e459a0 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x564de3e87de0 con 0x564de3e89a90 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa928a42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e44d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa928a42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e44d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57736/0 (socket says 172.21.15.3:57736) 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa928a42700 1 -- 172.21.15.3:0/2732213373 learned_addr learned my addr 172.21.15.3:0/2732213373 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa929243700 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e89a90 0x564de3e6bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.683+0000 7fa923fff700 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e459a0 0x564de3e878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa928a42700 1 -- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e89a90 msgr2=0x564de3e6bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa928a42700 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e89a90 0x564de3e6bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa928a42700 1 -- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e459a0 msgr2=0x564de3e878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa928a42700 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e459a0 0x564de3e878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa928a42700 1 -- 172.21.15.3:0/2732213373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564de3d61780 con 0x564de3e43bd0 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa928a42700 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e44d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9180090b0 tx=0x7fa918014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702ae99c61f4580a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa90e7fc700 1 -- 172.21.15.3:0/2732213373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91802f040 con 0x564de3e43bd0 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa90e7fc700 1 -- 172.21.15.3:0/2732213373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918030040 con 0x564de3e43bd0 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa90e7fc700 1 -- 172.21.15.3:0/2732213373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918021070 con 0x564de3e43bd0 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 -- 172.21.15.3:0/2732213373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 msgr2=0x564de3e44d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e44d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9180090b0 tx=0x7fa918014000 comp rx=0 tx=0).stop 2023-10-06T04:52:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 -- 172.21.15.3:0/2732213373 shutdown_connections 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e89a90 0x564de3e6bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e43bd0 0x564de3e44d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/2732213373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e459a0 0x564de3e878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 -- 172.21.15.3:0/2732213373 >> 172.21.15.3:0/2732213373 conn(0x564de3e44070 msgr2=0x564de3e68990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.684+0000 7fa933f9eb80 1 -- 172.21.15.3:0/2732213373 shutdown_connections 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.685+0000 7fa933f9eb80 1 -- 172.21.15.3:0/2732213373 wait complete. 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.685+0000 7fa933f9eb80 1 -- start start 2023-10-06T04:52:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.685+0000 7fa933f9eb80 1 Processor -- start 2023-10-06T04:52:33.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3eee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa933f9eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3eee640 con 0x564de3e89a90 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564de3e71bc0 con 0x564de3e43bd0 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa933f9eb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x564de3e83dc0 con 0x564de3e459a0 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa933f9eb80 10 client.0 _ll_register_callbacks cb 0x564de3c2ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa923fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e82f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57752/0 (socket says 172.21.15.3:57752) 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 -- 172.21.15.3:0/1598645959 learned_addr learned my addr 172.21.15.3:0/1598645959 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa928a42700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3eee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 -- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 msgr2=0x564de3e829e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e829e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 -- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 msgr2=0x564de3eee100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3eee100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.686+0000 7fa929243700 1 -- 172.21.15.3:0/1598645959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92415d830 con 0x564de3e89a90 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa929243700 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e82f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa91c026510 tx=0x7fa91c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa447a73627cf07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa902ffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa902ffd700 1 -- 172.21.15.3:0/1598645959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00e040 con 0x564de3e89a90 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa902ffd700 1 -- 172.21.15.3:0/1598645959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa91c026610 con 0x564de3e89a90 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564de3e6ee80 con 0x564de3e89a90 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa902ffd700 1 -- 172.21.15.3:0/1598645959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91c026a00 con 0x564de3e89a90 2023-10-06T04:52:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 10 client.4656 fetch_fsmap learned FSMap version 7 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x564de3d48470 con 0x564de3e89a90 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa902ffd700 1 -- 172.21.15.3:0/1598645959 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564de3e6ee80 con 0x564de3e89a90 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa902ffd700 1 -- 172.21.15.3:0/1598645959 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa91c0119a0 con 0x564de3e89a90 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 10 client.4656 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 10 client.4656 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 20 client.4656 populate_metadata read hostname 'smithi003' 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e69b20 0x564de3e69ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 4 client.4656 mds_command: new command op to 4303 tid=0[{"prefix": "status"}] 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 --> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fa91c00f040 con 0x564de3e69b20 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.687+0000 7fa928a42700 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e69b20 0x564de3e69ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa902ffd700 10 client.4656 ms_handle_connect on v2:172.21.15.153:6836/3245320636 2023-10-06T04:52:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa928a42700 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e69b20 0x564de3e69ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c14f0af75527b9 server_cookie=b8e0ca8878a2fb6 in_seq=0 out_seq=0 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa902ffd700 1 -- 172.21.15.3:0/1598645959 <== mds.0 v2:172.21.15.153:6836/3245320636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x7fa91c00f040 con 0x564de3e69b20 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa902ffd700 10 client.4656 handle_command_reply: tid=0 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 client.4656 shutdown 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 msgr2=0x564de3e82f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e82f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa91c026510 tx=0x7fa91c01f040 comp rx=0 tx=0).stop 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 shutdown_connections 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564de3e43bd0 0x564de3eee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:6836/3245320636,v1:172.21.15.153:6837/3245320636] conn(0x564de3e69b20 0x564de3e69ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564de3e89a90 0x564de3e82f20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa902ffd700 0 client.4656 ms_handle_reset on v2:172.21.15.153:6836/3245320636 2023-10-06T04:52:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 --2- 172.21.15.3:0/1598645959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564de3e459a0 0x564de3e829e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 >> 172.21.15.3:0/1598645959 conn(0x564de3e44070 msgr2=0x564de3e6abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 shutdown_connections 2023-10-06T04:52:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 1 -- 172.21.15.3:0/1598645959 wait complete. 2023-10-06T04:52:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.688+0000 7fa933f9eb80 20 client.4656 trim_cache size 0 max 16384 2023-10-06T04:52:33.699 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:52:33.699 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T04:52:33.699 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T04:52:33.699 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4303, 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 15.055659626000001, 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 7, 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 20, 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 20, 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.053928812999999 2023-10-06T04:52:33.700 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:52:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa908066f30 msgr2=0x7fa9080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa908066f30 0x7fa9080693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b2620 tx=0x7fa918017000 comp rx=0 tx=0).stop 2023-10-06T04:52:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 msgr2=0x7fa9240b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa9240b0af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c4e0 tx=0x7fa91000c7f0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 shutdown_connections 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa908066f30 0x7fa9080693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92415ede0 0x7fa924169a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92406aad0 0x7fa92416cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 --2- 172.21.15.3:0/1715933885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92415c930 0x7fa9240b0af0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 >> 172.21.15.3:0/1715933885 conn(0x7fa924066fb0 msgr2=0x7fa9240b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 shutdown_connections 2023-10-06T04:52:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:33.689+0000 7fa92aca6700 1 -- 172.21.15.3:0/1715933885 wait complete. 2023-10-06T04:52:33.703 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-06T04:52:33.728 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-06T04:52:33.729 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T04:52:33.729 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-06T04:52:33.729 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-06T04:52:33.729 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-06T04:52:33.729 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-06T04:52:33.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:33.729 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:52:33.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:33.749 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-10-06T04:52:33.820 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2023-10-06T04:52:33.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:52:33.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:33.822 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:52:33.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:33.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-10-06T04:52:33.905 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2023-10-06T04:52:33.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:52:33.906 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-06T04:52:33.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:52:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.063+0000 7f3b38165700 1 Processor -- start 2023-10-06T04:52:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.063+0000 7f3b38165700 1 -- start start 2023-10-06T04:52:34.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b38165700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015c8f0 0x7f3b3015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:34.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b38165700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b3015ccc0 0x7f3b300d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b38165700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b300cf6c0 con 0x7f3b300d06a0 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b38165700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b300cfb50 con 0x7f3b3015ccc0 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b38165700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3015e050 con 0x7f3b3015c8f0 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35700700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b3015ccc0 0x7f3b300d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b36702700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3b300d06a0 0x7f3b3006d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35874/0 (socket says 172.21.15.3:35874) 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b36702700 1 -- 172.21.15.3:0/1267028957 learned_addr learned my addr 172.21.15.3:0/1267028957 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35f01700 1 --2- 172.21.15.3:0/1267028957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015c8f0 0x7f3b3015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35700700 1 -- 172.21.15.3:0/1267028957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015c8f0 msgr2=0x7f3b3015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35700700 1 --2- 172.21.15.3:0/1267028957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015c8f0 0x7f3b3015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35700700 1 -- 172.21.15.3:0/1267028957 >> v1:172.21.15.3:6789/0 conn(0x7f3b300d06a0 legacy=0x7f3b3006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35700700 1 -- 172.21.15.3:0/1267028957 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b300d0af0 con 0x7f3b3015ccc0 2023-10-06T04:52:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b34eff700 1 -- 172.21.15.3:0/1267028957 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223404345 0 0) 0x7f3b300cf6c0 con 0x7f3b300d06a0 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35f01700 1 --2- 172.21.15.3:0/1267028957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015c8f0 0x7f3b3015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b35700700 1 --2- 172.21.15.3:0/1267028957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b3015ccc0 0x7f3b300d0160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098e0 tx=0x7f3b240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb98af94fbff6166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b34eff700 1 -- 172.21.15.3:0/1267028957 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24009e80 con 0x7f3b3015ccc0 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b34eff700 1 -- 172.21.15.3:0/1267028957 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2400b860 con 0x7f3b3015ccc0 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.064+0000 7f3b34eff700 1 -- 172.21.15.3:0/1267028957 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400bbe0 con 0x7f3b3015ccc0 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 -- 172.21.15.3:0/1267028957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b3015ccc0 msgr2=0x7f3b300d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 --2- 172.21.15.3:0/1267028957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b3015ccc0 0x7f3b300d0160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098e0 tx=0x7f3b240091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 -- 172.21.15.3:0/1267028957 shutdown_connections 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 --2- 172.21.15.3:0/1267028957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015c8f0 0x7f3b3015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 --2- 172.21.15.3:0/1267028957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b3015ccc0 0x7f3b300d0160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 -- 172.21.15.3:0/1267028957 >> 172.21.15.3:0/1267028957 conn(0x7f3b30070310 msgr2=0x7f3b30070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 -- 172.21.15.3:0/1267028957 shutdown_connections 2023-10-06T04:52:34.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.065+0000 7f3b38165700 1 -- 172.21.15.3:0/1267028957 wait complete. 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.066+0000 7f3b38165700 1 Processor -- start 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.066+0000 7f3b38165700 1 -- start start 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b38165700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b300d06a0 0x7f3b30154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b38165700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b3015c8f0 0x7f3b301693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b38165700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015ccc0 0x7f3b3016d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b38165700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3016ca50 con 0x7f3b300d06a0 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b38165700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30169c50 con 0x7f3b3015c8f0 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b38165700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30169f30 con 0x7f3b3015ccc0 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b300d06a0 0x7f3b30154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b300d06a0 0x7f3b30154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57756/0 (socket says 172.21.15.3:57756) 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 -- 172.21.15.3:0/991176912 learned_addr learned my addr 172.21.15.3:0/991176912 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 -- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015ccc0 msgr2=0x7f3b3016d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015ccc0 0x7f3b3016d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 -- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b3015c8f0 msgr2=0x7f3b301693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b3015c8f0 0x7f3b301693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 -- 172.21.15.3:0/991176912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24009420 con 0x7f3b300d06a0 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b35f01700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b300d06a0 0x7f3b30154cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800e8d0 tx=0x7f3b28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b2dffb700 1 -- 172.21.15.3:0/991176912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800ee70 con 0x7f3b300d06a0 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b2dffb700 1 -- 172.21.15.3:0/991176912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2800c5e0 con 0x7f3b300d06a0 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3016ccc0 con 0x7f3b300d06a0 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.067+0000 7f3b2dffb700 1 -- 172.21.15.3:0/991176912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800c980 con 0x7f3b300d06a0 2023-10-06T04:52:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.068+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b301698f0 con 0x7f3b300d06a0 2023-10-06T04:52:34.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.069+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b30071680 con 0x7f3b300d06a0 2023-10-06T04:52:34.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.069+0000 7f3b2dffb700 1 -- 172.21.15.3:0/991176912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b28013410 con 0x7f3b300d06a0 2023-10-06T04:52:34.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.069+0000 7f3b2dffb700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b1c066f30 0x7f3b1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:34.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.069+0000 7f3b2dffb700 1 -- 172.21.15.3:0/991176912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b28084480 con 0x7f3b300d06a0 2023-10-06T04:52:34.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.069+0000 7f3b35700700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b1c066f30 0x7f3b1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:34.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.070+0000 7f3b35700700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b1c066f30 0x7f3b1c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b3015d0b0 tx=0x7f3b24017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.073+0000 7f3b2dffb700 1 -- 172.21.15.3:0/991176912 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b28051590 con 0x7f3b300d06a0 2023-10-06T04:52:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.311+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3b300b35a0 con 0x7f3b300d06a0 2023-10-06T04:52:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.312+0000 7f3b2dffb700 1 -- 172.21.15.3:0/991176912 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3b28054bb0 con 0x7f3b300d06a0 2023-10-06T04:52:34.314 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:52:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b1c066f30 msgr2=0x7f3b1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b1c066f30 0x7f3b1c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b3015d0b0 tx=0x7f3b24017000 comp rx=0 tx=0).stop 2023-10-06T04:52:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b300d06a0 msgr2=0x7f3b30154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b300d06a0 0x7f3b30154cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800e8d0 tx=0x7f3b28008040 comp rx=0 tx=0).stop 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 shutdown_connections 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b1c066f30 0x7f3b1c0693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b3015ccc0 0x7f3b3016d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b300d06a0 0x7f3b30154cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 --2- 172.21.15.3:0/991176912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b3015c8f0 0x7f3b301693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 >> 172.21.15.3:0/991176912 conn(0x7f3b30070310 msgr2=0x7f3b301567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 shutdown_connections 2023-10-06T04:52:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:34.314+0000 7f3b38165700 1 -- 172.21.15.3:0/991176912 wait complete. 2023-10-06T04:52:34.328 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:52:34.328 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:52:34.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:52:34.329 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:52:34.329 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:52:34.329 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:52:34.329 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:52:34.329 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:52:34.329 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:52:34.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.330 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:52:34.344 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:34.345 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42476sec preferred_lft 42476sec 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:34.346 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:34.347 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:52:34.347 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:34.347 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:34.347 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:52:34.347 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T04:52:34.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.347 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:34.348 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:34.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2023-10-06T04:52:34.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2023-10-06T04:52:34.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2023-10-06T04:52:34.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T04:52:34.348 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:34.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.509 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:52:34.555 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:52:34.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.557 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:52:34.587 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:52:34.587 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:52:34.587 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:52:34.587 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:52:34.587 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:52:34.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.588 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:34.588 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:34.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:52:34.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:52:34.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:52:34.588 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:34.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.866 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:52:34.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.882 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-10-06T04:52:34.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:34.937 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:34.937 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:34.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:52:34.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T04:52:34.937 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:35.028 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T04:52:35.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:35.028 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:35.028 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:35.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T04:52:35.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T04:52:35.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T04:52:35.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T04:52:35.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T04:52:35.029 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:35.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:35.209 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:35.209 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:35.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:52:35.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T04:52:35.210 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:35.303 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:52:35.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:35.303 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:52:35.318 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:52:35.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:35.318 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:52:35.373 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:52:35.439 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:35.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:35.504 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:35.504 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:35.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:35.505 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:52:35.561 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:52:35.562 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:52:35.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:52:35.628 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:35.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:35.693 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:35.695 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:35.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:35.695 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:52:35.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:52:35.693+0000 7f4e12e31540 -1 init, newargv = 0x55b6713fc960 newargc=15 2023-10-06T04:52:35.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[99738]: starting ceph client 2023-10-06T04:52:35.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[99738]: starting fuse 2023-10-06T04:52:35.715 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:52:35.715 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:52:35.716 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:52:35.717 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:52:35.718 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:35.718 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:52:35.718 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:52:35.839 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.99738.asok 2023-10-06T04:52:36.021 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.99738.asok 2023-10-06T04:52:36.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:36.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.99738.asok status 2023-10-06T04:52:36.177 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "99738", 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:52:36.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4674, 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 4674 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:52:36.179 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 574803314 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 574803314 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.4674 192.168.0.1:0/574803314", 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/574803314", 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 7, 2023-10-06T04:52:36.180 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 20, 2023-10-06T04:52:36.181 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:52:36.181 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:52:36.181 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:52:36.181 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:52:36.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:52:36.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.304+0000 7ff7b96b6700 1 Processor -- start 2023-10-06T04:52:36.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.304+0000 7ff7b96b6700 1 -- start start 2023-10-06T04:52:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b96b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b40ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b96b6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b4067a10 0x7ff7b40a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b96b6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b40a9d90 con 0x7ff7b415a370 2023-10-06T04:52:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b96b6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b4159bb0 con 0x7ff7b4067a10 2023-10-06T04:52:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b96b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40a9ff0 con 0x7ff7b4067640 2023-10-06T04:52:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b4067a10 0x7ff7b40a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b4067a10 0x7ff7b40a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57766/0 (socket says 172.21.15.3:57766) 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 -- 172.21.15.3:0/3714359082 learned_addr learned my addr 172.21.15.3:0/3714359082 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 -- 172.21.15.3:0/3714359082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 msgr2=0x7ff7b40ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b3fff700 1 --2- 172.21.15.3:0/3714359082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b40ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 --2- 172.21.15.3:0/3714359082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b40ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 -- 172.21.15.3:0/3714359082 >> v1:172.21.15.153:6789/0 conn(0x7ff7b415a370 legacy=0x7ff7b415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 -- 172.21.15.3:0/3714359082 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40aa2d0 con 0x7ff7b4067a10 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b3fff700 1 --2- 172.21.15.3:0/3714359082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b40ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.305+0000 7ff7b37fe700 1 --2- 172.21.15.3:0/3714359082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b4067a10 0x7ff7b40a9850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40099b0 tx=0x7ff7a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6e61fc0813737 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b2ffd700 1 -- 172.21.15.3:0/3714359082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400b820 con 0x7ff7b4067a10 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b2ffd700 1 -- 172.21.15.3:0/3714359082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a400b980 con 0x7ff7b4067a10 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b2ffd700 1 -- 172.21.15.3:0/3714359082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400bd20 con 0x7ff7b4067a10 2023-10-06T04:52:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3714359082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b4067a10 msgr2=0x7ff7b40a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3714359082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b4067a10 0x7ff7b40a9850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40099b0 tx=0x7ff7a4009250 comp rx=0 tx=0).stop 2023-10-06T04:52:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3714359082 shutdown_connections 2023-10-06T04:52:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3714359082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b40ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3714359082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b4067a10 0x7ff7b40a9850 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3714359082 >> 172.21.15.3:0/3714359082 conn(0x7ff7b406d480 msgr2=0x7ff7b406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3714359082 shutdown_connections 2023-10-06T04:52:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.306+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3714359082 wait complete. 2023-10-06T04:52:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.307+0000 7ff7b96b6700 1 Processor -- start 2023-10-06T04:52:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b96b6700 1 -- start start 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b96b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b96b6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b4067a10 0x7ff7b414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b96b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b415a370 0x7ff7b40b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b96b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b415cce0 con 0x7ff7b415a370 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b96b6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40af000 con 0x7ff7b4067a10 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b96b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b415ca00 con 0x7ff7b4067640 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b8eb5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b415a370 0x7ff7b40b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b8eb5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b415a370 0x7ff7b40b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57776/0 (socket says 172.21.15.3:57776) 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b8eb5700 1 -- 172.21.15.3:0/3941478554 learned_addr learned my addr 172.21.15.3:0/3941478554 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b4067a10 0x7ff7b414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.308+0000 7ff7b3fff700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b8eb5700 1 -- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 msgr2=0x7ff7b414d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b8eb5700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b414d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b8eb5700 1 -- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b4067a10 msgr2=0x7ff7b414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b8eb5700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b4067a10 0x7ff7b414bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b8eb5700 1 -- 172.21.15.3:0/3941478554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40094f0 con 0x7ff7b415a370 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b8eb5700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b415a370 0x7ff7b40b0fd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00bc30 tx=0x7ff7ac009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7abfff700 1 -- 172.21.15.3:0/3941478554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00c840 con 0x7ff7b415a370 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7abfff700 1 -- 172.21.15.3:0/3941478554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac00c9e0 con 0x7ff7b415a370 2023-10-06T04:52:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7abfff700 1 -- 172.21.15.3:0/3941478554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00cda0 con 0x7ff7b415a370 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b40ae9b0 con 0x7ff7b415a370 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.309+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b414c320 con 0x7ff7b415a370 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.310+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b40af390 con 0x7ff7b415a370 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.310+0000 7ff7abfff700 1 -- 172.21.15.3:0/3941478554 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7ac014620 con 0x7ff7b415a370 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.311+0000 7ff7abfff700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.311+0000 7ff7abfff700 1 -- 172.21.15.3:0/3941478554 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff7ac084670 con 0x7ff7b415a370 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.311+0000 7ff7b3fff700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.312+0000 7ff7b3fff700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00adc0 tx=0x7ff79c006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.315+0000 7ff7abfff700 1 -- 172.21.15.3:0/3941478554 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7ac04c5a0 con 0x7ff7b415a370 2023-10-06T04:52:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.554+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7b40af170 con 0x7ff7b415a370 2023-10-06T04:52:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.555+0000 7ff7abfff700 1 -- 172.21.15.3:0/3941478554 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff7b40af390 con 0x7ff7b415a370 2023-10-06T04:52:36.558 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:52:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 msgr2=0x7ff7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00adc0 tx=0x7ff79c006f90 comp rx=0 tx=0).stop 2023-10-06T04:52:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b415a370 msgr2=0x7ff7b40b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b415a370 0x7ff7b40b0fd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00bc30 tx=0x7ff7ac009820 comp rx=0 tx=0).stop 2023-10-06T04:52:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 shutdown_connections 2023-10-06T04:52:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b415a370 0x7ff7b40b0fd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7b4067640 0x7ff7b414d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 --2- 172.21.15.3:0/3941478554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b4067a10 0x7ff7b414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 >> 172.21.15.3:0/3941478554 conn(0x7ff7b406d480 msgr2=0x7ff7b40afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 shutdown_connections 2023-10-06T04:52:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:36.557+0000 7ff7b96b6700 1 -- 172.21.15.3:0/3941478554 wait complete. 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:52:36.572 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T04:52:36.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.573 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:36.588 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42474sec preferred_lft 42474sec 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:52:36.589 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link tentative 2023-10-06T04:52:36.590 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:36.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.591 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:52:36.652 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:52:36.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.653 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:52:36.709 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:52:36.710 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:52:36.710 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:52:36.710 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:52:36.710 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:52:36.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.710 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:36.710 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:36.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:52:36.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:52:36.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:52:36.711 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:36.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.812 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:52:36.826 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:52:36.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.827 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-10-06T04:52:36.882 INFO:teuthology.orchestra.run.smithi003.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T04:52:36.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.883 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:36.883 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:36.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:52:36.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T04:52:36.883 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:36.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:36.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T04:52:37.005 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether de:0d:dd:98:ce:5c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::dc0d:ddff:fe98:ce5c/64 scope link tentative 2023-10-06T04:52:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:37.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T04:52:37.097 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:37.097 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:37.097 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:37.097 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether de:0d:dd:98:ce:5c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::dc0d:ddff:fe98:ce5c/64 scope link tentative 2023-10-06T04:52:37.098 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:37.114 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T04:52:37.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:37.114 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:37.114 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:37.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T04:52:37.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T04:52:37.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T04:52:37.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T04:52:37.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T04:52:37.115 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:37.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:37.332 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:52:37.332 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:52:37.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.1 up 2023-10-06T04:52:37.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T04:52:37.332 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:52:37.425 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-06T04:52:37.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:37.426 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T04:52:37.440 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T04:52:37.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:37.441 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T04:52:37.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:52:37.561 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:37.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:37.626 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:37.627 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:37.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:37.628 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:52:37.683 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:52:37.683 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-10-06T04:52:37.683 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-06T04:52:37.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:52:37.750 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:37.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:37.808 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi003.stderr:2023-10-06T04:52:37.806+0000 7f108f4de540 -1 init, newargv = 0x55cdd6ed68d0 newargc=15 2023-10-06T04:52:37.808 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi003.stderr:ceph-fuse[100273]: starting ceph client 2023-10-06T04:52:37.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi003.stderr:ceph-fuse[100273]: starting fuse 2023-10-06T04:52:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:37.817 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:37.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:37.817 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:52:37.873 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:52:37.873 INFO:teuthology.orchestra.run.smithi003.stdout:48 2023-10-06T04:52:37.873 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-10-06T04:52:37.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:52:37.873 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:52:37.873 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:52:37.874 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:52:37.874 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:52:37.874 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:37.874 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:52:37.874 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:52:37.875 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.2")) 2023-10-06T04:52:37.876 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:52:37.996 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.2.100273.asok 2023-10-06T04:52:38.178 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.100273.asok 2023-10-06T04:52:38.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:38.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.100273.asok status 2023-10-06T04:52:38.333 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:52:38.333 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "2", 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "100273", 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:52:38.334 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4686, 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 4686 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3458062705 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:38.335 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3458062705 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.4686 192.168.0.2:0/3458062705", 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.2:0/3458062705", 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 7, 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 20, 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:52:38.336 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:52:38.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:52:38.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.458+0000 7f93bafcf700 1 Processor -- start 2023-10-06T04:52:38.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.458+0000 7f93bafcf700 1 -- start start 2023-10-06T04:52:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.459+0000 7f93bafcf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f93b415c8f0 0x7f93b415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.459+0000 7f93bafcf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 0x7f93b40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.459+0000 7f93bafcf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b40cf6c0 con 0x7f93b415ccc0 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.459+0000 7f93bafcf700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b40cfb50 con 0x7f93b415c8f0 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.459+0000 7f93bafcf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b415e050 con 0x7f93b40d0870 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.459+0000 7f93b3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f93b415ccc0 0x7f93b40d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35878/0 (socket says 172.21.15.3:35878) 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.459+0000 7f93b3fff700 1 -- 172.21.15.3:0/3389275226 learned_addr learned my addr 172.21.15.3:0/3389275226 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b8d6b700 1 --2- 172.21.15.3:0/3389275226 >> v2:172.21.15.153:3300/0 conn(0x7f93b415c8f0 0x7f93b415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b956c700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 0x7f93b40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004785189 0 0) 0x7f93b40cf6c0 con 0x7f93b415ccc0 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a0003570 con 0x7f93b415ccc0 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387664082 0 0) 0x7f93a0003570 con 0x7f93b415ccc0 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 msgr2=0x7f93b40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 0x7f93b40b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 >> v2:172.21.15.153:3300/0 conn(0x7f93b415c8f0 msgr2=0x7f93b415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 --2- 172.21.15.3:0/3389275226 >> v2:172.21.15.153:3300/0 conn(0x7f93b415c8f0 0x7f93b415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b40ceab0 con 0x7f93b415ccc0 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f93a4001470 con 0x7f93b415ccc0 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 >> v1:172.21.15.3:6789/0 conn(0x7f93b415ccc0 legacy=0x7f93b40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 0x7f93a0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0003fe0 0x7f93a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93a0004d50 0x7f93a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40cfb50 con 0x7f93a0003fe0 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b415e050 con 0x7f93a0004d50 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a000ba90 con 0x7f93b40d0870 2023-10-06T04:52:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b3fff700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0003fe0 0x7f93a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b8d6b700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 0x7f93a0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b956c700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93a0004d50 0x7f93a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b3fff700 1 -- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 msgr2=0x7f93a0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b3fff700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93b40d0870 0x7f93a0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b3fff700 1 -- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93a0004d50 msgr2=0x7f93a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b3fff700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93a0004d50 0x7f93a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.460+0000 7f93b3fff700 1 -- 172.21.15.3:0/3389275226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0003570 con 0x7f93a0003fe0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93b3fff700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0003fe0 0x7f93a0004810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c390 tx=0x7f93a4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a3129a7fa9bc23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a400cc00 con 0x7f93a0003fe0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a400cd60 con 0x7f93a0003fe0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93b37fe700 1 -- 172.21.15.3:0/3389275226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4005b50 con 0x7f93a0003fe0 2023-10-06T04:52:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93bafcf700 1 -- 172.21.15.3:0/3389275226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0003fe0 msgr2=0x7f93a0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:38.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93bafcf700 1 --2- 172.21.15.3:0/3389275226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0003fe0 0x7f93a0004810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c390 tx=0x7f93a4001e90 comp rx=0 tx=0).stop 2023-10-06T04:52:38.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93b956c700 1 -- 172.21.15.3:0/3389275226 reap_dead start 2023-10-06T04:52:38.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93bafcf700 1 -- 172.21.15.3:0/3389275226 shutdown_connections 2023-10-06T04:52:38.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93bafcf700 1 -- 172.21.15.3:0/3389275226 >> 172.21.15.3:0/3389275226 conn(0x7f93b4070310 msgr2=0x7f93b4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:38.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93bafcf700 1 -- 172.21.15.3:0/3389275226 shutdown_connections 2023-10-06T04:52:38.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.461+0000 7f93bafcf700 1 -- 172.21.15.3:0/3389275226 wait complete. 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.462+0000 7f93bafcf700 1 Processor -- start 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93bafcf700 1 -- start start 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93bafcf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93a0003fe0 0x7f93b413a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93bafcf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93b4150b40 0x7f93b4150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93bafcf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b40b4020 0x7f93b40b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93bafcf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40b5840 con 0x7f93b40b4020 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93bafcf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b41514f0 con 0x7f93b4150b40 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93bafcf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40b49e0 con 0x7f93a0003fe0 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93b956c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b40b4020 0x7f93b40b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.463+0000 7f93b956c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b40b4020 0x7f93b40b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57802/0 (socket says 172.21.15.3:57802) 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b956c700 1 -- 172.21.15.3:0/3898314673 learned_addr learned my addr 172.21.15.3:0/3898314673 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b3fff700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93b4150b40 0x7f93b4150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b8d6b700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93a0003fe0 0x7f93b413a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b956c700 1 -- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93a0003fe0 msgr2=0x7f93b413a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b956c700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93a0003fe0 0x7f93b413a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b956c700 1 -- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93b4150b40 msgr2=0x7f93b4150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b956c700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93b4150b40 0x7f93b4150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b956c700 1 -- 172.21.15.3:0/3898314673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4152f40 con 0x7f93b40b4020 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b8d6b700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93a0003fe0 0x7f93b413a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b956c700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b40b4020 0x7f93b40b44a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f939c006f00 tx=0x7f939c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b0ff9700 1 -- 172.21.15.3:0/3898314673 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00f910 con 0x7f93b40b4020 2023-10-06T04:52:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b0ff9700 1 -- 172.21.15.3:0/3898314673 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c00fa70 con 0x7f93b40b4020 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93b0ff9700 1 -- 172.21.15.3:0/3898314673 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00fe10 con 0x7f93b40b4020 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b40b4c30 con 0x7f93b40b4020 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.464+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b40b51f0 con 0x7f93b40b4020 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.465+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b40b59f0 con 0x7f93b40b4020 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.465+0000 7f93b0ff9700 1 -- 172.21.15.3:0/3898314673 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f939c014650 con 0x7f93b40b4020 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.466+0000 7f93b0ff9700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9394066f30 0x7f93940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.466+0000 7f93b0ff9700 1 -- 172.21.15.3:0/3898314673 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f939c084c10 con 0x7f93b40b4020 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.466+0000 7f93b8d6b700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9394066f30 0x7f93940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.467+0000 7f93b8d6b700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9394066f30 0x7f93940693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0099f0 tx=0x7f93ac00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:38.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.470+0000 7f93b0ff9700 1 -- 172.21.15.3:0/3898314673 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b40b59f0 con 0x7f93b40b4020 2023-10-06T04:52:38.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.710+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f93b40b59f0 con 0x7f93b40b4020 2023-10-06T04:52:38.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.710+0000 7f93b0ff9700 1 -- 172.21.15.3:0/3898314673 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f93b40b59f0 con 0x7f93b40b4020 2023-10-06T04:52:38.713 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9394066f30 msgr2=0x7f93940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9394066f30 0x7f93940693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0099f0 tx=0x7f93ac00b040 comp rx=0 tx=0).stop 2023-10-06T04:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b40b4020 msgr2=0x7f93b40b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b40b4020 0x7f93b40b44a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f939c006f00 tx=0x7f939c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 shutdown_connections 2023-10-06T04:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9394066f30 0x7f93940693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b40b4020 0x7f93b40b44a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93a0003fe0 0x7f93b413a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 --2- 172.21.15.3:0/3898314673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93b4150b40 0x7f93b4150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 >> 172.21.15.3:0/3898314673 conn(0x7f93b4070310 msgr2=0x7f93b4157660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 shutdown_connections 2023-10-06T04:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:38.712+0000 7f93bafcf700 1 -- 172.21.15.3:0/3898314673 wait complete. 2023-10-06T04:52:38.726 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:52:38.726 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:52:38.727 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:52:38.727 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:52:38.727 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:52:38.727 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:52:38.727 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:52:38.727 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:52:38.727 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:52:38.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:38.727 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:52:38.745 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:38.746 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42474sec preferred_lft 42474sec 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:52:38.747 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:38.747 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T04:52:38.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:38.748 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:38.748 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:38.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add name ceph-brx type bridge 2023-10-06T04:52:38.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr flush dev ceph-brx 2023-10-06T04:52:38.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set ceph-brx up 2023-10-06T04:52:38.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T04:52:38.748 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:38.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:38.929 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:52:38.969 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:52:38.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:38.970 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:52:39.033 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:52:39.033 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:52:39.033 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:52:39.033 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:52:39.033 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:52:39.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:39.033 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:39.034 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:39.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:52:39.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:52:39.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:52:39.034 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:39.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:39.275 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:52:39.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:39.290 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2023-10-06T04:52:39.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:39.344 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:39.344 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:39.345 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:52:39.345 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T04:52:39.345 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:39.434 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T04:52:39.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:39.434 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:39.434 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:39.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T04:52:39.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T04:52:39.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T04:52:39.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T04:52:39.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T04:52:39.435 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:39.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:39.623 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:39.624 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:39.624 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T04:52:39.624 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T04:52:39.624 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:39.716 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:52:39.716 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:39.716 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:52:39.730 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:52:39.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:39.731 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:52:39.785 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:52:39.850 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:39.851 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:39.914 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:39.915 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:39.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:39.916 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:52:39.970 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:52:39.971 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:52:40.013 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:52:40.037 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:40.037 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:40.102 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:40.102 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:40.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:40.103 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:52:40.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T04:52:40.109+0000 7f9c77422540 -1 init, newargv = 0x55d7a2dfdd50 newargc=15 2023-10-06T04:52:40.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[96975]: starting ceph client 2023-10-06T04:52:40.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[96975]: starting fuse 2023-10-06T04:52:40.126 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:52:40.126 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:52:40.127 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:52:40.128 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T04:52:40.129 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:52:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.96975.asok 2023-10-06T04:52:40.429 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.96975.asok 2023-10-06T04:52:40.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:40.429 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.96975.asok status 2023-10-06T04:52:40.579 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:52:40.579 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:52:40.579 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:52:40.579 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "96975", 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 4544, 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:52:40.580 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 4544 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 4035485636 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:52:40.581 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 4035485636 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.4544 192.168.0.1:0/4035485636", 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/4035485636", 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 7, 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 20, 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:52:40.582 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:52:40.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:52:40.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.721+0000 7f76f1572700 1 Processor -- start 2023-10-06T04:52:40.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.722+0000 7f76f1572700 1 -- start start 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.722+0000 7f76f1572700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f76ec15ccc0 0x7f76ec0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.722+0000 7f76f1572700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec0d06a0 0x7f76ec0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.722+0000 7f76f1572700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ec0cf6c0 con 0x7f76ec15c8f0 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.722+0000 7f76f1572700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ec0cfb50 con 0x7f76ec15ccc0 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.722+0000 7f76f1572700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0d0ad0 con 0x7f76ec0d06a0 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eaffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f76ec15c8f0 0x7f76ec15db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35894/0 (socket says 172.21.15.3:35894) 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eaffd700 1 -- 172.21.15.3:0/322197781 learned_addr learned my addr 172.21.15.3:0/322197781 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:40.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eb7fe700 1 --2- 172.21.15.3:0/322197781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec0d06a0 0x7f76ec0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76ea7fc700 1 --2- 172.21.15.3:0/322197781 >> v2:172.21.15.153:3300/0 conn(0x7f76ec15ccc0 0x7f76ec0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76e9ffb700 1 -- 172.21.15.3:0/322197781 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826176275 0 0) 0x7f76ec0cf6c0 con 0x7f76ec15c8f0 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76e9ffb700 1 -- 172.21.15.3:0/322197781 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76d8003570 con 0x7f76ec15c8f0 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76e9ffb700 1 -- 172.21.15.3:0/322197781 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2462883308 0 0) 0x7f76d8003570 con 0x7f76ec15c8f0 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eb7fe700 1 -- 172.21.15.3:0/322197781 >> v2:172.21.15.153:3300/0 conn(0x7f76ec15ccc0 msgr2=0x7f76ec0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eb7fe700 1 --2- 172.21.15.3:0/322197781 >> v2:172.21.15.153:3300/0 conn(0x7f76ec15ccc0 0x7f76ec0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eb7fe700 1 -- 172.21.15.3:0/322197781 >> v1:172.21.15.3:6789/0 conn(0x7f76ec15c8f0 legacy=0x7f76ec15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eb7fe700 1 -- 172.21.15.3:0/322197781 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0ceab0 con 0x7f76ec0d06a0 2023-10-06T04:52:40.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76ea7fc700 1 --2- 172.21.15.3:0/322197781 >> v2:172.21.15.153:3300/0 conn(0x7f76ec15ccc0 0x7f76ec0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76eb7fe700 1 --2- 172.21.15.3:0/322197781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec0d06a0 0x7f76ec0b3ba0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098e0 tx=0x7f76e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932f424aba97ed33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.723+0000 7f76e9ffb700 1 -- 172.21.15.3:0/322197781 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0017f0 con 0x7f76ec0d06a0 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76e9ffb700 1 -- 172.21.15.3:0/322197781 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4011420 con 0x7f76ec0d06a0 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76e9ffb700 1 -- 172.21.15.3:0/322197781 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40117c0 con 0x7f76ec0d06a0 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 -- 172.21.15.3:0/322197781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec0d06a0 msgr2=0x7f76ec0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 --2- 172.21.15.3:0/322197781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec0d06a0 0x7f76ec0b3ba0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098e0 tx=0x7f76e40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 -- 172.21.15.3:0/322197781 shutdown_connections 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 --2- 172.21.15.3:0/322197781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec0d06a0 0x7f76ec0b3ba0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 --2- 172.21.15.3:0/322197781 >> v2:172.21.15.153:3300/0 conn(0x7f76ec15ccc0 0x7f76ec0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 -- 172.21.15.3:0/322197781 >> 172.21.15.3:0/322197781 conn(0x7f76ec070310 msgr2=0x7f76ec070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:40.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 -- 172.21.15.3:0/322197781 shutdown_connections 2023-10-06T04:52:40.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.724+0000 7f76f1572700 1 -- 172.21.15.3:0/322197781 wait complete. 2023-10-06T04:52:40.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.725+0000 7f76f1572700 1 Processor -- start 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.725+0000 7f76f1572700 1 -- start start 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76f1572700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76ec0d06a0 0x7f76ec0c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76f1572700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec15c8f0 0x7f76ec15b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76f1572700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ec15ccc0 0x7f76ec0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76f1572700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec158d40 con 0x7f76ec15ccc0 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76f1572700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec158f90 con 0x7f76ec0d06a0 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76f1572700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec159270 con 0x7f76ec15c8f0 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ec15ccc0 0x7f76ec0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76ea7fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec15c8f0 0x7f76ec15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76eaffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76ec0d06a0 0x7f76ec0c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:40.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ec15ccc0 0x7f76ec0c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57818/0 (socket says 172.21.15.3:57818) 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.726+0000 7f76eb7fe700 1 -- 172.21.15.3:0/3986188772 learned_addr learned my addr 172.21.15.3:0/3986188772 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76eb7fe700 1 -- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec15c8f0 msgr2=0x7f76ec15b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76eb7fe700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec15c8f0 0x7f76ec15b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76eb7fe700 1 -- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76ec0d06a0 msgr2=0x7f76ec0c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76eb7fe700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76ec0d06a0 0x7f76ec0c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76eb7fe700 1 -- 172.21.15.3:0/3986188772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4009420 con 0x7f76ec15ccc0 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76eaffd700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76ec0d06a0 0x7f76ec0c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76eb7fe700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ec15ccc0 0x7f76ec0c9230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f76e40030c0 tx=0x7f76e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76e2ffd700 1 -- 172.21.15.3:0/3986188772 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e401cd10 con 0x7f76ec15ccc0 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76e2ffd700 1 -- 172.21.15.3:0/3986188772 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e401ce70 con 0x7f76ec15ccc0 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e4009420 con 0x7f76ec15ccc0 2023-10-06T04:52:40.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76e2ffd700 1 -- 172.21.15.3:0/3986188772 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e401a680 con 0x7f76ec15ccc0 2023-10-06T04:52:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.727+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec159830 con 0x7f76ec15ccc0 2023-10-06T04:52:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.728+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ec159a50 con 0x7f76ec15ccc0 2023-10-06T04:52:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.729+0000 7f76e2ffd700 1 -- 172.21.15.3:0/3986188772 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76e401aa70 con 0x7f76ec15ccc0 2023-10-06T04:52:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.729+0000 7f76e2ffd700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f76cc066f30 0x7f76cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.729+0000 7f76e2ffd700 1 -- 172.21.15.3:0/3986188772 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f76e4086020 con 0x7f76ec15ccc0 2023-10-06T04:52:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.729+0000 7f76eaffd700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f76cc066f30 0x7f76cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.730+0000 7f76eaffd700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f76cc066f30 0x7f76cc0693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76dc001560 tx=0x7f76dc001640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:40.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.733+0000 7f76e2ffd700 1 -- 172.21.15.3:0/3986188772 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76e4053170 con 0x7f76ec15ccc0 2023-10-06T04:52:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.978+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f76ec071680 con 0x7f76ec15ccc0 2023-10-06T04:52:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.978+0000 7f76e2ffd700 1 -- 172.21.15.3:0/3986188772 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f76e4056750 con 0x7f76ec15ccc0 2023-10-06T04:52:40.981 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f76cc066f30 msgr2=0x7f76cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f76cc066f30 0x7f76cc0693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76dc001560 tx=0x7f76dc001640 comp rx=0 tx=0).stop 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ec15ccc0 msgr2=0x7f76ec0c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ec15ccc0 0x7f76ec0c9230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f76e40030c0 tx=0x7f76e4016000 comp rx=0 tx=0).stop 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 shutdown_connections 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f76cc066f30 0x7f76cc0693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76ec0d06a0 0x7f76ec0c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ec15ccc0 0x7f76ec0c9230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 --2- 172.21.15.3:0/3986188772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76ec15c8f0 0x7f76ec15b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 >> 172.21.15.3:0/3986188772 conn(0x7f76ec070310 msgr2=0x7f76ec0af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 shutdown_connections 2023-10-06T04:52:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:40.980+0000 7f76f1572700 1 -- 172.21.15.3:0/3986188772 wait complete. 2023-10-06T04:52:40.994 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:52:40.994 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:52:40.994 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:52:40.994 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T04:52:40.994 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T04:52:40.995 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:52:40.995 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:52:40.995 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:52:40.995 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T04:52:40.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:40.995 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:52:41.011 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:41.011 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:41.011 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:41.011 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.011 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:41.012 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.012 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:41.012 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:41.012 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:52:41.012 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:41.012 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:41.012 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42472sec preferred_lft 42472sec 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:52:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link tentative 2023-10-06T04:52:41.014 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.015 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:52:41.074 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:52:41.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.076 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:52:41.133 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:52:41.133 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:52:41.133 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:52:41.133 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:52:41.133 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:52:41.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.134 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:41.134 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:41.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:52:41.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:52:41.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:52:41.134 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:41.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.233 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:52:41.247 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:52:41.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.248 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2023-10-06T04:52:41.302 INFO:teuthology.orchestra.run.smithi153.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T04:52:41.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.303 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:41.303 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:41.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:52:41.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T04:52:41.303 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:41.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.391 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether e6:12:ef:5f:5c:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:52:41.428 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.429 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::e412:efff:fe5f:5cf2/64 scope link 2023-10-06T04:52:41.429 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:52:41.517 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether e6:12:ef:5f:5c:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:52:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:52:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::e412:efff:fe5f:5cf2/64 scope link 2023-10-06T04:52:41.518 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:52:41.530 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T04:52:41.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T04:52:41.531 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:41.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:41.749 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:52:41.749 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:52:41.749 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.1 up 2023-10-06T04:52:41.749 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T04:52:41.750 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:52:41.842 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-06T04:52:41.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:41.842 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T04:52:41.857 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T04:52:41.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:52:41.857 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T04:52:41.911 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:52:41.976 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:41.976 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:42.040 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:42.041 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:42.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.041 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:52:42.097 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:52:42.097 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-10-06T04:52:42.097 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-06T04:52:42.139 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:52:42.164 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:52:42.164 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:52:42.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi153.stderr:2023-10-06T04:52:42.219+0000 7f64fef12540 -1 init, newargv = 0x55e1ffe3df50 newargc=15 2023-10-06T04:52:42.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi153.stderr:ceph-fuse[97464]: starting ceph client 2023-10-06T04:52:42.225 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi153.stderr:ceph-fuse[97464]: starting fuse 2023-10-06T04:52:42.230 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:52:42.231 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:52:42.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.232 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:52:42.287 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:52:42.287 INFO:teuthology.orchestra.run.smithi153.stdout:48 2023-10-06T04:52:42.287 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-10-06T04:52:42.287 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:52:42.287 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:52:42.287 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.3")) 2023-10-06T04:52:42.289 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:52:42.406 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.3.97464.asok 2023-10-06T04:52:42.589 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.97464.asok 2023-10-06T04:52:42.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.590 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.97464.asok status 2023-10-06T04:52:42.741 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:52:42.741 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "3", 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "97464", 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:52:42.742 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 4553, 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 4553 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 1115852659 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:52:42.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 1115852659 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.4553 192.168.0.2:0/1115852659", 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.2:0/1115852659", 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 7, 2023-10-06T04:52:42.744 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 20, 2023-10-06T04:52:42.745 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:52:42.745 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:52:42.745 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:52:42.745 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:52:42.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.751 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:52:42.751 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:52:42.771 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:52:42.771 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:52:42.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:52:42.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.844 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:52:42.844 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-06T04:52:42.902 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:52:42.902 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-06T04:52:42.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T04:52:42.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.974 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:52:42.974 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:52:42.990 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:52:42.990 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:52:42.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:42.990 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:52:43.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:43.054 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:52:43.055 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-06T04:52:43.110 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:52:43.110 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-06T04:52:43.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:52:43.111 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T04:52:43.174 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T04:52:43.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.323+0000 7fd0f763a700 1 Processor -- start 2023-10-06T04:52:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.323+0000 7fd0f763a700 1 -- start start 2023-10-06T04:52:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f763a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157030 0x7fd0f0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f763a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd0f015a220 0x7fd0f00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f763a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f015aeb0 con 0x7fd0f0157400 2023-10-06T04:52:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f763a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f015a7a0 con 0x7fd0f015a220 2023-10-06T04:52:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f763a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f015aa80 con 0x7fd0f0157030 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f5bd7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd0f0157400 0x7fd0f0069340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53852/0 (socket says 172.21.15.3:53852) 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f5bd7700 1 -- 172.21.15.3:0/1372460770 learned_addr learned my addr 172.21.15.3:0/1372460770 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f53d6700 1 --2- 172.21.15.3:0/1372460770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157030 0x7fd0f0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f4bd5700 1 --2- 172.21.15.3:0/1372460770 >> v2:172.21.15.153:3300/0 conn(0x7fd0f015a220 0x7fd0f00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0effff700 1 -- 172.21.15.3:0/1372460770 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199827538 0 0) 0x7fd0f015aeb0 con 0x7fd0f0157400 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0effff700 1 -- 172.21.15.3:0/1372460770 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d8003570 con 0x7fd0f0157400 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f53d6700 1 -- 172.21.15.3:0/1372460770 >> v2:172.21.15.153:3300/0 conn(0x7fd0f015a220 msgr2=0x7fd0f00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f53d6700 1 --2- 172.21.15.3:0/1372460770 >> v2:172.21.15.153:3300/0 conn(0x7fd0f015a220 0x7fd0f00b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f53d6700 1 -- 172.21.15.3:0/1372460770 >> v1:172.21.15.3:6789/0 conn(0x7fd0f0157400 legacy=0x7fd0f0069340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f53d6700 1 -- 172.21.15.3:0/1372460770 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f006dba0 con 0x7fd0f0157030 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f4bd5700 1 --2- 172.21.15.3:0/1372460770 >> v2:172.21.15.153:3300/0 conn(0x7fd0f015a220 0x7fd0f00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.324+0000 7fd0f53d6700 1 --2- 172.21.15.3:0/1372460770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157030 0x7fd0f0159ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80098e0 tx=0x7fd0e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef77969c6aa76682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0effff700 1 -- 172.21.15.3:0/1372460770 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8011630 con 0x7fd0f0157030 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0effff700 1 -- 172.21.15.3:0/1372460770 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e8011790 con 0x7fd0f0157030 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0effff700 1 -- 172.21.15.3:0/1372460770 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8011b30 con 0x7fd0f0157030 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 -- 172.21.15.3:0/1372460770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157030 msgr2=0x7fd0f0159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1372460770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157030 0x7fd0f0159ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80098e0 tx=0x7fd0e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 -- 172.21.15.3:0/1372460770 shutdown_connections 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1372460770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157030 0x7fd0f0159ce0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1372460770 >> v2:172.21.15.153:3300/0 conn(0x7fd0f015a220 0x7fd0f00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 -- 172.21.15.3:0/1372460770 >> 172.21.15.3:0/1372460770 conn(0x7fd0f00ce270 msgr2=0x7fd0f00ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:43.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 -- 172.21.15.3:0/1372460770 shutdown_connections 2023-10-06T04:52:43.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.325+0000 7fd0f763a700 1 -- 172.21.15.3:0/1372460770 wait complete. 2023-10-06T04:52:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.326+0000 7fd0f763a700 1 Processor -- start 2023-10-06T04:52:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.327+0000 7fd0f763a700 1 -- start start 2023-10-06T04:52:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.327+0000 7fd0f763a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f0157030 0x7fd0f013ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.327+0000 7fd0f763a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157400 0x7fd0f013f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.327+0000 7fd0f763a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0f015a220 0x7fd0f00c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.327+0000 7fd0f763a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00b0580 con 0x7fd0f015a220 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.327+0000 7fd0f763a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f0140480 con 0x7fd0f0157030 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.327+0000 7fd0f763a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f0140720 con 0x7fd0f0157400 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0f015a220 0x7fd0f00c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0f015a220 0x7fd0f00c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47912/0 (socket says 172.21.15.3:47912) 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 -- 172.21.15.3:0/1857091891 learned_addr learned my addr 172.21.15.3:0/1857091891 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f4bd5700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157400 0x7fd0f013f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f53d6700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f0157030 0x7fd0f013ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 -- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157400 msgr2=0x7fd0f013f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157400 0x7fd0f013f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 -- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f0157030 msgr2=0x7fd0f013ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f0157030 0x7fd0f013ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 -- 172.21.15.3:0/1857091891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e4009420 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f5bd7700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0f015a220 0x7fd0f00c7420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0001460 tx=0x7fd0e0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0ed7fa700 1 -- 172.21.15.3:0/1857091891 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e000ce80 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0ed7fa700 1 -- 172.21.15.3:0/1857091891 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e000ab70 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e8009420 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0ed7fa700 1 -- 172.21.15.3:0/1857091891 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00070f0 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.328+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f0140ce0 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.329+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00644c0 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.330+0000 7fd0ed7fa700 1 -- 172.21.15.3:0/1857091891 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0e0007520 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.330+0000 7fd0ed7fa700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.330+0000 7fd0ed7fa700 1 -- 172.21.15.3:0/1857091891 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd0e0084bb0 con 0x7fd0f015a220 2023-10-06T04:52:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.330+0000 7fd0f53d6700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.331+0000 7fd0f53d6700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0dc066f30 0x7fd0dc0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80091e0 tx=0x7fd0e8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:43.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.334+0000 7fd0ed7fa700 1 -- 172.21.15.3:0/1857091891 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0e0051cb0 con 0x7fd0f015a220 2023-10-06T04:52:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.529+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0f0071680 con 0x7fd0f015a220 2023-10-06T04:52:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.530+0000 7fd0ed7fa700 1 -- 172.21.15.3:0/1857091891 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fd0e00552d0 con 0x7fd0f015a220 2023-10-06T04:52:43.533 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:43.533 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:43.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0dc066f30 msgr2=0x7fd0dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0dc066f30 0x7fd0dc0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80091e0 tx=0x7fd0e8017000 comp rx=0 tx=0).stop 2023-10-06T04:52:43.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0f015a220 msgr2=0x7fd0f00c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0f015a220 0x7fd0f00c7420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0001460 tx=0x7fd0e0001ae0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 shutdown_connections 2023-10-06T04:52:43.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f0157030 0x7fd0f013ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0f015a220 0x7fd0f00c7420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 --2- 172.21.15.3:0/1857091891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0f0157400 0x7fd0f013f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 >> 172.21.15.3:0/1857091891 conn(0x7fd0f00ce270 msgr2=0x7fd0f00b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:43.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 shutdown_connections 2023-10-06T04:52:43.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.532+0000 7fd0f763a700 1 -- 172.21.15.3:0/1857091891 wait complete. 2023-10-06T04:52:43.546 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:52:15.735846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:52:17.015278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:52:43.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.666+0000 7fc6c602e700 1 Processor -- start 2023-10-06T04:52:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.666+0000 7fc6c602e700 1 -- start start 2023-10-06T04:52:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6c602e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 0x7fc6c00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6c602e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00cb190 con 0x7fc6c00cd190 2023-10-06T04:52:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6c602e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c0159950 con 0x7fc6c00af290 2023-10-06T04:52:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6c602e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00cd6a0 con 0x7fc6c00af660 2023-10-06T04:52:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6c582d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc6c00cd190 0x7fc6c00c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53860/0 (socket says 172.21.15.3:53860) 2023-10-06T04:52:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6c582d700 1 -- 172.21.15.3:0/3009211681 learned_addr learned my addr 172.21.15.3:0/3009211681 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6c482b700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 0x7fc6c00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495181736 0 0) 0x7fc6c00cb190 con 0x7fc6c00cd190 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.667+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a8003570 con 0x7fc6c00cd190 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969033773 0 0) 0x7fc6a8003570 con 0x7fc6c00cd190 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 msgr2=0x7fc6c00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 0x7fc6c00ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 >> v1:172.21.15.153:6789/0 conn(0x7fc6c00af290 legacy=0x7fc6c00ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00cb3f0 con 0x7fc6c00cd190 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc6b0001470 con 0x7fc6c00cd190 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 >> v1:172.21.15.3:6789/0 conn(0x7fc6c00cd190 legacy=0x7fc6c00c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 0x7fc6a8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8003fd0 0x7fc6a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8004d50 0x7fc6a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00cd6a0 con 0x7fc6a8003fd0 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a800bb40 con 0x7fc6a8004d50 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a800bd00 con 0x7fc6c00af660 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c482b700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8003fd0 0x7fc6a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c502c700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 0x7fc6a8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c582d700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8004d50 0x7fc6a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c482b700 1 -- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 msgr2=0x7fc6a8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c482b700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00af660 0x7fc6a8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c482b700 1 -- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8004d50 msgr2=0x7fc6a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c482b700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8004d50 0x7fc6a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c482b700 1 -- 172.21.15.3:0/3009211681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8003570 con 0x7fc6a8003fd0 2023-10-06T04:52:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c582d700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8004d50 0x7fc6a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6c482b700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8003fd0 0x7fc6a8004810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009bf0 tx=0x7fc6b4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64d9175e3de364c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4005a00 con 0x7fc6a8003fd0 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b4005b60 con 0x7fc6a8003fd0 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.668+0000 7fc6bffff700 1 -- 172.21.15.3:0/3009211681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400e710 con 0x7fc6a8003fd0 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.669+0000 7fc6c602e700 1 -- 172.21.15.3:0/3009211681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8003fd0 msgr2=0x7fc6a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.669+0000 7fc6c602e700 1 --2- 172.21.15.3:0/3009211681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8003fd0 0x7fc6a8004810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009bf0 tx=0x7fc6b4009120 comp rx=0 tx=0).stop 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.669+0000 7fc6c582d700 1 -- 172.21.15.3:0/3009211681 reap_dead start 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.669+0000 7fc6c602e700 1 -- 172.21.15.3:0/3009211681 shutdown_connections 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.669+0000 7fc6c602e700 1 -- 172.21.15.3:0/3009211681 >> 172.21.15.3:0/3009211681 conn(0x7fc6c0089dd0 msgr2=0x7fc6c00996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.669+0000 7fc6c602e700 1 -- 172.21.15.3:0/3009211681 shutdown_connections 2023-10-06T04:52:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.669+0000 7fc6c602e700 1 -- 172.21.15.3:0/3009211681 wait complete. 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.670+0000 7fc6c602e700 1 Processor -- start 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.670+0000 7fc6c602e700 1 -- start start 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c602e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8003fd0 0x7fc6c014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c602e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00b2960 0x7fc6c0169860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c602e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c00b37c0 0x7fc6c00b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c602e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00b1250 con 0x7fc6c00b37c0 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c602e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00b3050 con 0x7fc6a8003fd0 2023-10-06T04:52:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c602e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00b3330 con 0x7fc6c00b2960 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c00b37c0 0x7fc6c00b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c00b37c0 0x7fc6c00b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47924/0 (socket says 172.21.15.3:47924) 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 -- 172.21.15.3:0/3751927653 learned_addr learned my addr 172.21.15.3:0/3751927653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c502c700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8003fd0 0x7fc6c014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c482b700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00b2960 0x7fc6c0169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 -- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00b2960 msgr2=0x7fc6c0169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00b2960 0x7fc6c0169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 -- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8003fd0 msgr2=0x7fc6c014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8003fd0 0x7fc6c014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.671+0000 7fc6c582d700 1 -- 172.21.15.3:0/3751927653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c01504a0 con 0x7fc6c00b37c0 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.672+0000 7fc6c582d700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c00b37c0 0x7fc6c00b3c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00cd560 tx=0x7fc6b0001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.672+0000 7fc6bd7fa700 1 -- 172.21.15.3:0/3751927653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0010520 con 0x7fc6c00b37c0 2023-10-06T04:52:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.672+0000 7fc6bd7fa700 1 -- 172.21.15.3:0/3751927653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b00106a0 con 0x7fc6c00b37c0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.672+0000 7fc6bd7fa700 1 -- 172.21.15.3:0/3751927653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0010a60 con 0x7fc6c00b37c0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.672+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c016a100 con 0x7fc6c00b37c0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.672+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c016a6c0 con 0x7fc6c00b37c0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.673+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c00b34c0 con 0x7fc6c00b37c0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.673+0000 7fc6bd7fa700 1 -- 172.21.15.3:0/3751927653 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6b00133e0 con 0x7fc6c00b37c0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.674+0000 7fc6bd7fa700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc6ac066f30 0x7fc6ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.674+0000 7fc6bd7fa700 1 -- 172.21.15.3:0/3751927653 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc6b0083fe0 con 0x7fc6c00b37c0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.674+0000 7fc6c502c700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc6ac066f30 0x7fc6ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.674+0000 7fc6c502c700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc6ac066f30 0x7fc6ac0693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00ce860 tx=0x7fc6b8001b10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.678+0000 7fc6bd7fa700 1 -- 172.21.15.3:0/3751927653 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6b0054b80 con 0x7fc6c00b37c0 2023-10-06T04:52:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.874+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6c004c380 con 0x7fc6c00b37c0 2023-10-06T04:52:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.875+0000 7fc6bd7fa700 1 -- 172.21.15.3:0/3751927653 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6b0054710 con 0x7fc6c00b37c0 2023-10-06T04:52:43.877 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.876+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc6ac066f30 msgr2=0x7fc6ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.876+0000 7fc6c602e700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc6ac066f30 0x7fc6ac0693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00ce860 tx=0x7fc6b8001b10 comp rx=0 tx=0).stop 2023-10-06T04:52:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.876+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c00b37c0 msgr2=0x7fc6c00b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.876+0000 7fc6c602e700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c00b37c0 0x7fc6c00b3c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00cd560 tx=0x7fc6b0001470 comp rx=0 tx=0).stop 2023-10-06T04:52:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 shutdown_connections 2023-10-06T04:52:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc6ac066f30 0x7fc6ac0693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6a8003fd0 0x7fc6c014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c00b37c0 0x7fc6c00b3c30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 --2- 172.21.15.3:0/3751927653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6c00b2960 0x7fc6c0169860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 >> 172.21.15.3:0/3751927653 conn(0x7fc6c0089dd0 msgr2=0x7fc6c006dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 shutdown_connections 2023-10-06T04:52:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:43.877+0000 7fc6c602e700 1 -- 172.21.15.3:0/3751927653 wait complete. 2023-10-06T04:52:43.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:44.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.009+0000 7fd47416e700 1 Processor -- start 2023-10-06T04:52:44.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.010+0000 7fd47416e700 1 -- start start 2023-10-06T04:52:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.010+0000 7fd47416e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c0ccd00 0x7fd46c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.010+0000 7fd47416e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd46c157570 0x7fd46c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.010+0000 7fd47416e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c15be90 con 0x7fd46c1571a0 2023-10-06T04:52:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.010+0000 7fd47416e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c15b030 con 0x7fd46c157570 2023-10-06T04:52:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.010+0000 7fd47416e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c15b280 con 0x7fd46c0ccd00 2023-10-06T04:52:44.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd471709700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c0ccd00 0x7fd46c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd46c157570 0x7fd46c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd46c157570 0x7fd46c0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:43110/0 (socket says 172.21.15.3:43110) 2023-10-06T04:52:44.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd471f0a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd46c1571a0 0x7fd46c0cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34902/0 (socket says 172.21.15.3:34902) 2023-10-06T04:52:44.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 -- 172.21.15.3:0/2281581707 learned_addr learned my addr 172.21.15.3:0/2281581707 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:44.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 -- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c0ccd00 msgr2=0x7fd46c0b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:44.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 --2- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c0ccd00 0x7fd46c0b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 -- 172.21.15.3:0/2281581707 >> v1:172.21.15.153:6789/0 conn(0x7fd46c1571a0 legacy=0x7fd46c0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 -- 172.21.15.3:0/2281581707 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c15b560 con 0x7fd46c157570 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd471709700 1 --2- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c0ccd00 0x7fd46c0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd470f08700 1 -- 172.21.15.3:0/2281581707 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237982484 0 0) 0x7fd46c15be90 con 0x7fd46c1571a0 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd47270b700 1 --2- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c157570 0x7fd46c0b2720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009f20 tx=0x7fd45c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16f7247d1b5aa9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd470f08700 1 -- 172.21.15.3:0/2281581707 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00e070 con 0x7fd46c157570 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd470f08700 1 -- 172.21.15.3:0/2281581707 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c00b9d0 con 0x7fd46c157570 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.011+0000 7fd470f08700 1 -- 172.21.15.3:0/2281581707 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0092e0 con 0x7fd46c157570 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 -- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c157570 msgr2=0x7fd46c0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 --2- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c157570 0x7fd46c0b2720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009f20 tx=0x7fd45c009840 comp rx=0 tx=0).stop 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 -- 172.21.15.3:0/2281581707 shutdown_connections 2023-10-06T04:52:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 --2- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c157570 0x7fd46c0b2720 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 --2- 172.21.15.3:0/2281581707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c0ccd00 0x7fd46c0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 -- 172.21.15.3:0/2281581707 >> 172.21.15.3:0/2281581707 conn(0x7fd46c06df50 msgr2=0x7fd46c0aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 -- 172.21.15.3:0/2281581707 shutdown_connections 2023-10-06T04:52:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.012+0000 7fd47416e700 1 -- 172.21.15.3:0/2281581707 wait complete. 2023-10-06T04:52:44.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.013+0000 7fd47416e700 1 Processor -- start 2023-10-06T04:52:44.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.013+0000 7fd47416e700 1 -- start start 2023-10-06T04:52:44.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47416e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c0ccd00 0x7fd46c146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47416e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c1571a0 0x7fd46c0aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47416e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c157570 0x7fd46c0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47416e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0b09a0 con 0x7fd46c0ccd00 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47416e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0aaea0 con 0x7fd46c1571a0 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47416e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0b0070 con 0x7fd46c157570 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c0ccd00 0x7fd46c146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c0ccd00 0x7fd46c146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47930/0 (socket says 172.21.15.3:47930) 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 -- 172.21.15.3:0/3741654709 learned_addr learned my addr 172.21.15.3:0/3741654709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471709700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c1571a0 0x7fd46c0aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47270b700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c157570 0x7fd46c0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 -- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c157570 msgr2=0x7fd46c0ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c157570 0x7fd46c0ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 -- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c1571a0 msgr2=0x7fd46c0aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c1571a0 0x7fd46c0aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 -- 172.21.15.3:0/3741654709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460009420 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd47270b700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c157570 0x7fd46c0ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.014+0000 7fd471f0a700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c0ccd00 0x7fd46c146aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4640011e0 tx=0x7fd464007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.015+0000 7fd469ffb700 1 -- 172.21.15.3:0/3741654709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464005a70 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.015+0000 7fd469ffb700 1 -- 172.21.15.3:0/3741654709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd464005bd0 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.015+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c009ae0 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.015+0000 7fd469ffb700 1 -- 172.21.15.3:0/3741654709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640070d0 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.015+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c0b0630 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.016+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c04c380 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.016+0000 7fd469ffb700 1 -- 172.21.15.3:0/3741654709 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd464007480 con 0x7fd46c0ccd00 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.016+0000 7fd469ffb700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd458066f30 0x7fd4580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.016+0000 7fd469ffb700 1 -- 172.21.15.3:0/3741654709 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd464083b60 con 0x7fd46c0ccd00 2023-10-06T04:52:44.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.017+0000 7fd471709700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd458066f30 0x7fd4580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.017+0000 7fd471709700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd458066f30 0x7fd4580693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd46c154f40 tx=0x7fd4600091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.021+0000 7fd469ffb700 1 -- 172.21.15.3:0/3741654709 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd464050d20 con 0x7fd46c0ccd00 2023-10-06T04:52:44.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.214+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd46c15b840 con 0x7fd46c0ccd00 2023-10-06T04:52:44.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.214+0000 7fd469ffb700 1 -- 172.21.15.3:0/3741654709 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd464017070 con 0x7fd46c0ccd00 2023-10-06T04:52:44.217 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd458066f30 msgr2=0x7fd4580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd458066f30 0x7fd4580693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd46c154f40 tx=0x7fd4600091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c0ccd00 msgr2=0x7fd46c146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c0ccd00 0x7fd46c146aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4640011e0 tx=0x7fd464007e10 comp rx=0 tx=0).stop 2023-10-06T04:52:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 shutdown_connections 2023-10-06T04:52:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd458066f30 0x7fd4580693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd46c157570 0x7fd46c0ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c0ccd00 0x7fd46c146aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 --2- 172.21.15.3:0/3741654709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd46c1571a0 0x7fd46c0aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 >> 172.21.15.3:0/3741654709 conn(0x7fd46c06df50 msgr2=0x7fd46c150430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 shutdown_connections 2023-10-06T04:52:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.216+0000 7fd47416e700 1 -- 172.21.15.3:0/3741654709 wait complete. 2023-10-06T04:52:44.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:52:44.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.350+0000 7f89a7169700 1 Processor -- start 2023-10-06T04:52:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.350+0000 7f89a7169700 1 -- start start 2023-10-06T04:52:44.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a7169700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89a00af290 0x7f89a00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a7169700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a00cb190 con 0x7f89a00af660 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a7169700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a0159950 con 0x7f89a00c5c20 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a7169700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a00cd6a0 con 0x7f89a00af290 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a5966700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f89a00af660 0x7f89a00ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53862/0 (socket says 172.21.15.3:53862) 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a5966700 1 -- 172.21.15.3:0/3273298707 learned_addr learned my addr 172.21.15.3:0/3273298707 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a6167700 1 --2- 172.21.15.3:0/3273298707 >> v2:172.21.15.153:3300/0 conn(0x7f89a00af290 0x7f89a00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680915034 0 0) 0x7f89a00cb190 con 0x7f89a00af660 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8988003570 con 0x7f89a00af660 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323267019 0 0) 0x7f89a0159950 con 0x7f89a00c5c20 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.351+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a00cb190 con 0x7f89a00c5c20 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a6167700 1 -- 172.21.15.3:0/3273298707 >> v1:172.21.15.153:6789/0 conn(0x7f89a00c5c20 legacy=0x7f89a00cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 628901343 0 0) 0x7f8988003570 con 0x7f89a00af660 2023-10-06T04:52:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a6167700 1 -- 172.21.15.3:0/3273298707 >> v1:172.21.15.3:6789/0 conn(0x7f89a00af660 legacy=0x7f89a00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a6167700 1 -- 172.21.15.3:0/3273298707 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00cb3f0 con 0x7f89a00af290 2023-10-06T04:52:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a6167700 1 --2- 172.21.15.3:0/3273298707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a00ce1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89a0159b40 tx=0x7f8998009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f88c34b7991063 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940015f0 con 0x7f89a00af290 2023-10-06T04:52:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998005a00 con 0x7f89a00af290 2023-10-06T04:52:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a5165700 1 -- 172.21.15.3:0/3273298707 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998005da0 con 0x7f89a00af290 2023-10-06T04:52:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a7169700 1 -- 172.21.15.3:0/3273298707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 msgr2=0x7f89a00ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.352+0000 7f89a7169700 1 --2- 172.21.15.3:0/3273298707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a00ce1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89a0159b40 tx=0x7f8998009120 comp rx=0 tx=0).stop 2023-10-06T04:52:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.353+0000 7f89a7169700 1 -- 172.21.15.3:0/3273298707 shutdown_connections 2023-10-06T04:52:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.353+0000 7f89a7169700 1 --2- 172.21.15.3:0/3273298707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a00ce1b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.353+0000 7f89a7169700 1 -- 172.21.15.3:0/3273298707 >> 172.21.15.3:0/3273298707 conn(0x7f89a0089dd0 msgr2=0x7f89a00996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.353+0000 7f89a7169700 1 -- 172.21.15.3:0/3273298707 shutdown_connections 2023-10-06T04:52:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.353+0000 7f89a7169700 1 -- 172.21.15.3:0/3273298707 wait complete. 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 Processor -- start 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 -- start start 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a016a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89a00af660 0x7f89a015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89a00c5c20 0x7f89a015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00b29b0 con 0x7f89a00af660 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00b3280 con 0x7f89a00af290 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.354+0000 7f89a7169700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00b2110 con 0x7f89a00c5c20 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89a00af660 0x7f89a015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89a00af660 0x7f89a015dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47938/0 (socket says 172.21.15.3:47938) 2023-10-06T04:52:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 -- 172.21.15.3:0/1737831458 learned_addr learned my addr 172.21.15.3:0/1737831458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a6968700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89a00c5c20 0x7f89a015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a6167700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a016a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 -- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89a00c5c20 msgr2=0x7f89a015e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89a00c5c20 0x7f89a015e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 -- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 msgr2=0x7f89a016a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a016a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 -- 172.21.15.3:0/1737831458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980093c0 con 0x7f89a00af660 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a6167700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a016a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:44.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a6968700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89a00c5c20 0x7f89a015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a5966700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89a00af660 0x7f89a015dfb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f899400cb60 tx=0x7f8994002bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f899e7fc700 1 -- 172.21.15.3:0/1737831458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994007300 con 0x7f89a00af660 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f899e7fc700 1 -- 172.21.15.3:0/1737831458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8994007460 con 0x7f89a00af660 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a00b23f0 con 0x7f89a00af660 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f899e7fc700 1 -- 172.21.15.3:0/1737831458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940057f0 con 0x7f89a00af660 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.355+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a016ac00 con 0x7f89a00af660 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.356+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a00b3050 con 0x7f89a00af660 2023-10-06T04:52:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.357+0000 7f899e7fc700 1 -- 172.21.15.3:0/1737831458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8994005ba0 con 0x7f89a00af660 2023-10-06T04:52:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.357+0000 7f899e7fc700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f898c066f30 0x7f898c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.357+0000 7f899e7fc700 1 -- 172.21.15.3:0/1737831458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8994084460 con 0x7f89a00af660 2023-10-06T04:52:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.357+0000 7f89a6167700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f898c066f30 0x7f898c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.358+0000 7f89a6167700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f898c066f30 0x7f898c0693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8998003080 tx=0x7f8998015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.361+0000 7f899e7fc700 1 -- 172.21.15.3:0/1737831458 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89a00b3050 con 0x7f89a00af660 2023-10-06T04:52:44.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.555+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89a00b3050 con 0x7f89a00af660 2023-10-06T04:52:44.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.555+0000 7f899e7fc700 1 -- 172.21.15.3:0/1737831458 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89a00b3050 con 0x7f89a00af660 2023-10-06T04:52:44.558 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:52:44.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f898c066f30 msgr2=0x7f898c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f898c066f30 0x7f898c0693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8998003080 tx=0x7f8998015000 comp rx=0 tx=0).stop 2023-10-06T04:52:44.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89a00af660 msgr2=0x7f89a015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89a00af660 0x7f89a015dfb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f899400cb60 tx=0x7f8994002bb0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 shutdown_connections 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f898c066f30 0x7f898c0693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89a00af290 0x7f89a016a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89a00c5c20 0x7f89a015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 --2- 172.21.15.3:0/1737831458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89a00af660 0x7f89a015dfb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 >> 172.21.15.3:0/1737831458 conn(0x7f89a0089dd0 msgr2=0x7f89a00637f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 shutdown_connections 2023-10-06T04:52:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.557+0000 7f89a7169700 1 -- 172.21.15.3:0/1737831458 wait complete. 2023-10-06T04:52:44.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:52:44.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.690+0000 7f3f325a9700 1 Processor -- start 2023-10-06T04:52:44.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.691+0000 7f3f325a9700 1 -- start start 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.691+0000 7f3f325a9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c0cba50 0x7f3f2c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.691+0000 7f3f325a9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f2c15d540 0x7f3f2c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.691+0000 7f3f325a9700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c15a290 con 0x7f3f2c150690 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.691+0000 7f3f325a9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c0d0020 con 0x7f3f2c15d540 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.691+0000 7f3f325a9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c159f40 con 0x7f3f2c0cba50 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f30b46700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f2c15d540 0x7f3f2c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f2b7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c0cba50 0x7f3f2c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f2bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3f2c150690 0x7f3f2c15a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34910/0 (socket says 172.21.15.3:34910) 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f2bfff700 1 -- 172.21.15.3:0/3702561694 learned_addr learned my addr 172.21.15.3:0/3702561694 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f30b46700 1 -- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c0cba50 msgr2=0x7f3f2c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f30b46700 1 --2- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c0cba50 0x7f3f2c150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f30b46700 1 -- 172.21.15.3:0/3702561694 >> v1:172.21.15.153:6789/0 conn(0x7f3f2c150690 legacy=0x7f3f2c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f30b46700 1 -- 172.21.15.3:0/3702561694 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c15dcb0 con 0x7f3f2c15d540 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f2b7fe700 1 --2- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c0cba50 0x7f3f2c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f30b46700 1 --2- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c15d540 0x7f3f2c0af190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009e60 tx=0x7f3f24009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7127027eab8f3060 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.692+0000 7f3f2affd700 1 -- 172.21.15.3:0/3702561694 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400e070 con 0x7f3f2c15d540 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f2affd700 1 -- 172.21.15.3:0/3702561694 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2400bb60 con 0x7f3f2c15d540 2023-10-06T04:52:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f2affd700 1 -- 172.21.15.3:0/3702561694 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24012430 con 0x7f3f2c15d540 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 -- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c15d540 msgr2=0x7f3f2c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c15d540 0x7f3f2c0af190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009e60 tx=0x7f3f24009780 comp rx=0 tx=0).stop 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 -- 172.21.15.3:0/3702561694 shutdown_connections 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c15d540 0x7f3f2c0af190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3702561694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c0cba50 0x7f3f2c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 -- 172.21.15.3:0/3702561694 >> 172.21.15.3:0/3702561694 conn(0x7f3f2c070310 msgr2=0x7f3f2c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 -- 172.21.15.3:0/3702561694 shutdown_connections 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.693+0000 7f3f325a9700 1 -- 172.21.15.3:0/3702561694 wait complete. 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.694+0000 7f3f325a9700 1 Processor -- start 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.694+0000 7f3f325a9700 1 -- start start 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f325a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2c0cba50 0x7f3f2c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f325a9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c150690 0x7f3f2c15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f325a9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c15d540 0x7f3f2c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f325a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c150510 con 0x7f3f2c0cba50 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f325a9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c152400 con 0x7f3f2c150690 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f325a9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0b45d0 con 0x7f3f2c15d540 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2c0cba50 0x7f3f2c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2c0cba50 0x7f3f2c14ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47940/0 (socket says 172.21.15.3:47940) 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f2bfff700 1 -- 172.21.15.3:0/3071330002 learned_addr learned my addr 172.21.15.3:0/3071330002 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f30b46700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c15d540 0x7f3f2c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.695+0000 7f3f2b7fe700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c150690 0x7f3f2c15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f2bfff700 1 -- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c15d540 msgr2=0x7f3f2c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f2bfff700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c15d540 0x7f3f2c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f2bfff700 1 -- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c150690 msgr2=0x7f3f2c15b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f2bfff700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c150690 0x7f3f2c15b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f2bfff700 1 -- 172.21.15.3:0/3071330002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f20009420 con 0x7f3f2c0cba50 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f2bfff700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2c0cba50 0x7f3f2c14ffd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c7c0 tx=0x7f3f1c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f13fff700 1 -- 172.21.15.3:0/3071330002 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c005b30 con 0x7f3f2c0cba50 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f13fff700 1 -- 172.21.15.3:0/3071330002 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c005c90 con 0x7f3f2c0cba50 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f24009a20 con 0x7f3f2c0cba50 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f13fff700 1 -- 172.21.15.3:0/3071330002 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c004a80 con 0x7f3f2c0cba50 2023-10-06T04:52:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.696+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c152090 con 0x7f3f2c0cba50 2023-10-06T04:52:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.697+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2c04c380 con 0x7f3f2c0cba50 2023-10-06T04:52:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.697+0000 7f3f13fff700 1 -- 172.21.15.3:0/3071330002 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f1c0070d0 con 0x7f3f2c0cba50 2023-10-06T04:52:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.698+0000 7f3f13fff700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f18066f30 0x7f3f180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.698+0000 7f3f13fff700 1 -- 172.21.15.3:0/3071330002 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f1c083a10 con 0x7f3f2c0cba50 2023-10-06T04:52:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.698+0000 7f3f2b7fe700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f18066f30 0x7f3f180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.698+0000 7f3f2b7fe700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f18066f30 0x7f3f180693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c1539e0 tx=0x7f3f200091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.702+0000 7f3f13fff700 1 -- 172.21.15.3:0/3071330002 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f1c04b5a0 con 0x7f3f2c0cba50 2023-10-06T04:52:44.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.895+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f2c15df90 con 0x7f3f2c0cba50 2023-10-06T04:52:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.895+0000 7f3f13fff700 1 -- 172.21.15.3:0/3071330002 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f1c050a80 con 0x7f3f2c0cba50 2023-10-06T04:52:44.898 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:52:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f18066f30 msgr2=0x7f3f180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f18066f30 0x7f3f180693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c1539e0 tx=0x7f3f200091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2c0cba50 msgr2=0x7f3f2c14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2c0cba50 0x7f3f2c14ffd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c7c0 tx=0x7f3f1c007f70 comp rx=0 tx=0).stop 2023-10-06T04:52:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 shutdown_connections 2023-10-06T04:52:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f18066f30 0x7f3f180693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2c15d540 0x7f3f2c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2c0cba50 0x7f3f2c14ffd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 --2- 172.21.15.3:0/3071330002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2c150690 0x7f3f2c15b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:44.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 >> 172.21.15.3:0/3071330002 conn(0x7f3f2c070310 msgr2=0x7f3f2c0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:44.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 shutdown_connections 2023-10-06T04:52:44.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:44.897+0000 7f3f325a9700 1 -- 172.21.15.3:0/3071330002 wait complete. 2023-10-06T04:52:44.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:45.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.029+0000 7ff072cc2700 1 Processor -- start 2023-10-06T04:52:45.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.029+0000 7ff072cc2700 1 -- start start 2023-10-06T04:52:45.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff072cc2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff06c15c8f0 0x7ff06c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff072cc2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c0d0870 0x7ff06c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff072cc2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c0cf6c0 con 0x7ff06c15ccc0 2023-10-06T04:52:45.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff072cc2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0cfb50 con 0x7ff06c15c8f0 2023-10-06T04:52:45.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff072cc2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c15e050 con 0x7ff06c0d0870 2023-10-06T04:52:45.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff06bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff06c15ccc0 0x7ff06c0d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53864/0 (socket says 172.21.15.3:53864) 2023-10-06T04:52:45.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff06bfff700 1 -- 172.21.15.3:0/3733080069 learned_addr learned my addr 172.21.15.3:0/3733080069 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff07125f700 1 --2- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c0d0870 0x7ff06c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff070a5e700 1 --2- 172.21.15.3:0/3733080069 >> v2:172.21.15.153:3300/0 conn(0x7ff06c15c8f0 0x7ff06c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff06b7fe700 1 -- 172.21.15.3:0/3733080069 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999151046 0 0) 0x7ff06c0cf6c0 con 0x7ff06c15ccc0 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff06b7fe700 1 -- 172.21.15.3:0/3733080069 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff05c003570 con 0x7ff06c15ccc0 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff070a5e700 1 -- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c0d0870 msgr2=0x7ff06c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff070a5e700 1 --2- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c0d0870 0x7ff06c0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff070a5e700 1 -- 172.21.15.3:0/3733080069 >> v1:172.21.15.3:6789/0 conn(0x7ff06c15ccc0 legacy=0x7ff06c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.030+0000 7ff06b7fe700 1 -- 172.21.15.3:0/3733080069 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421400371 0 0) 0x7ff05c003570 con 0x7ff06c15ccc0 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff070a5e700 1 -- 172.21.15.3:0/3733080069 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0ceab0 con 0x7ff06c15c8f0 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff07125f700 1 --2- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c0d0870 0x7ff06c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff070a5e700 1 --2- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c15c8f0 0x7ff06c15db10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff0600098e0 tx=0x7ff0600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7078962a37b4a1c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff06b7fe700 1 -- 172.21.15.3:0/3733080069 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058001470 con 0x7ff06c15c8f0 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff06b7fe700 1 -- 172.21.15.3:0/3733080069 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060011420 con 0x7ff06c15c8f0 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff06b7fe700 1 -- 172.21.15.3:0/3733080069 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600117c0 con 0x7ff06c15c8f0 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 -- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c15c8f0 msgr2=0x7ff06c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 --2- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c15c8f0 0x7ff06c15db10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff0600098e0 tx=0x7ff0600091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 -- 172.21.15.3:0/3733080069 shutdown_connections 2023-10-06T04:52:45.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 --2- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c0d0870 0x7ff06c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 --2- 172.21.15.3:0/3733080069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c15c8f0 0x7ff06c15db10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 -- 172.21.15.3:0/3733080069 >> 172.21.15.3:0/3733080069 conn(0x7ff06c070310 msgr2=0x7ff06c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:45.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 -- 172.21.15.3:0/3733080069 shutdown_connections 2023-10-06T04:52:45.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.031+0000 7ff072cc2700 1 -- 172.21.15.3:0/3733080069 wait complete. 2023-10-06T04:52:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.032+0000 7ff072cc2700 1 Processor -- start 2023-10-06T04:52:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff072cc2700 1 -- start start 2023-10-06T04:52:45.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff072cc2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c0d0870 0x7ff06c16ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff072cc2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff06c15c8f0 0x7ff06c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff072cc2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c15ccc0 0x7ff06c0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff072cc2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c159dc0 con 0x7ff06c15c8f0 2023-10-06T04:52:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff072cc2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c154ea0 con 0x7ff06c0d0870 2023-10-06T04:52:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff072cc2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c15ac10 con 0x7ff06c15ccc0 2023-10-06T04:52:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.033+0000 7ff06bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff06c15c8f0 0x7ff06c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff06c15c8f0 0x7ff06c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47954/0 (socket says 172.21.15.3:47954) 2023-10-06T04:52:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 -- 172.21.15.3:0/2731082296 learned_addr learned my addr 172.21.15.3:0/2731082296 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff070a5e700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c0d0870 0x7ff06c16ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff07125f700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c15ccc0 0x7ff06c0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 -- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c15ccc0 msgr2=0x7ff06c0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c15ccc0 0x7ff06c0b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 -- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c0d0870 msgr2=0x7ff06c16ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c0d0870 0x7ff06c16ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 -- 172.21.15.3:0/2731082296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009a20 con 0x7ff06c15c8f0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff06bfff700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff06c15c8f0 0x7ff06c154960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff05800cbc0 tx=0x7ff058001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff068ff9700 1 -- 172.21.15.3:0/2731082296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058001b30 con 0x7ff06c15c8f0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff068ff9700 1 -- 172.21.15.3:0/2731082296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff058005b30 con 0x7ff06c15c8f0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff060009420 con 0x7ff06c15c8f0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff068ff9700 1 -- 172.21.15.3:0/2731082296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580070f0 con 0x7ff06c15c8f0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.034+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c15a5c0 con 0x7ff06c15c8f0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.035+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c15ad80 con 0x7ff06c15c8f0 2023-10-06T04:52:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.036+0000 7ff068ff9700 1 -- 172.21.15.3:0/2731082296 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff058007520 con 0x7ff06c15c8f0 2023-10-06T04:52:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.036+0000 7ff068ff9700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff050066f30 0x7ff0500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.036+0000 7ff068ff9700 1 -- 172.21.15.3:0/2731082296 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff058084280 con 0x7ff06c15c8f0 2023-10-06T04:52:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.036+0000 7ff070a5e700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff050066f30 0x7ff0500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.037+0000 7ff070a5e700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff050066f30 0x7ff0500693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0600030c0 tx=0x7ff060016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.040+0000 7ff068ff9700 1 -- 172.21.15.3:0/2731082296 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0580514d0 con 0x7ff06c15c8f0 2023-10-06T04:52:45.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.236+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff06c071680 con 0x7ff06c15c8f0 2023-10-06T04:52:45.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.237+0000 7ff068ff9700 1 -- 172.21.15.3:0/2731082296 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7ff058054af0 con 0x7ff06c15c8f0 2023-10-06T04:52:45.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:45.240 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff050066f30 msgr2=0x7ff0500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff050066f30 0x7ff0500693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0600030c0 tx=0x7ff060016000 comp rx=0 tx=0).stop 2023-10-06T04:52:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff06c15c8f0 msgr2=0x7ff06c154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff06c15c8f0 0x7ff06c154960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff05800cbc0 tx=0x7ff058001aa0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 shutdown_connections 2023-10-06T04:52:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff050066f30 0x7ff0500693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c0d0870 0x7ff06c16ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff06c15ccc0 0x7ff06c0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 --2- 172.21.15.3:0/2731082296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff06c15c8f0 0x7ff06c154960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 >> 172.21.15.3:0/2731082296 conn(0x7ff06c070310 msgr2=0x7ff06c0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 shutdown_connections 2023-10-06T04:52:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.239+0000 7ff072cc2700 1 -- 172.21.15.3:0/2731082296 wait complete. 2023-10-06T04:52:45.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:52:15.735846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:52:17.015278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:52:45.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:45.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.375+0000 7f802612c700 1 Processor -- start 2023-10-06T04:52:45.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.375+0000 7f802612c700 1 -- start start 2023-10-06T04:52:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.375+0000 7f802612c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8020067dc0 0x7f80200b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.375+0000 7f802612c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80200b1b00 0x7f802015c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.375+0000 7f802612c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80200691c0 con 0x7f8020068190 2023-10-06T04:52:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.375+0000 7f802612c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200694a0 con 0x7f80200b1b00 2023-10-06T04:52:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.375+0000 7f802612c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8020069780 con 0x7f8020067dc0 2023-10-06T04:52:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80200b1b00 0x7f802015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8020068190 0x7f802015d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53876/0 (socket says 172.21.15.3:53876) 2023-10-06T04:52:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801ffff700 1 -- 172.21.15.3:0/2738932857 learned_addr learned my addr 172.21.15.3:0/2738932857 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801f7fe700 1 --2- 172.21.15.3:0/2738932857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8020067dc0 0x7f80200b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801effd700 1 -- 172.21.15.3:0/2738932857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8020067dc0 msgr2=0x7f80200b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801effd700 1 --2- 172.21.15.3:0/2738932857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8020067dc0 0x7f80200b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801effd700 1 -- 172.21.15.3:0/2738932857 >> v1:172.21.15.3:6789/0 conn(0x7f8020068190 legacy=0x7f802015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801effd700 1 -- 172.21.15.3:0/2738932857 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8020069a60 con 0x7f80200b1b00 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801e7fc700 1 -- 172.21.15.3:0/2738932857 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325031742 0 0) 0x7f80200691c0 con 0x7f8020068190 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801f7fe700 1 --2- 172.21.15.3:0/2738932857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8020067dc0 0x7f80200b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801effd700 1 --2- 172.21.15.3:0/2738932857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80200b1b00 0x7f802015c800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80140098e0 tx=0x7f80140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7db774913b597b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801e7fc700 1 -- 172.21.15.3:0/2738932857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014009e80 con 0x7f80200b1b00 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801e7fc700 1 -- 172.21.15.3:0/2738932857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801400b860 con 0x7f80200b1b00 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.376+0000 7f801e7fc700 1 -- 172.21.15.3:0/2738932857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400bbe0 con 0x7f80200b1b00 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 -- 172.21.15.3:0/2738932857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80200b1b00 msgr2=0x7f802015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 --2- 172.21.15.3:0/2738932857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80200b1b00 0x7f802015c800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80140098e0 tx=0x7f80140091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 -- 172.21.15.3:0/2738932857 shutdown_connections 2023-10-06T04:52:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 --2- 172.21.15.3:0/2738932857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8020067dc0 0x7f80200b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 --2- 172.21.15.3:0/2738932857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80200b1b00 0x7f802015c800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 -- 172.21.15.3:0/2738932857 >> 172.21.15.3:0/2738932857 conn(0x7f80200ce270 msgr2=0x7f80200ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 -- 172.21.15.3:0/2738932857 shutdown_connections 2023-10-06T04:52:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.377+0000 7f802612c700 1 -- 172.21.15.3:0/2738932857 wait complete. 2023-10-06T04:52:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.378+0000 7f802612c700 1 Processor -- start 2023-10-06T04:52:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.378+0000 7f802612c700 1 -- start start 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f802612c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8020067dc0 0x7f80200adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f802612c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020068190 0x7f80200b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f802612c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f80200b1b00 0x7f802015c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f802612c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200aee00 con 0x7f8020068190 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f802612c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200ae1f0 con 0x7f8020067dc0 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f802612c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200ae4d0 con 0x7f80200b1b00 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020068190 0x7f80200b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020068190 0x7f80200b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47976/0 (socket says 172.21.15.3:47976) 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 -- 172.21.15.3:0/704363152 learned_addr learned my addr 172.21.15.3:0/704363152 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801ffff700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f80200b1b00 0x7f802015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801f7fe700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8020067dc0 0x7f80200adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 -- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f80200b1b00 msgr2=0x7f802015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f80200b1b00 0x7f802015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 -- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8020067dc0 msgr2=0x7f80200adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8020067dc0 0x7f80200adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.379+0000 7f801effd700 1 -- 172.21.15.3:0/704363152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014009420 con 0x7f8020068190 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.380+0000 7f801effd700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020068190 0x7f80200b0590 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f802006b700 tx=0x7f8014017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.380+0000 7f80077fe700 1 -- 172.21.15.3:0/704363152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400bbe0 con 0x7f8020068190 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.380+0000 7f80077fe700 1 -- 172.21.15.3:0/704363152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014011420 con 0x7f8020068190 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.380+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80200ae7b0 con 0x7f8020068190 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.380+0000 7f80077fe700 1 -- 172.21.15.3:0/704363152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140117c0 con 0x7f8020068190 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.380+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802015cd60 con 0x7f8020068190 2023-10-06T04:52:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.381+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80200aec50 con 0x7f8020068190 2023-10-06T04:52:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.381+0000 7f80077fe700 1 -- 172.21.15.3:0/704363152 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8014011bf0 con 0x7f8020068190 2023-10-06T04:52:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.382+0000 7f80077fe700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f800c066f30 0x7f800c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.382+0000 7f80077fe700 1 -- 172.21.15.3:0/704363152 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8014086000 con 0x7f8020068190 2023-10-06T04:52:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.382+0000 7f801f7fe700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f800c066f30 0x7f800c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.382+0000 7f801f7fe700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f800c066f30 0x7f800c0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8010006ea0 tx=0x7f8010008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.386+0000 7f80077fe700 1 -- 172.21.15.3:0/704363152 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8014053110 con 0x7f8020068190 2023-10-06T04:52:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.577+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f802013e0b0 con 0x7f8020068190 2023-10-06T04:52:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.577+0000 7f80077fe700 1 -- 172.21.15.3:0/704363152 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8014056730 con 0x7f8020068190 2023-10-06T04:52:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f800c066f30 msgr2=0x7f800c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f800c066f30 0x7f800c0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8010006ea0 tx=0x7f8010008040 comp rx=0 tx=0).stop 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020068190 msgr2=0x7f80200b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020068190 0x7f80200b0590 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f802006b700 tx=0x7f8014017000 comp rx=0 tx=0).stop 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 shutdown_connections 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f800c066f30 0x7f800c0693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8020067dc0 0x7f80200adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f80200b1b00 0x7f802015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 --2- 172.21.15.3:0/704363152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020068190 0x7f80200b0590 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 >> 172.21.15.3:0/704363152 conn(0x7f80200ce270 msgr2=0x7f802015dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 shutdown_connections 2023-10-06T04:52:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.580+0000 7f802612c700 1 -- 172.21.15.3:0/704363152 wait complete. 2023-10-06T04:52:45.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.714+0000 7f6e91ab0700 1 Processor -- start 2023-10-06T04:52:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.714+0000 7f6e91ab0700 1 -- start start 2023-10-06T04:52:45.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e91ab0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0cfb80 0x7f6e8c0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e91ab0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6e8c0afad0 0x7f6e8c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e91ab0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8c15d510 con 0x7f6e8c0cef10 2023-10-06T04:52:45.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e91ab0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c15a480 con 0x7f6e8c0afad0 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e91ab0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c15a760 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8affd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6e8c0afad0 0x7f6e8c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8b7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0cfb80 0x7f6e8c0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f6e8c0cef10 0x7f6e8c15da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34918/0 (socket says 172.21.15.3:34918) 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8bfff700 1 -- 172.21.15.3:0/886507408 learned_addr learned my addr 172.21.15.3:0/886507408 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8a7fc700 1 -- 172.21.15.3:0/886507408 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847517469 0 0) 0x7f6e8c15d510 con 0x7f6e8c0cef10 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8a7fc700 1 -- 172.21.15.3:0/886507408 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e74003570 con 0x7f6e8c0cef10 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8affd700 1 -- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0cfb80 msgr2=0x7f6e8c0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8affd700 1 --2- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0cfb80 0x7f6e8c0af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8affd700 1 -- 172.21.15.3:0/886507408 >> v1:172.21.15.153:6789/0 conn(0x7f6e8c0cef10 legacy=0x7f6e8c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8affd700 1 -- 172.21.15.3:0/886507408 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c15aa40 con 0x7f6e8c0afad0 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.715+0000 7f6e8b7fe700 1 --2- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0cfb80 0x7f6e8c0af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e8affd700 1 --2- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0afad0 0x7f6e8c0ce9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0098e0 tx=0x7f6e7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb3d2de10e59e92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e8a7fc700 1 -- 172.21.15.3:0/886507408 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c011630 con 0x7f6e8c0afad0 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e8a7fc700 1 -- 172.21.15.3:0/886507408 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c011790 con 0x7f6e8c0afad0 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e8a7fc700 1 -- 172.21.15.3:0/886507408 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c011b30 con 0x7f6e8c0afad0 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 -- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0afad0 msgr2=0x7f6e8c0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0afad0 0x7f6e8c0ce9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0098e0 tx=0x7f6e7c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 -- 172.21.15.3:0/886507408 shutdown_connections 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0cfb80 0x7f6e8c0af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/886507408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0afad0 0x7f6e8c0ce9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 -- 172.21.15.3:0/886507408 >> 172.21.15.3:0/886507408 conn(0x7f6e8c070310 msgr2=0x7f6e8c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 -- 172.21.15.3:0/886507408 shutdown_connections 2023-10-06T04:52:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.716+0000 7f6e91ab0700 1 -- 172.21.15.3:0/886507408 wait complete. 2023-10-06T04:52:45.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 Processor -- start 2023-10-06T04:52:45.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 -- start start 2023-10-06T04:52:45.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0afad0 0x7f6e8c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0cef10 0x7f6e8c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8c0cfb80 0x7f6e8c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0ca9e0 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0b0350 con 0x7f6e8c0cef10 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.718+0000 7f6e91ab0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0b11b0 con 0x7f6e8c0afad0 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8c0cfb80 0x7f6e8c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8c0cfb80 0x7f6e8c0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47986/0 (socket says 172.21.15.3:47986) 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 -- 172.21.15.3:0/262066539 learned_addr learned my addr 172.21.15.3:0/262066539 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8affd700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0cef10 0x7f6e8c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8b7fe700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0afad0 0x7f6e8c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 -- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0afad0 msgr2=0x7f6e8c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0afad0 0x7f6e8c0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 -- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0cef10 msgr2=0x7f6e8c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0cef10 0x7f6e8c152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 -- 172.21.15.3:0/262066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e80009420 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e8bfff700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8c0cfb80 0x7f6e8c0b1640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400cc40 tx=0x7f6e84001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e737fe700 1 -- 172.21.15.3:0/262066539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8400a9c0 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e737fe700 1 -- 172.21.15.3:0/262066539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8400ab20 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c009420 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e737fe700 1 -- 172.21.15.3:0/262066539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8400aea0 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.719+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c0b0b60 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.720+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8c04c380 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.721+0000 7f6e737fe700 1 -- 172.21.15.3:0/262066539 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e84007300 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.721+0000 7f6e737fe700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e78066f30 0x7f6e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:45.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.721+0000 7f6e737fe700 1 -- 172.21.15.3:0/262066539 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e84084b70 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.721+0000 7f6e8b7fe700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e78066f30 0x7f6e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:45.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.722+0000 7f6e8b7fe700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e78066f30 0x7f6e780693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e800093f0 tx=0x7f6e80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:45.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.725+0000 7f6e737fe700 1 -- 172.21.15.3:0/262066539 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e84051c70 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.920+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e8c0ce470 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.921+0000 7f6e737fe700 1 -- 172.21.15.3:0/262066539 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e84055290 con 0x7f6e8c0cfb80 2023-10-06T04:52:45.923 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e78066f30 msgr2=0x7f6e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e78066f30 0x7f6e780693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e800093f0 tx=0x7f6e80008040 comp rx=0 tx=0).stop 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8c0cfb80 msgr2=0x7f6e8c0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8c0cfb80 0x7f6e8c0b1640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400cc40 tx=0x7f6e84001aa0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 shutdown_connections 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e78066f30 0x7f6e780693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8c0cfb80 0x7f6e8c0b1640 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e8c0afad0 0x7f6e8c0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 --2- 172.21.15.3:0/262066539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e8c0cef10 0x7f6e8c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 >> 172.21.15.3:0/262066539 conn(0x7f6e8c070310 msgr2=0x7f6e8c15b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 shutdown_connections 2023-10-06T04:52:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:45.923+0000 7f6e91ab0700 1 -- 172.21.15.3:0/262066539 wait complete. 2023-10-06T04:52:45.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:52:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.056+0000 7f8b5a082700 1 Processor -- start 2023-10-06T04:52:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.056+0000 7f8b5a082700 1 -- start start 2023-10-06T04:52:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b5a082700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b540cfb80 0x7f8b540af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b5a082700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b54159eb0 0x7f8b5415da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b5a082700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5415d510 con 0x7f8b540afad0 2023-10-06T04:52:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b5a082700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5415a480 con 0x7f8b540cfb80 2023-10-06T04:52:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b5a082700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5415a760 con 0x7f8b54159eb0 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b537fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b540cfb80 0x7f8b540af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b537fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b540cfb80 0x7f8b540af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47992/0 (socket says 172.21.15.3:47992) 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.057+0000 7f8b537fe700 1 -- 172.21.15.3:0/2602794616 learned_addr learned my addr 172.21.15.3:0/2602794616 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b53fff700 1 --2- 172.21.15.3:0/2602794616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b54159eb0 0x7f8b5415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b537fe700 1 -- 172.21.15.3:0/2602794616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b54159eb0 msgr2=0x7f8b5415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b537fe700 1 --2- 172.21.15.3:0/2602794616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b54159eb0 0x7f8b5415da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b537fe700 1 -- 172.21.15.3:0/2602794616 >> v1:172.21.15.153:6789/0 conn(0x7f8b540afad0 legacy=0x7f8b540ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b537fe700 1 -- 172.21.15.3:0/2602794616 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5415aa40 con 0x7f8b540cfb80 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b537fe700 1 --2- 172.21.15.3:0/2602794616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b540cfb80 0x7f8b540af590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b440099b0 tx=0x7f8b44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eaf505194ebb75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b527fc700 1 -- 172.21.15.3:0/2602794616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400b860 con 0x7f8b540cfb80 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b527fc700 1 -- 172.21.15.3:0/2602794616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4400b9c0 con 0x7f8b540cfb80 2023-10-06T04:52:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b527fc700 1 -- 172.21.15.3:0/2602794616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400bd60 con 0x7f8b540cfb80 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b5a082700 1 -- 172.21.15.3:0/2602794616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b540cfb80 msgr2=0x7f8b540af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b5a082700 1 --2- 172.21.15.3:0/2602794616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b540cfb80 0x7f8b540af590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b440099b0 tx=0x7f8b44009250 comp rx=0 tx=0).stop 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b5a082700 1 -- 172.21.15.3:0/2602794616 shutdown_connections 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b5a082700 1 --2- 172.21.15.3:0/2602794616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b54159eb0 0x7f8b5415da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.058+0000 7f8b5a082700 1 --2- 172.21.15.3:0/2602794616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b540cfb80 0x7f8b540af590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.059+0000 7f8b5a082700 1 -- 172.21.15.3:0/2602794616 >> 172.21.15.3:0/2602794616 conn(0x7f8b54070310 msgr2=0x7f8b54070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.059+0000 7f8b5a082700 1 -- 172.21.15.3:0/2602794616 shutdown_connections 2023-10-06T04:52:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.059+0000 7f8b5a082700 1 -- 172.21.15.3:0/2602794616 wait complete. 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 Processor -- start 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 -- start start 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b540afad0 0x7f8b540b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b540cfb80 0x7f8b540b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b54159eb0 0x7f8b54169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540b1270 con 0x7f8b54159eb0 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540b35a0 con 0x7f8b540afad0 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.060+0000 7f8b5a082700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b54163f30 con 0x7f8b540cfb80 2023-10-06T04:52:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b54159eb0 0x7f8b54169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b54159eb0 0x7f8b54169a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48008/0 (socket says 172.21.15.3:48008) 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 -- 172.21.15.3:0/1696330513 learned_addr learned my addr 172.21.15.3:0/1696330513 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b537fe700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b540afad0 0x7f8b540b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b52ffd700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b540cfb80 0x7f8b540b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 -- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b540cfb80 msgr2=0x7f8b540b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b540cfb80 0x7f8b540b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 -- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b540afad0 msgr2=0x7f8b540b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b540afad0 0x7f8b540b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 -- 172.21.15.3:0/1696330513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440094f0 con 0x7f8b54159eb0 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b53fff700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b54159eb0 0x7f8b54169a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c007f20 tx=0x7f8b4c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b3b7fe700 1 -- 172.21.15.3:0/1696330513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00eca0 con 0x7f8b54159eb0 2023-10-06T04:52:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b3b7fe700 1 -- 172.21.15.3:0/1696330513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c00ee20 con 0x7f8b54159eb0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b54155b60 con 0x7f8b54159eb0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b3b7fe700 1 -- 172.21.15.3:0/1696330513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c014670 con 0x7f8b54159eb0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.061+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b540b1790 con 0x7f8b54159eb0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.062+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b540b3370 con 0x7f8b54159eb0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.063+0000 7f8b3b7fe700 1 -- 172.21.15.3:0/1696330513 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b4c0149e0 con 0x7f8b54159eb0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.063+0000 7f8b3b7fe700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b40066f30 0x7f8b400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.063+0000 7f8b3b7fe700 1 -- 172.21.15.3:0/1696330513 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b4c084d80 con 0x7f8b54159eb0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.063+0000 7f8b537fe700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b40066f30 0x7f8b400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.064+0000 7f8b537fe700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b40066f30 0x7f8b400693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b440099b0 tx=0x7f8b44017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.067+0000 7f8b3b7fe700 1 -- 172.21.15.3:0/1696330513 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b540b3370 con 0x7f8b54159eb0 2023-10-06T04:52:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.260+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b540b3370 con 0x7f8b54159eb0 2023-10-06T04:52:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.261+0000 7f8b3b7fe700 1 -- 172.21.15.3:0/1696330513 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b540b3370 con 0x7f8b54159eb0 2023-10-06T04:52:46.264 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:52:46.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b40066f30 msgr2=0x7f8b400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b40066f30 0x7f8b400693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b440099b0 tx=0x7f8b44017000 comp rx=0 tx=0).stop 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b54159eb0 msgr2=0x7f8b54169a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b54159eb0 0x7f8b54169a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c007f20 tx=0x7f8b4c009860 comp rx=0 tx=0).stop 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 shutdown_connections 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b40066f30 0x7f8b400693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b540afad0 0x7f8b540b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b54159eb0 0x7f8b54169a20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 --2- 172.21.15.3:0/1696330513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b540cfb80 0x7f8b540b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 >> 172.21.15.3:0/1696330513 conn(0x7f8b54070310 msgr2=0x7f8b5415c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:46.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 shutdown_connections 2023-10-06T04:52:46.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.263+0000 7f8b5a082700 1 -- 172.21.15.3:0/1696330513 wait complete. 2023-10-06T04:52:46.277 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:52:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.396+0000 7f7b6ed46700 1 Processor -- start 2023-10-06T04:52:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.396+0000 7f7b6ed46700 1 -- start start 2023-10-06T04:52:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.397+0000 7f7b6ed46700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 0x7f7b681596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.397+0000 7f7b6ed46700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6806dac0 con 0x7f7b680ae9d0 2023-10-06T04:52:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.397+0000 7f7b6ed46700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b680af1b0 con 0x7f7b6806adb0 2023-10-06T04:52:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.397+0000 7f7b6ed46700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6806d4b0 con 0x7f7b6806a340 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.397+0000 7f7b6d2e3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7b680ae9d0 0x7f7b6806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53882/0 (socket says 172.21.15.3:53882) 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.397+0000 7f7b6d2e3700 1 -- 172.21.15.3:0/3301664433 learned_addr learned my addr 172.21.15.3:0/3301664433 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6cae2700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 0x7f7b681596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029123235 0 0) 0x7f7b6806dac0 con 0x7f7b680ae9d0 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b50003570 con 0x7f7b680ae9d0 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3898414180 0 0) 0x7f7b50003570 con 0x7f7b680ae9d0 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 msgr2=0x7f7b681596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 0x7f7b681596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 >> v1:172.21.15.153:6789/0 conn(0x7f7b6806adb0 legacy=0x7f7b6806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680aeed0 con 0x7f7b680ae9d0 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f7b58001470 con 0x7f7b680ae9d0 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 >> v1:172.21.15.3:6789/0 conn(0x7f7b680ae9d0 legacy=0x7f7b6806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b50003a90 0x7f7b50003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 0x7f7b50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6806d4b0 con 0x7f7b50004d50 2023-10-06T04:52:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5000bb70 con 0x7f7b50003a90 2023-10-06T04:52:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5000bd30 con 0x7f7b6806a340 2023-10-06T04:52:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6d2e3700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6cae2700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b50003a90 0x7f7b50003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b67fff700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 0x7f7b50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6d2e3700 1 -- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 msgr2=0x7f7b50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6d2e3700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 0x7f7b50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6d2e3700 1 -- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b50003a90 msgr2=0x7f7b50003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6d2e3700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b50003a90 0x7f7b50003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.398+0000 7f7b6d2e3700 1 -- 172.21.15.3:0/3301664433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b50003570 con 0x7f7b50004d50 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b67fff700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b6806a340 0x7f7b50004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6d2e3700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b500051c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c390 tx=0x7f7b6806da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98ff149cc186f84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5800cc60 con 0x7f7b50004d50 2023-10-06T04:52:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5800cdc0 con 0x7f7b50004d50 2023-10-06T04:52:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b677fe700 1 -- 172.21.15.3:0/3301664433 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58005b70 con 0x7f7b50004d50 2023-10-06T04:52:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3301664433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 msgr2=0x7f7b500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6ed46700 1 --2- 172.21.15.3:0/3301664433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b500051c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c390 tx=0x7f7b6806da30 comp rx=0 tx=0).stop 2023-10-06T04:52:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6d2e3700 1 -- 172.21.15.3:0/3301664433 reap_dead start 2023-10-06T04:52:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3301664433 shutdown_connections 2023-10-06T04:52:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3301664433 >> 172.21.15.3:0/3301664433 conn(0x7f7b68067a40 msgr2=0x7f7b680ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:46.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3301664433 shutdown_connections 2023-10-06T04:52:46.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.399+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3301664433 wait complete. 2023-10-06T04:52:46.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.400+0000 7f7b6ed46700 1 Processor -- start 2023-10-06T04:52:46.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6ed46700 1 -- start start 2023-10-06T04:52:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6ed46700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b681535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6ed46700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6815a590 0x7f7b6816a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6ed46700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b680afb50 0x7f7b680affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6ed46700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68153b00 con 0x7f7b50004d50 2023-10-06T04:52:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6ed46700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6816ad50 con 0x7f7b6815a590 2023-10-06T04:52:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6ed46700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680b04f0 con 0x7f7b680afb50 2023-10-06T04:52:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6cae2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b681535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6cae2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b681535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48022/0 (socket says 172.21.15.3:48022) 2023-10-06T04:52:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.401+0000 7f7b6cae2700 1 -- 172.21.15.3:0/3028017947 learned_addr learned my addr 172.21.15.3:0/3028017947 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6d2e3700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b680afb50 0x7f7b680affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b67fff700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6815a590 0x7f7b6816a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6cae2700 1 -- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b680afb50 msgr2=0x7f7b680affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6cae2700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b680afb50 0x7f7b680affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6cae2700 1 -- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6815a590 msgr2=0x7f7b6816a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6cae2700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6815a590 0x7f7b6816a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6cae2700 1 -- 172.21.15.3:0/3028017947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6816d390 con 0x7f7b50004d50 2023-10-06T04:52:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6cae2700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b681535c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b60011f50 tx=0x7f7b6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b64ff9700 1 -- 172.21.15.3:0/3028017947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000e040 con 0x7f7b50004d50 2023-10-06T04:52:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b64ff9700 1 -- 172.21.15.3:0/3028017947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000f040 con 0x7f7b50004d50 2023-10-06T04:52:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6816dc30 con 0x7f7b50004d50 2023-10-06T04:52:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b64ff9700 1 -- 172.21.15.3:0/3028017947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60015430 con 0x7f7b50004d50 2023-10-06T04:52:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.402+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6815aac0 con 0x7f7b50004d50 2023-10-06T04:52:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.403+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6806c000 con 0x7f7b50004d50 2023-10-06T04:52:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.404+0000 7f7b64ff9700 1 -- 172.21.15.3:0/3028017947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b60015820 con 0x7f7b50004d50 2023-10-06T04:52:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.404+0000 7f7b64ff9700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b54066f30 0x7f7b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.404+0000 7f7b64ff9700 1 -- 172.21.15.3:0/3028017947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7b600853e0 con 0x7f7b50004d50 2023-10-06T04:52:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.404+0000 7f7b67fff700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b54066f30 0x7f7b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.405+0000 7f7b67fff700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b54066f30 0x7f7b540693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007bc0 tx=0x7f7b5c00c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.408+0000 7f7b64ff9700 1 -- 172.21.15.3:0/3028017947 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b600524f0 con 0x7f7b50004d50 2023-10-06T04:52:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.601+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b6804c380 con 0x7f7b50004d50 2023-10-06T04:52:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.601+0000 7f7b64ff9700 1 -- 172.21.15.3:0/3028017947 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b60055b10 con 0x7f7b50004d50 2023-10-06T04:52:46.605 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:52:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.603+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b54066f30 msgr2=0x7f7b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.603+0000 7f7b6ed46700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b54066f30 0x7f7b540693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007bc0 tx=0x7f7b5c00c040 comp rx=0 tx=0).stop 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.603+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 msgr2=0x7f7b681535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.603+0000 7f7b6ed46700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b681535c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b60011f50 tx=0x7f7b6000b040 comp rx=0 tx=0).stop 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 shutdown_connections 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b54066f30 0x7f7b540693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b680afb50 0x7f7b680affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b50004d50 0x7f7b681535c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 --2- 172.21.15.3:0/3028017947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6815a590 0x7f7b6816a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 >> 172.21.15.3:0/3028017947 conn(0x7f7b68067a40 msgr2=0x7f7b680cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 shutdown_connections 2023-10-06T04:52:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.604+0000 7f7b6ed46700 1 -- 172.21.15.3:0/3028017947 wait complete. 2023-10-06T04:52:46.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:46.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.735+0000 7fb8cce23700 1 Processor -- start 2023-10-06T04:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.735+0000 7fb8cce23700 1 -- start start 2023-10-06T04:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8cce23700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8cce23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8c80d0870 0x7fb8c80b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8cce23700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c80cf6c0 con 0x7fb8c815ccc0 2023-10-06T04:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8cce23700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c80cfb50 con 0x7fb8c80d0870 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8cce23700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c815e050 con 0x7fb8c815c8f0 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8c80d0870 0x7fb8c80b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb8c815ccc0 0x7fb8c80d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53892/0 (socket says 172.21.15.3:53892) 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8c80d0870 0x7fb8c80b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48026/0 (socket says 172.21.15.3:48026) 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c5d9b700 1 -- 172.21.15.3:0/2878526140 learned_addr learned my addr 172.21.15.3:0/2878526140 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c659c700 1 --2- 172.21.15.3:0/2878526140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c6d9d700 1 -- 172.21.15.3:0/2878526140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 msgr2=0x7fb8c815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c6d9d700 1 --2- 172.21.15.3:0/2878526140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c6d9d700 1 -- 172.21.15.3:0/2878526140 >> v1:172.21.15.3:6789/0 conn(0x7fb8c815ccc0 legacy=0x7fb8c80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c6d9d700 1 -- 172.21.15.3:0/2878526140 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c80ceab0 con 0x7fb8c80d0870 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.736+0000 7fb8c559a700 1 -- 172.21.15.3:0/2878526140 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213108514 0 0) 0x7fb8c80cf6c0 con 0x7fb8c815ccc0 2023-10-06T04:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8c6d9d700 1 --2- 172.21.15.3:0/2878526140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b3ba0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80098e0 tx=0x7fb8b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3833ee70d232919 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8c559a700 1 -- 172.21.15.3:0/2878526140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800b640 con 0x7fb8c80d0870 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8c559a700 1 -- 172.21.15.3:0/2878526140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b800b7a0 con 0x7fb8c80d0870 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8c559a700 1 -- 172.21.15.3:0/2878526140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800bb40 con 0x7fb8c80d0870 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 -- 172.21.15.3:0/2878526140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c80d0870 msgr2=0x7fb8c80b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 --2- 172.21.15.3:0/2878526140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b3ba0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80098e0 tx=0x7fb8b80091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 -- 172.21.15.3:0/2878526140 shutdown_connections 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 --2- 172.21.15.3:0/2878526140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b3ba0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 --2- 172.21.15.3:0/2878526140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 -- 172.21.15.3:0/2878526140 >> 172.21.15.3:0/2878526140 conn(0x7fb8c8070310 msgr2=0x7fb8c8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 -- 172.21.15.3:0/2878526140 shutdown_connections 2023-10-06T04:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.737+0000 7fb8cce23700 1 -- 172.21.15.3:0/2878526140 wait complete. 2023-10-06T04:52:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.738+0000 7fb8cce23700 1 Processor -- start 2023-10-06T04:52:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8cce23700 1 -- start start 2023-10-06T04:52:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8cce23700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8cce23700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c816a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8cce23700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c815ccc0 0x7fb8c8153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8cce23700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80b4f80 con 0x7fb8c815ccc0 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8cce23700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80b5b00 con 0x7fb8c80d0870 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8cce23700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80b51d0 con 0x7fb8c815c8f0 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8c6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c815ccc0 0x7fb8c8153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8c6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c815ccc0 0x7fb8c8153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48032/0 (socket says 172.21.15.3:48032) 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.739+0000 7fb8c6d9d700 1 -- 172.21.15.3:0/273687107 learned_addr learned my addr 172.21.15.3:0/273687107 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c659c700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c5d9b700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c816a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c6d9d700 1 -- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 msgr2=0x7fb8c816a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c6d9d700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c816a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c6d9d700 1 -- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c80d0870 msgr2=0x7fb8c80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c6d9d700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c6d9d700 1 -- 172.21.15.3:0/273687107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8009420 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c659c700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8c6d9d700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c815ccc0 0x7fb8c8153e70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8c806eaa0 tx=0x7fb8b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8be7fc700 1 -- 172.21.15.3:0/273687107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8011420 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8be7fc700 1 -- 172.21.15.3:0/273687107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b8011580 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c80b54b0 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8be7fc700 1 -- 172.21.15.3:0/273687107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8011920 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.740+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c816a8d0 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.741+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c80b5c70 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.742+0000 7fb8be7fc700 1 -- 172.21.15.3:0/273687107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8b8011c90 con 0x7fb8c815ccc0 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.742+0000 7fb8be7fc700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8a8066f30 0x7fb8a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.742+0000 7fb8be7fc700 1 -- 172.21.15.3:0/273687107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb8b8085c50 con 0x7fb8c815ccc0 2023-10-06T04:52:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.742+0000 7fb8c659c700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8a8066f30 0x7fb8a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.743+0000 7fb8c659c700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8a8066f30 0x7fb8a80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0007780 tx=0x7fb8c000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.746+0000 7fb8be7fc700 1 -- 172.21.15.3:0/273687107 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8b808b020 con 0x7fb8c815ccc0 2023-10-06T04:52:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.945+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8c8071680 con 0x7fb8c815ccc0 2023-10-06T04:52:46.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.945+0000 7fb8be7fc700 1 -- 172.21.15.3:0/273687107 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fb8b8052d60 con 0x7fb8c815ccc0 2023-10-06T04:52:46.948 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:46.948 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:46.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.947+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8a8066f30 msgr2=0x7fb8a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.947+0000 7fb8cce23700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8a8066f30 0x7fb8a80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0007780 tx=0x7fb8c000b040 comp rx=0 tx=0).stop 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c815ccc0 msgr2=0x7fb8c8153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c815ccc0 0x7fb8c8153e70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8c806eaa0 tx=0x7fb8b8017000 comp rx=0 tx=0).stop 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 shutdown_connections 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8a8066f30 0x7fb8a80693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c80d0870 0x7fb8c80b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8c815ccc0 0x7fb8c8153e70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 --2- 172.21.15.3:0/273687107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8c815c8f0 0x7fb8c816a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 >> 172.21.15.3:0/273687107 conn(0x7fb8c8070310 msgr2=0x7fb8c80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:46.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 shutdown_connections 2023-10-06T04:52:46.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:46.948+0000 7fb8cce23700 1 -- 172.21.15.3:0/273687107 wait complete. 2023-10-06T04:52:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:52:15.735846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:52:17.015278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:52:46.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.083+0000 7fa1ed513700 1 Processor -- start 2023-10-06T04:52:47.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.083+0000 7fa1ed513700 1 -- start start 2023-10-06T04:52:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1ed513700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa1e8153740 0x7fa1e80ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1ed513700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 0x7fa1e80a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1ed513700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e815a960 con 0x7fa1e8153b10 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1ed513700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e80ac7e0 con 0x7fa1e8153740 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1ed513700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8159d50 con 0x7fa1e806ca80 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1e67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1e8153b10 0x7fa1e806c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53904/0 (socket says 172.21.15.3:53904) 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1e67fc700 1 -- 172.21.15.3:0/2378767893 learned_addr learned my addr 172.21.15.3:0/2378767893 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973212360 0 0) 0x7fa1e815a960 con 0x7fa1e8153b10 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/2378767893 >> v2:172.21.15.153:3300/0 conn(0x7fa1e8153740 0x7fa1e80ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.084+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 0x7fa1e80a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d4003570 con 0x7fa1e8153b10 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944593529 0 0) 0x7fa1d4003570 con 0x7fa1e8153b10 2023-10-06T04:52:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 msgr2=0x7fa1e80a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 0x7fa1e80a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 >> v2:172.21.15.153:3300/0 conn(0x7fa1e8153740 msgr2=0x7fa1e80ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/2378767893 >> v2:172.21.15.153:3300/0 conn(0x7fa1e8153740 0x7fa1e80ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e815a030 con 0x7fa1e8153b10 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fa1d0001470 con 0x7fa1e8153b10 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 >> v1:172.21.15.3:6789/0 conn(0x7fa1e8153b10 legacy=0x7fa1e806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 0x7fa1d4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4003fe0 0x7fa1d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4004d50 0x7fa1d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80ac7e0 con 0x7fa1d4004d50 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8159d50 con 0x7fa1d4003fe0 2023-10-06T04:52:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d400ba90 con 0x7fa1e806ca80 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4004d50 0x7fa1d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 0x7fa1d4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4003fe0 0x7fa1d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e77fe700 1 -- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 msgr2=0x7fa1d4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e806ca80 0x7fa1d4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e77fe700 1 -- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4003fe0 msgr2=0x7fa1d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4003fe0 0x7fa1d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e77fe700 1 -- 172.21.15.3:0/2378767893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4003570 con 0x7fa1d4004d50 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4003fe0 0x7fa1d4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.085+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4004d50 0x7fa1d40051c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e210 tx=0x7fa1d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872229978b2ae273 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800ea60 con 0x7fa1d4004d50 2023-10-06T04:52:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d800f040 con 0x7fa1d4004d50 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/2378767893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800ee10 con 0x7fa1d4004d50 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1ed513700 1 -- 172.21.15.3:0/2378767893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4004d50 msgr2=0x7fa1d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1ed513700 1 --2- 172.21.15.3:0/2378767893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4004d50 0x7fa1d40051c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e210 tx=0x7fa1d800b040 comp rx=0 tx=0).stop 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1e77fe700 1 -- 172.21.15.3:0/2378767893 reap_dead start 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1ed513700 1 -- 172.21.15.3:0/2378767893 shutdown_connections 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1ed513700 1 -- 172.21.15.3:0/2378767893 >> 172.21.15.3:0/2378767893 conn(0x7fa1e80cdab0 msgr2=0x7fa1e80aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1ed513700 1 -- 172.21.15.3:0/2378767893 shutdown_connections 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.086+0000 7fa1ed513700 1 -- 172.21.15.3:0/2378767893 wait complete. 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.087+0000 7fa1ed513700 1 Processor -- start 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1ed513700 1 -- start start 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1ed513700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e816a1d0 0x7fa1e80ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1ed513700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e80acef0 0x7fa1e814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1ed513700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8150050 0x7fa1e8151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1ed513700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e814e470 con 0x7fa1e816a1d0 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1ed513700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80ae080 con 0x7fa1e8150050 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1ed513700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e816a5a0 con 0x7fa1e80acef0 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1e6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e816a1d0 0x7fa1e80ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1e6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e816a1d0 0x7fa1e80ac9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48046/0 (socket says 172.21.15.3:48046) 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1e6ffd700 1 -- 172.21.15.3:0/4140838262 learned_addr learned my addr 172.21.15.3:0/4140838262 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.088+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e80acef0 0x7fa1e814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8150050 0x7fa1e8151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1e6ffd700 1 -- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e80acef0 msgr2=0x7fa1e814fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e80acef0 0x7fa1e814fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1e6ffd700 1 -- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8150050 msgr2=0x7fa1e8151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8150050 0x7fa1e8151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1e6ffd700 1 -- 172.21.15.3:0/4140838262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80ad470 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e816a1d0 0x7fa1e80ac9b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0011b10 tx=0x7fa1e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1deffd700 1 -- 172.21.15.3:0/4140838262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000f990 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1deffd700 1 -- 172.21.15.3:0/4140838262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e000faf0 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e80ad750 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1deffd700 1 -- 172.21.15.3:0/4140838262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000fe70 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.089+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e80add10 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.090+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e80ae360 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.090+0000 7fa1deffd700 1 -- 172.21.15.3:0/4140838262 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1e0014650 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.091+0000 7fa1deffd700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.091+0000 7fa1deffd700 1 -- 172.21.15.3:0/4140838262 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa1e0018030 con 0x7fa1e816a1d0 2023-10-06T04:52:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.091+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.091+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0001d20 tx=0x7fa1d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.095+0000 7fa1deffd700 1 -- 172.21.15.3:0/4140838262 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1e0055ca0 con 0x7fa1e816a1d0 2023-10-06T04:52:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.289+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1e804c380 con 0x7fa1e816a1d0 2023-10-06T04:52:47.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.289+0000 7fa1deffd700 1 -- 172.21.15.3:0/4140838262 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa1e0059e60 con 0x7fa1e816a1d0 2023-10-06T04:52:47.368 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:47.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 msgr2=0x7fa1c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0001d20 tx=0x7fa1d0009040 comp rx=0 tx=0).stop 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e816a1d0 msgr2=0x7fa1e80ac9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e816a1d0 0x7fa1e80ac9b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0011b10 tx=0x7fa1e00091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 shutdown_connections 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8150050 0x7fa1e8151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e816a1d0 0x7fa1e80ac9b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 --2- 172.21.15.3:0/4140838262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e80acef0 0x7fa1e814fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 >> 172.21.15.3:0/4140838262 conn(0x7fa1e80cdab0 msgr2=0x7fa1e80a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 shutdown_connections 2023-10-06T04:52:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.292+0000 7fa1ed513700 1 -- 172.21.15.3:0/4140838262 wait complete. 2023-10-06T04:52:47.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:47.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.490+0000 7ffb7e59e700 1 Processor -- start 2023-10-06T04:52:47.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.491+0000 7ffb7e59e700 1 -- start start 2023-10-06T04:52:47.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.491+0000 7ffb7e59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb780d0260 0x7ffb7815da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.491+0000 7ffb7e59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.491+0000 7ffb7e59e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb780ce630 con 0x7ffb780b35b0 2023-10-06T04:52:47.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.491+0000 7ffb7e59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb780b5ef0 con 0x7ffb780d0260 2023-10-06T04:52:47.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.491+0000 7ffb7e59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780ce910 con 0x7ffb780d0630 2023-10-06T04:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb780d0260 0x7ffb7815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7cd9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb780d0260 0x7ffb7815da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:43236/0 (socket says 172.21.15.3:43236) 2023-10-06T04:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7dd9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ffb780b35b0 0x7ffb781428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34932/0 (socket says 172.21.15.3:34932) 2023-10-06T04:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3355203163 learned_addr learned my addr 172.21.15.3:0/3355203163 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3355203163 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 msgr2=0x7ffb780b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 --2- 172.21.15.3:0/3355203163 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3355203163 >> v1:172.21.15.153:6789/0 conn(0x7ffb780b35b0 legacy=0x7ffb781428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3355203163 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780cebf0 con 0x7ffb780d0260 2023-10-06T04:52:47.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.492+0000 7ffb7d59c700 1 --2- 172.21.15.3:0/3355203163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 0x7ffb7815da30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009f20 tx=0x7ffb68009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23bc9a825795d788 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb77fff700 1 -- 172.21.15.3:0/3355203163 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6800e070 con 0x7ffb780d0260 2023-10-06T04:52:47.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb77fff700 1 -- 172.21.15.3:0/3355203163 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6800b9d0 con 0x7ffb780d0260 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb77fff700 1 -- 172.21.15.3:0/3355203163 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb680092e0 con 0x7ffb780d0260 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3355203163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 msgr2=0x7ffb7815da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3355203163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 0x7ffb7815da30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009f20 tx=0x7ffb68009840 comp rx=0 tx=0).stop 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3355203163 shutdown_connections 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3355203163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 0x7ffb7815da30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3355203163 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3355203163 >> 172.21.15.3:0/3355203163 conn(0x7ffb7806ecc0 msgr2=0x7ffb7806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3355203163 shutdown_connections 2023-10-06T04:52:47.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.493+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3355203163 wait complete. 2023-10-06T04:52:47.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.494+0000 7ffb7e59e700 1 Processor -- start 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.494+0000 7ffb7e59e700 1 -- start start 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7e59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb780b35b0 0x7ffb7815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7e59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 0x7ffb780b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7e59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7e59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7815a6c0 con 0x7ffb780b35b0 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7e59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780c9750 con 0x7ffb780d0260 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7e59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780c9a30 con 0x7ffb780d0630 2023-10-06T04:52:47.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7d59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb780b35b0 0x7ffb7815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7d59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb780b35b0 0x7ffb7815a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48062/0 (socket says 172.21.15.3:48062) 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3215210251 learned_addr learned my addr 172.21.15.3:0/3215210251 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7dd9d700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.495+0000 7ffb7cd9b700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 0x7ffb780b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 msgr2=0x7ffb780b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7d59c700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 msgr2=0x7ffb780b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7d59c700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 0x7ffb780b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7d59c700 1 -- 172.21.15.3:0/3215210251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb68009ae0 con 0x7ffb780b35b0 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7d59c700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb780b35b0 0x7ffb7815a180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800b610 tx=0x7ffb68018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb757fa700 1 -- 172.21.15.3:0/3215210251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb680088f0 con 0x7ffb780b35b0 2023-10-06T04:52:47.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb757fa700 1 -- 172.21.15.3:0/3215210251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb68008a50 con 0x7ffb780b35b0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb780c9d10 con 0x7ffb780b35b0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb757fa700 1 -- 172.21.15.3:0/3215210251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68008e10 con 0x7ffb780b35b0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.496+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb78169ce0 con 0x7ffb780b35b0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.497+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7806f920 con 0x7ffb780b35b0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.497+0000 7ffb757fa700 1 -- 172.21.15.3:0/3215210251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb68003040 con 0x7ffb780b35b0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.498+0000 7ffb757fa700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb64066f30 0x7ffb640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.498+0000 7ffb757fa700 1 -- 172.21.15.3:0/3215210251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffb68085a70 con 0x7ffb780b35b0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.498+0000 7ffb7cd9b700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb64066f30 0x7ffb640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.499+0000 7ffb7cd9b700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb64066f30 0x7ffb640693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009d10 tx=0x7ffb6c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.502+0000 7ffb757fa700 1 -- 172.21.15.3:0/3215210251 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb68052c30 con 0x7ffb780b35b0 2023-10-06T04:52:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.696+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb7804c380 con 0x7ffb780b35b0 2023-10-06T04:52:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.696+0000 7ffb757fa700 1 -- 172.21.15.3:0/3215210251 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb68056250 con 0x7ffb780b35b0 2023-10-06T04:52:47.699 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb64066f30 msgr2=0x7ffb640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb64066f30 0x7ffb640693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009d10 tx=0x7ffb6c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb780b35b0 msgr2=0x7ffb7815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb780b35b0 0x7ffb7815a180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800b610 tx=0x7ffb68018000 comp rx=0 tx=0).stop 2023-10-06T04:52:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 shutdown_connections 2023-10-06T04:52:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb64066f30 0x7ffb640693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb780d0630 0x7ffb780b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb780b35b0 0x7ffb7815a180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 --2- 172.21.15.3:0/3215210251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb780d0260 0x7ffb780b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 >> 172.21.15.3:0/3215210251 conn(0x7ffb7806ecc0 msgr2=0x7ffb780af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 shutdown_connections 2023-10-06T04:52:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.698+0000 7ffb7e59e700 1 -- 172.21.15.3:0/3215210251 wait complete. 2023-10-06T04:52:47.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:52:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.833+0000 7f9790f59700 1 Processor -- start 2023-10-06T04:52:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.833+0000 7f9790f59700 1 -- start start 2023-10-06T04:52:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9790f59700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c159eb0 0x7f978c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9790f59700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f978c0cef10 0x7f978c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9790f59700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c0afca0 con 0x7f978c0cfb80 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9790f59700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c15d510 con 0x7f978c0cef10 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9790f59700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c15a480 con 0x7f978c159eb0 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f978ad9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f978c0cef10 0x7f978c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9789d9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c159eb0 0x7f978c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f978a59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f978c0cfb80 0x7f978c0af590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34934/0 (socket says 172.21.15.3:34934) 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f978a59c700 1 -- 172.21.15.3:0/380808994 learned_addr learned my addr 172.21.15.3:0/380808994 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f978959a700 1 -- 172.21.15.3:0/380808994 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838562405 0 0) 0x7f978c0afca0 con 0x7f978c0cfb80 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f978959a700 1 -- 172.21.15.3:0/380808994 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9778003570 con 0x7f978c0cfb80 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9789d9b700 1 -- 172.21.15.3:0/380808994 >> v2:172.21.15.153:3300/0 conn(0x7f978c0cef10 msgr2=0x7f978c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9789d9b700 1 --2- 172.21.15.3:0/380808994 >> v2:172.21.15.153:3300/0 conn(0x7f978c0cef10 0x7f978c15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9789d9b700 1 -- 172.21.15.3:0/380808994 >> v1:172.21.15.153:6789/0 conn(0x7f978c0cfb80 legacy=0x7f978c0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f9789d9b700 1 -- 172.21.15.3:0/380808994 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c15a760 con 0x7f978c159eb0 2023-10-06T04:52:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.834+0000 7f978ad9d700 1 --2- 172.21.15.3:0/380808994 >> v2:172.21.15.153:3300/0 conn(0x7f978c0cef10 0x7f978c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9789d9b700 1 --2- 172.21.15.3:0/380808994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c159eb0 0x7f978c0ce9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97740098e0 tx=0x7f97740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac832fc6ff6b215 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f978959a700 1 -- 172.21.15.3:0/380808994 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774011630 con 0x7f978c159eb0 2023-10-06T04:52:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f978959a700 1 -- 172.21.15.3:0/380808994 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9774011790 con 0x7f978c159eb0 2023-10-06T04:52:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f978959a700 1 -- 172.21.15.3:0/380808994 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774011b30 con 0x7f978c159eb0 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 -- 172.21.15.3:0/380808994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c159eb0 msgr2=0x7f978c0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 --2- 172.21.15.3:0/380808994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c159eb0 0x7f978c0ce9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97740098e0 tx=0x7f97740091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 -- 172.21.15.3:0/380808994 shutdown_connections 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 --2- 172.21.15.3:0/380808994 >> v2:172.21.15.153:3300/0 conn(0x7f978c0cef10 0x7f978c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 --2- 172.21.15.3:0/380808994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c159eb0 0x7f978c0ce9d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 -- 172.21.15.3:0/380808994 >> 172.21.15.3:0/380808994 conn(0x7f978c070310 msgr2=0x7f978c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 -- 172.21.15.3:0/380808994 shutdown_connections 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.835+0000 7f9790f59700 1 -- 172.21.15.3:0/380808994 wait complete. 2023-10-06T04:52:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 Processor -- start 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 -- start start 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f978c0cef10 0x7f978c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c0cfb80 0x7f978c0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c159eb0 0x7f978c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0b0f80 con 0x7f978c0cef10 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0b03e0 con 0x7f978c159eb0 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f9790f59700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c15e590 con 0x7f978c0cfb80 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f978a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f978c0cef10 0x7f978c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f978a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f978c0cef10 0x7f978c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48078/0 (socket says 172.21.15.3:48078) 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.837+0000 7f978a59c700 1 -- 172.21.15.3:0/2255036624 learned_addr learned my addr 172.21.15.3:0/2255036624 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f978ad9d700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c159eb0 0x7f978c0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f9789d9b700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c0cfb80 0x7f978c0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f978a59c700 1 -- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c0cfb80 msgr2=0x7f978c0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f978a59c700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c0cfb80 0x7f978c0c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f978a59c700 1 -- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c159eb0 msgr2=0x7f978c0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f978a59c700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c159eb0 0x7f978c0b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f978a59c700 1 -- 172.21.15.3:0/2255036624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9784009a20 con 0x7f978c0cef10 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f978a59c700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f978c0cef10 0x7f978c155250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c7f0 tx=0x7f977c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f97827fc700 1 -- 172.21.15.3:0/2255036624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c005b30 con 0x7f978c0cef10 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f97827fc700 1 -- 172.21.15.3:0/2255036624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c005c90 con 0x7f978c0cef10 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9774009420 con 0x7f978c0cef10 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f97827fc700 1 -- 172.21.15.3:0/2255036624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c007340 con 0x7f978c0cef10 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.838+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c0b0930 con 0x7f978c0cef10 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.839+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c0b1260 con 0x7f978c0cef10 2023-10-06T04:52:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.839+0000 7f97827fc700 1 -- 172.21.15.3:0/2255036624 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f977c01b400 con 0x7f978c0cef10 2023-10-06T04:52:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.840+0000 7f97827fc700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f976c066f30 0x7f976c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.840+0000 7f97827fc700 1 -- 172.21.15.3:0/2255036624 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f977c084190 con 0x7f978c0cef10 2023-10-06T04:52:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.840+0000 7f9789d9b700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f976c066f30 0x7f976c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.841+0000 7f9789d9b700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f976c066f30 0x7f976c0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97740091e0 tx=0x7f9774017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:47.844+0000 7f97827fc700 1 -- 172.21.15.3:0/2255036624 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f977c0513d0 con 0x7f978c0cef10 2023-10-06T04:52:48.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.036+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f978c04c380 con 0x7f978c0cef10 2023-10-06T04:52:48.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.037+0000 7f97827fc700 1 -- 172.21.15.3:0/2255036624 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f977c0076f0 con 0x7f978c0cef10 2023-10-06T04:52:48.039 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:52:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f976c066f30 msgr2=0x7f976c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f976c066f30 0x7f976c0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97740091e0 tx=0x7f9774017000 comp rx=0 tx=0).stop 2023-10-06T04:52:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f978c0cef10 msgr2=0x7f978c155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f978c0cef10 0x7f978c155250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c7f0 tx=0x7f977c007e00 comp rx=0 tx=0).stop 2023-10-06T04:52:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 shutdown_connections 2023-10-06T04:52:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f976c066f30 0x7f976c0693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c159eb0 0x7f978c0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f978c0cef10 0x7f978c155250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 --2- 172.21.15.3:0/2255036624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f978c0cfb80 0x7f978c0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 >> 172.21.15.3:0/2255036624 conn(0x7f978c070310 msgr2=0x7f978c0c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:48.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 shutdown_connections 2023-10-06T04:52:48.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.039+0000 7f9790f59700 1 -- 172.21.15.3:0/2255036624 wait complete. 2023-10-06T04:52:48.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:52:48.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.172+0000 7fa34cb91700 1 Processor -- start 2023-10-06T04:52:48.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.172+0000 7fa34cb91700 1 -- start start 2023-10-06T04:52:48.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.172+0000 7fa34cb91700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa3480ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa34cb91700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3480afca0 con 0x7fa3480cef10 2023-10-06T04:52:48.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa34cb91700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa34815d510 con 0x7fa3480cfb80 2023-10-06T04:52:48.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa34cb91700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34815a480 con 0x7fa348159eb0 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa346d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa3480cef10 0x7fa34815da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53906/0 (socket says 172.21.15.3:53906) 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa346d9d700 1 -- 172.21.15.3:0/2364997220 learned_addr learned my addr 172.21.15.3:0/2364997220 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa345d9b700 1 --2- 172.21.15.3:0/2364997220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa3480ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa34559a700 1 -- 172.21.15.3:0/2364997220 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310567997 0 0) 0x7fa3480afca0 con 0x7fa3480cef10 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa34559a700 1 -- 172.21.15.3:0/2364997220 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa330003570 con 0x7fa3480cef10 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa34559a700 1 -- 172.21.15.3:0/2364997220 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227193882 0 0) 0x7fa34815d510 con 0x7fa3480cfb80 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa34559a700 1 -- 172.21.15.3:0/2364997220 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3480afca0 con 0x7fa3480cfb80 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa345d9b700 1 -- 172.21.15.3:0/2364997220 >> v1:172.21.15.153:6789/0 conn(0x7fa3480cfb80 legacy=0x7fa3480af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa345d9b700 1 -- 172.21.15.3:0/2364997220 >> v1:172.21.15.3:6789/0 conn(0x7fa3480cef10 legacy=0x7fa34815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa345d9b700 1 -- 172.21.15.3:0/2364997220 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34815a760 con 0x7fa348159eb0 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.173+0000 7fa345d9b700 1 --2- 172.21.15.3:0/2364997220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa3480ce9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009910 tx=0x7fa33c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c47178b8d996a8d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34559a700 1 -- 172.21.15.3:0/2364997220 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c011630 con 0x7fa348159eb0 2023-10-06T04:52:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34559a700 1 -- 172.21.15.3:0/2364997220 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c011790 con 0x7fa348159eb0 2023-10-06T04:52:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34559a700 1 -- 172.21.15.3:0/2364997220 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c011b30 con 0x7fa348159eb0 2023-10-06T04:52:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34cb91700 1 -- 172.21.15.3:0/2364997220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 msgr2=0x7fa3480ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2364997220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa3480ce9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009910 tx=0x7fa33c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34cb91700 1 -- 172.21.15.3:0/2364997220 shutdown_connections 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2364997220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa3480ce9d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34cb91700 1 -- 172.21.15.3:0/2364997220 >> 172.21.15.3:0/2364997220 conn(0x7fa348070310 msgr2=0x7fa348070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34cb91700 1 -- 172.21.15.3:0/2364997220 shutdown_connections 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.174+0000 7fa34cb91700 1 -- 172.21.15.3:0/2364997220 wait complete. 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.175+0000 7fa34cb91700 1 Processor -- start 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.175+0000 7fa34cb91700 1 -- start start 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa34cb91700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3480cef10 0x7fa3480b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa34cb91700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3480cfb80 0x7fa34815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa34cb91700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa348155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa34cb91700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480b5cf0 con 0x7fa3480cfb80 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa34cb91700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480b50e0 con 0x7fa3480cef10 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa34cb91700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480b53c0 con 0x7fa348159eb0 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa345d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3480cfb80 0x7fa34815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa345d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3480cfb80 0x7fa34815eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48094/0 (socket says 172.21.15.3:48094) 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa345d9b700 1 -- 172.21.15.3:0/2355358652 learned_addr learned my addr 172.21.15.3:0/2355358652 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa34659c700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3480cef10 0x7fa3480b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.176+0000 7fa346d9d700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa348155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa345d9b700 1 -- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 msgr2=0x7fa348155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa345d9b700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa348155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa345d9b700 1 -- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3480cef10 msgr2=0x7fa3480b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa345d9b700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3480cef10 0x7fa3480b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa345d9b700 1 -- 172.21.15.3:0/2355358652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c009420 con 0x7fa3480cfb80 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa345d9b700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3480cfb80 0x7fa34815eec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009f00 tx=0x7fa33c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa3367fc700 1 -- 172.21.15.3:0/2355358652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c01f070 con 0x7fa3480cfb80 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa3367fc700 1 -- 172.21.15.3:0/2355358652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c002f70 con 0x7fa3480cfb80 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa3367fc700 1 -- 172.21.15.3:0/2355358652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c011900 con 0x7fa3480cfb80 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3480b56a0 con 0x7fa3480cfb80 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.177+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa348155b60 con 0x7fa3480cfb80 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.178+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34804c380 con 0x7fa3480cfb80 2023-10-06T04:52:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.178+0000 7fa3367fc700 1 -- 172.21.15.3:0/2355358652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa33c023070 con 0x7fa3480cfb80 2023-10-06T04:52:48.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.179+0000 7fa3367fc700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa328066f30 0x7fa3280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.179+0000 7fa3367fc700 1 -- 172.21.15.3:0/2355358652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa33c085980 con 0x7fa3480cfb80 2023-10-06T04:52:48.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.179+0000 7fa34659c700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa328066f30 0x7fa3280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.179+0000 7fa34659c700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa328066f30 0x7fa3280693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa338007fd0 tx=0x7fa338007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.183+0000 7fa3367fc700 1 -- 172.21.15.3:0/2355358652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa33c052a90 con 0x7fa3480cfb80 2023-10-06T04:52:48.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.374+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3481574e0 con 0x7fa3480cfb80 2023-10-06T04:52:48.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.375+0000 7fa3367fc700 1 -- 172.21.15.3:0/2355358652 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa33c02a0a0 con 0x7fa3480cfb80 2023-10-06T04:52:48.377 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:52:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.376+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa328066f30 msgr2=0x7fa3280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.376+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa328066f30 0x7fa3280693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa338007fd0 tx=0x7fa338007d50 comp rx=0 tx=0).stop 2023-10-06T04:52:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.376+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3480cfb80 msgr2=0x7fa34815eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.376+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3480cfb80 0x7fa34815eec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009f00 tx=0x7fa33c017000 comp rx=0 tx=0).stop 2023-10-06T04:52:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 shutdown_connections 2023-10-06T04:52:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa328066f30 0x7fa3280693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3480cef10 0x7fa3480b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa348159eb0 0x7fa348155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 --2- 172.21.15.3:0/2355358652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3480cfb80 0x7fa34815eec0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 >> 172.21.15.3:0/2355358652 conn(0x7fa348070310 msgr2=0x7fa348157b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 shutdown_connections 2023-10-06T04:52:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.377+0000 7fa34cb91700 1 -- 172.21.15.3:0/2355358652 wait complete. 2023-10-06T04:52:48.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:48.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.507+0000 7fd493036700 1 Processor -- start 2023-10-06T04:52:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.507+0000 7fd493036700 1 -- start start 2023-10-06T04:52:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd493036700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd493036700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd48c0cc030 0x7fd48c15db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd493036700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0cf2b0 con 0x7fd48c15d6c0 2023-10-06T04:52:48.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd493036700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0cc4b0 con 0x7fd48c0cc030 2023-10-06T04:52:48.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd493036700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0b2bd0 con 0x7fd48c15c9c0 2023-10-06T04:52:48.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd492034700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd492835700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd48c0cc030 0x7fd48c15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd492835700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd48c0cc030 0x7fd48c15db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:43278/0 (socket says 172.21.15.3:43278) 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd492034700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c15d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:45690/0 (socket says 172.21.15.3:45690) 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd492835700 1 -- 172.21.15.3:0/3919235948 learned_addr learned my addr 172.21.15.3:0/3919235948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.508+0000 7fd491833700 1 --1- 172.21.15.3:0/3919235948 >> v1:172.21.15.153:6789/0 conn(0x7fd48c15d6c0 0x7fd48c0c50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34958/0 (socket says 172.21.15.3:34958) 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd491032700 1 -- 172.21.15.3:0/3919235948 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065131525 0 0) 0x7fd48c0cf2b0 con 0x7fd48c15d6c0 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd492034700 1 -- 172.21.15.3:0/3919235948 >> v2:172.21.15.153:3300/0 conn(0x7fd48c0cc030 msgr2=0x7fd48c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd492034700 1 --2- 172.21.15.3:0/3919235948 >> v2:172.21.15.153:3300/0 conn(0x7fd48c0cc030 0x7fd48c15db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd492034700 1 -- 172.21.15.3:0/3919235948 >> v1:172.21.15.153:6789/0 conn(0x7fd48c15d6c0 legacy=0x7fd48c0c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd492034700 1 -- 172.21.15.3:0/3919235948 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0ce9c0 con 0x7fd48c15c9c0 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd492034700 1 --2- 172.21.15.3:0/3919235948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c15d180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4840098d0 tx=0x7fd484009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2f5557325c3b79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd491032700 1 -- 172.21.15.3:0/3919235948 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484005c10 con 0x7fd48c15c9c0 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd491032700 1 -- 172.21.15.3:0/3919235948 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd484005d70 con 0x7fd48c15c9c0 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.509+0000 7fd491032700 1 -- 172.21.15.3:0/3919235948 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484006260 con 0x7fd48c15c9c0 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 -- 172.21.15.3:0/3919235948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 msgr2=0x7fd48c15d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 --2- 172.21.15.3:0/3919235948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c15d180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4840098d0 tx=0x7fd484009120 comp rx=0 tx=0).stop 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 -- 172.21.15.3:0/3919235948 shutdown_connections 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 --2- 172.21.15.3:0/3919235948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c15d180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 --2- 172.21.15.3:0/3919235948 >> v2:172.21.15.153:3300/0 conn(0x7fd48c0cc030 0x7fd48c15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 -- 172.21.15.3:0/3919235948 >> 172.21.15.3:0/3919235948 conn(0x7fd48c070af0 msgr2=0x7fd48c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 -- 172.21.15.3:0/3919235948 shutdown_connections 2023-10-06T04:52:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.510+0000 7fd493036700 1 -- 172.21.15.3:0/3919235948 wait complete. 2023-10-06T04:52:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.511+0000 7fd493036700 1 Processor -- start 2023-10-06T04:52:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.511+0000 7fd493036700 1 -- start start 2023-10-06T04:52:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c0cc030 0x7fd48c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd48c15d6c0 0x7fd48c0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c157da0 con 0x7fd48c0cc030 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c156ef0 con 0x7fd48c15d6c0 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c157190 con 0x7fd48c15c9c0 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c0cc030 0x7fd48c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c0cc030 0x7fd48c0c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48098/0 (socket says 172.21.15.3:48098) 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 -- 172.21.15.3:0/4107742497 learned_addr learned my addr 172.21.15.3:0/4107742497 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd491833700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492835700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd48c15d6c0 0x7fd48c0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 -- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 msgr2=0x7fd48c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 -- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd48c15d6c0 msgr2=0x7fd48c0cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd48c15d6c0 0x7fd48c0cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 -- 172.21.15.3:0/4107742497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840093c0 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd491833700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd492034700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c0cc030 0x7fd48c0c8e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd484009f70 tx=0x7fd484016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd48a7fc700 1 -- 172.21.15.3:0/4107742497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48401d730 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd48a7fc700 1 -- 172.21.15.3:0/4107742497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48401d890 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c157470 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd48a7fc700 1 -- 172.21.15.3:0/4107742497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48401dc30 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.512+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c157a30 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.513+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c157bc0 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.514+0000 7fd48a7fc700 1 -- 172.21.15.3:0/4107742497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd48401b690 con 0x7fd48c0cc030 2023-10-06T04:52:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.514+0000 7fd48a7fc700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd478066f30 0x7fd4780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.514+0000 7fd48a7fc700 1 -- 172.21.15.3:0/4107742497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd484086430 con 0x7fd48c0cc030 2023-10-06T04:52:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.514+0000 7fd491833700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd478066f30 0x7fd4780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.515+0000 7fd491833700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd478066f30 0x7fd4780693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd480007fd0 tx=0x7fd480007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.518+0000 7fd48a7fc700 1 -- 172.21.15.3:0/4107742497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd484053540 con 0x7fd48c0cc030 2023-10-06T04:52:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.712+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd48c04c380 con 0x7fd48c0cc030 2023-10-06T04:52:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.713+0000 7fd48a7fc700 1 -- 172.21.15.3:0/4107742497 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fd48402a090 con 0x7fd48c0cc030 2023-10-06T04:52:48.716 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:48.716 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd478066f30 msgr2=0x7fd4780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd478066f30 0x7fd4780693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd480007fd0 tx=0x7fd480007e30 comp rx=0 tx=0).stop 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c0cc030 msgr2=0x7fd48c0c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c0cc030 0x7fd48c0c8e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd484009f70 tx=0x7fd484016000 comp rx=0 tx=0).stop 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 shutdown_connections 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd478066f30 0x7fd4780693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd48c15d6c0 0x7fd48c0cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c0cc030 0x7fd48c0c8e80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 --2- 172.21.15.3:0/4107742497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd48c15c9c0 0x7fd48c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 >> 172.21.15.3:0/4107742497 conn(0x7fd48c070af0 msgr2=0x7fd48c0ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:48.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 shutdown_connections 2023-10-06T04:52:48.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.715+0000 7fd493036700 1 -- 172.21.15.3:0/4107742497 wait complete. 2023-10-06T04:52:48.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:52:15.735846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:52:17.015278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:52:48.730 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:48.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.850+0000 7f2fd088e700 1 Processor -- start 2023-10-06T04:52:48.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.850+0000 7f2fd088e700 1 -- start start 2023-10-06T04:52:48.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fd088e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2fcc0af290 0x7f2fcc0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fd088e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2fcc0af660 0x7f2fcc0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fd088e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0cd190 0x7f2fcc0c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fd088e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fcc0cb190 con 0x7f2fcc0af660 2023-10-06T04:52:48.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fd088e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fcc159950 con 0x7f2fcc0af290 2023-10-06T04:52:48.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fd088e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc0cd6a0 con 0x7f2fcc0cd190 2023-10-06T04:52:48.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2fcc0af660 0x7f2fcc0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2fcc0af660 0x7f2fcc0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48102/0 (socket says 172.21.15.3:48102) 2023-10-06T04:52:48.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 -- 172.21.15.3:0/1474803641 learned_addr learned my addr 172.21.15.3:0/1474803641 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:48.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fcad9d700 1 --2- 172.21.15.3:0/1474803641 >> v2:172.21.15.153:3300/0 conn(0x7f2fcc0af290 0x7f2fcc0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fcb59e700 1 --2- 172.21.15.3:0/1474803641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0cd190 0x7f2fcc0c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 -- 172.21.15.3:0/1474803641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0cd190 msgr2=0x7f2fcc0c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:48.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 --2- 172.21.15.3:0/1474803641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0cd190 0x7f2fcc0c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 -- 172.21.15.3:0/1474803641 >> v2:172.21.15.153:3300/0 conn(0x7f2fcc0af290 msgr2=0x7f2fcc0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:48.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 --2- 172.21.15.3:0/1474803641 >> v2:172.21.15.153:3300/0 conn(0x7f2fcc0af290 0x7f2fcc0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 -- 172.21.15.3:0/1474803641 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0cb3f0 con 0x7f2fcc0af660 2023-10-06T04:52:48.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fcad9d700 1 --2- 172.21.15.3:0/1474803641 >> v2:172.21.15.153:3300/0 conn(0x7f2fcc0af290 0x7f2fcc0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:48.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fca59c700 1 --2- 172.21.15.3:0/1474803641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af660 0x7f2fcc0ccc50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc089580 tx=0x7f2fb4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73828f723326cd54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.851+0000 7f2fc9d9b700 1 -- 172.21.15.3:0/1474803641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4005c10 con 0x7f2fcc0af660 2023-10-06T04:52:48.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fc9d9b700 1 -- 172.21.15.3:0/1474803641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4005d70 con 0x7f2fcc0af660 2023-10-06T04:52:48.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fc9d9b700 1 -- 172.21.15.3:0/1474803641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4006260 con 0x7f2fcc0af660 2023-10-06T04:52:48.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 -- 172.21.15.3:0/1474803641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af660 msgr2=0x7f2fcc0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 --2- 172.21.15.3:0/1474803641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af660 0x7f2fcc0ccc50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc089580 tx=0x7f2fb4009220 comp rx=0 tx=0).stop 2023-10-06T04:52:48.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 -- 172.21.15.3:0/1474803641 shutdown_connections 2023-10-06T04:52:48.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 --2- 172.21.15.3:0/1474803641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0cd190 0x7f2fcc0c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 --2- 172.21.15.3:0/1474803641 >> v2:172.21.15.153:3300/0 conn(0x7f2fcc0af290 0x7f2fcc0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 --2- 172.21.15.3:0/1474803641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af660 0x7f2fcc0ccc50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 -- 172.21.15.3:0/1474803641 >> 172.21.15.3:0/1474803641 conn(0x7f2fcc089dd0 msgr2=0x7f2fcc0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:48.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 -- 172.21.15.3:0/1474803641 shutdown_connections 2023-10-06T04:52:48.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.852+0000 7f2fd088e700 1 -- 172.21.15.3:0/1474803641 wait complete. 2023-10-06T04:52:48.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.853+0000 7f2fd088e700 1 Processor -- start 2023-10-06T04:52:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.853+0000 7f2fd088e700 1 -- start start 2023-10-06T04:52:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fd088e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af290 0x7f2fcc157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fd088e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0af660 0x7f2fcc0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fd088e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fcc0cd190 0x7f2fcc0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fd088e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc0b04e0 con 0x7f2fcc0af290 2023-10-06T04:52:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fd088e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc16a3c0 con 0x7f2fcc0cd190 2023-10-06T04:52:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fd088e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc0afbb0 con 0x7f2fcc0af660 2023-10-06T04:52:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af290 0x7f2fcc157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af290 0x7f2fcc157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48104/0 (socket says 172.21.15.3:48104) 2023-10-06T04:52:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 -- 172.21.15.3:0/739540037 learned_addr learned my addr 172.21.15.3:0/739540037 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcb59e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fcc0cd190 0x7f2fcc0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fca59c700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0af660 0x7f2fcc0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 -- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0af660 msgr2=0x7f2fcc0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0af660 0x7f2fcc0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 -- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fcc0cd190 msgr2=0x7f2fcc0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fcc0cd190 0x7f2fcc0b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcad9d700 1 -- 172.21.15.3:0/739540037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb4009480 con 0x7f2fcc0af290 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fcb59e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fcc0cd190 0x7f2fcc0b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.854+0000 7f2fca59c700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0af660 0x7f2fcc0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.855+0000 7f2fcad9d700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af290 0x7f2fcc157240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400d220 tx=0x7f2fc4009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.855+0000 7f2fc2ffd700 1 -- 172.21.15.3:0/739540037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4015440 con 0x7f2fcc0af290 2023-10-06T04:52:48.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.855+0000 7f2fc2ffd700 1 -- 172.21.15.3:0/739540037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc400f040 con 0x7f2fcc0af290 2023-10-06T04:52:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.855+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fcc0afe90 con 0x7f2fcc0af290 2023-10-06T04:52:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.855+0000 7f2fc2ffd700 1 -- 172.21.15.3:0/739540037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40157d0 con 0x7f2fcc0af290 2023-10-06T04:52:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.855+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fcc0b0970 con 0x7f2fcc0af290 2023-10-06T04:52:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.856+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fcc0b07c0 con 0x7f2fcc0af290 2023-10-06T04:52:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.856+0000 7f2fc2ffd700 1 -- 172.21.15.3:0/739540037 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fc4015bc0 con 0x7f2fcc0af290 2023-10-06T04:52:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.857+0000 7f2fc2ffd700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2fac066f30 0x7f2fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.857+0000 7f2fc2ffd700 1 -- 172.21.15.3:0/739540037 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2fc4084c50 con 0x7f2fcc0af290 2023-10-06T04:52:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.857+0000 7f2fca59c700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2fac066f30 0x7f2fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.857+0000 7f2fca59c700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2fac066f30 0x7f2fac0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc089580 tx=0x7f2fb4016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:48.861+0000 7f2fc2ffd700 1 -- 172.21.15.3:0/739540037 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fc404e5a0 con 0x7f2fcc0af290 2023-10-06T04:52:49.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.054+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2fcc04c380 con 0x7f2fcc0af290 2023-10-06T04:52:49.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.054+0000 7f2fc2ffd700 1 -- 172.21.15.3:0/739540037 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2fc401a020 con 0x7f2fcc0af290 2023-10-06T04:52:49.057 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2fac066f30 msgr2=0x7f2fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2fac066f30 0x7f2fac0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc089580 tx=0x7f2fb4016000 comp rx=0 tx=0).stop 2023-10-06T04:52:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af290 msgr2=0x7f2fcc157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af290 0x7f2fcc157240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400d220 tx=0x7f2fc4009ae0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 shutdown_connections 2023-10-06T04:52:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2fac066f30 0x7f2fac0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fcc0cd190 0x7f2fcc0b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fcc0af290 0x7f2fcc157240 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 --2- 172.21.15.3:0/739540037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fcc0af660 0x7f2fcc0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 >> 172.21.15.3:0/739540037 conn(0x7f2fcc089dd0 msgr2=0x7f2fcc06d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 shutdown_connections 2023-10-06T04:52:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.056+0000 7f2fd088e700 1 -- 172.21.15.3:0/739540037 wait complete. 2023-10-06T04:52:49.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:49.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.190+0000 7f5947499700 1 Processor -- start 2023-10-06T04:52:49.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.190+0000 7f5947499700 1 -- start start 2023-10-06T04:52:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5947499700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5940156050 0x7f594006f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5947499700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f59400c6d70 0x7f594006f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5947499700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400c7f60 con 0x7f5940156420 2023-10-06T04:52:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5947499700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400c7350 con 0x7f5940156050 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5947499700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400c7630 con 0x7f59400c6d70 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5945235700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5940156050 0x7f594006f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5945235700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5940156050 0x7f594006f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48110/0 (socket says 172.21.15.3:48110) 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.191+0000 7f5945235700 1 -- 172.21.15.3:0/542820278 learned_addr learned my addr 172.21.15.3:0/542820278 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f5945a36700 1 --2- 172.21.15.3:0/542820278 >> v2:172.21.15.153:3300/0 conn(0x7f59400c6d70 0x7f594006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f5945235700 1 -- 172.21.15.3:0/542820278 >> v2:172.21.15.153:3300/0 conn(0x7f59400c6d70 msgr2=0x7f594006f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f5945235700 1 --2- 172.21.15.3:0/542820278 >> v2:172.21.15.153:3300/0 conn(0x7f59400c6d70 0x7f594006f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f5945235700 1 -- 172.21.15.3:0/542820278 >> v1:172.21.15.153:6789/0 conn(0x7f5940156420 legacy=0x7f59400c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f5945235700 1 -- 172.21.15.3:0/542820278 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400c7910 con 0x7f5940156050 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f5945235700 1 --2- 172.21.15.3:0/542820278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f594006f230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f59400c8560 tx=0x7f5938009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6615c25424bcd93d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f593ffff700 1 -- 172.21.15.3:0/542820278 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800e070 con 0x7f5940156050 2023-10-06T04:52:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f593ffff700 1 -- 172.21.15.3:0/542820278 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800ba10 con 0x7f5940156050 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.192+0000 7f593ffff700 1 -- 172.21.15.3:0/542820278 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59380092e0 con 0x7f5940156050 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 -- 172.21.15.3:0/542820278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 msgr2=0x7f594006f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 --2- 172.21.15.3:0/542820278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f594006f230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f59400c8560 tx=0x7f5938009840 comp rx=0 tx=0).stop 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 -- 172.21.15.3:0/542820278 shutdown_connections 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 --2- 172.21.15.3:0/542820278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f594006f230 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 --2- 172.21.15.3:0/542820278 >> v2:172.21.15.153:3300/0 conn(0x7f59400c6d70 0x7f594006f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 -- 172.21.15.3:0/542820278 >> 172.21.15.3:0/542820278 conn(0x7f594006ce40 msgr2=0x7f59400cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 -- 172.21.15.3:0/542820278 shutdown_connections 2023-10-06T04:52:49.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.193+0000 7f5947499700 1 -- 172.21.15.3:0/542820278 wait complete. 2023-10-06T04:52:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.194+0000 7f5947499700 1 Processor -- start 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.194+0000 7f5947499700 1 -- start start 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5947499700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59400c6d70 0x7f5940167a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5947499700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f5940139c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5947499700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5940156420 0x7f594006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5947499700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594013ac90 con 0x7f5940156050 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5947499700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594013a1c0 con 0x7f59400c6d70 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5947499700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59400b0ab0 con 0x7f5940156420 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f5940139c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f5940139c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48120/0 (socket says 172.21.15.3:48120) 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 -- 172.21.15.3:0/3261073273 learned_addr learned my addr 172.21.15.3:0/3261073273 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5945235700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59400c6d70 0x7f5940167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5945a36700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5940156420 0x7f594006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 -- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5940156420 msgr2=0x7f594006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5940156420 0x7f594006e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 -- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59400c6d70 msgr2=0x7f5940167a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59400c6d70 0x7f5940167a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 -- 172.21.15.3:0/3261073273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5938009ae0 con 0x7f5940156050 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5945a36700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5940156420 0x7f594006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5945235700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59400c6d70 0x7f5940167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5944a34700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f5940139c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f593400b5e0 tx=0x7f59340090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f593d7fa700 1 -- 172.21.15.3:0/3261073273 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593400f510 con 0x7f5940156050 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f593d7fa700 1 -- 172.21.15.3:0/3261073273 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593400f6b0 con 0x7f5940156050 2023-10-06T04:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594013a3f0 con 0x7f5940156050 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.195+0000 7f593d7fa700 1 -- 172.21.15.3:0/3261073273 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593400fa70 con 0x7f5940156050 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.196+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594013a9b0 con 0x7f5940156050 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.196+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400b0910 con 0x7f5940156050 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.197+0000 7f593d7fa700 1 -- 172.21.15.3:0/3261073273 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f593401a400 con 0x7f5940156050 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.197+0000 7f593d7fa700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f592c066f30 0x7f592c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.197+0000 7f593d7fa700 1 -- 172.21.15.3:0/3261073273 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5934084280 con 0x7f5940156050 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.197+0000 7f5945235700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f592c066f30 0x7f592c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.198+0000 7f5945235700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f592c066f30 0x7f592c0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5938009ab0 tx=0x7f5938017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.201+0000 7f593d7fa700 1 -- 172.21.15.3:0/3261073273 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59400b0910 con 0x7f5940156050 2023-10-06T04:52:49.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.400+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59400b0910 con 0x7f5940156050 2023-10-06T04:52:49.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.400+0000 7f593d7fa700 1 -- 172.21.15.3:0/3261073273 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59400b0910 con 0x7f5940156050 2023-10-06T04:52:49.403 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f592c066f30 msgr2=0x7f592c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f592c066f30 0x7f592c0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5938009ab0 tx=0x7f5938017000 comp rx=0 tx=0).stop 2023-10-06T04:52:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 msgr2=0x7f5940139c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f5940139c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f593400b5e0 tx=0x7f59340090f0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 shutdown_connections 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f592c066f30 0x7f592c0693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59400c6d70 0x7f5940167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5940156420 0x7f594006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 --2- 172.21.15.3:0/3261073273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940156050 0x7f5940139c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.402+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 >> 172.21.15.3:0/3261073273 conn(0x7f594006ce40 msgr2=0x7f5940151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.403+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 shutdown_connections 2023-10-06T04:52:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.403+0000 7f5947499700 1 -- 172.21.15.3:0/3261073273 wait complete. 2023-10-06T04:52:49.417 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:52:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.535+0000 7f6e38c27700 1 Processor -- start 2023-10-06T04:52:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.535+0000 7f6e38c27700 1 -- start start 2023-10-06T04:52:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e38c27700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340cfb80 0x7f6e340af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e38c27700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e34159eb0 0x7f6e3415da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e38c27700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415d510 con 0x7f6e340afad0 2023-10-06T04:52:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e38c27700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3415a480 con 0x7f6e34159eb0 2023-10-06T04:52:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e38c27700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415a760 con 0x7f6e340cfb80 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e32d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e34159eb0 0x7f6e3415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e31d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6e340afad0 0x7f6e340ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53912/0 (socket says 172.21.15.3:53912) 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.536+0000 7f6e31d9b700 1 -- 172.21.15.3:0/3392772577 learned_addr learned my addr 172.21.15.3:0/3392772577 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e3259c700 1 --2- 172.21.15.3:0/3392772577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340cfb80 0x7f6e340af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e32d9d700 1 -- 172.21.15.3:0/3392772577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340cfb80 msgr2=0x7f6e340af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e32d9d700 1 --2- 172.21.15.3:0/3392772577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340cfb80 0x7f6e340af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e32d9d700 1 -- 172.21.15.3:0/3392772577 >> v1:172.21.15.3:6789/0 conn(0x7f6e340afad0 legacy=0x7f6e340ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e32d9d700 1 -- 172.21.15.3:0/3392772577 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3415aa40 con 0x7f6e34159eb0 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e3159a700 1 -- 172.21.15.3:0/3392772577 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645434288 0 0) 0x7f6e3415d510 con 0x7f6e340afad0 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e3259c700 1 --2- 172.21.15.3:0/3392772577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340cfb80 0x7f6e340af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e32d9d700 1 --2- 172.21.15.3:0/3392772577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e3415da10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009e60 tx=0x7f6e24009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf7b58e378586c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e3159a700 1 -- 172.21.15.3:0/3392772577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400e070 con 0x7f6e34159eb0 2023-10-06T04:52:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e3159a700 1 -- 172.21.15.3:0/3392772577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2400bc30 con 0x7f6e34159eb0 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e3159a700 1 -- 172.21.15.3:0/3392772577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24012430 con 0x7f6e34159eb0 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e38c27700 1 -- 172.21.15.3:0/3392772577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 msgr2=0x7f6e3415da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.537+0000 7f6e38c27700 1 --2- 172.21.15.3:0/3392772577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e3415da10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009e60 tx=0x7f6e24009780 comp rx=0 tx=0).stop 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.538+0000 7f6e38c27700 1 -- 172.21.15.3:0/3392772577 shutdown_connections 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.538+0000 7f6e38c27700 1 --2- 172.21.15.3:0/3392772577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e3415da10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.538+0000 7f6e38c27700 1 --2- 172.21.15.3:0/3392772577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340cfb80 0x7f6e340af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.538+0000 7f6e38c27700 1 -- 172.21.15.3:0/3392772577 >> 172.21.15.3:0/3392772577 conn(0x7f6e34070310 msgr2=0x7f6e34070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.538+0000 7f6e38c27700 1 -- 172.21.15.3:0/3392772577 shutdown_connections 2023-10-06T04:52:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.538+0000 7f6e38c27700 1 -- 172.21.15.3:0/3392772577 wait complete. 2023-10-06T04:52:49.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.539+0000 7f6e38c27700 1 Processor -- start 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.539+0000 7f6e38c27700 1 -- start start 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.539+0000 7f6e38c27700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340afad0 0x7f6e34154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e38c27700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e340cfb80 0x7f6e3415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e38c27700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e340c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e38c27700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340c9c90 con 0x7f6e34159eb0 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e38c27700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34155470 con 0x7f6e340cfb80 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e38c27700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340c9080 con 0x7f6e340afad0 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e340c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e340c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48130/0 (socket says 172.21.15.3:48130) 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 -- 172.21.15.3:0/2481667692 learned_addr learned my addr 172.21.15.3:0/2481667692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e3259c700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340afad0 0x7f6e34154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e31d9b700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e340cfb80 0x7f6e3415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 -- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340afad0 msgr2=0x7f6e34154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340afad0 0x7f6e34154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 -- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e340cfb80 msgr2=0x7f6e3415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e340cfb80 0x7f6e3415bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 -- 172.21.15.3:0/2481667692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24009a20 con 0x7f6e34159eb0 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e3259c700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340afad0 0x7f6e34154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e32d9d700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e340c5f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6e34089720 tx=0x7f6e24019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e2a7fc700 1 -- 172.21.15.3:0/2481667692 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24027470 con 0x7f6e34159eb0 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e2a7fc700 1 -- 172.21.15.3:0/2481667692 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e240275f0 con 0x7f6e34159eb0 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e340c9360 con 0x7f6e34159eb0 2023-10-06T04:52:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e2a7fc700 1 -- 172.21.15.3:0/2481667692 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240279b0 con 0x7f6e34159eb0 2023-10-06T04:52:49.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.540+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e340c9920 con 0x7f6e34159eb0 2023-10-06T04:52:49.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.541+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340c9e00 con 0x7f6e34159eb0 2023-10-06T04:52:49.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.542+0000 7f6e2a7fc700 1 -- 172.21.15.3:0/2481667692 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e24024070 con 0x7f6e34159eb0 2023-10-06T04:52:49.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.542+0000 7f6e2a7fc700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e14066f30 0x7f6e140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.542+0000 7f6e2a7fc700 1 -- 172.21.15.3:0/2481667692 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e24086ed0 con 0x7f6e34159eb0 2023-10-06T04:52:49.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.542+0000 7f6e3259c700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e14066f30 0x7f6e140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.543+0000 7f6e3259c700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e14066f30 0x7f6e140693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6e340c6dc0 tx=0x7f6e2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.546+0000 7f6e2a7fc700 1 -- 172.21.15.3:0/2481667692 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e24053fe0 con 0x7f6e34159eb0 2023-10-06T04:52:49.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.741+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e3404c380 con 0x7f6e34159eb0 2023-10-06T04:52:49.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.741+0000 7f6e2a7fc700 1 -- 172.21.15.3:0/2481667692 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e24057600 con 0x7f6e34159eb0 2023-10-06T04:52:49.744 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:52:49.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e14066f30 msgr2=0x7f6e140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e14066f30 0x7f6e140693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6e340c6dc0 tx=0x7f6e2c008040 comp rx=0 tx=0).stop 2023-10-06T04:52:49.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 msgr2=0x7f6e340c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e340c5f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6e34089720 tx=0x7f6e24019000 comp rx=0 tx=0).stop 2023-10-06T04:52:49.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 shutdown_connections 2023-10-06T04:52:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e14066f30 0x7f6e140693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e34159eb0 0x7f6e340c5f90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e340afad0 0x7f6e34154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 --2- 172.21.15.3:0/2481667692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e340cfb80 0x7f6e3415bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 >> 172.21.15.3:0/2481667692 conn(0x7f6e34070310 msgr2=0x7f6e34157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 shutdown_connections 2023-10-06T04:52:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.743+0000 7f6e38c27700 1 -- 172.21.15.3:0/2481667692 wait complete. 2023-10-06T04:52:49.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:52:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.878+0000 7ff3c428d700 1 Processor -- start 2023-10-06T04:52:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.878+0000 7ff3c428d700 1 -- start start 2023-10-06T04:52:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.878+0000 7ff3c428d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 0x7ff3bc0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.878+0000 7ff3c428d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff3bc0afad0 0x7ff3bc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.878+0000 7ff3c428d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc15d510 con 0x7ff3bc0cef10 2023-10-06T04:52:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c428d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc15a480 con 0x7ff3bc0afad0 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c428d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc15a760 con 0x7ff3bc0cfb80 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c282a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff3bc0cef10 0x7ff3bc15da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53922/0 (socket says 172.21.15.3:53922) 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c282a700 1 -- 172.21.15.3:0/629204585 learned_addr learned my addr 172.21.15.3:0/629204585 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c2029700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 0x7ff3bc0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1828700 1 --2- 172.21.15.3:0/629204585 >> v2:172.21.15.153:3300/0 conn(0x7ff3bc0afad0 0x7ff3bc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413204442 0 0) 0x7ff3bc15d510 con 0x7ff3bc0cef10 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a4003570 con 0x7ff3bc0cef10 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2001518396 0 0) 0x7ff3a4003570 con 0x7ff3bc0cef10 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 msgr2=0x7ff3bc0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 0x7ff3bc0af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 >> v2:172.21.15.153:3300/0 conn(0x7ff3bc0afad0 msgr2=0x7ff3bc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 --2- 172.21.15.3:0/629204585 >> v2:172.21.15.153:3300/0 conn(0x7ff3bc0afad0 0x7ff3bc0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc15aa40 con 0x7ff3bc0cef10 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7ff3ac0017f0 con 0x7ff3bc0cef10 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 >> v1:172.21.15.3:6789/0 conn(0x7ff3bc0cef10 legacy=0x7ff3bc15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a4004810 0x7ff3a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 0x7ff3a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc15a480 con 0x7ff3a4003a60 2023-10-06T04:52:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc15a760 con 0x7ff3a4004810 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.879+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a400ba60 con 0x7ff3bc0cfb80 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c2029700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c1828700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a4004810 0x7ff3a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c282a700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 0x7ff3a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c2029700 1 -- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 msgr2=0x7ff3a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c2029700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc0cfb80 0x7ff3a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c2029700 1 -- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a4004810 msgr2=0x7ff3a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c2029700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a4004810 0x7ff3a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c2029700 1 -- 172.21.15.3:0/629204585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a4003570 con 0x7ff3a4003a60 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c2029700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3a4003e70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009c30 tx=0x7ff3b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4a60e4e916f6e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400e710 con 0x7ff3a4003a60 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b400e870 con 0x7ff3a4003a60 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.880+0000 7ff3c1027700 1 -- 172.21.15.3:0/629204585 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400ec30 con 0x7ff3a4003a60 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.881+0000 7ff3c428d700 1 -- 172.21.15.3:0/629204585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 msgr2=0x7ff3a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.881+0000 7ff3c428d700 1 --2- 172.21.15.3:0/629204585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3a4003e70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009c30 tx=0x7ff3b40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.881+0000 7ff3c282a700 1 -- 172.21.15.3:0/629204585 reap_dead start 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.881+0000 7ff3c428d700 1 -- 172.21.15.3:0/629204585 shutdown_connections 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.881+0000 7ff3c428d700 1 -- 172.21.15.3:0/629204585 >> 172.21.15.3:0/629204585 conn(0x7ff3bc070310 msgr2=0x7ff3bc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.881+0000 7ff3c428d700 1 -- 172.21.15.3:0/629204585 shutdown_connections 2023-10-06T04:52:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.881+0000 7ff3c428d700 1 -- 172.21.15.3:0/629204585 wait complete. 2023-10-06T04:52:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.882+0000 7ff3c428d700 1 Processor -- start 2023-10-06T04:52:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.882+0000 7ff3c428d700 1 -- start start 2023-10-06T04:52:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3bc0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3bc0b0fe0 0x7ff3bc1697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc169ce0 0x7ff3bc16a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0b31e0 con 0x7ff3a4003a60 2023-10-06T04:52:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc16a6a0 con 0x7ff3bc0b0fe0 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0b1740 con 0x7ff3bc169ce0 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3bc0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3bc0b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48152/0 (socket says 172.21.15.3:48152) 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 -- 172.21.15.3:0/3525921713 learned_addr learned my addr 172.21.15.3:0/3525921713 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c282a700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc169ce0 0x7ff3bc16a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c1828700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3bc0b0fe0 0x7ff3bc1697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 -- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc169ce0 msgr2=0x7ff3bc16a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc169ce0 0x7ff3bc16a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 -- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3bc0b0fe0 msgr2=0x7ff3bc1697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3bc0b0fe0 0x7ff3bc1697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 -- 172.21.15.3:0/3525921713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc0b1a20 con 0x7ff3a4003a60 2023-10-06T04:52:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c1828700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3bc0b0fe0 0x7ff3bc1697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c2029700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3bc0b0aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff3b401ef70 tx=0x7ff3b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3ba7fc700 1 -- 172.21.15.3:0/3525921713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4023070 con 0x7ff3a4003a60 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3ba7fc700 1 -- 172.21.15.3:0/3525921713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b400e9e0 con 0x7ff3a4003a60 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc0b1d00 con 0x7ff3a4003a60 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3ba7fc700 1 -- 172.21.15.3:0/3525921713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b401e030 con 0x7ff3a4003a60 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.883+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc0b1450 con 0x7ff3a4003a60 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.884+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc04c380 con 0x7ff3a4003a60 2023-10-06T04:52:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.885+0000 7ff3ba7fc700 1 -- 172.21.15.3:0/3525921713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3b4013470 con 0x7ff3a4003a60 2023-10-06T04:52:49.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.885+0000 7ff3ba7fc700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:49.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.885+0000 7ff3ba7fc700 1 -- 172.21.15.3:0/3525921713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3b4085e30 con 0x7ff3a4003a60 2023-10-06T04:52:49.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.885+0000 7ff3c1828700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:49.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.886+0000 7ff3c1828700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff3a8066f30 0x7ff3a80693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0006dc0 tx=0x7ff3b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:49.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:49.890+0000 7ff3ba7fc700 1 -- 172.21.15.3:0/3525921713 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3b4052f40 con 0x7ff3a4003a60 2023-10-06T04:52:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.081+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3bc0ce470 con 0x7ff3a4003a60 2023-10-06T04:52:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.081+0000 7ff3ba7fc700 1 -- 172.21.15.3:0/3525921713 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3b4056560 con 0x7ff3a4003a60 2023-10-06T04:52:50.084 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:52:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff3a8066f30 msgr2=0x7ff3a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff3a8066f30 0x7ff3a80693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0006dc0 tx=0x7ff3b0008040 comp rx=0 tx=0).stop 2023-10-06T04:52:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 msgr2=0x7ff3bc0b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3bc0b0aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff3b401ef70 tx=0x7ff3b401a000 comp rx=0 tx=0).stop 2023-10-06T04:52:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 shutdown_connections 2023-10-06T04:52:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3bc169ce0 0x7ff3bc16a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a4003a60 0x7ff3bc0b0aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 --2- 172.21.15.3:0/3525921713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3bc0b0fe0 0x7ff3bc1697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 >> 172.21.15.3:0/3525921713 conn(0x7ff3bc070310 msgr2=0x7ff3bc06d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 shutdown_connections 2023-10-06T04:52:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.083+0000 7ff3c428d700 1 -- 172.21.15.3:0/3525921713 wait complete. 2023-10-06T04:52:50.127 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2023-10-06T04:52:50.127 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T04:52:50.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-06T04:52:50.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 Processor -- start 2023-10-06T04:52:50.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 -- start start 2023-10-06T04:52:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e68159500 0x7f8e680cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e681598d0 0x7f8e680cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e680b2220 0x7f8e680b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680cb7b0 con 0x7f8e681598d0 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680cc5c0 con 0x7f8e68159500 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.244+0000 7f8e6d60c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680cb9b0 con 0x7f8e680b2220 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e681598d0 0x7f8e680cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e681598d0 0x7f8e680cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48166/0 (socket says 172.21.15.3:48166) 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 -- 172.21.15.3:0/50276522 learned_addr learned my addr 172.21.15.3:0/50276522 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e677fe700 1 --2- 172.21.15.3:0/50276522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e680b2220 0x7f8e680b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e66ffd700 1 --2- 172.21.15.3:0/50276522 >> v2:172.21.15.153:3300/0 conn(0x7f8e68159500 0x7f8e680cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 -- 172.21.15.3:0/50276522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e680b2220 msgr2=0x7f8e680b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 --2- 172.21.15.3:0/50276522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e680b2220 0x7f8e680b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 -- 172.21.15.3:0/50276522 >> v2:172.21.15.153:3300/0 conn(0x7f8e68159500 msgr2=0x7f8e680cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 --2- 172.21.15.3:0/50276522 >> v2:172.21.15.153:3300/0 conn(0x7f8e68159500 0x7f8e680cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 -- 172.21.15.3:0/50276522 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e680cbc90 con 0x7f8e681598d0 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e5e7fc700 1 --2- 172.21.15.3:0/50276522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e680cb270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e68089390 tx=0x7f8e50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2133c5291add60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e667fc700 1 -- 172.21.15.3:0/50276522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5000b640 con 0x7f8e681598d0 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e667fc700 1 -- 172.21.15.3:0/50276522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5000b7a0 con 0x7f8e681598d0 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.245+0000 7f8e667fc700 1 -- 172.21.15.3:0/50276522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5000bb60 con 0x7f8e681598d0 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 -- 172.21.15.3:0/50276522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 msgr2=0x7f8e680cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/50276522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e680cb270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e68089390 tx=0x7f8e50009250 comp rx=0 tx=0).stop 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 -- 172.21.15.3:0/50276522 shutdown_connections 2023-10-06T04:52:50.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/50276522 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e680b2220 0x7f8e680b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/50276522 >> v2:172.21.15.153:3300/0 conn(0x7f8e68159500 0x7f8e680cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/50276522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e680cb270 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 -- 172.21.15.3:0/50276522 >> 172.21.15.3:0/50276522 conn(0x7f8e68070310 msgr2=0x7f8e68070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:50.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 -- 172.21.15.3:0/50276522 shutdown_connections 2023-10-06T04:52:50.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.246+0000 7f8e6d60c700 1 -- 172.21.15.3:0/50276522 wait complete. 2023-10-06T04:52:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.247+0000 7f8e6d60c700 1 Processor -- start 2023-10-06T04:52:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.247+0000 7f8e6d60c700 1 -- start start 2023-10-06T04:52:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e6d60c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680b2220 0x7f8e681558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e6d60c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e68159500 0x7f8e680c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e6d60c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e6815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e6d60c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680c8ad0 con 0x7f8e681598d0 2023-10-06T04:52:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e6d60c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680c7ec0 con 0x7f8e680b2220 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e6d60c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680c81a0 con 0x7f8e68159500 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e6815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e6815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48180/0 (socket says 172.21.15.3:48180) 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 -- 172.21.15.3:0/2203707283 learned_addr learned my addr 172.21.15.3:0/2203707283 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e5e7fc700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e68159500 0x7f8e680c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e66ffd700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680b2220 0x7f8e681558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 -- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e68159500 msgr2=0x7f8e680c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e68159500 0x7f8e680c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 -- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680b2220 msgr2=0x7f8e681558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680b2220 0x7f8e681558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e677fe700 1 -- 172.21.15.3:0/2203707283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e500094f0 con 0x7f8e681598d0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.248+0000 7f8e66ffd700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680b2220 0x7f8e681558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.249+0000 7f8e677fe700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e6815bcd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009e90 tx=0x7f8e6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.249+0000 7f8e5f7fe700 1 -- 172.21.15.3:0/2203707283 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000cc20 con 0x7f8e681598d0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.249+0000 7f8e5f7fe700 1 -- 172.21.15.3:0/2203707283 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6000f040 con 0x7f8e681598d0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.249+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e680c8480 con 0x7f8e681598d0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.249+0000 7f8e5f7fe700 1 -- 172.21.15.3:0/2203707283 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60014480 con 0x7f8e681598d0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.249+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e68155df0 con 0x7f8e681598d0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.250+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e680c8c40 con 0x7f8e681598d0 2023-10-06T04:52:50.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.250+0000 7f8e5f7fe700 1 -- 172.21.15.3:0/2203707283 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e60014830 con 0x7f8e681598d0 2023-10-06T04:52:50.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.251+0000 7f8e5f7fe700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8e48066f30 0x7f8e480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.251+0000 7f8e5f7fe700 1 -- 172.21.15.3:0/2203707283 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8e600850b0 con 0x7f8e681598d0 2023-10-06T04:52:50.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.251+0000 7f8e66ffd700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8e48066f30 0x7f8e480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.251+0000 7f8e66ffd700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8e48066f30 0x7f8e480693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009d80 tx=0x7f8e58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:50.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.255+0000 7f8e5f7fe700 1 -- 172.21.15.3:0/2203707283 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e60055c50 con 0x7f8e681598d0 2023-10-06T04:52:50.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.531+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f8e68071680 con 0x7f8e681598d0 2023-10-06T04:52:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.783+0000 7f8e5f7fe700 1 -- 172.21.15.3:0/2203707283 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v48) v1 ==== 168+0+0 (secure 0 0 0) 0x7f8e600557e0 con 0x7f8e681598d0 2023-10-06T04:52:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8e48066f30 msgr2=0x7f8e480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8e48066f30 0x7f8e480693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009d80 tx=0x7f8e58008040 comp rx=0 tx=0).stop 2023-10-06T04:52:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 msgr2=0x7f8e6815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e6815bcd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009e90 tx=0x7f8e6000b040 comp rx=0 tx=0).stop 2023-10-06T04:52:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 shutdown_connections 2023-10-06T04:52:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8e48066f30 0x7f8e480693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680b2220 0x7f8e681558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e681598d0 0x7f8e6815bcd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 --2- 172.21.15.3:0/2203707283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e68159500 0x7f8e680c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 >> 172.21.15.3:0/2203707283 conn(0x7f8e68070310 msgr2=0x7f8e6815d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 shutdown_connections 2023-10-06T04:52:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.786+0000 7f8e6d60c700 1 -- 172.21.15.3:0/2203707283 wait complete. 2023-10-06T04:52:50.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:52:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.921+0000 7f5795a62700 1 Processor -- start 2023-10-06T04:52:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.921+0000 7f5795a62700 1 -- start start 2023-10-06T04:52:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f5795a62700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900cfb80 0x7f57900af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f5795a62700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5790159eb0 0x7f579015da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f5795a62700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579015d510 con 0x7f57900afad0 2023-10-06T04:52:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f5795a62700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579015a480 con 0x7f5790159eb0 2023-10-06T04:52:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f5795a62700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579015a760 con 0x7f57900cfb80 2023-10-06T04:52:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f578ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5790159eb0 0x7f579015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f578ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5790159eb0 0x7f579015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48196/0 (socket says 172.21.15.3:48196) 2023-10-06T04:52:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.922+0000 7f578ffff700 1 -- 172.21.15.3:0/4030686419 learned_addr learned my addr 172.21.15.3:0/4030686419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578f7fe700 1 --2- 172.21.15.3:0/4030686419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900cfb80 0x7f57900af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578ffff700 1 -- 172.21.15.3:0/4030686419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900cfb80 msgr2=0x7f57900af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578ffff700 1 --2- 172.21.15.3:0/4030686419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900cfb80 0x7f57900af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578ffff700 1 -- 172.21.15.3:0/4030686419 >> v1:172.21.15.153:6789/0 conn(0x7f57900afad0 legacy=0x7f57900ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578ffff700 1 -- 172.21.15.3:0/4030686419 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579015aa40 con 0x7f5790159eb0 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578ffff700 1 --2- 172.21.15.3:0/4030686419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5790159eb0 0x7f579015da10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5788009f20 tx=0x7f5788009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9496230ac7572623 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578e7fc700 1 -- 172.21.15.3:0/4030686419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800e070 con 0x7f5790159eb0 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578e7fc700 1 -- 172.21.15.3:0/4030686419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800ba80 con 0x7f5790159eb0 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f578e7fc700 1 -- 172.21.15.3:0/4030686419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57880092e0 con 0x7f5790159eb0 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f5795a62700 1 -- 172.21.15.3:0/4030686419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5790159eb0 msgr2=0x7f579015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f5795a62700 1 --2- 172.21.15.3:0/4030686419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5790159eb0 0x7f579015da10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5788009f20 tx=0x7f5788009840 comp rx=0 tx=0).stop 2023-10-06T04:52:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.923+0000 7f5795a62700 1 -- 172.21.15.3:0/4030686419 shutdown_connections 2023-10-06T04:52:50.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.924+0000 7f5795a62700 1 --2- 172.21.15.3:0/4030686419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5790159eb0 0x7f579015da10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.924+0000 7f5795a62700 1 --2- 172.21.15.3:0/4030686419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900cfb80 0x7f57900af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.924+0000 7f5795a62700 1 -- 172.21.15.3:0/4030686419 >> 172.21.15.3:0/4030686419 conn(0x7f5790070310 msgr2=0x7f5790070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:50.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.924+0000 7f5795a62700 1 -- 172.21.15.3:0/4030686419 shutdown_connections 2023-10-06T04:52:50.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.924+0000 7f5795a62700 1 -- 172.21.15.3:0/4030686419 wait complete. 2023-10-06T04:52:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 Processor -- start 2023-10-06T04:52:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 -- start start 2023-10-06T04:52:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900afad0 0x7f57900b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57900cfb80 0x7f579016ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5790159eb0 0x7f579016d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579016ac90 con 0x7f57900cfb80 2023-10-06T04:52:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579016a990 con 0x7f5790159eb0 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.925+0000 7f5795a62700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900b1740 con 0x7f57900afad0 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57900cfb80 0x7f579016ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57900cfb80 0x7f579016ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48206/0 (socket says 172.21.15.3:48206) 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 -- 172.21.15.3:0/490433377 learned_addr learned my addr 172.21.15.3:0/490433377 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578ffff700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5790159eb0 0x7f579016d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578f7fe700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900afad0 0x7f57900b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 -- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900afad0 msgr2=0x7f57900b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900afad0 0x7f57900b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 -- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5790159eb0 msgr2=0x7f579016d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5790159eb0 0x7f579016d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 -- 172.21.15.3:0/490433377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788009ae0 con 0x7f57900cfb80 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578ffff700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5790159eb0 0x7f579016d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f578effd700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57900cfb80 0x7f579016ce80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f578400c5c0 tx=0x7f578400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f57777fe700 1 -- 172.21.15.3:0/490433377 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784011590 con 0x7f57900cfb80 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f57777fe700 1 -- 172.21.15.3:0/490433377 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5784011710 con 0x7f57900cfb80 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57900b1a20 con 0x7f57900cfb80 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f57777fe700 1 -- 172.21.15.3:0/490433377 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784011ad0 con 0x7f57900cfb80 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.926+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57900b1380 con 0x7f57900cfb80 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.927+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579016ae40 con 0x7f57900cfb80 2023-10-06T04:52:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.928+0000 7f57777fe700 1 -- 172.21.15.3:0/490433377 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f578401b3b0 con 0x7f57900cfb80 2023-10-06T04:52:50.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.928+0000 7f57777fe700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f577c066f30 0x7f577c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:50.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.928+0000 7f57777fe700 1 -- 172.21.15.3:0/490433377 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5784083c70 con 0x7f57900cfb80 2023-10-06T04:52:50.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.928+0000 7f578f7fe700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f577c066f30 0x7f577c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:50.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.929+0000 7f578f7fe700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f577c066f30 0x7f577c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5780006fd0 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:50.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:50.932+0000 7f57777fe700 1 -- 172.21.15.3:0/490433377 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f579016ae40 con 0x7f57900cfb80 2023-10-06T04:52:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.126+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f579016ae40 con 0x7f57900cfb80 2023-10-06T04:52:51.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.130+0000 7f57777fe700 1 -- 172.21.15.3:0/490433377 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f579016ae40 con 0x7f57900cfb80 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f577c066f30 msgr2=0x7f577c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f577c066f30 0x7f577c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5780006fd0 tx=0x7f5780008040 comp rx=0 tx=0).stop 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57900cfb80 msgr2=0x7f579016ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57900cfb80 0x7f579016ce80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f578400c5c0 tx=0x7f578400c980 comp rx=0 tx=0).stop 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 shutdown_connections 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f577c066f30 0x7f577c0693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5790159eb0 0x7f579016d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57900afad0 0x7f57900b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 --2- 172.21.15.3:0/490433377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57900cfb80 0x7f579016ce80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 >> 172.21.15.3:0/490433377 conn(0x7f5790070310 msgr2=0x7f5790158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 shutdown_connections 2023-10-06T04:52:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.132+0000 7f5795a62700 1 -- 172.21.15.3:0/490433377 wait complete. 2023-10-06T04:52:51.147 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:52:51.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:52:51.185 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:52:51.230 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:52:51.230 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:52:51.230 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:52:51.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:52:51.266 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:52:51.266 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:52:51.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:51.266 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:52:51.321 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:52:51.322 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:52:51.322 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:52:51.357 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:52:51.357 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:52:51.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:51.358 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:52:51.417 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:52:51.417 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:52:51.417 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-06T04:52:51.453 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:52:51.454 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:52:51.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:51.454 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:52:51.509 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:52:51.509 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:52:51.509 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-06T04:52:51.545 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:52:51.546 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:52:51.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:52:51.546 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:52:51.601 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:52:51.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.718+0000 7f048e727700 1 Processor -- start 2023-10-06T04:52:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.718+0000 7f048e727700 1 -- start start 2023-10-06T04:52:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048e727700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0488157160 0x7f048815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048e727700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04880ce020 0x7f04880ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048e727700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048815abc0 con 0x7f04880c8630 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048e727700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04880cea00 con 0x7f0488157160 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048e727700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048815e140 con 0x7f04880ce020 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048cf24700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f04880c8630 0x7f048815a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53936/0 (socket says 172.21.15.3:53936) 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048cf24700 1 -- 172.21.15.3:0/3735363902 learned_addr learned my addr 172.21.15.3:0/3735363902 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048d725700 1 --2- 172.21.15.3:0/3735363902 >> v2:172.21.15.153:3300/0 conn(0x7f0488157160 0x7f048815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f048df26700 1 --2- 172.21.15.3:0/3735363902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04880ce020 0x7f04880ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f0487fff700 1 -- 172.21.15.3:0/3735363902 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250134212 0 0) 0x7f047c001110 con 0x7f04880c8630 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.719+0000 7f0487fff700 1 -- 172.21.15.3:0/3735363902 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0470003570 con 0x7f04880c8630 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f048df26700 1 -- 172.21.15.3:0/3735363902 >> v2:172.21.15.153:3300/0 conn(0x7f0488157160 msgr2=0x7f048815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f048df26700 1 --2- 172.21.15.3:0/3735363902 >> v2:172.21.15.153:3300/0 conn(0x7f0488157160 0x7f048815dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f048df26700 1 -- 172.21.15.3:0/3735363902 >> v1:172.21.15.3:6789/0 conn(0x7f04880c8630 legacy=0x7f048815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f0487fff700 1 -- 172.21.15.3:0/3735363902 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109129670 0 0) 0x7f0470003570 con 0x7f04880c8630 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f048df26700 1 -- 172.21.15.3:0/3735363902 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048815d1b0 con 0x7f04880ce020 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f048d725700 1 --2- 172.21.15.3:0/3735363902 >> v2:172.21.15.153:3300/0 conn(0x7f0488157160 0x7f048815dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f048df26700 1 --2- 172.21.15.3:0/3735363902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04880ce020 0x7f04880ce4c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0480009e60 tx=0x7f0480009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899619bc94913ea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f0487fff700 1 -- 172.21.15.3:0/3735363902 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000e070 con 0x7f04880ce020 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f0487fff700 1 -- 172.21.15.3:0/3735363902 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048000b7a0 con 0x7f04880ce020 2023-10-06T04:52:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.720+0000 7f0487fff700 1 -- 172.21.15.3:0/3735363902 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c001600 con 0x7f04880ce020 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 -- 172.21.15.3:0/3735363902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04880ce020 msgr2=0x7f04880ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 --2- 172.21.15.3:0/3735363902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04880ce020 0x7f04880ce4c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0480009e60 tx=0x7f0480009780 comp rx=0 tx=0).stop 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 -- 172.21.15.3:0/3735363902 shutdown_connections 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 --2- 172.21.15.3:0/3735363902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04880ce020 0x7f04880ce4c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 --2- 172.21.15.3:0/3735363902 >> v2:172.21.15.153:3300/0 conn(0x7f0488157160 0x7f048815dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 -- 172.21.15.3:0/3735363902 >> 172.21.15.3:0/3735363902 conn(0x7f0488070310 msgr2=0x7f0488070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 -- 172.21.15.3:0/3735363902 shutdown_connections 2023-10-06T04:52:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.721+0000 7f048e727700 1 -- 172.21.15.3:0/3735363902 wait complete. 2023-10-06T04:52:51.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.722+0000 7f048e727700 1 Processor -- start 2023-10-06T04:52:51.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.722+0000 7f048e727700 1 -- start start 2023-10-06T04:52:51.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048e727700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04880c8630 0x7f04880c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:51.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048e727700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04880ce020 0x7f048815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:51.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048e727700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0488157160 0x7f04880c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048e727700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048815c780 con 0x7f04880ce020 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048e727700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048815bc90 con 0x7f04880c8630 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048e727700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048816afb0 con 0x7f0488157160 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04880ce020 0x7f048815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04880ce020 0x7f048815b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48220/0 (socket says 172.21.15.3:48220) 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 -- 172.21.15.3:0/804765152 learned_addr learned my addr 172.21.15.3:0/804765152 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048d725700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04880c8630 0x7f04880c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048df26700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0488157160 0x7f04880c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 -- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0488157160 msgr2=0x7f04880c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0488157160 0x7f04880c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 -- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04880c8630 msgr2=0x7f04880c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04880c8630 0x7f04880c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 -- 172.21.15.3:0/804765152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478009420 con 0x7f04880ce020 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048d725700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04880c8630 0x7f04880c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048df26700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0488157160 0x7f04880c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048cf24700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04880ce020 0x7f048815b750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c930 tx=0x7f047c001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f04857fa700 1 -- 172.21.15.3:0/804765152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ce70 con 0x7f04880ce020 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f04857fa700 1 -- 172.21.15.3:0/804765152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c005b30 con 0x7f04880ce020 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f04857fa700 1 -- 172.21.15.3:0/804765152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0070f0 con 0x7f04880ce020 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.723+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0480009a20 con 0x7f04880ce020 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.724+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048815bf90 con 0x7f04880ce020 2023-10-06T04:52:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.724+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048816b170 con 0x7f04880ce020 2023-10-06T04:52:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.725+0000 7f04857fa700 1 -- 172.21.15.3:0/804765152 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f047c007460 con 0x7f04880ce020 2023-10-06T04:52:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.725+0000 7f04857fa700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0474066f30 0x7f04740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.725+0000 7f04857fa700 1 -- 172.21.15.3:0/804765152 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f047c083a80 con 0x7f04880ce020 2023-10-06T04:52:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.726+0000 7f048d725700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0474066f30 0x7f04740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.726+0000 7f048d725700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0474066f30 0x7f04740693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f04780093f0 tx=0x7f0478008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.729+0000 7f04857fa700 1 -- 172.21.15.3:0/804765152 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f047c04b5a0 con 0x7f04880ce020 2023-10-06T04:52:51.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.967+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f048815abc0 con 0x7f04880ce020 2023-10-06T04:52:51.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.968+0000 7f04857fa700 1 -- 172.21.15.3:0/804765152 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7f048815abc0 con 0x7f04880ce020 2023-10-06T04:52:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:51.971 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:51.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0474066f30 msgr2=0x7f04740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0474066f30 0x7f04740693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f04780093f0 tx=0x7f0478008040 comp rx=0 tx=0).stop 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04880ce020 msgr2=0x7f048815b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04880ce020 0x7f048815b750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c930 tx=0x7f047c001d70 comp rx=0 tx=0).stop 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 shutdown_connections 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0474066f30 0x7f04740693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04880c8630 0x7f04880c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0488157160 0x7f04880c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 --2- 172.21.15.3:0/804765152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04880ce020 0x7f048815b750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:51.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 >> 172.21.15.3:0/804765152 conn(0x7f0488070310 msgr2=0x7f04880cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:51.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 shutdown_connections 2023-10-06T04:52:51.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:51.970+0000 7f048e727700 1 -- 172.21.15.3:0/804765152 wait complete. 2023-10-06T04:52:51.974 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:51.984 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:52.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.100+0000 7f6fb5985700 1 Processor -- start 2023-10-06T04:52:52.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.100+0000 7f6fb5985700 1 -- start start 2023-10-06T04:52:52.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6fb5985700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6fb00cb3b0 0x7f6fb00cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6fb5985700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb00cf130 con 0x7f6fb00cb780 2023-10-06T04:52:52.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6fb5985700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb00ce520 con 0x7f6fb01596d0 2023-10-06T04:52:52.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6fb5985700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb00ce800 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6faffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6fb00cb780 0x7f6fb0158fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53940/0 (socket says 172.21.15.3:53940) 2023-10-06T04:52:52.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6faffff700 1 -- 172.21.15.3:0/3128753276 learned_addr learned my addr 172.21.15.3:0/3128753276 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:52.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6fb4983700 1 --2- 172.21.15.3:0/3128753276 >> v2:172.21.15.153:3300/0 conn(0x7f6fb00cb3b0 0x7f6fb00cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6faf7fe700 1 -- 172.21.15.3:0/3128753276 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047662876 0 0) 0x7f6fb00cf130 con 0x7f6fb00cb780 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6faf7fe700 1 -- 172.21.15.3:0/3128753276 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa0003570 con 0x7f6fb00cb780 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6faf7fe700 1 -- 172.21.15.3:0/3128753276 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77702220 0 0) 0x7f6fb00ce520 con 0x7f6fb01596d0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6faf7fe700 1 -- 172.21.15.3:0/3128753276 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb00cf130 con 0x7f6fb01596d0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.101+0000 7f6fb4983700 1 -- 172.21.15.3:0/3128753276 >> v1:172.21.15.153:6789/0 conn(0x7f6fb01596d0 legacy=0x7f6fb00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb4983700 1 -- 172.21.15.3:0/3128753276 >> v1:172.21.15.3:6789/0 conn(0x7f6fb00cb780 legacy=0x7f6fb0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb4983700 1 -- 172.21.15.3:0/3128753276 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb00ceae0 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb4983700 1 --2- 172.21.15.3:0/3128753276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb00cf5c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40098e0 tx=0x7f6fa40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b0329d1ad5b0b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6faf7fe700 1 -- 172.21.15.3:0/3128753276 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4011630 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6faf7fe700 1 -- 172.21.15.3:0/3128753276 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4011790 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6faf7fe700 1 -- 172.21.15.3:0/3128753276 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4011b30 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb5985700 1 -- 172.21.15.3:0/3128753276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb3b0 msgr2=0x7f6fb00cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3128753276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb00cf5c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40098e0 tx=0x7f6fa40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb5985700 1 -- 172.21.15.3:0/3128753276 shutdown_connections 2023-10-06T04:52:52.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3128753276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb00cf5c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.102+0000 7f6fb5985700 1 -- 172.21.15.3:0/3128753276 >> 172.21.15.3:0/3128753276 conn(0x7f6fb0070310 msgr2=0x7f6fb0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:52.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.103+0000 7f6fb5985700 1 -- 172.21.15.3:0/3128753276 shutdown_connections 2023-10-06T04:52:52.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.103+0000 7f6fb5985700 1 -- 172.21.15.3:0/3128753276 wait complete. 2023-10-06T04:52:52.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 Processor -- start 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 -- start start 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb0154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb780 0x7f6fb01536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6fb01596d0 0x7f6fb00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb015c700 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb015c3d0 con 0x7f6fb00cb780 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.104+0000 7f6fb5985700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0153dd0 con 0x7f6fb01596d0 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb0154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb0154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48222/0 (socket says 172.21.15.3:48222) 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 -- 172.21.15.3:0/3866528201 learned_addr learned my addr 172.21.15.3:0/3866528201 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb5184700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6fb01596d0 0x7f6fb00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6faffff700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb780 0x7f6fb01536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 -- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6fb01596d0 msgr2=0x7f6fb00c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6fb01596d0 0x7f6fb00c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 -- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb780 msgr2=0x7f6fb01536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb780 0x7f6fb01536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 -- 172.21.15.3:0/3866528201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4009420 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6faffff700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb780 0x7f6fb01536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb4983700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb0154c10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00893d0 tx=0x7f6fa4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6facff9700 1 -- 172.21.15.3:0/3866528201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa401f070 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6facff9700 1 -- 172.21.15.3:0/3866528201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4011900 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb01540b0 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6facff9700 1 -- 172.21.15.3:0/3866528201 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4023410 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.105+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb00c9490 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.106+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb01545d0 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.107+0000 7f6facff9700 1 -- 172.21.15.3:0/3866528201 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6fa402c3b0 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.107+0000 7f6facff9700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f94066f30 0x7f6f940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.107+0000 7f6facff9700 1 -- 172.21.15.3:0/3866528201 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6fa4085930 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.107+0000 7f6faffff700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f94066f30 0x7f6f940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.108+0000 7f6faffff700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f94066f30 0x7f6f940693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fb016d110 tx=0x7f6f9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.111+0000 7f6facff9700 1 -- 172.21.15.3:0/3866528201 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fb01545d0 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.306+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fb015c510 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.306+0000 7f6facff9700 1 -- 172.21.15.3:0/3866528201 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f6fb015c510 con 0x7f6fb00cb3b0 2023-10-06T04:52:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:52.309 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:52.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f94066f30 msgr2=0x7f6f940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f94066f30 0x7f6f940693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fb016d110 tx=0x7f6f9c009040 comp rx=0 tx=0).stop 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fb00cb3b0 msgr2=0x7f6fb0154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb0154c10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00893d0 tx=0x7f6fa4017000 comp rx=0 tx=0).stop 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 shutdown_connections 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f94066f30 0x7f6f940693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6fb01596d0 0x7f6fb00c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fb00cb3b0 0x7f6fb0154c10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 --2- 172.21.15.3:0/3866528201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb00cb780 0x7f6fb01536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 >> 172.21.15.3:0/3866528201 conn(0x7f6fb0070310 msgr2=0x7f6fb01581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 shutdown_connections 2023-10-06T04:52:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.309+0000 7f6fb5985700 1 -- 172.21.15.3:0/3866528201 wait complete. 2023-10-06T04:52:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:52:15.735846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:52:17.015278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:52:52.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:52:52.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.443+0000 7face3444700 1 Processor -- start 2023-10-06T04:52:52.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.443+0000 7face3444700 1 -- start start 2023-10-06T04:52:52.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face3444700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face3444700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7facdc159eb0 0x7facdc15da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face3444700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc15d510 con 0x7facdc0afad0 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face3444700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc15a480 con 0x7facdc159eb0 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face3444700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc15a760 con 0x7facdc0cfb80 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face09df700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7facdc0afad0 0x7facdc0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54358/0 (socket says 172.21.15.3:54358) 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face09df700 1 -- 172.21.15.3:0/125672842 learned_addr learned my addr 172.21.15.3:0/125672842 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face11e0700 1 --2- 172.21.15.3:0/125672842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.444+0000 7face19e1700 1 --2- 172.21.15.3:0/125672842 >> v2:172.21.15.153:3300/0 conn(0x7facdc159eb0 0x7facdc15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7facdbfff700 1 -- 172.21.15.3:0/125672842 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408692977 0 0) 0x7facdc15d510 con 0x7facdc0afad0 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7facdbfff700 1 -- 172.21.15.3:0/125672842 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc4003570 con 0x7facdc0afad0 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7face11e0700 1 -- 172.21.15.3:0/125672842 >> v2:172.21.15.153:3300/0 conn(0x7facdc159eb0 msgr2=0x7facdc15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7face11e0700 1 --2- 172.21.15.3:0/125672842 >> v2:172.21.15.153:3300/0 conn(0x7facdc159eb0 0x7facdc15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7face11e0700 1 -- 172.21.15.3:0/125672842 >> v1:172.21.15.3:6789/0 conn(0x7facdc0afad0 legacy=0x7facdc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7face11e0700 1 -- 172.21.15.3:0/125672842 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc15aa40 con 0x7facdc0cfb80 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7face19e1700 1 --2- 172.21.15.3:0/125672842 >> v2:172.21.15.153:3300/0 conn(0x7facdc159eb0 0x7facdc15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7face11e0700 1 --2- 172.21.15.3:0/125672842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0af590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7facd40098e0 tx=0x7facd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c487ea2a5fd0be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7facdbfff700 1 -- 172.21.15.3:0/125672842 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4011630 con 0x7facdc0cfb80 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7facdbfff700 1 -- 172.21.15.3:0/125672842 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd4011790 con 0x7facdc0cfb80 2023-10-06T04:52:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.445+0000 7facdbfff700 1 -- 172.21.15.3:0/125672842 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4011b10 con 0x7facdc0cfb80 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 -- 172.21.15.3:0/125672842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 msgr2=0x7facdc0af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 --2- 172.21.15.3:0/125672842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0af590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7facd40098e0 tx=0x7facd40091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 -- 172.21.15.3:0/125672842 shutdown_connections 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 --2- 172.21.15.3:0/125672842 >> v2:172.21.15.153:3300/0 conn(0x7facdc159eb0 0x7facdc15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 --2- 172.21.15.3:0/125672842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0af590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 -- 172.21.15.3:0/125672842 >> 172.21.15.3:0/125672842 conn(0x7facdc070310 msgr2=0x7facdc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 -- 172.21.15.3:0/125672842 shutdown_connections 2023-10-06T04:52:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.446+0000 7face3444700 1 -- 172.21.15.3:0/125672842 wait complete. 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.447+0000 7face3444700 1 Processor -- start 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.447+0000 7face3444700 1 -- start start 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face3444700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facdc0afad0 0x7facdc153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face3444700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face3444700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facdc159eb0 0x7facdc0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face3444700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0b1d60 con 0x7facdc159eb0 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face3444700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0b1150 con 0x7facdc0afad0 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face3444700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0b1430 con 0x7facdc0cfb80 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facdc159eb0 0x7facdc0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facdc159eb0 0x7facdc0b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57280/0 (socket says 172.21.15.3:57280) 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 -- 172.21.15.3:0/3730057917 learned_addr learned my addr 172.21.15.3:0/3730057917 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face11e0700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facdc0afad0 0x7facdc153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face09df700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 -- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 msgr2=0x7facdc0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 -- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facdc0afad0 msgr2=0x7facdc153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facdc0afad0 0x7facdc153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.448+0000 7face19e1700 1 -- 172.21.15.3:0/3730057917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc009a20 con 0x7facdc159eb0 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.449+0000 7face19e1700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facdc159eb0 0x7facdc0b0890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faccc009f20 tx=0x7faccc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.449+0000 7facd97fa700 1 -- 172.21.15.3:0/3730057917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc00cb70 con 0x7facdc159eb0 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.449+0000 7facd97fa700 1 -- 172.21.15.3:0/3730057917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc00f040 con 0x7facdc159eb0 2023-10-06T04:52:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.449+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd4009420 con 0x7facdc159eb0 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.449+0000 7facd97fa700 1 -- 172.21.15.3:0/3730057917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc014480 con 0x7facdc159eb0 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.449+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc0b19f0 con 0x7facdc159eb0 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.450+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc04c380 con 0x7facdc159eb0 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.450+0000 7facd97fa700 1 -- 172.21.15.3:0/3730057917 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faccc003020 con 0x7facdc159eb0 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.451+0000 7facd97fa700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7facc8066f30 0x7facc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.451+0000 7facd97fa700 1 -- 172.21.15.3:0/3730057917 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faccc084ef0 con 0x7facdc159eb0 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.451+0000 7face11e0700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7facc8066f30 0x7facc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.451+0000 7face11e0700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7facc8066f30 0x7facc80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facd40091e0 tx=0x7facd4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.455+0000 7facd97fa700 1 -- 172.21.15.3:0/3730057917 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faccc052000 con 0x7facdc159eb0 2023-10-06T04:52:52.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.647+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7facdc0b1ea0 con 0x7facdc159eb0 2023-10-06T04:52:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.647+0000 7facd97fa700 1 -- 172.21.15.3:0/3730057917 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7faccc018020 con 0x7facdc159eb0 2023-10-06T04:52:52.650 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:52:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7facc8066f30 msgr2=0x7facc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7facc8066f30 0x7facc80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facd40091e0 tx=0x7facd4017000 comp rx=0 tx=0).stop 2023-10-06T04:52:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facdc159eb0 msgr2=0x7facdc0b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facdc159eb0 0x7facdc0b0890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faccc009f20 tx=0x7faccc00b040 comp rx=0 tx=0).stop 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 shutdown_connections 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7facc8066f30 0x7facc80693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facdc0afad0 0x7facdc153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facdc159eb0 0x7facdc0b0890 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 --2- 172.21.15.3:0/3730057917 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7facdc0cfb80 0x7facdc0b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 >> 172.21.15.3:0/3730057917 conn(0x7facdc070310 msgr2=0x7facdc0c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.649+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 shutdown_connections 2023-10-06T04:52:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.650+0000 7face3444700 1 -- 172.21.15.3:0/3730057917 wait complete. 2023-10-06T04:52:52.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:52:52.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.783+0000 7fc567f26700 1 Processor -- start 2023-10-06T04:52:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.783+0000 7fc567f26700 1 -- start start 2023-10-06T04:52:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc567f26700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc5600af8a0 0x7fc560150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc567f26700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56015a290 con 0x7fc560150690 2023-10-06T04:52:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc567f26700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5600d0020 con 0x7fc5600cba50 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc567f26700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc560159f40 con 0x7fc5600af8a0 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc5664c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc5600af8a0 0x7fc560150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc565cc2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc560150690 0x7fc56015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54366/0 (socket says 172.21.15.3:54366) 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc565cc2700 1 -- 172.21.15.3:0/488978303 learned_addr learned my addr 172.21.15.3:0/488978303 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc5664c3700 1 -- 172.21.15.3:0/488978303 >> v1:172.21.15.153:6789/0 conn(0x7fc5600cba50 legacy=0x7fc5600af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc5664c3700 1 -- 172.21.15.3:0/488978303 >> v1:172.21.15.3:6789/0 conn(0x7fc560150690 legacy=0x7fc56015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.784+0000 7fc5664c3700 1 -- 172.21.15.3:0/488978303 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56015dcb0 con 0x7fc5600af8a0 2023-10-06T04:52:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.785+0000 7fc5664c3700 1 --2- 172.21.15.3:0/488978303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5600af8a0 0x7fc560150a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc558009e60 tx=0x7fc558009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78776847ece592d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc564cc0700 1 -- 172.21.15.3:0/488978303 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55800e070 con 0x7fc5600af8a0 2023-10-06T04:52:52.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc564cc0700 1 -- 172.21.15.3:0/488978303 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55800bc30 con 0x7fc5600af8a0 2023-10-06T04:52:52.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc564cc0700 1 -- 172.21.15.3:0/488978303 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558012430 con 0x7fc5600af8a0 2023-10-06T04:52:52.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc567f26700 1 -- 172.21.15.3:0/488978303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5600af8a0 msgr2=0x7fc560150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc567f26700 1 --2- 172.21.15.3:0/488978303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5600af8a0 0x7fc560150a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc558009e60 tx=0x7fc558009780 comp rx=0 tx=0).stop 2023-10-06T04:52:52.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc567f26700 1 -- 172.21.15.3:0/488978303 shutdown_connections 2023-10-06T04:52:52.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc567f26700 1 --2- 172.21.15.3:0/488978303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5600af8a0 0x7fc560150a60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.786+0000 7fc567f26700 1 -- 172.21.15.3:0/488978303 >> 172.21.15.3:0/488978303 conn(0x7fc560070310 msgr2=0x7fc560070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:52.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.787+0000 7fc567f26700 1 -- 172.21.15.3:0/488978303 shutdown_connections 2023-10-06T04:52:52.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.787+0000 7fc567f26700 1 -- 172.21.15.3:0/488978303 wait complete. 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 Processor -- start 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 -- start start 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5600af8a0 0x7fc56016a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5600cba50 0x7fc5601538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc560150690 0x7fc56015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56014ea40 con 0x7fc560150690 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56015cfa0 con 0x7fc5600cba50 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.788+0000 7fc567f26700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56015a720 con 0x7fc5600af8a0 2023-10-06T04:52:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc560150690 0x7fc56015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc560150690 0x7fc56015c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57300/0 (socket says 172.21.15.3:57300) 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 -- 172.21.15.3:0/3497482851 learned_addr learned my addr 172.21.15.3:0/3497482851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc565cc2700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5600af8a0 0x7fc56016a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5654c1700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5600cba50 0x7fc5601538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 -- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5600af8a0 msgr2=0x7fc56016a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5600af8a0 0x7fc56016a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 -- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5600cba50 msgr2=0x7fc5601538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5600cba50 0x7fc5601538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 -- 172.21.15.3:0/3497482851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc558009a20 con 0x7fc560150690 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc5664c3700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc560150690 0x7fc56015c3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc558013000 tx=0x7fc558019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc555ffb700 1 -- 172.21.15.3:0/3497482851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558027470 con 0x7fc560150690 2023-10-06T04:52:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc555ffb700 1 -- 172.21.15.3:0/3497482851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5580275f0 con 0x7fc560150690 2023-10-06T04:52:52.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56015c900 con 0x7fc560150690 2023-10-06T04:52:52.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc555ffb700 1 -- 172.21.15.3:0/3497482851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580279b0 con 0x7fc560150690 2023-10-06T04:52:52.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.789+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56015d4d0 con 0x7fc560150690 2023-10-06T04:52:52.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.790+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56004c380 con 0x7fc560150690 2023-10-06T04:52:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.791+0000 7fc555ffb700 1 -- 172.21.15.3:0/3497482851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc558024070 con 0x7fc560150690 2023-10-06T04:52:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.792+0000 7fc555ffb700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc544066f30 0x7fc5440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.792+0000 7fc555ffb700 1 -- 172.21.15.3:0/3497482851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc558087410 con 0x7fc560150690 2023-10-06T04:52:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.792+0000 7fc565cc2700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc544066f30 0x7fc5440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.792+0000 7fc565cc2700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc544066f30 0x7fc5440693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc560071020 tx=0x7fc55c000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:52.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.795+0000 7fc555ffb700 1 -- 172.21.15.3:0/3497482851 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc558054650 con 0x7fc560150690 2023-10-06T04:52:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.987+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc56015df90 con 0x7fc560150690 2023-10-06T04:52:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.987+0000 7fc555ffb700 1 -- 172.21.15.3:0/3497482851 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc558057c70 con 0x7fc560150690 2023-10-06T04:52:52.990 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:52:52.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc544066f30 msgr2=0x7fc5440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc544066f30 0x7fc5440693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc560071020 tx=0x7fc55c000bc0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc560150690 msgr2=0x7fc56015c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:52.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc560150690 0x7fc56015c3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc558013000 tx=0x7fc558019000 comp rx=0 tx=0).stop 2023-10-06T04:52:52.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 shutdown_connections 2023-10-06T04:52:52.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc544066f30 0x7fc5440693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc560150690 0x7fc56015c3c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5600af8a0 0x7fc56016a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 --2- 172.21.15.3:0/3497482851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5600cba50 0x7fc5601538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 >> 172.21.15.3:0/3497482851 conn(0x7fc560070310 msgr2=0x7fc5600b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 shutdown_connections 2023-10-06T04:52:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:52.989+0000 7fc567f26700 1 -- 172.21.15.3:0/3497482851 wait complete. 2023-10-06T04:52:53.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:52:53.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.124+0000 7f705e9ad700 1 Processor -- start 2023-10-06T04:52:53.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.124+0000 7f705e9ad700 1 -- start start 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.124+0000 7f705e9ad700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7058150690 0x7f7058150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.124+0000 7f705e9ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70580cba50 0x7f705815d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.124+0000 7f705e9ad700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705815da80 con 0x7f705815a720 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.124+0000 7f705e9ad700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705815a290 con 0x7f70580cba50 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.124+0000 7f705e9ad700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70580d0020 con 0x7f7058150690 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70580cba50 0x7f705815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70580cba50 0x7f705815d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57310/0 (socket says 172.21.15.3:57310) 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 -- 172.21.15.3:0/1082025370 learned_addr learned my addr 172.21.15.3:0/1082025370 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f7057fff700 1 --2- 172.21.15.3:0/1082025370 >> v2:172.21.15.153:3300/0 conn(0x7f7058150690 0x7f7058150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 -- 172.21.15.3:0/1082025370 >> v2:172.21.15.153:3300/0 conn(0x7f7058150690 msgr2=0x7f7058150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 --2- 172.21.15.3:0/1082025370 >> v2:172.21.15.153:3300/0 conn(0x7f7058150690 0x7f7058150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 -- 172.21.15.3:0/1082025370 >> v1:172.21.15.153:6789/0 conn(0x7f705815a720 legacy=0x7f70580af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 -- 172.21.15.3:0/1082025370 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7058159f40 con 0x7f70580cba50 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f7057fff700 1 --2- 172.21.15.3:0/1082025370 >> v2:172.21.15.153:3300/0 conn(0x7f7058150690 0x7f7058150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f70577fe700 1 --2- 172.21.15.3:0/1082025370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705815d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70580cdf80 tx=0x7f704c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4769625d91f9f49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f7056ffd700 1 -- 172.21.15.3:0/1082025370 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00b820 con 0x7f70580cba50 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f7056ffd700 1 -- 172.21.15.3:0/1082025370 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c00b980 con 0x7f70580cba50 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.125+0000 7f7056ffd700 1 -- 172.21.15.3:0/1082025370 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00bd20 con 0x7f70580cba50 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 -- 172.21.15.3:0/1082025370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 msgr2=0x7f705815d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 --2- 172.21.15.3:0/1082025370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705815d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70580cdf80 tx=0x7f704c009250 comp rx=0 tx=0).stop 2023-10-06T04:52:53.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 -- 172.21.15.3:0/1082025370 shutdown_connections 2023-10-06T04:52:53.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 --2- 172.21.15.3:0/1082025370 >> v2:172.21.15.153:3300/0 conn(0x7f7058150690 0x7f7058150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 --2- 172.21.15.3:0/1082025370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705815d540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 -- 172.21.15.3:0/1082025370 >> 172.21.15.3:0/1082025370 conn(0x7f7058070310 msgr2=0x7f7058070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:53.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 -- 172.21.15.3:0/1082025370 shutdown_connections 2023-10-06T04:52:53.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.126+0000 7f705e9ad700 1 -- 172.21.15.3:0/1082025370 wait complete. 2023-10-06T04:52:53.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.127+0000 7f705e9ad700 1 Processor -- start 2023-10-06T04:52:53.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.127+0000 7f705e9ad700 1 -- start start 2023-10-06T04:52:53.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f705e9ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705814fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f705e9ad700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7058150690 0x7f7058154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f705e9ad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f705815a720 0x7f7058156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f705e9ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70580b4ea0 con 0x7f70580cba50 2023-10-06T04:52:53.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f705e9ad700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70580b4290 con 0x7f7058150690 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f705e9ad700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70580b4570 con 0x7f705815a720 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705814fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705814fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57322/0 (socket says 172.21.15.3:57322) 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 -- 172.21.15.3:0/4269133513 learned_addr learned my addr 172.21.15.3:0/4269133513 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 -- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f705815a720 msgr2=0x7f7058156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f705cf4a700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f705815a720 0x7f7058156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f70577fe700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7058150690 0x7f7058154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f705815a720 0x7f7058156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 -- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7058150690 msgr2=0x7f7058154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7058150690 0x7f7058154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 -- 172.21.15.3:0/4269133513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0094f0 con 0x7f70580cba50 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f7057fff700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705814fe30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f704800c830 tx=0x7f7048008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.128+0000 7f703ffff700 1 -- 172.21.15.3:0/4269133513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048011600 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.129+0000 7f703ffff700 1 -- 172.21.15.3:0/4269133513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7048011760 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.129+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70580b4850 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.129+0000 7f703ffff700 1 -- 172.21.15.3:0/4269133513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048011b20 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.129+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7058150370 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.129+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705804c380 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.130+0000 7f703ffff700 1 -- 172.21.15.3:0/4269133513 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f704800a3e0 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.130+0000 7f703ffff700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7044066f30 0x7f70440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.130+0000 7f703ffff700 1 -- 172.21.15.3:0/4269133513 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7048084ba0 con 0x7f70580cba50 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.131+0000 7f70577fe700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7044066f30 0x7f70440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.131+0000 7f70577fe700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7044066f30 0x7f70440693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70580cdf80 tx=0x7f704c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.135+0000 7f703ffff700 1 -- 172.21.15.3:0/4269133513 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7048051cb0 con 0x7f70580cba50 2023-10-06T04:52:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.326+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f705815dcb0 con 0x7f70580cba50 2023-10-06T04:52:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.326+0000 7f703ffff700 1 -- 172.21.15.3:0/4269133513 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7048015020 con 0x7f70580cba50 2023-10-06T04:52:53.329 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:52:53.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7044066f30 msgr2=0x7f70440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7044066f30 0x7f70440693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70580cdf80 tx=0x7f704c016000 comp rx=0 tx=0).stop 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 msgr2=0x7f705814fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705814fe30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f704800c830 tx=0x7f7048008040 comp rx=0 tx=0).stop 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 shutdown_connections 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7044066f30 0x7f70440693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f705815a720 0x7f7058156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580cba50 0x7f705814fe30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 --2- 172.21.15.3:0/4269133513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7058150690 0x7f7058154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 >> 172.21.15.3:0/4269133513 conn(0x7f7058070310 msgr2=0x7f7058157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 shutdown_connections 2023-10-06T04:52:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.328+0000 7f705e9ad700 1 -- 172.21.15.3:0/4269133513 wait complete. 2023-10-06T04:52:53.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:52:53.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.463+0000 7fac063a0700 1 Processor -- start 2023-10-06T04:52:53.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.463+0000 7fac063a0700 1 -- start start 2023-10-06T04:52:53.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.464+0000 7fac063a0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac001595c0 0x7fac00067ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.464+0000 7fac063a0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0015dfe0 con 0x7fac001591f0 2023-10-06T04:52:53.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.464+0000 7fac063a0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac000a9fc0 con 0x7fac000a9850 2023-10-06T04:52:53.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.464+0000 7fac063a0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac000aa2a0 con 0x7fac001595c0 2023-10-06T04:52:53.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.464+0000 7fac04b9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac001595c0 0x7fac00067ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.464+0000 7fac0539e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fac001591f0 0x7fac0015d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54380/0 (socket says 172.21.15.3:54380) 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.464+0000 7fac0539e700 1 -- 172.21.15.3:0/896623418 learned_addr learned my addr 172.21.15.3:0/896623418 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fac04b9d700 1 -- 172.21.15.3:0/896623418 >> v1:172.21.15.153:6789/0 conn(0x7fac000a9850 legacy=0x7fac000ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fac04b9d700 1 -- 172.21.15.3:0/896623418 >> v1:172.21.15.3:6789/0 conn(0x7fac001591f0 legacy=0x7fac0015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fabf7fff700 1 -- 172.21.15.3:0/896623418 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20953992 0 0) 0x7fac0015dfe0 con 0x7fac001591f0 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fac04b9d700 1 -- 172.21.15.3:0/896623418 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac000aa580 con 0x7fac001595c0 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fac04b9d700 1 --2- 172.21.15.3:0/896623418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001595c0 0x7fac00067ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fabf00098e0 tx=0x7fabf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c175262ae00cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fabf7fff700 1 -- 172.21.15.3:0/896623418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0009e80 con 0x7fac001595c0 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fabf7fff700 1 -- 172.21.15.3:0/896623418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf000b860 con 0x7fac001595c0 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.465+0000 7fabf7fff700 1 -- 172.21.15.3:0/896623418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf000bbe0 con 0x7fac001595c0 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.466+0000 7fac063a0700 1 -- 172.21.15.3:0/896623418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001595c0 msgr2=0x7fac00067ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.466+0000 7fac063a0700 1 --2- 172.21.15.3:0/896623418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001595c0 0x7fac00067ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fabf00098e0 tx=0x7fabf00091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.466+0000 7fac063a0700 1 -- 172.21.15.3:0/896623418 shutdown_connections 2023-10-06T04:52:53.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.466+0000 7fac063a0700 1 --2- 172.21.15.3:0/896623418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001595c0 0x7fac00067ad0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.466+0000 7fac063a0700 1 -- 172.21.15.3:0/896623418 >> 172.21.15.3:0/896623418 conn(0x7fac0006d480 msgr2=0x7fac0006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:53.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.466+0000 7fac063a0700 1 -- 172.21.15.3:0/896623418 shutdown_connections 2023-10-06T04:52:53.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.466+0000 7fac063a0700 1 -- 172.21.15.3:0/896623418 wait complete. 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.467+0000 7fac063a0700 1 Processor -- start 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.467+0000 7fac063a0700 1 -- start start 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac000a9850 0x7fac000cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001591f0 0x7fac00169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac001595c0 0x7fac0016a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0013f1b0 con 0x7fac001591f0 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000b2bf0 con 0x7fac001595c0 2023-10-06T04:52:53.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac000b1fe0 con 0x7fac000a9850 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001591f0 0x7fac00169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001591f0 0x7fac00169ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57340/0 (socket says 172.21.15.3:57340) 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 -- 172.21.15.3:0/192089704 learned_addr learned my addr 172.21.15.3:0/192089704 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac05b9f700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac001595c0 0x7fac0016a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac0539e700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac000a9850 0x7fac000cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 -- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac000a9850 msgr2=0x7fac000cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac000a9850 0x7fac000cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 -- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac001595c0 msgr2=0x7fac0016a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac001595c0 0x7fac0016a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 -- 172.21.15.3:0/192089704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0009420 con 0x7fac001591f0 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac05b9f700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac001595c0 0x7fac0016a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac04b9d700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001591f0 0x7fac00169ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac00070e30 tx=0x7fabf0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fabf57fa700 1 -- 172.21.15.3:0/192089704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf001f070 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fabf57fa700 1 -- 172.21.15.3:0/192089704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf0011420 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac000b22c0 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fabf57fa700 1 -- 172.21.15.3:0/192089704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf000bbe0 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.468+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac000b2880 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.469+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac00158b10 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.470+0000 7fabf57fa700 1 -- 172.21.15.3:0/192089704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabf0011a50 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.470+0000 7fabf57fa700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.470+0000 7fabf57fa700 1 -- 172.21.15.3:0/192089704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fabf0085bf0 con 0x7fac001591f0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.470+0000 7fac0539e700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.471+0000 7fac0539e700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabe4066f30 0x7fabe40693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0011e0 tx=0x7fabfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.474+0000 7fabf57fa700 1 -- 172.21.15.3:0/192089704 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabf0052d00 con 0x7fac001591f0 2023-10-06T04:52:53.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.665+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac0004c380 con 0x7fac001591f0 2023-10-06T04:52:53.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.666+0000 7fabf57fa700 1 -- 172.21.15.3:0/192089704 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabf00150a0 con 0x7fac001591f0 2023-10-06T04:52:53.668 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:52:53.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabe4066f30 msgr2=0x7fabe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabe4066f30 0x7fabe40693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0011e0 tx=0x7fabfc009040 comp rx=0 tx=0).stop 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001591f0 msgr2=0x7fac00169ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001591f0 0x7fac00169ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac00070e30 tx=0x7fabf0017000 comp rx=0 tx=0).stop 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 shutdown_connections 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac001595c0 0x7fac0016a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac000a9850 0x7fac000cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 --2- 172.21.15.3:0/192089704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac001591f0 0x7fac00169ae0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 >> 172.21.15.3:0/192089704 conn(0x7fac0006d480 msgr2=0x7fac000cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:53.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 shutdown_connections 2023-10-06T04:52:53.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.668+0000 7fac063a0700 1 -- 172.21.15.3:0/192089704 wait complete. 2023-10-06T04:52:53.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:53.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.799+0000 7fdee813b700 1 Processor -- start 2023-10-06T04:52:53.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee813b700 1 -- start start 2023-10-06T04:52:53.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee813b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0155980 0x7fdee006aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee813b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdee0156a30 0x7fdee006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee813b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdee0156ea0 0x7fdee006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee813b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00ae9e0 con 0x7fdee0156a30 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee813b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee0069790 con 0x7fdee0156ea0 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee813b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00699e0 con 0x7fdee0155980 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee56d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdee0156a30 0x7fdee006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee56d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdee0156a30 0x7fdee006e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57356/0 (socket says 172.21.15.3:57356) 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.800+0000 7fdee56d6700 1 -- 172.21.15.3:0/3446859159 learned_addr learned my addr 172.21.15.3:0/3446859159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee66d8700 1 --2- 172.21.15.3:0/3446859159 >> v2:172.21.15.153:3300/0 conn(0x7fdee0156ea0 0x7fdee006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee5ed7700 1 --2- 172.21.15.3:0/3446859159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0155980 0x7fdee006aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee56d6700 1 -- 172.21.15.3:0/3446859159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0155980 msgr2=0x7fdee006aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee56d6700 1 --2- 172.21.15.3:0/3446859159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0155980 0x7fdee006aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee56d6700 1 -- 172.21.15.3:0/3446859159 >> v2:172.21.15.153:3300/0 conn(0x7fdee0156ea0 msgr2=0x7fdee006ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee56d6700 1 --2- 172.21.15.3:0/3446859159 >> v2:172.21.15.153:3300/0 conn(0x7fdee0156ea0 0x7fdee006ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee56d6700 1 -- 172.21.15.3:0/3446859159 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0069cc0 con 0x7fdee0156a30 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee56d6700 1 --2- 172.21.15.3:0/3446859159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee006e530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdee00c8980 tx=0x7fded0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bab8c2c2873f4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee4ed5700 1 -- 172.21.15.3:0/3446859159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000b860 con 0x7fdee0156a30 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee4ed5700 1 -- 172.21.15.3:0/3446859159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded000b9c0 con 0x7fdee0156a30 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.801+0000 7fdee4ed5700 1 -- 172.21.15.3:0/3446859159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000bd60 con 0x7fdee0156a30 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 -- 172.21.15.3:0/3446859159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 msgr2=0x7fdee006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 --2- 172.21.15.3:0/3446859159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee006e530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdee00c8980 tx=0x7fded0009250 comp rx=0 tx=0).stop 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 -- 172.21.15.3:0/3446859159 shutdown_connections 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 --2- 172.21.15.3:0/3446859159 >> v2:172.21.15.153:3300/0 conn(0x7fdee0156ea0 0x7fdee006ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 --2- 172.21.15.3:0/3446859159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0155980 0x7fdee006aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 --2- 172.21.15.3:0/3446859159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee006e530 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 -- 172.21.15.3:0/3446859159 >> 172.21.15.3:0/3446859159 conn(0x7fdee006c370 msgr2=0x7fdee006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:53.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 -- 172.21.15.3:0/3446859159 shutdown_connections 2023-10-06T04:52:53.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.802+0000 7fdee813b700 1 -- 172.21.15.3:0/3446859159 wait complete. 2023-10-06T04:52:53.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.803+0000 7fdee813b700 1 Processor -- start 2023-10-06T04:52:53.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.803+0000 7fdee813b700 1 -- start start 2023-10-06T04:52:53.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee813b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdee0155980 0x7fdee00c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee813b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee00c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee813b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0156ea0 0x7fdee00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee813b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00cc930 con 0x7fdee0156a30 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee813b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b1120 con 0x7fdee0155980 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee813b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00c5ee0 con 0x7fdee0156ea0 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee00c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee00c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57372/0 (socket says 172.21.15.3:57372) 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 -- 172.21.15.3:0/279774817 learned_addr learned my addr 172.21.15.3:0/279774817 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee66d8700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0156ea0 0x7fdee00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee5ed7700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdee0155980 0x7fdee00c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 -- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0156ea0 msgr2=0x7fdee00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0156ea0 0x7fdee00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 -- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdee0155980 msgr2=0x7fdee00c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdee0155980 0x7fdee00c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 -- 172.21.15.3:0/279774817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00094f0 con 0x7fdee0156a30 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee56d6700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee00c9b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdee0068a00 tx=0x7fded0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fded5ffb700 1 -- 172.21.15.3:0/279774817 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0024440 con 0x7fdee0156a30 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fded5ffb700 1 -- 172.21.15.3:0/279774817 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded00245a0 con 0x7fdee0156a30 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.804+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00b0ad0 con 0x7fdee0156a30 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.805+0000 7fded5ffb700 1 -- 172.21.15.3:0/279774817 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0024960 con 0x7fdee0156a30 2023-10-06T04:52:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.805+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee00b07a0 con 0x7fdee0156a30 2023-10-06T04:52:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.806+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee014aa30 con 0x7fdee0156a30 2023-10-06T04:52:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.806+0000 7fded5ffb700 1 -- 172.21.15.3:0/279774817 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fded001a420 con 0x7fdee0156a30 2023-10-06T04:52:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.807+0000 7fded5ffb700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdec4066f30 0x7fdec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.807+0000 7fded5ffb700 1 -- 172.21.15.3:0/279774817 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fded00864b0 con 0x7fdee0156a30 2023-10-06T04:52:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.807+0000 7fdee5ed7700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdec4066f30 0x7fdec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.807+0000 7fdee5ed7700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdec4066f30 0x7fdec40693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdedc006fd0 tx=0x7fdedc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:53.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:53.811+0000 7fded5ffb700 1 -- 172.21.15.3:0/279774817 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdee014aa30 con 0x7fdee0156a30 2023-10-06T04:52:53.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:53.886 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.051+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee014aa30 con 0x7fdee0156a30 2023-10-06T04:52:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.052+0000 7fded5ffb700 1 -- 172.21.15.3:0/279774817 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7fdee014aa30 con 0x7fdee0156a30 2023-10-06T04:52:54.054 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:54.055 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdec4066f30 msgr2=0x7fdec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdec4066f30 0x7fdec40693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdedc006fd0 tx=0x7fdedc008040 comp rx=0 tx=0).stop 2023-10-06T04:52:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 msgr2=0x7fdee00c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee00c9b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdee0068a00 tx=0x7fded0017000 comp rx=0 tx=0).stop 2023-10-06T04:52:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 shutdown_connections 2023-10-06T04:52:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdec4066f30 0x7fdec40693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdee0155980 0x7fdee00c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdee0156ea0 0x7fdee00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 --2- 172.21.15.3:0/279774817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee0156a30 0x7fdee00c9b50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 >> 172.21.15.3:0/279774817 conn(0x7fdee006c370 msgr2=0x7fdee00ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 shutdown_connections 2023-10-06T04:52:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.054+0000 7fdee813b700 1 -- 172.21.15.3:0/279774817 wait complete. 2023-10-06T04:52:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:54.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.185+0000 7f5ca6299700 1 Processor -- start 2023-10-06T04:52:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.185+0000 7f5ca6299700 1 -- start start 2023-10-06T04:52:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca6299700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca015cb50 0x7f5ca00b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca6299700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ca00cd950 0x7f5ca00cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca6299700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca00b0ce0 con 0x7f5ca00b0590 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca6299700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca00cb820 con 0x7f5ca00cd950 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca6299700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00b0a00 con 0x7f5ca015cb50 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca4836700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ca00cd950 0x7f5ca00cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca4836700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ca00cd950 0x7f5ca00cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57374/0 (socket says 172.21.15.3:57374) 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.186+0000 7f5ca4836700 1 -- 172.21.15.3:0/3055979812 learned_addr learned my addr 172.21.15.3:0/3055979812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5c9ffff700 1 --2- 172.21.15.3:0/3055979812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca015cb50 0x7f5ca00b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca4836700 1 -- 172.21.15.3:0/3055979812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca015cb50 msgr2=0x7f5ca00b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca4836700 1 --2- 172.21.15.3:0/3055979812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca015cb50 0x7f5ca00b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca4836700 1 -- 172.21.15.3:0/3055979812 >> v1:172.21.15.153:6789/0 conn(0x7f5ca00b0590 legacy=0x7f5ca00b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca4836700 1 -- 172.21.15.3:0/3055979812 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca00cb1d0 con 0x7f5ca00cd950 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca4836700 1 --2- 172.21.15.3:0/3055979812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00cd950 0x7f5ca00cbcb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009f20 tx=0x7f5c98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cc7be7da83f283 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5c9effd700 1 -- 172.21.15.3:0/3055979812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800e070 con 0x7f5ca00cd950 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5c9effd700 1 -- 172.21.15.3:0/3055979812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9800ba40 con 0x7f5ca00cd950 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5c9effd700 1 -- 172.21.15.3:0/3055979812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c980092e0 con 0x7f5ca00cd950 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca6299700 1 -- 172.21.15.3:0/3055979812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00cd950 msgr2=0x7f5ca00cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca6299700 1 --2- 172.21.15.3:0/3055979812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00cd950 0x7f5ca00cbcb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009f20 tx=0x7f5c98009840 comp rx=0 tx=0).stop 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca6299700 1 -- 172.21.15.3:0/3055979812 shutdown_connections 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca6299700 1 --2- 172.21.15.3:0/3055979812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00cd950 0x7f5ca00cbcb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca6299700 1 --2- 172.21.15.3:0/3055979812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca015cb50 0x7f5ca00b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.187+0000 7f5ca6299700 1 -- 172.21.15.3:0/3055979812 >> 172.21.15.3:0/3055979812 conn(0x7f5ca0070310 msgr2=0x7f5ca0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.188+0000 7f5ca6299700 1 -- 172.21.15.3:0/3055979812 shutdown_connections 2023-10-06T04:52:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.188+0000 7f5ca6299700 1 -- 172.21.15.3:0/3055979812 wait complete. 2023-10-06T04:52:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.189+0000 7f5ca6299700 1 Processor -- start 2023-10-06T04:52:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.189+0000 7f5ca6299700 1 -- start start 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.189+0000 7f5ca6299700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00b0590 0x7f5ca0157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca6299700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca00cd950 0x7f5ca00c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca6299700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ca015cb50 0x7f5ca00c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca6299700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c52d0 con 0x7f5ca00b0590 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca6299700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c7220 con 0x7f5ca015cb50 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca6299700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0169940 con 0x7f5ca00cd950 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00b0590 0x7f5ca0157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00b0590 0x7f5ca0157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57382/0 (socket says 172.21.15.3:57382) 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 -- 172.21.15.3:0/2330975714 learned_addr learned my addr 172.21.15.3:0/2330975714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 -- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca00cd950 msgr2=0x7f5ca00c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca4836700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ca015cb50 0x7f5ca00c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9f7fe700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca00cd950 0x7f5ca00c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca00cd950 0x7f5ca00c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 -- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ca015cb50 msgr2=0x7f5ca00c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ca015cb50 0x7f5ca00c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 -- 172.21.15.3:0/2330975714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98009ae0 con 0x7f5ca00b0590 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca4836700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ca015cb50 0x7f5ca00c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9f7fe700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca00cd950 0x7f5ca00c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c9ffff700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00b0590 0x7f5ca0157d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d890 tx=0x7f5c90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c97fff700 1 -- 172.21.15.3:0/2330975714 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90018070 con 0x7f5ca00b0590 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c97fff700 1 -- 172.21.15.3:0/2330975714 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c90005a00 con 0x7f5ca00b0590 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca0169c20 con 0x7f5ca00b0590 2023-10-06T04:52:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5c97fff700 1 -- 172.21.15.3:0/2330975714 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90005dc0 con 0x7f5ca00b0590 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.190+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca0169fb0 con 0x7f5ca00b0590 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.191+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca016a420 con 0x7f5ca00b0590 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.192+0000 7f5c97fff700 1 -- 172.21.15.3:0/2330975714 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c90012800 con 0x7f5ca00b0590 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.192+0000 7f5c97fff700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c80066f30 0x7f5c800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.192+0000 7f5c97fff700 1 -- 172.21.15.3:0/2330975714 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5c900847c0 con 0x7f5ca00b0590 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.192+0000 7f5c9f7fe700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c80066f30 0x7f5c800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.193+0000 7f5c9f7fe700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c80066f30 0x7f5c800693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015b840 tx=0x7f5c88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.196+0000 7f5c97fff700 1 -- 172.21.15.3:0/2330975714 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ca016a420 con 0x7f5ca00b0590 2023-10-06T04:52:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.390+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ca0158890 con 0x7f5ca00b0590 2023-10-06T04:52:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.391+0000 7f5c97fff700 1 -- 172.21.15.3:0/2330975714 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f5c90055360 con 0x7f5ca00b0590 2023-10-06T04:52:54.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:54.394 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c80066f30 msgr2=0x7f5c800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c80066f30 0x7f5c800693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015b840 tx=0x7f5c88009040 comp rx=0 tx=0).stop 2023-10-06T04:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00b0590 msgr2=0x7f5ca0157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00b0590 0x7f5ca0157d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d890 tx=0x7f5c90008040 comp rx=0 tx=0).stop 2023-10-06T04:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 shutdown_connections 2023-10-06T04:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c80066f30 0x7f5c800693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ca015cb50 0x7f5ca00c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ca00b0590 0x7f5ca0157d90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 --2- 172.21.15.3:0/2330975714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ca00cd950 0x7f5ca00c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 >> 172.21.15.3:0/2330975714 conn(0x7f5ca0070310 msgr2=0x7f5ca00b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 shutdown_connections 2023-10-06T04:52:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.393+0000 7f5ca6299700 1 -- 172.21.15.3:0/2330975714 wait complete. 2023-10-06T04:52:54.408 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:52:54.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.524+0000 7fbb89d9e700 1 Processor -- start 2023-10-06T04:52:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.524+0000 7fbb89d9e700 1 -- start start 2023-10-06T04:52:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.525+0000 7fbb89d9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 0x7fbb840b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.525+0000 7fbb89d9e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb840b0ce0 con 0x7fbb840b0590 2023-10-06T04:52:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.525+0000 7fbb89d9e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb840cb820 con 0x7fbb840cd950 2023-10-06T04:52:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.525+0000 7fbb89d9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b0a00 con 0x7fbb8415cb50 2023-10-06T04:52:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.525+0000 7fbb82ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbb840b0590 0x7fbb840b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54390/0 (socket says 172.21.15.3:54390) 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.525+0000 7fbb82ffd700 1 -- 172.21.15.3:0/2065692626 learned_addr learned my addr 172.21.15.3:0/2065692626 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb837fe700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 0x7fbb840b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419448303 0 0) 0x7fbb840b0ce0 con 0x7fbb840b0590 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6c003570 con 0x7fbb840b0590 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561513798 0 0) 0x7fbb840cb820 con 0x7fbb840cd950 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb840b0ce0 con 0x7fbb840cd950 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894082799 0 0) 0x7fbb6c003570 con 0x7fbb840b0590 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 msgr2=0x7fbb840b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 0x7fbb840b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 >> v1:172.21.15.153:6789/0 conn(0x7fbb840cd950 legacy=0x7fbb840cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840cb1d0 con 0x7fbb840b0590 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fbb78001470 con 0x7fbb840b0590 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 >> v1:172.21.15.3:6789/0 conn(0x7fbb840b0590 legacy=0x7fbb840b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb6c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 0x7fbb6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c004d50 0x7fbb6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b0a00 con 0x7fbb6c003c20 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c00bbf0 con 0x7fbb6c004d50 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c00bd60 con 0x7fbb8415cb50 2023-10-06T04:52:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb837fe700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb6c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb82ffd700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 0x7fbb6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb83fff700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c004d50 0x7fbb6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb837fe700 1 -- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 msgr2=0x7fbb6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb837fe700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 0x7fbb6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb837fe700 1 -- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c004d50 msgr2=0x7fbb6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb837fe700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c004d50 0x7fbb6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb837fe700 1 -- 172.21.15.3:0/2065692626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c003570 con 0x7fbb6c003c20 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.526+0000 7fbb82ffd700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb8415cb50 0x7fbb6c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:54.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb837fe700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb6c004010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbb74006ea0 tx=0x7fbb74009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca5e8e2ae22e040 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74005a00 con 0x7fbb6c003c20 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb74005b60 con 0x7fbb6c003c20 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb827fc700 1 -- 172.21.15.3:0/2065692626 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400e640 con 0x7fbb6c003c20 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb89d9e700 1 -- 172.21.15.3:0/2065692626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 msgr2=0x7fbb6c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb89d9e700 1 --2- 172.21.15.3:0/2065692626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb6c004010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbb74006ea0 tx=0x7fbb74009120 comp rx=0 tx=0).stop 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb83fff700 1 -- 172.21.15.3:0/2065692626 reap_dead start 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb89d9e700 1 -- 172.21.15.3:0/2065692626 shutdown_connections 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb89d9e700 1 -- 172.21.15.3:0/2065692626 >> 172.21.15.3:0/2065692626 conn(0x7fbb84070310 msgr2=0x7fbb84070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb89d9e700 1 -- 172.21.15.3:0/2065692626 shutdown_connections 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.527+0000 7fbb89d9e700 1 -- 172.21.15.3:0/2065692626 wait complete. 2023-10-06T04:52:54.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 Processor -- start 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 -- start start 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb8416d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb840c9730 0x7fbb84153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb840c9ba0 0x7fbb84154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8416ddb0 con 0x7fbb6c003c20 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840c6520 con 0x7fbb840c9730 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb89d9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840ce100 con 0x7fbb840c9ba0 2023-10-06T04:52:54.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb837fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb8416d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb837fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb8416d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57392/0 (socket says 172.21.15.3:57392) 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb837fe700 1 -- 172.21.15.3:0/4108084098 learned_addr learned my addr 172.21.15.3:0/4108084098 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb82ffd700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb840c9730 0x7fbb84153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.529+0000 7fbb83fff700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb840c9ba0 0x7fbb84154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb837fe700 1 -- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb840c9ba0 msgr2=0x7fbb84154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb837fe700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb840c9ba0 0x7fbb84154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb837fe700 1 -- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb840c9730 msgr2=0x7fbb84153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb837fe700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb840c9730 0x7fbb84153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb837fe700 1 -- 172.21.15.3:0/4108084098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840ce3e0 con 0x7fbb6c003c20 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb82ffd700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb840c9730 0x7fbb84153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb837fe700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb8416d870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400e4b0 tx=0x7fbb74019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb6b7fe700 1 -- 172.21.15.3:0/4108084098 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74005a00 con 0x7fbb6c003c20 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb6b7fe700 1 -- 172.21.15.3:0/4108084098 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb74006240 con 0x7fbb6c003c20 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb840ce6c0 con 0x7fbb6c003c20 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb6b7fe700 1 -- 172.21.15.3:0/4108084098 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74006620 con 0x7fbb6c003c20 2023-10-06T04:52:54.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.530+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb840c61b0 con 0x7fbb6c003c20 2023-10-06T04:52:54.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.531+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb840cdf60 con 0x7fbb6c003c20 2023-10-06T04:52:54.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.531+0000 7fbb6b7fe700 1 -- 172.21.15.3:0/4108084098 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb74013470 con 0x7fbb6c003c20 2023-10-06T04:52:54.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.532+0000 7fbb6b7fe700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb70066f30 0x7fbb700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.532+0000 7fbb6b7fe700 1 -- 172.21.15.3:0/4108084098 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbb74085d50 con 0x7fbb6c003c20 2023-10-06T04:52:54.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.532+0000 7fbb82ffd700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb70066f30 0x7fbb700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.533+0000 7fbb82ffd700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb70066f30 0x7fbb700693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb780015a0 tx=0x7fbb78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.536+0000 7fbb6b7fe700 1 -- 172.21.15.3:0/4108084098 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb840cdf60 con 0x7fbb6c003c20 2023-10-06T04:52:54.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.775+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb840cdf60 con 0x7fbb6c003c20 2023-10-06T04:52:54.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.776+0000 7fbb6b7fe700 1 -- 172.21.15.3:0/4108084098 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7fbb840cdf60 con 0x7fbb6c003c20 2023-10-06T04:52:54.779 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:54.779 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:54.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb70066f30 msgr2=0x7fbb700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb70066f30 0x7fbb700693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb780015a0 tx=0x7fbb78009040 comp rx=0 tx=0).stop 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 msgr2=0x7fbb8416d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb8416d870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400e4b0 tx=0x7fbb74019000 comp rx=0 tx=0).stop 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 shutdown_connections 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb70066f30 0x7fbb700693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb840c9ba0 0x7fbb84154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6c003c20 0x7fbb8416d870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 --2- 172.21.15.3:0/4108084098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb840c9730 0x7fbb84153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 >> 172.21.15.3:0/4108084098 conn(0x7fbb84070310 msgr2=0x7fbb84154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 shutdown_connections 2023-10-06T04:52:54.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.778+0000 7fbb89d9e700 1 -- 172.21.15.3:0/4108084098 wait complete. 2023-10-06T04:52:54.782 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:54.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:52:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.908+0000 7f2b2628e700 1 Processor -- start 2023-10-06T04:52:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.909+0000 7f2b2628e700 1 -- start start 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.909+0000 7f2b2628e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b2015a260 0x7f2b2015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.909+0000 7f2b2628e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2015d7a0 con 0x7f2b200b29c0 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.909+0000 7f2b2628e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b200cc5d0 con 0x7f2b200b25f0 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.909+0000 7f2b2628e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2015a730 con 0x7f2b2015a260 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b2482b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2b200b29c0 0x7f2b200d06b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54398/0 (socket says 172.21.15.3:54398) 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b2482b700 1 -- 172.21.15.3:0/2959324908 learned_addr learned my addr 172.21.15.3:0/2959324908 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1f7fe700 1 --2- 172.21.15.3:0/2959324908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b2015a260 0x7f2b2015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1effd700 1 -- 172.21.15.3:0/2959324908 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143652321 0 0) 0x7f2b2015d7a0 con 0x7f2b200b29c0 2023-10-06T04:52:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1effd700 1 -- 172.21.15.3:0/2959324908 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b0c003570 con 0x7f2b200b29c0 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1effd700 1 -- 172.21.15.3:0/2959324908 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951016828 0 0) 0x7f2b200cc5d0 con 0x7f2b200b25f0 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1effd700 1 -- 172.21.15.3:0/2959324908 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b2015d7a0 con 0x7f2b200b25f0 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1f7fe700 1 -- 172.21.15.3:0/2959324908 >> v1:172.21.15.153:6789/0 conn(0x7f2b200b25f0 legacy=0x7f2b20159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1f7fe700 1 -- 172.21.15.3:0/2959324908 >> v1:172.21.15.3:6789/0 conn(0x7f2b200b29c0 legacy=0x7f2b200d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1f7fe700 1 -- 172.21.15.3:0/2959324908 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2015aa10 con 0x7f2b2015a260 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1f7fe700 1 --2- 172.21.15.3:0/2959324908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b2015a260 0x7f2b2015dc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009910 tx=0x7f2b080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d25ab2ab39dfce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1effd700 1 -- 172.21.15.3:0/2959324908 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08011630 con 0x7f2b2015a260 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1effd700 1 -- 172.21.15.3:0/2959324908 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b08011790 con 0x7f2b2015a260 2023-10-06T04:52:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.910+0000 7f2b1effd700 1 -- 172.21.15.3:0/2959324908 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08011b30 con 0x7f2b2015a260 2023-10-06T04:52:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.911+0000 7f2b2628e700 1 -- 172.21.15.3:0/2959324908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b2015a260 msgr2=0x7f2b2015dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.911+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2959324908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b2015a260 0x7f2b2015dc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009910 tx=0x7f2b080091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.911+0000 7f2b2628e700 1 -- 172.21.15.3:0/2959324908 shutdown_connections 2023-10-06T04:52:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.911+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2959324908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b2015a260 0x7f2b2015dc30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.911+0000 7f2b2628e700 1 -- 172.21.15.3:0/2959324908 >> 172.21.15.3:0/2959324908 conn(0x7f2b2006b3c0 msgr2=0x7f2b2006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.911+0000 7f2b2628e700 1 -- 172.21.15.3:0/2959324908 shutdown_connections 2023-10-06T04:52:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.911+0000 7f2b2628e700 1 -- 172.21.15.3:0/2959324908 wait complete. 2023-10-06T04:52:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.912+0000 7f2b2628e700 1 Processor -- start 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.912+0000 7f2b2628e700 1 -- start start 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b2628e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b200b25f0 0x7f2b201468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b2628e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b200b29c0 0x7f2b200c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b2628e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2015a260 0x7f2b200c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b2628e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200c83e0 con 0x7f2b200b25f0 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b2628e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200c86c0 con 0x7f2b2015a260 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b2628e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200c89a0 con 0x7f2b200b29c0 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b200b25f0 0x7f2b201468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b200b25f0 0x7f2b201468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57396/0 (socket says 172.21.15.3:57396) 2023-10-06T04:52:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 -- 172.21.15.3:0/2695193415 learned_addr learned my addr 172.21.15.3:0/2695193415 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1f7fe700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b200b29c0 0x7f2b200c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b2482b700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2015a260 0x7f2b200c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 -- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b200b29c0 msgr2=0x7f2b200c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b200b29c0 0x7f2b200c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 -- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2015a260 msgr2=0x7f2b200c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2015a260 0x7f2b200c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.913+0000 7f2b1ffff700 1 -- 172.21.15.3:0/2695193415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08009420 con 0x7f2b200b25f0 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.914+0000 7f2b1ffff700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b200b25f0 0x7f2b201468e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c790 tx=0x7f2b10007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.914+0000 7f2b17fff700 1 -- 172.21.15.3:0/2695193415 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10010510 con 0x7f2b200b25f0 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.914+0000 7f2b17fff700 1 -- 172.21.15.3:0/2695193415 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b10010670 con 0x7f2b200b25f0 2023-10-06T04:52:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.914+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b200c8c80 con 0x7f2b200b25f0 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.914+0000 7f2b17fff700 1 -- 172.21.15.3:0/2695193415 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10010a10 con 0x7f2b200b25f0 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.914+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b200b3190 con 0x7f2b200b25f0 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.915+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b20071680 con 0x7f2b200b25f0 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.915+0000 7f2b17fff700 1 -- 172.21.15.3:0/2695193415 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b1001b400 con 0x7f2b200b25f0 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.915+0000 7f2b17fff700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b00066f30 0x7f2b000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.916+0000 7f2b17fff700 1 -- 172.21.15.3:0/2695193415 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2b10084500 con 0x7f2b200b25f0 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.916+0000 7f2b1f7fe700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b00066f30 0x7f2b000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.916+0000 7f2b1f7fe700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b00066f30 0x7f2b000693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009860 tx=0x7f2b08017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:54.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:54.920+0000 7f2b17fff700 1 -- 172.21.15.3:0/2695193415 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b10051570 con 0x7f2b200b25f0 2023-10-06T04:52:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.155+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b2015d150 con 0x7f2b200b25f0 2023-10-06T04:52:55.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.155+0000 7f2b17fff700 1 -- 172.21.15.3:0/2695193415 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4585 (secure 0 0 0) 0x7f2b10054b90 con 0x7f2b200b25f0 2023-10-06T04:52:55.190 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:55.191 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/109190604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":109190604},{"type":"v1","addr":"172.21.15.153:6835","nonce":109190604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/903663498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":903663498},{"type":"v1","addr":"172.21.15.3:6834","nonce":903663498}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1192477517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1192477517},{"type":"v1","addr":"172.21.15.3:6835","nonce":1192477517}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:52:18.046513+0000","modified":"2023-10-06T04:52:21.661968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3245320636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3245320636},{"type":"v1","addr":"172.21.15.153:6837","nonce":3245320636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:52:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b00066f30 msgr2=0x7f2b000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b00066f30 0x7f2b000693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009860 tx=0x7f2b08017000 comp rx=0 tx=0).stop 2023-10-06T04:52:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b200b25f0 msgr2=0x7f2b201468e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b200b25f0 0x7f2b201468e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c790 tx=0x7f2b10007e10 comp rx=0 tx=0).stop 2023-10-06T04:52:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 shutdown_connections 2023-10-06T04:52:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b00066f30 0x7f2b000693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2015a260 0x7f2b200c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b200b25f0 0x7f2b201468e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.157+0000 7f2b2628e700 1 --2- 172.21.15.3:0/2695193415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b200b29c0 0x7f2b200c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.158+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 >> 172.21.15.3:0/2695193415 conn(0x7f2b2006b3c0 msgr2=0x7f2b20155880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.158+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 shutdown_connections 2023-10-06T04:52:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.158+0000 7f2b2628e700 1 -- 172.21.15.3:0/2695193415 wait complete. 2023-10-06T04:52:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-10-06T04:52:55.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:52:55.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.311+0000 7f3c3cd42700 1 Processor -- start 2023-10-06T04:52:55.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.311+0000 7f3c3cd42700 1 -- start start 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3cd42700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3c3815c8f0 0x7f3c3815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3cd42700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3c3815ccc0 0x7f3c380d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3cd42700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c380cf6c0 con 0x7f3c380d06a0 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3cd42700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c380cfb50 con 0x7f3c3815c8f0 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3cd42700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c380d0b20 con 0x7f3c3815ccc0 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3c3815c8f0 0x7f3c3815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c36d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3c380d06a0 0x7f3c3806d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54412/0 (socket says 172.21.15.3:54412) 2023-10-06T04:52:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c36d9d700 1 -- 172.21.15.3:0/1668045072 learned_addr learned my addr 172.21.15.3:0/1668045072 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c35d9b700 1 --2- 172.21.15.3:0/1668045072 >> v2:172.21.15.153:3300/0 conn(0x7f3c3815ccc0 0x7f3c380d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3659c700 1 -- 172.21.15.3:0/1668045072 >> v2:172.21.15.153:3300/0 conn(0x7f3c3815ccc0 msgr2=0x7f3c380d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3659c700 1 --2- 172.21.15.3:0/1668045072 >> v2:172.21.15.153:3300/0 conn(0x7f3c3815ccc0 0x7f3c380d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3659c700 1 -- 172.21.15.3:0/1668045072 >> v1:172.21.15.3:6789/0 conn(0x7f3c380d06a0 legacy=0x7f3c3806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3559a700 1 -- 172.21.15.3:0/1668045072 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521441447 0 0) 0x7f3c380cf6c0 con 0x7f3c380d06a0 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3659c700 1 -- 172.21.15.3:0/1668045072 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3815e050 con 0x7f3c3815c8f0 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.312+0000 7f3c3659c700 1 --2- 172.21.15.3:0/1668045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815c8f0 0x7f3c3815db10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009e60 tx=0x7f3c28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd67f5f792ee233a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3559a700 1 -- 172.21.15.3:0/1668045072 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800e070 con 0x7f3c3815c8f0 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3559a700 1 -- 172.21.15.3:0/1668045072 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2800bbf0 con 0x7f3c3815c8f0 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3559a700 1 -- 172.21.15.3:0/1668045072 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28012430 con 0x7f3c3815c8f0 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 -- 172.21.15.3:0/1668045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815c8f0 msgr2=0x7f3c3815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/1668045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815c8f0 0x7f3c3815db10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009e60 tx=0x7f3c28009780 comp rx=0 tx=0).stop 2023-10-06T04:52:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 -- 172.21.15.3:0/1668045072 shutdown_connections 2023-10-06T04:52:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/1668045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815c8f0 0x7f3c3815db10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/1668045072 >> v2:172.21.15.153:3300/0 conn(0x7f3c3815ccc0 0x7f3c380d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 -- 172.21.15.3:0/1668045072 >> 172.21.15.3:0/1668045072 conn(0x7f3c38070310 msgr2=0x7f3c38070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 -- 172.21.15.3:0/1668045072 shutdown_connections 2023-10-06T04:52:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.313+0000 7f3c3cd42700 1 -- 172.21.15.3:0/1668045072 wait complete. 2023-10-06T04:52:55.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.314+0000 7f3c3cd42700 1 Processor -- start 2023-10-06T04:52:55.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.314+0000 7f3c3cd42700 1 -- start start 2023-10-06T04:52:55.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c3cd42700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c380d06a0 0x7f3c3814c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c3cd42700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3815c8f0 0x7f3c3816d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c3cd42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815ccc0 0x7f3c380b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c3cd42700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3816de40 con 0x7f3c3815ccc0 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c3cd42700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3816a930 con 0x7f3c3815c8f0 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c3cd42700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38169db0 con 0x7f3c380d06a0 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815ccc0 0x7f3c380b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815ccc0 0x7f3c380b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57424/0 (socket says 172.21.15.3:57424) 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 -- 172.21.15.3:0/3812127392 learned_addr learned my addr 172.21.15.3:0/3812127392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c3659c700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c380d06a0 0x7f3c3814c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c35d9b700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3815c8f0 0x7f3c3816d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 -- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c380d06a0 msgr2=0x7f3c3814c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c380d06a0 0x7f3c3814c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 -- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3815c8f0 msgr2=0x7f3c3816d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3815c8f0 0x7f3c3816d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.315+0000 7f3c36d9d700 1 -- 172.21.15.3:0/3812127392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28009a20 con 0x7f3c3815ccc0 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.316+0000 7f3c36d9d700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815ccc0 0x7f3c380b4d20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000c530 tx=0x7f3c30007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.316+0000 7f3c2e7fc700 1 -- 172.21.15.3:0/3812127392 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30005b30 con 0x7f3c3815ccc0 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.316+0000 7f3c2e7fc700 1 -- 172.21.15.3:0/3812127392 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c30005cb0 con 0x7f3c3815ccc0 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.316+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3816a090 con 0x7f3c3815ccc0 2023-10-06T04:52:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.316+0000 7f3c2e7fc700 1 -- 172.21.15.3:0/3812127392 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30007320 con 0x7f3c3815ccc0 2023-10-06T04:52:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.316+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3816a650 con 0x7f3c3815ccc0 2023-10-06T04:52:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.317+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c38071680 con 0x7f3c3815ccc0 2023-10-06T04:52:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.317+0000 7f3c2e7fc700 1 -- 172.21.15.3:0/3812127392 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c3001b3b0 con 0x7f3c3815ccc0 2023-10-06T04:52:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.318+0000 7f3c2e7fc700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c18066f30 0x7f3c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.318+0000 7f3c2e7fc700 1 -- 172.21.15.3:0/3812127392 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3c30083980 con 0x7f3c3815ccc0 2023-10-06T04:52:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.318+0000 7f3c3659c700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c18066f30 0x7f3c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.318+0000 7f3c3659c700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c18066f30 0x7f3c180693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c280097c0 tx=0x7f3c28019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.322+0000 7f3c2e7fc700 1 -- 172.21.15.3:0/3812127392 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c30050a90 con 0x7f3c3815ccc0 2023-10-06T04:52:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.516+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c380cf060 con 0x7f3c3815ccc0 2023-10-06T04:52:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.517+0000 7f3c2e7fc700 1 -- 172.21.15.3:0/3812127392 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f3c30017020 con 0x7f3c3815ccc0 2023-10-06T04:52:55.519 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:52:55.520 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:52:19.632337+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:52:15.735846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:52:17.015278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:52:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c18066f30 msgr2=0x7f3c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c18066f30 0x7f3c180693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c280097c0 tx=0x7f3c28019000 comp rx=0 tx=0).stop 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815ccc0 msgr2=0x7f3c380b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815ccc0 0x7f3c380b4d20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000c530 tx=0x7f3c30007f90 comp rx=0 tx=0).stop 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 shutdown_connections 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c18066f30 0x7f3c180693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c3815ccc0 0x7f3c380b4d20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c380d06a0 0x7f3c3814c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 --2- 172.21.15.3:0/3812127392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3815c8f0 0x7f3c3816d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 >> 172.21.15.3:0/3812127392 conn(0x7f3c38070310 msgr2=0x7f3c380b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 shutdown_connections 2023-10-06T04:52:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.519+0000 7f3c3cd42700 1 -- 172.21.15.3:0/3812127392 wait complete. 2023-10-06T04:52:55.535 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:52:55.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.652+0000 7f931d91e700 1 Processor -- start 2023-10-06T04:52:55.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.652+0000 7f931d91e700 1 -- start start 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.653+0000 7f931d91e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f931815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.653+0000 7f931d91e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93180cf6c0 con 0x7f931815ccc0 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.653+0000 7f931d91e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93180cfb50 con 0x7f93180d0870 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.653+0000 7f931d91e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931815e050 con 0x7f931815c8f0 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.653+0000 7f93167fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f931815ccc0 0x7f93180d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54424/0 (socket says 172.21.15.3:54424) 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.653+0000 7f93167fc700 1 -- 172.21.15.3:0/2470340810 learned_addr learned my addr 172.21.15.3:0/2470340810 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.653+0000 7f9316ffd700 1 --2- 172.21.15.3:0/2470340810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f931815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9315ffb700 1 -- 172.21.15.3:0/2470340810 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156130949 0 0) 0x7f93180cf6c0 con 0x7f931815ccc0 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9315ffb700 1 -- 172.21.15.3:0/2470340810 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9304003570 con 0x7f931815ccc0 2023-10-06T04:52:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9315ffb700 1 -- 172.21.15.3:0/2470340810 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028122695 0 0) 0x7f93180cfb50 con 0x7f93180d0870 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9315ffb700 1 -- 172.21.15.3:0/2470340810 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93180cf6c0 con 0x7f93180d0870 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9316ffd700 1 -- 172.21.15.3:0/2470340810 >> v1:172.21.15.153:6789/0 conn(0x7f93180d0870 legacy=0x7f93180b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9316ffd700 1 -- 172.21.15.3:0/2470340810 >> v1:172.21.15.3:6789/0 conn(0x7f931815ccc0 legacy=0x7f93180d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9316ffd700 1 -- 172.21.15.3:0/2470340810 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180ceab0 con 0x7f931815c8f0 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9316ffd700 1 --2- 172.21.15.3:0/2470340810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f931815db10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9310009e90 tx=0x7f9310009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45a56dd1c939d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9315ffb700 1 -- 172.21.15.3:0/2470340810 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000e070 con 0x7f931815c8f0 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9315ffb700 1 -- 172.21.15.3:0/2470340810 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931000bb80 con 0x7f931815c8f0 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.654+0000 7f9315ffb700 1 -- 172.21.15.3:0/2470340810 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310012430 con 0x7f931815c8f0 2023-10-06T04:52:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.655+0000 7f931d91e700 1 -- 172.21.15.3:0/2470340810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 msgr2=0x7f931815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.655+0000 7f931d91e700 1 --2- 172.21.15.3:0/2470340810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f931815db10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9310009e90 tx=0x7f9310009780 comp rx=0 tx=0).stop 2023-10-06T04:52:55.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.655+0000 7f931d91e700 1 -- 172.21.15.3:0/2470340810 shutdown_connections 2023-10-06T04:52:55.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.655+0000 7f931d91e700 1 --2- 172.21.15.3:0/2470340810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f931815db10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.655+0000 7f931d91e700 1 -- 172.21.15.3:0/2470340810 >> 172.21.15.3:0/2470340810 conn(0x7f9318070310 msgr2=0x7f9318070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:55.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.655+0000 7f931d91e700 1 -- 172.21.15.3:0/2470340810 shutdown_connections 2023-10-06T04:52:55.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.655+0000 7f931d91e700 1 -- 172.21.15.3:0/2470340810 wait complete. 2023-10-06T04:52:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.656+0000 7f931d91e700 1 Processor -- start 2023-10-06T04:52:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.656+0000 7f931d91e700 1 -- start start 2023-10-06T04:52:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f931d91e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93180d0870 0x7f9318149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f931d91e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f93180c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f931d91e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f931815ccc0 0x7f93180c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f931d91e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180c5760 con 0x7f93180d0870 2023-10-06T04:52:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f931d91e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180c62e0 con 0x7f931815ccc0 2023-10-06T04:52:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f931d91e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931814a510 con 0x7f931815c8f0 2023-10-06T04:52:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93180d0870 0x7f9318149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93180d0870 0x7f9318149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57430/0 (socket says 172.21.15.3:57430) 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 -- 172.21.15.3:0/3605993276 learned_addr learned my addr 172.21.15.3:0/3605993276 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f93167fc700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f93180c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f93177fe700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f931815ccc0 0x7f93180c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 -- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 msgr2=0x7f93180c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f93180c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 -- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f931815ccc0 msgr2=0x7f93180c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f931815ccc0 0x7f93180c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 -- 172.21.15.3:0/3605993276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310009a20 con 0x7f93180d0870 2023-10-06T04:52:55.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f9316ffd700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93180d0870 0x7f9318149fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f93100097c0 tx=0x7f9310019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f930effd700 1 -- 172.21.15.3:0/3605993276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310012e30 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f930effd700 1 -- 172.21.15.3:0/3605993276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310026480 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180c59b0 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.657+0000 7f930effd700 1 -- 172.21.15.3:0/3605993276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310026840 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.658+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180c5f70 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.658+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9318071680 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.659+0000 7f930effd700 1 -- 172.21.15.3:0/3605993276 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f931001c400 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.659+0000 7f930effd700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92f8066f30 0x7f92f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.659+0000 7f930effd700 1 -- 172.21.15.3:0/3605993276 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9310086420 con 0x7f93180d0870 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.659+0000 7f93167fc700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92f8066f30 0x7f92f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:55.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.660+0000 7f93167fc700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92f8066f30 0x7f92f80693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9318157730 tx=0x7f9300009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:55.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.664+0000 7f930effd700 1 -- 172.21.15.3:0/3605993276 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9310056fc0 con 0x7f93180d0870 2023-10-06T04:52:55.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.903+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f93180b35a0 con 0x7f93180d0870 2023-10-06T04:52:55.907 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T04:52:55.904+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:52:55.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.907+0000 7f930effd700 1 -- 172.21.15.3:0/3605993276 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9310056b50 con 0x7f93180d0870 2023-10-06T04:52:55.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.909+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92f8066f30 msgr2=0x7f92f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.909+0000 7f931d91e700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92f8066f30 0x7f92f80693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9318157730 tx=0x7f9300009040 comp rx=0 tx=0).stop 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.909+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93180d0870 msgr2=0x7f9318149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.909+0000 7f931d91e700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93180d0870 0x7f9318149fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f93100097c0 tx=0x7f9310019000 comp rx=0 tx=0).stop 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 shutdown_connections 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f92f8066f30 0x7f92f80693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f931815ccc0 0x7f93180c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93180d0870 0x7f9318149fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 --2- 172.21.15.3:0/3605993276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f931815c8f0 0x7f93180c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 >> 172.21.15.3:0/3605993276 conn(0x7f9318070310 msgr2=0x7f9318155d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 shutdown_connections 2023-10-06T04:52:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:55.910+0000 7f931d91e700 1 -- 172.21.15.3:0/3605993276 wait complete. 2023-10-06T04:52:55.915 INFO:tasks.ceph.mds.d.smithi153.stderr: -2185> 2023-10-06T04:52:22.646+0000 7ff5c55c9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:52:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:52:55.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:52:55.937 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:52:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.043+0000 7f22c1469700 1 Processor -- start 2023-10-06T04:52:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.043+0000 7f22c1469700 1 -- start start 2023-10-06T04:52:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22c1469700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 0x7f22bc150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22c1469700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc15a290 con 0x7f22bc0cba50 2023-10-06T04:52:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22c1469700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0d0020 con 0x7f22bc150690 2023-10-06T04:52:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22c1469700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc159f40 con 0x7f22bc0af8a0 2023-10-06T04:52:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22ba7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f22bc0cba50 0x7f22bc0af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54436/0 (socket says 172.21.15.3:54436) 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22ba7fc700 1 -- 172.21.15.3:0/2923290096 learned_addr learned my addr 172.21.15.3:0/2923290096 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22bb7fe700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 0x7f22bc150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286028028 0 0) 0x7f22bc15a290 con 0x7f22bc0cba50 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a4003570 con 0x7f22bc0cba50 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098699242 0 0) 0x7f22bc0d0020 con 0x7f22bc150690 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22bc15a290 con 0x7f22bc150690 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639465479 0 0) 0x7f22a4003570 con 0x7f22bc0cba50 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 msgr2=0x7f22bc150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 0x7f22bc150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 >> v1:172.21.15.153:6789/0 conn(0x7f22bc150690 legacy=0x7f22bc15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc15dcb0 con 0x7f22bc0cba50 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f22b0001470 con 0x7f22bc0cba50 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 >> v1:172.21.15.3:6789/0 conn(0x7f22bc0cba50 legacy=0x7f22bc0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4003c20 0x7f22a4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a4004810 0x7f22a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.044+0000 7f22b9ffb700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 0x7f22a4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc159f40 con 0x7f22a4004810 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a400bbf0 con 0x7f22a4003c20 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a400bdb0 con 0x7f22bc0af8a0 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22ba7fc700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a4004810 0x7f22a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22baffd700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4003c20 0x7f22a4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22bb7fe700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 0x7f22a4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22ba7fc700 1 -- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 msgr2=0x7f22a4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22ba7fc700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22bc0af8a0 0x7f22a4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22ba7fc700 1 -- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4003c20 msgr2=0x7f22a4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22ba7fc700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22a4003c20 0x7f22a4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22ba7fc700 1 -- 172.21.15.3:0/2923290096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4003570 con 0x7f22a4004810 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22ba7fc700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a4004810 0x7f22a4004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f22b000c210 tx=0x7f22b0001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfa9d9aff19355d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000cc10 con 0x7f22a4004810 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b000cd70 con 0x7f22a4004810 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.045+0000 7f22b9ffb700 1 -- 172.21.15.3:0/2923290096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0005b70 con 0x7f22a4004810 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.046+0000 7f22c1469700 1 -- 172.21.15.3:0/2923290096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a4004810 msgr2=0x7f22a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.046+0000 7f22c1469700 1 --2- 172.21.15.3:0/2923290096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a4004810 0x7f22a4004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f22b000c210 tx=0x7f22b0001d40 comp rx=0 tx=0).stop 2023-10-06T04:52:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.046+0000 7f22bb7fe700 1 -- 172.21.15.3:0/2923290096 reap_dead start 2023-10-06T04:52:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.046+0000 7f22c1469700 1 -- 172.21.15.3:0/2923290096 shutdown_connections 2023-10-06T04:52:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.046+0000 7f22c1469700 1 -- 172.21.15.3:0/2923290096 >> 172.21.15.3:0/2923290096 conn(0x7f22bc070310 msgr2=0x7f22bc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.046+0000 7f22c1469700 1 -- 172.21.15.3:0/2923290096 shutdown_connections 2023-10-06T04:52:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.046+0000 7f22c1469700 1 -- 172.21.15.3:0/2923290096 wait complete. 2023-10-06T04:52:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.047+0000 7f22c1469700 1 Processor -- start 2023-10-06T04:52:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.047+0000 7f22c1469700 1 -- start start 2023-10-06T04:52:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22c1469700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22a4004810 0x7f22bc14df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22c1469700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22bc0b5910 0x7f22bc0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22c1469700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22bc0b5d80 0x7f22bc0c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22c1469700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0b62d0 con 0x7f22bc0b5d80 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22c1469700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c8990 con 0x7f22bc0b5910 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22c1469700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c8060 con 0x7f22a4004810 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22bc0b5d80 0x7f22bc0c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22bc0b5d80 0x7f22bc0c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57440/0 (socket says 172.21.15.3:57440) 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 -- 172.21.15.3:0/1172463954 learned_addr learned my addr 172.21.15.3:0/1172463954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22baffd700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22a4004810 0x7f22bc14df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22ba7fc700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22bc0b5910 0x7f22bc0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 -- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22a4004810 msgr2=0x7f22bc14df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22a4004810 0x7f22bc14df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 -- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22bc0b5910 msgr2=0x7f22bc0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22bc0b5910 0x7f22bc0c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 -- 172.21.15.3:0/1172463954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0c8340 con 0x7f22bc0b5d80 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.048+0000 7f22bb7fe700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22bc0b5d80 0x7f22bc0c5720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f22b40094d0 tx=0x7f22b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.049+0000 7f22a2ffd700 1 -- 172.21.15.3:0/1172463954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b400e040 con 0x7f22bc0b5d80 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.049+0000 7f22a2ffd700 1 -- 172.21.15.3:0/1172463954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b400f040 con 0x7f22bc0b5d80 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.049+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc0c8620 con 0x7f22bc0b5d80 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.049+0000 7f22a2ffd700 1 -- 172.21.15.3:0/1172463954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b400fd70 con 0x7f22bc0b5d80 2023-10-06T04:52:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.049+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc0b3720 con 0x7f22bc0b5d80 2023-10-06T04:52:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.050+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc04c380 con 0x7f22bc0b5d80 2023-10-06T04:52:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.050+0000 7f22a2ffd700 1 -- 172.21.15.3:0/1172463954 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22b4015640 con 0x7f22bc0b5d80 2023-10-06T04:52:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.050+0000 7f22a2ffd700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.051+0000 7f22a2ffd700 1 -- 172.21.15.3:0/1172463954 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f22b4084e40 con 0x7f22bc0b5d80 2023-10-06T04:52:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.051+0000 7f22baffd700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.051+0000 7f22baffd700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f22a8066f30 0x7f22a80693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0010e0 tx=0x7f22ac007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.055+0000 7f22a2ffd700 1 -- 172.21.15.3:0/1172463954 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22b4051e80 con 0x7f22bc0b5d80 2023-10-06T04:52:56.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.292+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f22bc15df90 con 0x7f22bc0b5d80 2023-10-06T04:52:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.911+0000 7f22a2ffd700 1 -- 172.21.15.3:0/1172463954 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f22b40554a0 con 0x7f22bc0b5d80 2023-10-06T04:52:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f22a8066f30 msgr2=0x7f22a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f22a8066f30 0x7f22a80693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0010e0 tx=0x7f22ac007d30 comp rx=0 tx=0).stop 2023-10-06T04:52:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22bc0b5d80 msgr2=0x7f22bc0c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22bc0b5d80 0x7f22bc0c5720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f22b40094d0 tx=0x7f22b400b040 comp rx=0 tx=0).stop 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 shutdown_connections 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22bc0b5d80 0x7f22bc0c5720 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22a4004810 0x7f22bc14df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 --2- 172.21.15.3:0/1172463954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22bc0b5910 0x7f22bc0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 >> 172.21.15.3:0/1172463954 conn(0x7f22bc070310 msgr2=0x7f22bc14fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 shutdown_connections 2023-10-06T04:52:56.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:56.913+0000 7f22c1469700 1 -- 172.21.15.3:0/1172463954 wait complete. 2023-10-06T04:52:56.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:52:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.058+0000 7f282759e700 1 Processor -- start 2023-10-06T04:52:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.058+0000 7f282759e700 1 -- start start 2023-10-06T04:52:57.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f282759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f282809d670 0x7f282809da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:57.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f282759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28280cf6a0 0x7f282815dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f282759e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2828159b70 con 0x7f28280b4a20 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f282759e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28280cfb10 con 0x7f28280cf6a0 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f282759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282815d8b0 con 0x7f282809d670 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f2825d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28280cf6a0 0x7f282815dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f2826d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f28280b4a20 0x7f28280ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54440/0 (socket says 172.21.15.3:54440) 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f2826d9d700 1 -- 172.21.15.3:0/114976423 learned_addr learned my addr 172.21.15.3:0/114976423 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.059+0000 7f282659c700 1 --2- 172.21.15.3:0/114976423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f282809d670 0x7f282809da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f2825d9b700 1 -- 172.21.15.3:0/114976423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f282809d670 msgr2=0x7f282809da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f2825d9b700 1 --2- 172.21.15.3:0/114976423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f282809d670 0x7f282809da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f2825d9b700 1 -- 172.21.15.3:0/114976423 >> v1:172.21.15.3:6789/0 conn(0x7f28280b4a20 legacy=0x7f28280ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f2825d9b700 1 -- 172.21.15.3:0/114976423 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282815d2d0 con 0x7f28280cf6a0 2023-10-06T04:52:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f282559a700 1 -- 172.21.15.3:0/114976423 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830892109 0 0) 0x7f2828159b70 con 0x7f28280b4a20 2023-10-06T04:52:57.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f2825d9b700 1 --2- 172.21.15.3:0/114976423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f282815dd40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f28100098e0 tx=0x7f28100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5293e0a58e66fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:57.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f282559a700 1 -- 172.21.15.3:0/114976423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000b640 con 0x7f28280cf6a0 2023-10-06T04:52:57.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f282559a700 1 -- 172.21.15.3:0/114976423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281000b7a0 con 0x7f28280cf6a0 2023-10-06T04:52:57.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f282559a700 1 -- 172.21.15.3:0/114976423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000bb40 con 0x7f28280cf6a0 2023-10-06T04:52:57.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f282759e700 1 -- 172.21.15.3:0/114976423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 msgr2=0x7f282815dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.060+0000 7f282759e700 1 --2- 172.21.15.3:0/114976423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f282815dd40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f28100098e0 tx=0x7f28100091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.061+0000 7f282759e700 1 -- 172.21.15.3:0/114976423 shutdown_connections 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.061+0000 7f282759e700 1 --2- 172.21.15.3:0/114976423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f282809d670 0x7f282809da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.061+0000 7f282759e700 1 --2- 172.21.15.3:0/114976423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f282815dd40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.061+0000 7f282759e700 1 -- 172.21.15.3:0/114976423 >> 172.21.15.3:0/114976423 conn(0x7f2828070310 msgr2=0x7f2828070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.061+0000 7f282759e700 1 -- 172.21.15.3:0/114976423 shutdown_connections 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.061+0000 7f282759e700 1 -- 172.21.15.3:0/114976423 wait complete. 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 Processor -- start 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 -- start start 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f282809d670 0x7f282816a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28280b4a20 0x7f28280b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:57.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f2828150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28280b1de0 con 0x7f28280cf6a0 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282816d900 con 0x7f282809d670 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.062+0000 7f282759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282816d010 con 0x7f28280b4a20 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f2828150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f2828150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57458/0 (socket says 172.21.15.3:57458) 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 -- 172.21.15.3:0/789674456 learned_addr learned my addr 172.21.15.3:0/789674456 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f282659c700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f282809d670 0x7f282816a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2825d9b700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28280b4a20 0x7f28280b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 -- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28280b4a20 msgr2=0x7f28280b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28280b4a20 0x7f28280b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 -- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f282809d670 msgr2=0x7f282816a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f282809d670 0x7f282816a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 -- 172.21.15.3:0/789674456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810009420 con 0x7f28280cf6a0 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f282659c700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f282809d670 0x7f282816a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f2826d9d700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f2828150ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f281800cc90 tx=0x7f2818001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f281e7fc700 1 -- 172.21.15.3:0/789674456 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281800cea0 con 0x7f28280cf6a0 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f281e7fc700 1 -- 172.21.15.3:0/789674456 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281800ab70 con 0x7f28280cf6a0 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282816d2b0 con 0x7f28280cf6a0 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f281e7fc700 1 -- 172.21.15.3:0/789674456 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180070f0 con 0x7f28280cf6a0 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.063+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28280b2010 con 0x7f28280cf6a0 2023-10-06T04:52:57.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.064+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282816da90 con 0x7f28280cf6a0 2023-10-06T04:52:57.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.064+0000 7f281e7fc700 1 -- 172.21.15.3:0/789674456 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28180074a0 con 0x7f28280cf6a0 2023-10-06T04:52:57.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.065+0000 7f281e7fc700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2808066f30 0x7f28080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:57.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.065+0000 7f281e7fc700 1 -- 172.21.15.3:0/789674456 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f2818084bc0 con 0x7f28280cf6a0 2023-10-06T04:52:57.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.065+0000 7f282659c700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2808066f30 0x7f28080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:57.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.066+0000 7f282659c700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2808066f30 0x7f28080693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2820009d80 tx=0x7f2820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:57.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.069+0000 7f281e7fc700 1 -- 172.21.15.3:0/789674456 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2818051d00 con 0x7f28280cf6a0 2023-10-06T04:52:57.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.263+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f282804c380 con 0x7f28280cf6a0 2023-10-06T04:52:57.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.918+0000 7f281e7fc700 1 -- 172.21.15.3:0/789674456 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2818055320 con 0x7f28280cf6a0 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2808066f30 msgr2=0x7f28080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2808066f30 0x7f28080693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2820009d80 tx=0x7f2820008040 comp rx=0 tx=0).stop 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 msgr2=0x7f2828150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f2828150ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f281800cc90 tx=0x7f2818001a80 comp rx=0 tx=0).stop 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 shutdown_connections 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2808066f30 0x7f28080693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f282809d670 0x7f282816a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28280cf6a0 0x7f2828150ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 --2- 172.21.15.3:0/789674456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28280b4a20 0x7f28280b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:57.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 >> 172.21.15.3:0/789674456 conn(0x7f2828070310 msgr2=0x7f282806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:57.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 shutdown_connections 2023-10-06T04:52:57.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:57.921+0000 7f282759e700 1 -- 172.21.15.3:0/789674456 wait complete. 2023-10-06T04:52:57.925 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:52:57.940 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:52:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.065+0000 7f33f59dd700 1 Processor -- start 2023-10-06T04:52:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.065+0000 7f33f59dd700 1 -- start start 2023-10-06T04:52:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33f59dd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33f015dca0 0x7f33f00cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33f59dd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f00d15f0 con 0x7f33f01598f0 2023-10-06T04:52:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33f59dd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f00ccda0 con 0x7f33f00d1170 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33f59dd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f015e400 con 0x7f33f015dca0 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33ee7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33f015dca0 0x7f33f00cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33eeffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f33f01598f0 0x7f33f015d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54454/0 (socket says 172.21.15.3:54454) 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33eeffd700 1 -- 172.21.15.3:0/1200066573 learned_addr learned my addr 172.21.15.3:0/1200066573 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33ee7fc700 1 -- 172.21.15.3:0/1200066573 >> v1:172.21.15.153:6789/0 conn(0x7f33f00d1170 legacy=0x7f33f0159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33ee7fc700 1 -- 172.21.15.3:0/1200066573 >> v1:172.21.15.3:6789/0 conn(0x7f33f01598f0 legacy=0x7f33f015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33ee7fc700 1 -- 172.21.15.3:0/1200066573 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f00cf7d0 con 0x7f33f015dca0 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33edffb700 1 -- 172.21.15.3:0/1200066573 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151607895 0 0) 0x7f33e8001110 con 0x7f33f01598f0 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33ee7fc700 1 --2- 172.21.15.3:0/1200066573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f015dca0 0x7f33f00cebd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f33d80098e0 tx=0x7f33d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78468a38dd20e318 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33edffb700 1 -- 172.21.15.3:0/1200066573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8009e80 con 0x7f33f015dca0 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33edffb700 1 -- 172.21.15.3:0/1200066573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d800b8b0 con 0x7f33f015dca0 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.066+0000 7f33edffb700 1 -- 172.21.15.3:0/1200066573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d800bc30 con 0x7f33f015dca0 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.067+0000 7f33f59dd700 1 -- 172.21.15.3:0/1200066573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f015dca0 msgr2=0x7f33f00cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.067+0000 7f33f59dd700 1 --2- 172.21.15.3:0/1200066573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f015dca0 0x7f33f00cebd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f33d80098e0 tx=0x7f33d80091a0 comp rx=0 tx=0).stop 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.067+0000 7f33f59dd700 1 -- 172.21.15.3:0/1200066573 shutdown_connections 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.067+0000 7f33f59dd700 1 --2- 172.21.15.3:0/1200066573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f015dca0 0x7f33f00cebd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.067+0000 7f33f59dd700 1 -- 172.21.15.3:0/1200066573 >> 172.21.15.3:0/1200066573 conn(0x7f33f0070af0 msgr2=0x7f33f00d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.067+0000 7f33f59dd700 1 -- 172.21.15.3:0/1200066573 shutdown_connections 2023-10-06T04:52:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.067+0000 7f33f59dd700 1 -- 172.21.15.3:0/1200066573 wait complete. 2023-10-06T04:52:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.068+0000 7f33f59dd700 1 Processor -- start 2023-10-06T04:52:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.068+0000 7f33f59dd700 1 -- start start 2023-10-06T04:52:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33f59dd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d1170 0x7f33f00b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33f59dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f01598f0 0x7f33f0152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33f59dd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f015dca0 0x7f33f015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33f59dd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0153510 con 0x7f33f01598f0 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33f59dd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00b5bd0 con 0x7f33f00d1170 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33f59dd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00b4fc0 con 0x7f33f015dca0 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f01598f0 0x7f33f0152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f01598f0 0x7f33f0152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57468/0 (socket says 172.21.15.3:57468) 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 -- 172.21.15.3:0/77188554 learned_addr learned my addr 172.21.15.3:0/77188554 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33eeffd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d1170 0x7f33f00b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 -- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f015dca0 msgr2=0x7f33f015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f015dca0 0x7f33f015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 -- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d1170 msgr2=0x7f33f00b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d1170 0x7f33f00b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.069+0000 7f33ee7fc700 1 -- 172.21.15.3:0/77188554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8009420 con 0x7f33f01598f0 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.070+0000 7f33ee7fc700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f01598f0 0x7f33f0152fd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33d80030c0 tx=0x7f33d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.070+0000 7f33e6ffd700 1 -- 172.21.15.3:0/77188554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d801b750 con 0x7f33f01598f0 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.070+0000 7f33e6ffd700 1 -- 172.21.15.3:0/77188554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d801b8b0 con 0x7f33f01598f0 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.070+0000 7f33e6ffd700 1 -- 172.21.15.3:0/77188554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d801bc70 con 0x7f33f01598f0 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.070+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f00b52a0 con 0x7f33f01598f0 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.070+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f00b5860 con 0x7f33f01598f0 2023-10-06T04:52:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.071+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f006b790 con 0x7f33f01598f0 2023-10-06T04:52:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.071+0000 7f33e6ffd700 1 -- 172.21.15.3:0/77188554 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33d801c660 con 0x7f33f01598f0 2023-10-06T04:52:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.072+0000 7f33e6ffd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33d0066f30 0x7f33d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.072+0000 7f33e6ffd700 1 -- 172.21.15.3:0/77188554 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f33d80865e0 con 0x7f33f01598f0 2023-10-06T04:52:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.072+0000 7f33eeffd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33d0066f30 0x7f33d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.072+0000 7f33eeffd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33d0066f30 0x7f33d00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f33e8001750 tx=0x7f33e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.076+0000 7f33e6ffd700 1 -- 172.21.15.3:0/77188554 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33d8053850 con 0x7f33f01598f0 2023-10-06T04:52:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.274+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33f0071680 con 0x7f33f01598f0 2023-10-06T04:52:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.923+0000 7f33e6ffd700 1 -- 172.21.15.3:0/77188554 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f33d8056e70 con 0x7f33f01598f0 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33d0066f30 msgr2=0x7f33d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33d0066f30 0x7f33d00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f33e8001750 tx=0x7f33e8009040 comp rx=0 tx=0).stop 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f01598f0 msgr2=0x7f33f0152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f01598f0 0x7f33f0152fd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33d80030c0 tx=0x7f33d8016000 comp rx=0 tx=0).stop 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 shutdown_connections 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33d0066f30 0x7f33d00693e0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f33e8001750 tx=0x7f33e8009040 comp rx=0 tx=0).stop 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d1170 0x7f33f00b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f015dca0 0x7f33f015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 --2- 172.21.15.3:0/77188554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f01598f0 0x7f33f0152fd0 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33d80030c0 tx=0x7f33d8016000 comp rx=0 tx=0).stop 2023-10-06T04:52:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.927+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 >> 172.21.15.3:0/77188554 conn(0x7f33f0070af0 msgr2=0x7f33f006f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.930+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 shutdown_connections 2023-10-06T04:52:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:58.931+0000 7f33f59dd700 1 -- 172.21.15.3:0/77188554 wait complete. 2023-10-06T04:52:58.935 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T04:52:58.955 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:52:58.955 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:52:58.955 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:52:58.955 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:52:58.956 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:52:58.956 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:52:58.956 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:52:58.956 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:52:58.956 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:52:58.956 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:52:58.957 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:52:58.957 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:52:58.957 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:52:58.957 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:52:58.957 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:52:58.957 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:52:58.966 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:52:58.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:52:58.967 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:52:58.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:52:58.969 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:52:58.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:52:58.970 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:52:58.970 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:52:59.009 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:52:59.010 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:59.010 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:52:59.011 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:59.011 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:52:59.012 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:59.012 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:52:59.012 DEBUG:teuthology.parallel:result is None 2023-10-06T04:52:59.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:52:59.086 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:52:59.086 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:52:59.088 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:52:59.089 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:52:59.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.135+0000 7f1cfa35a700 1 Processor -- start 2023-10-06T04:52:59.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.135+0000 7f1cfa35a700 1 -- start start 2023-10-06T04:52:59.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.136+0000 7f1cfa35a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.136+0000 7f1cfa35a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1cf40d0870 0x7f1cf40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.136+0000 7f1cfa35a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf40cf6c0 con 0x7f1cf415ccc0 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.136+0000 7f1cfa35a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf40cfb50 con 0x7f1cf40d0870 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.136+0000 7f1cfa35a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf415e050 con 0x7f1cf415c8f0 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1cf40d0870 0x7f1cf40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf3fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1cf40d0870 0x7f1cf40b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:59840/0 (socket says 172.21.15.3:59840) 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 -- 172.21.15.3:0/1107076004 learned_addr learned my addr 172.21.15.3:0/1107076004 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf37fe700 1 --1- 172.21.15.3:0/1107076004 >> v1:172.21.15.153:6789/0 conn(0x7f1cf415ccc0 0x7f1cf40d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:36388/0 (socket says 172.21.15.3:36388) 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 -- 172.21.15.3:0/1107076004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 msgr2=0x7f1cf415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:52:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 --2- 172.21.15.3:0/1107076004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 -- 172.21.15.3:0/1107076004 >> v1:172.21.15.153:6789/0 conn(0x7f1cf415ccc0 legacy=0x7f1cf40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 -- 172.21.15.3:0/1107076004 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40ceab0 con 0x7f1cf40d0870 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf2ffd700 1 -- 172.21.15.3:0/1107076004 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372586613 0 0) 0x7f1cf40cf6c0 con 0x7f1cf415ccc0 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf88f7700 1 --2- 172.21.15.3:0/1107076004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf40d0870 0x7f1cf40b3ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009f20 tx=0x7f1cec009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204d51c467319316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf2ffd700 1 -- 172.21.15.3:0/1107076004 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00e070 con 0x7f1cf40d0870 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf2ffd700 1 -- 172.21.15.3:0/1107076004 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec00b9d0 con 0x7f1cf40d0870 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.137+0000 7f1cf2ffd700 1 -- 172.21.15.3:0/1107076004 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0092e0 con 0x7f1cf40d0870 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 -- 172.21.15.3:0/1107076004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf40d0870 msgr2=0x7f1cf40b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/1107076004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf40d0870 0x7f1cf40b3ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009f20 tx=0x7f1cec009840 comp rx=0 tx=0).stop 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 -- 172.21.15.3:0/1107076004 shutdown_connections 2023-10-06T04:52:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/1107076004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/1107076004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf40d0870 0x7f1cf40b3ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 -- 172.21.15.3:0/1107076004 >> 172.21.15.3:0/1107076004 conn(0x7f1cf4070310 msgr2=0x7f1cf4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:52:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 -- 172.21.15.3:0/1107076004 shutdown_connections 2023-10-06T04:52:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.138+0000 7f1cfa35a700 1 -- 172.21.15.3:0/1107076004 wait complete. 2023-10-06T04:52:59.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.139+0000 7f1cfa35a700 1 Processor -- start 2023-10-06T04:52:59.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.139+0000 7f1cfa35a700 1 -- start start 2023-10-06T04:52:59.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf40d0870 0x7f1cf4144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:59.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:59.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf415ccc0 0x7f1cf40c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:59.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf415a820 con 0x7f1cf40d0870 2023-10-06T04:52:59.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4159930 con 0x7f1cf415ccc0 2023-10-06T04:52:59.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4159c10 con 0x7f1cf415c8f0 2023-10-06T04:52:59.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf40d0870 0x7f1cf4144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf40d0870 0x7f1cf4144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57498/0 (socket says 172.21.15.3:57498) 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 -- 172.21.15.3:0/2926440533 learned_addr learned my addr 172.21.15.3:0/2926440533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf37fe700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf88f7700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf415ccc0 0x7f1cf40c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 -- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 msgr2=0x7f1cf415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 -- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf415ccc0 msgr2=0x7f1cf40c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf415ccc0 0x7f1cf40c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 -- 172.21.15.3:0/2926440533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec009ae0 con 0x7f1cf40d0870 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cf3fff700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf40d0870 0x7f1cf4144110 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400ec80 tx=0x7f1ce40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:59.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cdbfff700 1 -- 172.21.15.3:0/2926440533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400c850 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cdbfff700 1 -- 172.21.15.3:0/2926440533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce400c9b0 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cdbfff700 1 -- 172.21.15.3:0/2926440533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400cd50 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf4159ef0 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.141+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf415a4b0 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.142+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf4071680 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.142+0000 7f1cdbfff700 1 -- 172.21.15.3:0/2926440533 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ce4013680 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.142+0000 7f1cdbfff700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ce0066f30 0x7f1ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.142+0000 7f1cdbfff700 1 -- 172.21.15.3:0/2926440533 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1ce40845f0 con 0x7f1cf40d0870 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.143+0000 7f1cf37fe700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ce0066f30 0x7f1ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:52:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.143+0000 7f1cf37fe700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ce0066f30 0x7f1ce00693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1cf416cfb0 tx=0x7f1ce8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:52:59.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.147+0000 7f1cdbfff700 1 -- 172.21.15.3:0/2926440533 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ce4051a00 con 0x7f1cf40d0870 2023-10-06T04:52:59.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:52:59.354+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1cf4156380 con 0x7f1cf40d0870 2023-10-06T04:53:00.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.510+0000 7f1cdbfff700 1 -- 172.21.15.3:0/2926440533 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1ce4055020 con 0x7f1cf40d0870 2023-10-06T04:53:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ce0066f30 msgr2=0x7f1ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ce0066f30 0x7f1ce00693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1cf416cfb0 tx=0x7f1ce8007d50 comp rx=0 tx=0).stop 2023-10-06T04:53:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf40d0870 msgr2=0x7f1cf4144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf40d0870 0x7f1cf4144110 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400ec80 tx=0x7f1ce40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 shutdown_connections 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ce0066f30 0x7f1ce00693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf415ccc0 0x7f1cf40c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf40d0870 0x7f1cf4144110 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 --2- 172.21.15.3:0/2926440533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cf415c8f0 0x7f1cf415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 >> 172.21.15.3:0/2926440533 conn(0x7f1cf4070310 msgr2=0x7f1cf40b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.513+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 shutdown_connections 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.514+0000 7f1cfa35a700 1 -- 172.21.15.3:0/2926440533 wait complete. 2023-10-06T04:53:00.516 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T04:53:00.531 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:53:00.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.655+0000 7f566e054700 1 Processor -- start 2023-10-06T04:53:00.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.655+0000 7f566e054700 1 -- start start 2023-10-06T04:53:00.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f566e054700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f56680cb3b0 0x7f56680cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f566e054700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56680cf130 con 0x7f56681596d0 2023-10-06T04:53:00.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f566e054700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56680ce520 con 0x7f56680cb780 2023-10-06T04:53:00.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f566e054700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56680ce800 con 0x7f56680cb3b0 2023-10-06T04:53:00.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f566d853700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f56681596d0 0x7f56680cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54468/0 (socket says 172.21.15.3:54468) 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f566d853700 1 -- 172.21.15.3:0/74901716 learned_addr learned my addr 172.21.15.3:0/74901716 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651628687 0 0) 0x7f56680cf130 con 0x7f56681596d0 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5650003570 con 0x7f56681596d0 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.656+0000 7f566d052700 1 --2- 172.21.15.3:0/74901716 >> v2:172.21.15.153:3300/0 conn(0x7f56680cb3b0 0x7f56680cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 390826114 0 0) 0x7f5650003570 con 0x7f56681596d0 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 >> v2:172.21.15.153:3300/0 conn(0x7f56680cb3b0 msgr2=0x7f56680cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 --2- 172.21.15.3:0/74901716 >> v2:172.21.15.153:3300/0 conn(0x7f56680cb3b0 0x7f56680cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 >> v1:172.21.15.153:6789/0 conn(0x7f56680cb780 legacy=0x7f5668158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680ceae0 con 0x7f56681596d0 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f5660001470 con 0x7f56681596d0 2023-10-06T04:53:00.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 >> v1:172.21.15.3:6789/0 conn(0x7f56681596d0 legacy=0x7f56680cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5650003a60 0x7f5650003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5650004810 0x7f5650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56500051c0 0x7f565000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680ce800 con 0x7f56500051c0 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565000c160 con 0x7f5650004810 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565000c320 con 0x7f5650003a60 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56500051c0 0x7f565000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566c851700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5650004810 0x7f5650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d052700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5650003a60 0x7f5650003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 -- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5650003a60 msgr2=0x7f5650003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:00.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5650003a60 0x7f5650003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 -- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5650004810 msgr2=0x7f5650004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5650004810 0x7f5650004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 -- 172.21.15.3:0/74901716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650003570 con 0x7f56500051c0 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 -- 172.21.15.3:0/74901716 reap_dead start 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f566d853700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56500051c0 0x7f565000ba60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f566000c3c0 tx=0x7f5660001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299438cd5dea41da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566000cc90 con 0x7f56500051c0 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566000cdf0 con 0x7f56500051c0 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.657+0000 7f5667fff700 1 -- 172.21.15.3:0/74901716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660005b50 con 0x7f56500051c0 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.658+0000 7f566e054700 1 -- 172.21.15.3:0/74901716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56500051c0 msgr2=0x7f565000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.658+0000 7f566e054700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56500051c0 0x7f565000ba60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f566000c3c0 tx=0x7f5660001e50 comp rx=0 tx=0).stop 2023-10-06T04:53:00.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.658+0000 7f566e054700 1 -- 172.21.15.3:0/74901716 shutdown_connections 2023-10-06T04:53:00.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.658+0000 7f566e054700 1 --2- 172.21.15.3:0/74901716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56500051c0 0x7f565000ba60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.658+0000 7f566e054700 1 -- 172.21.15.3:0/74901716 >> 172.21.15.3:0/74901716 conn(0x7f5668070310 msgr2=0x7f5668070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:00.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.658+0000 7f566e054700 1 -- 172.21.15.3:0/74901716 shutdown_connections 2023-10-06T04:53:00.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.658+0000 7f566e054700 1 -- 172.21.15.3:0/74901716 wait complete. 2023-10-06T04:53:00.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.659+0000 7f566e054700 1 Processor -- start 2023-10-06T04:53:00.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.659+0000 7f566e054700 1 -- start start 2023-10-06T04:53:00.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566e054700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56500051c0 0x7f56681578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566e054700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5668169850 0x7f56680d0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566e054700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56680b4820 0x7f56680b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566e054700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566816a090 con 0x7f5668169850 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566e054700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566816a370 con 0x7f56680b4820 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566e054700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566816a650 con 0x7f56500051c0 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5668169850 0x7f56680d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5668169850 0x7f56680d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57522/0 (socket says 172.21.15.3:57522) 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 -- 172.21.15.3:0/967855804 learned_addr learned my addr 172.21.15.3:0/967855804 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566d853700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56680b4820 0x7f56680b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566d052700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56500051c0 0x7f56681578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 -- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56500051c0 msgr2=0x7f56681578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56500051c0 0x7f56681578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 -- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56680b4820 msgr2=0x7f56680b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56680b4820 0x7f56680b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.660+0000 7f566c851700 1 -- 172.21.15.3:0/967855804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680b4020 con 0x7f5668169850 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.661+0000 7f566c851700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5668169850 0x7f56680d0410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f565c002950 tx=0x7f565c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.661+0000 7f56657fa700 1 -- 172.21.15.3:0/967855804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c003390 con 0x7f5668169850 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.661+0000 7f56657fa700 1 -- 172.21.15.3:0/967855804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c006820 con 0x7f5668169850 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.661+0000 7f56657fa700 1 -- 172.21.15.3:0/967855804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c006bc0 con 0x7f5668169850 2023-10-06T04:53:00.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.661+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56680b4300 con 0x7f5668169850 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.661+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56680d0950 con 0x7f5668169850 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.662+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56680b4540 con 0x7f5668169850 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.662+0000 7f56657fa700 1 -- 172.21.15.3:0/967855804 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f565c007030 con 0x7f5668169850 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.662+0000 7f56657fa700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5654066f30 0x7f56540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.663+0000 7f56657fa700 1 -- 172.21.15.3:0/967855804 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f565c0843d0 con 0x7f5668169850 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.663+0000 7f566d052700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5654066f30 0x7f56540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.663+0000 7f566d052700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5654066f30 0x7f56540693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5658009fd0 tx=0x7f565800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:00.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.667+0000 7f56657fa700 1 -- 172.21.15.3:0/967855804 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56680b4540 con 0x7f5668169850 2023-10-06T04:53:00.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.943+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f56680b4540 con 0x7f5668169850 2023-10-06T04:53:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.944+0000 7f56657fa700 1 -- 172.21.15.3:0/967855804 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f56680b4540 con 0x7f5668169850 2023-10-06T04:53:00.945 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:00.945 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.947 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T04:53:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.949 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.951 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T04:53:00.953 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T04:53:00.954 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:00.955 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:53:00.956 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:53:00.957 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.959 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mds", 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBvkh9l4Zp1LBAAJDdOObsD7LilXJ3VAzqrCQ==", 2023-10-06T04:53:00.960 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBvkh9ljqR1LBAAD5ukzeecqTJ7yL+j6mgI1g==", 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.961 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-osd", 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBvkh9l8K11LBAATFCATOBZhsUfFZpdHdPtsw==", 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBvkh9l1rZ1LBAAiAylbLYUwE5ZUL2g2REx9Q==", 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.962 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBvkh9lzr51LBAA0pQfbs1dut9yhTSFzoa0Fg==", 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T04:53:00.963 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBvkh9l7cZ1LBAAXmfzUvEJk+RfU5NR+HcxbA==", 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.964 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:00.965 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:53:00.966 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:53:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5654066f30 msgr2=0x7f56540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5654066f30 0x7f56540693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5658009fd0 tx=0x7f565800b040 comp rx=0 tx=0).stop 2023-10-06T04:53:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5668169850 msgr2=0x7f56680d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5668169850 0x7f56680d0410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f565c002950 tx=0x7f565c007d30 comp rx=0 tx=0).stop 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 shutdown_connections 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5654066f30 0x7f56540693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56680b4820 0x7f56680b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56500051c0 0x7f56681578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 --2- 172.21.15.3:0/967855804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5668169850 0x7f56680d0410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 >> 172.21.15.3:0/967855804 conn(0x7f5668070310 msgr2=0x7f56680b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 shutdown_connections 2023-10-06T04:53:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:00.946+0000 7f566e054700 1 -- 172.21.15.3:0/967855804 wait complete. 2023-10-06T04:53:00.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T04:53:01.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.087+0000 7fed48027700 1 Processor -- start 2023-10-06T04:53:01.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.088+0000 7fed48027700 1 -- start start 2023-10-06T04:53:01.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.088+0000 7fed48027700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 0x7fed400d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.088+0000 7fed48027700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed400cf6c0 con 0x7fed4015c8f0 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.088+0000 7fed48027700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed400cfb50 con 0x7fed400d06a0 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.088+0000 7fed48027700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed400d0b20 con 0x7fed4015ccc0 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed45dc3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed4015c8f0 0x7fed4015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54484/0 (socket says 172.21.15.3:54484) 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed45dc3700 1 -- 172.21.15.3:0/4205641339 learned_addr learned my addr 172.21.15.3:0/4205641339 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed455c2700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 0x7fed400d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779923558 0 0) 0x7fed400cf6c0 con 0x7fed4015c8f0 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed28003570 con 0x7fed4015c8f0 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1724256558 0 0) 0x7fed28003570 con 0x7fed4015c8f0 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 msgr2=0x7fed400d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 0x7fed400d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 >> v1:172.21.15.153:6789/0 conn(0x7fed400d06a0 legacy=0x7fed400b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed400ceab0 con 0x7fed4015c8f0 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fed380017f0 con 0x7fed4015c8f0 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 >> v1:172.21.15.3:6789/0 conn(0x7fed4015c8f0 legacy=0x7fed4015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed28003a90 0x7fed28003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 0x7fed28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed28004d50 0x7fed280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed400d0b20 con 0x7fed28003a90 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2800bb70 con 0x7fed28004d50 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2800bd30 con 0x7fed4015ccc0 2023-10-06T04:53:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed45dc3700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed28003a90 0x7fed28003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed455c2700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 0x7fed28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.089+0000 7fed465c4700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed28004d50 0x7fed280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed45dc3700 1 -- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 msgr2=0x7fed28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed45dc3700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed4015ccc0 0x7fed28004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed45dc3700 1 -- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed28004d50 msgr2=0x7fed280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed45dc3700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed28004d50 0x7fed280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed45dc3700 1 -- 172.21.15.3:0/4205641339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28003570 con 0x7fed28003a90 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed465c4700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed28004d50 0x7fed280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed45dc3700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed28003a90 0x7fed28003ea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed3800a930 tx=0x7fed380021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751f0548baada27c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed380070d0 con 0x7fed28003a90 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38007230 con 0x7fed28003a90 2023-10-06T04:53:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed44dc1700 1 -- 172.21.15.3:0/4205641339 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed380075d0 con 0x7fed28003a90 2023-10-06T04:53:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed48027700 1 -- 172.21.15.3:0/4205641339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed28003a90 msgr2=0x7fed28003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed48027700 1 --2- 172.21.15.3:0/4205641339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed28003a90 0x7fed28003ea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed3800a930 tx=0x7fed380021d0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed465c4700 1 -- 172.21.15.3:0/4205641339 reap_dead start 2023-10-06T04:53:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed48027700 1 -- 172.21.15.3:0/4205641339 shutdown_connections 2023-10-06T04:53:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.090+0000 7fed48027700 1 -- 172.21.15.3:0/4205641339 >> 172.21.15.3:0/4205641339 conn(0x7fed40070310 msgr2=0x7fed40070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.091+0000 7fed48027700 1 -- 172.21.15.3:0/4205641339 shutdown_connections 2023-10-06T04:53:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.091+0000 7fed48027700 1 -- 172.21.15.3:0/4205641339 wait complete. 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.092+0000 7fed48027700 1 Processor -- start 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.092+0000 7fed48027700 1 -- start start 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.092+0000 7fed48027700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed28003a90 0x7fed40151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.092+0000 7fed48027700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed400b4b70 0x7fed400b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.092+0000 7fed48027700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed401546c0 0x7fed40154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed48027700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed40151c90 con 0x7fed400b4b70 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed48027700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed400b5520 con 0x7fed401546c0 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed48027700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4014e8b0 con 0x7fed28003a90 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed400b4b70 0x7fed400b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed400b4b70 0x7fed400b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57526/0 (socket says 172.21.15.3:57526) 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 -- 172.21.15.3:0/2033674286 learned_addr learned my addr 172.21.15.3:0/2033674286 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed45dc3700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed28003a90 0x7fed40151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed465c4700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed401546c0 0x7fed40154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 -- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed28003a90 msgr2=0x7fed40151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed28003a90 0x7fed40151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 -- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed401546c0 msgr2=0x7fed40154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed401546c0 0x7fed40154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 -- 172.21.15.3:0/2033674286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4014f430 con 0x7fed400b4b70 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed45dc3700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed28003a90 0x7fed40151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed455c2700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed400b4b70 0x7fed400b4fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fed34011b40 tx=0x7fed340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed3dffb700 1 -- 172.21.15.3:0/2033674286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400fa70 con 0x7fed400b4b70 2023-10-06T04:53:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed3dffb700 1 -- 172.21.15.3:0/2033674286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3400fbd0 con 0x7fed400b4b70 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4015be40 con 0x7fed400b4b70 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.093+0000 7fed3dffb700 1 -- 172.21.15.3:0/2033674286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34014440 con 0x7fed400b4b70 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.094+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4014eb00 con 0x7fed400b4b70 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.094+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4014f5c0 con 0x7fed400b4b70 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.095+0000 7fed3dffb700 1 -- 172.21.15.3:0/2033674286 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed34014830 con 0x7fed400b4b70 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.095+0000 7fed3dffb700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed2c066f30 0x7fed2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.095+0000 7fed3dffb700 1 -- 172.21.15.3:0/2033674286 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fed34018030 con 0x7fed400b4b70 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.095+0000 7fed45dc3700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed2c066f30 0x7fed2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.096+0000 7fed45dc3700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed2c066f30 0x7fed2c0693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed3800a3f0 tx=0x7fed38016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.099+0000 7fed3dffb700 1 -- 172.21.15.3:0/2033674286 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed34052210 con 0x7fed400b4b70 2023-10-06T04:53:01.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.373+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fed40071680 con 0x7fed400b4b70 2023-10-06T04:53:01.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.377+0000 7fed3dffb700 1 -- 172.21.15.3:0/2033674286 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fed34059e60 con 0x7fed400b4b70 2023-10-06T04:53:01.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed2c066f30 msgr2=0x7fed2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed2c066f30 0x7fed2c0693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed3800a3f0 tx=0x7fed38016000 comp rx=0 tx=0).stop 2023-10-06T04:53:01.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed400b4b70 msgr2=0x7fed400b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed400b4b70 0x7fed400b4fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fed34011b40 tx=0x7fed340091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 shutdown_connections 2023-10-06T04:53:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed2c066f30 0x7fed2c0693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed401546c0 0x7fed40154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed28003a90 0x7fed40151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 --2- 172.21.15.3:0/2033674286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed400b4b70 0x7fed400b4fe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 >> 172.21.15.3:0/2033674286 conn(0x7fed40070310 msgr2=0x7fed400b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 shutdown_connections 2023-10-06T04:53:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.379+0000 7fed48027700 1 -- 172.21.15.3:0/2033674286 wait complete. 2023-10-06T04:53:01.393 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T04:53:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.512+0000 7efffc4e8700 1 Processor -- start 2023-10-06T04:53:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.512+0000 7efffc4e8700 1 -- start start 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efffc4e8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efff415cb50 0x7efff40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efffc4e8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efff40b0590 0x7efff40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efffc4e8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff40b0ce0 con 0x7efff40cd950 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efffc4e8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff40cccc0 con 0x7efff40b0590 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efffc4e8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff40cb820 con 0x7efff415cb50 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efff40b0590 0x7efff40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efff40b0590 0x7efff40b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57530/0 (socket says 172.21.15.3:57530) 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 -- 172.21.15.3:0/2558236084 learned_addr learned my addr 172.21.15.3:0/2558236084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 -- 172.21.15.3:0/2558236084 >> v2:172.21.15.153:3300/0 conn(0x7efff415cb50 msgr2=0x7efff40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 --2- 172.21.15.3:0/2558236084 >> v2:172.21.15.153:3300/0 conn(0x7efff415cb50 0x7efff40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 -- 172.21.15.3:0/2558236084 >> v1:172.21.15.153:6789/0 conn(0x7efff40cd950 legacy=0x7efff406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 -- 172.21.15.3:0/2558236084 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40b0a00 con 0x7efff40b0590 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9a83700 1 --2- 172.21.15.3:0/2558236084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff40b2200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efff40897d0 tx=0x7effe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a8308e3d796b2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.513+0000 7efff9282700 1 -- 172.21.15.3:0/2558236084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe400b820 con 0x7efff40b0590 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efff9282700 1 -- 172.21.15.3:0/2558236084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe400b980 con 0x7efff40b0590 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efff9282700 1 -- 172.21.15.3:0/2558236084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe400bd20 con 0x7efff40b0590 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 -- 172.21.15.3:0/2558236084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 msgr2=0x7efff40b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 --2- 172.21.15.3:0/2558236084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff40b2200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efff40897d0 tx=0x7effe4009250 comp rx=0 tx=0).stop 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 -- 172.21.15.3:0/2558236084 shutdown_connections 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 --2- 172.21.15.3:0/2558236084 >> v2:172.21.15.153:3300/0 conn(0x7efff415cb50 0x7efff40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 --2- 172.21.15.3:0/2558236084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff40b2200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 -- 172.21.15.3:0/2558236084 >> 172.21.15.3:0/2558236084 conn(0x7efff4070310 msgr2=0x7efff4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 -- 172.21.15.3:0/2558236084 shutdown_connections 2023-10-06T04:53:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.514+0000 7efffc4e8700 1 -- 172.21.15.3:0/2558236084 wait complete. 2023-10-06T04:53:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.515+0000 7efffc4e8700 1 Processor -- start 2023-10-06T04:53:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.515+0000 7efffc4e8700 1 -- start start 2023-10-06T04:53:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffc4e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff4153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffc4e8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efff40cd950 0x7efff416acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffc4e8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efff415cb50 0x7efff40d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffc4e8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40ceb90 con 0x7efff40b0590 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffc4e8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40cdf80 con 0x7efff40cd950 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffc4e8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff40ce260 con 0x7efff415cb50 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff4153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff4153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57544/0 (socket says 172.21.15.3:57544) 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 -- 172.21.15.3:0/408282918 learned_addr learned my addr 172.21.15.3:0/408282918 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efff9a83700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efff40cd950 0x7efff416acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffaa85700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efff415cb50 0x7efff40d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 -- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efff415cb50 msgr2=0x7efff40d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efff415cb50 0x7efff40d04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 -- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efff40cd950 msgr2=0x7efff416acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efff40cd950 0x7efff416acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.516+0000 7efffa284700 1 -- 172.21.15.3:0/408282918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40094f0 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.517+0000 7efffa284700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff4153f80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efff000b600 tx=0x7efff00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.517+0000 7effea7fc700 1 -- 172.21.15.3:0/408282918 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff000f510 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.517+0000 7effea7fc700 1 -- 172.21.15.3:0/408282918 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff000f670 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.517+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff40ce540 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.517+0000 7effea7fc700 1 -- 172.21.15.3:0/408282918 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff000fa30 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.517+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff41544c0 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.518+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff4071680 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.518+0000 7effea7fc700 1 -- 172.21.15.3:0/408282918 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efff001a400 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.519+0000 7effea7fc700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effd8066f30 0x7effd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.519+0000 7effea7fc700 1 -- 172.21.15.3:0/408282918 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efff0083d40 con 0x7efff40b0590 2023-10-06T04:53:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.519+0000 7efff9a83700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effd8066f30 0x7effd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.519+0000 7efff9a83700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effd8066f30 0x7effd80693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efff4089980 tx=0x7effe4016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.523+0000 7effea7fc700 1 -- 172.21.15.3:0/408282918 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efff0051220 con 0x7efff40b0590 2023-10-06T04:53:01.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.797+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7efff40cb1d0 con 0x7efff40b0590 2023-10-06T04:53:01.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.801+0000 7effea7fc700 1 -- 172.21.15.3:0/408282918 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7efff0054840 con 0x7efff40b0590 2023-10-06T04:53:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effd8066f30 msgr2=0x7effd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effd8066f30 0x7effd80693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efff4089980 tx=0x7effe4016000 comp rx=0 tx=0).stop 2023-10-06T04:53:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 msgr2=0x7efff4153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff4153f80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efff000b600 tx=0x7efff00091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 shutdown_connections 2023-10-06T04:53:01.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effd8066f30 0x7effd80693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efff415cb50 0x7efff40d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff40b0590 0x7efff4153f80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 --2- 172.21.15.3:0/408282918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efff40cd950 0x7efff416acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 >> 172.21.15.3:0/408282918 conn(0x7efff4070310 msgr2=0x7efff40b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:01.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 shutdown_connections 2023-10-06T04:53:01.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.803+0000 7efffc4e8700 1 -- 172.21.15.3:0/408282918 wait complete. 2023-10-06T04:53:01.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T04:53:01.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.935+0000 7f8b092bf700 1 Processor -- start 2023-10-06T04:53:01.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.935+0000 7f8b092bf700 1 -- start start 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b092bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b040b1a50 0x7f8b0406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b092bf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040aedc0 0x7f8b04151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b092bf700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0406dae0 con 0x7f8b040b1e20 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b092bf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b04158930 con 0x7f8b040b1a50 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b092bf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040cbcd0 con 0x7f8b040aedc0 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b02ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b040b1a50 0x7f8b0406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b02ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b040b1a50 0x7f8b0406ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57558/0 (socket says 172.21.15.3:57558) 2023-10-06T04:53:01.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b02ffd700 1 -- 172.21.15.3:0/384242455 learned_addr learned my addr 172.21.15.3:0/384242455 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b027fc700 1 --2- 172.21.15.3:0/384242455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040aedc0 0x7f8b04151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b02ffd700 1 -- 172.21.15.3:0/384242455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040aedc0 msgr2=0x7f8b04151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b02ffd700 1 --2- 172.21.15.3:0/384242455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040aedc0 0x7f8b04151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b02ffd700 1 -- 172.21.15.3:0/384242455 >> v1:172.21.15.153:6789/0 conn(0x7f8b040b1e20 legacy=0x7f8b0406a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b02ffd700 1 -- 172.21.15.3:0/384242455 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0406d800 con 0x7f8b040b1a50 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.936+0000 7f8b027fc700 1 --2- 172.21.15.3:0/384242455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040aedc0 0x7f8b04151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b02ffd700 1 --2- 172.21.15.3:0/384242455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0406ceb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8af40099b0 tx=0x7f8af4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8c26234c08fad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b01ffb700 1 -- 172.21.15.3:0/384242455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400b860 con 0x7f8b040b1a50 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b01ffb700 1 -- 172.21.15.3:0/384242455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af400b9c0 con 0x7f8b040b1a50 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b01ffb700 1 -- 172.21.15.3:0/384242455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400bd60 con 0x7f8b040b1a50 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 -- 172.21.15.3:0/384242455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 msgr2=0x7f8b0406ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 --2- 172.21.15.3:0/384242455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0406ceb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8af40099b0 tx=0x7f8af4009250 comp rx=0 tx=0).stop 2023-10-06T04:53:01.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 -- 172.21.15.3:0/384242455 shutdown_connections 2023-10-06T04:53:01.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 --2- 172.21.15.3:0/384242455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0406ceb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 --2- 172.21.15.3:0/384242455 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040aedc0 0x7f8b04151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 -- 172.21.15.3:0/384242455 >> 172.21.15.3:0/384242455 conn(0x7f8b04066fb0 msgr2=0x7f8b040673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:01.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 -- 172.21.15.3:0/384242455 shutdown_connections 2023-10-06T04:53:01.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.937+0000 7f8b092bf700 1 -- 172.21.15.3:0/384242455 wait complete. 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 Processor -- start 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 -- start start 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b040aedc0 0x7f8b04148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0416b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040b1e20 0x7f8b041465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04149030 con 0x7f8b040b1a50 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0416b530 con 0x7f8b040aedc0 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b092bf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0414c520 con 0x7f8b040b1e20 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b027fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0416b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b027fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0416b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57570/0 (socket says 172.21.15.3:57570) 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.939+0000 7f8b027fc700 1 -- 172.21.15.3:0/3173236998 learned_addr learned my addr 172.21.15.3:0/3173236998 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b02ffd700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b040aedc0 0x7f8b04148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b037fe700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040b1e20 0x7f8b041465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b027fc700 1 -- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040b1e20 msgr2=0x7f8b041465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b027fc700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040b1e20 0x7f8b041465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b027fc700 1 -- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b040aedc0 msgr2=0x7f8b04148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b027fc700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b040aedc0 0x7f8b04148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b027fc700 1 -- 172.21.15.3:0/3173236998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40094f0 con 0x7f8b040b1a50 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b027fc700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0416b9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8af800e8a0 tx=0x7f8af8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8aeaffd700 1 -- 172.21.15.3:0/3173236998 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800c430 con 0x7f8b040b1a50 2023-10-06T04:53:01.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8aeaffd700 1 -- 172.21.15.3:0/3173236998 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af800c590 con 0x7f8b040b1a50 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0416b200 con 0x7f8b040b1a50 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8aeaffd700 1 -- 172.21.15.3:0/3173236998 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800c950 con 0x7f8b040b1a50 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.940+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0414c240 con 0x7f8b040b1a50 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.941+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0416b6c0 con 0x7f8b040b1a50 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.941+0000 7f8aeaffd700 1 -- 172.21.15.3:0/3173236998 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8af800cd40 con 0x7f8b040b1a50 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.942+0000 7f8aeaffd700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8af0066f30 0x7f8af00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.942+0000 7f8aeaffd700 1 -- 172.21.15.3:0/3173236998 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8af80840d0 con 0x7f8b040b1a50 2023-10-06T04:53:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.942+0000 7f8b02ffd700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8af0066f30 0x7f8af00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:01.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.943+0000 7f8b02ffd700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8af0066f30 0x7f8af00693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8af40099b0 tx=0x7f8af4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:01.946+0000 7f8aeaffd700 1 -- 172.21.15.3:0/3173236998 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8af8051530 con 0x7f8b040b1a50 2023-10-06T04:53:02.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.220+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8b04071680 con 0x7f8b040b1a50 2023-10-06T04:53:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.224+0000 7f8aeaffd700 1 -- 172.21.15.3:0/3173236998 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8af8054b50 con 0x7f8b040b1a50 2023-10-06T04:53:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8af0066f30 msgr2=0x7f8af00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8af0066f30 0x7f8af00693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8af40099b0 tx=0x7f8af4017000 comp rx=0 tx=0).stop 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 msgr2=0x7f8b0416b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0416b9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8af800e8a0 tx=0x7f8af8008040 comp rx=0 tx=0).stop 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 shutdown_connections 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8af0066f30 0x7f8af00693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b040aedc0 0x7f8b04148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b040b1e20 0x7f8b041465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 --2- 172.21.15.3:0/3173236998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b040b1a50 0x7f8b0416b9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 >> 172.21.15.3:0/3173236998 conn(0x7f8b04066fb0 msgr2=0x7f8b040cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 shutdown_connections 2023-10-06T04:53:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.226+0000 7f8b092bf700 1 -- 172.21.15.3:0/3173236998 wait complete. 2023-10-06T04:53:02.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T04:53:02.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.358+0000 7fbd6b57a700 1 Processor -- start 2023-10-06T04:53:02.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.358+0000 7fbd6b57a700 1 -- start start 2023-10-06T04:53:02.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd6b57a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbd640cd950 0x7fbd640b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd6b57a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd640cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd6b57a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640b0ce0 con 0x7fbd6415cb50 2023-10-06T04:53:02.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd6b57a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd640cb820 con 0x7fbd640cd950 2023-10-06T04:53:02.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd6b57a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640b0760 con 0x7fbd640b2740 2023-10-06T04:53:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd69b17700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd640cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd68b15700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbd640cd950 0x7fbd640b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd68b15700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbd640cd950 0x7fbd640b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:44060/0 (socket says 172.21.15.3:44060) 2023-10-06T04:53:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd69b17700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd640cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:47246/0 (socket says 172.21.15.3:47246) 2023-10-06T04:53:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd68b15700 1 -- 172.21.15.3:0/3185909476 learned_addr learned my addr 172.21.15.3:0/3185909476 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd69316700 1 --1- 172.21.15.3:0/3185909476 >> v1:172.21.15.153:6789/0 conn(0x7fbd6415cb50 0x7fbd640b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53898/0 (socket says 172.21.15.3:53898) 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd68b15700 1 -- 172.21.15.3:0/3185909476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 msgr2=0x7fbd640cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd68b15700 1 --2- 172.21.15.3:0/3185909476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd640cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd68b15700 1 -- 172.21.15.3:0/3185909476 >> v1:172.21.15.153:6789/0 conn(0x7fbd6415cb50 legacy=0x7fbd640b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd5bfff700 1 -- 172.21.15.3:0/3185909476 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446828792 0 0) 0x7fbd640b0ce0 con 0x7fbd6415cb50 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.359+0000 7fbd68b15700 1 -- 172.21.15.3:0/3185909476 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640b0a00 con 0x7fbd640cd950 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd68b15700 1 --2- 172.21.15.3:0/3185909476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 0x7fbd640b2200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbd540099b0 tx=0x7fbd54009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582004f95bce7d9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd5bfff700 1 -- 172.21.15.3:0/3185909476 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54009ea0 con 0x7fbd640cd950 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd5bfff700 1 -- 172.21.15.3:0/3185909476 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5400b860 con 0x7fbd640cd950 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd5bfff700 1 -- 172.21.15.3:0/3185909476 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5400bc00 con 0x7fbd640cd950 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 -- 172.21.15.3:0/3185909476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 msgr2=0x7fbd640b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/3185909476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 0x7fbd640b2200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbd540099b0 tx=0x7fbd54009250 comp rx=0 tx=0).stop 2023-10-06T04:53:02.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 -- 172.21.15.3:0/3185909476 shutdown_connections 2023-10-06T04:53:02.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/3185909476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd640cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/3185909476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 0x7fbd640b2200 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 -- 172.21.15.3:0/3185909476 >> 172.21.15.3:0/3185909476 conn(0x7fbd64070310 msgr2=0x7fbd64070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:02.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 -- 172.21.15.3:0/3185909476 shutdown_connections 2023-10-06T04:53:02.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.360+0000 7fbd6b57a700 1 -- 172.21.15.3:0/3185909476 wait complete. 2023-10-06T04:53:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 Processor -- start 2023-10-06T04:53:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 -- start start 2023-10-06T04:53:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd6414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 0x7fbd640c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6415cb50 0x7fbd640c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6415d8f0 con 0x7fbd6415cb50 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6414fe70 con 0x7fbd640cd950 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.362+0000 7fbd6b57a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6414f930 con 0x7fbd640b2740 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6415cb50 0x7fbd640c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6415cb50 0x7fbd640c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40636/0 (socket says 172.21.15.3:40636) 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 -- 172.21.15.3:0/2211748412 learned_addr learned my addr 172.21.15.3:0/2211748412 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd68b15700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 0x7fbd640c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69316700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd6414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 -- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 msgr2=0x7fbd6414f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd6414f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 -- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 msgr2=0x7fbd640c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 0x7fbd640c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 -- 172.21.15.3:0/2211748412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540094f0 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd69b17700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6415cb50 0x7fbd640c97d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009e90 tx=0x7fbd5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd597fa700 1 -- 172.21.15.3:0/2211748412 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00cb70 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd597fa700 1 -- 172.21.15.3:0/2211748412 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c00f040 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd640ca370 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd597fa700 1 -- 172.21.15.3:0/2211748412 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c014480 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.363+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6415d2a0 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.364+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6415da60 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.365+0000 7fbd597fa700 1 -- 172.21.15.3:0/2211748412 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd5c014830 con 0x7fbd6415cb50 2023-10-06T04:53:02.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.365+0000 7fbd597fa700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbd48066f30 0x7fbd480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.365+0000 7fbd597fa700 1 -- 172.21.15.3:0/2211748412 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbd5c084d10 con 0x7fbd6415cb50 2023-10-06T04:53:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.365+0000 7fbd69316700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbd48066f30 0x7fbd480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.366+0000 7fbd69316700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbd48066f30 0x7fbd480693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbd60000fb0 tx=0x7fbd60007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:02.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.369+0000 7fbd597fa700 1 -- 172.21.15.3:0/2211748412 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd5c052170 con 0x7fbd6415cb50 2023-10-06T04:53:02.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.642+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fbd64071680 con 0x7fbd6415cb50 2023-10-06T04:53:02.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.646+0000 7fbd597fa700 1 -- 172.21.15.3:0/2211748412 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbd5c055790 con 0x7fbd6415cb50 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbd48066f30 msgr2=0x7fbd480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbd48066f30 0x7fbd480693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbd60000fb0 tx=0x7fbd60007f00 comp rx=0 tx=0).stop 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6415cb50 msgr2=0x7fbd640c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6415cb50 0x7fbd640c97d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009e90 tx=0x7fbd5c00b040 comp rx=0 tx=0).stop 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 shutdown_connections 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbd48066f30 0x7fbd480693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6415cb50 0x7fbd640c97d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd640b2740 0x7fbd6414f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 --2- 172.21.15.3:0/2211748412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd640cd950 0x7fbd640c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 >> 172.21.15.3:0/2211748412 conn(0x7fbd64070310 msgr2=0x7fbd640c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:02.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 shutdown_connections 2023-10-06T04:53:02.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.648+0000 7fbd6b57a700 1 -- 172.21.15.3:0/2211748412 wait complete. 2023-10-06T04:53:02.662 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T04:53:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.780+0000 7f14b506b700 1 Processor -- start 2023-10-06T04:53:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.780+0000 7f14b506b700 1 -- start start 2023-10-06T04:53:02.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14b506b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14b00c9840 0x7f14b00cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14b506b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00b1f80 0x7f14b0158e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14b506b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b00678e0 con 0x7f14b00c9470 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14b506b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b00cc430 con 0x7f14b00c9840 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14b506b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b0066fb0 con 0x7f14b00b1f80 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14affff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14b00c9470 0x7f14b006c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39424/0 (socket says 172.21.15.3:39424) 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14affff700 1 -- 172.21.15.3:0/249965035 learned_addr learned my addr 172.21.15.3:0/249965035 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.781+0000 7f14af7fe700 1 --2- 172.21.15.3:0/249965035 >> v2:172.21.15.3:3300/0 conn(0x7f14b00c9840 0x7f14b00cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14b486a700 1 --2- 172.21.15.3:0/249965035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00b1f80 0x7f14b0158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14aeffd700 1 -- 172.21.15.3:0/249965035 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554056186 0 0) 0x7f14b00678e0 con 0x7f14b00c9470 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14aeffd700 1 -- 172.21.15.3:0/249965035 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1498003570 con 0x7f14b00c9470 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14af7fe700 1 -- 172.21.15.3:0/249965035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00b1f80 msgr2=0x7f14b0158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14af7fe700 1 --2- 172.21.15.3:0/249965035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00b1f80 0x7f14b0158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14af7fe700 1 -- 172.21.15.3:0/249965035 >> v1:172.21.15.3:6789/0 conn(0x7f14b00c9470 legacy=0x7f14b006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14af7fe700 1 -- 172.21.15.3:0/249965035 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0067290 con 0x7f14b00c9840 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14af7fe700 1 --2- 172.21.15.3:0/249965035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b00cbe50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f14a40098e0 tx=0x7f14a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b2566871c8743 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14aeffd700 1 -- 172.21.15.3:0/249965035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4011630 con 0x7f14b00c9840 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14aeffd700 1 -- 172.21.15.3:0/249965035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4011790 con 0x7f14b00c9840 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.782+0000 7f14aeffd700 1 -- 172.21.15.3:0/249965035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4011b10 con 0x7f14b00c9840 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 -- 172.21.15.3:0/249965035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 msgr2=0x7f14b00cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 --2- 172.21.15.3:0/249965035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b00cbe50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f14a40098e0 tx=0x7f14a40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 -- 172.21.15.3:0/249965035 shutdown_connections 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 --2- 172.21.15.3:0/249965035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00b1f80 0x7f14b0158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 --2- 172.21.15.3:0/249965035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b00cbe50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 -- 172.21.15.3:0/249965035 >> 172.21.15.3:0/249965035 conn(0x7f14b00cd1f0 msgr2=0x7f14b00b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:02.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 -- 172.21.15.3:0/249965035 shutdown_connections 2023-10-06T04:53:02.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.783+0000 7f14b506b700 1 -- 172.21.15.3:0/249965035 wait complete. 2023-10-06T04:53:02.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.784+0000 7f14b506b700 1 Processor -- start 2023-10-06T04:53:02.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.784+0000 7f14b506b700 1 -- start start 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14b00b1f80 0x7f14b016d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00c9470 0x7f14b00afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b0151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b015e4d0 con 0x7f14b00c9840 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b015ece0 con 0x7f14b00b1f80 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b016d460 con 0x7f14b00c9470 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b0151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40652/0 (socket says 172.21.15.3:40652) 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 -- 172.21.15.3:0/113950725 learned_addr learned my addr 172.21.15.3:0/113950725 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14af7fe700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00c9470 0x7f14b00afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14affff700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14b00b1f80 0x7f14b016d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 -- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00c9470 msgr2=0x7f14b00afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00c9470 0x7f14b00afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 -- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14b00b1f80 msgr2=0x7f14b016d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14b00b1f80 0x7f14b016d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 -- 172.21.15.3:0/113950725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4009420 con 0x7f14b00c9840 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14af7fe700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00c9470 0x7f14b00afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b486a700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b0151160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009ec0 tx=0x7f14a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f1497fff700 1 -- 172.21.15.3:0/113950725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800cc20 con 0x7f14b00c9840 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f1497fff700 1 -- 172.21.15.3:0/113950725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a800f040 con 0x7f14b00c9840 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b016de30 con 0x7f14b00c9840 2023-10-06T04:53:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f1497fff700 1 -- 172.21.15.3:0/113950725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8014480 con 0x7f14b00c9840 2023-10-06T04:53:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.785+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b015ea00 con 0x7f14b00c9840 2023-10-06T04:53:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.786+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b016d5f0 con 0x7f14b00c9840 2023-10-06T04:53:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.787+0000 7f1497fff700 1 -- 172.21.15.3:0/113950725 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14a8014830 con 0x7f14b00c9840 2023-10-06T04:53:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.787+0000 7f1497fff700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f149c066f30 0x7f149c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.787+0000 7f1497fff700 1 -- 172.21.15.3:0/113950725 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f14a8084dd0 con 0x7f14b00c9840 2023-10-06T04:53:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.787+0000 7f14affff700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f149c066f30 0x7f149c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.788+0000 7f14affff700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f149c066f30 0x7f149c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14a00058d0 tx=0x7f14a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:02.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:02.791+0000 7f1497fff700 1 -- 172.21.15.3:0/113950725 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14a8055cc0 con 0x7f14b00c9840 2023-10-06T04:53:03.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.070+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f14b004c380 con 0x7f14b00c9840 2023-10-06T04:53:03.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.073+0000 7f1497fff700 1 -- 172.21.15.3:0/113950725 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f14a8055850 con 0x7f14b00c9840 2023-10-06T04:53:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f149c066f30 msgr2=0x7f149c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f149c066f30 0x7f149c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14a00058d0 tx=0x7f14a0009040 comp rx=0 tx=0).stop 2023-10-06T04:53:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 msgr2=0x7f14b0151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b0151160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009ec0 tx=0x7f14a800b040 comp rx=0 tx=0).stop 2023-10-06T04:53:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 shutdown_connections 2023-10-06T04:53:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f149c066f30 0x7f149c0693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14b00b1f80 0x7f14b016d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14b00c9840 0x7f14b0151160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 --2- 172.21.15.3:0/113950725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14b00c9470 0x7f14b00afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 >> 172.21.15.3:0/113950725 conn(0x7f14b00cd1f0 msgr2=0x7f14b00b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 shutdown_connections 2023-10-06T04:53:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.075+0000 7f14b506b700 1 -- 172.21.15.3:0/113950725 wait complete. 2023-10-06T04:53:03.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T04:53:03.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.207+0000 7fd033560700 1 Processor -- start 2023-10-06T04:53:03.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.207+0000 7fd033560700 1 -- start start 2023-10-06T04:53:03.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd033560700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15ccc0 0x7fd02c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd033560700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd02c0d06a0 0x7fd02c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd033560700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c0cf6c0 con 0x7fd02c15c8f0 2023-10-06T04:53:03.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd033560700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c0cfb50 con 0x7fd02c0d06a0 2023-10-06T04:53:03.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd033560700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0d0b20 con 0x7fd02c15ccc0 2023-10-06T04:53:03.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd031afd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd02c0d06a0 0x7fd02c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd031afd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd02c0d06a0 0x7fd02c0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40664/0 (socket says 172.21.15.3:40664) 2023-10-06T04:53:03.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.208+0000 7fd031afd700 1 -- 172.21.15.3:0/3350549161 learned_addr learned my addr 172.21.15.3:0/3350549161 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:03.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd031afd700 1 -- 172.21.15.3:0/3350549161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15ccc0 msgr2=0x7fd02c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:03.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd031afd700 1 --2- 172.21.15.3:0/3350549161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15ccc0 0x7fd02c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd031afd700 1 -- 172.21.15.3:0/3350549161 >> v1:172.21.15.153:6789/0 conn(0x7fd02c15c8f0 legacy=0x7fd02c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd031afd700 1 -- 172.21.15.3:0/3350549161 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0ceab0 con 0x7fd02c0d06a0 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd031afd700 1 --2- 172.21.15.3:0/3350549161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0d06a0 0x7fd02c0b3ba0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd024009f20 tx=0x7fd024009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3181b1cfae568c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd023fff700 1 -- 172.21.15.3:0/3350549161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02400e070 con 0x7fd02c0d06a0 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd023fff700 1 -- 172.21.15.3:0/3350549161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02400ba40 con 0x7fd02c0d06a0 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd023fff700 1 -- 172.21.15.3:0/3350549161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0240092e0 con 0x7fd02c0d06a0 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd033560700 1 -- 172.21.15.3:0/3350549161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0d06a0 msgr2=0x7fd02c0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd033560700 1 --2- 172.21.15.3:0/3350549161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0d06a0 0x7fd02c0b3ba0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd024009f20 tx=0x7fd024009840 comp rx=0 tx=0).stop 2023-10-06T04:53:03.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd033560700 1 -- 172.21.15.3:0/3350549161 shutdown_connections 2023-10-06T04:53:03.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd033560700 1 --2- 172.21.15.3:0/3350549161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0d06a0 0x7fd02c0b3ba0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd033560700 1 --2- 172.21.15.3:0/3350549161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15ccc0 0x7fd02c0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.209+0000 7fd033560700 1 -- 172.21.15.3:0/3350549161 >> 172.21.15.3:0/3350549161 conn(0x7fd02c070310 msgr2=0x7fd02c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:03.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.210+0000 7fd033560700 1 -- 172.21.15.3:0/3350549161 shutdown_connections 2023-10-06T04:53:03.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.210+0000 7fd033560700 1 -- 172.21.15.3:0/3350549161 wait complete. 2023-10-06T04:53:03.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 Processor -- start 2023-10-06T04:53:03.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 -- start start 2023-10-06T04:53:03.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd02c0d06a0 0x7fd02c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15c8f0 0x7fd02c14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c15ccc0 0x7fd02c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c158cb0 con 0x7fd02c15ccc0 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c158f90 con 0x7fd02c0d06a0 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.211+0000 7fd033560700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c159270 con 0x7fd02c15c8f0 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c15ccc0 0x7fd02c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c15ccc0 0x7fd02c15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40672/0 (socket says 172.21.15.3:40672) 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 -- 172.21.15.3:0/559289335 learned_addr learned my addr 172.21.15.3:0/559289335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd030afb700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15c8f0 0x7fd02c14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 -- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15c8f0 msgr2=0x7fd02c14ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15c8f0 0x7fd02c14ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 -- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd02c0d06a0 msgr2=0x7fd02c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd0312fc700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd02c0d06a0 0x7fd02c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd02c0d06a0 0x7fd02c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 -- 172.21.15.3:0/559289335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd024009ae0 con 0x7fd02c15ccc0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd0312fc700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd02c0d06a0 0x7fd02c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd031afd700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c15ccc0 0x7fd02c15a030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd02c06eaa0 tx=0x7fd024017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd0217fa700 1 -- 172.21.15.3:0/559289335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024012e60 con 0x7fd02c15ccc0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd0217fa700 1 -- 172.21.15.3:0/559289335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0240092e0 con 0x7fd02c15ccc0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c159550 con 0x7fd02c15ccc0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd0217fa700 1 -- 172.21.15.3:0/559289335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02401e660 con 0x7fd02c15ccc0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.212+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02c159b10 con 0x7fd02c15ccc0 2023-10-06T04:53:03.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.213+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02c0d0f10 con 0x7fd02c15ccc0 2023-10-06T04:53:03.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.214+0000 7fd0217fa700 1 -- 172.21.15.3:0/559289335 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd02401ea10 con 0x7fd02c15ccc0 2023-10-06T04:53:03.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.214+0000 7fd0217fa700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.214+0000 7fd0217fa700 1 -- 172.21.15.3:0/559289335 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd024085ff0 con 0x7fd02c15ccc0 2023-10-06T04:53:03.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.214+0000 7fd0312fc700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.215+0000 7fd0312fc700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd010066f30 0x7fd0100693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd028006fd0 tx=0x7fd028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.218+0000 7fd0217fa700 1 -- 172.21.15.3:0/559289335 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd024053450 con 0x7fd02c15ccc0 2023-10-06T04:53:03.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.489+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd02c159d30 con 0x7fd02c15ccc0 2023-10-06T04:53:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.492+0000 7fd0217fa700 1 -- 172.21.15.3:0/559289335 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd024056a70 con 0x7fd02c15ccc0 2023-10-06T04:53:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.494+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd010066f30 msgr2=0x7fd0100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.494+0000 7fd033560700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd010066f30 0x7fd0100693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd028006fd0 tx=0x7fd028008040 comp rx=0 tx=0).stop 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.494+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c15ccc0 msgr2=0x7fd02c15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.494+0000 7fd033560700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c15ccc0 0x7fd02c15a030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd02c06eaa0 tx=0x7fd024017000 comp rx=0 tx=0).stop 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 shutdown_connections 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd02c0d06a0 0x7fd02c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c15ccc0 0x7fd02c15a030 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 --2- 172.21.15.3:0/559289335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd02c15c8f0 0x7fd02c14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 >> 172.21.15.3:0/559289335 conn(0x7fd02c070310 msgr2=0x7fd02c1584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 shutdown_connections 2023-10-06T04:53:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.495+0000 7fd033560700 1 -- 172.21.15.3:0/559289335 wait complete. 2023-10-06T04:53:03.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:03.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.625+0000 7f976062c700 1 Processor -- start 2023-10-06T04:53:03.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.625+0000 7f976062c700 1 -- start start 2023-10-06T04:53:03.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.626+0000 7f976062c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9758157030 0x7f9758159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.626+0000 7f976062c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f97580b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.626+0000 7f976062c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f975815aeb0 con 0x7f9758157400 2023-10-06T04:53:03.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.626+0000 7f976062c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975815a7a0 con 0x7f9758157030 2023-10-06T04:53:03.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.626+0000 7f976062c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975815aa80 con 0x7f975815a220 2023-10-06T04:53:03.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975dbc7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f97580b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975e3c8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9758157030 0x7f9758159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975ebc9700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9758157400 0x7f9758069340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53916/0 (socket says 172.21.15.3:53916) 2023-10-06T04:53:03.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975ebc9700 1 -- 172.21.15.3:0/2157960130 learned_addr learned my addr 172.21.15.3:0/2157960130 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:03.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975e3c8700 1 -- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 msgr2=0x7f97580b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975e3c8700 1 --2- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f97580b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975e3c8700 1 -- 172.21.15.3:0/2157960130 >> v1:172.21.15.153:6789/0 conn(0x7f9758157400 legacy=0x7f9758069340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975e3c8700 1 -- 172.21.15.3:0/2157960130 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975806dba0 con 0x7f9758157030 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975d3c6700 1 -- 172.21.15.3:0/2157960130 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125058381 0 0) 0x7f975815aeb0 con 0x7f9758157400 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975dbc7700 1 --2- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f97580b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.627+0000 7f975e3c8700 1 --2- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157030 0x7f9758159ce0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9748009e60 tx=0x7f9748009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96da9969f7b3fcec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f975d3c6700 1 -- 172.21.15.3:0/2157960130 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974800e070 con 0x7f9758157030 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f975d3c6700 1 -- 172.21.15.3:0/2157960130 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974800bb60 con 0x7f9758157030 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f975d3c6700 1 -- 172.21.15.3:0/2157960130 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748012430 con 0x7f9758157030 2023-10-06T04:53:03.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 -- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157030 msgr2=0x7f9758159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 --2- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157030 0x7f9758159ce0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9748009e60 tx=0x7f9748009780 comp rx=0 tx=0).stop 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 -- 172.21.15.3:0/2157960130 shutdown_connections 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 --2- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157030 0x7f9758159ce0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 --2- 172.21.15.3:0/2157960130 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f97580b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 -- 172.21.15.3:0/2157960130 >> 172.21.15.3:0/2157960130 conn(0x7f97580ce270 msgr2=0x7f97580ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 -- 172.21.15.3:0/2157960130 shutdown_connections 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.628+0000 7f976062c700 1 -- 172.21.15.3:0/2157960130 wait complete. 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.629+0000 7f976062c700 1 Processor -- start 2023-10-06T04:53:03.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.629+0000 7f976062c700 1 -- start start 2023-10-06T04:53:03.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f976062c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758157030 0x7f97580cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f976062c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157400 0x7f97580b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f976062c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f975815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f976062c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580afac0 con 0x7f9758157030 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f976062c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975815c8e0 con 0x7f9758157400 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f976062c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580cdb40 con 0x7f975815a220 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758157030 0x7f97580cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758157030 0x7f97580cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40678/0 (socket says 172.21.15.3:40678) 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 -- 172.21.15.3:0/50355350 learned_addr learned my addr 172.21.15.3:0/50355350 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975dbc7700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157400 0x7f97580b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975ebc9700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f975815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 -- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 msgr2=0x7f975815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f975815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 -- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157400 msgr2=0x7f97580b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157400 0x7f97580b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.630+0000 7f975e3c8700 1 -- 172.21.15.3:0/50355350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c009420 con 0x7f9758157030 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.631+0000 7f975e3c8700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758157030 0x7f97580cd540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97580676c0 tx=0x7f9748019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.631+0000 7f97567fc700 1 -- 172.21.15.3:0/50355350 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748012cc0 con 0x7f9758157030 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.631+0000 7f97567fc700 1 -- 172.21.15.3:0/50355350 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9748012e40 con 0x7f9758157030 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.631+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9748009a20 con 0x7f9758157030 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.631+0000 7f97567fc700 1 -- 172.21.15.3:0/50355350 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97480276c0 con 0x7f9758157030 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.631+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97580affd0 con 0x7f9758157030 2023-10-06T04:53:03.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.632+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97580644c0 con 0x7f9758157030 2023-10-06T04:53:03.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.632+0000 7f97567fc700 1 -- 172.21.15.3:0/50355350 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9748024070 con 0x7f9758157030 2023-10-06T04:53:03.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.633+0000 7f97567fc700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9744066f30 0x7f97440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.633+0000 7f97567fc700 1 -- 172.21.15.3:0/50355350 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9748086850 con 0x7f9758157030 2023-10-06T04:53:03.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.633+0000 7f975dbc7700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9744066f30 0x7f97440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.633+0000 7f975dbc7700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9744066f30 0x7f97440693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f97580cd300 tx=0x7f974c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.637+0000 7f97567fc700 1 -- 172.21.15.3:0/50355350 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9748053cb0 con 0x7f9758157030 2023-10-06T04:53:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.832+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97580b4170 con 0x7f9758157030 2023-10-06T04:53:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.833+0000 7f97567fc700 1 -- 172.21.15.3:0/50355350 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+13602 (secure 0 0 0) 0x7f97480572d0 con 0x7f9758157030 2023-10-06T04:53:03.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:03.835 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:00.506874+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9744066f30 msgr2=0x7f97440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9744066f30 0x7f97440693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f97580cd300 tx=0x7f974c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758157030 msgr2=0x7f97580cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758157030 0x7f97580cd540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97580676c0 tx=0x7f9748019000 comp rx=0 tx=0).stop 2023-10-06T04:53:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 shutdown_connections 2023-10-06T04:53:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9744066f30 0x7f97440693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f975815a220 0x7f975815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758157030 0x7f97580cd540 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 --2- 172.21.15.3:0/50355350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758157400 0x7f97580b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 >> 172.21.15.3:0/50355350 conn(0x7f97580ce270 msgr2=0x7f97580b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 shutdown_connections 2023-10-06T04:53:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.835+0000 7f976062c700 1 -- 172.21.15.3:0/50355350 wait complete. 2023-10-06T04:53:03.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:53:03.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:53:03.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.970+0000 7f4a6c9b7700 1 Processor -- start 2023-10-06T04:53:03.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.970+0000 7f4a6c9b7700 1 -- start start 2023-10-06T04:53:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6c9b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a680af660 0x7f4a680ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6c9b7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680cd190 0x7f4a680c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6c9b7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a680cb190 con 0x7f4a680af290 2023-10-06T04:53:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6c9b7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a68159950 con 0x7f4a680af660 2023-10-06T04:53:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6c9b7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680cd6a0 con 0x7f4a680cd190 2023-10-06T04:53:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a680af660 0x7f4a680ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a680af660 0x7f4a680ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40688/0 (socket says 172.21.15.3:40688) 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6659c700 1 -- 172.21.15.3:0/677631015 learned_addr learned my addr 172.21.15.3:0/677631015 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6759e700 1 --2- 172.21.15.3:0/677631015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680cd190 0x7f4a680c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6659c700 1 -- 172.21.15.3:0/677631015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680cd190 msgr2=0x7f4a680c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6659c700 1 --2- 172.21.15.3:0/677631015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680cd190 0x7f4a680c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6659c700 1 -- 172.21.15.3:0/677631015 >> v1:172.21.15.153:6789/0 conn(0x7f4a680af290 legacy=0x7f4a680ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.971+0000 7f4a6659c700 1 -- 172.21.15.3:0/677631015 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a680cb3f0 con 0x7f4a680af660 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6659c700 1 --2- 172.21.15.3:0/677631015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a680ccc50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009990 tx=0x7f4a50009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e21f945f86236fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a65d9b700 1 -- 172.21.15.3:0/677631015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50009e80 con 0x7f4a680af660 2023-10-06T04:53:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a65d9b700 1 -- 172.21.15.3:0/677631015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a50005c10 con 0x7f4a680af660 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a65d9b700 1 -- 172.21.15.3:0/677631015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000b650 con 0x7f4a680af660 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/677631015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 msgr2=0x7f4a680ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/677631015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a680ccc50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009990 tx=0x7f4a50009220 comp rx=0 tx=0).stop 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/677631015 shutdown_connections 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/677631015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680cd190 0x7f4a680c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/677631015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a680ccc50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/677631015 >> 172.21.15.3:0/677631015 conn(0x7f4a68089dd0 msgr2=0x7f4a680996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/677631015 shutdown_connections 2023-10-06T04:53:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.972+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/677631015 wait complete. 2023-10-06T04:53:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 Processor -- start 2023-10-06T04:53:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 -- start start 2023-10-06T04:53:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680af290 0x7f4a680c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a6816a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a680cd190 0x7f4a680b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680c8510 con 0x7f4a680af660 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680c7be0 con 0x7f4a680cd190 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6c9b7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680c7ec0 con 0x7f4a680af290 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a6816a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a6816a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40690/0 (socket says 172.21.15.3:40690) 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.974+0000 7f4a6659c700 1 -- 172.21.15.3:0/1948222399 learned_addr learned my addr 172.21.15.3:0/1948222399 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a66d9d700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680af290 0x7f4a680c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6759e700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a680cd190 0x7f4a680b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6659c700 1 -- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680af290 msgr2=0x7f4a680c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6659c700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680af290 0x7f4a680c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6659c700 1 -- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a680cd190 msgr2=0x7f4a680b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6659c700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a680cd190 0x7f4a680b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6659c700 1 -- 172.21.15.3:0/1948222399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50009480 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6659c700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a6816a050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a680cb320 tx=0x7f4a50016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a5effd700 1 -- 172.21.15.3:0/1948222399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50011950 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a5effd700 1 -- 172.21.15.3:0/1948222399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a50020070 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a680c81a0 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a5effd700 1 -- 172.21.15.3:0/1948222399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50011d00 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.975+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a680b23f0 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.976+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a680c86c0 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.977+0000 7f4a5effd700 1 -- 172.21.15.3:0/1948222399 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a50022030 con 0x7f4a680af660 2023-10-06T04:53:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.977+0000 7f4a5effd700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.977+0000 7f4a5effd700 1 -- 172.21.15.3:0/1948222399 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4a50085db0 con 0x7f4a680af660 2023-10-06T04:53:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.977+0000 7f4a66d9d700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.978+0000 7f4a66d9d700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a48066f30 0x7f4a480693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c580 tx=0x7f4a58001b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:03.981+0000 7f4a5effd700 1 -- 172.21.15.3:0/1948222399 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a680c86c0 con 0x7f4a680af660 2023-10-06T04:53:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.175+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4a680c86c0 con 0x7f4a680af660 2023-10-06T04:53:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.176+0000 7f4a5effd700 1 -- 172.21.15.3:0/1948222399 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4a680c86c0 con 0x7f4a680af660 2023-10-06T04:53:04.178 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:53:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.177+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a48066f30 msgr2=0x7f4a480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.177+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a48066f30 0x7f4a480693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c580 tx=0x7f4a58001b20 comp rx=0 tx=0).stop 2023-10-06T04:53:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.177+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 msgr2=0x7f4a6816a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.177+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a6816a050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a680cb320 tx=0x7f4a50016000 comp rx=0 tx=0).stop 2023-10-06T04:53:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.177+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 shutdown_connections 2023-10-06T04:53:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.178+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.178+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a680cd190 0x7f4a680b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.178+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a680af290 0x7f4a680c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.178+0000 7f4a6c9b7700 1 --2- 172.21.15.3:0/1948222399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a680af660 0x7f4a6816a050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.178+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 >> 172.21.15.3:0/1948222399 conn(0x7f4a68089dd0 msgr2=0x7f4a680ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:04.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.178+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 shutdown_connections 2023-10-06T04:53:04.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.178+0000 7f4a6c9b7700 1 -- 172.21.15.3:0/1948222399 wait complete. 2023-10-06T04:53:04.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:53:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.312+0000 7fb0d2c47700 1 Processor -- start 2023-10-06T04:53:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.312+0000 7fb0d2c47700 1 -- start start 2023-10-06T04:53:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.312+0000 7fb0d2c47700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb0cc0681c0 0x7fb0cc1646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.312+0000 7fb0d2c47700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0cc069a80 con 0x7fb0cc143140 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.312+0000 7fb0d2c47700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0cc069150 con 0x7fb0cc142d70 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.312+0000 7fb0d2c47700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0cc0cc880 con 0x7fb0cc0681c0 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0cbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb0cc143140 0x7fb0cc067ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39432/0 (socket says 172.21.15.3:39432) 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0cbfff700 1 -- 172.21.15.3:0/802797072 learned_addr learned my addr 172.21.15.3:0/802797072 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0d11e4700 1 --2- 172.21.15.3:0/802797072 >> v2:172.21.15.3:3300/0 conn(0x7fb0cc0681c0 0x7fb0cc1646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0cb7fe700 1 -- 172.21.15.3:0/802797072 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802459281 0 0) 0x7fb0cc069a80 con 0x7fb0cc143140 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0cb7fe700 1 -- 172.21.15.3:0/802797072 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0b8003570 con 0x7fb0cc143140 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0d11e4700 1 -- 172.21.15.3:0/802797072 >> v1:172.21.15.153:6789/0 conn(0x7fb0cc142d70 legacy=0x7fb0cc0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0d11e4700 1 -- 172.21.15.3:0/802797072 >> v1:172.21.15.3:6789/0 conn(0x7fb0cc143140 legacy=0x7fb0cc067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0d11e4700 1 -- 172.21.15.3:0/802797072 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc069430 con 0x7fb0cc0681c0 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0d11e4700 1 --2- 172.21.15.3:0/802797072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc0681c0 0x7fb0cc1646f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40098e0 tx=0x7fb0b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8260ed934ca06e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0cb7fe700 1 -- 172.21.15.3:0/802797072 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400b640 con 0x7fb0cc0681c0 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0cb7fe700 1 -- 172.21.15.3:0/802797072 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400b7a0 con 0x7fb0cc0681c0 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.313+0000 7fb0cb7fe700 1 -- 172.21.15.3:0/802797072 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400bb40 con 0x7fb0cc0681c0 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.314+0000 7fb0d2c47700 1 -- 172.21.15.3:0/802797072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc0681c0 msgr2=0x7fb0cc1646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.314+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/802797072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc0681c0 0x7fb0cc1646f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40098e0 tx=0x7fb0b40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.314+0000 7fb0d2c47700 1 -- 172.21.15.3:0/802797072 shutdown_connections 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.314+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/802797072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc0681c0 0x7fb0cc1646f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.314+0000 7fb0d2c47700 1 -- 172.21.15.3:0/802797072 >> 172.21.15.3:0/802797072 conn(0x7fb0cc0ce210 msgr2=0x7fb0cc0ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.314+0000 7fb0d2c47700 1 -- 172.21.15.3:0/802797072 shutdown_connections 2023-10-06T04:53:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.314+0000 7fb0d2c47700 1 -- 172.21.15.3:0/802797072 wait complete. 2023-10-06T04:53:04.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.315+0000 7fb0d2c47700 1 Processor -- start 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.315+0000 7fb0d2c47700 1 -- start start 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d2c47700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0cc0681c0 0x7fb0cc13eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d2c47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc142d70 0x7fb0cc0c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d2c47700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0cc143140 0x7fb0cc159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d2c47700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc0c7310 con 0x7fb0cc142d70 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d2c47700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc0c6be0 con 0x7fb0cc143140 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d2c47700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc0cb6f0 con 0x7fb0cc0681c0 2023-10-06T04:53:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc142d70 0x7fb0cc0c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc142d70 0x7fb0cc0c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40696/0 (socket says 172.21.15.3:40696) 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 -- 172.21.15.3:0/1122377282 learned_addr learned my addr 172.21.15.3:0/1122377282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d11e4700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0cc143140 0x7fb0cc159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0d09e3700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0cc0681c0 0x7fb0cc13eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 -- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0cc0681c0 msgr2=0x7fb0cc13eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0cc0681c0 0x7fb0cc13eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 -- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0cc143140 msgr2=0x7fb0cc159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0cc143140 0x7fb0cc159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.316+0000 7fb0cbfff700 1 -- 172.21.15.3:0/1122377282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4009420 con 0x7fb0cc142d70 2023-10-06T04:53:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.317+0000 7fb0cbfff700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc142d70 0x7fb0cc0c66a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00cc70 tx=0x7fb0bc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.317+0000 7fb0c8ff9700 1 -- 172.21.15.3:0/1122377282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00ce80 con 0x7fb0cc142d70 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.317+0000 7fb0c8ff9700 1 -- 172.21.15.3:0/1122377282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0bc00ab10 con 0x7fb0cc142d70 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.317+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc0cb9d0 con 0x7fb0cc142d70 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.317+0000 7fb0c8ff9700 1 -- 172.21.15.3:0/1122377282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0070f0 con 0x7fb0cc142d70 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.317+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc0cb310 con 0x7fb0cc142d70 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.318+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0cc04c380 con 0x7fb0cc142d70 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.318+0000 7fb0c8ff9700 1 -- 172.21.15.3:0/1122377282 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0bc007520 con 0x7fb0cc142d70 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.319+0000 7fb0c8ff9700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb0ac066f30 0x7fb0ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.319+0000 7fb0c8ff9700 1 -- 172.21.15.3:0/1122377282 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb0bc0847a0 con 0x7fb0cc142d70 2023-10-06T04:53:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.319+0000 7fb0d09e3700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb0ac066f30 0x7fb0ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.319+0000 7fb0d09e3700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb0ac066f30 0x7fb0ac0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40037c0 tx=0x7fb0c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.323+0000 7fb0c8ff9700 1 -- 172.21.15.3:0/1122377282 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0bc051c80 con 0x7fb0cc142d70 2023-10-06T04:53:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.516+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0cc06f450 con 0x7fb0cc142d70 2023-10-06T04:53:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.517+0000 7fb0c8ff9700 1 -- 172.21.15.3:0/1122377282 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0bc0552a0 con 0x7fb0cc142d70 2023-10-06T04:53:04.519 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb0ac066f30 msgr2=0x7fb0ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb0ac066f30 0x7fb0ac0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40037c0 tx=0x7fb0c4009040 comp rx=0 tx=0).stop 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc142d70 msgr2=0x7fb0cc0c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc142d70 0x7fb0cc0c66a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00cc70 tx=0x7fb0bc001a40 comp rx=0 tx=0).stop 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 shutdown_connections 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb0ac066f30 0x7fb0ac0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0cc143140 0x7fb0cc159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0cc0681c0 0x7fb0cc13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 --2- 172.21.15.3:0/1122377282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0cc142d70 0x7fb0cc0c66a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 >> 172.21.15.3:0/1122377282 conn(0x7fb0cc0ce210 msgr2=0x7fb0cc0adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 shutdown_connections 2023-10-06T04:53:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.519+0000 7fb0d2c47700 1 -- 172.21.15.3:0/1122377282 wait complete. 2023-10-06T04:53:04.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:53:04.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.650+0000 7fa64777c700 1 Processor -- start 2023-10-06T04:53:04.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.650+0000 7fa64777c700 1 -- start start 2023-10-06T04:53:04.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa64777c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa64015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa64777c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa64015ccc0 0x7fa6400d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa64777c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6400cf6c0 con 0x7fa6400d06a0 2023-10-06T04:53:04.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa64777c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6400cfb50 con 0x7fa64015ccc0 2023-10-06T04:53:04.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa64777c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64015e050 con 0x7fa64015c8f0 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa644d17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa64015ccc0 0x7fa6400d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa644d17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa64015ccc0 0x7fa6400d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40704/0 (socket says 172.21.15.3:40704) 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa644d17700 1 -- 172.21.15.3:0/2244111088 learned_addr learned my addr 172.21.15.3:0/2244111088 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa645d19700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa6400d06a0 0x7fa64006d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39444/0 (socket says 172.21.15.3:39444) 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa645518700 1 --2- 172.21.15.3:0/2244111088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa64015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa644d17700 1 -- 172.21.15.3:0/2244111088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 msgr2=0x7fa64015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa644d17700 1 --2- 172.21.15.3:0/2244111088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa64015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa644d17700 1 -- 172.21.15.3:0/2244111088 >> v1:172.21.15.3:6789/0 conn(0x7fa6400d06a0 legacy=0x7fa64006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa644d17700 1 -- 172.21.15.3:0/2244111088 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6400d0af0 con 0x7fa64015ccc0 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.651+0000 7fa63ffff700 1 -- 172.21.15.3:0/2244111088 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150530245 0 0) 0x7fa6400cf6c0 con 0x7fa6400d06a0 2023-10-06T04:53:04.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa644d17700 1 --2- 172.21.15.3:0/2244111088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400d0160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6340099b0 tx=0x7fa634009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f510a0685c5ff4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:04.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa63ffff700 1 -- 172.21.15.3:0/2244111088 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400b860 con 0x7fa64015ccc0 2023-10-06T04:53:04.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa63ffff700 1 -- 172.21.15.3:0/2244111088 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63400b9c0 con 0x7fa64015ccc0 2023-10-06T04:53:04.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa63ffff700 1 -- 172.21.15.3:0/2244111088 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400bd60 con 0x7fa64015ccc0 2023-10-06T04:53:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 -- 172.21.15.3:0/2244111088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 msgr2=0x7fa6400d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 --2- 172.21.15.3:0/2244111088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400d0160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6340099b0 tx=0x7fa634009250 comp rx=0 tx=0).stop 2023-10-06T04:53:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 -- 172.21.15.3:0/2244111088 shutdown_connections 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 --2- 172.21.15.3:0/2244111088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa64015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 --2- 172.21.15.3:0/2244111088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400d0160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 -- 172.21.15.3:0/2244111088 >> 172.21.15.3:0/2244111088 conn(0x7fa640070310 msgr2=0x7fa640070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 -- 172.21.15.3:0/2244111088 shutdown_connections 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.652+0000 7fa64777c700 1 -- 172.21.15.3:0/2244111088 wait complete. 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.653+0000 7fa64777c700 1 Processor -- start 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.653+0000 7fa64777c700 1 -- start start 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa64777c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6400d06a0 0x7fa640154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa64777c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa6400b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa64777c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa64777c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa640159f80 con 0x7fa64015ccc0 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa64777c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa640159c50 con 0x7fa6400d06a0 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa64777c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400b5950 con 0x7fa64015c8f0 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa645d19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa645d19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40716/0 (socket says 172.21.15.3:40716) 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.654+0000 7fa645d19700 1 -- 172.21.15.3:0/286975430 learned_addr learned my addr 172.21.15.3:0/286975430 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa645518700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6400d06a0 0x7fa640154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa644d17700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa6400b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa645d19700 1 -- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 msgr2=0x7fa6400b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa645d19700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa6400b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa645d19700 1 -- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6400d06a0 msgr2=0x7fa640154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa645d19700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6400d06a0 0x7fa640154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa645d19700 1 -- 172.21.15.3:0/286975430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340094f0 con 0x7fa64015ccc0 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa645d19700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400c55d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa63000cd50 tx=0x7fa630001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa63d7fa700 1 -- 172.21.15.3:0/286975430 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630005df0 con 0x7fa64015ccc0 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa63d7fa700 1 -- 172.21.15.3:0/286975430 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63000a8b0 con 0x7fa64015ccc0 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6400b5c30 con 0x7fa64015ccc0 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa63d7fa700 1 -- 172.21.15.3:0/286975430 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63000ac70 con 0x7fa64015ccc0 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.655+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa640154f20 con 0x7fa64015ccc0 2023-10-06T04:53:04.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.656+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64015a0f0 con 0x7fa64015ccc0 2023-10-06T04:53:04.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.657+0000 7fa63d7fa700 1 -- 172.21.15.3:0/286975430 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa630007300 con 0x7fa64015ccc0 2023-10-06T04:53:04.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.657+0000 7fa63d7fa700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa62c066f30 0x7fa62c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.657+0000 7fa63d7fa700 1 -- 172.21.15.3:0/286975430 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa630084420 con 0x7fa64015ccc0 2023-10-06T04:53:04.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.657+0000 7fa645518700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa62c066f30 0x7fa62c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:04.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.658+0000 7fa645518700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa62c066f30 0x7fa62c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa638006fd0 tx=0x7fa638008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:04.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.661+0000 7fa63d7fa700 1 -- 172.21.15.3:0/286975430 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa630051900 con 0x7fa64015ccc0 2023-10-06T04:53:04.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.858+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa640071680 con 0x7fa64015ccc0 2023-10-06T04:53:04.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.859+0000 7fa63d7fa700 1 -- 172.21.15.3:0/286975430 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa630054f20 con 0x7fa64015ccc0 2023-10-06T04:53:04.861 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:04.861 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:53:04.861 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:04.861 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:53:04.861 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:53:04.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:04.861 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:04.862 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:53:04.862 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:53:04.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:04.862 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa62c066f30 msgr2=0x7fa62c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa62c066f30 0x7fa62c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa638006fd0 tx=0x7fa638008040 comp rx=0 tx=0).stop 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 msgr2=0x7fa6400c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400c55d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa63000cd50 tx=0x7fa630001b10 comp rx=0 tx=0).stop 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 shutdown_connections 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa62c066f30 0x7fa62c0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6400d06a0 0x7fa640154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa64015ccc0 0x7fa6400c55d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 --2- 172.21.15.3:0/286975430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa64015c8f0 0x7fa6400b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:04.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 >> 172.21.15.3:0/286975430 conn(0x7fa640070310 msgr2=0x7fa6400afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:04.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 shutdown_connections 2023-10-06T04:53:04.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.861+0000 7fa64777c700 1 -- 172.21.15.3:0/286975430 wait complete. 2023-10-06T04:53:04.874 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:53:04.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:53:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.996+0000 7fcb61ebf700 1 Processor -- start 2023-10-06T04:53:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.996+0000 7fcb61ebf700 1 -- start start 2023-10-06T04:53:04.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.996+0000 7fcb61ebf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157460 0x7fcb5c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:04.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.996+0000 7fcb61ebf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb5c15a8f0 0x7fcb5c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.996+0000 7fcb61ebf700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c0b3b80 con 0x7fcb5c157090 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb61ebf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c0b3dc0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb61ebf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0b40a0 con 0x7fcb5c157460 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb5c15a8f0 0x7fcb5c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb5c15a8f0 0x7fcb5c0b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40732/0 (socket says 172.21.15.3:40732) 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3528524115 learned_addr learned my addr 172.21.15.3:0/3528524115 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5affd700 1 --2- 172.21.15.3:0/3528524115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157460 0x7fcb5c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3528524115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157460 msgr2=0x7fcb5c0687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 --2- 172.21.15.3:0/3528524115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157460 0x7fcb5c0687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3528524115 >> v1:172.21.15.153:6789/0 conn(0x7fcb5c157090 legacy=0x7fcb5c0b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3528524115 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c067cf0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5bfff700 1 --2- 172.21.15.3:0/3528524115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c0b1450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009f20 tx=0x7fcb54009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7621e5487e2221e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5a7fc700 1 -- 172.21.15.3:0/3528524115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400e070 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5a7fc700 1 -- 172.21.15.3:0/3528524115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5400ba80 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.997+0000 7fcb5a7fc700 1 -- 172.21.15.3:0/3528524115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540092e0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3528524115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 msgr2=0x7fcb5c0b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3528524115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c0b1450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009f20 tx=0x7fcb54009840 comp rx=0 tx=0).stop 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3528524115 shutdown_connections 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3528524115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c0b1450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3528524115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157460 0x7fcb5c0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3528524115 >> 172.21.15.3:0/3528524115 conn(0x7fcb5c06a360 msgr2=0x7fcb5c06c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3528524115 shutdown_connections 2023-10-06T04:53:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.998+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3528524115 wait complete. 2023-10-06T04:53:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.999+0000 7fcb61ebf700 1 Processor -- start 2023-10-06T04:53:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:04.999+0000 7fcb61ebf700 1 -- start start 2023-10-06T04:53:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb61ebf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157090 0x7fcb5c15e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb61ebf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c157460 0x7fcb5c145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb61ebf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb61ebf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c16e000 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb61ebf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c16d3f0 con 0x7fcb5c157460 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb61ebf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c16d6d0 con 0x7fcb5c157090 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c146020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40734/0 (socket says 172.21.15.3:40734) 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3742370156 learned_addr learned my addr 172.21.15.3:0/3742370156 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5affd700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c157460 0x7fcb5c145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5b7fe700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157090 0x7fcb5c15e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157090 msgr2=0x7fcb5c15e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157090 0x7fcb5c15e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c157460 msgr2=0x7fcb5c145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c157460 0x7fcb5c145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 -- 172.21.15.3:0/3742370156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009ae0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5b7fe700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157090 0x7fcb5c15e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.000+0000 7fcb5bfff700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c146020 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b6c0 tx=0x7fcb54018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.001+0000 7fcb437fe700 1 -- 172.21.15.3:0/3742370156 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540032f0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.001+0000 7fcb437fe700 1 -- 172.21.15.3:0/3742370156 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54008dd0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.001+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c16d9b0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.001+0000 7fcb437fe700 1 -- 172.21.15.3:0/3742370156 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54025670 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.001+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c15e9a0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.002+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c16e170 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.002+0000 7fcb437fe700 1 -- 172.21.15.3:0/3742370156 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb54003040 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.002+0000 7fcb437fe700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcb48066f30 0x7fcb480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.003+0000 7fcb437fe700 1 -- 172.21.15.3:0/3742370156 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fcb54085cb0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.003+0000 7fcb5b7fe700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcb48066f30 0x7fcb480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.003+0000 7fcb5b7fe700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcb48066f30 0x7fcb480693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c007fd0 tx=0x7fcb4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.007+0000 7fcb437fe700 1 -- 172.21.15.3:0/3742370156 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb540531c0 con 0x7fcb5c15a8f0 2023-10-06T04:53:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:05.201+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcb5c071680 con 0x7fcb5c15a8f0 2023-10-06T04:53:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.083+0000 7fcb437fe700 1 -- 172.21.15.3:0/3742370156 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcb540567e0 con 0x7fcb5c15a8f0 2023-10-06T04:53:06.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.087+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcb48066f30 msgr2=0x7fcb480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:06.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.087+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcb48066f30 0x7fcb480693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c007fd0 tx=0x7fcb4c009040 comp rx=0 tx=0).stop 2023-10-06T04:53:06.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.087+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 msgr2=0x7fcb5c146020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:06.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.087+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c146020 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b6c0 tx=0x7fcb54018000 comp rx=0 tx=0).stop 2023-10-06T04:53:06.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 shutdown_connections 2023-10-06T04:53:06.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcb48066f30 0x7fcb480693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb5c15a8f0 0x7fcb5c146020 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb5c157090 0x7fcb5c15e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 --2- 172.21.15.3:0/3742370156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c157460 0x7fcb5c145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 >> 172.21.15.3:0/3742370156 conn(0x7fcb5c06a360 msgr2=0x7fcb5c155c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 shutdown_connections 2023-10-06T04:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.090+0000 7fcb61ebf700 1 -- 172.21.15.3:0/3742370156 wait complete. 2023-10-06T04:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:53:06.108 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:53:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.238+0000 7ff84b50c700 1 Processor -- start 2023-10-06T04:53:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.238+0000 7ff84b50c700 1 -- start start 2023-10-06T04:53:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff84b50c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff844150690 0x7ff844150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff84b50c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84415a290 con 0x7ff84415d540 2023-10-06T04:53:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff84b50c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8440d0020 con 0x7ff8440cba50 2023-10-06T04:53:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff84b50c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff844159f40 con 0x7ff844150690 2023-10-06T04:53:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff849aa9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff84415d540 0x7ff8440af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39446/0 (socket says 172.21.15.3:39446) 2023-10-06T04:53:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff849aa9700 1 -- 172.21.15.3:0/1851161769 learned_addr learned my addr 172.21.15.3:0/1851161769 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994239241 0 0) 0x7ff84415a290 con 0x7ff84415d540 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff82c003570 con 0x7ff84415d540 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff8492a8700 1 --2- 172.21.15.3:0/1851161769 >> v2:172.21.15.153:3300/0 conn(0x7ff844150690 0x7ff844150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2653831220 0 0) 0x7ff82c003570 con 0x7ff84415d540 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 >> v2:172.21.15.153:3300/0 conn(0x7ff844150690 msgr2=0x7ff844150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 --2- 172.21.15.3:0/1851161769 >> v2:172.21.15.153:3300/0 conn(0x7ff844150690 0x7ff844150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 >> v1:172.21.15.153:6789/0 conn(0x7ff8440cba50 legacy=0x7ff84415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84415dcb0 con 0x7ff84415d540 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.239+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7ff834001470 con 0x7ff84415d540 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 >> v1:172.21.15.3:6789/0 conn(0x7ff84415d540 legacy=0x7ff8440af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff82c003a60 0x7ff82c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff82c004810 0x7ff82c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c0051c0 0x7ff82c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844159f40 con 0x7ff82c003a60 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c00c160 con 0x7ff82c004810 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c00c320 con 0x7ff82c0051c0 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff8492a8700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff82c003a60 0x7ff82c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff848aa7700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff82c004810 0x7ff82c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff849aa9700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c0051c0 0x7ff82c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff8492a8700 1 -- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c0051c0 msgr2=0x7ff82c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff8492a8700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c0051c0 0x7ff82c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff8492a8700 1 -- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff82c004810 msgr2=0x7ff82c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff8492a8700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff82c004810 0x7ff82c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff8492a8700 1 -- 172.21.15.3:0/1851161769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c003570 con 0x7ff82c003a60 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff848aa7700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff82c004810 0x7ff82c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff849aa9700 1 -- 172.21.15.3:0/1851161769 reap_dead start 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff8492a8700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff82c003a60 0x7ff82c003e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e210 tx=0x7ff83c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6c38d9ac6371a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00eac0 con 0x7ff82c003a60 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c00f040 con 0x7ff82c003a60 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.240+0000 7ff843fff700 1 -- 172.21.15.3:0/1851161769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00ee70 con 0x7ff82c003a60 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.241+0000 7ff84b50c700 1 -- 172.21.15.3:0/1851161769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff82c003a60 msgr2=0x7ff82c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.241+0000 7ff84b50c700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff82c003a60 0x7ff82c003e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e210 tx=0x7ff83c00b040 comp rx=0 tx=0).stop 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.241+0000 7ff84b50c700 1 -- 172.21.15.3:0/1851161769 shutdown_connections 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.241+0000 7ff84b50c700 1 --2- 172.21.15.3:0/1851161769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff82c003a60 0x7ff82c003e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.241+0000 7ff84b50c700 1 -- 172.21.15.3:0/1851161769 >> 172.21.15.3:0/1851161769 conn(0x7ff844070310 msgr2=0x7ff844070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.241+0000 7ff84b50c700 1 -- 172.21.15.3:0/1851161769 shutdown_connections 2023-10-06T04:53:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.241+0000 7ff84b50c700 1 -- 172.21.15.3:0/1851161769 wait complete. 2023-10-06T04:53:06.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.242+0000 7ff84b50c700 1 Processor -- start 2023-10-06T04:53:06.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.242+0000 7ff84b50c700 1 -- start start 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff84b50c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c003a60 0x7ff8440b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff84b50c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440af630 0x7ff8440c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff84b50c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8440cd1c0 0x7ff8440cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff84b50c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440cdb70 con 0x7ff8440cd1c0 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff84b50c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844155490 con 0x7ff8440af630 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff84b50c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff844154880 con 0x7ff82c003a60 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8440cd1c0 0x7ff8440cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8440cd1c0 0x7ff8440cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40750/0 (socket says 172.21.15.3:40750) 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 -- 172.21.15.3:0/3363588269 learned_addr learned my addr 172.21.15.3:0/3363588269 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff848aa7700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440af630 0x7ff8440c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff8492a8700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c003a60 0x7ff8440b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 -- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c003a60 msgr2=0x7ff8440b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c003a60 0x7ff8440b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 -- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440af630 msgr2=0x7ff8440c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440af630 0x7ff8440c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 -- 172.21.15.3:0/3363588269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844154b60 con 0x7ff8440cd1c0 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.243+0000 7ff849aa9700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8440cd1c0 0x7ff8440cd630 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8440b2f80 tx=0x7ff834000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.244+0000 7ff8417fa700 1 -- 172.21.15.3:0/3363588269 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8340114f0 con 0x7ff8440cd1c0 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.244+0000 7ff8417fa700 1 -- 172.21.15.3:0/3363588269 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff834011650 con 0x7ff8440cd1c0 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.244+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff844154e40 con 0x7ff8440cd1c0 2023-10-06T04:53:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.244+0000 7ff8417fa700 1 -- 172.21.15.3:0/3363588269 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834011a10 con 0x7ff8440cd1c0 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.244+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8440b40e0 con 0x7ff8440cd1c0 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.245+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff844155640 con 0x7ff8440cd1c0 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.245+0000 7ff8417fa700 1 -- 172.21.15.3:0/3363588269 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff83401e440 con 0x7ff8440cd1c0 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.245+0000 7ff8417fa700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff830066f30 0x7ff8300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.246+0000 7ff8417fa700 1 -- 172.21.15.3:0/3363588269 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5576+0+0 (secure 0 0 0) 0x7ff834083a90 con 0x7ff8440cd1c0 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.246+0000 7ff8492a8700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff830066f30 0x7ff8300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.246+0000 7ff8492a8700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff830066f30 0x7ff8300693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00afd0 tx=0x7ff83c01a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.250+0000 7ff8417fa700 1 -- 172.21.15.3:0/3363588269 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff844155640 con 0x7ff8440cd1c0 2023-10-06T04:53:06.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:06.443+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff844155600 con 0x7ff8440cd1c0 2023-10-06T04:53:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.089+0000 7ff8417fa700 1 -- 172.21.15.3:0/3363588269 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff834054810 con 0x7ff8440cd1c0 2023-10-06T04:53:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff830066f30 msgr2=0x7ff8300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff830066f30 0x7ff8300693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00afd0 tx=0x7ff83c01a000 comp rx=0 tx=0).stop 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8440cd1c0 msgr2=0x7ff8440cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8440cd1c0 0x7ff8440cd630 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8440b2f80 tx=0x7ff834000bc0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 shutdown_connections 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff830066f30 0x7ff8300693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8440cd1c0 0x7ff8440cd630 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff82c003a60 0x7ff8440b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 --2- 172.21.15.3:0/3363588269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440af630 0x7ff8440c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.102+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 >> 172.21.15.3:0/3363588269 conn(0x7ff844070310 msgr2=0x7ff844158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.103+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 shutdown_connections 2023-10-06T04:53:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.103+0000 7ff84b50c700 1 -- 172.21.15.3:0/3363588269 wait complete. 2023-10-06T04:53:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:53:07.121 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:53:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.245+0000 7f499c99d700 1 Processor -- start 2023-10-06T04:53:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.245+0000 7f499c99d700 1 -- start start 2023-10-06T04:53:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f499c99d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f49981571a0 0x7f49980cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f499c99d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4998157570 0x7f49980ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f499c99d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49980b21e0 0x7f49980b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f499c99d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499815be90 con 0x7f4998157570 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f499c99d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49980b2b80 con 0x7f49981571a0 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f499c99d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499815b030 con 0x7f49980b21e0 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f4995d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4998157570 0x7f49980ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f4995d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4998157570 0x7f49980ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40756/0 (socket says 172.21.15.3:40756) 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.246+0000 7f4995d9b700 1 -- 172.21.15.3:0/207337434 learned_addr learned my addr 172.21.15.3:0/207337434 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f499659c700 1 --2- 172.21.15.3:0/207337434 >> v2:172.21.15.153:3300/0 conn(0x7f49981571a0 0x7f49980cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f4996d9d700 1 --2- 172.21.15.3:0/207337434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49980b21e0 0x7f49980b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f4995d9b700 1 -- 172.21.15.3:0/207337434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49980b21e0 msgr2=0x7f49980b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f4995d9b700 1 --2- 172.21.15.3:0/207337434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49980b21e0 0x7f49980b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f4995d9b700 1 -- 172.21.15.3:0/207337434 >> v2:172.21.15.153:3300/0 conn(0x7f49981571a0 msgr2=0x7f49980cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f4995d9b700 1 --2- 172.21.15.3:0/207337434 >> v2:172.21.15.153:3300/0 conn(0x7f49981571a0 0x7f49980cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f4995d9b700 1 -- 172.21.15.3:0/207337434 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499815b280 con 0x7f4998157570 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f4995d9b700 1 --2- 172.21.15.3:0/207337434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998157570 0x7f49980ccb30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f499806d3b0 tx=0x7f4980009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5979d6a7363b755 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f499559a700 1 -- 172.21.15.3:0/207337434 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000b640 con 0x7f4998157570 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f499559a700 1 -- 172.21.15.3:0/207337434 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498000b7a0 con 0x7f4998157570 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.247+0000 7f499559a700 1 -- 172.21.15.3:0/207337434 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000bb60 con 0x7f4998157570 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 -- 172.21.15.3:0/207337434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998157570 msgr2=0x7f49980ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 --2- 172.21.15.3:0/207337434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998157570 0x7f49980ccb30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f499806d3b0 tx=0x7f4980009250 comp rx=0 tx=0).stop 2023-10-06T04:53:07.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 -- 172.21.15.3:0/207337434 shutdown_connections 2023-10-06T04:53:07.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 --2- 172.21.15.3:0/207337434 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49980b21e0 0x7f49980b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 --2- 172.21.15.3:0/207337434 >> v2:172.21.15.153:3300/0 conn(0x7f49981571a0 0x7f49980cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 --2- 172.21.15.3:0/207337434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998157570 0x7f49980ccb30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 -- 172.21.15.3:0/207337434 >> 172.21.15.3:0/207337434 conn(0x7f499806df50 msgr2=0x7f49980aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:07.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 -- 172.21.15.3:0/207337434 shutdown_connections 2023-10-06T04:53:07.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.248+0000 7f499c99d700 1 -- 172.21.15.3:0/207337434 wait complete. 2023-10-06T04:53:07.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.249+0000 7f499c99d700 1 Processor -- start 2023-10-06T04:53:07.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.249+0000 7f499c99d700 1 -- start start 2023-10-06T04:53:07.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980b21e0 0x7f49980b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:07.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49981571a0 0x7f49980c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:07.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998157570 0x7f4998067470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499816acc0 con 0x7f49980b21e0 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980c9180 con 0x7f4998157570 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499816de10 con 0x7f49981571a0 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980b21e0 0x7f49980b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980b21e0 0x7f49980b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40764/0 (socket says 172.21.15.3:40764) 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 -- 172.21.15.3:0/3908878808 learned_addr learned my addr 172.21.15.3:0/3908878808 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f4995d9b700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49981571a0 0x7f49980c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 -- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49981571a0 msgr2=0x7f49980c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f4996d9d700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998157570 0x7f4998067470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49981571a0 0x7f49980c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 -- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998157570 msgr2=0x7f4998067470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:07.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998157570 0x7f4998067470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 -- 172.21.15.3:0/3908878808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800094f0 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f4995d9b700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49981571a0 0x7f49980c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499659c700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980b21e0 0x7f49980b1bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f499000ec80 tx=0x7f49900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f498e7fc700 1 -- 172.21.15.3:0/3908878808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499000c9d0 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f498e7fc700 1 -- 172.21.15.3:0/3908878808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499000cb30 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49980b1090 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f498e7fc700 1 -- 172.21.15.3:0/3908878808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990013470 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.250+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499816d7c0 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.251+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499804c380 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.252+0000 7f498e7fc700 1 -- 172.21.15.3:0/3908878808 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4990013820 con 0x7f49980b21e0 2023-10-06T04:53:07.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.252+0000 7f498e7fc700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4978066f30 0x7f49780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:07.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.252+0000 7f498e7fc700 1 -- 172.21.15.3:0/3908878808 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f4990084d70 con 0x7f49980b21e0 2023-10-06T04:53:07.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.253+0000 7f4995d9b700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4978066f30 0x7f49780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:07.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.253+0000 7f4995d9b700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4978066f30 0x7f49780693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f499806d3b0 tx=0x7f4980017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:07.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.257+0000 7f498e7fc700 1 -- 172.21.15.3:0/3908878808 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4990051ee0 con 0x7f49980b21e0 2023-10-06T04:53:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:07.495+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f499815b560 con 0x7f49980b21e0 2023-10-06T04:53:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.100+0000 7f498e7fc700 1 -- 172.21.15.3:0/3908878808 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4990055500 con 0x7f49980b21e0 2023-10-06T04:53:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4978066f30 msgr2=0x7f49780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4978066f30 0x7f49780693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f499806d3b0 tx=0x7f4980017000 comp rx=0 tx=0).stop 2023-10-06T04:53:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980b21e0 msgr2=0x7f49980b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980b21e0 0x7f49980b1bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f499000ec80 tx=0x7f49900091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 shutdown_connections 2023-10-06T04:53:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4978066f30 0x7f49780693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998157570 0x7f4998067470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980b21e0 0x7f49980b1bd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 --2- 172.21.15.3:0/3908878808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49981571a0 0x7f49980c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.104+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 >> 172.21.15.3:0/3908878808 conn(0x7f499806df50 msgr2=0x7f4998153e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.105+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 shutdown_connections 2023-10-06T04:53:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.105+0000 7f499c99d700 1 -- 172.21.15.3:0/3908878808 wait complete. 2023-10-06T04:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T04:53:08.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:08.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.249+0000 7fc96cf86700 1 Processor -- start 2023-10-06T04:53:08.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.249+0000 7fc96cf86700 1 -- start start 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96cf86700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96cf86700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9680b0590 0x7fc9680b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96cf86700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9680b0ce0 con 0x7fc9680cd950 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96cf86700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680cccc0 con 0x7fc9680b0590 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96cf86700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680cb820 con 0x7fc96815cb50 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc965d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9680b0590 0x7fc9680b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc966d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc9680cd950 0x7fc96806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39454/0 (socket says 172.21.15.3:39454) 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc966d9d700 1 -- 172.21.15.3:0/1515701161 learned_addr learned my addr 172.21.15.3:0/1515701161 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96659c700 1 --2- 172.21.15.3:0/1515701161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc965d9b700 1 -- 172.21.15.3:0/1515701161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 msgr2=0x7fc9680b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96559a700 1 -- 172.21.15.3:0/1515701161 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714302775 0 0) 0x7fc9680b0ce0 con 0x7fc9680cd950 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc965d9b700 1 --2- 172.21.15.3:0/1515701161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc965d9b700 1 -- 172.21.15.3:0/1515701161 >> v1:172.21.15.3:6789/0 conn(0x7fc9680cd950 legacy=0x7fc96806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc965d9b700 1 -- 172.21.15.3:0/1515701161 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680b0a00 con 0x7fc9680b0590 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96659c700 1 --2- 172.21.15.3:0/1515701161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc965d9b700 1 --2- 172.21.15.3:0/1515701161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680b0590 0x7fc9680b2200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc9500098e0 tx=0x7fc9500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631a2d408abac40b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.250+0000 7fc96559a700 1 -- 172.21.15.3:0/1515701161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950009e80 con 0x7fc9680b0590 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96559a700 1 -- 172.21.15.3:0/1515701161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95000b860 con 0x7fc9680b0590 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96559a700 1 -- 172.21.15.3:0/1515701161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000bbe0 con 0x7fc9680b0590 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 -- 172.21.15.3:0/1515701161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680b0590 msgr2=0x7fc9680b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1515701161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680b0590 0x7fc9680b2200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc9500098e0 tx=0x7fc9500091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 -- 172.21.15.3:0/1515701161 shutdown_connections 2023-10-06T04:53:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1515701161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1515701161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680b0590 0x7fc9680b2200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 -- 172.21.15.3:0/1515701161 >> 172.21.15.3:0/1515701161 conn(0x7fc968070310 msgr2=0x7fc968070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 -- 172.21.15.3:0/1515701161 shutdown_connections 2023-10-06T04:53:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.251+0000 7fc96cf86700 1 -- 172.21.15.3:0/1515701161 wait complete. 2023-10-06T04:53:08.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.252+0000 7fc96cf86700 1 Processor -- start 2023-10-06T04:53:08.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.252+0000 7fc96cf86700 1 -- start start 2023-10-06T04:53:08.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc96cf86700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9680b0590 0x7fc9680ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc96cf86700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680cd950 0x7fc9681478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc96cf86700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc96cf86700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96815d050 con 0x7fc9680cd950 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc96cf86700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc968147e10 con 0x7fc9680b0590 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc96cf86700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680d0f30 con 0x7fc96815cb50 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680cd950 0x7fc9681478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680cd950 0x7fc9681478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40786/0 (socket says 172.21.15.3:40786) 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 -- 172.21.15.3:0/1138845850 learned_addr learned my addr 172.21.15.3:0/1138845850 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc966d9d700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 -- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 msgr2=0x7fc9680d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 -- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9680b0590 msgr2=0x7fc9680ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9680b0590 0x7fc9680ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.253+0000 7fc965d9b700 1 -- 172.21.15.3:0/1138845850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950009420 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.254+0000 7fc965d9b700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680cd950 0x7fc9681478d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc968064a20 tx=0x7fc950017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.254+0000 7fc95e7fc700 1 -- 172.21.15.3:0/1138845850 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000bbe0 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.254+0000 7fc95e7fc700 1 -- 172.21.15.3:0/1138845850 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950011420 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.254+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9680cefb0 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.254+0000 7fc95e7fc700 1 -- 172.21.15.3:0/1138845850 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500117a0 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.254+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9680cf570 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.255+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc968071680 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.255+0000 7fc95e7fc700 1 -- 172.21.15.3:0/1138845850 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc950011bd0 con 0x7fc9680cd950 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.255+0000 7fc95e7fc700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc948066f30 0x7fc9480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.256+0000 7fc95e7fc700 1 -- 172.21.15.3:0/1138845850 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc950085c20 con 0x7fc9680cd950 2023-10-06T04:53:08.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.256+0000 7fc96659c700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc948066f30 0x7fc9480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.256+0000 7fc96659c700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc948066f30 0x7fc9480693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc960006ea0 tx=0x7fc960008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.260+0000 7fc95e7fc700 1 -- 172.21.15.3:0/1138845850 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc950052e60 con 0x7fc9680cd950 2023-10-06T04:53:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.455+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc96815d1c0 con 0x7fc9680cd950 2023-10-06T04:53:08.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.456+0000 7fc95e7fc700 1 -- 172.21.15.3:0/1138845850 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16831 (secure 0 0 0) 0x7fc96815d1c0 con 0x7fc9680cd950 2023-10-06T04:53:08.458 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:08.458 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:07.496022+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:05.231194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:06.473725+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:08.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc948066f30 msgr2=0x7fc9480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc948066f30 0x7fc9480693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc960006ea0 tx=0x7fc960008040 comp rx=0 tx=0).stop 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680cd950 msgr2=0x7fc9681478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680cd950 0x7fc9681478d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc968064a20 tx=0x7fc950017000 comp rx=0 tx=0).stop 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 shutdown_connections 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc948066f30 0x7fc9480693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9680b0590 0x7fc9680ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc96815cb50 0x7fc9680d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 --2- 172.21.15.3:0/1138845850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9680cd950 0x7fc9681478d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.458+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 >> 172.21.15.3:0/1138845850 conn(0x7fc968070310 msgr2=0x7fc96815b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.459+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 shutdown_connections 2023-10-06T04:53:08.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.459+0000 7fc96cf86700 1 -- 172.21.15.3:0/1138845850 wait complete. 2023-10-06T04:53:08.472 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.589+0000 7efd43bf9700 1 Processor -- start 2023-10-06T04:53:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.590+0000 7efd43bf9700 1 -- start start 2023-10-06T04:53:08.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.590+0000 7efd43bf9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd3c159500 0x7efd3c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.590+0000 7efd43bf9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efd3c1598d0 0x7efd3c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.590+0000 7efd43bf9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.590+0000 7efd43bf9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0cb7b0 con 0x7efd3c159500 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.590+0000 7efd43bf9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0cc5c0 con 0x7efd3c1598d0 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.590+0000 7efd43bf9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0cb9b0 con 0x7efd3c0b2220 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd3c159500 0x7efd3c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd3c159500 0x7efd3c0cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40792/0 (socket says 172.21.15.3:40792) 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 -- 172.21.15.3:0/2643668950 learned_addr learned my addr 172.21.15.3:0/2643668950 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd42196700 1 --2- 172.21.15.3:0/2643668950 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41194700 1 --2- 172.21.15.3:0/2643668950 >> v2:172.21.15.153:3300/0 conn(0x7efd3c1598d0 0x7efd3c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 -- 172.21.15.3:0/2643668950 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 msgr2=0x7efd3c0b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 --2- 172.21.15.3:0/2643668950 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 -- 172.21.15.3:0/2643668950 >> v2:172.21.15.153:3300/0 conn(0x7efd3c1598d0 msgr2=0x7efd3c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 --2- 172.21.15.3:0/2643668950 >> v2:172.21.15.153:3300/0 conn(0x7efd3c1598d0 0x7efd3c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 -- 172.21.15.3:0/2643668950 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0cbc90 con 0x7efd3c159500 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41194700 1 --2- 172.21.15.3:0/2643668950 >> v2:172.21.15.153:3300/0 conn(0x7efd3c1598d0 0x7efd3c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd41995700 1 --2- 172.21.15.3:0/2643668950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c0cad30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efd3c15d130 tx=0x7efd34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f95e82337205b59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd40993700 1 -- 172.21.15.3:0/2643668950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34009e70 con 0x7efd3c159500 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd40993700 1 -- 172.21.15.3:0/2643668950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3400b860 con 0x7efd3c159500 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.591+0000 7efd40993700 1 -- 172.21.15.3:0/2643668950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400bc00 con 0x7efd3c159500 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 -- 172.21.15.3:0/2643668950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 msgr2=0x7efd3c0cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2643668950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c0cad30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efd3c15d130 tx=0x7efd34009250 comp rx=0 tx=0).stop 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 -- 172.21.15.3:0/2643668950 shutdown_connections 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2643668950 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2643668950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c0cad30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2643668950 >> v2:172.21.15.153:3300/0 conn(0x7efd3c1598d0 0x7efd3c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 -- 172.21.15.3:0/2643668950 >> 172.21.15.3:0/2643668950 conn(0x7efd3c070310 msgr2=0x7efd3c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 -- 172.21.15.3:0/2643668950 shutdown_connections 2023-10-06T04:53:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.592+0000 7efd43bf9700 1 -- 172.21.15.3:0/2643668950 wait complete. 2023-10-06T04:53:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.593+0000 7efd43bf9700 1 Processor -- start 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.593+0000 7efd43bf9700 1 -- start start 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd43bf9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd43bf9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c16d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd43bf9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd3c1598d0 0x7efd3c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd43bf9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0c5830 con 0x7efd3c159500 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd43bf9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0b58d0 con 0x7efd3c1598d0 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd43bf9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0c5500 con 0x7efd3c0b2220 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c16d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c16d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40808/0 (socket says 172.21.15.3:40808) 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 -- 172.21.15.3:0/2546275587 learned_addr learned my addr 172.21.15.3:0/2546275587 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41995700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd42196700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd3c1598d0 0x7efd3c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 -- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 msgr2=0x7efd3c0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 -- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd3c1598d0 msgr2=0x7efd3c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd3c1598d0 0x7efd3c0ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 -- 172.21.15.3:0/2546275587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340094f0 con 0x7efd3c159500 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd41194700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c16d890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efd3000d860 tx=0x7efd30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd39ffb700 1 -- 172.21.15.3:0/2546275587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300125f0 con 0x7efd3c159500 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.594+0000 7efd39ffb700 1 -- 172.21.15.3:0/2546275587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30012750 con 0x7efd3c159500 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.595+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c16ddd0 con 0x7efd3c159500 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.595+0000 7efd39ffb700 1 -- 172.21.15.3:0/2546275587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30012af0 con 0x7efd3c159500 2023-10-06T04:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.595+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c0b5e50 con 0x7efd3c159500 2023-10-06T04:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.596+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c0b6070 con 0x7efd3c159500 2023-10-06T04:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.596+0000 7efd39ffb700 1 -- 172.21.15.3:0/2546275587 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd30014430 con 0x7efd3c159500 2023-10-06T04:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.596+0000 7efd39ffb700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd28066f30 0x7efd280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.596+0000 7efd39ffb700 1 -- 172.21.15.3:0/2546275587 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efd30084a30 con 0x7efd3c159500 2023-10-06T04:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.596+0000 7efd41995700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd28066f30 0x7efd280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.597+0000 7efd41995700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd28066f30 0x7efd280693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efd3c15d130 tx=0x7efd34017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.600+0000 7efd39ffb700 1 -- 172.21.15.3:0/2546275587 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd30051b40 con 0x7efd3c159500 2023-10-06T04:53:08.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.794+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd3c071680 con 0x7efd3c159500 2023-10-06T04:53:08.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.795+0000 7efd39ffb700 1 -- 172.21.15.3:0/2546275587 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16831 (secure 0 0 0) 0x7efd30055160 con 0x7efd3c159500 2023-10-06T04:53:08.797 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:08.797 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:07.496022+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:05.231194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:06.473725+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:08.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.797+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd28066f30 msgr2=0x7efd280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.797+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd28066f30 0x7efd280693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efd3c15d130 tx=0x7efd34017000 comp rx=0 tx=0).stop 2023-10-06T04:53:08.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.797+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 msgr2=0x7efd3c16d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.797+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c16d890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efd3000d860 tx=0x7efd30008040 comp rx=0 tx=0).stop 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 shutdown_connections 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd28066f30 0x7efd280693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd3c1598d0 0x7efd3c0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd3c0b2220 0x7efd3c0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 --2- 172.21.15.3:0/2546275587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd3c159500 0x7efd3c16d890 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 >> 172.21.15.3:0/2546275587 conn(0x7efd3c070310 msgr2=0x7efd3c0cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 shutdown_connections 2023-10-06T04:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.798+0000 7efd43bf9700 1 -- 172.21.15.3:0/2546275587 wait complete. 2023-10-06T04:53:08.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:53:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.931+0000 7f995b59e700 1 Processor -- start 2023-10-06T04:53:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.931+0000 7f995b59e700 1 -- start start 2023-10-06T04:53:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995b59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f995c0af290 0x7f995c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995b59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f995c0c5c20 0x7f995c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995b59e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995c0cb190 con 0x7f995c0af660 2023-10-06T04:53:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995b59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995c0c65a0 con 0x7f995c0c5c20 2023-10-06T04:53:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995b59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995c159950 con 0x7f995c0af290 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f995c0c5c20 0x7f995c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f995c0c5c20 0x7f995c0c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40810/0 (socket says 172.21.15.3:40810) 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995ad9d700 1 -- 172.21.15.3:0/4162935672 learned_addr learned my addr 172.21.15.3:0/4162935672 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.932+0000 7f995a59c700 1 --2- 172.21.15.3:0/4162935672 >> v2:172.21.15.153:3300/0 conn(0x7f995c0af290 0x7f995c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995ad9d700 1 -- 172.21.15.3:0/4162935672 >> v2:172.21.15.153:3300/0 conn(0x7f995c0af290 msgr2=0x7f995c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995ad9d700 1 --2- 172.21.15.3:0/4162935672 >> v2:172.21.15.153:3300/0 conn(0x7f995c0af290 0x7f995c0ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995ad9d700 1 -- 172.21.15.3:0/4162935672 >> v1:172.21.15.153:6789/0 conn(0x7f995c0af660 legacy=0x7f995c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995ad9d700 1 -- 172.21.15.3:0/4162935672 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0cd6a0 con 0x7f995c0c5c20 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995ad9d700 1 --2- 172.21.15.3:0/4162935672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0c5c20 0x7f995c0c6060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f995c089580 tx=0x7f9954009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef249b62dc2761f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995959a700 1 -- 172.21.15.3:0/4162935672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400b820 con 0x7f995c0c5c20 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995959a700 1 -- 172.21.15.3:0/4162935672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995400b980 con 0x7f995c0c5c20 2023-10-06T04:53:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995959a700 1 -- 172.21.15.3:0/4162935672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400bd20 con 0x7f995c0c5c20 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995b59e700 1 -- 172.21.15.3:0/4162935672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0c5c20 msgr2=0x7f995c0c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995b59e700 1 --2- 172.21.15.3:0/4162935672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0c5c20 0x7f995c0c6060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f995c089580 tx=0x7f9954009250 comp rx=0 tx=0).stop 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995b59e700 1 -- 172.21.15.3:0/4162935672 shutdown_connections 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995b59e700 1 --2- 172.21.15.3:0/4162935672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0c5c20 0x7f995c0c6060 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995b59e700 1 --2- 172.21.15.3:0/4162935672 >> v2:172.21.15.153:3300/0 conn(0x7f995c0af290 0x7f995c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.933+0000 7f995b59e700 1 -- 172.21.15.3:0/4162935672 >> 172.21.15.3:0/4162935672 conn(0x7f995c089dd0 msgr2=0x7f995c0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.934+0000 7f995b59e700 1 -- 172.21.15.3:0/4162935672 shutdown_connections 2023-10-06T04:53:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.934+0000 7f995b59e700 1 -- 172.21.15.3:0/4162935672 wait complete. 2023-10-06T04:53:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 Processor -- start 2023-10-06T04:53:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 -- start start 2023-10-06T04:53:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f995c0af290 0x7f995c0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0af660 0x7f995c0b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f995c0c5c20 0x7f995c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c150950 con 0x7f995c0af660 2023-10-06T04:53:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0c93f0 con 0x7f995c0af290 2023-10-06T04:53:08.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f995b59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0c9880 con 0x7f995c0c5c20 2023-10-06T04:53:08.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.935+0000 7f9959d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0af660 0x7f995c0b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0af660 0x7f995c0b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40820/0 (socket says 172.21.15.3:40820) 2023-10-06T04:53:08.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 -- 172.21.15.3:0/3546554851 learned_addr learned my addr 172.21.15.3:0/3546554851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f995ad9d700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f995c0c5c20 0x7f995c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 -- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f995c0c5c20 msgr2=0x7f995c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f995a59c700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f995c0af290 0x7f995c0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f995c0c5c20 0x7f995c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 -- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f995c0af290 msgr2=0x7f995c0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f995c0af290 0x7f995c0d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 -- 172.21.15.3:0/3546554851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540094f0 con 0x7f995c0af660 2023-10-06T04:53:08.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f9959d9b700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0af660 0x7f995c0b5740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f994400c510 tx=0x7f994400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f99527fc700 1 -- 172.21.15.3:0/3546554851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944011590 con 0x7f995c0af660 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f99527fc700 1 -- 172.21.15.3:0/3546554851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944011710 con 0x7f995c0af660 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c0c87e0 con 0x7f995c0af660 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f99527fc700 1 -- 172.21.15.3:0/3546554851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944011ad0 con 0x7f995c0af660 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.936+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c0c8da0 con 0x7f995c0af660 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.937+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c0c9560 con 0x7f995c0af660 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.938+0000 7f99527fc700 1 -- 172.21.15.3:0/3546554851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9944012420 con 0x7f995c0af660 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.938+0000 7f99527fc700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f993c066f30 0x7f993c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.938+0000 7f99527fc700 1 -- 172.21.15.3:0/3546554851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f99440841c0 con 0x7f995c0af660 2023-10-06T04:53:08.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.938+0000 7f995a59c700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f993c066f30 0x7f993c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:08.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.939+0000 7f995a59c700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f993c066f30 0x7f993c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f994c00d430 tx=0x7f994c007560 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:08.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:08.943+0000 7f99527fc700 1 -- 172.21.15.3:0/3546554851 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99440512d0 con 0x7f995c0af660 2023-10-06T04:53:09.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:09.183+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f995c04c380 con 0x7f995c0af660 2023-10-06T04:53:09.938 INFO:tasks.ceph.mds.b.smithi153.stderr:2023-10-06T04:53:09.936+0000 7fe7399a6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:53:10.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.107+0000 7f99527fc700 1 -- 172.21.15.3:0/3546554851 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f99440548f0 con 0x7f995c0af660 2023-10-06T04:53:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f993c066f30 msgr2=0x7f993c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f993c066f30 0x7f993c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f994c00d430 tx=0x7f994c007560 comp rx=0 tx=0).stop 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0af660 msgr2=0x7f995c0b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0af660 0x7f995c0b5740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f994400c510 tx=0x7f994400c8d0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 shutdown_connections 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f993c066f30 0x7f993c0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f995c0af290 0x7f995c0d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f995c0c5c20 0x7f995c06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 --2- 172.21.15.3:0/3546554851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c0af660 0x7f995c0b5740 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 >> 172.21.15.3:0/3546554851 conn(0x7f995c089dd0 msgr2=0x7f995c158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 shutdown_connections 2023-10-06T04:53:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.110+0000 7f995b59e700 1 -- 172.21.15.3:0/3546554851 wait complete. 2023-10-06T04:53:10.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:53:10.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.254+0000 7fe0019e1700 1 Processor -- start 2023-10-06T04:53:10.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.254+0000 7fe0019e1700 1 -- start start 2023-10-06T04:53:10.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.254+0000 7fe0019e1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc150690 0x7fdffc150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:10.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.254+0000 7fe0019e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdffc0cba50 0x7fdffc15d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.254+0000 7fe0019e1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdffc15da80 con 0x7fdffc15a720 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.254+0000 7fe0019e1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc15a290 con 0x7fdffc0cba50 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.254+0000 7fe0019e1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0d0020 con 0x7fdffc150690 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffa7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdffc0cba50 0x7fdffc15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdffc15a720 0x7fdffc0af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39458/0 (socket says 172.21.15.3:39458) 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffb7fe700 1 -- 172.21.15.3:0/3476677700 learned_addr learned my addr 172.21.15.3:0/3476677700 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffaffd700 1 --2- 172.21.15.3:0/3476677700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc150690 0x7fdffc150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffa7fc700 1 -- 172.21.15.3:0/3476677700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc150690 msgr2=0x7fdffc150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffa7fc700 1 --2- 172.21.15.3:0/3476677700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc150690 0x7fdffc150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffa7fc700 1 -- 172.21.15.3:0/3476677700 >> v1:172.21.15.3:6789/0 conn(0x7fdffc15a720 legacy=0x7fdffc0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffa7fc700 1 -- 172.21.15.3:0/3476677700 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc159f40 con 0x7fdffc0cba50 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdff9ffb700 1 -- 172.21.15.3:0/3476677700 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027014857 0 0) 0x7fdffc15da80 con 0x7fdffc15a720 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffaffd700 1 --2- 172.21.15.3:0/3476677700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc150690 0x7fdffc150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdffa7fc700 1 --2- 172.21.15.3:0/3476677700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc0cba50 0x7fdffc15d540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098e0 tx=0x7fdfe40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b365b21b658c53a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdff9ffb700 1 -- 172.21.15.3:0/3476677700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4009e80 con 0x7fdffc0cba50 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdff9ffb700 1 -- 172.21.15.3:0/3476677700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400b860 con 0x7fdffc0cba50 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.255+0000 7fdff9ffb700 1 -- 172.21.15.3:0/3476677700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400bbe0 con 0x7fdffc0cba50 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 -- 172.21.15.3:0/3476677700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc0cba50 msgr2=0x7fdffc15d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3476677700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc0cba50 0x7fdffc15d540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098e0 tx=0x7fdfe40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 -- 172.21.15.3:0/3476677700 shutdown_connections 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3476677700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc150690 0x7fdffc150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3476677700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc0cba50 0x7fdffc15d540 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 -- 172.21.15.3:0/3476677700 >> 172.21.15.3:0/3476677700 conn(0x7fdffc070310 msgr2=0x7fdffc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:10.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 -- 172.21.15.3:0/3476677700 shutdown_connections 2023-10-06T04:53:10.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.256+0000 7fe0019e1700 1 -- 172.21.15.3:0/3476677700 wait complete. 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.257+0000 7fe0019e1700 1 Processor -- start 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.257+0000 7fe0019e1700 1 -- start start 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fe0019e1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc0cba50 0x7fdffc0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fe0019e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc150690 0x7fdffc1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fe0019e1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc15a720 0x7fdffc153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fe0019e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc150dd0 con 0x7fdffc150690 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fe0019e1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc156780 con 0x7fdffc0cba50 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fe0019e1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc156a60 con 0x7fdffc15a720 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc150690 0x7fdffc1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc150690 0x7fdffc1530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40846/0 (socket says 172.21.15.3:40846) 2023-10-06T04:53:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 -- 172.21.15.3:0/3552723480 learned_addr learned my addr 172.21.15.3:0/3552723480 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffaffd700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc0cba50 0x7fdffc0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffb7fe700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc15a720 0x7fdffc153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 -- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc15a720 msgr2=0x7fdffc153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc15a720 0x7fdffc153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 -- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc0cba50 msgr2=0x7fdffc0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc0cba50 0x7fdffc0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 -- 172.21.15.3:0/3552723480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009420 con 0x7fdffc150690 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffaffd700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc0cba50 0x7fdffc0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffb7fe700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc15a720 0x7fdffc153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.258+0000 7fdffa7fc700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc150690 0x7fdffc1530d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0c71f0 tx=0x7fdfe4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.259+0000 7fdff2ffd700 1 -- 172.21.15.3:0/3552723480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400bbe0 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.259+0000 7fdff2ffd700 1 -- 172.21.15.3:0/3552723480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4011420 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.259+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc156d40 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.259+0000 7fdff2ffd700 1 -- 172.21.15.3:0/3552723480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40117c0 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.259+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc1559a0 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.259+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdffc150f80 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.260+0000 7fdff2ffd700 1 -- 172.21.15.3:0/3552723480 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfe4011bb0 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.260+0000 7fdff2ffd700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdfdc066f30 0x7fdfdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.260+0000 7fdff2ffd700 1 -- 172.21.15.3:0/3552723480 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdfe4085ee0 con 0x7fdffc150690 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.261+0000 7fdffaffd700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdfdc066f30 0x7fdfdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.261+0000 7fdffaffd700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdfdc066f30 0x7fdfdc0693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00d640 tx=0x7fdfec007670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.265+0000 7fdff2ffd700 1 -- 172.21.15.3:0/3552723480 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdffc150f80 con 0x7fdffc150690 2023-10-06T04:53:10.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:10.501+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdffc04c380 con 0x7fdffc150690 2023-10-06T04:53:11.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.108+0000 7fdff2ffd700 1 -- 172.21.15.3:0/3552723480 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdfe4011e60 con 0x7fdffc150690 2023-10-06T04:53:11.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdfdc066f30 msgr2=0x7fdfdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdfdc066f30 0x7fdfdc0693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00d640 tx=0x7fdfec007670 comp rx=0 tx=0).stop 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc150690 msgr2=0x7fdffc1530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc150690 0x7fdffc1530d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0c71f0 tx=0x7fdfe4017000 comp rx=0 tx=0).stop 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 shutdown_connections 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdfdc066f30 0x7fdfdc0693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc0cba50 0x7fdffc0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc15a720 0x7fdffc153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 --2- 172.21.15.3:0/3552723480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdffc150690 0x7fdffc1530d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 >> 172.21.15.3:0/3552723480 conn(0x7fdffc070310 msgr2=0x7fdffc0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 shutdown_connections 2023-10-06T04:53:11.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.111+0000 7fe0019e1700 1 -- 172.21.15.3:0/3552723480 wait complete. 2023-10-06T04:53:11.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:11.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.248+0000 7f83f1c36700 1 Processor -- start 2023-10-06T04:53:11.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.248+0000 7f83f1c36700 1 -- start start 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83f1c36700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f83ec15c8f0 0x7f83ec15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83f1c36700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f83ec15ccc0 0x7f83ec0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83f1c36700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0cf6c0 con 0x7f83ec0d06a0 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83f1c36700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0cfb50 con 0x7f83ec15ccc0 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83f1c36700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0d0b20 con 0x7f83ec15c8f0 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83eaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f83ec15ccc0 0x7f83ec0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83ebfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f83ec0d06a0 0x7f83ec06d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39474/0 (socket says 172.21.15.3:39474) 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83ebfff700 1 -- 172.21.15.3:0/2574137383 learned_addr learned my addr 172.21.15.3:0/2574137383 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83eb7fe700 1 --2- 172.21.15.3:0/2574137383 >> v2:172.21.15.153:3300/0 conn(0x7f83ec15c8f0 0x7f83ec15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83eaffd700 1 -- 172.21.15.3:0/2574137383 >> v2:172.21.15.153:3300/0 conn(0x7f83ec15c8f0 msgr2=0x7f83ec15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83eaffd700 1 --2- 172.21.15.3:0/2574137383 >> v2:172.21.15.153:3300/0 conn(0x7f83ec15c8f0 0x7f83ec15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83eaffd700 1 -- 172.21.15.3:0/2574137383 >> v1:172.21.15.3:6789/0 conn(0x7f83ec0d06a0 legacy=0x7f83ec06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83eaffd700 1 -- 172.21.15.3:0/2574137383 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec15e050 con 0x7f83ec15ccc0 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83ea7fc700 1 -- 172.21.15.3:0/2574137383 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181810613 0 0) 0x7f83ec0cf6c0 con 0x7f83ec0d06a0 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83eaffd700 1 --2- 172.21.15.3:0/2574137383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0d0160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f83e00098e0 tx=0x7f83e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffc747b8e9592b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.249+0000 7f83ea7fc700 1 -- 172.21.15.3:0/2574137383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0009e80 con 0x7f83ec15ccc0 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83ea7fc700 1 -- 172.21.15.3:0/2574137383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e000b820 con 0x7f83ec15ccc0 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83ea7fc700 1 -- 172.21.15.3:0/2574137383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000bba0 con 0x7f83ec15ccc0 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 -- 172.21.15.3:0/2574137383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 msgr2=0x7f83ec0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2574137383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0d0160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f83e00098e0 tx=0x7f83e00091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 -- 172.21.15.3:0/2574137383 shutdown_connections 2023-10-06T04:53:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2574137383 >> v2:172.21.15.153:3300/0 conn(0x7f83ec15c8f0 0x7f83ec15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2574137383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0d0160 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 -- 172.21.15.3:0/2574137383 >> 172.21.15.3:0/2574137383 conn(0x7f83ec070310 msgr2=0x7f83ec070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 -- 172.21.15.3:0/2574137383 shutdown_connections 2023-10-06T04:53:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.250+0000 7f83f1c36700 1 -- 172.21.15.3:0/2574137383 wait complete. 2023-10-06T04:53:11.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.251+0000 7f83f1c36700 1 Processor -- start 2023-10-06T04:53:11.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.251+0000 7f83f1c36700 1 -- start start 2023-10-06T04:53:11.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83f1c36700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83ec0d06a0 0x7f83ec0c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83f1c36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83ec15c8f0 0x7f83ec0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83f1c36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83f1c36700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec15bde0 con 0x7f83ec15ccc0 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83f1c36700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0ca9f0 con 0x7f83ec15c8f0 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83f1c36700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0cacd0 con 0x7f83ec0d06a0 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40856/0 (socket says 172.21.15.3:40856) 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83ebfff700 1 -- 172.21.15.3:0/2886939805 learned_addr learned my addr 172.21.15.3:0/2886939805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83eb7fe700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83ec0d06a0 0x7f83ec0c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.252+0000 7f83eaffd700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83ec15c8f0 0x7f83ec0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83ebfff700 1 -- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83ec0d06a0 msgr2=0x7f83ec0c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83ebfff700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83ec0d06a0 0x7f83ec0c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83ebfff700 1 -- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83ec15c8f0 msgr2=0x7f83ec0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83ebfff700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83ec15c8f0 0x7f83ec0c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83ebfff700 1 -- 172.21.15.3:0/2886939805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0009420 con 0x7f83ec15ccc0 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83eaffd700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83ec15c8f0 0x7f83ec0c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83ebfff700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0ca3c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83e400cbf0 tx=0x7f83e40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83d37fe700 1 -- 172.21.15.3:0/2886939805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400a910 con 0x7f83ec15ccc0 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83d37fe700 1 -- 172.21.15.3:0/2886939805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e400aa70 con 0x7f83ec15ccc0 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83d37fe700 1 -- 172.21.15.3:0/2886939805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400ae30 con 0x7f83ec15ccc0 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec0cafb0 con 0x7f83ec15ccc0 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.253+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec0cb570 con 0x7f83ec15ccc0 2023-10-06T04:53:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.254+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec0cb760 con 0x7f83ec15ccc0 2023-10-06T04:53:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.254+0000 7f83d37fe700 1 -- 172.21.15.3:0/2886939805 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83e4007300 con 0x7f83ec15ccc0 2023-10-06T04:53:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.255+0000 7f83d37fe700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83d8066f30 0x7f83d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.255+0000 7f83d37fe700 1 -- 172.21.15.3:0/2886939805 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83e40849d0 con 0x7f83ec15ccc0 2023-10-06T04:53:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.255+0000 7f83eb7fe700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83d8066f30 0x7f83d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.256+0000 7f83eb7fe700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83d8066f30 0x7f83d80693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83dc006fd0 tx=0x7f83dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.259+0000 7f83d37fe700 1 -- 172.21.15.3:0/2886939805 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83e4051b60 con 0x7f83ec15ccc0 2023-10-06T04:53:11.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.496+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83ec071680 con 0x7f83ec15ccc0 2023-10-06T04:53:11.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.496+0000 7f83d37fe700 1 -- 172.21.15.3:0/2886939805 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f83e4055180 con 0x7f83ec15ccc0 2023-10-06T04:53:11.498 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:11.498 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83d8066f30 msgr2=0x7f83d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83d8066f30 0x7f83d80693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83dc006fd0 tx=0x7f83dc008040 comp rx=0 tx=0).stop 2023-10-06T04:53:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 msgr2=0x7f83ec0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0ca3c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83e400cbf0 tx=0x7f83e40019b0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 shutdown_connections 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83d8066f30 0x7f83d80693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83ec15ccc0 0x7f83ec0ca3c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83ec0d06a0 0x7f83ec0c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 --2- 172.21.15.3:0/2886939805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83ec15c8f0 0x7f83ec0c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.498+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 >> 172.21.15.3:0/2886939805 conn(0x7f83ec070310 msgr2=0x7f83ec0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.499+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 shutdown_connections 2023-10-06T04:53:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.499+0000 7f83f1c36700 1 -- 172.21.15.3:0/2886939805 wait complete. 2023-10-06T04:53:11.502 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:11.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:11.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.630+0000 7feda1c74700 1 Processor -- start 2023-10-06T04:53:11.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.630+0000 7feda1c74700 1 -- start start 2023-10-06T04:53:11.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.630+0000 7feda1c74700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed9c15ccc0 0x7fed9c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.630+0000 7feda1c74700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.630+0000 7feda1c74700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9c0cf6c0 con 0x7fed9c15c8f0 2023-10-06T04:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.630+0000 7feda1c74700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9c0cfb50 con 0x7fed9c15ccc0 2023-10-06T04:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.630+0000 7feda1c74700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0d0ad0 con 0x7fed9c0d06a0 2023-10-06T04:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed9c15c8f0 0x7fed9c15db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39490/0 (socket says 172.21.15.3:39490) 2023-10-06T04:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9b7fe700 1 -- 172.21.15.3:0/1120571123 learned_addr learned my addr 172.21.15.3:0/1120571123 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9affd700 1 --2- 172.21.15.3:0/1120571123 >> v2:172.21.15.3:3300/0 conn(0x7fed9c15ccc0 0x7fed9c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9bfff700 1 --2- 172.21.15.3:0/1120571123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9a7fc700 1 -- 172.21.15.3:0/1120571123 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122512535 0 0) 0x7fed9c0cf6c0 con 0x7fed9c15c8f0 2023-10-06T04:53:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9a7fc700 1 -- 172.21.15.3:0/1120571123 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed84003570 con 0x7fed9c15c8f0 2023-10-06T04:53:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9affd700 1 -- 172.21.15.3:0/1120571123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 msgr2=0x7fed9c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9affd700 1 --2- 172.21.15.3:0/1120571123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9affd700 1 -- 172.21.15.3:0/1120571123 >> v1:172.21.15.3:6789/0 conn(0x7fed9c15c8f0 legacy=0x7fed9c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9affd700 1 -- 172.21.15.3:0/1120571123 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c0ceab0 con 0x7fed9c15ccc0 2023-10-06T04:53:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9affd700 1 --2- 172.21.15.3:0/1120571123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c0d0160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fed900098e0 tx=0x7fed900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4161502e322a00f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9a7fc700 1 -- 172.21.15.3:0/1120571123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90011630 con 0x7fed9c15ccc0 2023-10-06T04:53:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9a7fc700 1 -- 172.21.15.3:0/1120571123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90011790 con 0x7fed9c15ccc0 2023-10-06T04:53:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.631+0000 7fed9a7fc700 1 -- 172.21.15.3:0/1120571123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90011b30 con 0x7fed9c15ccc0 2023-10-06T04:53:11.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 -- 172.21.15.3:0/1120571123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 msgr2=0x7fed9c0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 --2- 172.21.15.3:0/1120571123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c0d0160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fed900098e0 tx=0x7fed900091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 -- 172.21.15.3:0/1120571123 shutdown_connections 2023-10-06T04:53:11.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 --2- 172.21.15.3:0/1120571123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 --2- 172.21.15.3:0/1120571123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c0d0160 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 -- 172.21.15.3:0/1120571123 >> 172.21.15.3:0/1120571123 conn(0x7fed9c070310 msgr2=0x7fed9c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 -- 172.21.15.3:0/1120571123 shutdown_connections 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.632+0000 7feda1c74700 1 -- 172.21.15.3:0/1120571123 wait complete. 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.633+0000 7feda1c74700 1 Processor -- start 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.633+0000 7feda1c74700 1 -- start start 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7feda1c74700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7feda1c74700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed9c15c8f0 0x7fed9c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7feda1c74700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7feda1c74700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c169d40 con 0x7fed9c15ccc0 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7feda1c74700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c169fb0 con 0x7fed9c15c8f0 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7feda1c74700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c16a290 con 0x7fed9c0d06a0 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40870/0 (socket says 172.21.15.3:40870) 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 -- 172.21.15.3:0/1843379988 learned_addr learned my addr 172.21.15.3:0/1843379988 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9b7fe700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9affd700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed9c15c8f0 0x7fed9c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 -- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 msgr2=0x7fed9c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 -- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed9c15c8f0 msgr2=0x7fed9c15d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed9c15c8f0 0x7fed9c15d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9bfff700 1 -- 172.21.15.3:0/1843379988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90009420 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.634+0000 7fed9affd700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed9c15c8f0 0x7fed9c15d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.635+0000 7fed9bfff700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c15d600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed94009e90 tx=0x7fed9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.635+0000 7fed837fe700 1 -- 172.21.15.3:0/1843379988 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9400cc20 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.635+0000 7fed837fe700 1 -- 172.21.15.3:0/1843379988 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9400f040 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.635+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c16a570 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.635+0000 7fed837fe700 1 -- 172.21.15.3:0/1843379988 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94014480 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.635+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c1567c0 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.636+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9c071680 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.636+0000 7fed837fe700 1 -- 172.21.15.3:0/1843379988 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed94014830 con 0x7fed9c15ccc0 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.637+0000 7fed837fe700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed88066f30 0x7fed880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.637+0000 7fed837fe700 1 -- 172.21.15.3:0/1843379988 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fed94085100 con 0x7fed9c15ccc0 2023-10-06T04:53:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.637+0000 7fed9b7fe700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed88066f30 0x7fed880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.637+0000 7fed9b7fe700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed88066f30 0x7fed880693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed8c007fd0 tx=0x7fed8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.641+0000 7fed837fe700 1 -- 172.21.15.3:0/1843379988 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed94055ca0 con 0x7fed9c15ccc0 2023-10-06T04:53:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.839+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed9c0cdcb0 con 0x7fed9c15ccc0 2023-10-06T04:53:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.840+0000 7fed837fe700 1 -- 172.21.15.3:0/1843379988 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16818 (secure 0 0 0) 0x7fed94059e60 con 0x7fed9c15ccc0 2023-10-06T04:53:11.842 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:11.842 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:09.096476+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:05.231194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:06.473725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:11.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed88066f30 msgr2=0x7fed880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed88066f30 0x7fed880693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed8c007fd0 tx=0x7fed8c009040 comp rx=0 tx=0).stop 2023-10-06T04:53:11.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 msgr2=0x7fed9c15d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c15d600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed94009e90 tx=0x7fed9400b040 comp rx=0 tx=0).stop 2023-10-06T04:53:11.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 shutdown_connections 2023-10-06T04:53:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed88066f30 0x7fed880693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed9c15ccc0 0x7fed9c15d600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed9c0d06a0 0x7fed9c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 --2- 172.21.15.3:0/1843379988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed9c15c8f0 0x7fed9c15d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 >> 172.21.15.3:0/1843379988 conn(0x7fed9c070310 msgr2=0x7fed9c1570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 shutdown_connections 2023-10-06T04:53:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.842+0000 7feda1c74700 1 -- 172.21.15.3:0/1843379988 wait complete. 2023-10-06T04:53:11.856 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:53:11.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:53:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.974+0000 7fe52a1a5700 1 Processor -- start 2023-10-06T04:53:11.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.974+0000 7fe52a1a5700 1 -- start start 2023-10-06T04:53:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe52a1a5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe524159500 0x7fe5240cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe52a1a5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe5241598d0 0x7fe5240cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe52a1a5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5240b2220 0x7fe5240b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe52a1a5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5240cb7b0 con 0x7fe524159500 2023-10-06T04:53:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe52a1a5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5240cc5c0 con 0x7fe5241598d0 2023-10-06T04:53:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe52a1a5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240cb9b0 con 0x7fe5240b2220 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe524159500 0x7fe5240cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe524159500 0x7fe5240cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40886/0 (socket says 172.21.15.3:40886) 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 -- 172.21.15.3:0/2766918058 learned_addr learned my addr 172.21.15.3:0/2766918058 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe523fff700 1 --2- 172.21.15.3:0/2766918058 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5240b2220 0x7fe5240b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe51bfff700 1 --2- 172.21.15.3:0/2766918058 >> v2:172.21.15.153:3300/0 conn(0x7fe5241598d0 0x7fe5240cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 -- 172.21.15.3:0/2766918058 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5240b2220 msgr2=0x7fe5240b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 --2- 172.21.15.3:0/2766918058 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5240b2220 0x7fe5240b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 -- 172.21.15.3:0/2766918058 >> v2:172.21.15.153:3300/0 conn(0x7fe5241598d0 msgr2=0x7fe5240cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 --2- 172.21.15.3:0/2766918058 >> v2:172.21.15.153:3300/0 conn(0x7fe5241598d0 0x7fe5240cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.975+0000 7fe5237fe700 1 -- 172.21.15.3:0/2766918058 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5240cbc90 con 0x7fe524159500 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe5237fe700 1 --2- 172.21.15.3:0/2766918058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe524159500 0x7fe5240cad30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe52415d130 tx=0x7fe514009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4591a6f9a54209 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe522ffd700 1 -- 172.21.15.3:0/2766918058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51400b640 con 0x7fe524159500 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe522ffd700 1 -- 172.21.15.3:0/2766918058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51400b7a0 con 0x7fe524159500 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe522ffd700 1 -- 172.21.15.3:0/2766918058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51400bb60 con 0x7fe524159500 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2766918058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe524159500 msgr2=0x7fe5240cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2766918058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe524159500 0x7fe5240cad30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe52415d130 tx=0x7fe514009250 comp rx=0 tx=0).stop 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2766918058 shutdown_connections 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2766918058 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5240b2220 0x7fe5240b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2766918058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe524159500 0x7fe5240cad30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2766918058 >> v2:172.21.15.153:3300/0 conn(0x7fe5241598d0 0x7fe5240cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2766918058 >> 172.21.15.3:0/2766918058 conn(0x7fe524070310 msgr2=0x7fe524070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2766918058 shutdown_connections 2023-10-06T04:53:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.976+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2766918058 wait complete. 2023-10-06T04:53:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.977+0000 7fe52a1a5700 1 Processor -- start 2023-10-06T04:53:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.977+0000 7fe52a1a5700 1 -- start start 2023-10-06T04:53:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe52a1a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5240b2220 0x7fe5240ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe52a1a5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524159500 0x7fe5240c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe52a1a5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5241598d0 0x7fe52415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe52a1a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240b31e0 con 0x7fe5240b2220 2023-10-06T04:53:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe52a1a5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240b3d60 con 0x7fe524159500 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe52a1a5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240b3430 con 0x7fe5241598d0 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe5237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5240b2220 0x7fe5240ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe5237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5240b2220 0x7fe5240ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40896/0 (socket says 172.21.15.3:40896) 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe5237fe700 1 -- 172.21.15.3:0/2938328448 learned_addr learned my addr 172.21.15.3:0/2938328448 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe523fff700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5241598d0 0x7fe52415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.978+0000 7fe51bfff700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524159500 0x7fe5240c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe5237fe700 1 -- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5241598d0 msgr2=0x7fe52415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe5237fe700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5241598d0 0x7fe52415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe5237fe700 1 -- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524159500 msgr2=0x7fe5240c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe5237fe700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524159500 0x7fe5240c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe5237fe700 1 -- 172.21.15.3:0/2938328448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140094f0 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe5237fe700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5240b2220 0x7fe5240ca540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe514006fd0 tx=0x7fe514017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe51b7fe700 1 -- 172.21.15.3:0/2938328448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514000d10 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe51b7fe700 1 -- 172.21.15.3:0/2938328448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51400bb20 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5240b3710 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe51b7fe700 1 -- 172.21.15.3:0/2938328448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514011670 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.979+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5240c6c00 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.980+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe524071680 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.980+0000 7fe51b7fe700 1 -- 172.21.15.3:0/2938328448 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5140119e0 con 0x7fe5240b2220 2023-10-06T04:53:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.981+0000 7fe51b7fe700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.981+0000 7fe51b7fe700 1 -- 172.21.15.3:0/2938328448 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe514085e50 con 0x7fe5240b2220 2023-10-06T04:53:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.981+0000 7fe51bfff700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.982+0000 7fe51bfff700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe5240c40b0 tx=0x7fe50c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:11.985+0000 7fe51b7fe700 1 -- 172.21.15.3:0/2938328448 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe514052f60 con 0x7fe5240b2220 2023-10-06T04:53:12.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.173+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5240b3ea0 con 0x7fe504066f30 2023-10-06T04:53:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.182+0000 7fe51b7fe700 1 -- 172.21.15.3:0/2938328448 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286931 (secure 0 0 0) 0x7fe50c0098e0 con 0x7fe504066f30 2023-10-06T04:53:12.186 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 msgr2=0x7fe5040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe5240c40b0 tx=0x7fe50c008040 comp rx=0 tx=0).stop 2023-10-06T04:53:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5240b2220 msgr2=0x7fe5240ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5240b2220 0x7fe5240ca540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe514006fd0 tx=0x7fe514017000 comp rx=0 tx=0).stop 2023-10-06T04:53:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 shutdown_connections 2023-10-06T04:53:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5241598d0 0x7fe52415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5240b2220 0x7fe5240ca540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 --2- 172.21.15.3:0/2938328448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe524159500 0x7fe5240c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 >> 172.21.15.3:0/2938328448 conn(0x7fe524070310 msgr2=0x7fe5240ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 shutdown_connections 2023-10-06T04:53:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.185+0000 7fe52a1a5700 1 -- 172.21.15.3:0/2938328448 wait complete. 2023-10-06T04:53:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:53:12.199 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2023-10-06T04:53:11.435556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382332,"kb_used_data":4028,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592388,"statfs":{"total":773094113280,"available":770654605312,"internally_reserved":0,"allocated":4124672,"data_stored":2355617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.350234"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.108563+0000","last_change":"2023-10-06T04:53:08.103583+0000","last_active":"2023-10-06T04:53:08.108563+0000","last_peered":"2023-10-06T04:53:08.108563+0000","last_clean":"2023-10-06T04:53:08.108563+0000","last_became_active":"2023-10-06T04:53:08.102833+0000","last_became_peered":"2023-10-06T04:53:08.102833+0000","last_unstale":"2023-10-06T04:53:08.108563+0000","last_undegraded":"2023-10-06T04:53:08.108563+0000","last_fullsized":"2023-10-06T04:53:08.108563+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.546936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110034+0000","last_change":"2023-10-06T04:53:08.106695+0000","last_active":"2023-10-06T04:53:08.110034+0000","last_peered":"2023-10-06T04:53:08.110034+0000","last_clean":"2023-10-06T04:53:08.110034+0000","last_became_active":"2023-10-06T04:53:08.106375+0000","last_became_peered":"2023-10-06T04:53:08.106375+0000","last_unstale":"2023-10-06T04:53:08.110034+0000","last_undegraded":"2023-10-06T04:53:08.110034+0000","last_fullsized":"2023-10-06T04:53:08.110034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:33.319235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119197+0000","last_change":"2023-10-06T04:53:08.112093+0000","last_active":"2023-10-06T04:53:08.119197+0000","last_peered":"2023-10-06T04:53:08.119197+0000","last_clean":"2023-10-06T04:53:08.119197+0000","last_became_active":"2023-10-06T04:53:08.111813+0000","last_became_peered":"2023-10-06T04:53:08.111813+0000","last_unstale":"2023-10-06T04:53:08.119197+0000","last_undegraded":"2023-10-06T04:53:08.119197+0000","last_fullsized":"2023-10-06T04:53:08.119197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:50.075234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109744+0000","last_change":"2023-10-06T04:53:08.103857+0000","last_active":"2023-10-06T04:53:08.109744+0000","last_peered":"2023-10-06T04:53:08.109744+0000","last_clean":"2023-10-06T04:53:08.109744+0000","last_became_active":"2023-10-06T04:53:08.103436+0000","last_became_peered":"2023-10-06T04:53:08.103436+0000","last_unstale":"2023-10-06T04:53:08.109744+0000","last_undegraded":"2023-10-06T04:53:08.109744+0000","last_fullsized":"2023-10-06T04:53:08.109744+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:14.317772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.117525+0000","last_change":"2023-10-06T04:53:08.112255+0000","last_active":"2023-10-06T04:53:08.117525+0000","last_peered":"2023-10-06T04:53:08.117525+0000","last_clean":"2023-10-06T04:53:08.117525+0000","last_became_active":"2023-10-06T04:53:08.111963+0000","last_became_peered":"2023-10-06T04:53:08.111963+0000","last_unstale":"2023-10-06T04:53:08.117525+0000","last_undegraded":"2023-10-06T04:53:08.117525+0000","last_fullsized":"2023-10-06T04:53:08.117525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:01.830386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.106082+0000","last_change":"2023-10-06T04:53:08.099092+0000","last_active":"2023-10-06T04:53:08.106082+0000","last_peered":"2023-10-06T04:53:08.106082+0000","last_clean":"2023-10-06T04:53:08.106082+0000","last_became_active":"2023-10-06T04:53:08.098824+0000","last_became_peered":"2023-10-06T04:53:08.098824+0000","last_unstale":"2023-10-06T04:53:08.106082+0000","last_undegraded":"2023-10-06T04:53:08.106082+0000","last_fullsized":"2023-10-06T04:53:08.106082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:26.746368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118633+0000","last_change":"2023-10-06T04:53:08.111080+0000","last_active":"2023-10-06T04:53:08.118633+0000","last_peered":"2023-10-06T04:53:08.118633+0000","last_clean":"2023-10-06T04:53:08.118633+0000","last_became_active":"2023-10-06T04:53:08.110785+0000","last_became_peered":"2023-10-06T04:53:08.110785+0000","last_unstale":"2023-10-06T04:53:08.118633+0000","last_undegraded":"2023-10-06T04:53:08.118633+0000","last_fullsized":"2023-10-06T04:53:08.118633+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:33.933870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109694+0000","last_change":"2023-10-06T04:53:08.107323+0000","last_active":"2023-10-06T04:53:08.109694+0000","last_peered":"2023-10-06T04:53:08.109694+0000","last_clean":"2023-10-06T04:53:08.109694+0000","last_became_active":"2023-10-06T04:53:08.107025+0000","last_became_peered":"2023-10-06T04:53:08.107025+0000","last_unstale":"2023-10-06T04:53:08.109694+0000","last_undegraded":"2023-10-06T04:53:08.109694+0000","last_fullsized":"2023-10-06T04:53:08.109694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119570+0000","last_change":"2023-10-06T04:53:08.112717+0000","last_active":"2023-10-06T04:53:08.119570+0000","last_peered":"2023-10-06T04:53:08.119570+0000","last_clean":"2023-10-06T04:53:08.119570+0000","last_became_active":"2023-10-06T04:53:08.112459+0000","last_became_peered":"2023-10-06T04:53:08.112459+0000","last_unstale":"2023-10-06T04:53:08.119570+0000","last_undegraded":"2023-10-06T04:53:08.119570+0000","last_fullsized":"2023-10-06T04:53:08.119570+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:12.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118845+0000","last_change":"2023-10-06T04:53:08.113858+0000","last_active":"2023-10-06T04:53:08.118845+0000","last_peered":"2023-10-06T04:53:08.118845+0000","last_clean":"2023-10-06T04:53:08.118845+0000","last_became_active":"2023-10-06T04:53:08.113584+0000","last_became_peered":"2023-10-06T04:53:08.113584+0000","last_unstale":"2023-10-06T04:53:08.118845+0000","last_undegraded":"2023-10-06T04:53:08.118845+0000","last_fullsized":"2023-10-06T04:53:08.118845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:17.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118792+0000","last_change":"2023-10-06T04:53:08.112570+0000","last_active":"2023-10-06T04:53:08.118792+0000","last_peered":"2023-10-06T04:53:08.118792+0000","last_clean":"2023-10-06T04:53:08.118792+0000","last_became_active":"2023-10-06T04:53:08.112144+0000","last_became_peered":"2023-10-06T04:53:08.112144+0000","last_unstale":"2023-10-06T04:53:08.118792+0000","last_undegraded":"2023-10-06T04:53:08.118792+0000","last_fullsized":"2023-10-06T04:53:08.118792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:29.212376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119244+0000","last_change":"2023-10-06T04:53:08.109792+0000","last_active":"2023-10-06T04:53:08.119244+0000","last_peered":"2023-10-06T04:53:08.119244+0000","last_clean":"2023-10-06T04:53:08.119244+0000","last_became_active":"2023-10-06T04:53:08.109510+0000","last_became_peered":"2023-10-06T04:53:08.109510+0000","last_unstale":"2023-10-06T04:53:08.119244+0000","last_undegraded":"2023-10-06T04:53:08.119244+0000","last_fullsized":"2023-10-06T04:53:08.119244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:29.212376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122676+0000","last_change":"2023-10-06T04:53:08.119816+0000","last_active":"2023-10-06T04:53:08.122676+0000","last_peered":"2023-10-06T04:53:08.122676+0000","last_clean":"2023-10-06T04:53:08.122676+0000","last_became_active":"2023-10-06T04:53:08.119536+0000","last_became_peered":"2023-10-06T04:53:08.119536+0000","last_unstale":"2023-10-06T04:53:08.122676+0000","last_undegraded":"2023-10-06T04:53:08.122676+0000","last_fullsized":"2023-10-06T04:53:08.122676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.546936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120990+0000","last_change":"2023-10-06T04:53:08.110528+0000","last_active":"2023-10-06T04:53:08.120990+0000","last_peered":"2023-10-06T04:53:08.120990+0000","last_clean":"2023-10-06T04:53:08.120990+0000","last_became_active":"2023-10-06T04:53:08.110301+0000","last_became_peered":"2023-10-06T04:53:08.110301+0000","last_unstale":"2023-10-06T04:53:08.120990+0000","last_undegraded":"2023-10-06T04:53:08.120990+0000","last_fullsized":"2023-10-06T04:53:08.120990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:20.022067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121238+0000","last_change":"2023-10-06T04:53:08.114696+0000","last_active":"2023-10-06T04:53:08.121238+0000","last_peered":"2023-10-06T04:53:08.121238+0000","last_clean":"2023-10-06T04:53:08.121238+0000","last_became_active":"2023-10-06T04:53:08.113005+0000","last_became_peered":"2023-10-06T04:53:08.113005+0000","last_unstale":"2023-10-06T04:53:08.121238+0000","last_undegraded":"2023-10-06T04:53:08.121238+0000","last_fullsized":"2023-10-06T04:53:08.121238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:52.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121817+0000","last_change":"2023-10-06T04:53:08.115316+0000","last_active":"2023-10-06T04:53:08.121817+0000","last_peered":"2023-10-06T04:53:08.121817+0000","last_clean":"2023-10-06T04:53:08.121817+0000","last_became_active":"2023-10-06T04:53:08.114411+0000","last_became_peered":"2023-10-06T04:53:08.114411+0000","last_unstale":"2023-10-06T04:53:08.121817+0000","last_undegraded":"2023-10-06T04:53:08.121817+0000","last_fullsized":"2023-10-06T04:53:08.121817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:29.212376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120249+0000","last_change":"2023-10-06T04:53:08.113641+0000","last_active":"2023-10-06T04:53:08.120249+0000","last_peered":"2023-10-06T04:53:08.120249+0000","last_clean":"2023-10-06T04:53:08.120249+0000","last_became_active":"2023-10-06T04:53:08.112929+0000","last_became_peered":"2023-10-06T04:53:08.112929+0000","last_unstale":"2023-10-06T04:53:08.120249+0000","last_undegraded":"2023-10-06T04:53:08.120249+0000","last_fullsized":"2023-10-06T04:53:08.120249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:50.075234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118642+0000","last_change":"2023-10-06T04:53:08.113744+0000","last_active":"2023-10-06T04:53:08.118642+0000","last_peered":"2023-10-06T04:53:08.118642+0000","last_clean":"2023-10-06T04:53:08.118642+0000","last_became_active":"2023-10-06T04:53:08.113456+0000","last_became_peered":"2023-10-06T04:53:08.113456+0000","last_unstale":"2023-10-06T04:53:08.118642+0000","last_undegraded":"2023-10-06T04:53:08.118642+0000","last_fullsized":"2023-10-06T04:53:08.118642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:01.830386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.108837+0000","last_change":"2023-10-06T04:53:08.099322+0000","last_active":"2023-10-06T04:53:08.108837+0000","last_peered":"2023-10-06T04:53:08.108837+0000","last_clean":"2023-10-06T04:53:08.108837+0000","last_became_active":"2023-10-06T04:53:08.099066+0000","last_became_peered":"2023-10-06T04:53:08.099066+0000","last_unstale":"2023-10-06T04:53:08.108837+0000","last_undegraded":"2023-10-06T04:53:08.108837+0000","last_fullsized":"2023-10-06T04:53:08.108837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:50.183857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113845+0000","last_change":"2023-10-06T04:53:08.106650+0000","last_active":"2023-10-06T04:53:08.113845+0000","last_peered":"2023-10-06T04:53:08.113845+0000","last_clean":"2023-10-06T04:53:08.113845+0000","last_became_active":"2023-10-06T04:53:08.106280+0000","last_became_peered":"2023-10-06T04:53:08.106280+0000","last_unstale":"2023-10-06T04:53:08.113845+0000","last_undegraded":"2023-10-06T04:53:08.113845+0000","last_fullsized":"2023-10-06T04:53:08.113845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:05.753883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118802+0000","last_change":"2023-10-06T04:53:08.112581+0000","last_active":"2023-10-06T04:53:08.118802+0000","last_peered":"2023-10-06T04:53:08.118802+0000","last_clean":"2023-10-06T04:53:08.118802+0000","last_became_active":"2023-10-06T04:53:08.112244+0000","last_became_peered":"2023-10-06T04:53:08.112244+0000","last_unstale":"2023-10-06T04:53:08.118802+0000","last_undegraded":"2023-10-06T04:53:08.118802+0000","last_fullsized":"2023-10-06T04:53:08.118802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:12.639355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123111+0000","last_change":"2023-10-06T04:53:08.118537+0000","last_active":"2023-10-06T04:53:08.123111+0000","last_peered":"2023-10-06T04:53:08.123111+0000","last_clean":"2023-10-06T04:53:08.123111+0000","last_became_active":"2023-10-06T04:53:08.116925+0000","last_became_peered":"2023-10-06T04:53:08.116925+0000","last_unstale":"2023-10-06T04:53:08.123111+0000","last_undegraded":"2023-10-06T04:53:08.123111+0000","last_fullsized":"2023-10-06T04:53:08.123111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:11.783956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.116568+0000","last_change":"2023-10-06T04:53:08.109517+0000","last_active":"2023-10-06T04:53:08.116568+0000","last_peered":"2023-10-06T04:53:08.116568+0000","last_clean":"2023-10-06T04:53:08.116568+0000","last_became_active":"2023-10-06T04:53:08.109226+0000","last_became_peered":"2023-10-06T04:53:08.109226+0000","last_unstale":"2023-10-06T04:53:08.116568+0000","last_undegraded":"2023-10-06T04:53:08.116568+0000","last_fullsized":"2023-10-06T04:53:08.116568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:12.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109896+0000","last_change":"2023-10-06T04:53:08.108399+0000","last_active":"2023-10-06T04:53:08.109896+0000","last_peered":"2023-10-06T04:53:08.109896+0000","last_clean":"2023-10-06T04:53:08.109896+0000","last_became_active":"2023-10-06T04:53:08.108126+0000","last_became_peered":"2023-10-06T04:53:08.108126+0000","last_unstale":"2023-10-06T04:53:08.109896+0000","last_undegraded":"2023-10-06T04:53:08.109896+0000","last_fullsized":"2023-10-06T04:53:08.109896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:05.753883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.114384+0000","last_change":"2023-10-06T04:53:08.106546+0000","last_active":"2023-10-06T04:53:08.114384+0000","last_peered":"2023-10-06T04:53:08.114384+0000","last_clean":"2023-10-06T04:53:08.114384+0000","last_became_active":"2023-10-06T04:53:08.106241+0000","last_became_peered":"2023-10-06T04:53:08.106241+0000","last_unstale":"2023-10-06T04:53:08.114384+0000","last_undegraded":"2023-10-06T04:53:08.114384+0000","last_fullsized":"2023-10-06T04:53:08.114384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:48.709130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.111739+0000","last_change":"2023-10-06T04:53:08.105986+0000","last_active":"2023-10-06T04:53:08.111739+0000","last_peered":"2023-10-06T04:53:08.111739+0000","last_clean":"2023-10-06T04:53:08.111739+0000","last_became_active":"2023-10-06T04:53:08.105553+0000","last_became_peered":"2023-10-06T04:53:08.105553+0000","last_unstale":"2023-10-06T04:53:08.111739+0000","last_undegraded":"2023-10-06T04:53:08.111739+0000","last_fullsized":"2023-10-06T04:53:08.111739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:58.365320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118689+0000","last_change":"2023-10-06T04:53:08.113131+0000","last_active":"2023-10-06T04:53:08.118689+0000","last_peered":"2023-10-06T04:53:08.118689+0000","last_clean":"2023-10-06T04:53:08.118689+0000","last_became_active":"2023-10-06T04:53:08.112825+0000","last_became_peered":"2023-10-06T04:53:08.112825+0000","last_unstale":"2023-10-06T04:53:08.118689+0000","last_undegraded":"2023-10-06T04:53:08.118689+0000","last_fullsized":"2023-10-06T04:53:08.118689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:41.247813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118582+0000","last_change":"2023-10-06T04:53:08.110462+0000","last_active":"2023-10-06T04:53:08.118582+0000","last_peered":"2023-10-06T04:53:08.118582+0000","last_clean":"2023-10-06T04:53:08.118582+0000","last_became_active":"2023-10-06T04:53:08.110144+0000","last_became_peered":"2023-10-06T04:53:08.110144+0000","last_unstale":"2023-10-06T04:53:08.118582+0000","last_undegraded":"2023-10-06T04:53:08.118582+0000","last_fullsized":"2023-10-06T04:53:08.118582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113794+0000","last_change":"2023-10-06T04:53:08.108356+0000","last_active":"2023-10-06T04:53:08.113794+0000","last_peered":"2023-10-06T04:53:08.113794+0000","last_clean":"2023-10-06T04:53:08.113794+0000","last_became_active":"2023-10-06T04:53:08.108020+0000","last_became_peered":"2023-10-06T04:53:08.108020+0000","last_unstale":"2023-10-06T04:53:08.113794+0000","last_undegraded":"2023-10-06T04:53:08.113794+0000","last_fullsized":"2023-10-06T04:53:08.113794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:14.317772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113475+0000","last_change":"2023-10-06T04:53:08.105781+0000","last_active":"2023-10-06T04:53:08.113475+0000","last_peered":"2023-10-06T04:53:08.113475+0000","last_clean":"2023-10-06T04:53:08.113475+0000","last_became_active":"2023-10-06T04:53:08.105559+0000","last_became_peered":"2023-10-06T04:53:08.105559+0000","last_unstale":"2023-10-06T04:53:08.113475+0000","last_undegraded":"2023-10-06T04:53:08.113475+0000","last_fullsized":"2023-10-06T04:53:08.113475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:29.052698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121916+0000","last_change":"2023-10-06T04:53:08.115148+0000","last_active":"2023-10-06T04:53:08.121916+0000","last_peered":"2023-10-06T04:53:08.121916+0000","last_clean":"2023-10-06T04:53:08.121916+0000","last_became_active":"2023-10-06T04:53:08.114930+0000","last_became_peered":"2023-10-06T04:53:08.114930+0000","last_unstale":"2023-10-06T04:53:08.121916+0000","last_undegraded":"2023-10-06T04:53:08.121916+0000","last_fullsized":"2023-10-06T04:53:08.121916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:52.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.106261+0000","last_change":"2023-10-06T04:53:08.098992+0000","last_active":"2023-10-06T04:53:08.106261+0000","last_peered":"2023-10-06T04:53:08.106261+0000","last_clean":"2023-10-06T04:53:08.106261+0000","last_became_active":"2023-10-06T04:53:08.098744+0000","last_became_peered":"2023-10-06T04:53:08.098744+0000","last_unstale":"2023-10-06T04:53:08.106261+0000","last_undegraded":"2023-10-06T04:53:08.106261+0000","last_fullsized":"2023-10-06T04:53:08.106261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:10.735694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122303+0000","last_change":"2023-10-06T04:53:08.113103+0000","last_active":"2023-10-06T04:53:08.122303+0000","last_peered":"2023-10-06T04:53:08.122303+0000","last_clean":"2023-10-06T04:53:08.122303+0000","last_became_active":"2023-10-06T04:53:08.112806+0000","last_became_peered":"2023-10-06T04:53:08.112806+0000","last_unstale":"2023-10-06T04:53:08.122303+0000","last_undegraded":"2023-10-06T04:53:08.122303+0000","last_fullsized":"2023-10-06T04:53:08.122303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:12.639355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123222+0000","last_change":"2023-10-06T04:53:07.104308+0000","last_active":"2023-10-06T04:53:08.123222+0000","last_peered":"2023-10-06T04:53:08.123222+0000","last_clean":"2023-10-06T04:53:08.123222+0000","last_became_active":"2023-10-06T04:53:07.103696+0000","last_became_peered":"2023-10-06T04:53:07.103696+0000","last_unstale":"2023-10-06T04:53:08.123222+0000","last_undegraded":"2023-10-06T04:53:08.123222+0000","last_fullsized":"2023-10-06T04:53:08.123222+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:57.514050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.117347+0000","last_change":"2023-10-06T04:53:08.113134+0000","last_active":"2023-10-06T04:53:08.117347+0000","last_peered":"2023-10-06T04:53:08.117347+0000","last_clean":"2023-10-06T04:53:08.117347+0000","last_became_active":"2023-10-06T04:53:08.112336+0000","last_became_peered":"2023-10-06T04:53:08.112336+0000","last_unstale":"2023-10-06T04:53:08.117347+0000","last_undegraded":"2023-10-06T04:53:08.117347+0000","last_fullsized":"2023-10-06T04:53:08.117347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:29.364724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120507+0000","last_change":"2023-10-06T04:53:07.106919+0000","last_active":"2023-10-06T04:53:08.120507+0000","last_peered":"2023-10-06T04:53:08.120507+0000","last_clean":"2023-10-06T04:53:08.120507+0000","last_became_active":"2023-10-06T04:53:07.106653+0000","last_became_peered":"2023-10-06T04:53:07.106653+0000","last_unstale":"2023-10-06T04:53:08.120507+0000","last_undegraded":"2023-10-06T04:53:08.120507+0000","last_fullsized":"2023-10-06T04:53:08.120507+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:40.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120712+0000","last_change":"2023-10-06T04:53:08.110653+0000","last_active":"2023-10-06T04:53:08.120712+0000","last_peered":"2023-10-06T04:53:08.120712+0000","last_clean":"2023-10-06T04:53:08.120712+0000","last_became_active":"2023-10-06T04:53:08.110390+0000","last_became_peered":"2023-10-06T04:53:08.110390+0000","last_unstale":"2023-10-06T04:53:08.120712+0000","last_undegraded":"2023-10-06T04:53:08.120712+0000","last_fullsized":"2023-10-06T04:53:08.120712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:21.350284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122650+0000","last_change":"2023-10-06T04:53:07.098879+0000","last_active":"2023-10-06T04:53:08.122650+0000","last_peered":"2023-10-06T04:53:08.122650+0000","last_clean":"2023-10-06T04:53:08.122650+0000","last_became_active":"2023-10-06T04:53:07.097717+0000","last_became_peered":"2023-10-06T04:53:07.097717+0000","last_unstale":"2023-10-06T04:53:08.122650+0000","last_undegraded":"2023-10-06T04:53:08.122650+0000","last_fullsized":"2023-10-06T04:53:08.122650+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:00.823641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110470+0000","last_change":"2023-10-06T04:53:08.107923+0000","last_active":"2023-10-06T04:53:08.110470+0000","last_peered":"2023-10-06T04:53:08.110470+0000","last_clean":"2023-10-06T04:53:08.110470+0000","last_became_active":"2023-10-06T04:53:08.107634+0000","last_became_peered":"2023-10-06T04:53:08.107634+0000","last_unstale":"2023-10-06T04:53:08.110470+0000","last_undegraded":"2023-10-06T04:53:08.110470+0000","last_fullsized":"2023-10-06T04:53:08.110470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123274+0000","last_change":"2023-10-06T04:53:07.103838+0000","last_active":"2023-10-06T04:53:08.123274+0000","last_peered":"2023-10-06T04:53:08.123274+0000","last_clean":"2023-10-06T04:53:08.123274+0000","last_became_active":"2023-10-06T04:53:07.103574+0000","last_became_peered":"2023-10-06T04:53:07.103574+0000","last_unstale":"2023-10-06T04:53:08.123274+0000","last_undegraded":"2023-10-06T04:53:08.123274+0000","last_fullsized":"2023-10-06T04:53:08.123274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:57.358575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113236+0000","last_change":"2023-10-06T04:53:08.106864+0000","last_active":"2023-10-06T04:53:08.113236+0000","last_peered":"2023-10-06T04:53:08.113236+0000","last_clean":"2023-10-06T04:53:08.113236+0000","last_became_active":"2023-10-06T04:53:08.106603+0000","last_became_peered":"2023-10-06T04:53:08.106603+0000","last_unstale":"2023-10-06T04:53:08.113236+0000","last_undegraded":"2023-10-06T04:53:08.113236+0000","last_fullsized":"2023-10-06T04:53:08.113236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:23.857041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121358+0000","last_change":"2023-10-06T04:53:07.103893+0000","last_active":"2023-10-06T04:53:08.121358+0000","last_peered":"2023-10-06T04:53:08.121358+0000","last_clean":"2023-10-06T04:53:08.121358+0000","last_became_active":"2023-10-06T04:53:07.103652+0000","last_became_peered":"2023-10-06T04:53:07.103652+0000","last_unstale":"2023-10-06T04:53:08.121358+0000","last_undegraded":"2023-10-06T04:53:08.121358+0000","last_fullsized":"2023-10-06T04:53:08.121358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:47.702385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118986+0000","last_change":"2023-10-06T04:53:08.114549+0000","last_active":"2023-10-06T04:53:08.118986+0000","last_peered":"2023-10-06T04:53:08.118986+0000","last_clean":"2023-10-06T04:53:08.118986+0000","last_became_active":"2023-10-06T04:53:08.113921+0000","last_became_peered":"2023-10-06T04:53:08.113921+0000","last_unstale":"2023-10-06T04:53:08.118986+0000","last_undegraded":"2023-10-06T04:53:08.118986+0000","last_fullsized":"2023-10-06T04:53:08.118986+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:26.083562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.124847+0000","last_change":"2023-10-06T04:53:07.103898+0000","last_active":"2023-10-06T04:53:08.124847+0000","last_peered":"2023-10-06T04:53:08.124847+0000","last_clean":"2023-10-06T04:53:08.124847+0000","last_became_active":"2023-10-06T04:53:07.103669+0000","last_became_peered":"2023-10-06T04:53:07.103669+0000","last_unstale":"2023-10-06T04:53:08.124847+0000","last_undegraded":"2023-10-06T04:53:08.124847+0000","last_fullsized":"2023-10-06T04:53:08.124847+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:28.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123092+0000","last_change":"2023-10-06T04:53:08.118542+0000","last_active":"2023-10-06T04:53:08.123092+0000","last_peered":"2023-10-06T04:53:08.123092+0000","last_clean":"2023-10-06T04:53:08.123092+0000","last_became_active":"2023-10-06T04:53:08.118282+0000","last_became_peered":"2023-10-06T04:53:08.118282+0000","last_unstale":"2023-10-06T04:53:08.123092+0000","last_undegraded":"2023-10-06T04:53:08.123092+0000","last_fullsized":"2023-10-06T04:53:08.123092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:33.754555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091744+0000","last_change":"2023-10-06T04:53:07.107097+0000","last_active":"2023-10-06T04:53:08.091744+0000","last_peered":"2023-10-06T04:53:08.091744+0000","last_clean":"2023-10-06T04:53:08.091744+0000","last_became_active":"2023-10-06T04:53:07.106755+0000","last_became_peered":"2023-10-06T04:53:07.106755+0000","last_unstale":"2023-10-06T04:53:08.091744+0000","last_undegraded":"2023-10-06T04:53:08.091744+0000","last_fullsized":"2023-10-06T04:53:08.091744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:43.651185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.126318+0000","last_change":"2023-10-06T04:53:08.122305+0000","last_active":"2023-10-06T04:53:08.126318+0000","last_peered":"2023-10-06T04:53:08.126318+0000","last_clean":"2023-10-06T04:53:08.126318+0000","last_became_active":"2023-10-06T04:53:08.121988+0000","last_became_peered":"2023-10-06T04:53:08.121988+0000","last_unstale":"2023-10-06T04:53:08.126318+0000","last_undegraded":"2023-10-06T04:53:08.126318+0000","last_fullsized":"2023-10-06T04:53:08.126318+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:17.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098795+0000","last_change":"2023-10-06T04:53:07.102376+0000","last_active":"2023-10-06T04:53:08.098795+0000","last_peered":"2023-10-06T04:53:08.098795+0000","last_clean":"2023-10-06T04:53:08.098795+0000","last_became_active":"2023-10-06T04:53:07.102103+0000","last_became_peered":"2023-10-06T04:53:07.102103+0000","last_unstale":"2023-10-06T04:53:08.098795+0000","last_undegraded":"2023-10-06T04:53:08.098795+0000","last_fullsized":"2023-10-06T04:53:08.098795+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:28.205631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109721+0000","last_change":"2023-10-06T04:53:08.103674+0000","last_active":"2023-10-06T04:53:08.109721+0000","last_peered":"2023-10-06T04:53:08.109721+0000","last_clean":"2023-10-06T04:53:08.109721+0000","last_became_active":"2023-10-06T04:53:08.103115+0000","last_became_peered":"2023-10-06T04:53:08.103115+0000","last_unstale":"2023-10-06T04:53:08.109721+0000","last_undegraded":"2023-10-06T04:53:08.109721+0000","last_fullsized":"2023-10-06T04:53:08.109721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:44.657930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.092586+0000","last_change":"2023-10-06T04:53:07.104906+0000","last_active":"2023-10-06T04:53:08.092586+0000","last_peered":"2023-10-06T04:53:08.092586+0000","last_clean":"2023-10-06T04:53:08.092586+0000","last_became_active":"2023-10-06T04:53:07.104622+0000","last_became_peered":"2023-10-06T04:53:07.104622+0000","last_unstale":"2023-10-06T04:53:08.092586+0000","last_undegraded":"2023-10-06T04:53:08.092586+0000","last_fullsized":"2023-10-06T04:53:08.092586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:17.815747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120272+0000","last_change":"2023-10-06T04:53:08.113699+0000","last_active":"2023-10-06T04:53:08.120272+0000","last_peered":"2023-10-06T04:53:08.120272+0000","last_clean":"2023-10-06T04:53:08.120272+0000","last_became_active":"2023-10-06T04:53:08.113354+0000","last_became_peered":"2023-10-06T04:53:08.113354+0000","last_unstale":"2023-10-06T04:53:08.120272+0000","last_undegraded":"2023-10-06T04:53:08.120272+0000","last_fullsized":"2023-10-06T04:53:08.120272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:33.933870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.102480+0000","last_change":"2023-10-06T04:53:07.107135+0000","last_active":"2023-10-06T04:53:08.102480+0000","last_peered":"2023-10-06T04:53:08.102480+0000","last_clean":"2023-10-06T04:53:08.102480+0000","last_became_active":"2023-10-06T04:53:07.106792+0000","last_became_peered":"2023-10-06T04:53:07.106792+0000","last_unstale":"2023-10-06T04:53:08.102480+0000","last_undegraded":"2023-10-06T04:53:08.102480+0000","last_fullsized":"2023-10-06T04:53:08.102480+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:48.589723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119313+0000","last_change":"2023-10-06T04:53:08.107557+0000","last_active":"2023-10-06T04:53:08.119313+0000","last_peered":"2023-10-06T04:53:08.119313+0000","last_clean":"2023-10-06T04:53:08.119313+0000","last_became_active":"2023-10-06T04:53:08.107247+0000","last_became_peered":"2023-10-06T04:53:08.107247+0000","last_unstale":"2023-10-06T04:53:08.119313+0000","last_undegraded":"2023-10-06T04:53:08.119313+0000","last_fullsized":"2023-10-06T04:53:08.119313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121633+0000","last_change":"2023-10-06T04:53:07.104291+0000","last_active":"2023-10-06T04:53:08.121633+0000","last_peered":"2023-10-06T04:53:08.121633+0000","last_clean":"2023-10-06T04:53:08.121633+0000","last_became_active":"2023-10-06T04:53:07.104022+0000","last_became_peered":"2023-10-06T04:53:07.104022+0000","last_unstale":"2023-10-06T04:53:08.121633+0000","last_undegraded":"2023-10-06T04:53:08.121633+0000","last_fullsized":"2023-10-06T04:53:08.121633+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:40.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122121+0000","last_change":"2023-10-06T04:53:08.112439+0000","last_active":"2023-10-06T04:53:08.122121+0000","last_peered":"2023-10-06T04:53:08.122121+0000","last_clean":"2023-10-06T04:53:08.122121+0000","last_became_active":"2023-10-06T04:53:08.112137+0000","last_became_peered":"2023-10-06T04:53:08.112137+0000","last_unstale":"2023-10-06T04:53:08.122121+0000","last_undegraded":"2023-10-06T04:53:08.122121+0000","last_fullsized":"2023-10-06T04:53:08.122121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:26.083562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.126601+0000","last_change":"2023-10-06T04:53:07.109471+0000","last_active":"2023-10-06T04:53:08.126601+0000","last_peered":"2023-10-06T04:53:08.126601+0000","last_clean":"2023-10-06T04:53:08.126601+0000","last_became_active":"2023-10-06T04:53:07.109147+0000","last_became_peered":"2023-10-06T04:53:07.109147+0000","last_unstale":"2023-10-06T04:53:08.126601+0000","last_undegraded":"2023-10-06T04:53:08.126601+0000","last_fullsized":"2023-10-06T04:53:08.126601+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:25.739623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119459+0000","last_change":"2023-10-06T04:53:08.111156+0000","last_active":"2023-10-06T04:53:08.119459+0000","last_peered":"2023-10-06T04:53:08.119459+0000","last_clean":"2023-10-06T04:53:08.119459+0000","last_became_active":"2023-10-06T04:53:08.110818+0000","last_became_peered":"2023-10-06T04:53:08.110818+0000","last_unstale":"2023-10-06T04:53:08.119459+0000","last_undegraded":"2023-10-06T04:53:08.119459+0000","last_fullsized":"2023-10-06T04:53:08.119459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.206832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123201+0000","last_change":"2023-10-06T04:53:07.104633+0000","last_active":"2023-10-06T04:53:08.123201+0000","last_peered":"2023-10-06T04:53:08.123201+0000","last_clean":"2023-10-06T04:53:08.123201+0000","last_became_active":"2023-10-06T04:53:07.104373+0000","last_became_peered":"2023-10-06T04:53:07.104373+0000","last_unstale":"2023-10-06T04:53:08.123201+0000","last_undegraded":"2023-10-06T04:53:08.123201+0000","last_fullsized":"2023-10-06T04:53:08.123201+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:11.829383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109577+0000","last_change":"2023-10-06T04:53:08.106581+0000","last_active":"2023-10-06T04:53:08.109577+0000","last_peered":"2023-10-06T04:53:08.109577+0000","last_clean":"2023-10-06T04:53:08.109577+0000","last_became_active":"2023-10-06T04:53:08.106214+0000","last_became_peered":"2023-10-06T04:53:08.106214+0000","last_unstale":"2023-10-06T04:53:08.109577+0000","last_undegraded":"2023-10-06T04:53:08.109577+0000","last_fullsized":"2023-10-06T04:53:08.109577+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:26.083562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.092303+0000","last_change":"2023-10-06T04:53:07.106979+0000","last_active":"2023-10-06T04:53:08.092303+0000","last_peered":"2023-10-06T04:53:08.092303+0000","last_clean":"2023-10-06T04:53:08.092303+0000","last_became_active":"2023-10-06T04:53:07.106690+0000","last_became_peered":"2023-10-06T04:53:07.106690+0000","last_unstale":"2023-10-06T04:53:08.092303+0000","last_undegraded":"2023-10-06T04:53:08.092303+0000","last_fullsized":"2023-10-06T04:53:08.092303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:42.878681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120202+0000","last_change":"2023-10-06T04:53:08.112824+0000","last_active":"2023-10-06T04:53:08.120202+0000","last_peered":"2023-10-06T04:53:08.120202+0000","last_clean":"2023-10-06T04:53:08.120202+0000","last_became_active":"2023-10-06T04:53:08.112532+0000","last_became_peered":"2023-10-06T04:53:08.112532+0000","last_unstale":"2023-10-06T04:53:08.120202+0000","last_undegraded":"2023-10-06T04:53:08.120202+0000","last_fullsized":"2023-10-06T04:53:08.120202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.206832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.099152+0000","last_change":"2023-10-06T04:53:07.103726+0000","last_active":"2023-10-06T04:53:08.099152+0000","last_peered":"2023-10-06T04:53:08.099152+0000","last_clean":"2023-10-06T04:53:08.099152+0000","last_became_active":"2023-10-06T04:53:07.103457+0000","last_became_peered":"2023-10-06T04:53:07.103457+0000","last_unstale":"2023-10-06T04:53:08.099152+0000","last_undegraded":"2023-10-06T04:53:08.099152+0000","last_fullsized":"2023-10-06T04:53:08.099152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:14.302187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110437+0000","last_change":"2023-10-06T04:53:08.107829+0000","last_active":"2023-10-06T04:53:08.110437+0000","last_peered":"2023-10-06T04:53:08.110437+0000","last_clean":"2023-10-06T04:53:08.110437+0000","last_became_active":"2023-10-06T04:53:08.107522+0000","last_became_peered":"2023-10-06T04:53:08.107522+0000","last_unstale":"2023-10-06T04:53:08.110437+0000","last_undegraded":"2023-10-06T04:53:08.110437+0000","last_fullsized":"2023-10-06T04:53:08.110437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:11.783956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.125199+0000","last_change":"2023-10-06T04:53:07.106993+0000","last_active":"2023-10-06T04:53:08.125199+0000","last_peered":"2023-10-06T04:53:08.125199+0000","last_clean":"2023-10-06T04:53:08.125199+0000","last_became_active":"2023-10-06T04:53:07.106326+0000","last_became_peered":"2023-10-06T04:53:07.106326+0000","last_unstale":"2023-10-06T04:53:08.125199+0000","last_undegraded":"2023-10-06T04:53:08.125199+0000","last_fullsized":"2023-10-06T04:53:08.125199+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:04.747138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110453+0000","last_change":"2023-10-06T04:53:08.107921+0000","last_active":"2023-10-06T04:53:08.110453+0000","last_peered":"2023-10-06T04:53:08.110453+0000","last_clean":"2023-10-06T04:53:08.110453+0000","last_became_active":"2023-10-06T04:53:08.107656+0000","last_became_peered":"2023-10-06T04:53:08.107656+0000","last_unstale":"2023-10-06T04:53:08.110453+0000","last_undegraded":"2023-10-06T04:53:08.110453+0000","last_fullsized":"2023-10-06T04:53:08.110453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.206832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123598+0000","last_change":"2023-10-06T04:53:07.107107+0000","last_active":"2023-10-06T04:53:08.123598+0000","last_peered":"2023-10-06T04:53:08.123598+0000","last_clean":"2023-10-06T04:53:08.123598+0000","last_became_active":"2023-10-06T04:53:07.106851+0000","last_became_peered":"2023-10-06T04:53:07.106851+0000","last_unstale":"2023-10-06T04:53:08.123598+0000","last_undegraded":"2023-10-06T04:53:08.123598+0000","last_fullsized":"2023-10-06T04:53:08.123598+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:28.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118614+0000","last_change":"2023-10-06T04:53:08.110867+0000","last_active":"2023-10-06T04:53:08.118614+0000","last_peered":"2023-10-06T04:53:08.118614+0000","last_clean":"2023-10-06T04:53:08.118614+0000","last_became_active":"2023-10-06T04:53:08.110563+0000","last_became_peered":"2023-10-06T04:53:08.110563+0000","last_unstale":"2023-10-06T04:53:08.118614+0000","last_undegraded":"2023-10-06T04:53:08.118614+0000","last_fullsized":"2023-10-06T04:53:08.118614+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123615+0000","last_change":"2023-10-06T04:53:07.108371+0000","last_active":"2023-10-06T04:53:08.123615+0000","last_peered":"2023-10-06T04:53:08.123615+0000","last_clean":"2023-10-06T04:53:08.123615+0000","last_became_active":"2023-10-06T04:53:07.108067+0000","last_became_peered":"2023-10-06T04:53:07.108067+0000","last_unstale":"2023-10-06T04:53:08.123615+0000","last_undegraded":"2023-10-06T04:53:08.123615+0000","last_fullsized":"2023-10-06T04:53:08.123615+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:22.850296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109883+0000","last_change":"2023-10-06T04:53:08.107665+0000","last_active":"2023-10-06T04:53:08.109883+0000","last_peered":"2023-10-06T04:53:08.109883+0000","last_clean":"2023-10-06T04:53:08.109883+0000","last_became_active":"2023-10-06T04:53:08.107386+0000","last_became_peered":"2023-10-06T04:53:08.107386+0000","last_unstale":"2023-10-06T04:53:08.109883+0000","last_undegraded":"2023-10-06T04:53:08.109883+0000","last_fullsized":"2023-10-06T04:53:08.109883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:17.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122840+0000","last_change":"2023-10-06T04:53:07.107782+0000","last_active":"2023-10-06T04:53:08.122840+0000","last_peered":"2023-10-06T04:53:08.122840+0000","last_clean":"2023-10-06T04:53:08.122840+0000","last_became_active":"2023-10-06T04:53:07.107463+0000","last_became_peered":"2023-10-06T04:53:07.107463+0000","last_unstale":"2023-10-06T04:53:08.122840+0000","last_undegraded":"2023-10-06T04:53:08.122840+0000","last_fullsized":"2023-10-06T04:53:08.122840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:00.567630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109355+0000","last_change":"2023-10-06T04:53:08.102393+0000","last_active":"2023-10-06T04:53:08.109355+0000","last_peered":"2023-10-06T04:53:08.109355+0000","last_clean":"2023-10-06T04:53:08.109355+0000","last_became_active":"2023-10-06T04:53:08.102159+0000","last_became_peered":"2023-10-06T04:53:08.102159+0000","last_unstale":"2023-10-06T04:53:08.109355+0000","last_undegraded":"2023-10-06T04:53:08.109355+0000","last_fullsized":"2023-10-06T04:53:08.109355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.885426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091890+0000","last_change":"2023-10-06T04:53:07.098564+0000","last_active":"2023-10-06T04:53:08.091890+0000","last_peered":"2023-10-06T04:53:08.091890+0000","last_clean":"2023-10-06T04:53:08.091890+0000","last_became_active":"2023-10-06T04:53:07.098284+0000","last_became_peered":"2023-10-06T04:53:07.098284+0000","last_unstale":"2023-10-06T04:53:08.091890+0000","last_undegraded":"2023-10-06T04:53:08.091890+0000","last_fullsized":"2023-10-06T04:53:08.091890+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:13.311027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121848+0000","last_change":"2023-10-06T04:53:08.116342+0000","last_active":"2023-10-06T04:53:08.121848+0000","last_peered":"2023-10-06T04:53:08.121848+0000","last_clean":"2023-10-06T04:53:08.121848+0000","last_became_active":"2023-10-06T04:53:08.114084+0000","last_became_peered":"2023-10-06T04:53:08.114084+0000","last_unstale":"2023-10-06T04:53:08.121848+0000","last_undegraded":"2023-10-06T04:53:08.121848+0000","last_fullsized":"2023-10-06T04:53:08.121848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123303+0000","last_change":"2023-10-06T04:53:07.103458+0000","last_active":"2023-10-06T04:53:08.123303+0000","last_peered":"2023-10-06T04:53:08.123303+0000","last_clean":"2023-10-06T04:53:08.123303+0000","last_became_active":"2023-10-06T04:53:07.103165+0000","last_became_peered":"2023-10-06T04:53:07.103165+0000","last_unstale":"2023-10-06T04:53:08.123303+0000","last_undegraded":"2023-10-06T04:53:08.123303+0000","last_fullsized":"2023-10-06T04:53:08.123303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.469905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120714+0000","last_change":"2023-10-06T04:53:08.109571+0000","last_active":"2023-10-06T04:53:08.120714+0000","last_peered":"2023-10-06T04:53:08.120714+0000","last_clean":"2023-10-06T04:53:08.120714+0000","last_became_active":"2023-10-06T04:53:08.109240+0000","last_became_peered":"2023-10-06T04:53:08.109240+0000","last_unstale":"2023-10-06T04:53:08.120714+0000","last_undegraded":"2023-10-06T04:53:08.120714+0000","last_fullsized":"2023-10-06T04:53:08.120714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:12.639355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.094970+0000","last_change":"2023-10-06T04:53:07.102516+0000","last_active":"2023-10-06T04:53:08.094970+0000","last_peered":"2023-10-06T04:53:08.094970+0000","last_clean":"2023-10-06T04:53:08.094970+0000","last_became_active":"2023-10-06T04:53:07.102192+0000","last_became_peered":"2023-10-06T04:53:07.102192+0000","last_unstale":"2023-10-06T04:53:08.094970+0000","last_undegraded":"2023-10-06T04:53:08.094970+0000","last_fullsized":"2023-10-06T04:53:08.094970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:25.069789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119603+0000","last_change":"2023-10-06T04:53:08.109152+0000","last_active":"2023-10-06T04:53:08.119603+0000","last_peered":"2023-10-06T04:53:08.119603+0000","last_clean":"2023-10-06T04:53:08.119603+0000","last_became_active":"2023-10-06T04:53:08.108892+0000","last_became_peered":"2023-10-06T04:53:08.108892+0000","last_unstale":"2023-10-06T04:53:08.119603+0000","last_undegraded":"2023-10-06T04:53:08.119603+0000","last_fullsized":"2023-10-06T04:53:08.119603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:50.075234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096792+0000","last_change":"2023-10-06T04:53:07.106534+0000","last_active":"2023-10-06T04:53:08.096792+0000","last_peered":"2023-10-06T04:53:08.096792+0000","last_clean":"2023-10-06T04:53:08.096792+0000","last_became_active":"2023-10-06T04:53:07.106150+0000","last_became_peered":"2023-10-06T04:53:07.106150+0000","last_unstale":"2023-10-06T04:53:08.096792+0000","last_undegraded":"2023-10-06T04:53:08.096792+0000","last_fullsized":"2023-10-06T04:53:08.096792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:13.311027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.126304+0000","last_change":"2023-10-06T04:53:08.122237+0000","last_active":"2023-10-06T04:53:08.126304+0000","last_peered":"2023-10-06T04:53:08.126304+0000","last_clean":"2023-10-06T04:53:08.126304+0000","last_became_active":"2023-10-06T04:53:08.121962+0000","last_became_peered":"2023-10-06T04:53:08.121962+0000","last_unstale":"2023-10-06T04:53:08.126304+0000","last_undegraded":"2023-10-06T04:53:08.126304+0000","last_fullsized":"2023-10-06T04:53:08.126304+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:21.350284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.095883+0000","last_change":"2023-10-06T04:53:07.102961+0000","last_active":"2023-10-06T04:53:08.095883+0000","last_peered":"2023-10-06T04:53:08.095883+0000","last_clean":"2023-10-06T04:53:08.095883+0000","last_became_active":"2023-10-06T04:53:07.102626+0000","last_became_peered":"2023-10-06T04:53:07.102626+0000","last_unstale":"2023-10-06T04:53:08.095883+0000","last_undegraded":"2023-10-06T04:53:08.095883+0000","last_fullsized":"2023-10-06T04:53:08.095883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:36.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.108381+0000","last_change":"2023-10-06T04:53:08.105138+0000","last_active":"2023-10-06T04:53:08.108381+0000","last_peered":"2023-10-06T04:53:08.108381+0000","last_clean":"2023-10-06T04:53:08.108381+0000","last_became_active":"2023-10-06T04:53:08.104913+0000","last_became_peered":"2023-10-06T04:53:08.104913+0000","last_unstale":"2023-10-06T04:53:08.108381+0000","last_undegraded":"2023-10-06T04:53:08.108381+0000","last_fullsized":"2023-10-06T04:53:08.108381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:33.319235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096504+0000","last_change":"2023-10-06T04:53:07.105054+0000","last_active":"2023-10-06T04:53:08.096504+0000","last_peered":"2023-10-06T04:53:08.096504+0000","last_clean":"2023-10-06T04:53:08.096504+0000","last_became_active":"2023-10-06T04:53:07.104816+0000","last_became_peered":"2023-10-06T04:53:07.104816+0000","last_unstale":"2023-10-06T04:53:08.096504+0000","last_undegraded":"2023-10-06T04:53:08.096504+0000","last_fullsized":"2023-10-06T04:53:08.096504+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:11.632610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097847+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:53:08.097847+0000","last_peered":"2023-10-06T04:53:08.097847+0000","last_clean":"2023-10-06T04:53:08.097847+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:53:08.097847+0000","last_undegraded":"2023-10-06T04:53:08.097847+0000","last_fullsized":"2023-10-06T04:53:08.097847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122550+0000","last_change":"2023-10-06T04:53:08.116880+0000","last_active":"2023-10-06T04:53:08.122550+0000","last_peered":"2023-10-06T04:53:08.122550+0000","last_clean":"2023-10-06T04:53:08.122550+0000","last_became_active":"2023-10-06T04:53:08.112104+0000","last_became_peered":"2023-10-06T04:53:08.112104+0000","last_unstale":"2023-10-06T04:53:08.122550+0000","last_undegraded":"2023-10-06T04:53:08.122550+0000","last_fullsized":"2023-10-06T04:53:08.122550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:48.507690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091912+0000","last_change":"2023-10-06T04:53:07.105451+0000","last_active":"2023-10-06T04:53:08.091912+0000","last_peered":"2023-10-06T04:53:08.091912+0000","last_clean":"2023-10-06T04:53:08.091912+0000","last_became_active":"2023-10-06T04:53:07.105182+0000","last_became_peered":"2023-10-06T04:53:07.105182+0000","last_unstale":"2023-10-06T04:53:08.091912+0000","last_undegraded":"2023-10-06T04:53:08.091912+0000","last_fullsized":"2023-10-06T04:53:08.091912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:09.728948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.094007+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:53:08.094007+0000","last_peered":"2023-10-06T04:53:08.094007+0000","last_clean":"2023-10-06T04:53:08.094007+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:53:08.094007+0000","last_undegraded":"2023-10-06T04:53:08.094007+0000","last_fullsized":"2023-10-06T04:53:08.094007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.117296+0000","last_change":"2023-10-06T04:53:08.112860+0000","last_active":"2023-10-06T04:53:08.117296+0000","last_peered":"2023-10-06T04:53:08.117296+0000","last_clean":"2023-10-06T04:53:08.117296+0000","last_became_active":"2023-10-06T04:53:08.112557+0000","last_became_peered":"2023-10-06T04:53:08.112557+0000","last_unstale":"2023-10-06T04:53:08.117296+0000","last_undegraded":"2023-10-06T04:53:08.117296+0000","last_fullsized":"2023-10-06T04:53:08.117296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:43.415231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123524+0000","last_change":"2023-10-06T04:53:07.107653+0000","last_active":"2023-10-06T04:53:08.123524+0000","last_peered":"2023-10-06T04:53:08.123524+0000","last_clean":"2023-10-06T04:53:08.123524+0000","last_became_active":"2023-10-06T04:53:07.106536+0000","last_became_peered":"2023-10-06T04:53:07.106536+0000","last_unstale":"2023-10-06T04:53:08.123524+0000","last_undegraded":"2023-10-06T04:53:08.123524+0000","last_fullsized":"2023-10-06T04:53:08.123524+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:04.747138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091398+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:53:08.091398+0000","last_peered":"2023-10-06T04:53:08.091398+0000","last_clean":"2023-10-06T04:53:08.091398+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:53:08.091398+0000","last_undegraded":"2023-10-06T04:53:08.091398+0000","last_fullsized":"2023-10-06T04:53:08.091398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097644+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:53:08.097644+0000","last_peered":"2023-10-06T04:53:08.097644+0000","last_clean":"2023-10-06T04:53:08.097644+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:53:08.097644+0000","last_undegraded":"2023-10-06T04:53:08.097644+0000","last_fullsized":"2023-10-06T04:53:08.097644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120213+0000","last_change":"2023-10-06T04:53:08.113479+0000","last_active":"2023-10-06T04:53:08.120213+0000","last_peered":"2023-10-06T04:53:08.120213+0000","last_clean":"2023-10-06T04:53:08.120213+0000","last_became_active":"2023-10-06T04:53:08.113234+0000","last_became_peered":"2023-10-06T04:53:08.113234+0000","last_unstale":"2023-10-06T04:53:08.120213+0000","last_undegraded":"2023-10-06T04:53:08.120213+0000","last_fullsized":"2023-10-06T04:53:08.120213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097036+0000","last_change":"2023-10-06T04:53:07.107768+0000","last_active":"2023-10-06T04:53:08.097036+0000","last_peered":"2023-10-06T04:53:08.097036+0000","last_clean":"2023-10-06T04:53:08.097036+0000","last_became_active":"2023-10-06T04:53:07.107436+0000","last_became_peered":"2023-10-06T04:53:07.107436+0000","last_unstale":"2023-10-06T04:53:08.097036+0000","last_undegraded":"2023-10-06T04:53:08.097036+0000","last_fullsized":"2023-10-06T04:53:08.097036+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:24.715961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098834+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:53:08.098834+0000","last_peered":"2023-10-06T04:53:08.098834+0000","last_clean":"2023-10-06T04:53:08.098834+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:53:08.098834+0000","last_undegraded":"2023-10-06T04:53:08.098834+0000","last_fullsized":"2023-10-06T04:53:08.098834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118710+0000","last_change":"2023-10-06T04:53:08.112343+0000","last_active":"2023-10-06T04:53:08.118710+0000","last_peered":"2023-10-06T04:53:08.118710+0000","last_clean":"2023-10-06T04:53:08.118710+0000","last_became_active":"2023-10-06T04:53:08.111289+0000","last_became_peered":"2023-10-06T04:53:08.111289+0000","last_unstale":"2023-10-06T04:53:08.118710+0000","last_undegraded":"2023-10-06T04:53:08.118710+0000","last_fullsized":"2023-10-06T04:53:08.118710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:48.507690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098462+0000","last_change":"2023-10-06T04:53:07.103097+0000","last_active":"2023-10-06T04:53:08.098462+0000","last_peered":"2023-10-06T04:53:08.098462+0000","last_clean":"2023-10-06T04:53:08.098462+0000","last_became_active":"2023-10-06T04:53:07.102813+0000","last_became_peered":"2023-10-06T04:53:07.102813+0000","last_unstale":"2023-10-06T04:53:08.098462+0000","last_undegraded":"2023-10-06T04:53:08.098462+0000","last_fullsized":"2023-10-06T04:53:08.098462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:40.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097581+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:53:08.097581+0000","last_peered":"2023-10-06T04:53:08.097581+0000","last_clean":"2023-10-06T04:53:08.097581+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:53:08.097581+0000","last_undegraded":"2023-10-06T04:53:08.097581+0000","last_fullsized":"2023-10-06T04:53:08.097581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110022+0000","last_change":"2023-10-06T04:53:08.104332+0000","last_active":"2023-10-06T04:53:08.110022+0000","last_peered":"2023-10-06T04:53:08.110022+0000","last_clean":"2023-10-06T04:53:08.110022+0000","last_became_active":"2023-10-06T04:53:08.104075+0000","last_became_peered":"2023-10-06T04:53:08.104075+0000","last_unstale":"2023-10-06T04:53:08.110022+0000","last_undegraded":"2023-10-06T04:53:08.110022+0000","last_fullsized":"2023-10-06T04:53:08.110022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:52.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097584+0000","last_change":"2023-10-06T04:53:07.098446+0000","last_active":"2023-10-06T04:53:08.097584+0000","last_peered":"2023-10-06T04:53:08.097584+0000","last_clean":"2023-10-06T04:53:08.097584+0000","last_became_active":"2023-10-06T04:53:07.098140+0000","last_became_peered":"2023-10-06T04:53:07.098140+0000","last_unstale":"2023-10-06T04:53:08.097584+0000","last_undegraded":"2023-10-06T04:53:08.097584+0000","last_fullsized":"2023-10-06T04:53:08.097584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:47.702385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.105276+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:53:08.105276+0000","last_peered":"2023-10-06T04:53:08.105276+0000","last_clean":"2023-10-06T04:53:08.105276+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:53:08.105276+0000","last_undegraded":"2023-10-06T04:53:08.105276+0000","last_fullsized":"2023-10-06T04:53:08.105276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113189+0000","last_change":"2023-10-06T04:53:08.109536+0000","last_active":"2023-10-06T04:53:08.113189+0000","last_peered":"2023-10-06T04:53:08.113189+0000","last_clean":"2023-10-06T04:53:08.113189+0000","last_became_active":"2023-10-06T04:53:08.109269+0000","last_became_peered":"2023-10-06T04:53:08.109269+0000","last_unstale":"2023-10-06T04:53:08.113189+0000","last_undegraded":"2023-10-06T04:53:08.113189+0000","last_fullsized":"2023-10-06T04:53:08.113189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:18.822493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096187+0000","last_change":"2023-10-06T04:53:07.104825+0000","last_active":"2023-10-06T04:53:08.096187+0000","last_peered":"2023-10-06T04:53:08.096187+0000","last_clean":"2023-10-06T04:53:08.096187+0000","last_became_active":"2023-10-06T04:53:07.104572+0000","last_became_peered":"2023-10-06T04:53:07.104572+0000","last_unstale":"2023-10-06T04:53:08.096187+0000","last_undegraded":"2023-10-06T04:53:08.096187+0000","last_fullsized":"2023-10-06T04:53:08.096187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:43.511233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098274+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:53:08.098274+0000","last_peered":"2023-10-06T04:53:08.098274+0000","last_clean":"2023-10-06T04:53:08.098274+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:53:08.098274+0000","last_undegraded":"2023-10-06T04:53:08.098274+0000","last_fullsized":"2023-10-06T04:53:08.098274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110259+0000","last_change":"2023-10-06T04:53:08.103680+0000","last_active":"2023-10-06T04:53:08.110259+0000","last_peered":"2023-10-06T04:53:08.110259+0000","last_clean":"2023-10-06T04:53:08.110259+0000","last_became_active":"2023-10-06T04:53:08.103393+0000","last_became_peered":"2023-10-06T04:53:08.103393+0000","last_unstale":"2023-10-06T04:53:08.110259+0000","last_undegraded":"2023-10-06T04:53:08.110259+0000","last_fullsized":"2023-10-06T04:53:08.110259+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096340+0000","last_change":"2023-10-06T04:53:07.103554+0000","last_active":"2023-10-06T04:53:08.096340+0000","last_peered":"2023-10-06T04:53:08.096340+0000","last_clean":"2023-10-06T04:53:08.096340+0000","last_became_active":"2023-10-06T04:53:07.103281+0000","last_became_peered":"2023-10-06T04:53:07.103281+0000","last_unstale":"2023-10-06T04:53:08.096340+0000","last_undegraded":"2023-10-06T04:53:08.096340+0000","last_fullsized":"2023-10-06T04:53:08.096340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:28.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097695+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:53:08.097695+0000","last_peered":"2023-10-06T04:53:08.097695+0000","last_clean":"2023-10-06T04:53:08.097695+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:53:08.097695+0000","last_undegraded":"2023-10-06T04:53:08.097695+0000","last_fullsized":"2023-10-06T04:53:08.097695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":737412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":954368,"data_stored":735896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":148361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":147367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":145550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:53:12.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:53:12.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.357+0000 7f735bd80700 1 Processor -- start 2023-10-06T04:53:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.357+0000 7f735bd80700 1 -- start start 2023-10-06T04:53:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735bd80700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7354159500 0x7f73540cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735bd80700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f73540cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735bd80700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73540cc5c0 con 0x7f73540b2220 2023-10-06T04:53:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735bd80700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73540cb7b0 con 0x7f7354159500 2023-10-06T04:53:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735bd80700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73540cda60 con 0x7f73541598d0 2023-10-06T04:53:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735931b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f73540cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f7359b1c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7354159500 0x7f73540cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735931b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f73540cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:58050/0 (socket says 172.21.15.3:58050) 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735931b700 1 -- 172.21.15.3:0/3111453682 learned_addr learned my addr 172.21.15.3:0/3111453682 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.358+0000 7f735a31d700 1 --1- 172.21.15.3:0/3111453682 >> v1:172.21.15.153:6789/0 conn(0x7f73540b2220 0x7f73540b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:48850/0 (socket says 172.21.15.3:48850) 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f735931b700 1 -- 172.21.15.3:0/3111453682 >> v2:172.21.15.153:3300/0 conn(0x7f7354159500 msgr2=0x7f73540cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f735931b700 1 --2- 172.21.15.3:0/3111453682 >> v2:172.21.15.153:3300/0 conn(0x7f7354159500 0x7f73540cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f735931b700 1 -- 172.21.15.3:0/3111453682 >> v1:172.21.15.153:6789/0 conn(0x7f73540b2220 legacy=0x7f73540b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f735931b700 1 -- 172.21.15.3:0/3111453682 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540cb9b0 con 0x7f73541598d0 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f7358b1a700 1 -- 172.21.15.3:0/3111453682 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451542273 0 0) 0x7f73540cc5c0 con 0x7f73540b2220 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f7359b1c700 1 --2- 172.21.15.3:0/3111453682 >> v2:172.21.15.153:3300/0 conn(0x7f7354159500 0x7f73540cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f735931b700 1 --2- 172.21.15.3:0/3111453682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f73540cb270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73440099b0 tx=0x7f7344009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ed9721d7046bec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f7358b1a700 1 -- 172.21.15.3:0/3111453682 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344009ea0 con 0x7f73541598d0 2023-10-06T04:53:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f7358b1a700 1 -- 172.21.15.3:0/3111453682 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734400b860 con 0x7f73541598d0 2023-10-06T04:53:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f7358b1a700 1 -- 172.21.15.3:0/3111453682 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400bc00 con 0x7f73541598d0 2023-10-06T04:53:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f735bd80700 1 -- 172.21.15.3:0/3111453682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 msgr2=0x7f73540cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.359+0000 7f735bd80700 1 --2- 172.21.15.3:0/3111453682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f73540cb270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73440099b0 tx=0x7f7344009250 comp rx=0 tx=0).stop 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.360+0000 7f735bd80700 1 -- 172.21.15.3:0/3111453682 shutdown_connections 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.360+0000 7f735bd80700 1 --2- 172.21.15.3:0/3111453682 >> v2:172.21.15.153:3300/0 conn(0x7f7354159500 0x7f73540cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.360+0000 7f735bd80700 1 --2- 172.21.15.3:0/3111453682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f73540cb270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.360+0000 7f735bd80700 1 -- 172.21.15.3:0/3111453682 >> 172.21.15.3:0/3111453682 conn(0x7f7354070310 msgr2=0x7f7354070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.360+0000 7f735bd80700 1 -- 172.21.15.3:0/3111453682 shutdown_connections 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.360+0000 7f735bd80700 1 -- 172.21.15.3:0/3111453682 wait complete. 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.361+0000 7f735bd80700 1 Processor -- start 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.361+0000 7f735bd80700 1 -- start start 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.361+0000 7f735bd80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73540b2220 0x7f73540c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.361+0000 7f735bd80700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7354159500 0x7f73540d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.361+0000 7f735bd80700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f735414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.361+0000 7f735bd80700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540c8fe0 con 0x7f73540b2220 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.361+0000 7f735bd80700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540c92c0 con 0x7f7354159500 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f735bd80700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73540c95a0 con 0x7f73541598d0 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73540b2220 0x7f73540c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73540b2220 0x7f73540c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57298/0 (socket says 172.21.15.3:57298) 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 -- 172.21.15.3:0/1644309799 learned_addr learned my addr 172.21.15.3:0/1644309799 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f735931b700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7354159500 0x7f73540d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f735a31d700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f735414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 -- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 msgr2=0x7f735414f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f735414f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 -- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7354159500 msgr2=0x7f73540d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7354159500 0x7f73540d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 -- 172.21.15.3:0/1644309799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350009420 con 0x7f73540b2220 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7359b1c700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73540b2220 0x7f73540c8080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f735000ec80 tx=0x7f73500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7349ffb700 1 -- 172.21.15.3:0/1644309799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735000ca60 con 0x7f73540b2220 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7349ffb700 1 -- 172.21.15.3:0/1644309799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735000cbc0 con 0x7f73540b2220 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73440094f0 con 0x7f73540b2220 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.362+0000 7f7349ffb700 1 -- 172.21.15.3:0/1644309799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350013470 con 0x7f73540b2220 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.363+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73540c85c0 con 0x7f73540b2220 2023-10-06T04:53:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.364+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73540c9990 con 0x7f73540b2220 2023-10-06T04:53:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.364+0000 7f7349ffb700 1 -- 172.21.15.3:0/1644309799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7350013820 con 0x7f73540b2220 2023-10-06T04:53:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.364+0000 7f7349ffb700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7338066f30 0x7f73380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.365+0000 7f7349ffb700 1 -- 172.21.15.3:0/1644309799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7350084bf0 con 0x7f73540b2220 2023-10-06T04:53:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.365+0000 7f735931b700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7338066f30 0x7f73380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.365+0000 7f735931b700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7338066f30 0x7f73380693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73440099b0 tx=0x7f7344017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.369+0000 7f7349ffb700 1 -- 172.21.15.3:0/1644309799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7350051d00 con 0x7f73540b2220 2023-10-06T04:53:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.557+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73540c97c0 con 0x7f7338066f30 2023-10-06T04:53:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.566+0000 7f7349ffb700 1 -- 172.21.15.3:0/1644309799 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286931 (secure 0 0 0) 0x7f734400b9d0 con 0x7f7338066f30 2023-10-06T04:53:12.570 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7338066f30 msgr2=0x7f73380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7338066f30 0x7f73380693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73440099b0 tx=0x7f7344017000 comp rx=0 tx=0).stop 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73540b2220 msgr2=0x7f73540c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73540b2220 0x7f73540c8080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f735000ec80 tx=0x7f73500091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 shutdown_connections 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7338066f30 0x7f73380693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73541598d0 0x7f735414f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73540b2220 0x7f73540c8080 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 --2- 172.21.15.3:0/1644309799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7354159500 0x7f73540d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 >> 172.21.15.3:0/1644309799 conn(0x7f7354070310 msgr2=0x7f7354150880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:12.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 shutdown_connections 2023-10-06T04:53:12.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.569+0000 7f735bd80700 1 -- 172.21.15.3:0/1644309799 wait complete. 2023-10-06T04:53:12.574 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:53:12.582 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2023-10-06T04:53:11.435556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382332,"kb_used_data":4028,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592388,"statfs":{"total":773094113280,"available":770654605312,"internally_reserved":0,"allocated":4124672,"data_stored":2355617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.350234"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.108563+0000","last_change":"2023-10-06T04:53:08.103583+0000","last_active":"2023-10-06T04:53:08.108563+0000","last_peered":"2023-10-06T04:53:08.108563+0000","last_clean":"2023-10-06T04:53:08.108563+0000","last_became_active":"2023-10-06T04:53:08.102833+0000","last_became_peered":"2023-10-06T04:53:08.102833+0000","last_unstale":"2023-10-06T04:53:08.108563+0000","last_undegraded":"2023-10-06T04:53:08.108563+0000","last_fullsized":"2023-10-06T04:53:08.108563+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.546936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110034+0000","last_change":"2023-10-06T04:53:08.106695+0000","last_active":"2023-10-06T04:53:08.110034+0000","last_peered":"2023-10-06T04:53:08.110034+0000","last_clean":"2023-10-06T04:53:08.110034+0000","last_became_active":"2023-10-06T04:53:08.106375+0000","last_became_peered":"2023-10-06T04:53:08.106375+0000","last_unstale":"2023-10-06T04:53:08.110034+0000","last_undegraded":"2023-10-06T04:53:08.110034+0000","last_fullsized":"2023-10-06T04:53:08.110034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:33.319235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119197+0000","last_change":"2023-10-06T04:53:08.112093+0000","last_active":"2023-10-06T04:53:08.119197+0000","last_peered":"2023-10-06T04:53:08.119197+0000","last_clean":"2023-10-06T04:53:08.119197+0000","last_became_active":"2023-10-06T04:53:08.111813+0000","last_became_peered":"2023-10-06T04:53:08.111813+0000","last_unstale":"2023-10-06T04:53:08.119197+0000","last_undegraded":"2023-10-06T04:53:08.119197+0000","last_fullsized":"2023-10-06T04:53:08.119197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:50.075234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109744+0000","last_change":"2023-10-06T04:53:08.103857+0000","last_active":"2023-10-06T04:53:08.109744+0000","last_peered":"2023-10-06T04:53:08.109744+0000","last_clean":"2023-10-06T04:53:08.109744+0000","last_became_active":"2023-10-06T04:53:08.103436+0000","last_became_peered":"2023-10-06T04:53:08.103436+0000","last_unstale":"2023-10-06T04:53:08.109744+0000","last_undegraded":"2023-10-06T04:53:08.109744+0000","last_fullsized":"2023-10-06T04:53:08.109744+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:14.317772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.117525+0000","last_change":"2023-10-06T04:53:08.112255+0000","last_active":"2023-10-06T04:53:08.117525+0000","last_peered":"2023-10-06T04:53:08.117525+0000","last_clean":"2023-10-06T04:53:08.117525+0000","last_became_active":"2023-10-06T04:53:08.111963+0000","last_became_peered":"2023-10-06T04:53:08.111963+0000","last_unstale":"2023-10-06T04:53:08.117525+0000","last_undegraded":"2023-10-06T04:53:08.117525+0000","last_fullsized":"2023-10-06T04:53:08.117525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:01.830386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.106082+0000","last_change":"2023-10-06T04:53:08.099092+0000","last_active":"2023-10-06T04:53:08.106082+0000","last_peered":"2023-10-06T04:53:08.106082+0000","last_clean":"2023-10-06T04:53:08.106082+0000","last_became_active":"2023-10-06T04:53:08.098824+0000","last_became_peered":"2023-10-06T04:53:08.098824+0000","last_unstale":"2023-10-06T04:53:08.106082+0000","last_undegraded":"2023-10-06T04:53:08.106082+0000","last_fullsized":"2023-10-06T04:53:08.106082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:26.746368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118633+0000","last_change":"2023-10-06T04:53:08.111080+0000","last_active":"2023-10-06T04:53:08.118633+0000","last_peered":"2023-10-06T04:53:08.118633+0000","last_clean":"2023-10-06T04:53:08.118633+0000","last_became_active":"2023-10-06T04:53:08.110785+0000","last_became_peered":"2023-10-06T04:53:08.110785+0000","last_unstale":"2023-10-06T04:53:08.118633+0000","last_undegraded":"2023-10-06T04:53:08.118633+0000","last_fullsized":"2023-10-06T04:53:08.118633+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:33.933870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109694+0000","last_change":"2023-10-06T04:53:08.107323+0000","last_active":"2023-10-06T04:53:08.109694+0000","last_peered":"2023-10-06T04:53:08.109694+0000","last_clean":"2023-10-06T04:53:08.109694+0000","last_became_active":"2023-10-06T04:53:08.107025+0000","last_became_peered":"2023-10-06T04:53:08.107025+0000","last_unstale":"2023-10-06T04:53:08.109694+0000","last_undegraded":"2023-10-06T04:53:08.109694+0000","last_fullsized":"2023-10-06T04:53:08.109694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119570+0000","last_change":"2023-10-06T04:53:08.112717+0000","last_active":"2023-10-06T04:53:08.119570+0000","last_peered":"2023-10-06T04:53:08.119570+0000","last_clean":"2023-10-06T04:53:08.119570+0000","last_became_active":"2023-10-06T04:53:08.112459+0000","last_became_peered":"2023-10-06T04:53:08.112459+0000","last_unstale":"2023-10-06T04:53:08.119570+0000","last_undegraded":"2023-10-06T04:53:08.119570+0000","last_fullsized":"2023-10-06T04:53:08.119570+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:12.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118845+0000","last_change":"2023-10-06T04:53:08.113858+0000","last_active":"2023-10-06T04:53:08.118845+0000","last_peered":"2023-10-06T04:53:08.118845+0000","last_clean":"2023-10-06T04:53:08.118845+0000","last_became_active":"2023-10-06T04:53:08.113584+0000","last_became_peered":"2023-10-06T04:53:08.113584+0000","last_unstale":"2023-10-06T04:53:08.118845+0000","last_undegraded":"2023-10-06T04:53:08.118845+0000","last_fullsized":"2023-10-06T04:53:08.118845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:17.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118792+0000","last_change":"2023-10-06T04:53:08.112570+0000","last_active":"2023-10-06T04:53:08.118792+0000","last_peered":"2023-10-06T04:53:08.118792+0000","last_clean":"2023-10-06T04:53:08.118792+0000","last_became_active":"2023-10-06T04:53:08.112144+0000","last_became_peered":"2023-10-06T04:53:08.112144+0000","last_unstale":"2023-10-06T04:53:08.118792+0000","last_undegraded":"2023-10-06T04:53:08.118792+0000","last_fullsized":"2023-10-06T04:53:08.118792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:29.212376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119244+0000","last_change":"2023-10-06T04:53:08.109792+0000","last_active":"2023-10-06T04:53:08.119244+0000","last_peered":"2023-10-06T04:53:08.119244+0000","last_clean":"2023-10-06T04:53:08.119244+0000","last_became_active":"2023-10-06T04:53:08.109510+0000","last_became_peered":"2023-10-06T04:53:08.109510+0000","last_unstale":"2023-10-06T04:53:08.119244+0000","last_undegraded":"2023-10-06T04:53:08.119244+0000","last_fullsized":"2023-10-06T04:53:08.119244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:29.212376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122676+0000","last_change":"2023-10-06T04:53:08.119816+0000","last_active":"2023-10-06T04:53:08.122676+0000","last_peered":"2023-10-06T04:53:08.122676+0000","last_clean":"2023-10-06T04:53:08.122676+0000","last_became_active":"2023-10-06T04:53:08.119536+0000","last_became_peered":"2023-10-06T04:53:08.119536+0000","last_unstale":"2023-10-06T04:53:08.122676+0000","last_undegraded":"2023-10-06T04:53:08.122676+0000","last_fullsized":"2023-10-06T04:53:08.122676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.546936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120990+0000","last_change":"2023-10-06T04:53:08.110528+0000","last_active":"2023-10-06T04:53:08.120990+0000","last_peered":"2023-10-06T04:53:08.120990+0000","last_clean":"2023-10-06T04:53:08.120990+0000","last_became_active":"2023-10-06T04:53:08.110301+0000","last_became_peered":"2023-10-06T04:53:08.110301+0000","last_unstale":"2023-10-06T04:53:08.120990+0000","last_undegraded":"2023-10-06T04:53:08.120990+0000","last_fullsized":"2023-10-06T04:53:08.120990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:20.022067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121238+0000","last_change":"2023-10-06T04:53:08.114696+0000","last_active":"2023-10-06T04:53:08.121238+0000","last_peered":"2023-10-06T04:53:08.121238+0000","last_clean":"2023-10-06T04:53:08.121238+0000","last_became_active":"2023-10-06T04:53:08.113005+0000","last_became_peered":"2023-10-06T04:53:08.113005+0000","last_unstale":"2023-10-06T04:53:08.121238+0000","last_undegraded":"2023-10-06T04:53:08.121238+0000","last_fullsized":"2023-10-06T04:53:08.121238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:52.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121817+0000","last_change":"2023-10-06T04:53:08.115316+0000","last_active":"2023-10-06T04:53:08.121817+0000","last_peered":"2023-10-06T04:53:08.121817+0000","last_clean":"2023-10-06T04:53:08.121817+0000","last_became_active":"2023-10-06T04:53:08.114411+0000","last_became_peered":"2023-10-06T04:53:08.114411+0000","last_unstale":"2023-10-06T04:53:08.121817+0000","last_undegraded":"2023-10-06T04:53:08.121817+0000","last_fullsized":"2023-10-06T04:53:08.121817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:29.212376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120249+0000","last_change":"2023-10-06T04:53:08.113641+0000","last_active":"2023-10-06T04:53:08.120249+0000","last_peered":"2023-10-06T04:53:08.120249+0000","last_clean":"2023-10-06T04:53:08.120249+0000","last_became_active":"2023-10-06T04:53:08.112929+0000","last_became_peered":"2023-10-06T04:53:08.112929+0000","last_unstale":"2023-10-06T04:53:08.120249+0000","last_undegraded":"2023-10-06T04:53:08.120249+0000","last_fullsized":"2023-10-06T04:53:08.120249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:50.075234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118642+0000","last_change":"2023-10-06T04:53:08.113744+0000","last_active":"2023-10-06T04:53:08.118642+0000","last_peered":"2023-10-06T04:53:08.118642+0000","last_clean":"2023-10-06T04:53:08.118642+0000","last_became_active":"2023-10-06T04:53:08.113456+0000","last_became_peered":"2023-10-06T04:53:08.113456+0000","last_unstale":"2023-10-06T04:53:08.118642+0000","last_undegraded":"2023-10-06T04:53:08.118642+0000","last_fullsized":"2023-10-06T04:53:08.118642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:01.830386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.108837+0000","last_change":"2023-10-06T04:53:08.099322+0000","last_active":"2023-10-06T04:53:08.108837+0000","last_peered":"2023-10-06T04:53:08.108837+0000","last_clean":"2023-10-06T04:53:08.108837+0000","last_became_active":"2023-10-06T04:53:08.099066+0000","last_became_peered":"2023-10-06T04:53:08.099066+0000","last_unstale":"2023-10-06T04:53:08.108837+0000","last_undegraded":"2023-10-06T04:53:08.108837+0000","last_fullsized":"2023-10-06T04:53:08.108837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:50.183857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113845+0000","last_change":"2023-10-06T04:53:08.106650+0000","last_active":"2023-10-06T04:53:08.113845+0000","last_peered":"2023-10-06T04:53:08.113845+0000","last_clean":"2023-10-06T04:53:08.113845+0000","last_became_active":"2023-10-06T04:53:08.106280+0000","last_became_peered":"2023-10-06T04:53:08.106280+0000","last_unstale":"2023-10-06T04:53:08.113845+0000","last_undegraded":"2023-10-06T04:53:08.113845+0000","last_fullsized":"2023-10-06T04:53:08.113845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:05.753883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118802+0000","last_change":"2023-10-06T04:53:08.112581+0000","last_active":"2023-10-06T04:53:08.118802+0000","last_peered":"2023-10-06T04:53:08.118802+0000","last_clean":"2023-10-06T04:53:08.118802+0000","last_became_active":"2023-10-06T04:53:08.112244+0000","last_became_peered":"2023-10-06T04:53:08.112244+0000","last_unstale":"2023-10-06T04:53:08.118802+0000","last_undegraded":"2023-10-06T04:53:08.118802+0000","last_fullsized":"2023-10-06T04:53:08.118802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:12.639355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123111+0000","last_change":"2023-10-06T04:53:08.118537+0000","last_active":"2023-10-06T04:53:08.123111+0000","last_peered":"2023-10-06T04:53:08.123111+0000","last_clean":"2023-10-06T04:53:08.123111+0000","last_became_active":"2023-10-06T04:53:08.116925+0000","last_became_peered":"2023-10-06T04:53:08.116925+0000","last_unstale":"2023-10-06T04:53:08.123111+0000","last_undegraded":"2023-10-06T04:53:08.123111+0000","last_fullsized":"2023-10-06T04:53:08.123111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:11.783956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.116568+0000","last_change":"2023-10-06T04:53:08.109517+0000","last_active":"2023-10-06T04:53:08.116568+0000","last_peered":"2023-10-06T04:53:08.116568+0000","last_clean":"2023-10-06T04:53:08.116568+0000","last_became_active":"2023-10-06T04:53:08.109226+0000","last_became_peered":"2023-10-06T04:53:08.109226+0000","last_unstale":"2023-10-06T04:53:08.116568+0000","last_undegraded":"2023-10-06T04:53:08.116568+0000","last_fullsized":"2023-10-06T04:53:08.116568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:12.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109896+0000","last_change":"2023-10-06T04:53:08.108399+0000","last_active":"2023-10-06T04:53:08.109896+0000","last_peered":"2023-10-06T04:53:08.109896+0000","last_clean":"2023-10-06T04:53:08.109896+0000","last_became_active":"2023-10-06T04:53:08.108126+0000","last_became_peered":"2023-10-06T04:53:08.108126+0000","last_unstale":"2023-10-06T04:53:08.109896+0000","last_undegraded":"2023-10-06T04:53:08.109896+0000","last_fullsized":"2023-10-06T04:53:08.109896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:05.753883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.114384+0000","last_change":"2023-10-06T04:53:08.106546+0000","last_active":"2023-10-06T04:53:08.114384+0000","last_peered":"2023-10-06T04:53:08.114384+0000","last_clean":"2023-10-06T04:53:08.114384+0000","last_became_active":"2023-10-06T04:53:08.106241+0000","last_became_peered":"2023-10-06T04:53:08.106241+0000","last_unstale":"2023-10-06T04:53:08.114384+0000","last_undegraded":"2023-10-06T04:53:08.114384+0000","last_fullsized":"2023-10-06T04:53:08.114384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:48.709130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.111739+0000","last_change":"2023-10-06T04:53:08.105986+0000","last_active":"2023-10-06T04:53:08.111739+0000","last_peered":"2023-10-06T04:53:08.111739+0000","last_clean":"2023-10-06T04:53:08.111739+0000","last_became_active":"2023-10-06T04:53:08.105553+0000","last_became_peered":"2023-10-06T04:53:08.105553+0000","last_unstale":"2023-10-06T04:53:08.111739+0000","last_undegraded":"2023-10-06T04:53:08.111739+0000","last_fullsized":"2023-10-06T04:53:08.111739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:58.365320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118689+0000","last_change":"2023-10-06T04:53:08.113131+0000","last_active":"2023-10-06T04:53:08.118689+0000","last_peered":"2023-10-06T04:53:08.118689+0000","last_clean":"2023-10-06T04:53:08.118689+0000","last_became_active":"2023-10-06T04:53:08.112825+0000","last_became_peered":"2023-10-06T04:53:08.112825+0000","last_unstale":"2023-10-06T04:53:08.118689+0000","last_undegraded":"2023-10-06T04:53:08.118689+0000","last_fullsized":"2023-10-06T04:53:08.118689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:41.247813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118582+0000","last_change":"2023-10-06T04:53:08.110462+0000","last_active":"2023-10-06T04:53:08.118582+0000","last_peered":"2023-10-06T04:53:08.118582+0000","last_clean":"2023-10-06T04:53:08.118582+0000","last_became_active":"2023-10-06T04:53:08.110144+0000","last_became_peered":"2023-10-06T04:53:08.110144+0000","last_unstale":"2023-10-06T04:53:08.118582+0000","last_undegraded":"2023-10-06T04:53:08.118582+0000","last_fullsized":"2023-10-06T04:53:08.118582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113794+0000","last_change":"2023-10-06T04:53:08.108356+0000","last_active":"2023-10-06T04:53:08.113794+0000","last_peered":"2023-10-06T04:53:08.113794+0000","last_clean":"2023-10-06T04:53:08.113794+0000","last_became_active":"2023-10-06T04:53:08.108020+0000","last_became_peered":"2023-10-06T04:53:08.108020+0000","last_unstale":"2023-10-06T04:53:08.113794+0000","last_undegraded":"2023-10-06T04:53:08.113794+0000","last_fullsized":"2023-10-06T04:53:08.113794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:14.317772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113475+0000","last_change":"2023-10-06T04:53:08.105781+0000","last_active":"2023-10-06T04:53:08.113475+0000","last_peered":"2023-10-06T04:53:08.113475+0000","last_clean":"2023-10-06T04:53:08.113475+0000","last_became_active":"2023-10-06T04:53:08.105559+0000","last_became_peered":"2023-10-06T04:53:08.105559+0000","last_unstale":"2023-10-06T04:53:08.113475+0000","last_undegraded":"2023-10-06T04:53:08.113475+0000","last_fullsized":"2023-10-06T04:53:08.113475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:29.052698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121916+0000","last_change":"2023-10-06T04:53:08.115148+0000","last_active":"2023-10-06T04:53:08.121916+0000","last_peered":"2023-10-06T04:53:08.121916+0000","last_clean":"2023-10-06T04:53:08.121916+0000","last_became_active":"2023-10-06T04:53:08.114930+0000","last_became_peered":"2023-10-06T04:53:08.114930+0000","last_unstale":"2023-10-06T04:53:08.121916+0000","last_undegraded":"2023-10-06T04:53:08.121916+0000","last_fullsized":"2023-10-06T04:53:08.121916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:52.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.106261+0000","last_change":"2023-10-06T04:53:08.098992+0000","last_active":"2023-10-06T04:53:08.106261+0000","last_peered":"2023-10-06T04:53:08.106261+0000","last_clean":"2023-10-06T04:53:08.106261+0000","last_became_active":"2023-10-06T04:53:08.098744+0000","last_became_peered":"2023-10-06T04:53:08.098744+0000","last_unstale":"2023-10-06T04:53:08.106261+0000","last_undegraded":"2023-10-06T04:53:08.106261+0000","last_fullsized":"2023-10-06T04:53:08.106261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:10.735694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122303+0000","last_change":"2023-10-06T04:53:08.113103+0000","last_active":"2023-10-06T04:53:08.122303+0000","last_peered":"2023-10-06T04:53:08.122303+0000","last_clean":"2023-10-06T04:53:08.122303+0000","last_became_active":"2023-10-06T04:53:08.112806+0000","last_became_peered":"2023-10-06T04:53:08.112806+0000","last_unstale":"2023-10-06T04:53:08.122303+0000","last_undegraded":"2023-10-06T04:53:08.122303+0000","last_fullsized":"2023-10-06T04:53:08.122303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:12.639355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123222+0000","last_change":"2023-10-06T04:53:07.104308+0000","last_active":"2023-10-06T04:53:08.123222+0000","last_peered":"2023-10-06T04:53:08.123222+0000","last_clean":"2023-10-06T04:53:08.123222+0000","last_became_active":"2023-10-06T04:53:07.103696+0000","last_became_peered":"2023-10-06T04:53:07.103696+0000","last_unstale":"2023-10-06T04:53:08.123222+0000","last_undegraded":"2023-10-06T04:53:08.123222+0000","last_fullsized":"2023-10-06T04:53:08.123222+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:57.514050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.117347+0000","last_change":"2023-10-06T04:53:08.113134+0000","last_active":"2023-10-06T04:53:08.117347+0000","last_peered":"2023-10-06T04:53:08.117347+0000","last_clean":"2023-10-06T04:53:08.117347+0000","last_became_active":"2023-10-06T04:53:08.112336+0000","last_became_peered":"2023-10-06T04:53:08.112336+0000","last_unstale":"2023-10-06T04:53:08.117347+0000","last_undegraded":"2023-10-06T04:53:08.117347+0000","last_fullsized":"2023-10-06T04:53:08.117347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:29.364724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120507+0000","last_change":"2023-10-06T04:53:07.106919+0000","last_active":"2023-10-06T04:53:08.120507+0000","last_peered":"2023-10-06T04:53:08.120507+0000","last_clean":"2023-10-06T04:53:08.120507+0000","last_became_active":"2023-10-06T04:53:07.106653+0000","last_became_peered":"2023-10-06T04:53:07.106653+0000","last_unstale":"2023-10-06T04:53:08.120507+0000","last_undegraded":"2023-10-06T04:53:08.120507+0000","last_fullsized":"2023-10-06T04:53:08.120507+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:40.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120712+0000","last_change":"2023-10-06T04:53:08.110653+0000","last_active":"2023-10-06T04:53:08.120712+0000","last_peered":"2023-10-06T04:53:08.120712+0000","last_clean":"2023-10-06T04:53:08.120712+0000","last_became_active":"2023-10-06T04:53:08.110390+0000","last_became_peered":"2023-10-06T04:53:08.110390+0000","last_unstale":"2023-10-06T04:53:08.120712+0000","last_undegraded":"2023-10-06T04:53:08.120712+0000","last_fullsized":"2023-10-06T04:53:08.120712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:21.350284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122650+0000","last_change":"2023-10-06T04:53:07.098879+0000","last_active":"2023-10-06T04:53:08.122650+0000","last_peered":"2023-10-06T04:53:08.122650+0000","last_clean":"2023-10-06T04:53:08.122650+0000","last_became_active":"2023-10-06T04:53:07.097717+0000","last_became_peered":"2023-10-06T04:53:07.097717+0000","last_unstale":"2023-10-06T04:53:08.122650+0000","last_undegraded":"2023-10-06T04:53:08.122650+0000","last_fullsized":"2023-10-06T04:53:08.122650+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:00.823641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110470+0000","last_change":"2023-10-06T04:53:08.107923+0000","last_active":"2023-10-06T04:53:08.110470+0000","last_peered":"2023-10-06T04:53:08.110470+0000","last_clean":"2023-10-06T04:53:08.110470+0000","last_became_active":"2023-10-06T04:53:08.107634+0000","last_became_peered":"2023-10-06T04:53:08.107634+0000","last_unstale":"2023-10-06T04:53:08.110470+0000","last_undegraded":"2023-10-06T04:53:08.110470+0000","last_fullsized":"2023-10-06T04:53:08.110470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123274+0000","last_change":"2023-10-06T04:53:07.103838+0000","last_active":"2023-10-06T04:53:08.123274+0000","last_peered":"2023-10-06T04:53:08.123274+0000","last_clean":"2023-10-06T04:53:08.123274+0000","last_became_active":"2023-10-06T04:53:07.103574+0000","last_became_peered":"2023-10-06T04:53:07.103574+0000","last_unstale":"2023-10-06T04:53:08.123274+0000","last_undegraded":"2023-10-06T04:53:08.123274+0000","last_fullsized":"2023-10-06T04:53:08.123274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:57.358575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113236+0000","last_change":"2023-10-06T04:53:08.106864+0000","last_active":"2023-10-06T04:53:08.113236+0000","last_peered":"2023-10-06T04:53:08.113236+0000","last_clean":"2023-10-06T04:53:08.113236+0000","last_became_active":"2023-10-06T04:53:08.106603+0000","last_became_peered":"2023-10-06T04:53:08.106603+0000","last_unstale":"2023-10-06T04:53:08.113236+0000","last_undegraded":"2023-10-06T04:53:08.113236+0000","last_fullsized":"2023-10-06T04:53:08.113236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:23.857041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121358+0000","last_change":"2023-10-06T04:53:07.103893+0000","last_active":"2023-10-06T04:53:08.121358+0000","last_peered":"2023-10-06T04:53:08.121358+0000","last_clean":"2023-10-06T04:53:08.121358+0000","last_became_active":"2023-10-06T04:53:07.103652+0000","last_became_peered":"2023-10-06T04:53:07.103652+0000","last_unstale":"2023-10-06T04:53:08.121358+0000","last_undegraded":"2023-10-06T04:53:08.121358+0000","last_fullsized":"2023-10-06T04:53:08.121358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:47.702385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118986+0000","last_change":"2023-10-06T04:53:08.114549+0000","last_active":"2023-10-06T04:53:08.118986+0000","last_peered":"2023-10-06T04:53:08.118986+0000","last_clean":"2023-10-06T04:53:08.118986+0000","last_became_active":"2023-10-06T04:53:08.113921+0000","last_became_peered":"2023-10-06T04:53:08.113921+0000","last_unstale":"2023-10-06T04:53:08.118986+0000","last_undegraded":"2023-10-06T04:53:08.118986+0000","last_fullsized":"2023-10-06T04:53:08.118986+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:26.083562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.124847+0000","last_change":"2023-10-06T04:53:07.103898+0000","last_active":"2023-10-06T04:53:08.124847+0000","last_peered":"2023-10-06T04:53:08.124847+0000","last_clean":"2023-10-06T04:53:08.124847+0000","last_became_active":"2023-10-06T04:53:07.103669+0000","last_became_peered":"2023-10-06T04:53:07.103669+0000","last_unstale":"2023-10-06T04:53:08.124847+0000","last_undegraded":"2023-10-06T04:53:08.124847+0000","last_fullsized":"2023-10-06T04:53:08.124847+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:28.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123092+0000","last_change":"2023-10-06T04:53:08.118542+0000","last_active":"2023-10-06T04:53:08.123092+0000","last_peered":"2023-10-06T04:53:08.123092+0000","last_clean":"2023-10-06T04:53:08.123092+0000","last_became_active":"2023-10-06T04:53:08.118282+0000","last_became_peered":"2023-10-06T04:53:08.118282+0000","last_unstale":"2023-10-06T04:53:08.123092+0000","last_undegraded":"2023-10-06T04:53:08.123092+0000","last_fullsized":"2023-10-06T04:53:08.123092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:33.754555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091744+0000","last_change":"2023-10-06T04:53:07.107097+0000","last_active":"2023-10-06T04:53:08.091744+0000","last_peered":"2023-10-06T04:53:08.091744+0000","last_clean":"2023-10-06T04:53:08.091744+0000","last_became_active":"2023-10-06T04:53:07.106755+0000","last_became_peered":"2023-10-06T04:53:07.106755+0000","last_unstale":"2023-10-06T04:53:08.091744+0000","last_undegraded":"2023-10-06T04:53:08.091744+0000","last_fullsized":"2023-10-06T04:53:08.091744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:43.651185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.126318+0000","last_change":"2023-10-06T04:53:08.122305+0000","last_active":"2023-10-06T04:53:08.126318+0000","last_peered":"2023-10-06T04:53:08.126318+0000","last_clean":"2023-10-06T04:53:08.126318+0000","last_became_active":"2023-10-06T04:53:08.121988+0000","last_became_peered":"2023-10-06T04:53:08.121988+0000","last_unstale":"2023-10-06T04:53:08.126318+0000","last_undegraded":"2023-10-06T04:53:08.126318+0000","last_fullsized":"2023-10-06T04:53:08.126318+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:17.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098795+0000","last_change":"2023-10-06T04:53:07.102376+0000","last_active":"2023-10-06T04:53:08.098795+0000","last_peered":"2023-10-06T04:53:08.098795+0000","last_clean":"2023-10-06T04:53:08.098795+0000","last_became_active":"2023-10-06T04:53:07.102103+0000","last_became_peered":"2023-10-06T04:53:07.102103+0000","last_unstale":"2023-10-06T04:53:08.098795+0000","last_undegraded":"2023-10-06T04:53:08.098795+0000","last_fullsized":"2023-10-06T04:53:08.098795+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:28.205631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109721+0000","last_change":"2023-10-06T04:53:08.103674+0000","last_active":"2023-10-06T04:53:08.109721+0000","last_peered":"2023-10-06T04:53:08.109721+0000","last_clean":"2023-10-06T04:53:08.109721+0000","last_became_active":"2023-10-06T04:53:08.103115+0000","last_became_peered":"2023-10-06T04:53:08.103115+0000","last_unstale":"2023-10-06T04:53:08.109721+0000","last_undegraded":"2023-10-06T04:53:08.109721+0000","last_fullsized":"2023-10-06T04:53:08.109721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:44.657930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.092586+0000","last_change":"2023-10-06T04:53:07.104906+0000","last_active":"2023-10-06T04:53:08.092586+0000","last_peered":"2023-10-06T04:53:08.092586+0000","last_clean":"2023-10-06T04:53:08.092586+0000","last_became_active":"2023-10-06T04:53:07.104622+0000","last_became_peered":"2023-10-06T04:53:07.104622+0000","last_unstale":"2023-10-06T04:53:08.092586+0000","last_undegraded":"2023-10-06T04:53:08.092586+0000","last_fullsized":"2023-10-06T04:53:08.092586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:17.815747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120272+0000","last_change":"2023-10-06T04:53:08.113699+0000","last_active":"2023-10-06T04:53:08.120272+0000","last_peered":"2023-10-06T04:53:08.120272+0000","last_clean":"2023-10-06T04:53:08.120272+0000","last_became_active":"2023-10-06T04:53:08.113354+0000","last_became_peered":"2023-10-06T04:53:08.113354+0000","last_unstale":"2023-10-06T04:53:08.120272+0000","last_undegraded":"2023-10-06T04:53:08.120272+0000","last_fullsized":"2023-10-06T04:53:08.120272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:33.933870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.102480+0000","last_change":"2023-10-06T04:53:07.107135+0000","last_active":"2023-10-06T04:53:08.102480+0000","last_peered":"2023-10-06T04:53:08.102480+0000","last_clean":"2023-10-06T04:53:08.102480+0000","last_became_active":"2023-10-06T04:53:07.106792+0000","last_became_peered":"2023-10-06T04:53:07.106792+0000","last_unstale":"2023-10-06T04:53:08.102480+0000","last_undegraded":"2023-10-06T04:53:08.102480+0000","last_fullsized":"2023-10-06T04:53:08.102480+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:48.589723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119313+0000","last_change":"2023-10-06T04:53:08.107557+0000","last_active":"2023-10-06T04:53:08.119313+0000","last_peered":"2023-10-06T04:53:08.119313+0000","last_clean":"2023-10-06T04:53:08.119313+0000","last_became_active":"2023-10-06T04:53:08.107247+0000","last_became_peered":"2023-10-06T04:53:08.107247+0000","last_unstale":"2023-10-06T04:53:08.119313+0000","last_undegraded":"2023-10-06T04:53:08.119313+0000","last_fullsized":"2023-10-06T04:53:08.119313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121633+0000","last_change":"2023-10-06T04:53:07.104291+0000","last_active":"2023-10-06T04:53:08.121633+0000","last_peered":"2023-10-06T04:53:08.121633+0000","last_clean":"2023-10-06T04:53:08.121633+0000","last_became_active":"2023-10-06T04:53:07.104022+0000","last_became_peered":"2023-10-06T04:53:07.104022+0000","last_unstale":"2023-10-06T04:53:08.121633+0000","last_undegraded":"2023-10-06T04:53:08.121633+0000","last_fullsized":"2023-10-06T04:53:08.121633+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:40.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122121+0000","last_change":"2023-10-06T04:53:08.112439+0000","last_active":"2023-10-06T04:53:08.122121+0000","last_peered":"2023-10-06T04:53:08.122121+0000","last_clean":"2023-10-06T04:53:08.122121+0000","last_became_active":"2023-10-06T04:53:08.112137+0000","last_became_peered":"2023-10-06T04:53:08.112137+0000","last_unstale":"2023-10-06T04:53:08.122121+0000","last_undegraded":"2023-10-06T04:53:08.122121+0000","last_fullsized":"2023-10-06T04:53:08.122121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:26.083562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.126601+0000","last_change":"2023-10-06T04:53:07.109471+0000","last_active":"2023-10-06T04:53:08.126601+0000","last_peered":"2023-10-06T04:53:08.126601+0000","last_clean":"2023-10-06T04:53:08.126601+0000","last_became_active":"2023-10-06T04:53:07.109147+0000","last_became_peered":"2023-10-06T04:53:07.109147+0000","last_unstale":"2023-10-06T04:53:08.126601+0000","last_undegraded":"2023-10-06T04:53:08.126601+0000","last_fullsized":"2023-10-06T04:53:08.126601+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:25.739623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119459+0000","last_change":"2023-10-06T04:53:08.111156+0000","last_active":"2023-10-06T04:53:08.119459+0000","last_peered":"2023-10-06T04:53:08.119459+0000","last_clean":"2023-10-06T04:53:08.119459+0000","last_became_active":"2023-10-06T04:53:08.110818+0000","last_became_peered":"2023-10-06T04:53:08.110818+0000","last_unstale":"2023-10-06T04:53:08.119459+0000","last_undegraded":"2023-10-06T04:53:08.119459+0000","last_fullsized":"2023-10-06T04:53:08.119459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.206832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123201+0000","last_change":"2023-10-06T04:53:07.104633+0000","last_active":"2023-10-06T04:53:08.123201+0000","last_peered":"2023-10-06T04:53:08.123201+0000","last_clean":"2023-10-06T04:53:08.123201+0000","last_became_active":"2023-10-06T04:53:07.104373+0000","last_became_peered":"2023-10-06T04:53:07.104373+0000","last_unstale":"2023-10-06T04:53:08.123201+0000","last_undegraded":"2023-10-06T04:53:08.123201+0000","last_fullsized":"2023-10-06T04:53:08.123201+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:11.829383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109577+0000","last_change":"2023-10-06T04:53:08.106581+0000","last_active":"2023-10-06T04:53:08.109577+0000","last_peered":"2023-10-06T04:53:08.109577+0000","last_clean":"2023-10-06T04:53:08.109577+0000","last_became_active":"2023-10-06T04:53:08.106214+0000","last_became_peered":"2023-10-06T04:53:08.106214+0000","last_unstale":"2023-10-06T04:53:08.109577+0000","last_undegraded":"2023-10-06T04:53:08.109577+0000","last_fullsized":"2023-10-06T04:53:08.109577+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:26.083562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.092303+0000","last_change":"2023-10-06T04:53:07.106979+0000","last_active":"2023-10-06T04:53:08.092303+0000","last_peered":"2023-10-06T04:53:08.092303+0000","last_clean":"2023-10-06T04:53:08.092303+0000","last_became_active":"2023-10-06T04:53:07.106690+0000","last_became_peered":"2023-10-06T04:53:07.106690+0000","last_unstale":"2023-10-06T04:53:08.092303+0000","last_undegraded":"2023-10-06T04:53:08.092303+0000","last_fullsized":"2023-10-06T04:53:08.092303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:42.878681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120202+0000","last_change":"2023-10-06T04:53:08.112824+0000","last_active":"2023-10-06T04:53:08.120202+0000","last_peered":"2023-10-06T04:53:08.120202+0000","last_clean":"2023-10-06T04:53:08.120202+0000","last_became_active":"2023-10-06T04:53:08.112532+0000","last_became_peered":"2023-10-06T04:53:08.112532+0000","last_unstale":"2023-10-06T04:53:08.120202+0000","last_undegraded":"2023-10-06T04:53:08.120202+0000","last_fullsized":"2023-10-06T04:53:08.120202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.206832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.099152+0000","last_change":"2023-10-06T04:53:07.103726+0000","last_active":"2023-10-06T04:53:08.099152+0000","last_peered":"2023-10-06T04:53:08.099152+0000","last_clean":"2023-10-06T04:53:08.099152+0000","last_became_active":"2023-10-06T04:53:07.103457+0000","last_became_peered":"2023-10-06T04:53:07.103457+0000","last_unstale":"2023-10-06T04:53:08.099152+0000","last_undegraded":"2023-10-06T04:53:08.099152+0000","last_fullsized":"2023-10-06T04:53:08.099152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:14.302187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110437+0000","last_change":"2023-10-06T04:53:08.107829+0000","last_active":"2023-10-06T04:53:08.110437+0000","last_peered":"2023-10-06T04:53:08.110437+0000","last_clean":"2023-10-06T04:53:08.110437+0000","last_became_active":"2023-10-06T04:53:08.107522+0000","last_became_peered":"2023-10-06T04:53:08.107522+0000","last_unstale":"2023-10-06T04:53:08.110437+0000","last_undegraded":"2023-10-06T04:53:08.110437+0000","last_fullsized":"2023-10-06T04:53:08.110437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:11.783956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.125199+0000","last_change":"2023-10-06T04:53:07.106993+0000","last_active":"2023-10-06T04:53:08.125199+0000","last_peered":"2023-10-06T04:53:08.125199+0000","last_clean":"2023-10-06T04:53:08.125199+0000","last_became_active":"2023-10-06T04:53:07.106326+0000","last_became_peered":"2023-10-06T04:53:07.106326+0000","last_unstale":"2023-10-06T04:53:08.125199+0000","last_undegraded":"2023-10-06T04:53:08.125199+0000","last_fullsized":"2023-10-06T04:53:08.125199+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:04.747138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110453+0000","last_change":"2023-10-06T04:53:08.107921+0000","last_active":"2023-10-06T04:53:08.110453+0000","last_peered":"2023-10-06T04:53:08.110453+0000","last_clean":"2023-10-06T04:53:08.110453+0000","last_became_active":"2023-10-06T04:53:08.107656+0000","last_became_peered":"2023-10-06T04:53:08.107656+0000","last_unstale":"2023-10-06T04:53:08.110453+0000","last_undegraded":"2023-10-06T04:53:08.110453+0000","last_fullsized":"2023-10-06T04:53:08.110453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.206832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123598+0000","last_change":"2023-10-06T04:53:07.107107+0000","last_active":"2023-10-06T04:53:08.123598+0000","last_peered":"2023-10-06T04:53:08.123598+0000","last_clean":"2023-10-06T04:53:08.123598+0000","last_became_active":"2023-10-06T04:53:07.106851+0000","last_became_peered":"2023-10-06T04:53:07.106851+0000","last_unstale":"2023-10-06T04:53:08.123598+0000","last_undegraded":"2023-10-06T04:53:08.123598+0000","last_fullsized":"2023-10-06T04:53:08.123598+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:28.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118614+0000","last_change":"2023-10-06T04:53:08.110867+0000","last_active":"2023-10-06T04:53:08.118614+0000","last_peered":"2023-10-06T04:53:08.118614+0000","last_clean":"2023-10-06T04:53:08.118614+0000","last_became_active":"2023-10-06T04:53:08.110563+0000","last_became_peered":"2023-10-06T04:53:08.110563+0000","last_unstale":"2023-10-06T04:53:08.118614+0000","last_undegraded":"2023-10-06T04:53:08.118614+0000","last_fullsized":"2023-10-06T04:53:08.118614+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123615+0000","last_change":"2023-10-06T04:53:07.108371+0000","last_active":"2023-10-06T04:53:08.123615+0000","last_peered":"2023-10-06T04:53:08.123615+0000","last_clean":"2023-10-06T04:53:08.123615+0000","last_became_active":"2023-10-06T04:53:07.108067+0000","last_became_peered":"2023-10-06T04:53:07.108067+0000","last_unstale":"2023-10-06T04:53:08.123615+0000","last_undegraded":"2023-10-06T04:53:08.123615+0000","last_fullsized":"2023-10-06T04:53:08.123615+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:22.850296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109883+0000","last_change":"2023-10-06T04:53:08.107665+0000","last_active":"2023-10-06T04:53:08.109883+0000","last_peered":"2023-10-06T04:53:08.109883+0000","last_clean":"2023-10-06T04:53:08.109883+0000","last_became_active":"2023-10-06T04:53:08.107386+0000","last_became_peered":"2023-10-06T04:53:08.107386+0000","last_unstale":"2023-10-06T04:53:08.109883+0000","last_undegraded":"2023-10-06T04:53:08.109883+0000","last_fullsized":"2023-10-06T04:53:08.109883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:17.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122840+0000","last_change":"2023-10-06T04:53:07.107782+0000","last_active":"2023-10-06T04:53:08.122840+0000","last_peered":"2023-10-06T04:53:08.122840+0000","last_clean":"2023-10-06T04:53:08.122840+0000","last_became_active":"2023-10-06T04:53:07.107463+0000","last_became_peered":"2023-10-06T04:53:07.107463+0000","last_unstale":"2023-10-06T04:53:08.122840+0000","last_undegraded":"2023-10-06T04:53:08.122840+0000","last_fullsized":"2023-10-06T04:53:08.122840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:00.567630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.109355+0000","last_change":"2023-10-06T04:53:08.102393+0000","last_active":"2023-10-06T04:53:08.109355+0000","last_peered":"2023-10-06T04:53:08.109355+0000","last_clean":"2023-10-06T04:53:08.109355+0000","last_became_active":"2023-10-06T04:53:08.102159+0000","last_became_peered":"2023-10-06T04:53:08.102159+0000","last_unstale":"2023-10-06T04:53:08.109355+0000","last_undegraded":"2023-10-06T04:53:08.109355+0000","last_fullsized":"2023-10-06T04:53:08.109355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.885426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091890+0000","last_change":"2023-10-06T04:53:07.098564+0000","last_active":"2023-10-06T04:53:08.091890+0000","last_peered":"2023-10-06T04:53:08.091890+0000","last_clean":"2023-10-06T04:53:08.091890+0000","last_became_active":"2023-10-06T04:53:07.098284+0000","last_became_peered":"2023-10-06T04:53:07.098284+0000","last_unstale":"2023-10-06T04:53:08.091890+0000","last_undegraded":"2023-10-06T04:53:08.091890+0000","last_fullsized":"2023-10-06T04:53:08.091890+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:13.311027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.121848+0000","last_change":"2023-10-06T04:53:08.116342+0000","last_active":"2023-10-06T04:53:08.121848+0000","last_peered":"2023-10-06T04:53:08.121848+0000","last_clean":"2023-10-06T04:53:08.121848+0000","last_became_active":"2023-10-06T04:53:08.114084+0000","last_became_peered":"2023-10-06T04:53:08.114084+0000","last_unstale":"2023-10-06T04:53:08.121848+0000","last_undegraded":"2023-10-06T04:53:08.121848+0000","last_fullsized":"2023-10-06T04:53:08.121848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:04.251592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123303+0000","last_change":"2023-10-06T04:53:07.103458+0000","last_active":"2023-10-06T04:53:08.123303+0000","last_peered":"2023-10-06T04:53:08.123303+0000","last_clean":"2023-10-06T04:53:08.123303+0000","last_became_active":"2023-10-06T04:53:07.103165+0000","last_became_peered":"2023-10-06T04:53:07.103165+0000","last_unstale":"2023-10-06T04:53:08.123303+0000","last_undegraded":"2023-10-06T04:53:08.123303+0000","last_fullsized":"2023-10-06T04:53:08.123303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.469905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120714+0000","last_change":"2023-10-06T04:53:08.109571+0000","last_active":"2023-10-06T04:53:08.120714+0000","last_peered":"2023-10-06T04:53:08.120714+0000","last_clean":"2023-10-06T04:53:08.120714+0000","last_became_active":"2023-10-06T04:53:08.109240+0000","last_became_peered":"2023-10-06T04:53:08.109240+0000","last_unstale":"2023-10-06T04:53:08.120714+0000","last_undegraded":"2023-10-06T04:53:08.120714+0000","last_fullsized":"2023-10-06T04:53:08.120714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:12.639355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.094970+0000","last_change":"2023-10-06T04:53:07.102516+0000","last_active":"2023-10-06T04:53:08.094970+0000","last_peered":"2023-10-06T04:53:08.094970+0000","last_clean":"2023-10-06T04:53:08.094970+0000","last_became_active":"2023-10-06T04:53:07.102192+0000","last_became_peered":"2023-10-06T04:53:07.102192+0000","last_unstale":"2023-10-06T04:53:08.094970+0000","last_undegraded":"2023-10-06T04:53:08.094970+0000","last_fullsized":"2023-10-06T04:53:08.094970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:25.069789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.119603+0000","last_change":"2023-10-06T04:53:08.109152+0000","last_active":"2023-10-06T04:53:08.119603+0000","last_peered":"2023-10-06T04:53:08.119603+0000","last_clean":"2023-10-06T04:53:08.119603+0000","last_became_active":"2023-10-06T04:53:08.108892+0000","last_became_peered":"2023-10-06T04:53:08.108892+0000","last_unstale":"2023-10-06T04:53:08.119603+0000","last_undegraded":"2023-10-06T04:53:08.119603+0000","last_fullsized":"2023-10-06T04:53:08.119603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:50.075234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096792+0000","last_change":"2023-10-06T04:53:07.106534+0000","last_active":"2023-10-06T04:53:08.096792+0000","last_peered":"2023-10-06T04:53:08.096792+0000","last_clean":"2023-10-06T04:53:08.096792+0000","last_became_active":"2023-10-06T04:53:07.106150+0000","last_became_peered":"2023-10-06T04:53:07.106150+0000","last_unstale":"2023-10-06T04:53:08.096792+0000","last_undegraded":"2023-10-06T04:53:08.096792+0000","last_fullsized":"2023-10-06T04:53:08.096792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:13.311027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.126304+0000","last_change":"2023-10-06T04:53:08.122237+0000","last_active":"2023-10-06T04:53:08.126304+0000","last_peered":"2023-10-06T04:53:08.126304+0000","last_clean":"2023-10-06T04:53:08.126304+0000","last_became_active":"2023-10-06T04:53:08.121962+0000","last_became_peered":"2023-10-06T04:53:08.121962+0000","last_unstale":"2023-10-06T04:53:08.126304+0000","last_undegraded":"2023-10-06T04:53:08.126304+0000","last_fullsized":"2023-10-06T04:53:08.126304+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:21.350284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.095883+0000","last_change":"2023-10-06T04:53:07.102961+0000","last_active":"2023-10-06T04:53:08.095883+0000","last_peered":"2023-10-06T04:53:08.095883+0000","last_clean":"2023-10-06T04:53:08.095883+0000","last_became_active":"2023-10-06T04:53:07.102626+0000","last_became_peered":"2023-10-06T04:53:07.102626+0000","last_unstale":"2023-10-06T04:53:08.095883+0000","last_undegraded":"2023-10-06T04:53:08.095883+0000","last_fullsized":"2023-10-06T04:53:08.095883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:36.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.108381+0000","last_change":"2023-10-06T04:53:08.105138+0000","last_active":"2023-10-06T04:53:08.108381+0000","last_peered":"2023-10-06T04:53:08.108381+0000","last_clean":"2023-10-06T04:53:08.108381+0000","last_became_active":"2023-10-06T04:53:08.104913+0000","last_became_peered":"2023-10-06T04:53:08.104913+0000","last_unstale":"2023-10-06T04:53:08.108381+0000","last_undegraded":"2023-10-06T04:53:08.108381+0000","last_fullsized":"2023-10-06T04:53:08.108381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:33.319235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096504+0000","last_change":"2023-10-06T04:53:07.105054+0000","last_active":"2023-10-06T04:53:08.096504+0000","last_peered":"2023-10-06T04:53:08.096504+0000","last_clean":"2023-10-06T04:53:08.096504+0000","last_became_active":"2023-10-06T04:53:07.104816+0000","last_became_peered":"2023-10-06T04:53:07.104816+0000","last_unstale":"2023-10-06T04:53:08.096504+0000","last_undegraded":"2023-10-06T04:53:08.096504+0000","last_fullsized":"2023-10-06T04:53:08.096504+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:11.632610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097847+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:53:08.097847+0000","last_peered":"2023-10-06T04:53:08.097847+0000","last_clean":"2023-10-06T04:53:08.097847+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:53:08.097847+0000","last_undegraded":"2023-10-06T04:53:08.097847+0000","last_fullsized":"2023-10-06T04:53:08.097847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.122550+0000","last_change":"2023-10-06T04:53:08.116880+0000","last_active":"2023-10-06T04:53:08.122550+0000","last_peered":"2023-10-06T04:53:08.122550+0000","last_clean":"2023-10-06T04:53:08.122550+0000","last_became_active":"2023-10-06T04:53:08.112104+0000","last_became_peered":"2023-10-06T04:53:08.112104+0000","last_unstale":"2023-10-06T04:53:08.122550+0000","last_undegraded":"2023-10-06T04:53:08.122550+0000","last_fullsized":"2023-10-06T04:53:08.122550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:48.507690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091912+0000","last_change":"2023-10-06T04:53:07.105451+0000","last_active":"2023-10-06T04:53:08.091912+0000","last_peered":"2023-10-06T04:53:08.091912+0000","last_clean":"2023-10-06T04:53:08.091912+0000","last_became_active":"2023-10-06T04:53:07.105182+0000","last_became_peered":"2023-10-06T04:53:07.105182+0000","last_unstale":"2023-10-06T04:53:08.091912+0000","last_undegraded":"2023-10-06T04:53:08.091912+0000","last_fullsized":"2023-10-06T04:53:08.091912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:09.728948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.094007+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:53:08.094007+0000","last_peered":"2023-10-06T04:53:08.094007+0000","last_clean":"2023-10-06T04:53:08.094007+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:53:08.094007+0000","last_undegraded":"2023-10-06T04:53:08.094007+0000","last_fullsized":"2023-10-06T04:53:08.094007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.117296+0000","last_change":"2023-10-06T04:53:08.112860+0000","last_active":"2023-10-06T04:53:08.117296+0000","last_peered":"2023-10-06T04:53:08.117296+0000","last_clean":"2023-10-06T04:53:08.117296+0000","last_became_active":"2023-10-06T04:53:08.112557+0000","last_became_peered":"2023-10-06T04:53:08.112557+0000","last_unstale":"2023-10-06T04:53:08.117296+0000","last_undegraded":"2023-10-06T04:53:08.117296+0000","last_fullsized":"2023-10-06T04:53:08.117296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:43.415231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.123524+0000","last_change":"2023-10-06T04:53:07.107653+0000","last_active":"2023-10-06T04:53:08.123524+0000","last_peered":"2023-10-06T04:53:08.123524+0000","last_clean":"2023-10-06T04:53:08.123524+0000","last_became_active":"2023-10-06T04:53:07.106536+0000","last_became_peered":"2023-10-06T04:53:07.106536+0000","last_unstale":"2023-10-06T04:53:08.123524+0000","last_undegraded":"2023-10-06T04:53:08.123524+0000","last_fullsized":"2023-10-06T04:53:08.123524+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:04.747138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.091398+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:53:08.091398+0000","last_peered":"2023-10-06T04:53:08.091398+0000","last_clean":"2023-10-06T04:53:08.091398+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:53:08.091398+0000","last_undegraded":"2023-10-06T04:53:08.091398+0000","last_fullsized":"2023-10-06T04:53:08.091398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097644+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:53:08.097644+0000","last_peered":"2023-10-06T04:53:08.097644+0000","last_clean":"2023-10-06T04:53:08.097644+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:53:08.097644+0000","last_undegraded":"2023-10-06T04:53:08.097644+0000","last_fullsized":"2023-10-06T04:53:08.097644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.120213+0000","last_change":"2023-10-06T04:53:08.113479+0000","last_active":"2023-10-06T04:53:08.120213+0000","last_peered":"2023-10-06T04:53:08.120213+0000","last_clean":"2023-10-06T04:53:08.120213+0000","last_became_active":"2023-10-06T04:53:08.113234+0000","last_became_peered":"2023-10-06T04:53:08.113234+0000","last_unstale":"2023-10-06T04:53:08.120213+0000","last_undegraded":"2023-10-06T04:53:08.120213+0000","last_fullsized":"2023-10-06T04:53:08.120213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097036+0000","last_change":"2023-10-06T04:53:07.107768+0000","last_active":"2023-10-06T04:53:08.097036+0000","last_peered":"2023-10-06T04:53:08.097036+0000","last_clean":"2023-10-06T04:53:08.097036+0000","last_became_active":"2023-10-06T04:53:07.107436+0000","last_became_peered":"2023-10-06T04:53:07.107436+0000","last_unstale":"2023-10-06T04:53:08.097036+0000","last_undegraded":"2023-10-06T04:53:08.097036+0000","last_fullsized":"2023-10-06T04:53:08.097036+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:24.715961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098834+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:53:08.098834+0000","last_peered":"2023-10-06T04:53:08.098834+0000","last_clean":"2023-10-06T04:53:08.098834+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:53:08.098834+0000","last_undegraded":"2023-10-06T04:53:08.098834+0000","last_fullsized":"2023-10-06T04:53:08.098834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.118710+0000","last_change":"2023-10-06T04:53:08.112343+0000","last_active":"2023-10-06T04:53:08.118710+0000","last_peered":"2023-10-06T04:53:08.118710+0000","last_clean":"2023-10-06T04:53:08.118710+0000","last_became_active":"2023-10-06T04:53:08.111289+0000","last_became_peered":"2023-10-06T04:53:08.111289+0000","last_unstale":"2023-10-06T04:53:08.118710+0000","last_undegraded":"2023-10-06T04:53:08.118710+0000","last_fullsized":"2023-10-06T04:53:08.118710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:48.507690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098462+0000","last_change":"2023-10-06T04:53:07.103097+0000","last_active":"2023-10-06T04:53:08.098462+0000","last_peered":"2023-10-06T04:53:08.098462+0000","last_clean":"2023-10-06T04:53:08.098462+0000","last_became_active":"2023-10-06T04:53:07.102813+0000","last_became_peered":"2023-10-06T04:53:07.102813+0000","last_unstale":"2023-10-06T04:53:08.098462+0000","last_undegraded":"2023-10-06T04:53:08.098462+0000","last_fullsized":"2023-10-06T04:53:08.098462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:40.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097581+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:53:08.097581+0000","last_peered":"2023-10-06T04:53:08.097581+0000","last_clean":"2023-10-06T04:53:08.097581+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:53:08.097581+0000","last_undegraded":"2023-10-06T04:53:08.097581+0000","last_fullsized":"2023-10-06T04:53:08.097581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110022+0000","last_change":"2023-10-06T04:53:08.104332+0000","last_active":"2023-10-06T04:53:08.110022+0000","last_peered":"2023-10-06T04:53:08.110022+0000","last_clean":"2023-10-06T04:53:08.110022+0000","last_became_active":"2023-10-06T04:53:08.104075+0000","last_became_peered":"2023-10-06T04:53:08.104075+0000","last_unstale":"2023-10-06T04:53:08.110022+0000","last_undegraded":"2023-10-06T04:53:08.110022+0000","last_fullsized":"2023-10-06T04:53:08.110022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:52.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097584+0000","last_change":"2023-10-06T04:53:07.098446+0000","last_active":"2023-10-06T04:53:08.097584+0000","last_peered":"2023-10-06T04:53:08.097584+0000","last_clean":"2023-10-06T04:53:08.097584+0000","last_became_active":"2023-10-06T04:53:07.098140+0000","last_became_peered":"2023-10-06T04:53:07.098140+0000","last_unstale":"2023-10-06T04:53:08.097584+0000","last_undegraded":"2023-10-06T04:53:08.097584+0000","last_fullsized":"2023-10-06T04:53:08.097584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:47.702385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.105276+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:53:08.105276+0000","last_peered":"2023-10-06T04:53:08.105276+0000","last_clean":"2023-10-06T04:53:08.105276+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:53:08.105276+0000","last_undegraded":"2023-10-06T04:53:08.105276+0000","last_fullsized":"2023-10-06T04:53:08.105276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.113189+0000","last_change":"2023-10-06T04:53:08.109536+0000","last_active":"2023-10-06T04:53:08.113189+0000","last_peered":"2023-10-06T04:53:08.113189+0000","last_clean":"2023-10-06T04:53:08.113189+0000","last_became_active":"2023-10-06T04:53:08.109269+0000","last_became_peered":"2023-10-06T04:53:08.109269+0000","last_unstale":"2023-10-06T04:53:08.113189+0000","last_undegraded":"2023-10-06T04:53:08.113189+0000","last_fullsized":"2023-10-06T04:53:08.113189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:18.822493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096187+0000","last_change":"2023-10-06T04:53:07.104825+0000","last_active":"2023-10-06T04:53:08.096187+0000","last_peered":"2023-10-06T04:53:08.096187+0000","last_clean":"2023-10-06T04:53:08.096187+0000","last_became_active":"2023-10-06T04:53:07.104572+0000","last_became_peered":"2023-10-06T04:53:07.104572+0000","last_unstale":"2023-10-06T04:53:08.096187+0000","last_undegraded":"2023-10-06T04:53:08.096187+0000","last_fullsized":"2023-10-06T04:53:08.096187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:43.511233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.098274+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:53:08.098274+0000","last_peered":"2023-10-06T04:53:08.098274+0000","last_clean":"2023-10-06T04:53:08.098274+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:53:08.098274+0000","last_undegraded":"2023-10-06T04:53:08.098274+0000","last_fullsized":"2023-10-06T04:53:08.098274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.110259+0000","last_change":"2023-10-06T04:53:08.103680+0000","last_active":"2023-10-06T04:53:08.110259+0000","last_peered":"2023-10-06T04:53:08.110259+0000","last_clean":"2023-10-06T04:53:08.110259+0000","last_became_active":"2023-10-06T04:53:08.103393+0000","last_became_peered":"2023-10-06T04:53:08.103393+0000","last_unstale":"2023-10-06T04:53:08.110259+0000","last_undegraded":"2023-10-06T04:53:08.110259+0000","last_fullsized":"2023-10-06T04:53:08.110259+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:07.082079+0000","last_clean_scrub_stamp":"2023-10-06T04:53:07.082079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.645548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.096340+0000","last_change":"2023-10-06T04:53:07.103554+0000","last_active":"2023-10-06T04:53:08.096340+0000","last_peered":"2023-10-06T04:53:08.096340+0000","last_clean":"2023-10-06T04:53:08.096340+0000","last_became_active":"2023-10-06T04:53:07.103281+0000","last_became_peered":"2023-10-06T04:53:07.103281+0000","last_unstale":"2023-10-06T04:53:08.096340+0000","last_undegraded":"2023-10-06T04:53:08.096340+0000","last_fullsized":"2023-10-06T04:53:08.096340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:06.075334+0000","last_clean_scrub_stamp":"2023-10-06T04:53:06.075334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:28.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T04:53:08.097695+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:53:08.097695+0000","last_peered":"2023-10-06T04:53:08.097695+0000","last_clean":"2023-10-06T04:53:08.097695+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:53:08.097695+0000","last_undegraded":"2023-10-06T04:53:08.097695+0000","last_fullsized":"2023-10-06T04:53:08.097695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":737412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":954368,"data_stored":735896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":148361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":147367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":145550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:53:12.589 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:53:12.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:53:12.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.747+0000 7fa51245b700 1 Processor -- start 2023-10-06T04:53:12.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.747+0000 7fa51245b700 1 -- start start 2023-10-06T04:53:12.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa51245b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa50c0af670 0x7fa50c0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa51245b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0ce420 con 0x7fa50c0cc510 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa51245b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c15a880 con 0x7fa50c0cc140 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa51245b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c15ab60 con 0x7fa50c0af670 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa50c0cc510 0x7fa50c0aef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57904/0 (socket says 172.21.15.3:57904) 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50b7fe700 1 -- 172.21.15.3:0/2800806321 learned_addr learned my addr 172.21.15.3:0/2800806321 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa5109f8700 1 --2- 172.21.15.3:0/2800806321 >> v2:172.21.15.153:3300/0 conn(0x7fa50c0af670 0x7fa50c0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082091203 0 0) 0x7fa50c0ce420 con 0x7fa50c0cc510 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f4003570 con 0x7fa50c0cc510 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075070316 0 0) 0x7fa50c15a880 con 0x7fa50c0cc140 2023-10-06T04:53:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa50c0ce420 con 0x7fa50c0cc140 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1633071808 0 0) 0x7fa4f4003570 con 0x7fa50c0cc510 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 >> v2:172.21.15.153:3300/0 conn(0x7fa50c0af670 msgr2=0x7fa50c0ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 --2- 172.21.15.3:0/2800806321 >> v2:172.21.15.153:3300/0 conn(0x7fa50c0af670 0x7fa50c0ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 >> v1:172.21.15.153:6789/0 conn(0x7fa50c0cc140 legacy=0x7fa50c159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0ce0b0 con 0x7fa50c0cc510 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fa500001470 con 0x7fa50c0cc510 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 >> v1:172.21.15.3:6789/0 conn(0x7fa50c0cc510 legacy=0x7fa50c0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4f4003bf0 0x7fa4f4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4f4004810 0x7fa4f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa4f400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c15ab60 con 0x7fa4f40051c0 2023-10-06T04:53:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f400c0b0 con 0x7fa4f4004810 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f400c270 con 0x7fa4f4003bf0 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa5109f8700 1 --2- 172.21.15.3:0/2800806321 >> v2:172.21.15.153:3300/0 conn(0x7fa50c0af670 0x7fa50c0ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa5109f8700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa4f400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.748+0000 7fa50b7fe700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4f4004810 0x7fa4f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa50bfff700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4f4003bf0 0x7fa4f4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa5109f8700 1 -- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4f4003bf0 msgr2=0x7fa4f4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa5109f8700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4f4003bf0 0x7fa4f4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa5109f8700 1 -- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4f4004810 msgr2=0x7fa4f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa5109f8700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4f4004810 0x7fa4f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa5109f8700 1 -- 172.21.15.3:0/2800806321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f4003570 con 0x7fa4f40051c0 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa5109f8700 1 -- 172.21.15.3:0/2800806321 reap_dead start 2023-10-06T04:53:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa5109f8700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa4f400ba60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa50400e3c0 tx=0x7fa50400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10e6996ae35df85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504009c50 con 0x7fa4f40051c0 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50400f040 con 0x7fa4f40051c0 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.749+0000 7fa50affd700 1 -- 172.21.15.3:0/2800806321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400eea0 con 0x7fa4f40051c0 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.750+0000 7fa51245b700 1 -- 172.21.15.3:0/2800806321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 msgr2=0x7fa4f400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.750+0000 7fa51245b700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa4f400ba60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa50400e3c0 tx=0x7fa50400b040 comp rx=0 tx=0).stop 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.750+0000 7fa51245b700 1 -- 172.21.15.3:0/2800806321 shutdown_connections 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.750+0000 7fa51245b700 1 --2- 172.21.15.3:0/2800806321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa4f400ba60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.750+0000 7fa51245b700 1 -- 172.21.15.3:0/2800806321 >> 172.21.15.3:0/2800806321 conn(0x7fa50c0d03e0 msgr2=0x7fa50c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.750+0000 7fa51245b700 1 -- 172.21.15.3:0/2800806321 shutdown_connections 2023-10-06T04:53:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.750+0000 7fa51245b700 1 -- 172.21.15.3:0/2800806321 wait complete. 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.751+0000 7fa51245b700 1 Processor -- start 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.751+0000 7fa51245b700 1 -- start start 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa50c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50c0c6aa0 0x7fa50c0c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa50c15e350 0x7fa50c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0c8490 con 0x7fa4f40051c0 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0c86e0 con 0x7fa50c0c6aa0 2023-10-06T04:53:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0c89c0 con 0x7fa50c15e350 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa50c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa50c0c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57320/0 (socket says 172.21.15.3:57320) 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 -- 172.21.15.3:0/62357420 learned_addr learned my addr 172.21.15.3:0/62357420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50b7fe700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50c0c6aa0 0x7fa50c0c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa5109f8700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa50c15e350 0x7fa50c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 -- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa50c15e350 msgr2=0x7fa50c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa50c15e350 0x7fa50c0b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 -- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50c0c6aa0 msgr2=0x7fa50c0c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50c0c6aa0 0x7fa50c0c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 -- 172.21.15.3:0/62357420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0c8ca0 con 0x7fa4f40051c0 2023-10-06T04:53:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa50bfff700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa50c0c94a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00f880 tx=0x7fa4fc007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa4f3fff700 1 -- 172.21.15.3:0/62357420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00d670 con 0x7fa4f40051c0 2023-10-06T04:53:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa4f3fff700 1 -- 172.21.15.3:0/62357420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc00d7d0 con 0x7fa4f40051c0 2023-10-06T04:53:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c0c8f80 con 0x7fa4f40051c0 2023-10-06T04:53:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa4f3fff700 1 -- 172.21.15.3:0/62357420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00db70 con 0x7fa4f40051c0 2023-10-06T04:53:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.752+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c0c4ef0 con 0x7fa4f40051c0 2023-10-06T04:53:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.754+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c0c9170 con 0x7fa4f40051c0 2023-10-06T04:53:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.754+0000 7fa4f3fff700 1 -- 172.21.15.3:0/62357420 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4fc0133f0 con 0x7fa4f40051c0 2023-10-06T04:53:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.754+0000 7fa4f3fff700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4f8066f30 0x7fa4f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.754+0000 7fa4f3fff700 1 -- 172.21.15.3:0/62357420 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4fc0843d0 con 0x7fa4f40051c0 2023-10-06T04:53:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.754+0000 7fa50b7fe700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4f8066f30 0x7fa4f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.755+0000 7fa50b7fe700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4f8066f30 0x7fa4f80693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa500001c80 tx=0x7fa500009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:12.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:12.759+0000 7fa4f3fff700 1 -- 172.21.15.3:0/62357420 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4fc0514e0 con 0x7fa4f40051c0 2023-10-06T04:53:13.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.030+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa50c04c380 con 0x7fa4f40051c0 2023-10-06T04:53:13.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.033+0000 7fa4f3fff700 1 -- 172.21.15.3:0/62357420 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa4fc054b00 con 0x7fa4f40051c0 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4f8066f30 msgr2=0x7fa4f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4f8066f30 0x7fa4f80693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa500001c80 tx=0x7fa500009040 comp rx=0 tx=0).stop 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 msgr2=0x7fa50c0c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa50c0c94a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00f880 tx=0x7fa4fc007d80 comp rx=0 tx=0).stop 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 shutdown_connections 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4f8066f30 0x7fa4f80693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa50c15e350 0x7fa50c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4f40051c0 0x7fa50c0c94a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 --2- 172.21.15.3:0/62357420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50c0c6aa0 0x7fa50c0c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 >> 172.21.15.3:0/62357420 conn(0x7fa50c0d03e0 msgr2=0x7fa50c0cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 shutdown_connections 2023-10-06T04:53:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.036+0000 7fa51245b700 1 -- 172.21.15.3:0/62357420 wait complete. 2023-10-06T04:53:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T04:53:13.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:13.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.167+0000 7f591915b700 1 Processor -- start 2023-10-06T04:53:13.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.167+0000 7f591915b700 1 -- start start 2023-10-06T04:53:13.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f591915b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5914157200 0x7f59140cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f591915b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f59140b16c0 0x7f59140c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f591915b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591406f3c0 con 0x7f59140cb980 2023-10-06T04:53:13.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f591915b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59140abc90 con 0x7f5914157200 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f591915b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59140abf70 con 0x7f59140b16c0 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f5913fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5914157200 0x7f59140cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f5913fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5914157200 0x7f59140cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57328/0 (socket says 172.21.15.3:57328) 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f5913fff700 1 -- 172.21.15.3:0/3410132378 learned_addr learned my addr 172.21.15.3:0/3410132378 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f591895a700 1 --2- 172.21.15.3:0/3410132378 >> v2:172.21.15.153:3300/0 conn(0x7f59140b16c0 0x7f59140c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f5913fff700 1 -- 172.21.15.3:0/3410132378 >> v2:172.21.15.153:3300/0 conn(0x7f59140b16c0 msgr2=0x7f59140c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.168+0000 7f5913fff700 1 --2- 172.21.15.3:0/3410132378 >> v2:172.21.15.153:3300/0 conn(0x7f59140b16c0 0x7f59140c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f5913fff700 1 -- 172.21.15.3:0/3410132378 >> v1:172.21.15.153:6789/0 conn(0x7f59140cb980 legacy=0x7f59140b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f5913fff700 1 -- 172.21.15.3:0/3410132378 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140ac250 con 0x7f5914157200 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f5913fff700 1 --2- 172.21.15.3:0/3410132378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5914157200 0x7f59140cb440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5914069230 tx=0x7f5904009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3076ed365dc42a44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f5912ffd700 1 -- 172.21.15.3:0/3410132378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590400b860 con 0x7f5914157200 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f5912ffd700 1 -- 172.21.15.3:0/3410132378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590400b9c0 con 0x7f5914157200 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f5912ffd700 1 -- 172.21.15.3:0/3410132378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590400bd60 con 0x7f5914157200 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 -- 172.21.15.3:0/3410132378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5914157200 msgr2=0x7f59140cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 --2- 172.21.15.3:0/3410132378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5914157200 0x7f59140cb440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5914069230 tx=0x7f5904009250 comp rx=0 tx=0).stop 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 -- 172.21.15.3:0/3410132378 shutdown_connections 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 --2- 172.21.15.3:0/3410132378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5914157200 0x7f59140cb440 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 --2- 172.21.15.3:0/3410132378 >> v2:172.21.15.153:3300/0 conn(0x7f59140b16c0 0x7f59140c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 -- 172.21.15.3:0/3410132378 >> 172.21.15.3:0/3410132378 conn(0x7f591406c370 msgr2=0x7f591406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 -- 172.21.15.3:0/3410132378 shutdown_connections 2023-10-06T04:53:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.169+0000 7f591915b700 1 -- 172.21.15.3:0/3410132378 wait complete. 2023-10-06T04:53:13.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 Processor -- start 2023-10-06T04:53:13.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 -- start start 2023-10-06T04:53:13.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59140b16c0 0x7f591414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cb980 0x7f591414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5914157200 0x7f591414ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59140a5440 con 0x7f59140cb980 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59140a5a00 con 0x7f59140b16c0 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.171+0000 7f591915b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591414f270 con 0x7f5914157200 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cb980 0x7f591414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cb980 0x7f591414e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57336/0 (socket says 172.21.15.3:57336) 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 -- 172.21.15.3:0/3640401822 learned_addr learned my addr 172.21.15.3:0/3640401822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f5913fff700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59140b16c0 0x7f591414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f591895a700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5914157200 0x7f591414ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 -- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5914157200 msgr2=0x7f591414ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5914157200 0x7f591414ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 -- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59140b16c0 msgr2=0x7f591414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59140b16c0 0x7f591414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 -- 172.21.15.3:0/3640401822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040094f0 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f591895a700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5914157200 0x7f591414ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f59137fe700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cb980 0x7f591414e690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f590800c7f0 tx=0x7f5908007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f58fbfff700 1 -- 172.21.15.3:0/3640401822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908011590 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f58fbfff700 1 -- 172.21.15.3:0/3640401822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5908011710 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591414f550 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f58fbfff700 1 -- 172.21.15.3:0/3640401822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908011ad0 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.172+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5914167690 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.173+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591404c380 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.174+0000 7f58fbfff700 1 -- 172.21.15.3:0/3640401822 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f590801b3b0 con 0x7f59140cb980 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.174+0000 7f58fbfff700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.174+0000 7f58fbfff700 1 -- 172.21.15.3:0/3640401822 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5908083b40 con 0x7f59140cb980 2023-10-06T04:53:13.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.174+0000 7f5913fff700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.175+0000 7f5913fff700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5914069230 tx=0x7f5904017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.179+0000 7f58fbfff700 1 -- 172.21.15.3:0/3640401822 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5908050c50 con 0x7f59140cb980 2023-10-06T04:53:13.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.418+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59140b1d00 con 0x7f59140cb980 2023-10-06T04:53:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.418+0000 7f58fbfff700 1 -- 172.21.15.3:0/3640401822 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f5908017070 con 0x7f59140cb980 2023-10-06T04:53:13.420 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:13.420 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.420+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 msgr2=0x7f59000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.420+0000 7f591915b700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5914069230 tx=0x7f5904017000 comp rx=0 tx=0).stop 2023-10-06T04:53:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.420+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cb980 msgr2=0x7f591414e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.420+0000 7f591915b700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cb980 0x7f591414e690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f590800c7f0 tx=0x7f5908007f50 comp rx=0 tx=0).stop 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.420+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 shutdown_connections 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.420+0000 7f591915b700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.421+0000 7f591915b700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59140b16c0 0x7f591414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.421+0000 7f591915b700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5914157200 0x7f591414ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.421+0000 7f591915b700 1 --2- 172.21.15.3:0/3640401822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cb980 0x7f591414e690 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.421+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 >> 172.21.15.3:0/3640401822 conn(0x7f591406c370 msgr2=0x7f591406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.421+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 shutdown_connections 2023-10-06T04:53:13.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.421+0000 7f591915b700 1 -- 172.21.15.3:0/3640401822 wait complete. 2023-10-06T04:53:13.424 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:13.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:53:07.496347+0000', 'modified': '2023-10-06T04:53:11.105282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4962}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4962': {'gid': 4962, 'name': 'b', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.153:6836/501211712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 501211712}, {'type': 'v1', 'addr': '172.21.15.153:6836', 'nonce': 501211712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:53:13.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:53:13.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T04:53:13.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.961+0000 7f91fa7c7700 1 Processor -- start 2023-10-06T04:53:13.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.961+0000 7f91fa7c7700 1 -- start start 2023-10-06T04:53:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.961+0000 7f91fa7c7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91f415cb50 0x7f91f40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.961+0000 7f91fa7c7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f40cd950 0x7f91f40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.961+0000 7f91fa7c7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f40b0ce0 con 0x7f91f40b0590 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.961+0000 7f91fa7c7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f40cb820 con 0x7f91f415cb50 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.961+0000 7f91fa7c7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40b0a00 con 0x7f91f40cd950 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91f415cb50 0x7f91f40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f8d64700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f40cd950 0x7f91f40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91f415cb50 0x7f91f40b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:40504/0 (socket says 172.21.15.3:40504) 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f37fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f91f40b0590 0x7f91f40b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:48888/0 (socket says 172.21.15.3:48888) 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f8d64700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f40cd950 0x7f91f40cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:58106/0 (socket says 172.21.15.3:58106) 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f3fff700 1 -- 172.21.15.3:0/1981969678 learned_addr learned my addr 172.21.15.3:0/1981969678 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f8d64700 1 -- 172.21.15.3:0/1981969678 >> v2:172.21.15.153:3300/0 conn(0x7f91f415cb50 msgr2=0x7f91f40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f8d64700 1 --2- 172.21.15.3:0/1981969678 >> v2:172.21.15.153:3300/0 conn(0x7f91f415cb50 0x7f91f40b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f8d64700 1 -- 172.21.15.3:0/1981969678 >> v1:172.21.15.153:6789/0 conn(0x7f91f40b0590 legacy=0x7f91f40b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f8d64700 1 -- 172.21.15.3:0/1981969678 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40cb1d0 con 0x7f91f40cd950 2023-10-06T04:53:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.962+0000 7f91f3fff700 1 --2- 172.21.15.3:0/1981969678 >> v2:172.21.15.153:3300/0 conn(0x7f91f415cb50 0x7f91f40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91f8d64700 1 --2- 172.21.15.3:0/1981969678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f40cd950 0x7f91f40cbcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009e60 tx=0x7f91ec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97746438d6f08a60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91f2ffd700 1 -- 172.21.15.3:0/1981969678 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00e070 con 0x7f91f40cd950 2023-10-06T04:53:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91f2ffd700 1 -- 172.21.15.3:0/1981969678 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec00bb60 con 0x7f91f40cd950 2023-10-06T04:53:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91f2ffd700 1 -- 172.21.15.3:0/1981969678 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec012430 con 0x7f91f40cd950 2023-10-06T04:53:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1981969678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f40cd950 msgr2=0x7f91f40cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1981969678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f40cd950 0x7f91f40cbcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009e60 tx=0x7f91ec009780 comp rx=0 tx=0).stop 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1981969678 shutdown_connections 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1981969678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f40cd950 0x7f91f40cbcb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1981969678 >> v2:172.21.15.153:3300/0 conn(0x7f91f415cb50 0x7f91f40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1981969678 >> 172.21.15.3:0/1981969678 conn(0x7f91f4070310 msgr2=0x7f91f4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1981969678 shutdown_connections 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.963+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1981969678 wait complete. 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.964+0000 7f91fa7c7700 1 Processor -- start 2023-10-06T04:53:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91fa7c7700 1 -- start start 2023-10-06T04:53:13.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91fa7c7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f40b0590 0x7f91f415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91fa7c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91f40cd950 0x7f91f415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91fa7c7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f415cb50 0x7f91f4150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91fa7c7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4155340 con 0x7f91f40cd950 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91fa7c7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40cdf80 con 0x7f91f40b0590 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91fa7c7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40ce190 con 0x7f91f415cb50 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91f37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91f40cd950 0x7f91f415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91f37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91f40cd950 0x7f91f415b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57348/0 (socket says 172.21.15.3:57348) 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.965+0000 7f91f37fe700 1 -- 172.21.15.3:0/1175751060 learned_addr learned my addr 172.21.15.3:0/1175751060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f3fff700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f40b0590 0x7f91f415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f8d64700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f415cb50 0x7f91f4150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f37fe700 1 -- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f415cb50 msgr2=0x7f91f4150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f37fe700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f415cb50 0x7f91f4150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f37fe700 1 -- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f40b0590 msgr2=0x7f91f415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f37fe700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f40b0590 0x7f91f415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f37fe700 1 -- 172.21.15.3:0/1175751060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4009480 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91f37fe700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91f40cd950 0x7f91f415b900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c790 tx=0x7f91e8007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91dbfff700 1 -- 172.21.15.3:0/1175751060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005a70 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91dbfff700 1 -- 172.21.15.3:0/1175751060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8005bd0 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91dbfff700 1 -- 172.21.15.3:0/1175751060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e80070d0 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec009a20 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.966+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f40ce7d0 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.968+0000 7f91dbfff700 1 -- 172.21.15.3:0/1175751060 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91e8007480 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.968+0000 7f91dbfff700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f91e0066f30 0x7f91e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.968+0000 7f91dbfff700 1 -- 172.21.15.3:0/1175751060 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91e8083b80 con 0x7f91f40cd950 2023-10-06T04:53:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.968+0000 7f91f3fff700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f91e0066f30 0x7f91e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.969+0000 7f91f3fff700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f91e0066f30 0x7f91e00693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006ce0 tx=0x7f91e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.970+0000 7f9203abfb80 1 Processor -- start 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.970+0000 7f9203abfb80 1 -- start start 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ae0a90 0x55a186ae0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186a99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f9203abfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a186a19c80 con 0x55a186a9abd0 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1869f0a70 con 0x55a186a9c9a0 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1869ee270 con 0x55a186ae0a90 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57364/0 (socket says 172.21.15.3:57364) 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 -- 172.21.15.3:0/596961482 learned_addr learned my addr 172.21.15.3:0/596961482 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f3fff700 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ae0a90 0x55a186ae0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f37fe700 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186a99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 -- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ae0a90 msgr2=0x55a186ae0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ae0a90 0x55a186ae0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 -- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 msgr2=0x55a186a99580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186a99580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 -- 172.21.15.3:0/596961482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a186884270 con 0x55a186a9abd0 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91f8d64700 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99ac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f91ec013000 tx=0x7f91ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8de667e5ebfba99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91d97fa700 1 -- 172.21.15.3:0/596961482 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00bcd0 con 0x55a186a9abd0 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91d97fa700 1 -- 172.21.15.3:0/596961482 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec00be50 con 0x55a186a9abd0 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.971+0000 7f91d97fa700 1 -- 172.21.15.3:0/596961482 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0129f0 con 0x55a186a9abd0 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 -- 172.21.15.3:0/596961482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 msgr2=0x55a186a99ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99ac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f91ec013000 tx=0x7f91ec019000 comp rx=0 tx=0).stop 2023-10-06T04:53:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 -- 172.21.15.3:0/596961482 shutdown_connections 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99ac0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ae0a90 0x55a186ae0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 --2- 172.21.15.3:0/596961482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186a99580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 -- 172.21.15.3:0/596961482 >> 172.21.15.3:0/596961482 conn(0x55a186a9b070 msgr2=0x55a186a9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 -- 172.21.15.3:0/596961482 shutdown_connections 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.972+0000 7f9203abfb80 1 -- 172.21.15.3:0/596961482 wait complete. 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.973+0000 7f9203abfb80 1 -- start start 2023-10-06T04:53:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.973+0000 7f9203abfb80 1 Processor -- start 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186ae3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ae0a90 0x55a186ae4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1869ea060 con 0x55a186a9abd0 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a186995890 con 0x55a186ae0a90 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1869b0ea0 con 0x55a186a9c9a0 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 10 client.0 _ll_register_callbacks cb 0x55a186886f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57370/0 (socket says 172.21.15.3:57370) 2023-10-06T04:53:13.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 -- 172.21.15.3:0/2457122088 learned_addr learned my addr 172.21.15.3:0/2457122088 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f8d64700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ae0a90 0x55a186ae4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f37fe700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186ae3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 -- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 msgr2=0x55a186ae3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186ae3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 -- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ae0a90 msgr2=0x55a186ae4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ae0a90 0x55a186ae4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 -- 172.21.15.3:0/2457122088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f40cb1d0 con 0x55a186a9abd0 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f37fe700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186ae3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91f3fff700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac0470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91e4015a10 tx=0x7f91e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdeca6c77fc422c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:53:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91d27fc700 1 -- 172.21.15.3:0/2457122088 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4015ea0 con 0x55a186a9abd0 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91d27fc700 1 -- 172.21.15.3:0/2457122088 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91e4014870 con 0x55a186a9abd0 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a186a8e690 con 0x55a186a9abd0 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f91d27fc700 1 -- 172.21.15.3:0/2457122088 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e4014c40 con 0x55a186a9abd0 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 10 client.5130 fetch_fsmap learned FSMap version 15 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.974+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55a186826fe0 con 0x55a186a9abd0 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f91d27fc700 1 -- 172.21.15.3:0/2457122088 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a186a8e690 con 0x55a186a9abd0 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f91d27fc700 1 -- 172.21.15.3:0/2457122088 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f91e4028020 con 0x55a186a9abd0 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f9203abfb80 10 client.5130 fetch_fsmap finished waiting for FSMap version 15 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f9203abfb80 10 client.5130 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f9203abfb80 20 client.5130 populate_metadata read hostname 'smithi003' 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f9203abfb80 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ac6ca0 0x55a186ac7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f9203abfb80 4 client.5130 mds_command: new command op to 4962 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:53:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 --> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a186a6b5c0 con 0x55a186ac6ca0 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f91f37fe700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ac6ca0 0x55a186ac7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f91d27fc700 10 client.5130 ms_handle_connect on v2:172.21.15.153:6834/501211712 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.975+0000 7f91f37fe700 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ac6ca0 0x55a186ac7070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a234fe41816fb702 server_cookie=c5339084e3deee91 in_seq=0 out_seq=0 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f91d27fc700 1 -- 172.21.15.3:0/2457122088 <== mds.0 v2:172.21.15.153:6834/501211712 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a186a6b5c0 con 0x55a186ac6ca0 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f91d27fc700 10 client.5130 handle_command_reply: tid=0 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 client.5130 shutdown 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 msgr2=0x55a186ac0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac0470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91e4015a10 tx=0x7f91e4018000 comp rx=0 tx=0).stop 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 shutdown_connections 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ac6ca0 0x55a186ac7070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ae0a90 0x55a186ae4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f91d27fc700 0 client.5130 ms_handle_reset on v2:172.21.15.153:6834/501211712 2023-10-06T04:53:13.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac0470 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 --2- 172.21.15.3:0/2457122088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186ae3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 >> 172.21.15.3:0/2457122088 conn(0x55a186a9b070 msgr2=0x55a186a94740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:13.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 shutdown_connections 2023-10-06T04:53:13.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.976+0000 7f9203abfb80 1 -- 172.21.15.3:0/2457122088 wait complete. 2023-10-06T04:53:13.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.977+0000 7f9203abfb80 20 client.5130 trim_cache size 0 max 16384 2023-10-06T04:53:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.993+0000 7f9203abfb80 1 Processor -- start 2023-10-06T04:53:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.993+0000 7f9203abfb80 1 -- start start 2023-10-06T04:53:13.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.994+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.994+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186abeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.994+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ac6ca0 0x55a186abf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.994+0000 7f9203abfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18686ce30 con 0x55a186a9abd0 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.994+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a186adc2a0 con 0x55a186a9c9a0 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.994+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a186a94350 con 0x55a186ac6ca0 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.994+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57384/0 (socket says 172.21.15.3:57384) 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 -- 172.21.15.3:0/3000675208 learned_addr learned my addr 172.21.15.3:0/3000675208 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 -- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ac6ca0 msgr2=0x55a186abf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f8d64700 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ac6ca0 0x55a186abf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ac6ca0 0x55a186abf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 -- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 msgr2=0x55a186abeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f37fe700 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186abeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186abeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 -- 172.21.15.3:0/3000675208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1869b8780 con 0x55a186a9abd0 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f37fe700 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186abeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91f3fff700 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac9a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f91e4012010 tx=0x7f91e401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33543fbf72fa23ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91d9ffb700 1 -- 172.21.15.3:0/3000675208 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4015dd0 con 0x55a186a9abd0 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91d9ffb700 1 -- 172.21.15.3:0/3000675208 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e4014c40 con 0x55a186a9abd0 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.995+0000 7f91d9ffb700 1 -- 172.21.15.3:0/3000675208 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e40148d0 con 0x55a186a9abd0 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 -- 172.21.15.3:0/3000675208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 msgr2=0x55a186ac9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac9a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f91e4012010 tx=0x7f91e401b000 comp rx=0 tx=0).stop 2023-10-06T04:53:13.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 -- 172.21.15.3:0/3000675208 shutdown_connections 2023-10-06T04:53:14.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186ac6ca0 0x55a186abf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186ac9a10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 --2- 172.21.15.3:0/3000675208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186a9c9a0 0x55a186abeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 -- 172.21.15.3:0/3000675208 >> 172.21.15.3:0/3000675208 conn(0x55a186a9b070 msgr2=0x55a186aef4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:14.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 -- 172.21.15.3:0/3000675208 shutdown_connections 2023-10-06T04:53:14.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 -- 172.21.15.3:0/3000675208 wait complete. 2023-10-06T04:53:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.996+0000 7f9203abfb80 1 -- start start 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 1 Processor -- start 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186a99ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ac6ca0 0x55a186abd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a186b3f770 con 0x55a186a9abd0 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a186b3dfa0 con 0x55a186ac6ca0 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a186a964c0 con 0x55a186a9c9a0 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f9203abfb80 10 client.0 _ll_register_callbacks cb 0x55a186886f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57394/0 (socket says 172.21.15.3:57394) 2023-10-06T04:53:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.997+0000 7f91f3fff700 1 -- 172.21.15.3:0/68353992 learned_addr learned my addr 172.21.15.3:0/68353992 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f37fe700 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186a99ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f8d64700 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ac6ca0 0x55a186abd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f3fff700 1 -- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 msgr2=0x55a186a99ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f3fff700 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186a99ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f3fff700 1 -- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ac6ca0 msgr2=0x55a186abd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f3fff700 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ac6ca0 0x55a186abd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f3fff700 1 -- 172.21.15.3:0/68353992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a186826fe0 con 0x55a186a9abd0 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91f3fff700 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99580 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f91e4012010 tx=0x7f91e402c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dff90d0d3168fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91d27fc700 1 -- 172.21.15.3:0/68353992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4015660 con 0x55a186a9abd0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91ec00be50 con 0x55a186a9abd0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91d27fc700 1 -- 172.21.15.3:0/68353992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91e40146c0 con 0x55a186a9abd0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91d27fc700 1 -- 172.21.15.3:0/68353992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e4014880 con 0x55a186a9abd0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 10 client.5136 fetch_fsmap learned FSMap version 15 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55a18699f470 con 0x55a186a9abd0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91d27fc700 1 -- 172.21.15.3:0/68353992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec00be50 con 0x55a186a9abd0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f91d27fc700 1 -- 172.21.15.3:0/68353992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f91e4028070 con 0x55a186a9abd0 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 10 client.5136 fetch_fsmap finished waiting for FSMap version 15 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 10 client.5136 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T04:53:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 20 client.5136 populate_metadata read hostname 'smithi003' 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ae0a90 0x55a186ad6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 4 client.5136 mds_command: new command op to 4962 tid=0[{"prefix": "status"}] 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.998+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 --> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a18698ce00 con 0x55a186ae0a90 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f91f37fe700 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ae0a90 0x55a186ad6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f91d27fc700 10 client.5136 ms_handle_connect on v2:172.21.15.153:6834/501211712 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f91f37fe700 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ae0a90 0x55a186ad6260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8013c653246e44f server_cookie=451e0cf7db53b389 in_seq=0 out_seq=0 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f91d27fc700 1 -- 172.21.15.3:0/68353992 <== mds.0 v2:172.21.15.153:6834/501211712 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a18698ce00 con 0x55a186ae0a90 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f91d27fc700 10 client.5136 handle_command_reply: tid=0 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 client.5136 shutdown 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 msgr2=0x55a186a99580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99580 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f91e4012010 tx=0x7f91e402c050 comp rx=0 tx=0).stop 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 shutdown_connections 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:6834/501211712,v1:172.21.15.153:6836/501211712] conn(0x55a186ae0a90 0x55a186ad6260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a186ac6ca0 0x55a186abd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f91d27fc700 0 client.5136 ms_handle_reset on v2:172.21.15.153:6834/501211712 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a186a9abd0 0x55a186a99580 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 --2- 172.21.15.3:0/68353992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a186a9c9a0 0x55a186a99ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:13.999+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 >> 172.21.15.3:0/68353992 conn(0x55a186a9b070 msgr2=0x55a186adf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 shutdown_connections 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f9203abfb80 1 -- 172.21.15.3:0/68353992 wait complete. 2023-10-06T04:53:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f9203abfb80 20 client.5136 trim_cache size 0 max 16384 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4962, 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 5.9005189979999999, 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 15, 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 32, 2023-10-06T04:53:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 32, 2023-10-06T04:53:14.010 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 14.910828753000001 2023-10-06T04:53:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:53:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f91e0066f30 msgr2=0x7f91e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f91e0066f30 0x7f91e00693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006ce0 tx=0x7f91e4008040 comp rx=0 tx=0).stop 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91f40cd950 msgr2=0x7f91f415b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91f40cd950 0x7f91f415b900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c790 tx=0x7f91e8007f00 comp rx=0 tx=0).stop 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 shutdown_connections 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f91e0066f30 0x7f91e00693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f40b0590 0x7f91f415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91f415cb50 0x7f91f4150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 --2- 172.21.15.3:0/1175751060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91f40cd950 0x7f91f415b900 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 >> 172.21.15.3:0/1175751060 conn(0x7f91f4070310 msgr2=0x7f91f41517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 shutdown_connections 2023-10-06T04:53:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.000+0000 7f91fa7c7700 1 -- 172.21.15.3:0/1175751060 wait complete. 2023-10-06T04:53:14.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:53:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.131+0000 7f7f806dc700 1 Processor -- start 2023-10-06T04:53:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.131+0000 7f7f806dc700 1 -- start start 2023-10-06T04:53:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f806dc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7f7815c8f0 0x7f7f7815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f806dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f7815ccc0 0x7f7f780d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f806dc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f780cf6c0 con 0x7f7f780d06a0 2023-10-06T04:53:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f806dc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f780cfb50 con 0x7f7f7815ccc0 2023-10-06T04:53:14.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f806dc700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f780d0b20 con 0x7f7f7815c8f0 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f7dc77700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f7815ccc0 0x7f7f780d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f7ec79700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7f780d06a0 0x7f7f7806d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57912/0 (socket says 172.21.15.3:57912) 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.132+0000 7f7f7ec79700 1 -- 172.21.15.3:0/4161498175 learned_addr learned my addr 172.21.15.3:0/4161498175 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7e478700 1 --2- 172.21.15.3:0/4161498175 >> v2:172.21.15.153:3300/0 conn(0x7f7f7815c8f0 0x7f7f7815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7dc77700 1 -- 172.21.15.3:0/4161498175 >> v2:172.21.15.153:3300/0 conn(0x7f7f7815c8f0 msgr2=0x7f7f7815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7dc77700 1 --2- 172.21.15.3:0/4161498175 >> v2:172.21.15.153:3300/0 conn(0x7f7f7815c8f0 0x7f7f7815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7dc77700 1 -- 172.21.15.3:0/4161498175 >> v1:172.21.15.3:6789/0 conn(0x7f7f780d06a0 legacy=0x7f7f7806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7dc77700 1 -- 172.21.15.3:0/4161498175 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7815e050 con 0x7f7f7815ccc0 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7d476700 1 -- 172.21.15.3:0/4161498175 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243225235 0 0) 0x7f7f780cf6c0 con 0x7f7f780d06a0 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7dc77700 1 --2- 172.21.15.3:0/4161498175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815ccc0 0x7f7f780d0160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0098e0 tx=0x7f7f6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f9caed4d0c99c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:14.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7d476700 1 -- 172.21.15.3:0/4161498175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c009e80 con 0x7f7f7815ccc0 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7d476700 1 -- 172.21.15.3:0/4161498175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c00b820 con 0x7f7f7815ccc0 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.133+0000 7f7f7d476700 1 -- 172.21.15.3:0/4161498175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00bba0 con 0x7f7f7815ccc0 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 -- 172.21.15.3:0/4161498175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815ccc0 msgr2=0x7f7f780d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 --2- 172.21.15.3:0/4161498175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815ccc0 0x7f7f780d0160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0098e0 tx=0x7f7f6c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 -- 172.21.15.3:0/4161498175 shutdown_connections 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 --2- 172.21.15.3:0/4161498175 >> v2:172.21.15.153:3300/0 conn(0x7f7f7815c8f0 0x7f7f7815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 --2- 172.21.15.3:0/4161498175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815ccc0 0x7f7f780d0160 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 -- 172.21.15.3:0/4161498175 >> 172.21.15.3:0/4161498175 conn(0x7f7f78070310 msgr2=0x7f7f78070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 -- 172.21.15.3:0/4161498175 shutdown_connections 2023-10-06T04:53:14.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.134+0000 7f7f806dc700 1 -- 172.21.15.3:0/4161498175 wait complete. 2023-10-06T04:53:14.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 Processor -- start 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 -- start start 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f780d06a0 0x7f7f780c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815c8f0 0x7f7f780c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f7815ccc0 0x7f7f780ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7815bde0 con 0x7f7f7815c8f0 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780ca9f0 con 0x7f7f780d06a0 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.135+0000 7f7f806dc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780cacd0 con 0x7f7f7815ccc0 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815c8f0 0x7f7f780c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815c8f0 0x7f7f780c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57414/0 (socket says 172.21.15.3:57414) 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 -- 172.21.15.3:0/49948818 learned_addr learned my addr 172.21.15.3:0/49948818 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7e478700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f780d06a0 0x7f7f780c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7ec79700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f7815ccc0 0x7f7f780ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 -- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f7815ccc0 msgr2=0x7f7f780ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f7815ccc0 0x7f7f780ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 -- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f780d06a0 msgr2=0x7f7f780c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f780d06a0 0x7f7f780c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 -- 172.21.15.3:0/49948818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c009420 con 0x7f7f7815c8f0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f7dc77700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815c8f0 0x7f7f780c9e80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c013000 tx=0x7f7f6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f767fc700 1 -- 172.21.15.3:0/49948818 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011420 con 0x7f7f7815c8f0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f767fc700 1 -- 172.21.15.3:0/49948818 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c0115a0 con 0x7f7f7815c8f0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f780cafb0 con 0x7f7f7815c8f0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f767fc700 1 -- 172.21.15.3:0/49948818 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011960 con 0x7f7f7815c8f0 2023-10-06T04:53:14.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.136+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f780cb570 con 0x7f7f7815c8f0 2023-10-06T04:53:14.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.137+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f780cb760 con 0x7f7f7815c8f0 2023-10-06T04:53:14.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.138+0000 7f7f767fc700 1 -- 172.21.15.3:0/49948818 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f6c011cd0 con 0x7f7f7815c8f0 2023-10-06T04:53:14.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.138+0000 7f7f767fc700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:14.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.138+0000 7f7f767fc700 1 -- 172.21.15.3:0/49948818 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f6c085ce0 con 0x7f7f7815c8f0 2023-10-06T04:53:14.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.138+0000 7f7f7e478700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:14.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.139+0000 7f7f7e478700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f64066f30 0x7f7f640693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f680074f0 tx=0x7f7f68009700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:14.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.142+0000 7f7f767fc700 1 -- 172.21.15.3:0/49948818 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f6c052df0 con 0x7f7f7815c8f0 2023-10-06T04:53:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.385+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7f78071680 con 0x7f7f7815c8f0 2023-10-06T04:53:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.385+0000 7f7f767fc700 1 -- 172.21.15.3:0/49948818 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7f6c015060 con 0x7f7f7815c8f0 2023-10-06T04:53:14.387 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f64066f30 msgr2=0x7f7f640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f64066f30 0x7f7f640693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f680074f0 tx=0x7f7f68009700 comp rx=0 tx=0).stop 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815c8f0 msgr2=0x7f7f780c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815c8f0 0x7f7f780c9e80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c013000 tx=0x7f7f6c017000 comp rx=0 tx=0).stop 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 shutdown_connections 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f780d06a0 0x7f7f780c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f7815ccc0 0x7f7f780ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 --2- 172.21.15.3:0/49948818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7815c8f0 0x7f7f780c9e80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:14.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 >> 172.21.15.3:0/49948818 conn(0x7f7f78070310 msgr2=0x7f7f780b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:14.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 shutdown_connections 2023-10-06T04:53:14.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:14.387+0000 7f7f806dc700 1 -- 172.21.15.3:0/49948818 wait complete. 2023-10-06T04:53:14.400 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:53:14.401 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:53:14.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:14.401 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:53:14.417 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:53:14.417 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:53:14.417 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:53:14.417 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:14.417 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42436sec preferred_lft 42436sec 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:14.418 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:53:14.419 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:14.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:14.421 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:53:14.481 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:53:14.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:14.484 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:53:14.539 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:53:14.539 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:53:14.539 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:53:14.539 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:53:14.539 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:53:14.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:14.539 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:53:14.540 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:53:14.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:53:14.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:53:14.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:53:14.540 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:53:14.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:14.641 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:53:14.656 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:53:14.656 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:53:14.656 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:53:14.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:14.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:53:14.721 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:53:14.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:53:14.722 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:53:14.776 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:53:14.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:53:14.777 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:53:14.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:53:14.897 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:53:14.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:53:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:53:14.962 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:53:14.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:14.963 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:53:15.018 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:53:15.018 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:53:15.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:53:15.085 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:53:15.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:53:15.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:53:15.142+0000 7facc4253540 -1 init, newargv = 0x560e4b0338a0 newargc=15 2023-10-06T04:53:15.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[103658]: starting ceph client 2023-10-06T04:53:15.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[103658]: starting fuse 2023-10-06T04:53:15.151 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:53:15.153 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:53:15.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:15.153 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:53:15.208 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:53:15.209 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:53:15.209 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:53:15.210 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:53:15.211 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:15.331 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.103658.asok 2023-10-06T04:53:15.513 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103658.asok 2023-10-06T04:53:15.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:15.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103658.asok status 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:53:15.668 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "103658", 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5154, 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5154 2023-10-06T04:53:15.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:15.670 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:53:15.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:53:15.670 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:53:15.670 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 475472563 2023-10-06T04:53:15.671 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:15.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:15.671 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:53:15.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:53:15.671 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:53:15.671 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 475472563 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5154 192.168.0.1:0/475472563", 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/475472563", 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 15, 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 32, 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:53:15.672 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:53:15.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:15.678 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:53:15.678 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:53:15.694 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:53:15.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:53:15.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:15.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:53:15.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:53:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.919+0000 7f58867c0700 1 Processor -- start 2023-10-06T04:53:15.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.919+0000 7f58867c0700 1 -- start start 2023-10-06T04:53:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f58867c0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5880157160 0x7f588015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f58867c0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800c8630 0x7f588015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f58867c0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588015d1b0 con 0x7f58800ce020 2023-10-06T04:53:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f58867c0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588015e140 con 0x7f5880157160 2023-10-06T04:53:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f58867c0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588015d770 con 0x7f58800c8630 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f5884fbd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800c8630 0x7f588015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f58857be700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5880157160 0x7f588015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f5884fbd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800c8630 0x7f588015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:58166/0 (socket says 172.21.15.3:58166) 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f58857be700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5880157160 0x7f588015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:40578/0 (socket says 172.21.15.3:40578) 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f5884fbd700 1 -- 172.21.15.3:0/2909659385 learned_addr learned my addr 172.21.15.3:0/2909659385 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.920+0000 7f5885fbf700 1 --1- 172.21.15.3:0/2909659385 >> v1:172.21.15.153:6789/0 conn(0x7f58800ce020 0x7f588006d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:48894/0 (socket says 172.21.15.3:48894) 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5884fbd700 1 -- 172.21.15.3:0/2909659385 >> v2:172.21.15.153:3300/0 conn(0x7f5880157160 msgr2=0x7f588015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5884fbd700 1 --2- 172.21.15.3:0/2909659385 >> v2:172.21.15.153:3300/0 conn(0x7f5880157160 0x7f588015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5884fbd700 1 -- 172.21.15.3:0/2909659385 >> v1:172.21.15.153:6789/0 conn(0x7f58800ce020 legacy=0x7f588006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:15.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5884fbd700 1 -- 172.21.15.3:0/2909659385 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800c8b80 con 0x7f58800c8630 2023-10-06T04:53:15.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f58857be700 1 --2- 172.21.15.3:0/2909659385 >> v2:172.21.15.153:3300/0 conn(0x7f5880157160 0x7f588015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:15.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5884fbd700 1 --2- 172.21.15.3:0/2909659385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800c8630 0x7f588015a4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f58700099b0 tx=0x7f5870009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac8392693208bef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5877fff700 1 -- 172.21.15.3:0/2909659385 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870009ea0 con 0x7f58800c8630 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5877fff700 1 -- 172.21.15.3:0/2909659385 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587000b860 con 0x7f58800c8630 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.921+0000 7f5877fff700 1 -- 172.21.15.3:0/2909659385 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000bc00 con 0x7f58800c8630 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 -- 172.21.15.3:0/2909659385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800c8630 msgr2=0x7f588015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 --2- 172.21.15.3:0/2909659385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800c8630 0x7f588015a4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f58700099b0 tx=0x7f5870009250 comp rx=0 tx=0).stop 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 -- 172.21.15.3:0/2909659385 shutdown_connections 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 --2- 172.21.15.3:0/2909659385 >> v2:172.21.15.153:3300/0 conn(0x7f5880157160 0x7f588015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 --2- 172.21.15.3:0/2909659385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800c8630 0x7f588015a4b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:15.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 -- 172.21.15.3:0/2909659385 >> 172.21.15.3:0/2909659385 conn(0x7f5880070310 msgr2=0x7f5880070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 -- 172.21.15.3:0/2909659385 shutdown_connections 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.922+0000 7f58867c0700 1 -- 172.21.15.3:0/2909659385 wait complete. 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.923+0000 7f58867c0700 1 Processor -- start 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.923+0000 7f58867c0700 1 -- start start 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58867c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800c8630 0x7f58800c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58867c0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800ce020 0x7f58800c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58867c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5880157160 0x7f58800cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58867c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c3260 con 0x7f58800c8630 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58867c0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c3540 con 0x7f5880157160 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58867c0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c3820 con 0x7f58800ce020 2023-10-06T04:53:15.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800c8630 0x7f58800c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800c8630 0x7f58800c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57426/0 (socket says 172.21.15.3:57426) 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 -- 172.21.15.3:0/1126728862 learned_addr learned my addr 172.21.15.3:0/1126728862 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f5884fbd700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800ce020 0x7f58800c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f5885fbf700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5880157160 0x7f58800cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 -- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800ce020 msgr2=0x7f58800c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800ce020 0x7f58800c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 -- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5880157160 msgr2=0x7f58800cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5880157160 0x7f58800cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 -- 172.21.15.3:0/1126728862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009420 con 0x7f58800c8630 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f5885fbf700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5880157160 0x7f58800cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:15.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58857be700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800c8630 0x7f58800c2960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ec80 tx=0x7f587c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58757fa700 1 -- 172.21.15.3:0/1126728862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00c9d0 con 0x7f58800c8630 2023-10-06T04:53:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58757fa700 1 -- 172.21.15.3:0/1126728862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00cb30 con 0x7f58800c8630 2023-10-06T04:53:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58757fa700 1 -- 172.21.15.3:0/1126728862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00ceb0 con 0x7f58800c8630 2023-10-06T04:53:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.924+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58700094f0 con 0x7f58800c8630 2023-10-06T04:53:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.925+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58800b11c0 con 0x7f58800c8630 2023-10-06T04:53:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.925+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588004c380 con 0x7f58800c8630 2023-10-06T04:53:15.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.926+0000 7f58757fa700 1 -- 172.21.15.3:0/1126728862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f587c013680 con 0x7f58800c8630 2023-10-06T04:53:15.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.926+0000 7f58757fa700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5864066f30 0x7f58640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:15.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.926+0000 7f58757fa700 1 -- 172.21.15.3:0/1126728862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f587c084ad0 con 0x7f58800c8630 2023-10-06T04:53:15.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.927+0000 7f5884fbd700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5864066f30 0x7f58640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:15.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.927+0000 7f5884fbd700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5864066f30 0x7f58640693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f58700099b0 tx=0x7f5870017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:15.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:15.931+0000 7f58757fa700 1 -- 172.21.15.3:0/1126728862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f587c051b40 con 0x7f58800c8630 2023-10-06T04:53:16.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.174+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58800c8e60 con 0x7f58800c8630 2023-10-06T04:53:16.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.174+0000 7f58757fa700 1 -- 172.21.15.3:0/1126728862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f587c055160 con 0x7f58800c8630 2023-10-06T04:53:16.176 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:53:16.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5864066f30 msgr2=0x7f58640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:16.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5864066f30 0x7f58640693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f58700099b0 tx=0x7f5870017000 comp rx=0 tx=0).stop 2023-10-06T04:53:16.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800c8630 msgr2=0x7f58800c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:16.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800c8630 0x7f58800c2960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ec80 tx=0x7f587c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:16.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 shutdown_connections 2023-10-06T04:53:16.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5864066f30 0x7f58640693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:16.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5880157160 0x7f58800cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:16.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800c8630 0x7f58800c2960 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:16.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 --2- 172.21.15.3:0/1126728862 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58800ce020 0x7f58800c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:16.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 >> 172.21.15.3:0/1126728862 conn(0x7f5880070310 msgr2=0x7f58800cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:16.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 shutdown_connections 2023-10-06T04:53:16.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:16.176+0000 7f58867c0700 1 -- 172.21.15.3:0/1126728862 wait complete. 2023-10-06T04:53:16.189 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:53:16.190 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:53:16.190 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:53:16.190 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:53:16.190 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:53:16.191 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:53:16.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:53:16.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:53:16.191 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:53:16.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:16.191 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:53:16.211 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:53:16.211 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:53:16.211 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:53:16.211 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:16.211 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:53:16.211 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:16.212 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42436sec preferred_lft 42436sec 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:53:16.213 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T04:53:16.214 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:16.214 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:53:16.214 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:53:16.214 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T04:53:16.214 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:53:16.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:16.215 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:53:16.278 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:53:16.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:16.280 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:53:16.338 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:53:16.338 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:53:16.338 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:53:16.338 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:53:16.338 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:53:16.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:16.339 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:53:16.339 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:53:16.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:53:16.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:53:16.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:53:16.340 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:53:16.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:16.443 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:53:16.458 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T04:53:16.458 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:53:16.458 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:53:16.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:16.459 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T04:53:16.523 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:53:16.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:53:16.523 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:53:16.578 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:53:16.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:53:16.578 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:53:16.632 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:53:16.698 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:53:16.698 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:53:16.761 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:53:16.762 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:53:16.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:16.763 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:53:16.817 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:53:16.817 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:53:16.860 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:53:16.884 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:53:16.884 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:53:16.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T04:53:16.938+0000 7f1313915540 -1 init, newargv = 0x558c6dc43ca0 newargc=15 2023-10-06T04:53:16.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[98077]: starting ceph client 2023-10-06T04:53:16.945 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[98077]: starting fuse 2023-10-06T04:53:16.950 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:53:16.951 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:53:16.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:16.951 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:53:17.007 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:53:17.007 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:53:17.007 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:17.007 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:53:17.007 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:53:17.007 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:53:17.007 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T04:53:17.009 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:53:17.127 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.98077.asok 2023-10-06T04:53:17.309 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.98077.asok 2023-10-06T04:53:17.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:17.310 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.98077.asok status 2023-10-06T04:53:17.461 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:53:17.461 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:53:17.461 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:53:17.461 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:53:17.461 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T04:53:17.461 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "98077", 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 5089, 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:53:17.462 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 5089 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 960979156 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 960979156 2023-10-06T04:53:17.463 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.5089 192.168.0.1:0/960979156", 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/960979156", 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 15, 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 32, 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:53:17.464 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:53:17.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:17.471 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:53:17.471 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:53:17.486 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:53:17.486 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:53:17.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:17.487 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:53:17.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 500 2023-10-06T04:53:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.686+0000 7fadf0cc4700 1 Processor -- start 2023-10-06T04:53:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.686+0000 7fadf0cc4700 1 -- start start 2023-10-06T04:53:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fadf0cc4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fadec0cfb80 0x7fadec0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fadf0cc4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec159eb0 0x7fadec15da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fadf0cc4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec15d510 con 0x7fadec0afad0 2023-10-06T04:53:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fadf0cc4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec15a480 con 0x7fadec0cfb80 2023-10-06T04:53:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fadf0cc4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec15a760 con 0x7fadec159eb0 2023-10-06T04:53:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fadead9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec159eb0 0x7fadec15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fadea59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fadec0cfb80 0x7fadec0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fade3fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fadec0afad0 0x7fadec0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:48900/0 (socket says 172.21.15.3:48900) 2023-10-06T04:53:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.687+0000 7fade3fff700 1 -- 172.21.15.3:0/2451344672 learned_addr learned my addr 172.21.15.3:0/2451344672 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadea59c700 1 -- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec159eb0 msgr2=0x7fadec15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadea59c700 1 --2- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec159eb0 0x7fadec15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadea59c700 1 -- 172.21.15.3:0/2451344672 >> v1:172.21.15.153:6789/0 conn(0x7fadec0afad0 legacy=0x7fadec0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadea59c700 1 -- 172.21.15.3:0/2451344672 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec15aa40 con 0x7fadec0cfb80 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fade9d9b700 1 -- 172.21.15.3:0/2451344672 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392764203 0 0) 0x7fadec15d510 con 0x7fadec0afad0 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadead9d700 1 --2- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec159eb0 0x7fadec15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadea59c700 1 --2- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 0x7fadec0af590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faddc0098e0 tx=0x7faddc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=818d58adc73ebb15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fade9d9b700 1 -- 172.21.15.3:0/2451344672 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00b640 con 0x7fadec0cfb80 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fade9d9b700 1 -- 172.21.15.3:0/2451344672 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00b7a0 con 0x7fadec0cfb80 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fade9d9b700 1 -- 172.21.15.3:0/2451344672 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00bb40 con 0x7fadec0cfb80 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadf0cc4700 1 -- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 msgr2=0x7fadec0af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 0x7fadec0af590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faddc0098e0 tx=0x7faddc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadf0cc4700 1 -- 172.21.15.3:0/2451344672 shutdown_connections 2023-10-06T04:53:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec159eb0 0x7fadec15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/2451344672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 0x7fadec0af590 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.688+0000 7fadf0cc4700 1 -- 172.21.15.3:0/2451344672 >> 172.21.15.3:0/2451344672 conn(0x7fadec070310 msgr2=0x7fadec070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.689+0000 7fadf0cc4700 1 -- 172.21.15.3:0/2451344672 shutdown_connections 2023-10-06T04:53:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.689+0000 7fadf0cc4700 1 -- 172.21.15.3:0/2451344672 wait complete. 2023-10-06T04:53:17.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 Processor -- start 2023-10-06T04:53:17.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 -- start start 2023-10-06T04:53:17.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec0afad0 0x7fadec16ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:17.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 0x7fadec1417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:17.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec159eb0 0x7fadec0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:17.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0b0ce0 con 0x7fadec159eb0 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0b1b00 con 0x7fadec0cfb80 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadf0cc4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0b0ef0 con 0x7fadec0afad0 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.690+0000 7fadead9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec159eb0 0x7fadec0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec159eb0 0x7fadec0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57436/0 (socket says 172.21.15.3:57436) 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 -- 172.21.15.3:0/3024517675 learned_addr learned my addr 172.21.15.3:0/3024517675 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadea59c700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec0afad0 0x7fadec16ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fade3fff700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 0x7fadec1417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 -- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec0afad0 msgr2=0x7fadec16ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec0afad0 0x7fadec16ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 -- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 msgr2=0x7fadec1417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 0x7fadec1417f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 -- 172.21.15.3:0/3024517675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009a20 con 0x7fadec159eb0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadead9d700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec159eb0 0x7fadec0b07a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fade4009f20 tx=0x7fade400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fade27fc700 1 -- 172.21.15.3:0/3024517675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400cbd0 con 0x7fadec159eb0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fade27fc700 1 -- 172.21.15.3:0/3024517675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400f040 con 0x7fadec159eb0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc009420 con 0x7fadec159eb0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fade27fc700 1 -- 172.21.15.3:0/3024517675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4014480 con 0x7fadec159eb0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.691+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec0b14b0 con 0x7fadec159eb0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.692+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec04c380 con 0x7fadec159eb0 2023-10-06T04:53:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.693+0000 7fade27fc700 1 -- 172.21.15.3:0/3024517675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fade4003020 con 0x7fadec159eb0 2023-10-06T04:53:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.693+0000 7fade27fc700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fadcc066f30 0x7fadcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.693+0000 7fade27fc700 1 -- 172.21.15.3:0/3024517675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fade4085090 con 0x7fadec159eb0 2023-10-06T04:53:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.693+0000 7fadea59c700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fadcc066f30 0x7fadcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.694+0000 7fadea59c700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fadcc066f30 0x7fadcc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faddc0093f0 tx=0x7faddc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.697+0000 7fade27fc700 1 -- 172.21.15.3:0/3024517675 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fade40521a0 con 0x7fadec159eb0 2023-10-06T04:53:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.893+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7fadec0b1c40 con 0x7fadec159eb0 2023-10-06T04:53:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.896+0000 7fade27fc700 1 -- 172.21.15.3:0/3024517675 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) v1 ==== 123+0+0 (secure 0 0 0) 0x7fade40557c0 con 0x7fadec159eb0 2023-10-06T04:53:17.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fadcc066f30 msgr2=0x7fadcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:17.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fadcc066f30 0x7fadcc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faddc0093f0 tx=0x7faddc017000 comp rx=0 tx=0).stop 2023-10-06T04:53:17.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec159eb0 msgr2=0x7fadec0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:17.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec159eb0 0x7fadec0b07a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fade4009f20 tx=0x7fade400b040 comp rx=0 tx=0).stop 2023-10-06T04:53:17.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 shutdown_connections 2023-10-06T04:53:17.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fadcc066f30 0x7fadcc0693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec159eb0 0x7fadec0b07a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadec0afad0 0x7fadec16ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 --2- 172.21.15.3:0/3024517675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec0cfb80 0x7fadec1417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 >> 172.21.15.3:0/3024517675 conn(0x7fadec070310 msgr2=0x7fadec0af330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 shutdown_connections 2023-10-06T04:53:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:17.898+0000 7fadf0cc4700 1 -- 172.21.15.3:0/3024517675 wait complete. 2023-10-06T04:53:17.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 250 2023-10-06T04:53:18.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.032+0000 7f5f08e45700 1 Processor -- start 2023-10-06T04:53:18.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.032+0000 7f5f08e45700 1 -- start start 2023-10-06T04:53:18.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.033+0000 7f5f08e45700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f040b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:18.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.033+0000 7f5f08e45700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f040b1e70 con 0x7f5f0406d480 2023-10-06T04:53:18.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.033+0000 7f5f08e45700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f040b1860 con 0x7f5f0406dfe0 2023-10-06T04:53:18.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.033+0000 7f5f08e45700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0406eaf0 con 0x7f5f0415cae0 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.033+0000 7f5f02ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5f0406d480 0x7f5f0406d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57926/0 (socket says 172.21.15.3:57926) 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.033+0000 7f5f02ffd700 1 -- 172.21.15.3:0/665443998 learned_addr learned my addr 172.21.15.3:0/665443998 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.033+0000 7f5f037fe700 1 --2- 172.21.15.3:0/665443998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f040b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f027fc700 1 -- 172.21.15.3:0/665443998 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808347292 0 0) 0x7f5f040b1e70 con 0x7f5f0406d480 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f027fc700 1 -- 172.21.15.3:0/665443998 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eec003570 con 0x7f5f0406d480 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f027fc700 1 -- 172.21.15.3:0/665443998 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308749407 0 0) 0x7f5f040b1860 con 0x7f5f0406dfe0 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f027fc700 1 -- 172.21.15.3:0/665443998 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f040b1e70 con 0x7f5f0406dfe0 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f037fe700 1 -- 172.21.15.3:0/665443998 >> v1:172.21.15.153:6789/0 conn(0x7f5f0406dfe0 legacy=0x7f5f040b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:18.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f037fe700 1 -- 172.21.15.3:0/665443998 >> v1:172.21.15.3:6789/0 conn(0x7f5f0406d480 legacy=0x7f5f0406d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f037fe700 1 -- 172.21.15.3:0/665443998 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0406e530 con 0x7f5f0415cae0 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f037fe700 1 --2- 172.21.15.3:0/665443998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f040b0430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ec7fa251adc60f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f027fc700 1 -- 172.21.15.3:0/665443998 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4005c10 con 0x7f5f0415cae0 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f027fc700 1 -- 172.21.15.3:0/665443998 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4005d70 con 0x7f5f0415cae0 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.034+0000 7f5f027fc700 1 -- 172.21.15.3:0/665443998 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4006260 con 0x7f5f0415cae0 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.035+0000 7f5f08e45700 1 -- 172.21.15.3:0/665443998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 msgr2=0x7f5f040b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.035+0000 7f5f08e45700 1 --2- 172.21.15.3:0/665443998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f040b0430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef4009120 comp rx=0 tx=0).stop 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.035+0000 7f5f08e45700 1 -- 172.21.15.3:0/665443998 shutdown_connections 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.035+0000 7f5f08e45700 1 --2- 172.21.15.3:0/665443998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f040b0430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.035+0000 7f5f08e45700 1 -- 172.21.15.3:0/665443998 >> 172.21.15.3:0/665443998 conn(0x7f5f0406cb50 msgr2=0x7f5f040ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:18.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.035+0000 7f5f08e45700 1 -- 172.21.15.3:0/665443998 shutdown_connections 2023-10-06T04:53:18.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.035+0000 7f5f08e45700 1 -- 172.21.15.3:0/665443998 wait complete. 2023-10-06T04:53:18.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.036+0000 7f5f08e45700 1 Processor -- start 2023-10-06T04:53:18.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.036+0000 7f5f08e45700 1 -- start start 2023-10-06T04:53:18.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f08e45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f0406d480 0x7f5f040b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:18.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f08e45700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0406dfe0 0x7f5f04153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f08e45700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f0409b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f08e45700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0409a330 con 0x7f5f0406d480 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f08e45700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0409a5d0 con 0x7f5f0406dfe0 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f08e45700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0409a8b0 con 0x7f5f0415cae0 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f0406d480 0x7f5f040b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f0406d480 0x7f5f040b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57442/0 (socket says 172.21.15.3:57442) 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 -- 172.21.15.3:0/4199378140 learned_addr learned my addr 172.21.15.3:0/4199378140 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f03fff700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f0409b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 -- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 msgr2=0x7f5f0409b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f0409b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 -- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0406dfe0 msgr2=0x7f5f04153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f02ffd700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0406dfe0 0x7f5f04153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0406dfe0 0x7f5f04153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 -- 172.21.15.3:0/4199378140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40093c0 con 0x7f5f0406d480 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f02ffd700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0406dfe0 0x7f5f04153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f037fe700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f0406d480 0x7f5f040b2550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009160 tx=0x7f5ef4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5eeb7fe700 1 -- 172.21.15.3:0/4199378140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef401d970 con 0x7f5f0406d480 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5eeb7fe700 1 -- 172.21.15.3:0/4199378140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef401dad0 con 0x7f5f0406d480 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0409ab90 con 0x7f5f0406d480 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.037+0000 7f5eeb7fe700 1 -- 172.21.15.3:0/4199378140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef401de90 con 0x7f5f0406d480 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.038+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f040b2a90 con 0x7f5f0406d480 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.038+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0409b0b0 con 0x7f5f0406d480 2023-10-06T04:53:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.039+0000 7f5eeb7fe700 1 -- 172.21.15.3:0/4199378140 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ef401b690 con 0x7f5f0406d480 2023-10-06T04:53:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.039+0000 7f5eeb7fe700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ef0066f30 0x7f5ef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.039+0000 7f5eeb7fe700 1 -- 172.21.15.3:0/4199378140 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5ef4086680 con 0x7f5f0406d480 2023-10-06T04:53:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.039+0000 7f5f02ffd700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ef0066f30 0x7f5ef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.040+0000 7f5f02ffd700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ef0066f30 0x7f5ef00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8007fd0 tx=0x7f5ef8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:18.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.044+0000 7f5eeb7fe700 1 -- 172.21.15.3:0/4199378140 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f0409b0b0 con 0x7f5f0406d480 2023-10-06T04:53:18.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.239+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) v1 -- 0x7f5f0409b0b0 con 0x7f5f0406d480 2023-10-06T04:53:18.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.242+0000 7f5eeb7fe700 1 -- 172.21.15.3:0/4199378140 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5f0409b0b0 con 0x7f5f0406d480 2023-10-06T04:53:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ef0066f30 msgr2=0x7f5ef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ef0066f30 0x7f5ef00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8007fd0 tx=0x7f5ef8009040 comp rx=0 tx=0).stop 2023-10-06T04:53:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f0406d480 msgr2=0x7f5f040b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f0406d480 0x7f5f040b2550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009160 tx=0x7f5ef4016000 comp rx=0 tx=0).stop 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 shutdown_connections 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ef0066f30 0x7f5ef00693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f0415cae0 0x7f5f0409b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f0406d480 0x7f5f040b2550 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 --2- 172.21.15.3:0/4199378140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0406dfe0 0x7f5f04153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.244+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 >> 172.21.15.3:0/4199378140 conn(0x7f5f0406cb50 msgr2=0x7f5f0415a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.245+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 shutdown_connections 2023-10-06T04:53:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:18.245+0000 7f5f08e45700 1 -- 172.21.15.3:0/4199378140 wait complete. 2023-10-06T04:53:18.258 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:18.258 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:18.258 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> n = 250 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:53:18.259 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:18.260 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:53:18.261 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:53:18.261 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:53:18.261 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:53:18.261 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:18.262 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:18.263 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:53:18.263 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:53:18.263 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> n = 250 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:53:19.518 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:53:19.519 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:53:19.520 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:53:19.521 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:19.521 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:19.521 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:53:19.521 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:53:19.521 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:20.812 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> n = 250 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:53:20.813 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:53:20.814 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:53:20.815 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:22.509 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:22.509 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:22.509 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:53:22.509 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:22.509 DEBUG:teuthology.orchestra.run.smithi003:> n = 250 2023-10-06T04:53:22.509 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:53:22.510 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:53:22.510 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:53:22.510 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2023-10-06T04:53:22.510 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:22.510 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:53:22.510 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:53:22.510 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:53:22.511 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:53:22.512 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:53:22.513 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:53:22.513 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:22.513 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:22.513 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:53:22.513 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:53:22.513 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:23.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:23.948 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> n = 250 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:53:24.007 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/dir5/file" 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:53:24.008 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:53:24.009 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> n = 250 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:53:25.303 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:53:25.304 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/dir6/file" 2023-10-06T04:53:25.304 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:25.304 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:53:25.304 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:53:25.304 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:53:25.304 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:25.304 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:53:25.305 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:53:25.305 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:25.305 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:53:25.305 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:53:25.305 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:53:25.305 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:53:25.306 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:53:25.306 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:53:25.306 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:53:25.306 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:25.306 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:53:25.307 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:26.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:53:26.798 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:53:26.798 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:53:26.798 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:53:26.798 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:53:26.798 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:53:26.799 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:53:26.800 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:53:26.921 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.103658.asok 2023-10-06T04:53:27.103 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103658.asok 2023-10-06T04:53:27.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:27.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103658.asok mds_sessions 2023-10-06T04:53:27.257 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5154, 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5154 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:53:27.258 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 475472563 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5154 v1:192.168.0.1:0/475472563", 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/475472563", 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6834", 2023-10-06T04:53:27.259 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 501211712 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6836", 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 501211712 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 3006, 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2023-10-06T04:54:16.145084+0000", 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2023-10-06T04:53:16.145084+0000", 2023-10-06T04:53:27.260 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2023-10-06T04:53:27.261 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 183, 2023-10-06T04:53:27.261 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2023-10-06T04:53:27.261 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:27.261 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-06T04:53:27.261 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 15 2023-10-06T04:53:27.261 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:53:27.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:53:27.268 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'find | wc') 2023-10-06T04:53:31.421 INFO:teuthology.orchestra.run.smithi003.stdout: 1507 1507 23384 2023-10-06T04:53:31.422 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:31.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.550+0000 7f38a4809700 1 Processor -- start 2023-10-06T04:53:31.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.550+0000 7f38a4809700 1 -- start start 2023-10-06T04:53:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f38a4809700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f38a4809700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00cc5c0 con 0x7f38a00b2930 2023-10-06T04:53:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f38a4809700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00cb9b0 con 0x7f38a01598d0 2023-10-06T04:53:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f38a4809700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00cbc90 con 0x7f38a0159500 2023-10-06T04:53:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f38a00b2930 0x7f38a00cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34462/0 (socket says 172.21.15.3:34462) 2023-10-06T04:53:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389ed9d700 1 -- 172.21.15.3:0/4273617848 learned_addr learned my addr 172.21.15.3:0/4273617848 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389e59c700 1 --2- 172.21.15.3:0/4273617848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389d59a700 1 -- 172.21.15.3:0/4273617848 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610542079 0 0) 0x7f38a00cc5c0 con 0x7f38a00b2930 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389d59a700 1 -- 172.21.15.3:0/4273617848 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f388c003570 con 0x7f38a00b2930 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389d59a700 1 -- 172.21.15.3:0/4273617848 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734503721 0 0) 0x7f38a00cb9b0 con 0x7f38a01598d0 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389d59a700 1 -- 172.21.15.3:0/4273617848 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38a00cc5c0 con 0x7f38a01598d0 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389e59c700 1 -- 172.21.15.3:0/4273617848 >> v1:172.21.15.153:6789/0 conn(0x7f38a01598d0 legacy=0x7f38a00b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389e59c700 1 -- 172.21.15.3:0/4273617848 >> v1:172.21.15.3:6789/0 conn(0x7f38a00b2930 legacy=0x7f38a00cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.551+0000 7f389e59c700 1 -- 172.21.15.3:0/4273617848 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00cbf70 con 0x7f38a0159500 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f389e59c700 1 --2- 172.21.15.3:0/4273617848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00cad30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3898009910 tx=0x7f38980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f287e0d7a372314 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f389d59a700 1 -- 172.21.15.3:0/4273617848 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898011630 con 0x7f38a0159500 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f389d59a700 1 -- 172.21.15.3:0/4273617848 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898011790 con 0x7f38a0159500 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f389d59a700 1 -- 172.21.15.3:0/4273617848 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898011b30 con 0x7f38a0159500 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f38a4809700 1 -- 172.21.15.3:0/4273617848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 msgr2=0x7f38a00cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f38a4809700 1 --2- 172.21.15.3:0/4273617848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00cad30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3898009910 tx=0x7f38980091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f38a4809700 1 -- 172.21.15.3:0/4273617848 shutdown_connections 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f38a4809700 1 --2- 172.21.15.3:0/4273617848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00cad30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.552+0000 7f38a4809700 1 -- 172.21.15.3:0/4273617848 >> 172.21.15.3:0/4273617848 conn(0x7f38a0070310 msgr2=0x7f38a0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.553+0000 7f38a4809700 1 -- 172.21.15.3:0/4273617848 shutdown_connections 2023-10-06T04:53:31.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.553+0000 7f38a4809700 1 -- 172.21.15.3:0/4273617848 wait complete. 2023-10-06T04:53:31.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 Processor -- start 2023-10-06T04:53:31.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 -- start start 2023-10-06T04:53:31.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a00b2930 0x7f38a00b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:31.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:31.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a01598d0 0x7f38a0156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:31.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015a640 con 0x7f38a00b2930 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00b3790 con 0x7f38a01598d0 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f38a4809700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a016d750 con 0x7f38a0159500 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.554+0000 7f389e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a00b2930 0x7f38a00b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a00b2930 0x7f38a00b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41736/0 (socket says 172.21.15.3:41736) 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 -- 172.21.15.3:0/4148169530 learned_addr learned my addr 172.21.15.3:0/4148169530 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389dd9b700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389ed9d700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a01598d0 0x7f38a0156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 -- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 msgr2=0x7f38a00b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 -- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a01598d0 msgr2=0x7f38a0156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a01598d0 0x7f38a0156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 -- 172.21.15.3:0/4148169530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898009420 con 0x7f38a00b2930 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389dd9b700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f389e59c700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a00b2930 0x7f38a00b3250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3898009f00 tx=0x7f3898017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f38967fc700 1 -- 172.21.15.3:0/4148169530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389801f070 con 0x7f38a00b2930 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f38967fc700 1 -- 172.21.15.3:0/4148169530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898000bc0 con 0x7f38a00b2930 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a016da30 con 0x7f38a00b2930 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f38967fc700 1 -- 172.21.15.3:0/4148169530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898025420 con 0x7f38a00b2930 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.555+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a016dff0 con 0x7f38a00b2930 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.556+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a0071680 con 0x7f38a00b2930 2023-10-06T04:53:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.556+0000 7f38967fc700 1 -- 172.21.15.3:0/4148169530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3898023070 con 0x7f38a00b2930 2023-10-06T04:53:31.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.557+0000 7f38967fc700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:31.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.557+0000 7f38967fc700 1 -- 172.21.15.3:0/4148169530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3898085b30 con 0x7f38a00b2930 2023-10-06T04:53:31.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.557+0000 7f389dd9b700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:31.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.558+0000 7f389dd9b700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38a016d140 tx=0x7f3888009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.561+0000 7f38967fc700 1 -- 172.21.15.3:0/4148169530 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3898052c40 con 0x7f38a00b2930 2023-10-06T04:53:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.800+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38a00cc250 con 0x7f38a00b2930 2023-10-06T04:53:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.801+0000 7f38967fc700 1 -- 172.21.15.3:0/4148169530 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f389802a020 con 0x7f38a00b2930 2023-10-06T04:53:31.802 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:31.803 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3880066f30 msgr2=0x7f38800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38a016d140 tx=0x7f3888009040 comp rx=0 tx=0).stop 2023-10-06T04:53:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a00b2930 msgr2=0x7f38a00b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a00b2930 0x7f38a00b3250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3898009f00 tx=0x7f3898017000 comp rx=0 tx=0).stop 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 shutdown_connections 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a01598d0 0x7f38a0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a00b2930 0x7f38a00b3250 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 --2- 172.21.15.3:0/4148169530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38a0159500 0x7f38a00b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 >> 172.21.15.3:0/4148169530 conn(0x7f38a0070310 msgr2=0x7f38a0150ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 shutdown_connections 2023-10-06T04:53:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:31.803+0000 7f38a4809700 1 -- 172.21.15.3:0/4148169530 wait complete. 2023-10-06T04:53:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:31.817 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T04:53:31.964 INFO:teuthology.orchestra.run.smithi153.stdout:[{"id":5089,"entity":{"name":{"type":"client","num":5089},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":960979156}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":15.011741269,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9040115935405364,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5089 192.168.0.1:0/960979156","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"98077","root":"/"}},{"id":5154,"entity":{"name":{"type":"client","num":5154},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":475472563}},"state":"open","num_leases":6,"num_caps":350,"request_load_avg":1582,"uptime":16.807710314000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":1972.9543157515459,"halflife":60},"release_caps":{"value":2350.7295249856584,"halflife":60},"recall_caps_throttle":{"value":922.46712905490222,"halflife":1.5},"recall_caps_throttle2o":{"value":212.41349198894872,"halflife":0.5},"session_cache_liveness":{"value":4218.3100822939277,"halflife":300},"cap_acquisition":{"value":1397.0484537218108,"halflife":30},"last_trim_completed_requests_tid":1754,"last_trim_completed_flushes_tid":1501,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5154 192.168.0.1:0/475472563","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000007d6","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"103658","root":"/"}}] 2023-10-06T04:53:31.965 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "98077", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 960979156, "type": "any" }, "name": { "num": 5089, "type": "client" } }, "id": 5089, "inst": "client.5089 192.168.0.1:0/960979156", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9040115935405364 }, "state": "open", "uptime": 15.011741269 }, { "cap_acquisition": { "halflife": 30, "value": 1397.0484537218108 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "103658", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 475472563, "type": "any" }, "name": { "num": 5154, "type": "client" } }, "id": 5154, "inst": "client.5154 192.168.0.1:0/475472563", "last_trim_completed_flushes_tid": 1501, "last_trim_completed_requests_tid": 1754, "num_caps": 350, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 6, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 1000, "start": "0x100000007d6" } ], "recall_caps": { "halflife": 60, "value": 1972.954315751546 }, "recall_caps_throttle": { "halflife": 1.5, "value": 922.4671290549022 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 212.41349198894872 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 2350.7295249856584 }, "request_load_avg": 1582, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 4218.310082293928 }, "state": "open", "uptime": 16.807710314 } ] 2023-10-06T04:53:31.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:32.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.083+0000 7f4cbca21700 1 Processor -- start 2023-10-06T04:53:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.083+0000 7f4cbca21700 1 -- start start 2023-10-06T04:53:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cbca21700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2530 0x7f4cb80b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cbca21700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4cb8159080 0x7f4cb8159500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cbca21700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb815e0d0 con 0x7f4cb80b2900 2023-10-06T04:53:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cbca21700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb80b0e00 con 0x7f4cb8159080 2023-10-06T04:53:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cbca21700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80cbd70 con 0x7f4cb80b2530 2023-10-06T04:53:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2530 0x7f4cb80b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb6d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4cb8159080 0x7f4cb8159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb5d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4cb80b2900 0x7f4cb815d9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:48624/0 (socket says 172.21.15.3:48624) 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb5d9b700 1 -- 172.21.15.3:0/2771761448 learned_addr learned my addr 172.21.15.3:0/2771761448 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb559a700 1 -- 172.21.15.3:0/2771761448 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309293555 0 0) 0x7f4ca0001110 con 0x7f4cb80b2900 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb559a700 1 -- 172.21.15.3:0/2771761448 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca4003570 con 0x7f4cb80b2900 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb659c700 1 -- 172.21.15.3:0/2771761448 >> v2:172.21.15.153:3300/0 conn(0x7f4cb8159080 msgr2=0x7f4cb8159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb659c700 1 --2- 172.21.15.3:0/2771761448 >> v2:172.21.15.153:3300/0 conn(0x7f4cb8159080 0x7f4cb8159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb659c700 1 -- 172.21.15.3:0/2771761448 >> v1:172.21.15.153:6789/0 conn(0x7f4cb80b2900 legacy=0x7f4cb815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb659c700 1 -- 172.21.15.3:0/2771761448 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb80cb160 con 0x7f4cb80b2530 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.084+0000 7f4cb6d9d700 1 --2- 172.21.15.3:0/2771761448 >> v2:172.21.15.153:3300/0 conn(0x7f4cb8159080 0x7f4cb8159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cb659c700 1 --2- 172.21.15.3:0/2771761448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2530 0x7f4cb80b08c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00098e0 tx=0x7f4cb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c864e7af8fbd2546 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cb559a700 1 -- 172.21.15.3:0/2771761448 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0011630 con 0x7f4cb80b2530 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cb559a700 1 -- 172.21.15.3:0/2771761448 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb0011790 con 0x7f4cb80b2530 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cb559a700 1 -- 172.21.15.3:0/2771761448 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0011b30 con 0x7f4cb80b2530 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 -- 172.21.15.3:0/2771761448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2530 msgr2=0x7f4cb80b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 --2- 172.21.15.3:0/2771761448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2530 0x7f4cb80b08c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00098e0 tx=0x7f4cb00091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 -- 172.21.15.3:0/2771761448 shutdown_connections 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 --2- 172.21.15.3:0/2771761448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2530 0x7f4cb80b08c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 --2- 172.21.15.3:0/2771761448 >> v2:172.21.15.153:3300/0 conn(0x7f4cb8159080 0x7f4cb8159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 -- 172.21.15.3:0/2771761448 >> 172.21.15.3:0/2771761448 conn(0x7f4cb8070310 msgr2=0x7f4cb8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 -- 172.21.15.3:0/2771761448 shutdown_connections 2023-10-06T04:53:32.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.085+0000 7f4cbca21700 1 -- 172.21.15.3:0/2771761448 wait complete. 2023-10-06T04:53:32.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.086+0000 7f4cbca21700 1 Processor -- start 2023-10-06T04:53:32.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cbca21700 1 -- start start 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cbca21700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb80b2530 0x7f4cb80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cbca21700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2900 0x7f4cb80c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cbca21700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cb8159080 0x7f4cb80c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cbca21700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80c7ec0 con 0x7f4cb80b2530 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cbca21700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80b4af0 con 0x7f4cb8159080 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cbca21700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80b3ee0 con 0x7f4cb80b2900 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cb659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb80b2530 0x7f4cb80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cb659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb80b2530 0x7f4cb80b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41738/0 (socket says 172.21.15.3:41738) 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cb5d9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2900 0x7f4cb80c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:32.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cb659c700 1 -- 172.21.15.3:0/3023138614 learned_addr learned my addr 172.21.15.3:0/3023138614 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.087+0000 7f4cb6d9d700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cb8159080 0x7f4cb80c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cb659c700 1 -- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2900 msgr2=0x7f4cb80c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cb659c700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2900 0x7f4cb80c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cb659c700 1 -- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cb8159080 msgr2=0x7f4cb80c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cb659c700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cb8159080 0x7f4cb80c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cb659c700 1 -- 172.21.15.3:0/3023138614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8009a20 con 0x7f4cb80b2530 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cb5d9b700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2900 0x7f4cb80c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cb659c700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb80b2530 0x7f4cb80b4f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80cbeb0 tx=0x7f4cb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cae7fc700 1 -- 172.21.15.3:0/3023138614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0011b10 con 0x7f4cb80b2530 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cae7fc700 1 -- 172.21.15.3:0/3023138614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb00253f0 con 0x7f4cb80b2530 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb0009420 con 0x7f4cb80b2530 2023-10-06T04:53:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cae7fc700 1 -- 172.21.15.3:0/3023138614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00257b0 con 0x7f4cb80b2530 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.088+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb80b44a0 con 0x7f4cb80b2530 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.089+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb8071680 con 0x7f4cb80b2530 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.089+0000 7f4cae7fc700 1 -- 172.21.15.3:0/3023138614 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cb0025ba0 con 0x7f4cb80b2530 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.090+0000 7f4cae7fc700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c98066f30 0x7f4c980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.090+0000 7f4cae7fc700 1 -- 172.21.15.3:0/3023138614 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4cb0085e10 con 0x7f4cb80b2530 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.090+0000 7f4cb5d9b700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c98066f30 0x7f4c980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.090+0000 7f4cb5d9b700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c98066f30 0x7f4c980693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80b5550 tx=0x7f4ca0007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.094+0000 7f4cae7fc700 1 -- 172.21.15.3:0/3023138614 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cb0052fe0 con 0x7f4cb80b2530 2023-10-06T04:53:32.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.334+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cb80cb440 con 0x7f4cb80b2530 2023-10-06T04:53:32.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.335+0000 7f4cae7fc700 1 -- 172.21.15.3:0/3023138614 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f4cb0056600 con 0x7f4cb80b2530 2023-10-06T04:53:32.336 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:32.337 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:32.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c98066f30 msgr2=0x7f4c980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c98066f30 0x7f4c980693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80b5550 tx=0x7f4ca0007bf0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb80b2530 msgr2=0x7f4cb80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb80b2530 0x7f4cb80b4f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80cbeb0 tx=0x7f4cb0017000 comp rx=0 tx=0).stop 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 shutdown_connections 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c98066f30 0x7f4c980693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cb8159080 0x7f4cb80c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb80b2530 0x7f4cb80b4f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 --2- 172.21.15.3:0/3023138614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4cb80b2900 0x7f4cb80c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 >> 172.21.15.3:0/3023138614 conn(0x7f4cb8070310 msgr2=0x7f4cb8156100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 shutdown_connections 2023-10-06T04:53:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:32.337+0000 7f4cbca21700 1 -- 172.21.15.3:0/3023138614 wait complete. 2023-10-06T04:53:32.340 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:32.350 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json perf dump 2023-10-06T04:53:32.485 INFO:teuthology.orchestra.run.smithi153.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":903,"msgr_send_messages":888,"msgr_recv_bytes":226083,"msgr_send_bytes":4313943,"msgr_created_connections":8,"msgr_active_connections":4,"msgr_running_total_time":0.060745849,"msgr_running_send_time":0.021348708,"msgr_running_recv_time":0.017040299,"msgr_running_fast_dispatch_time":0.017365630,"msgr_send_messages_queue_lat":{"avgcount":887,"sum":0.167951019,"avgtime":0.000189347},"msgr_handle_ack_lat":{"avgcount":25,"sum":0.000003120,"avgtime":0.000000124}},"AsyncMessenger::Worker-1":{"msgr_recv_messages":723,"msgr_send_messages":717,"msgr_recv_bytes":272364,"msgr_send_bytes":4416818,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.052132995,"msgr_running_send_time":0.020434329,"msgr_running_recv_time":0.018313228,"msgr_running_fast_dispatch_time":0.013135014,"msgr_send_messages_queue_lat":{"avgcount":716,"sum":0.131235020,"avgtime":0.000183289},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"AsyncMessenger::Worker-2":{"msgr_recv_messages":4804,"msgr_send_messages":6302,"msgr_recv_bytes":1338438,"msgr_send_bytes":3227136,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.234148894,"msgr_running_send_time":0.103826557,"msgr_running_recv_time":0.172437794,"msgr_running_fast_dispatch_time":0.000997412,"msgr_send_messages_queue_lat":{"avgcount":6301,"sum":0.102682934,"avgtime":0.000016296},"msgr_handle_ack_lat":{"avgcount":11032,"sum":0.001275798,"avgtime":0.000000115}},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":3066,"sum":1.229974084,"avgtime":0.000401165}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000015707,"avgtime":0.000007853}},"mds":{"request":1756,"reply":1756,"reply_latency":{"avgcount":1756,"sum":5.115662602,"avgtime":0.002913247},"slow_reply":0,"forward":0,"dir_fetch":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":1516,"inodes_top":0,"inodes_bottom":1516,"inodes_pin_tail":0,"inodes_pinned":1516,"inodes_expired":0,"inodes_with_caps":350,"caps":351,"subtrees":2,"traverse":2005,"traverse_hit":1564,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":382898,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1500,"root_rbytes":3840,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":11,"ceph_cap_op_grant":1500,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1511,"handle_client_caps_dirty":1500,"handle_client_cap_release":7,"process_request_cap_release":1507},"mds_cache":{"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4520,"evex":0,"evtrm":0,"ev":4520,"evexg":0,"evexd":0,"segadd":6,"segex":0,"segtrm":0,"seg":6,"segexg":0,"segexd":0,"expos":4194304,"wrpos":13616516,"rdpos":0,"jlat":{"avgcount":1515,"sum":2.417668498,"avgtime":0.001595820},"replayed":0},"mds_mem":{"ino":1519,"ino+":1519,"ino-":0,"dir":18,"dir+":18,"dir-":0,"dn":1516,"dn+":1516,"dn-":0,"cap":351,"cap+":2764,"cap-":2413,"rss":60024,"heap":207104},"mds_server":{"dispatch_client_request":1775,"dispatch_server_request":0,"handle_client_request":1756,"handle_client_session":1506,"handle_peer_request":0,"req_create_latency":{"avgcount":1500,"sum":0.891438032,"avgtime":0.000594292},"req_getattr_latency":{"avgcount":23,"sum":0.155049179,"avgtime":0.006741268},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":219,"sum":0.035842101,"avgtime":0.000163662},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":6,"sum":0.006205469,"avgtime":0.001034244},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":7,"sum":4.026646481,"avgtime":0.575235211},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000481340,"avgtime":0.000481340},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":8,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":1593,"average_load":796,"avg_session_uptime":13},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_Buffer_bytes":0,"bluestore_Buffer_items":0,"bluestore_Extent_bytes":0,"bluestore_Extent_items":0,"bluestore_Blob_bytes":0,"bluestore_Blob_items":0,"bluestore_SharedBlob_bytes":0,"bluestore_SharedBlob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":492095,"buffer_anon_items":158,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":6104,"osdmap_items":78,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":4498240,"mds_co_items":30067,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":1559,"op_send_bytes":9488674,"op_resend":26,"op_reply":1559,"oplen_avg":{"avgcount":1559,"sum":1588},"op":1559,"op_r":0,"op_w":1559,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":1515,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":8,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":32,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":21,"omap_rd":0,"omap_del":3},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":6425,"get_sum":1254538,"get_or_fail_fail":0,"get_or_fail_success":6425,"take":0,"take_sum":0,"put":6425,"put_sum":1254538,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":1559,"take_sum":9488674,"put":1559,"put_sum":9488674,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":1559,"take_sum":1559,"put":1559,"put_sum":1559,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ce24d4fe00":{"val":0,"max":3758096384,"get_started":0,"get":4520,"get_sum":9422212,"get_or_fail_fail":0,"get_or_fail_success":4520,"take":0,"take_sum":0,"put":1515,"put_sum":9422212,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-10-06T04:53:32.487 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 4, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 25, "avgtime": 1.24e-07, "sum": 3.12e-06 }, "msgr_recv_bytes": 226083, "msgr_recv_messages": 903, "msgr_running_fast_dispatch_time": 0.01736563, "msgr_running_recv_time": 0.017040299, "msgr_running_send_time": 0.021348708, "msgr_running_total_time": 0.060745849, "msgr_send_bytes": 4313943, "msgr_send_messages": 888, "msgr_send_messages_queue_lat": { "avgcount": 887, "avgtime": 0.000189347, "sum": 0.167951019 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 272364, "msgr_recv_messages": 723, "msgr_running_fast_dispatch_time": 0.013135014, "msgr_running_recv_time": 0.018313228, "msgr_running_send_time": 0.020434329, "msgr_running_total_time": 0.052132995, "msgr_send_bytes": 4416818, "msgr_send_messages": 717, "msgr_send_messages_queue_lat": { "avgcount": 716, "avgtime": 0.000183289, "sum": 0.13123502 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 11032, "avgtime": 1.15e-07, "sum": 0.001275798 }, "msgr_recv_bytes": 1338438, "msgr_recv_messages": 4804, "msgr_running_fast_dispatch_time": 0.000997412, "msgr_running_recv_time": 0.172437794, "msgr_running_send_time": 0.103826557, "msgr_running_total_time": 0.234148894, "msgr_send_bytes": 3227136, "msgr_send_messages": 6302, "msgr_send_messages_queue_lat": { "avgcount": 6301, "avgtime": 1.6296e-05, "sum": 0.102682934 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 3066, "avgtime": 0.000401165, "sum": 1.229974084 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 7.853e-06, "sum": 1.5707e-05 }, "queue_len": 0 }, "mds": { "caps": 351, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1500, "ceph_cap_op_revoke": 11, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 7, "handle_client_caps": 1511, "handle_client_caps_dirty": 1500, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 1516, "inodes_bottom": 1516, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 1516, "inodes_top": 0, "inodes_with_caps": 350, "load_cent": 382898, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1507, "q": 0, "reply": 1756, "reply_latency": { "avgcount": 1756, "avgtime": 0.002913247, "sum": 5.115662602 }, "request": 1756, "root_rbytes": 3840, "root_rfiles": 1500, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 2005, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 1564, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 4520, "evadd": 4520, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 1515, "avgtime": 0.00159582, "sum": 2.417668498 }, "rdpos": 0, "replayed": 0, "seg": 6, "segadd": 6, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 13616516 }, "mds_mem": { "cap": 351, "cap+": 2764, "cap-": 2413, "dir": 18, "dir+": 18, "dir-": 0, "dn": 1516, "dn+": 1516, "dn-": 0, "heap": 207104, "ino": 1519, "ino+": 1519, "ino-": 0, "rss": 60024 }, "mds_server": { "cap_acquisition_throttle": 8, "cap_revoke_eviction": 0, "dispatch_client_request": 1775, "dispatch_server_request": 0, "handle_client_request": 1756, "handle_client_session": 1506, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1500, "avgtime": 0.000594292, "sum": 0.891438032 }, "req_getattr_latency": { "avgcount": 23, "avgtime": 0.006741268, "sum": 0.155049179 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 219, "avgtime": 0.000163662, "sum": 0.035842101 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 6, "avgtime": 0.001034244, "sum": 0.006205469 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 7, "avgtime": 0.575235211, "sum": 4.026646481 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.00048134, "sum": 0.00048134 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 796, "avg_session_uptime": 13, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 1593 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_Blob_bytes": 0, "bluestore_Blob_items": 0, "bluestore_Buffer_bytes": 0, "bluestore_Buffer_items": 0, "bluestore_Extent_bytes": 0, "bluestore_Extent_items": 0, "bluestore_SharedBlob_bytes": 0, "bluestore_SharedBlob_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 492095, "buffer_anon_items": 158, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 4498240, "mds_co_items": 30067, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 6104, "osdmap_items": 78, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 32, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 0, "omap_wr": 21, "op": 1559, "op_active": 0, "op_laggy": 0, "op_pg": 0, "op_r": 0, "op_reply": 1559, "op_resend": 26, "op_rmw": 0, "op_send": 1559, "op_send_bytes": 9488674, "op_w": 1559, "oplen_avg": { "avgcount": 1559, "sum": 1588 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 8, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 1515, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 6425, "get_or_fail_fail": 0, "get_or_fail_success": 6425, "get_started": 0, "get_sum": 1254538, "max": 104857600, "put": 6425, "put_sum": 1254538, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 1559, "put_sum": 9488674, "take": 1559, "take_sum": 9488674, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 1559, "put_sum": 1559, "take": 1559, "take_sum": 1559, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ce24d4fe00": { "get": 4520, "get_or_fail_fail": 0, "get_or_fail_success": 4520, "get_started": 0, "get_sum": 9422212, "max": 3758096384, "put": 1515, "put_sum": 9422212, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-10-06T04:53:32.490 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:53:32.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:53:32.529 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:53:32.563 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:53:32.563 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:53:32.564 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:53:32.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:53:32.600 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[103658]: fuse finished with error 0 and tester_r 0 2023-10-06T04:53:32.602 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:53:32.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:32.602 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:53:32.758 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:53:32.759 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:53:32.759 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:53:32.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[98077]: fuse finished with error 0 and tester_r 0 2023-10-06T04:53:32.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:53:32.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:32.792 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:53:32.952 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:53:32.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:32.952 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:53:32.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:53:32.969 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T04:53:32.969 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:53:32.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:53:32.969 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:53:33.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:53:33.008 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T04:53:33.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:33.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.128+0000 7f3d98b9e700 1 Processor -- start 2023-10-06T04:53:33.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.128+0000 7f3d98b9e700 1 -- start start 2023-10-06T04:53:33.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.129+0000 7f3d98b9e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3d9415ccc0 0x7f3d940d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.129+0000 7f3d98b9e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940cf6c0 con 0x7f3d940d06a0 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.129+0000 7f3d98b9e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940cfb50 con 0x7f3d9415c8f0 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.129+0000 7f3d98b9e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d940d0b00 con 0x7f3d9415ccc0 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.129+0000 7f3d92d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3d940d06a0 0x7f3d940b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45290/0 (socket says 172.21.15.3:45290) 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.129+0000 7f3d92d9d700 1 -- 172.21.15.3:0/2298783576 learned_addr learned my addr 172.21.15.3:0/2298783576 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.129+0000 7f3d91d9b700 1 --2- 172.21.15.3:0/2298783576 >> v2:172.21.15.153:3300/0 conn(0x7f3d9415ccc0 0x7f3d940d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279483638 0 0) 0x7f3d940cf6c0 con 0x7f3d940d06a0 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d80003570 con 0x7f3d940d06a0 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963937051 0 0) 0x7f3d940cfb50 con 0x7f3d9415c8f0 2023-10-06T04:53:33.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d940cf6c0 con 0x7f3d9415c8f0 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d91d9b700 1 -- 172.21.15.3:0/2298783576 >> v1:172.21.15.153:6789/0 conn(0x7f3d9415c8f0 legacy=0x7f3d9415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296801022 0 0) 0x7f3d80003570 con 0x7f3d940d06a0 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d91d9b700 1 -- 172.21.15.3:0/2298783576 >> v1:172.21.15.3:6789/0 conn(0x7f3d940d06a0 legacy=0x7f3d940b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d91d9b700 1 -- 172.21.15.3:0/2298783576 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940ceab0 con 0x7f3d9415ccc0 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d91d9b700 1 --2- 172.21.15.3:0/2298783576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d940d0160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0098e0 tx=0x7f3d7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79240c947704e2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0017f0 con 0x7f3d9415ccc0 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c011420 con 0x7f3d9415ccc0 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.130+0000 7f3d9159a700 1 -- 172.21.15.3:0/2298783576 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0117c0 con 0x7f3d9415ccc0 2023-10-06T04:53:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.131+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2298783576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 msgr2=0x7f3d940d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.131+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2298783576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d940d0160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0098e0 tx=0x7f3d7c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.131+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2298783576 shutdown_connections 2023-10-06T04:53:33.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.131+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2298783576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d940d0160 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.131+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2298783576 >> 172.21.15.3:0/2298783576 conn(0x7f3d94070310 msgr2=0x7f3d94070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:33.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.131+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2298783576 shutdown_connections 2023-10-06T04:53:33.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.131+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2298783576 wait complete. 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.132+0000 7f3d98b9e700 1 Processor -- start 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.132+0000 7f3d98b9e700 1 -- start start 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d940d06a0 0x7f3d94154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9415c8f0 0x7f3d94169b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d9415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9415b180 con 0x7f3d9415c8f0 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9415a570 con 0x7f3d9415ccc0 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9415a850 con 0x7f3d940d06a0 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9415c8f0 0x7f3d94169b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9415c8f0 0x7f3d94169b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52790/0 (socket says 172.21.15.3:52790) 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 -- 172.21.15.3:0/2338437919 learned_addr learned my addr 172.21.15.3:0/2338437919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d92d9d700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d9415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d9259c700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d940d06a0 0x7f3d94154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 -- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d940d06a0 msgr2=0x7f3d94154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d940d06a0 0x7f3d94154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 -- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 msgr2=0x7f3d9415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d9415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 -- 172.21.15.3:0/2338437919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c009420 con 0x7f3d9415c8f0 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d92d9d700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d9415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d91d9b700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9415c8f0 0x7f3d94169b70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d9406eaa0 tx=0x7f3d7c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2338437919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c01cc90 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2338437919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c01cdf0 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9415ab30 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2338437919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c025680 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.133+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d94155390 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.134+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d940b2c70 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.135+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2338437919 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d7c0193f0 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.135+0000 7f3d8a7fc700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d74066f30 0x7f3d740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.135+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2338437919 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d7c0859c0 con 0x7f3d9415c8f0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.135+0000 7f3d9259c700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d74066f30 0x7f3d740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.136+0000 7f3d9259c700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d74066f30 0x7f3d740693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3d84003870 tx=0x7f3d84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.139+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2338437919 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d7c052b50 con 0x7f3d9415c8f0 2023-10-06T04:53:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.377+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d94071680 con 0x7f3d9415c8f0 2023-10-06T04:53:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.377+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2338437919 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f3d7c056170 con 0x7f3d9415c8f0 2023-10-06T04:53:33.382 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:33.382 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d74066f30 msgr2=0x7f3d740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d74066f30 0x7f3d740693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3d84003870 tx=0x7f3d84009040 comp rx=0 tx=0).stop 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9415c8f0 msgr2=0x7f3d94169b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9415c8f0 0x7f3d94169b70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d9406eaa0 tx=0x7f3d7c016000 comp rx=0 tx=0).stop 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 shutdown_connections 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d74066f30 0x7f3d740693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d9415ccc0 0x7f3d9415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d940d06a0 0x7f3d94154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 --2- 172.21.15.3:0/2338437919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9415c8f0 0x7f3d94169b70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 >> 172.21.15.3:0/2338437919 conn(0x7f3d94070310 msgr2=0x7f3d94156930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:33.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 shutdown_connections 2023-10-06T04:53:33.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.379+0000 7f3d98b9e700 1 -- 172.21.15.3:0/2338437919 wait complete. 2023-10-06T04:53:33.385 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:33.393 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:33.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.510+0000 7f2ac94fb700 1 Processor -- start 2023-10-06T04:53:33.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.510+0000 7f2ac94fb700 1 -- start start 2023-10-06T04:53:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac94fb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ac415dbd0 0x7f2ac40b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac94fb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ac4159570 0x7f2ac40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac94fb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac40b0a90 con 0x7f2ac41591a0 2023-10-06T04:53:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac94fb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac40b2a60 con 0x7f2ac415dbd0 2023-10-06T04:53:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac94fb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac40b3110 con 0x7f2ac4159570 2023-10-06T04:53:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ac415dbd0 0x7f2ac40b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2ac41591a0 0x7f2ac40affd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45292/0 (socket says 172.21.15.3:45292) 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac2ffd700 1 -- 172.21.15.3:0/90622743 learned_addr learned my addr 172.21.15.3:0/90622743 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac37fe700 1 --2- 172.21.15.3:0/90622743 >> v2:172.21.15.153:3300/0 conn(0x7f2ac4159570 0x7f2ac40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac27fc700 1 -- 172.21.15.3:0/90622743 >> v2:172.21.15.153:3300/0 conn(0x7f2ac4159570 msgr2=0x7f2ac40b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac27fc700 1 --2- 172.21.15.3:0/90622743 >> v2:172.21.15.153:3300/0 conn(0x7f2ac4159570 0x7f2ac40b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac27fc700 1 -- 172.21.15.3:0/90622743 >> v1:172.21.15.3:6789/0 conn(0x7f2ac41591a0 legacy=0x7f2ac40affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac27fc700 1 -- 172.21.15.3:0/90622743 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40b33f0 con 0x7f2ac415dbd0 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.511+0000 7f2ac1ffb700 1 -- 172.21.15.3:0/90622743 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672402181 0 0) 0x7f2ac40b0a90 con 0x7f2ac41591a0 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac27fc700 1 --2- 172.21.15.3:0/90622743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac415dbd0 0x7f2ac40b1fe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0098e0 tx=0x7f2aac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2436b9bbf8dd8cca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac1ffb700 1 -- 172.21.15.3:0/90622743 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011630 con 0x7f2ac415dbd0 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac1ffb700 1 -- 172.21.15.3:0/90622743 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac011790 con 0x7f2ac415dbd0 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac1ffb700 1 -- 172.21.15.3:0/90622743 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011b30 con 0x7f2ac415dbd0 2023-10-06T04:53:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 -- 172.21.15.3:0/90622743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac415dbd0 msgr2=0x7f2ac40b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/90622743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac415dbd0 0x7f2ac40b1fe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0098e0 tx=0x7f2aac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 -- 172.21.15.3:0/90622743 shutdown_connections 2023-10-06T04:53:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/90622743 >> v2:172.21.15.153:3300/0 conn(0x7f2ac4159570 0x7f2ac40b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/90622743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac415dbd0 0x7f2ac40b1fe0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 -- 172.21.15.3:0/90622743 >> 172.21.15.3:0/90622743 conn(0x7f2ac4070310 msgr2=0x7f2ac4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 -- 172.21.15.3:0/90622743 shutdown_connections 2023-10-06T04:53:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.512+0000 7f2ac94fb700 1 -- 172.21.15.3:0/90622743 wait complete. 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.514+0000 7f2ac94fb700 1 Processor -- start 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.514+0000 7f2ac94fb700 1 -- start start 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac94fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac41591a0 0x7f2ac4145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac94fb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ac4159570 0x7f2ac415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac94fb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ac415dbd0 0x7f2ac40c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac94fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac415c3f0 con 0x7f2ac41591a0 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac94fb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40c8070 con 0x7f2ac4159570 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac94fb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40c8350 con 0x7f2ac415dbd0 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac41591a0 0x7f2ac4145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac41591a0 0x7f2ac4145910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52800/0 (socket says 172.21.15.3:52800) 2023-10-06T04:53:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 -- 172.21.15.3:0/787395217 learned_addr learned my addr 172.21.15.3:0/787395217 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac27fc700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ac4159570 0x7f2ac415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac37fe700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ac415dbd0 0x7f2ac40c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 -- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ac415dbd0 msgr2=0x7f2ac40c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ac415dbd0 0x7f2ac40c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 -- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ac4159570 msgr2=0x7f2ac415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ac4159570 0x7f2ac415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 -- 172.21.15.3:0/787395217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac009420 con 0x7f2ac41591a0 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac37fe700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ac415dbd0 0x7f2ac40c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.515+0000 7f2ac2ffd700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac41591a0 0x7f2ac4145910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c7e0 tx=0x7f2ab4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.516+0000 7f2abaffd700 1 -- 172.21.15.3:0/787395217 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4005b30 con 0x7f2ac41591a0 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.516+0000 7f2abaffd700 1 -- 172.21.15.3:0/787395217 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab4005c90 con 0x7f2ac41591a0 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.516+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac40c8630 con 0x7f2ac41591a0 2023-10-06T04:53:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.516+0000 7f2abaffd700 1 -- 172.21.15.3:0/787395217 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4007300 con 0x7f2ac41591a0 2023-10-06T04:53:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.516+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac40c8bf0 con 0x7f2ac41591a0 2023-10-06T04:53:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.517+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac40c8e10 con 0x7f2ac41591a0 2023-10-06T04:53:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.517+0000 7f2abaffd700 1 -- 172.21.15.3:0/787395217 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ab4007670 con 0x7f2ac41591a0 2023-10-06T04:53:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.517+0000 7f2abaffd700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2aa4066f30 0x7f2aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.517+0000 7f2abaffd700 1 -- 172.21.15.3:0/787395217 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ab4083c10 con 0x7f2ac41591a0 2023-10-06T04:53:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.518+0000 7f2ac27fc700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2aa4066f30 0x7f2aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.518+0000 7f2ac27fc700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2aa4066f30 0x7f2aa40693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0091e0 tx=0x7f2aac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.523+0000 7f2abaffd700 1 -- 172.21.15.3:0/787395217 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ab4050d20 con 0x7f2ac41591a0 2023-10-06T04:53:33.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.716+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ac404c380 con 0x7f2ac41591a0 2023-10-06T04:53:33.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.717+0000 7f2abaffd700 1 -- 172.21.15.3:0/787395217 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f2ab4017020 con 0x7f2ac41591a0 2023-10-06T04:53:33.719 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:33.719 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:09.096476+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:05.231194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:06.473725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:33.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2aa4066f30 msgr2=0x7f2aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2aa4066f30 0x7f2aa40693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0091e0 tx=0x7f2aac017000 comp rx=0 tx=0).stop 2023-10-06T04:53:33.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac41591a0 msgr2=0x7f2ac4145910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac41591a0 0x7f2ac4145910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c7e0 tx=0x7f2ab4007df0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 shutdown_connections 2023-10-06T04:53:33.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2aa4066f30 0x7f2aa40693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ac415dbd0 0x7f2ac40c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ac41591a0 0x7f2ac4145910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 --2- 172.21.15.3:0/787395217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ac4159570 0x7f2ac415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 >> 172.21.15.3:0/787395217 conn(0x7f2ac4070310 msgr2=0x7f2ac40cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 shutdown_connections 2023-10-06T04:53:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.719+0000 7f2ac94fb700 1 -- 172.21.15.3:0/787395217 wait complete. 2023-10-06T04:53:33.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:53:05.231194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:53:06.473725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:53:33.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:53:33.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.855+0000 7f54f1776700 1 Processor -- start 2023-10-06T04:53:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.855+0000 7f54f1776700 1 -- start start 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54f1776700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f54ec15cae0 0x7f54ec0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54f1776700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec0b0520 0x7f54ec0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54f1776700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54ec0cd8e0 0x7f54ec0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54f1776700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0b0c70 con 0x7f54ec0cd8e0 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54f1776700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0cb7b0 con 0x7f54ec15cae0 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54f1776700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0b2c10 con 0x7f54ec0b0520 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54ec0cd8e0 0x7f54ec0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54ec0cd8e0 0x7f54ec0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52806/0 (socket says 172.21.15.3:52806) 2023-10-06T04:53:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 -- 172.21.15.3:0/1080534795 learned_addr learned my addr 172.21.15.3:0/1080534795 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54ea7fc700 1 --2- 172.21.15.3:0/1080534795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec0b0520 0x7f54ec0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eaffd700 1 --2- 172.21.15.3:0/1080534795 >> v2:172.21.15.153:3300/0 conn(0x7f54ec15cae0 0x7f54ec0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 -- 172.21.15.3:0/1080534795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec0b0520 msgr2=0x7f54ec0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 --2- 172.21.15.3:0/1080534795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec0b0520 0x7f54ec0b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 -- 172.21.15.3:0/1080534795 >> v2:172.21.15.153:3300/0 conn(0x7f54ec15cae0 msgr2=0x7f54ec0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 --2- 172.21.15.3:0/1080534795 >> v2:172.21.15.153:3300/0 conn(0x7f54ec15cae0 0x7f54ec0affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 -- 172.21.15.3:0/1080534795 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0b0990 con 0x7f54ec0cd8e0 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54ea7fc700 1 --2- 172.21.15.3:0/1080534795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec0b0520 0x7f54ec0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54eb7fe700 1 --2- 172.21.15.3:0/1080534795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0cd8e0 0x7f54ec0b26d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f54ec089760 tx=0x7f54dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe324ceab7ea53cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.856+0000 7f54e9ffb700 1 -- 172.21.15.3:0/1080534795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00e070 con 0x7f54ec0cd8e0 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54e9ffb700 1 -- 172.21.15.3:0/1080534795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00ba50 con 0x7f54ec0cd8e0 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54e9ffb700 1 -- 172.21.15.3:0/1080534795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0092e0 con 0x7f54ec0cd8e0 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 -- 172.21.15.3:0/1080534795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0cd8e0 msgr2=0x7f54ec0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 --2- 172.21.15.3:0/1080534795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0cd8e0 0x7f54ec0b26d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f54ec089760 tx=0x7f54dc009840 comp rx=0 tx=0).stop 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 -- 172.21.15.3:0/1080534795 shutdown_connections 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 --2- 172.21.15.3:0/1080534795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0cd8e0 0x7f54ec0b26d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 --2- 172.21.15.3:0/1080534795 >> v2:172.21.15.153:3300/0 conn(0x7f54ec15cae0 0x7f54ec0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 --2- 172.21.15.3:0/1080534795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec0b0520 0x7f54ec0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 -- 172.21.15.3:0/1080534795 >> 172.21.15.3:0/1080534795 conn(0x7f54ec070310 msgr2=0x7f54ec070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 -- 172.21.15.3:0/1080534795 shutdown_connections 2023-10-06T04:53:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.857+0000 7f54f1776700 1 -- 172.21.15.3:0/1080534795 wait complete. 2023-10-06T04:53:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.858+0000 7f54f1776700 1 Processor -- start 2023-10-06T04:53:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.858+0000 7f54f1776700 1 -- start start 2023-10-06T04:53:33.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54f1776700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0b0520 0x7f54ec0d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54f1776700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec0cd8e0 0x7f54ec0cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54f1776700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec15cae0 0x7f54ec15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54f1776700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0ceeb0 con 0x7f54ec0b0520 2023-10-06T04:53:33.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54f1776700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0d1030 con 0x7f54ec0cd8e0 2023-10-06T04:53:33.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54f1776700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0c6be0 con 0x7f54ec15cae0 2023-10-06T04:53:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0b0520 0x7f54ec0d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0b0520 0x7f54ec0d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52822/0 (socket says 172.21.15.3:52822) 2023-10-06T04:53:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 -- 172.21.15.3:0/3547864644 learned_addr learned my addr 172.21.15.3:0/3547864644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54ea7fc700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec0cd8e0 0x7f54ec0cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eb7fe700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec15cae0 0x7f54ec15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 -- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec15cae0 msgr2=0x7f54ec15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec15cae0 0x7f54ec15ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 -- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec0cd8e0 msgr2=0x7f54ec0cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec0cd8e0 0x7f54ec0cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.859+0000 7f54eaffd700 1 -- 172.21.15.3:0/3547864644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009ae0 con 0x7f54ec0b0520 2023-10-06T04:53:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.860+0000 7f54eaffd700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0b0520 0x7f54ec0d0af0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f54e400d890 tx=0x7f54e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.860+0000 7f54e2ffd700 1 -- 172.21.15.3:0/3547864644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40125f0 con 0x7f54ec0b0520 2023-10-06T04:53:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.860+0000 7f54e2ffd700 1 -- 172.21.15.3:0/3547864644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e4012750 con 0x7f54ec0b0520 2023-10-06T04:53:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.860+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec0cf710 con 0x7f54ec0b0520 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.860+0000 7f54e2ffd700 1 -- 172.21.15.3:0/3547864644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4012af0 con 0x7f54ec0b0520 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.860+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec0cf3a0 con 0x7f54ec0b0520 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.861+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec0cf9f0 con 0x7f54ec0b0520 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.861+0000 7f54e2ffd700 1 -- 172.21.15.3:0/3547864644 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54e401e460 con 0x7f54ec0b0520 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.862+0000 7f54e2ffd700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f54cc066f60 0x7f54cc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.862+0000 7f54e2ffd700 1 -- 172.21.15.3:0/3547864644 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f54e4084ab0 con 0x7f54ec0b0520 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.862+0000 7f54ea7fc700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f54cc066f60 0x7f54cc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.862+0000 7f54ea7fc700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f54cc066f60 0x7f54cc069410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54ec158530 tx=0x7f54d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:33.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:33.866+0000 7f54e2ffd700 1 -- 172.21.15.3:0/3547864644 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54e4051bc0 con 0x7f54ec0b0520 2023-10-06T04:53:34.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.059+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f54ec088300 con 0x7f54ec0b0520 2023-10-06T04:53:34.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.060+0000 7f54e2ffd700 1 -- 172.21.15.3:0/3547864644 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f54e40551e0 con 0x7f54ec0b0520 2023-10-06T04:53:34.062 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f54cc066f60 msgr2=0x7f54cc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f54cc066f60 0x7f54cc069410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54ec158530 tx=0x7f54d4008040 comp rx=0 tx=0).stop 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0b0520 msgr2=0x7f54ec0d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0b0520 0x7f54ec0d0af0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f54e400d890 tx=0x7f54e4008040 comp rx=0 tx=0).stop 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 shutdown_connections 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f54cc066f60 0x7f54cc069410 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec15cae0 0x7f54ec15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ec0b0520 0x7f54ec0d0af0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 --2- 172.21.15.3:0/3547864644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec0cd8e0 0x7f54ec0cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 >> 172.21.15.3:0/3547864644 conn(0x7f54ec070310 msgr2=0x7f54ec0b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 shutdown_connections 2023-10-06T04:53:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.062+0000 7f54f1776700 1 -- 172.21.15.3:0/3547864644 wait complete. 2023-10-06T04:53:34.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:53:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.196+0000 7fabd72e6700 1 Processor -- start 2023-10-06T04:53:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.196+0000 7fabd72e6700 1 -- start start 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.197+0000 7fabd72e6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabd015a450 0x7fabd015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.197+0000 7fabd72e6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a820 0x7fabd0070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.197+0000 7fabd72e6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd00ce1b0 con 0x7fabd00c5a80 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.197+0000 7fabd72e6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd015b3b0 con 0x7fabd015a450 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.197+0000 7fabd72e6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd00ce3e0 con 0x7fabd015a820 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.197+0000 7fabd5ae3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a820 0x7fabd0070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.197+0000 7fabd62e4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabd015a450 0x7fabd015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd62e4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabd015a450 0x7fabd015ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:54034/0 (socket says 172.21.15.3:54034) 2023-10-06T04:53:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd62e4700 1 -- 172.21.15.3:0/613754181 learned_addr learned my addr 172.21.15.3:0/613754181 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd52e2700 1 -- 172.21.15.3:0/613754181 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885395214 0 0) 0x7fabd00ce1b0 con 0x7fabd00c5a80 2023-10-06T04:53:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd5ae3700 1 -- 172.21.15.3:0/613754181 >> v2:172.21.15.153:3300/0 conn(0x7fabd015a450 msgr2=0x7fabd015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd5ae3700 1 --2- 172.21.15.3:0/613754181 >> v2:172.21.15.153:3300/0 conn(0x7fabd015a450 0x7fabd015ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd5ae3700 1 -- 172.21.15.3:0/613754181 >> v1:172.21.15.153:6789/0 conn(0x7fabd00c5a80 legacy=0x7fabd00cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd5ae3700 1 -- 172.21.15.3:0/613754181 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00ce6a0 con 0x7fabd015a820 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd5ae3700 1 --2- 172.21.15.3:0/613754181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a820 0x7fabd0070af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098e0 tx=0x7fabc40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81e74178e390377 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd52e2700 1 -- 172.21.15.3:0/613754181 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400b640 con 0x7fabd015a820 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd52e2700 1 -- 172.21.15.3:0/613754181 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc400b7a0 con 0x7fabd015a820 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.198+0000 7fabd52e2700 1 -- 172.21.15.3:0/613754181 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400bb40 con 0x7fabd015a820 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 -- 172.21.15.3:0/613754181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a820 msgr2=0x7fabd0070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 --2- 172.21.15.3:0/613754181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a820 0x7fabd0070af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098e0 tx=0x7fabc40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 -- 172.21.15.3:0/613754181 shutdown_connections 2023-10-06T04:53:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 --2- 172.21.15.3:0/613754181 >> v2:172.21.15.153:3300/0 conn(0x7fabd015a450 0x7fabd015ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 --2- 172.21.15.3:0/613754181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a820 0x7fabd0070af0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 -- 172.21.15.3:0/613754181 >> 172.21.15.3:0/613754181 conn(0x7fabd0089dd0 msgr2=0x7fabd00996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 -- 172.21.15.3:0/613754181 shutdown_connections 2023-10-06T04:53:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.199+0000 7fabd72e6700 1 -- 172.21.15.3:0/613754181 wait complete. 2023-10-06T04:53:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.200+0000 7fabd72e6700 1 Processor -- start 2023-10-06T04:53:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.200+0000 7fabd72e6700 1 -- start start 2023-10-06T04:53:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd72e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabd00c5a80 0x7fabd0145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd72e6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a450 0x7fabd0157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd72e6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd015a820 0x7fabd00d0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd72e6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0158ad0 con 0x7fabd00c5a80 2023-10-06T04:53:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd72e6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd00c8ad0 con 0x7fabd015a820 2023-10-06T04:53:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd72e6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd00c8d90 con 0x7fabd015a450 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabd00c5a80 0x7fabd0145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabd00c5a80 0x7fabd0145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52826/0 (socket says 172.21.15.3:52826) 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 -- 172.21.15.3:0/4001270495 learned_addr learned my addr 172.21.15.3:0/4001270495 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd5ae3700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a450 0x7fabd0157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd6ae5700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd015a820 0x7fabd00d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 -- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a450 msgr2=0x7fabd0157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a450 0x7fabd0157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 -- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd015a820 msgr2=0x7fabd00d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd015a820 0x7fabd00d0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 -- 172.21.15.3:0/4001270495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4009420 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.201+0000 7fabd62e4700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabd00c5a80 0x7fabd0145e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009f00 tx=0x7fabc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.202+0000 7fabce7fc700 1 -- 172.21.15.3:0/4001270495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000c960 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.202+0000 7fabce7fc700 1 -- 172.21.15.3:0/4001270495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc000f040 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.202+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd0157f90 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.202+0000 7fabce7fc700 1 -- 172.21.15.3:0/4001270495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000ccd0 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.202+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd0158510 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.202+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd004c380 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.203+0000 7fabce7fc700 1 -- 172.21.15.3:0/4001270495 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabc0014480 con 0x7fabd00c5a80 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.203+0000 7fabce7fc700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabbc066f30 0x7fabbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.203+0000 7fabce7fc700 1 -- 172.21.15.3:0/4001270495 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fabc0084b70 con 0x7fabd00c5a80 2023-10-06T04:53:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.204+0000 7fabd5ae3700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabbc066f30 0x7fabbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.204+0000 7fabd5ae3700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabbc066f30 0x7fabbc0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fabc40093f0 tx=0x7fabc4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.208+0000 7fabce7fc700 1 -- 172.21.15.3:0/4001270495 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabc0051c80 con 0x7fabd00c5a80 2023-10-06T04:53:34.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.399+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabd00c9070 con 0x7fabd00c5a80 2023-10-06T04:53:34.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.400+0000 7fabce7fc700 1 -- 172.21.15.3:0/4001270495 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabc00552a0 con 0x7fabd00c5a80 2023-10-06T04:53:34.401 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:53:34.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabbc066f30 msgr2=0x7fabbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabbc066f30 0x7fabbc0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fabc40093f0 tx=0x7fabc4017000 comp rx=0 tx=0).stop 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabd00c5a80 msgr2=0x7fabd0145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabd00c5a80 0x7fabd0145e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009f00 tx=0x7fabc000b040 comp rx=0 tx=0).stop 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 shutdown_connections 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fabbc066f30 0x7fabbc0693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd015a820 0x7fabd00d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabd00c5a80 0x7fabd0145e40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 --2- 172.21.15.3:0/4001270495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabd015a450 0x7fabd0157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 >> 172.21.15.3:0/4001270495 conn(0x7fabd0089dd0 msgr2=0x7fabd0070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 shutdown_connections 2023-10-06T04:53:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.402+0000 7fabd72e6700 1 -- 172.21.15.3:0/4001270495 wait complete. 2023-10-06T04:53:34.415 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:53:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.537+0000 7fc1ced0c700 1 Processor -- start 2023-10-06T04:53:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.537+0000 7fc1ced0c700 1 -- start start 2023-10-06T04:53:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.537+0000 7fc1ced0c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c8150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.537+0000 7fc1ced0c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c815a290 con 0x7fc1c815d540 2023-10-06T04:53:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1ced0c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80d0020 con 0x7fc1c8150690 2023-10-06T04:53:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1ced0c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c8159f40 con 0x7fc1c80cba50 2023-10-06T04:53:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1cd2a9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc1c815d540 0x7fc1c80af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45300/0 (socket says 172.21.15.3:45300) 2023-10-06T04:53:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1cd2a9700 1 -- 172.21.15.3:0/3686979617 learned_addr learned my addr 172.21.15.3:0/3686979617 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c7fff700 1 --2- 172.21.15.3:0/3686979617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c8150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c77fe700 1 -- 172.21.15.3:0/3686979617 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298547187 0 0) 0x7fc1c815a290 con 0x7fc1c815d540 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c77fe700 1 -- 172.21.15.3:0/3686979617 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b4003570 con 0x7fc1c815d540 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c7fff700 1 -- 172.21.15.3:0/3686979617 >> v1:172.21.15.153:6789/0 conn(0x7fc1c8150690 legacy=0x7fc1c815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c77fe700 1 -- 172.21.15.3:0/3686979617 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284541613 0 0) 0x7fc1c80d0020 con 0x7fc1c8150690 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c7fff700 1 -- 172.21.15.3:0/3686979617 >> v1:172.21.15.3:6789/0 conn(0x7fc1c815d540 legacy=0x7fc1c80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c7fff700 1 -- 172.21.15.3:0/3686979617 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c815dcb0 con 0x7fc1c80cba50 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.538+0000 7fc1c7fff700 1 --2- 172.21.15.3:0/3686979617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c8150a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009910 tx=0x7fc1b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbed9cf32a036af8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1c77fe700 1 -- 172.21.15.3:0/3686979617 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800b680 con 0x7fc1c80cba50 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1c77fe700 1 -- 172.21.15.3:0/3686979617 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b800b7e0 con 0x7fc1c80cba50 2023-10-06T04:53:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1c77fe700 1 -- 172.21.15.3:0/3686979617 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800bb60 con 0x7fc1c80cba50 2023-10-06T04:53:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1ced0c700 1 -- 172.21.15.3:0/3686979617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 msgr2=0x7fc1c8150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/3686979617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c8150a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009910 tx=0x7fc1b80091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1ced0c700 1 -- 172.21.15.3:0/3686979617 shutdown_connections 2023-10-06T04:53:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/3686979617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c8150a60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1ced0c700 1 -- 172.21.15.3:0/3686979617 >> 172.21.15.3:0/3686979617 conn(0x7fc1c8070310 msgr2=0x7fc1c8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1ced0c700 1 -- 172.21.15.3:0/3686979617 shutdown_connections 2023-10-06T04:53:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.539+0000 7fc1ced0c700 1 -- 172.21.15.3:0/3686979617 wait complete. 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.540+0000 7fc1ced0c700 1 Processor -- start 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.540+0000 7fc1ced0c700 1 -- start start 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1ced0c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c80b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1ced0c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c8150690 0x7fc1c8154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1ced0c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c815d540 0x7fc1c814f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1ced0c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80b4280 con 0x7fc1c8150690 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1ced0c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80b3670 con 0x7fc1c815d540 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1ced0c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80b3950 con 0x7fc1c80cba50 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c8150690 0x7fc1c8154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c8150690 0x7fc1c8154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52836/0 (socket says 172.21.15.3:52836) 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1c7fff700 1 -- 172.21.15.3:0/2926035251 learned_addr learned my addr 172.21.15.3:0/2926035251 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1ccaa8700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c80b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1cd2a9700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c815d540 0x7fc1c814f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1c7fff700 1 -- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 msgr2=0x7fc1c80b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1c7fff700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c80b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1c7fff700 1 -- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c815d540 msgr2=0x7fc1c814f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.541+0000 7fc1c7fff700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c815d540 0x7fc1c814f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1c7fff700 1 -- 172.21.15.3:0/2926035251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8009420 con 0x7fc1c8150690 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1cd2a9700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c815d540 0x7fc1c814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1c7fff700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c8150690 0x7fc1c8154710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80c71f0 tx=0x7fc1b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1c4ff9700 1 -- 172.21.15.3:0/2926035251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b801f070 con 0x7fc1c8150690 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1c4ff9700 1 -- 172.21.15.3:0/2926035251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b8000c90 con 0x7fc1c8150690 2023-10-06T04:53:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c80b3c30 con 0x7fc1c8150690 2023-10-06T04:53:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1c4ff9700 1 -- 172.21.15.3:0/2926035251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8025420 con 0x7fc1c8150690 2023-10-06T04:53:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.542+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c8150a60 con 0x7fc1c8150690 2023-10-06T04:53:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.543+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c80b43f0 con 0x7fc1c8150690 2023-10-06T04:53:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.543+0000 7fc1c4ff9700 1 -- 172.21.15.3:0/2926035251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1b8023070 con 0x7fc1c8150690 2023-10-06T04:53:34.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.544+0000 7fc1c4ff9700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1a8066f30 0x7fc1a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.544+0000 7fc1c4ff9700 1 -- 172.21.15.3:0/2926035251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1b8085b10 con 0x7fc1c8150690 2023-10-06T04:53:34.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.544+0000 7fc1ccaa8700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1a8066f30 0x7fc1a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.544+0000 7fc1ccaa8700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1a8066f30 0x7fc1a80693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0001560 tx=0x7fc1c00016f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.548+0000 7fc1c4ff9700 1 -- 172.21.15.3:0/2926035251 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1b8052ce0 con 0x7fc1c8150690 2023-10-06T04:53:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.743+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1c804c380 con 0x7fc1c8150690 2023-10-06T04:53:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.744+0000 7fc1c4ff9700 1 -- 172.21.15.3:0/2926035251 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1b8015020 con 0x7fc1c8150690 2023-10-06T04:53:34.745 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1a8066f30 msgr2=0x7fc1a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1a8066f30 0x7fc1a80693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0001560 tx=0x7fc1c00016f0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c8150690 msgr2=0x7fc1c8154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c8150690 0x7fc1c8154710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80c71f0 tx=0x7fc1b8017000 comp rx=0 tx=0).stop 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 shutdown_connections 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1a8066f30 0x7fc1a80693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c815d540 0x7fc1c814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1c80cba50 0x7fc1c80b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 --2- 172.21.15.3:0/2926035251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c8150690 0x7fc1c8154710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 >> 172.21.15.3:0/2926035251 conn(0x7fc1c8070310 msgr2=0x7fc1c8157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 shutdown_connections 2023-10-06T04:53:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.746+0000 7fc1ced0c700 1 -- 172.21.15.3:0/2926035251 wait complete. 2023-10-06T04:53:34.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:53:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.881+0000 7f69474af700 1 Processor -- start 2023-10-06T04:53:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.881+0000 7f69474af700 1 -- start start 2023-10-06T04:53:34.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f69474af700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69400cba50 0x7f6940150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f69474af700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f694015d540 0x7f69400af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f69474af700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694015a290 con 0x7f6940150690 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f69474af700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400d0020 con 0x7f69400cba50 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f69474af700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6940159f40 con 0x7f694015d540 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69400cba50 0x7f6940150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69400cba50 0x7f6940150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52852/0 (socket says 172.21.15.3:52852) 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 -- 172.21.15.3:0/2920048243 learned_addr learned my addr 172.21.15.3:0/2920048243 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6945a4c700 1 --2- 172.21.15.3:0/2920048243 >> v2:172.21.15.153:3300/0 conn(0x7f694015d540 0x7f69400af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 -- 172.21.15.3:0/2920048243 >> v2:172.21.15.153:3300/0 conn(0x7f694015d540 msgr2=0x7f69400af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 --2- 172.21.15.3:0/2920048243 >> v2:172.21.15.153:3300/0 conn(0x7f694015d540 0x7f69400af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 -- 172.21.15.3:0/2920048243 >> v1:172.21.15.153:6789/0 conn(0x7f6940150690 legacy=0x7f694015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 -- 172.21.15.3:0/2920048243 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694015dcb0 con 0x7f69400cba50 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6944a4a700 1 --2- 172.21.15.3:0/2920048243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69400cba50 0x7f6940150a60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f694015a200 tx=0x7f6930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ec019f710fccf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6937fff700 1 -- 172.21.15.3:0/2920048243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930009e70 con 0x7f69400cba50 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6937fff700 1 -- 172.21.15.3:0/2920048243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693000b860 con 0x7f69400cba50 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.882+0000 7f6937fff700 1 -- 172.21.15.3:0/2920048243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693000bc00 con 0x7f69400cba50 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 -- 172.21.15.3:0/2920048243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69400cba50 msgr2=0x7f6940150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 --2- 172.21.15.3:0/2920048243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69400cba50 0x7f6940150a60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f694015a200 tx=0x7f6930009250 comp rx=0 tx=0).stop 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 -- 172.21.15.3:0/2920048243 shutdown_connections 2023-10-06T04:53:34.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 --2- 172.21.15.3:0/2920048243 >> v2:172.21.15.153:3300/0 conn(0x7f694015d540 0x7f69400af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 --2- 172.21.15.3:0/2920048243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69400cba50 0x7f6940150a60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 -- 172.21.15.3:0/2920048243 >> 172.21.15.3:0/2920048243 conn(0x7f6940070310 msgr2=0x7f6940070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:34.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 -- 172.21.15.3:0/2920048243 shutdown_connections 2023-10-06T04:53:34.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.883+0000 7f69474af700 1 -- 172.21.15.3:0/2920048243 wait complete. 2023-10-06T04:53:34.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.884+0000 7f69474af700 1 Processor -- start 2023-10-06T04:53:34.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.884+0000 7f69474af700 1 -- start start 2023-10-06T04:53:34.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f69474af700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69400cba50 0x7f6940155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f69474af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6940150690 0x7f694016a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f69474af700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694015d540 0x7f694015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f69474af700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69400b41b0 con 0x7f6940150690 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f69474af700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69400b4a50 con 0x7f694015d540 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f69474af700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6940155fe0 con 0x7f69400cba50 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6940150690 0x7f694016a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6940150690 0x7f694016a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52858/0 (socket says 172.21.15.3:52858) 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 -- 172.21.15.3:0/976319927 learned_addr learned my addr 172.21.15.3:0/976319927 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f694524b700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69400cba50 0x7f6940155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6945a4c700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694015d540 0x7f694015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 -- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69400cba50 msgr2=0x7f6940155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69400cba50 0x7f6940155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 -- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694015d540 msgr2=0x7f694015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694015d540 0x7f694015c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 -- 172.21.15.3:0/976319927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300094f0 con 0x7f6940150690 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6945a4c700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694015d540 0x7f694015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.885+0000 7f6944a4a700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6940150690 0x7f694016a840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f69400c71f0 tx=0x7f6930017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.886+0000 7f69357fa700 1 -- 172.21.15.3:0/976319927 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693000b9d0 con 0x7f6940150690 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.886+0000 7f69357fa700 1 -- 172.21.15.3:0/976319927 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693000bb30 con 0x7f6940150690 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.886+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69400b4400 con 0x7f6940150690 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.886+0000 7f69357fa700 1 -- 172.21.15.3:0/976319927 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930025670 con 0x7f6940150690 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.886+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6940155bb0 con 0x7f6940150690 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.887+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69400b48a0 con 0x7f6940150690 2023-10-06T04:53:34.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.887+0000 7f69357fa700 1 -- 172.21.15.3:0/976319927 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6930023070 con 0x7f6940150690 2023-10-06T04:53:34.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.887+0000 7f69357fa700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6924066f30 0x7f69240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:34.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.888+0000 7f69357fa700 1 -- 172.21.15.3:0/976319927 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6930085b40 con 0x7f6940150690 2023-10-06T04:53:34.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.888+0000 7f694524b700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6924066f30 0x7f69240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:34.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.888+0000 7f694524b700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6924066f30 0x7f69240693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f693c003a80 tx=0x7f693c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:34.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:34.892+0000 7f69357fa700 1 -- 172.21.15.3:0/976319927 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6930052c50 con 0x7f6940150690 2023-10-06T04:53:35.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.084+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f694004c380 con 0x7f6940150690 2023-10-06T04:53:35.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.084+0000 7f69357fa700 1 -- 172.21.15.3:0/976319927 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6930015020 con 0x7f6940150690 2023-10-06T04:53:35.086 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:53:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6924066f30 msgr2=0x7f69240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6924066f30 0x7f69240693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f693c003a80 tx=0x7f693c007ec0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6940150690 msgr2=0x7f694016a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6940150690 0x7f694016a840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f69400c71f0 tx=0x7f6930017000 comp rx=0 tx=0).stop 2023-10-06T04:53:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 shutdown_connections 2023-10-06T04:53:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6924066f30 0x7f69240693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694015d540 0x7f694015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69400cba50 0x7f6940155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 --2- 172.21.15.3:0/976319927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6940150690 0x7f694016a840 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 >> 172.21.15.3:0/976319927 conn(0x7f6940070310 msgr2=0x7f69400b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 shutdown_connections 2023-10-06T04:53:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.086+0000 7f69474af700 1 -- 172.21.15.3:0/976319927 wait complete. 2023-10-06T04:53:35.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.218+0000 7f88a2bca700 1 Processor -- start 2023-10-06T04:53:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.218+0000 7f88a2bca700 1 -- start start 2023-10-06T04:53:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f88a2bca700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f889c15cb50 0x7f889c0b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f88a2bca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f889c0b0590 0x7f889c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f88a2bca700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c0b0ce0 con 0x7f889c0cd950 2023-10-06T04:53:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f88a2bca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c0cccc0 con 0x7f889c0b0590 2023-10-06T04:53:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f88a2bca700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c0cb820 con 0x7f889c15cb50 2023-10-06T04:53:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f889c0b0590 0x7f889c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f889c0b0590 0x7f889c0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52864/0 (socket says 172.21.15.3:52864) 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 -- 172.21.15.3:0/2854757734 learned_addr learned my addr 172.21.15.3:0/2854757734 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f88a0966700 1 --2- 172.21.15.3:0/2854757734 >> v2:172.21.15.153:3300/0 conn(0x7f889c15cb50 0x7f889c0b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 -- 172.21.15.3:0/2854757734 >> v2:172.21.15.153:3300/0 conn(0x7f889c15cb50 msgr2=0x7f889c0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 --2- 172.21.15.3:0/2854757734 >> v2:172.21.15.153:3300/0 conn(0x7f889c15cb50 0x7f889c0b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 -- 172.21.15.3:0/2854757734 >> v1:172.21.15.153:6789/0 conn(0x7f889c0cd950 legacy=0x7f889c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 -- 172.21.15.3:0/2854757734 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0b0a00 con 0x7f889c0b0590 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889bfff700 1 --2- 172.21.15.3:0/2854757734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c0b0590 0x7f889c0b2200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f889c0897d0 tx=0x7f888c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911460e05d7942c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.219+0000 7f889b7fe700 1 -- 172.21.15.3:0/2854757734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c009e70 con 0x7f889c0b0590 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f889b7fe700 1 -- 172.21.15.3:0/2854757734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c00b860 con 0x7f889c0b0590 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f889b7fe700 1 -- 172.21.15.3:0/2854757734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00bc00 con 0x7f889c0b0590 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 -- 172.21.15.3:0/2854757734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c0b0590 msgr2=0x7f889c0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 --2- 172.21.15.3:0/2854757734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c0b0590 0x7f889c0b2200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f889c0897d0 tx=0x7f888c009250 comp rx=0 tx=0).stop 2023-10-06T04:53:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 -- 172.21.15.3:0/2854757734 shutdown_connections 2023-10-06T04:53:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 --2- 172.21.15.3:0/2854757734 >> v2:172.21.15.153:3300/0 conn(0x7f889c15cb50 0x7f889c0b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 --2- 172.21.15.3:0/2854757734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c0b0590 0x7f889c0b2200 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 -- 172.21.15.3:0/2854757734 >> 172.21.15.3:0/2854757734 conn(0x7f889c070310 msgr2=0x7f889c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 -- 172.21.15.3:0/2854757734 shutdown_connections 2023-10-06T04:53:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.220+0000 7f88a2bca700 1 -- 172.21.15.3:0/2854757734 wait complete. 2023-10-06T04:53:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.221+0000 7f88a2bca700 1 Processor -- start 2023-10-06T04:53:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.221+0000 7f88a2bca700 1 -- start start 2023-10-06T04:53:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a2bca700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c0b0590 0x7f889c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a2bca700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f889c0cd950 0x7f889c155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a2bca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c15cb50 0x7f889c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a2bca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0c7040 con 0x7f889c15cb50 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a2bca700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c156960 con 0x7f889c0b0590 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a2bca700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0ce0b0 con 0x7f889c0cd950 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a1167700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c15cb50 0x7f889c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a1167700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c15cb50 0x7f889c155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52878/0 (socket says 172.21.15.3:52878) 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a1167700 1 -- 172.21.15.3:0/115433979 learned_addr learned my addr 172.21.15.3:0/115433979 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a0966700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c0b0590 0x7f889c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f889bfff700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f889c0cd950 0x7f889c155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a1167700 1 -- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f889c0cd950 msgr2=0x7f889c155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.222+0000 7f88a1167700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f889c0cd950 0x7f889c155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f88a1167700 1 -- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c0b0590 msgr2=0x7f889c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f88a1167700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c0b0590 0x7f889c153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f88a1167700 1 -- 172.21.15.3:0/115433979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0094f0 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f88a1167700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c15cb50 0x7f889c155a90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f888400c590 tx=0x7f8884007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f8898ff9700 1 -- 172.21.15.3:0/115433979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884011590 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f8898ff9700 1 -- 172.21.15.3:0/115433979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88840116f0 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c1560c0 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f8898ff9700 1 -- 172.21.15.3:0/115433979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884011ab0 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.223+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c156680 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.224+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c071680 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.224+0000 7f8898ff9700 1 -- 172.21.15.3:0/115433979 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f888401b3b0 con 0x7f889c15cb50 2023-10-06T04:53:35.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.225+0000 7f8898ff9700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f887c066f30 0x7f887c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.225+0000 7f8898ff9700 1 -- 172.21.15.3:0/115433979 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8884083a90 con 0x7f889c15cb50 2023-10-06T04:53:35.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.225+0000 7f88a0966700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f887c066f30 0x7f887c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.225+0000 7f88a0966700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f887c066f30 0x7f887c0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f889c0c6990 tx=0x7f889400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.229+0000 7f8898ff9700 1 -- 172.21.15.3:0/115433979 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8884050b00 con 0x7f889c15cb50 2023-10-06T04:53:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.466+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f889c0cb1d0 con 0x7f889c15cb50 2023-10-06T04:53:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.467+0000 7f8898ff9700 1 -- 172.21.15.3:0/115433979 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f8884017070 con 0x7f889c15cb50 2023-10-06T04:53:35.469 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:35.470 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f887c066f30 msgr2=0x7f887c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f887c066f30 0x7f887c0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f889c0c6990 tx=0x7f889400b040 comp rx=0 tx=0).stop 2023-10-06T04:53:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c15cb50 msgr2=0x7f889c155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c15cb50 0x7f889c155a90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f888400c590 tx=0x7f8884007ec0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 shutdown_connections 2023-10-06T04:53:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f887c066f30 0x7f887c0693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c0b0590 0x7f889c153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c15cb50 0x7f889c155a90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 --2- 172.21.15.3:0/115433979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f889c0cd950 0x7f889c155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 >> 172.21.15.3:0/115433979 conn(0x7f889c070310 msgr2=0x7f889c15b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 shutdown_connections 2023-10-06T04:53:35.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.469+0000 7f88a2bca700 1 -- 172.21.15.3:0/115433979 wait complete. 2023-10-06T04:53:35.474 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:35.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:35.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.602+0000 7f93cc5ce700 1 Processor -- start 2023-10-06T04:53:35.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.602+0000 7f93cc5ce700 1 -- start start 2023-10-06T04:53:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.603+0000 7f93cc5ce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f93c415c8f0 0x7f93c415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.603+0000 7f93cc5ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40cf6c0 con 0x7f93c415ccc0 2023-10-06T04:53:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.603+0000 7f93cc5ce700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40cfb50 con 0x7f93c40d0870 2023-10-06T04:53:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.603+0000 7f93cc5ce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c415e050 con 0x7f93c415c8f0 2023-10-06T04:53:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.603+0000 7f93c9b69700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f93c415ccc0 0x7f93c40d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45310/0 (socket says 172.21.15.3:45310) 2023-10-06T04:53:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.603+0000 7f93c9b69700 1 -- 172.21.15.3:0/3293531677 learned_addr learned my addr 172.21.15.3:0/3293531677 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:35.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680379538 0 0) 0x7f93c40cf6c0 con 0x7f93c415ccc0 2023-10-06T04:53:35.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ac003570 con 0x7f93c415ccc0 2023-10-06T04:53:35.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93ca36a700 1 --2- 172.21.15.3:0/3293531677 >> v2:172.21.15.153:3300/0 conn(0x7f93c415c8f0 0x7f93c415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2715151894 0 0) 0x7f93ac003570 con 0x7f93c415ccc0 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 >> v2:172.21.15.153:3300/0 conn(0x7f93c415c8f0 msgr2=0x7f93c415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 --2- 172.21.15.3:0/3293531677 >> v2:172.21.15.153:3300/0 conn(0x7f93c415c8f0 0x7f93c415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 >> v1:172.21.15.153:6789/0 conn(0x7f93c40d0870 legacy=0x7f93c40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40ceab0 con 0x7f93c415ccc0 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f93b8001470 con 0x7f93c415ccc0 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 >> v1:172.21.15.3:6789/0 conn(0x7f93c415ccc0 legacy=0x7f93c40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93ac004810 0x7f93ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93ac0051c0 0x7f93ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c415e050 con 0x7f93ac003a60 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac00c0d0 con 0x7f93ac0051c0 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac00c290 con 0x7f93ac004810 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93ca36a700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93cab6b700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93ac0051c0 0x7f93ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93c9b69700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93ac004810 0x7f93ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93ca36a700 1 -- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93ac004810 msgr2=0x7f93ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93ca36a700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93ac004810 0x7f93ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93ca36a700 1 -- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93ac0051c0 msgr2=0x7f93ac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.604+0000 7f93ca36a700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93ac0051c0 0x7f93ac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93ca36a700 1 -- 172.21.15.3:0/3293531677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac003570 con 0x7f93ac003a60 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93c9b69700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93ac004810 0x7f93ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cab6b700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93ac0051c0 0x7f93ac00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cab6b700 1 -- 172.21.15.3:0/3293531677 reap_dead start 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93ca36a700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93ac003e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009c00 tx=0x7f93b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac962f3ebdc32e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b400e710 con 0x7f93ac003a60 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b400e870 con 0x7f93ac003a60 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93c9368700 1 -- 172.21.15.3:0/3293531677 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b400ec30 con 0x7f93ac003a60 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cc5ce700 1 -- 172.21.15.3:0/3293531677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 msgr2=0x7f93ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93ac003e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009c00 tx=0x7f93b40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cc5ce700 1 -- 172.21.15.3:0/3293531677 shutdown_connections 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/3293531677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93ac003e70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cc5ce700 1 -- 172.21.15.3:0/3293531677 >> 172.21.15.3:0/3293531677 conn(0x7f93c4070310 msgr2=0x7f93c4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cc5ce700 1 -- 172.21.15.3:0/3293531677 shutdown_connections 2023-10-06T04:53:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.605+0000 7f93cc5ce700 1 -- 172.21.15.3:0/3293531677 wait complete. 2023-10-06T04:53:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 Processor -- start 2023-10-06T04:53:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 -- start start 2023-10-06T04:53:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93c41509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93c414e3d0 0x7f93c4169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93c416a1a0 0x7f93c416a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c416ab80 con 0x7f93ac003a60 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c414f490 con 0x7f93c414e3d0 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93cc5ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c414ebf0 con 0x7f93c416a1a0 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93ca36a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93c41509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93ca36a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93c41509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52898/0 (socket says 172.21.15.3:52898) 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.607+0000 7f93ca36a700 1 -- 172.21.15.3:0/1390441048 learned_addr learned my addr 172.21.15.3:0/1390441048 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93cab6b700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93c416a1a0 0x7f93c416a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93c9b69700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93c414e3d0 0x7f93c4169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93ca36a700 1 -- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93c416a1a0 msgr2=0x7f93c416a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93ca36a700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93c416a1a0 0x7f93c416a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93ca36a700 1 -- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93c414e3d0 msgr2=0x7f93c4169c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93ca36a700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93c414e3d0 0x7f93c4169c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93ca36a700 1 -- 172.21.15.3:0/1390441048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c414eed0 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93ca36a700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93c41509b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c0a0 tx=0x7f93b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93c27fc700 1 -- 172.21.15.3:0/1390441048 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4022070 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93c27fc700 1 -- 172.21.15.3:0/1390441048 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b400e9e0 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c414f1b0 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93c27fc700 1 -- 172.21.15.3:0/1390441048 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b401c450 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.608+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c415b7b0 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.609+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c4071680 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.609+0000 7f93c27fc700 1 -- 172.21.15.3:0/1390441048 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93b401c840 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.610+0000 7f93c27fc700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f93b0066f30 0x7f93b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.610+0000 7f93c27fc700 1 -- 172.21.15.3:0/1390441048 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f93b4085d00 con 0x7f93ac003a60 2023-10-06T04:53:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.610+0000 7f93c9b69700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f93b0066f30 0x7f93b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.610+0000 7f93c9b69700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f93b0066f30 0x7f93b00693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f93b8000e10 tx=0x7f93b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.614+0000 7f93c27fc700 1 -- 172.21.15.3:0/1390441048 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b4052ec0 con 0x7f93ac003a60 2023-10-06T04:53:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.807+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93c40b35a0 con 0x7f93ac003a60 2023-10-06T04:53:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.808+0000 7f93c27fc700 1 -- 172.21.15.3:0/1390441048 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f93b402b090 con 0x7f93ac003a60 2023-10-06T04:53:35.810 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:35.810 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:09.096476+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:05.231194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:06.473725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f93b0066f30 msgr2=0x7f93b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f93b0066f30 0x7f93b00693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f93b8000e10 tx=0x7f93b8009040 comp rx=0 tx=0).stop 2023-10-06T04:53:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 msgr2=0x7f93c41509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93c41509b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c0a0 tx=0x7f93b401a000 comp rx=0 tx=0).stop 2023-10-06T04:53:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 shutdown_connections 2023-10-06T04:53:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f93b0066f30 0x7f93b00693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93c416a1a0 0x7f93c416a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93ac003a60 0x7f93c41509b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 --2- 172.21.15.3:0/1390441048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93c414e3d0 0x7f93c4169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 >> 172.21.15.3:0/1390441048 conn(0x7f93c4070310 msgr2=0x7f93c40b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 shutdown_connections 2023-10-06T04:53:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.810+0000 7f93cc5ce700 1 -- 172.21.15.3:0/1390441048 wait complete. 2023-10-06T04:53:35.824 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:53:35.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.942+0000 7f96d554b700 1 Processor -- start 2023-10-06T04:53:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.942+0000 7f96d554b700 1 -- start start 2023-10-06T04:53:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96d554b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f96d015cdc0 0x7f96d00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96d554b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d006ea10 con 0x7f96d006daa0 2023-10-06T04:53:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96d554b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00b20c0 con 0x7f96d006d6d0 2023-10-06T04:53:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96d554b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d006e730 con 0x7f96d015cdc0 2023-10-06T04:53:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96cf7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f96d006daa0 0x7f96d006dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45322/0 (socket says 172.21.15.3:45322) 2023-10-06T04:53:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96cf7fe700 1 -- 172.21.15.3:0/2828723463 learned_addr learned my addr 172.21.15.3:0/2828723463 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96ce7fc700 1 --2- 172.21.15.3:0/2828723463 >> v2:172.21.15.153:3300/0 conn(0x7f96d015cdc0 0x7f96d00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96cdffb700 1 -- 172.21.15.3:0/2828723463 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616747871 0 0) 0x7f96d006ea10 con 0x7f96d006daa0 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96cdffb700 1 -- 172.21.15.3:0/2828723463 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96bc003570 con 0x7f96d006daa0 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96ce7fc700 1 -- 172.21.15.3:0/2828723463 >> v1:172.21.15.153:6789/0 conn(0x7f96d006d6d0 legacy=0x7f96d015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96ce7fc700 1 -- 172.21.15.3:0/2828723463 >> v1:172.21.15.3:6789/0 conn(0x7f96d006daa0 legacy=0x7f96d006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.943+0000 7f96ce7fc700 1 -- 172.21.15.3:0/2828723463 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00b1a70 con 0x7f96d015cdc0 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96ce7fc700 1 --2- 172.21.15.3:0/2828723463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d00b05f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96b80098d0 tx=0x7f96b8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c457bf4ba859911 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96cdffb700 1 -- 172.21.15.3:0/2828723463 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8005c10 con 0x7f96d015cdc0 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96cdffb700 1 -- 172.21.15.3:0/2828723463 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b8005d70 con 0x7f96d015cdc0 2023-10-06T04:53:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96cdffb700 1 -- 172.21.15.3:0/2828723463 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8006280 con 0x7f96d015cdc0 2023-10-06T04:53:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96d554b700 1 -- 172.21.15.3:0/2828723463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 msgr2=0x7f96d00b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96d554b700 1 --2- 172.21.15.3:0/2828723463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d00b05f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96b80098d0 tx=0x7f96b8009120 comp rx=0 tx=0).stop 2023-10-06T04:53:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96d554b700 1 -- 172.21.15.3:0/2828723463 shutdown_connections 2023-10-06T04:53:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96d554b700 1 --2- 172.21.15.3:0/2828723463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d00b05f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.944+0000 7f96d554b700 1 -- 172.21.15.3:0/2828723463 >> 172.21.15.3:0/2828723463 conn(0x7f96d006cb50 msgr2=0x7f96d00ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.945+0000 7f96d554b700 1 -- 172.21.15.3:0/2828723463 shutdown_connections 2023-10-06T04:53:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.945+0000 7f96d554b700 1 -- 172.21.15.3:0/2828723463 wait complete. 2023-10-06T04:53:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.946+0000 7f96d554b700 1 Processor -- start 2023-10-06T04:53:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.946+0000 7f96d554b700 1 -- start start 2023-10-06T04:53:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.946+0000 7f96d554b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96d006d6d0 0x7f96d0144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.946+0000 7f96d554b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96d006daa0 0x7f96d0143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.946+0000 7f96d554b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d016ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96d554b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0144300 con 0x7f96d006d6d0 2023-10-06T04:53:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96d554b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0099260 con 0x7f96d015cdc0 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96d554b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0099540 con 0x7f96d006daa0 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96d006d6d0 0x7f96d0144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96d006d6d0 0x7f96d0144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52908/0 (socket says 172.21.15.3:52908) 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 -- 172.21.15.3:0/120524979 learned_addr learned my addr 172.21.15.3:0/120524979 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ce7fc700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96d006daa0 0x7f96d0143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96cf7fe700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d016ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 -- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96d006daa0 msgr2=0x7f96d0143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96d006daa0 0x7f96d0143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 -- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 msgr2=0x7f96d016ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d016ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 -- 172.21.15.3:0/120524979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80093c0 con 0x7f96d006d6d0 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ce7fc700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96d006daa0 0x7f96d0143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96cf7fe700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d016ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96ceffd700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96d006d6d0 0x7f96d0144540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c7c0 tx=0x7f96c8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96c6ffd700 1 -- 172.21.15.3:0/120524979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8011590 con 0x7f96d006d6d0 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96c6ffd700 1 -- 172.21.15.3:0/120524979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c8011710 con 0x7f96d006d6d0 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d0099820 con 0x7f96d006d6d0 2023-10-06T04:53:35.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96c6ffd700 1 -- 172.21.15.3:0/120524979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8011ad0 con 0x7f96d006d6d0 2023-10-06T04:53:35.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.947+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d0099de0 con 0x7f96d006d6d0 2023-10-06T04:53:35.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.948+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d004c380 con 0x7f96d006d6d0 2023-10-06T04:53:35.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.949+0000 7f96c6ffd700 1 -- 172.21.15.3:0/120524979 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96c801b3b0 con 0x7f96d006d6d0 2023-10-06T04:53:35.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.949+0000 7f96c6ffd700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b0066f30 0x7f96b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:35.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.949+0000 7f96c6ffd700 1 -- 172.21.15.3:0/120524979 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f96c8083b90 con 0x7f96d006d6d0 2023-10-06T04:53:35.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.949+0000 7f96ce7fc700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b0066f30 0x7f96b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:35.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.950+0000 7f96ce7fc700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b0066f30 0x7f96b00693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f96d015d240 tx=0x7f96b8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:35.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:35.953+0000 7f96c6ffd700 1 -- 172.21.15.3:0/120524979 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96c8050d50 con 0x7f96d006d6d0 2023-10-06T04:53:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.194+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96d00ad550 con 0x7f96d006d6d0 2023-10-06T04:53:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.195+0000 7f96c6ffd700 1 -- 172.21.15.3:0/120524979 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f96c8017070 con 0x7f96d006d6d0 2023-10-06T04:53:36.197 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:36.197 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b0066f30 msgr2=0x7f96b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b0066f30 0x7f96b00693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f96d015d240 tx=0x7f96b8016000 comp rx=0 tx=0).stop 2023-10-06T04:53:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96d006d6d0 msgr2=0x7f96d0144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96d006d6d0 0x7f96d0144540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c7c0 tx=0x7f96c8007f70 comp rx=0 tx=0).stop 2023-10-06T04:53:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 shutdown_connections 2023-10-06T04:53:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b0066f30 0x7f96b00693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d015cdc0 0x7f96d016ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96d006d6d0 0x7f96d0144540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 --2- 172.21.15.3:0/120524979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96d006daa0 0x7f96d0143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 >> 172.21.15.3:0/120524979 conn(0x7f96d006cb50 msgr2=0x7f96d00b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 shutdown_connections 2023-10-06T04:53:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.197+0000 7f96d554b700 1 -- 172.21.15.3:0/120524979 wait complete. 2023-10-06T04:53:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:36.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:36.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.328+0000 7f60f6895700 1 Processor -- start 2023-10-06T04:53:36.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.328+0000 7f60f6895700 1 -- start start 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.329+0000 7f60f6895700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f0157700 0x7f60f00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.329+0000 7f60f6895700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f00b2e50 con 0x7f60f00b1b10 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.329+0000 7f60f6895700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f0158290 con 0x7f60f0157ad0 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.329+0000 7f60f6895700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00b1fc0 con 0x7f60f0157700 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.329+0000 7f60f4e32700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f60f00b1b10 0x7f60f0067b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45334/0 (socket says 172.21.15.3:45334) 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.329+0000 7f60f4e32700 1 -- 172.21.15.3:0/1860304972 learned_addr learned my addr 172.21.15.3:0/1860304972 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60effff700 1 --2- 172.21.15.3:0/1860304972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f0157700 0x7f60f00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912498937 0 0) 0x7f60f00b2e50 con 0x7f60f00b1b10 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60dc003570 con 0x7f60f00b1b10 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528857837 0 0) 0x7f60f0158290 con 0x7f60f0157ad0 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60f00b2e50 con 0x7f60f0157ad0 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60effff700 1 -- 172.21.15.3:0/1860304972 >> v1:172.21.15.153:6789/0 conn(0x7f60f0157ad0 legacy=0x7f60f00b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60effff700 1 -- 172.21.15.3:0/1860304972 >> v1:172.21.15.3:6789/0 conn(0x7f60f00b1b10 legacy=0x7f60f0067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2619473261 0 0) 0x7f60dc003570 con 0x7f60f00b1b10 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60effff700 1 -- 172.21.15.3:0/1860304972 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00b22a0 con 0x7f60f0157700 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60effff700 1 --2- 172.21.15.3:0/1860304972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f0157700 0x7f60f00cc270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009910 tx=0x7f60d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0237bd592bdfcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e80017f0 con 0x7f60f0157700 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d8011420 con 0x7f60f0157700 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.330+0000 7f60eeffd700 1 -- 172.21.15.3:0/1860304972 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d80117c0 con 0x7f60f0157700 2023-10-06T04:53:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.331+0000 7f60f6895700 1 -- 172.21.15.3:0/1860304972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f0157700 msgr2=0x7f60f00cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.331+0000 7f60f6895700 1 --2- 172.21.15.3:0/1860304972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f0157700 0x7f60f00cc270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009910 tx=0x7f60d80091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.331+0000 7f60f6895700 1 -- 172.21.15.3:0/1860304972 shutdown_connections 2023-10-06T04:53:36.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.331+0000 7f60f6895700 1 --2- 172.21.15.3:0/1860304972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f0157700 0x7f60f00cc270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.331+0000 7f60f6895700 1 -- 172.21.15.3:0/1860304972 >> 172.21.15.3:0/1860304972 conn(0x7f60f00ce2c0 msgr2=0x7f60f00ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:36.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.331+0000 7f60f6895700 1 -- 172.21.15.3:0/1860304972 shutdown_connections 2023-10-06T04:53:36.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.331+0000 7f60f6895700 1 -- 172.21.15.3:0/1860304972 wait complete. 2023-10-06T04:53:36.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.332+0000 7f60f6895700 1 Processor -- start 2023-10-06T04:53:36.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.332+0000 7f60f6895700 1 -- start start 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f6895700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f00b1b10 0x7f60f01531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f6895700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f0157700 0x7f60f00af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f6895700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f0157ad0 0x7f60f00af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f6895700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f006d480 con 0x7f60f0157ad0 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f6895700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f006e2e0 con 0x7f60f0157700 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f6895700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0153700 con 0x7f60f00b1b10 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f0157ad0 0x7f60f00af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f0157ad0 0x7f60f00af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52912/0 (socket says 172.21.15.3:52912) 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 -- 172.21.15.3:0/1752611245 learned_addr learned my addr 172.21.15.3:0/1752611245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60ef7fe700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f0157700 0x7f60f00af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60effff700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f00b1b10 0x7f60f01531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 -- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f00b1b10 msgr2=0x7f60f01531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f00b1b10 0x7f60f01531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 -- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f0157700 msgr2=0x7f60f00af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f0157700 0x7f60f00af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.333+0000 7f60f4e32700 1 -- 172.21.15.3:0/1752611245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8009420 con 0x7f60f0157ad0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.334+0000 7f60f4e32700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f0157ad0 0x7f60f00af740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f60e800ccb0 tx=0x7f60e8007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.334+0000 7f60e7fff700 1 -- 172.21.15.3:0/1752611245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e800ce60 con 0x7f60f0157ad0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.334+0000 7f60e7fff700 1 -- 172.21.15.3:0/1752611245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e8004dc0 con 0x7f60f0157ad0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.334+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f006d6d0 con 0x7f60f0157ad0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.334+0000 7f60e7fff700 1 -- 172.21.15.3:0/1752611245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e800ab10 con 0x7f60f0157ad0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.334+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f006dc90 con 0x7f60f0157ad0 2023-10-06T04:53:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.335+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f0158160 con 0x7f60f0157ad0 2023-10-06T04:53:36.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.335+0000 7f60e7fff700 1 -- 172.21.15.3:0/1752611245 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60e8007150 con 0x7f60f0157ad0 2023-10-06T04:53:36.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.335+0000 7f60e7fff700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60d0066f30 0x7f60d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.336+0000 7f60e7fff700 1 -- 172.21.15.3:0/1752611245 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f60e8084170 con 0x7f60f0157ad0 2023-10-06T04:53:36.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.336+0000 7f60effff700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60d0066f30 0x7f60d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.336+0000 7f60effff700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60d0066f30 0x7f60d00693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f60d80030c0 tx=0x7f60d8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.340+0000 7f60e7fff700 1 -- 172.21.15.3:0/1752611245 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60e8051300 con 0x7f60f0157ad0 2023-10-06T04:53:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.577+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60f00644c0 con 0x7f60f0157ad0 2023-10-06T04:53:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.578+0000 7f60e7fff700 1 -- 172.21.15.3:0/1752611245 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f60e8054920 con 0x7f60f0157ad0 2023-10-06T04:53:36.579 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:36.580 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:07.496347+0000","modified":"2023-10-06T04:53:11.105282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4962":{"gid":4962,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6836/501211712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":501211712},{"type":"v1","addr":"172.21.15.153:6836","nonce":501211712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:53:36.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60d0066f30 msgr2=0x7f60d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60d0066f30 0x7f60d00693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f60d80030c0 tx=0x7f60d8016000 comp rx=0 tx=0).stop 2023-10-06T04:53:36.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f0157ad0 msgr2=0x7f60f00af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f0157ad0 0x7f60f00af740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f60e800ccb0 tx=0x7f60e8007d20 comp rx=0 tx=0).stop 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 shutdown_connections 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f60d0066f30 0x7f60d00693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f0157ad0 0x7f60f00af740 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60f00b1b10 0x7f60f01531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 --2- 172.21.15.3:0/1752611245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f0157700 0x7f60f00af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 >> 172.21.15.3:0/1752611245 conn(0x7f60f00ce2c0 msgr2=0x7f60f01540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 shutdown_connections 2023-10-06T04:53:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.580+0000 7f60f6895700 1 -- 172.21.15.3:0/1752611245 wait complete. 2023-10-06T04:53:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 15 2023-10-06T04:53:36.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:36.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.710+0000 7f022506e700 1 Processor -- start 2023-10-06T04:53:36.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.711+0000 7f022506e700 1 -- start start 2023-10-06T04:53:36.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.711+0000 7f022506e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f022015c8f0 0x7f022015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.711+0000 7f022506e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f022015ccc0 0x7f02200d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.711+0000 7f022506e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200cf6c0 con 0x7f02200d06a0 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.711+0000 7f022506e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200cfb50 con 0x7f022015c8f0 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.711+0000 7f022506e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200d0b20 con 0x7f022015ccc0 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f022015c8f0 0x7f022015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021f59e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f02200d06a0 0x7f022006d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45340/0 (socket says 172.21.15.3:45340) 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021f59e700 1 -- 172.21.15.3:0/2352811258 learned_addr learned my addr 172.21.15.3:0/2352811258 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021e59c700 1 --2- 172.21.15.3:0/2352811258 >> v2:172.21.15.153:3300/0 conn(0x7f022015ccc0 0x7f02200d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021ed9d700 1 -- 172.21.15.3:0/2352811258 >> v2:172.21.15.153:3300/0 conn(0x7f022015ccc0 msgr2=0x7f02200d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021ed9d700 1 --2- 172.21.15.3:0/2352811258 >> v2:172.21.15.153:3300/0 conn(0x7f022015ccc0 0x7f02200d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021ed9d700 1 -- 172.21.15.3:0/2352811258 >> v1:172.21.15.3:6789/0 conn(0x7f02200d06a0 legacy=0x7f022006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021dd9b700 1 -- 172.21.15.3:0/2352811258 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432207420 0 0) 0x7f02200cf6c0 con 0x7f02200d06a0 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021ed9d700 1 -- 172.21.15.3:0/2352811258 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022015e050 con 0x7f022015c8f0 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021ed9d700 1 --2- 172.21.15.3:0/2352811258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f022015c8f0 0x7f022015db10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f02180098e0 tx=0x7f02180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b1c6bec0183993 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021dd9b700 1 -- 172.21.15.3:0/2352811258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218009e80 con 0x7f022015c8f0 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021dd9b700 1 -- 172.21.15.3:0/2352811258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021800b820 con 0x7f022015c8f0 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.712+0000 7f021dd9b700 1 -- 172.21.15.3:0/2352811258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800bba0 con 0x7f022015c8f0 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 -- 172.21.15.3:0/2352811258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f022015c8f0 msgr2=0x7f022015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 --2- 172.21.15.3:0/2352811258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f022015c8f0 0x7f022015db10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f02180098e0 tx=0x7f02180091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 -- 172.21.15.3:0/2352811258 shutdown_connections 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 --2- 172.21.15.3:0/2352811258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f022015c8f0 0x7f022015db10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 --2- 172.21.15.3:0/2352811258 >> v2:172.21.15.153:3300/0 conn(0x7f022015ccc0 0x7f02200d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 -- 172.21.15.3:0/2352811258 >> 172.21.15.3:0/2352811258 conn(0x7f0220070310 msgr2=0x7f0220070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 -- 172.21.15.3:0/2352811258 shutdown_connections 2023-10-06T04:53:36.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.713+0000 7f022506e700 1 -- 172.21.15.3:0/2352811258 wait complete. 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.714+0000 7f022506e700 1 Processor -- start 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.714+0000 7f022506e700 1 -- start start 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f022506e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02200d06a0 0x7f0220169a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f022506e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f022015c8f0 0x7f022016c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f022506e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f022015ccc0 0x7f0220158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f022506e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022015b8a0 con 0x7f02200d06a0 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f022506e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022015bd60 con 0x7f022015ccc0 2023-10-06T04:53:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f022506e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02201588a0 con 0x7f022015c8f0 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02200d06a0 0x7f0220169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02200d06a0 0x7f0220169a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52926/0 (socket says 172.21.15.3:52926) 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 -- 172.21.15.3:0/234372571 learned_addr learned my addr 172.21.15.3:0/234372571 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 -- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f022015c8f0 msgr2=0x7f022016c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f022015c8f0 0x7f022016c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 -- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f022015ccc0 msgr2=0x7f0220158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f022015ccc0 0x7f0220158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 -- 172.21.15.3:0/234372571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218009420 con 0x7f02200d06a0 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.715+0000 7f021ed9d700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02200d06a0 0x7f0220169a90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f022006eaa0 tx=0x7f0218017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.716+0000 7f0216ffd700 1 -- 172.21.15.3:0/234372571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218011420 con 0x7f02200d06a0 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.716+0000 7f0216ffd700 1 -- 172.21.15.3:0/234372571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02180115a0 con 0x7f02200d06a0 2023-10-06T04:53:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.716+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022015ba80 con 0x7f02200d06a0 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.716+0000 7f0216ffd700 1 -- 172.21.15.3:0/234372571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218011960 con 0x7f02200d06a0 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.716+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0220158530 con 0x7f02200d06a0 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.717+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220071680 con 0x7f02200d06a0 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.717+0000 7f0216ffd700 1 -- 172.21.15.3:0/234372571 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0218011cd0 con 0x7f02200d06a0 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.717+0000 7f0216ffd700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0200066f30 0x7f02000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.717+0000 7f0216ffd700 1 -- 172.21.15.3:0/234372571 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0218085be0 con 0x7f02200d06a0 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.718+0000 7f021e59c700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0200066f30 0x7f02000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.718+0000 7f021e59c700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0200066f30 0x7f02000693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02200b1420 tx=0x7f0208008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.722+0000 7f0216ffd700 1 -- 172.21.15.3:0/234372571 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0218052cf0 con 0x7f02200d06a0 2023-10-06T04:53:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.914+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02200b35a0 con 0x7f02200d06a0 2023-10-06T04:53:36.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.915+0000 7f0216ffd700 1 -- 172.21.15.3:0/234372571 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f0218015090 con 0x7f02200d06a0 2023-10-06T04:53:36.917 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:36.917 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:09.096476+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:05.231194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:06.473725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0200066f30 msgr2=0x7f02000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0200066f30 0x7f02000693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02200b1420 tx=0x7f0208008040 comp rx=0 tx=0).stop 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02200d06a0 msgr2=0x7f0220169a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02200d06a0 0x7f0220169a90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f022006eaa0 tx=0x7f0218017000 comp rx=0 tx=0).stop 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 shutdown_connections 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0200066f30 0x7f02000693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f022015ccc0 0x7f0220158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02200d06a0 0x7f0220169a90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 --2- 172.21.15.3:0/234372571 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f022015c8f0 0x7f022016c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 >> 172.21.15.3:0/234372571 conn(0x7f0220070310 msgr2=0x7f0220155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 shutdown_connections 2023-10-06T04:53:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:36.917+0000 7f022506e700 1 -- 172.21.15.3:0/234372571 wait complete. 2023-10-06T04:53:36.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:53:37.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.050+0000 7fcdc4081700 1 Processor -- start 2023-10-06T04:53:37.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.050+0000 7fcdc4081700 1 -- start start 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc4081700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 0x7fcdbc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc4081700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdbc0d0870 0x7fcdbc0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc4081700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdbc0cf6c0 con 0x7fcdbc15ccc0 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc4081700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc0cfb50 con 0x7fcdbc0d0870 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc4081700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc15e050 con 0x7fcdbc15c8f0 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc261e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdbc0d0870 0x7fcdbc0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc161c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcdbc15ccc0 0x7fcdbc0d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45356/0 (socket says 172.21.15.3:45356) 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.051+0000 7fcdc161c700 1 -- 172.21.15.3:0/2579315794 learned_addr learned my addr 172.21.15.3:0/2579315794 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc1e1d700 1 --2- 172.21.15.3:0/2579315794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 0x7fcdbc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc261e700 1 -- 172.21.15.3:0/2579315794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 msgr2=0x7fcdbc15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc261e700 1 --2- 172.21.15.3:0/2579315794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 0x7fcdbc15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc261e700 1 -- 172.21.15.3:0/2579315794 >> v1:172.21.15.3:6789/0 conn(0x7fcdbc15ccc0 legacy=0x7fcdbc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc261e700 1 -- 172.21.15.3:0/2579315794 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc0ceab0 con 0x7fcdbc0d0870 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc0e1b700 1 -- 172.21.15.3:0/2579315794 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337738143 0 0) 0x7fcdbc0cf6c0 con 0x7fcdbc15ccc0 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc261e700 1 --2- 172.21.15.3:0/2579315794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0b3ba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40098e0 tx=0x7fcdb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4900c210fd881d65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc0e1b700 1 -- 172.21.15.3:0/2579315794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4009e80 con 0x7fcdbc0d0870 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc0e1b700 1 -- 172.21.15.3:0/2579315794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb400b860 con 0x7fcdbc0d0870 2023-10-06T04:53:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.052+0000 7fcdc0e1b700 1 -- 172.21.15.3:0/2579315794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400bbe0 con 0x7fcdbc0d0870 2023-10-06T04:53:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 -- 172.21.15.3:0/2579315794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 msgr2=0x7fcdbc0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2579315794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0b3ba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40098e0 tx=0x7fcdb40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 -- 172.21.15.3:0/2579315794 shutdown_connections 2023-10-06T04:53:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2579315794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0b3ba0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2579315794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 0x7fcdbc15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 -- 172.21.15.3:0/2579315794 >> 172.21.15.3:0/2579315794 conn(0x7fcdbc070310 msgr2=0x7fcdbc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 -- 172.21.15.3:0/2579315794 shutdown_connections 2023-10-06T04:53:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.053+0000 7fcdc4081700 1 -- 172.21.15.3:0/2579315794 wait complete. 2023-10-06T04:53:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.054+0000 7fcdc4081700 1 Processor -- start 2023-10-06T04:53:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.054+0000 7fcdc4081700 1 -- start start 2023-10-06T04:53:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.054+0000 7fcdc4081700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.054+0000 7fcdc4081700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 0x7fcdbc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc4081700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdbc15ccc0 0x7fcdbc153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc4081700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc154e20 con 0x7fcdbc0d0870 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc4081700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc15a2b0 con 0x7fcdbc15ccc0 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc4081700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc0c5920 con 0x7fcdbc15c8f0 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52954/0 (socket says 172.21.15.3:52954) 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 -- 172.21.15.3:0/2650055214 learned_addr learned my addr 172.21.15.3:0/2650055214 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 -- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 msgr2=0x7fcdbc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 0x7fcdbc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 -- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdbc15ccc0 msgr2=0x7fcdbc153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdbc15ccc0 0x7fcdbc153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 -- 172.21.15.3:0/2650055214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4009420 con 0x7fcdbc0d0870 2023-10-06T04:53:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc1e1d700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0c5220 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e190 tx=0x7fcdac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdb9ffb700 1 -- 172.21.15.3:0/2650055214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ee30 con 0x7fcdbc0d0870 2023-10-06T04:53:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdb9ffb700 1 -- 172.21.15.3:0/2650055214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac014420 con 0x7fcdbc0d0870 2023-10-06T04:53:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc159980 con 0x7fcdbc0d0870 2023-10-06T04:53:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdb9ffb700 1 -- 172.21.15.3:0/2650055214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0147e0 con 0x7fcdbc0d0870 2023-10-06T04:53:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.055+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc159f40 con 0x7fcdbc0d0870 2023-10-06T04:53:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.056+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc071680 con 0x7fcdbc0d0870 2023-10-06T04:53:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.057+0000 7fcdb9ffb700 1 -- 172.21.15.3:0/2650055214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdac014b90 con 0x7fcdbc0d0870 2023-10-06T04:53:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.057+0000 7fcdb9ffb700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcda8066f00 0x7fcda80693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.057+0000 7fcdb9ffb700 1 -- 172.21.15.3:0/2650055214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcdac084eb0 con 0x7fcdbc0d0870 2023-10-06T04:53:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.058+0000 7fcdc161c700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcda8066f00 0x7fcda80693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.058+0000 7fcdc161c700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcda8066f00 0x7fcda80693b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00011e0 tx=0x7fcdb00079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.062+0000 7fcdb9ffb700 1 -- 172.21.15.3:0/2650055214 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdac051fc0 con 0x7fcdbc0d0870 2023-10-06T04:53:37.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.299+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcdbc15a640 con 0x7fcdbc0d0870 2023-10-06T04:53:37.301 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T04:53:37.299+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:53:37.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.306+0000 7fcdb9ffb700 1 -- 172.21.15.3:0/2650055214 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcdac009a60 con 0x7fcdbc0d0870 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcda8066f00 msgr2=0x7fcda80693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcda8066f00 0x7fcda80693b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00011e0 tx=0x7fcdb00079d0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 msgr2=0x7fcdbc0c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0c5220 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e190 tx=0x7fcdac009920 comp rx=0 tx=0).stop 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 shutdown_connections 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcda8066f00 0x7fcda80693b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdbc15ccc0 0x7fcdbc153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0d0870 0x7fcdbc0c5220 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 --2- 172.21.15.3:0/2650055214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcdbc15c8f0 0x7fcdbc15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 >> 172.21.15.3:0/2650055214 conn(0x7fcdbc070310 msgr2=0x7fcdbc0cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:37.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 shutdown_connections 2023-10-06T04:53:37.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.308+0000 7fcdc4081700 1 -- 172.21.15.3:0/2650055214 wait complete. 2023-10-06T04:53:37.311 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:53:37.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:53:37.341 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:53:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.441+0000 7f7af0bdd700 1 Processor -- start 2023-10-06T04:53:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.441+0000 7f7af0bdd700 1 -- start start 2023-10-06T04:53:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7af0bdd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7aec140e70 0x7f7aec0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7af0bdd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7af0bdd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec069f10 con 0x7f7aec0cd440 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7af0bdd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec0cca70 con 0x7f7aec140e70 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7af0bdd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0cc140 con 0x7f7aec0b16c0 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7ae9d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7aec0cd440 0x7f7aec0b0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45372/0 (socket says 172.21.15.3:45372) 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7ae9d9b700 1 -- 172.21.15.3:0/2426486236 learned_addr learned my addr 172.21.15.3:0/2426486236 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.442+0000 7f7aead9d700 1 --2- 172.21.15.3:0/2426486236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7aea59c700 1 --2- 172.21.15.3:0/2426486236 >> v2:172.21.15.153:3300/0 conn(0x7f7aec140e70 0x7f7aec0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7ae959a700 1 -- 172.21.15.3:0/2426486236 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017504232 0 0) 0x7f7aec069f10 con 0x7f7aec0cd440 2023-10-06T04:53:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7ae959a700 1 -- 172.21.15.3:0/2426486236 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad4003570 con 0x7f7aec0cd440 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7aead9d700 1 -- 172.21.15.3:0/2426486236 >> v2:172.21.15.153:3300/0 conn(0x7f7aec140e70 msgr2=0x7f7aec0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7aead9d700 1 --2- 172.21.15.3:0/2426486236 >> v2:172.21.15.153:3300/0 conn(0x7f7aec140e70 0x7f7aec0ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7aead9d700 1 -- 172.21.15.3:0/2426486236 >> v1:172.21.15.3:6789/0 conn(0x7f7aec0cd440 legacy=0x7f7aec0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7aead9d700 1 -- 172.21.15.3:0/2426486236 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0cc420 con 0x7f7aec0b16c0 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7aea59c700 1 --2- 172.21.15.3:0/2426486236 >> v2:172.21.15.153:3300/0 conn(0x7f7aec140e70 0x7f7aec0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7aead9d700 1 --2- 172.21.15.3:0/2426486236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec067ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009e60 tx=0x7f7ae4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68c2c98ff59f7aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7ae959a700 1 -- 172.21.15.3:0/2426486236 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400e070 con 0x7f7aec0b16c0 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7ae959a700 1 -- 172.21.15.3:0/2426486236 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae400bab0 con 0x7f7aec0b16c0 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.443+0000 7f7ae959a700 1 -- 172.21.15.3:0/2426486236 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400be70 con 0x7f7aec0b16c0 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 -- 172.21.15.3:0/2426486236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 msgr2=0x7f7aec067ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/2426486236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec067ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009e60 tx=0x7f7ae4009780 comp rx=0 tx=0).stop 2023-10-06T04:53:37.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 -- 172.21.15.3:0/2426486236 shutdown_connections 2023-10-06T04:53:37.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/2426486236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec067ab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/2426486236 >> v2:172.21.15.153:3300/0 conn(0x7f7aec140e70 0x7f7aec0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 -- 172.21.15.3:0/2426486236 >> 172.21.15.3:0/2426486236 conn(0x7f7aec0ce210 msgr2=0x7f7aec0ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:37.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 -- 172.21.15.3:0/2426486236 shutdown_connections 2023-10-06T04:53:37.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.444+0000 7f7af0bdd700 1 -- 172.21.15.3:0/2426486236 wait complete. 2023-10-06T04:53:37.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.445+0000 7f7af0bdd700 1 Processor -- start 2023-10-06T04:53:37.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.445+0000 7f7af0bdd700 1 -- start start 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7aec0cd440 0x7f7aec16a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7aec140e70 0x7f7aec16ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0c7be0 con 0x7f7aec0cd440 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0c6600 con 0x7f7aec140e70 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0c68e0 con 0x7f7aec0b16c0 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7aec0cd440 0x7f7aec16a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7aec0cd440 0x7f7aec16a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52966/0 (socket says 172.21.15.3:52966) 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 -- 172.21.15.3:0/1762039542 learned_addr learned my addr 172.21.15.3:0/1762039542 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7aea59c700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7aead9d700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7aec140e70 0x7f7aec16ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 -- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 msgr2=0x7f7aec0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 -- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7aec140e70 msgr2=0x7f7aec16ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7aec140e70 0x7f7aec16ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 -- 172.21.15.3:0/1762039542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc009420 con 0x7f7aec0cd440 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7aead9d700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7aec140e70 0x7f7aec16ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ae9d9b700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7aec0cd440 0x7f7aec16a900 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c7c0 tx=0x7f7ae0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:37.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ada7fc700 1 -- 172.21.15.3:0/1762039542 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0005b30 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ada7fc700 1 -- 172.21.15.3:0/1762039542 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae0005cb0 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae4009a20 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7ada7fc700 1 -- 172.21.15.3:0/1762039542 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0007320 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.446+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec0c6ea0 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.447+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec04c380 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.448+0000 7f7ada7fc700 1 -- 172.21.15.3:0/1762039542 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ae0007690 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.448+0000 7f7ada7fc700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7acc066f30 0x7f7acc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.448+0000 7f7ada7fc700 1 -- 172.21.15.3:0/1762039542 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7ae0083bb0 con 0x7f7aec0cd440 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.448+0000 7f7aea59c700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7acc066f30 0x7f7acc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:37.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.449+0000 7f7aea59c700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7acc066f30 0x7f7acc0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0cc3d0 tx=0x7f7adc0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.452+0000 7f7ada7fc700 1 -- 172.21.15.3:0/1762039542 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ae0050c70 con 0x7f7aec0cd440 2023-10-06T04:53:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:37.693+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7aec0cc700 con 0x7f7aec0cd440 2023-10-06T04:53:38.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.308+0000 7f7ada7fc700 1 -- 172.21.15.3:0/1762039542 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ae00170a0 con 0x7f7aec0cd440 2023-10-06T04:53:38.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7acc066f30 msgr2=0x7f7acc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:38.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7acc066f30 0x7f7acc0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0cc3d0 tx=0x7f7adc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7aec0cd440 msgr2=0x7f7aec16a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:38.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7aec0cd440 0x7f7aec16a900 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c7c0 tx=0x7f7ae0007e00 comp rx=0 tx=0).stop 2023-10-06T04:53:38.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 shutdown_connections 2023-10-06T04:53:38.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7acc066f30 0x7f7acc0693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7aec140e70 0x7f7aec16ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7aec0b16c0 0x7f7aec0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 --2- 172.21.15.3:0/1762039542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7aec0cd440 0x7f7aec16a900 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 >> 172.21.15.3:0/1762039542 conn(0x7f7aec0ce210 msgr2=0x7f7aec13fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:38.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 shutdown_connections 2023-10-06T04:53:38.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.311+0000 7f7af0bdd700 1 -- 172.21.15.3:0/1762039542 wait complete. 2023-10-06T04:53:38.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:53:38.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.453+0000 7f123f023700 1 Processor -- start 2023-10-06T04:53:38.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.453+0000 7f123f023700 1 -- start start 2023-10-06T04:53:38.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123f023700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f12380cd570 0x7f123815d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:38.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123f023700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f123815a560 0x7f1238153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:38.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123f023700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123815aa00 con 0x7f12381536e0 2023-10-06T04:53:38.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123f023700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1238153ff0 con 0x7f123815a560 2023-10-06T04:53:38.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123f023700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12380ce6b0 con 0x7f12380cd570 2023-10-06T04:53:38.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123d5c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f123815a560 0x7f1238153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123d5c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f123815a560 0x7f1238153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52976/0 (socket says 172.21.15.3:52976) 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123d5c0700 1 -- 172.21.15.3:0/2011296746 learned_addr learned my addr 172.21.15.3:0/2011296746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f1237fff700 1 --2- 172.21.15.3:0/2011296746 >> v2:172.21.15.153:3300/0 conn(0x7f12380cd570 0x7f123815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123d5c0700 1 -- 172.21.15.3:0/2011296746 >> v2:172.21.15.153:3300/0 conn(0x7f12380cd570 msgr2=0x7f123815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123d5c0700 1 --2- 172.21.15.3:0/2011296746 >> v2:172.21.15.153:3300/0 conn(0x7f12380cd570 0x7f123815d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123d5c0700 1 -- 172.21.15.3:0/2011296746 >> v1:172.21.15.153:6789/0 conn(0x7f12381536e0 legacy=0x7f12380cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.454+0000 7f123d5c0700 1 -- 172.21.15.3:0/2011296746 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123815d830 con 0x7f123815a560 2023-10-06T04:53:38.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123d5c0700 1 --2- 172.21.15.3:0/2011296746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815a560 0x7f1238153ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f12380896f0 tx=0x7f1228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc120655968b28e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f12377fe700 1 -- 172.21.15.3:0/2011296746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228009e70 con 0x7f123815a560 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f12377fe700 1 -- 172.21.15.3:0/2011296746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122800b860 con 0x7f123815a560 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f12377fe700 1 -- 172.21.15.3:0/2011296746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122800bc00 con 0x7f123815a560 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 -- 172.21.15.3:0/2011296746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815a560 msgr2=0x7f1238153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 --2- 172.21.15.3:0/2011296746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815a560 0x7f1238153ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f12380896f0 tx=0x7f1228009250 comp rx=0 tx=0).stop 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 -- 172.21.15.3:0/2011296746 shutdown_connections 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 --2- 172.21.15.3:0/2011296746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815a560 0x7f1238153ab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 --2- 172.21.15.3:0/2011296746 >> v2:172.21.15.153:3300/0 conn(0x7f12380cd570 0x7f123815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 -- 172.21.15.3:0/2011296746 >> 172.21.15.3:0/2011296746 conn(0x7f123806f980 msgr2=0x7f1238070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 -- 172.21.15.3:0/2011296746 shutdown_connections 2023-10-06T04:53:38.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.455+0000 7f123f023700 1 -- 172.21.15.3:0/2011296746 wait complete. 2023-10-06T04:53:38.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.456+0000 7f123f023700 1 Processor -- start 2023-10-06T04:53:38.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f123f023700 1 -- start start 2023-10-06T04:53:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f123f023700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12380cd570 0x7f1238169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f123f023700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12381536e0 0x7f123816c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f123f023700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f123815a560 0x7f12380c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:38.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f123f023700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b3200 con 0x7f12381536e0 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f123f023700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b34e0 con 0x7f12380cd570 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f123f023700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b37c0 con 0x7f123815a560 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.457+0000 7f1237fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12381536e0 0x7f123816c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12381536e0 0x7f123816c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52988/0 (socket says 172.21.15.3:52988) 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 -- 172.21.15.3:0/553019938 learned_addr learned my addr 172.21.15.3:0/553019938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 -- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f123815a560 msgr2=0x7f12380c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f123815a560 0x7f12380c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f123cdbf700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12380cd570 0x7f1238169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 -- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12380cd570 msgr2=0x7f1238169c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12380cd570 0x7f1238169c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 -- 172.21.15.3:0/553019938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280094f0 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1237fff700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12381536e0 0x7f123816c7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f122c00ec50 tx=0x7f122c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1234ff9700 1 -- 172.21.15.3:0/553019938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00c9d0 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1234ff9700 1 -- 172.21.15.3:0/553019938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122c00cb30 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12380b3aa0 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f1234ff9700 1 -- 172.21.15.3:0/553019938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c013470 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.458+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123816a1a0 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.459+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1238071680 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.460+0000 7f1234ff9700 1 -- 172.21.15.3:0/553019938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f122c013860 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.460+0000 7f1234ff9700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1224066f30 0x7f12240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.460+0000 7f1234ff9700 1 -- 172.21.15.3:0/553019938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f122c084a30 con 0x7f12381536e0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.460+0000 7f123cdbf700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1224066f30 0x7f12240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.461+0000 7f123cdbf700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1224066f30 0x7f12240693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1230007fd0 tx=0x7f1230009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.464+0000 7f1234ff9700 1 -- 172.21.15.3:0/553019938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f122c051af0 con 0x7f12381536e0 2023-10-06T04:53:38.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:38.657+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12380c5380 con 0x7f12381536e0 2023-10-06T04:53:39.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.533+0000 7f1234ff9700 1 -- 172.21.15.3:0/553019938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 181+0+0 (secure 0 0 0) 0x7f122c055110 con 0x7f12381536e0 2023-10-06T04:53:39.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1224066f30 msgr2=0x7f12240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:39.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1224066f30 0x7f12240693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1230007fd0 tx=0x7f1230009040 comp rx=0 tx=0).stop 2023-10-06T04:53:39.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12381536e0 msgr2=0x7f123816c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:39.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12381536e0 0x7f123816c7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f122c00ec50 tx=0x7f122c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 shutdown_connections 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1224066f30 0x7f12240693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12380cd570 0x7f1238169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f123815a560 0x7f12380c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 --2- 172.21.15.3:0/553019938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12381536e0 0x7f123816c7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 >> 172.21.15.3:0/553019938 conn(0x7f123806f980 msgr2=0x7f12380c5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 shutdown_connections 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.536+0000 7f123f023700 1 -- 172.21.15.3:0/553019938 wait complete. 2023-10-06T04:53:39.539 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:53:39.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:53:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.681+0000 7f0636e46700 1 Processor -- start 2023-10-06T04:53:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.681+0000 7f0636e46700 1 -- start start 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0636e46700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0630159560 0x7f0630068050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0636e46700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0630068590 con 0x7f063006c370 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0636e46700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06300abf00 con 0x7f0630159190 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0636e46700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06300ac1e0 con 0x7f0630159560 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0636645700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f063006c370 0x7f06300674d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45380/0 (socket says 172.21.15.3:45380) 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0636645700 1 -- 172.21.15.3:0/3653410718 learned_addr learned my addr 172.21.15.3:0/3653410718 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855731294 0 0) 0x7f0630068590 con 0x7f063006c370 2023-10-06T04:53:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0618003570 con 0x7f063006c370 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.682+0000 7f0635643700 1 --2- 172.21.15.3:0/3653410718 >> v2:172.21.15.153:3300/0 conn(0x7f0630159560 0x7f0630068050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3500871518 0 0) 0x7f0618003570 con 0x7f063006c370 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 >> v2:172.21.15.153:3300/0 conn(0x7f0630159560 msgr2=0x7f0630068050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 --2- 172.21.15.3:0/3653410718 >> v2:172.21.15.153:3300/0 conn(0x7f0630159560 0x7f0630068050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 >> v1:172.21.15.153:6789/0 conn(0x7f0630159190 legacy=0x7f063015cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300ac4c0 con 0x7f063006c370 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f0628001470 con 0x7f063006c370 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 >> v1:172.21.15.3:6789/0 conn(0x7f063006c370 legacy=0x7f06300674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0618003a60 0x7f0618003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f0618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06180052e0 0x7f061800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06300ac1e0 con 0x7f0618004810 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061800c230 con 0x7f0618003a60 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061800c3f0 con 0x7f06180052e0 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635643700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f0618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635e44700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0618003a60 0x7f0618003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0636645700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06180052e0 0x7f061800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635643700 1 -- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06180052e0 msgr2=0x7f061800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635643700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06180052e0 0x7f061800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635643700 1 -- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0618003a60 msgr2=0x7f0618003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635643700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0618003a60 0x7f0618003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635643700 1 -- 172.21.15.3:0/3653410718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618003570 con 0x7f0618004810 2023-10-06T04:53:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0636645700 1 -- 172.21.15.3:0/3653410718 reap_dead start 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.683+0000 7f0635643700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f0618004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f062400bb90 tx=0x7f0624008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd85e348cc0cd0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062400d5d0 con 0x7f0618004810 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062400d750 con 0x7f0618004810 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0634e42700 1 -- 172.21.15.3:0/3653410718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062400db10 con 0x7f0618004810 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0636e46700 1 -- 172.21.15.3:0/3653410718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 msgr2=0x7f0618004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0636e46700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f0618004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f062400bb90 tx=0x7f0624008040 comp rx=0 tx=0).stop 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0636e46700 1 -- 172.21.15.3:0/3653410718 shutdown_connections 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0636e46700 1 --2- 172.21.15.3:0/3653410718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f0618004c80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0636e46700 1 -- 172.21.15.3:0/3653410718 >> 172.21.15.3:0/3653410718 conn(0x7f063006df00 msgr2=0x7f06300ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:39.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0636e46700 1 -- 172.21.15.3:0/3653410718 shutdown_connections 2023-10-06T04:53:39.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.684+0000 7f0636e46700 1 -- 172.21.15.3:0/3653410718 wait complete. 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.685+0000 7f0636e46700 1 Processor -- start 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.685+0000 7f0636e46700 1 -- start start 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0636e46700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f063014abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0636e46700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06300afe30 0x7f06300b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0636e46700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06300b07e0 0x7f063015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0636e46700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063015a900 con 0x7f0618004810 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0636e46700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063014b120 con 0x7f06300afe30 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0636e46700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063014e6c0 con 0x7f06300b07e0 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f063014abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f063014abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53008/0 (socket says 172.21.15.3:53008) 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 -- 172.21.15.3:0/277292669 learned_addr learned my addr 172.21.15.3:0/277292669 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0636645700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06300b07e0 0x7f063015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635643700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06300afe30 0x7f06300b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 -- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06300b07e0 msgr2=0x7f063015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06300b07e0 0x7f063015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 -- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06300afe30 msgr2=0x7f06300b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06300afe30 0x7f06300b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 -- 172.21.15.3:0/277292669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300b0c50 con 0x7f0618004810 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635643700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06300afe30 0x7f06300b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.686+0000 7f0635e44700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f063014abe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f062000ebf0 tx=0x7f0620001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:39.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.687+0000 7f062dffb700 1 -- 172.21.15.3:0/277292669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620019070 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.687+0000 7f062dffb700 1 -- 172.21.15.3:0/277292669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062000c7e0 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.687+0000 7f062dffb700 1 -- 172.21.15.3:0/277292669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000cb80 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.687+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0630159cf0 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.687+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063015a2b0 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.688+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063004c380 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.688+0000 7f062dffb700 1 -- 172.21.15.3:0/277292669 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0620015430 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.688+0000 7f062dffb700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f061c066f30 0x7f061c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.689+0000 7f062dffb700 1 -- 172.21.15.3:0/277292669 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f06200847d0 con 0x7f0618004810 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.689+0000 7f0635643700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f061c066f30 0x7f061c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.689+0000 7f0635643700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f061c066f30 0x7f061c0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f062400bfd0 tx=0x7f062401b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:39.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.693+0000 7f062dffb700 1 -- 172.21.15.3:0/277292669 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0620051a40 con 0x7f0618004810 2023-10-06T04:53:39.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:39.886+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06300ac7a0 con 0x7f0618004810 2023-10-06T04:53:40.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.538+0000 7f062dffb700 1 -- 172.21.15.3:0/277292669 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0620055060 con 0x7f0618004810 2023-10-06T04:53:40.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.542+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f061c066f30 msgr2=0x7f061c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.542+0000 7f0636e46700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f061c066f30 0x7f061c0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f062400bfd0 tx=0x7f062401b000 comp rx=0 tx=0).stop 2023-10-06T04:53:40.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.542+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 msgr2=0x7f063014abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.542+0000 7f0636e46700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f063014abe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f062000ebf0 tx=0x7f0620001640 comp rx=0 tx=0).stop 2023-10-06T04:53:40.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.542+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 shutdown_connections 2023-10-06T04:53:40.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.543+0000 7f0636e46700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f061c066f30 0x7f061c0693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.543+0000 7f0636e46700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06300b07e0 0x7f063015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.543+0000 7f0636e46700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0618004810 0x7f063014abe0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.543+0000 7f0636e46700 1 --2- 172.21.15.3:0/277292669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06300afe30 0x7f06300b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.543+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 >> 172.21.15.3:0/277292669 conn(0x7f063006df00 msgr2=0x7f063015e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:40.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.543+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 shutdown_connections 2023-10-06T04:53:40.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.543+0000 7f0636e46700 1 -- 172.21.15.3:0/277292669 wait complete. 2023-10-06T04:53:40.546 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T04:53:40.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-10-06T04:53:40.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.689+0000 7f584b7c4700 1 Processor -- start 2023-10-06T04:53:40.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.689+0000 7f584b7c4700 1 -- start start 2023-10-06T04:53:40.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f584b7c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58440b0fb0 0x7f58440b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:40.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f584b7c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5844069f10 con 0x7f5844067ab0 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f584b7c4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58440cca70 con 0x7f5844140e70 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f584b7c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58440cc140 con 0x7f58440b0fb0 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f5848d5f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58440b0fb0 0x7f58440b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f5849d61700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5844067ab0 0x7f5844067ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45394/0 (socket says 172.21.15.3:45394) 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f5848d5f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58440b0fb0 0x7f58440b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53018/0 (socket says 172.21.15.3:53018) 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.690+0000 7f5849d61700 1 -- 172.21.15.3:0/2509791314 learned_addr learned my addr 172.21.15.3:0/2509791314 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5848d5f700 1 -- 172.21.15.3:0/2509791314 >> v1:172.21.15.153:6789/0 conn(0x7f5844140e70 legacy=0x7f58440ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5848d5f700 1 -- 172.21.15.3:0/2509791314 >> v1:172.21.15.3:6789/0 conn(0x7f5844067ab0 legacy=0x7f5844067ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5848d5f700 1 -- 172.21.15.3:0/2509791314 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440cc420 con 0x7f58440b0fb0 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5843fff700 1 -- 172.21.15.3:0/2509791314 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374363080 0 0) 0x7f5844069f10 con 0x7f5844067ab0 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5848d5f700 1 --2- 172.21.15.3:0/2509791314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440b0fb0 0x7f58440b1420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f58380098e0 tx=0x7f58380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20aaa879c58b846d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:40.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5843fff700 1 -- 172.21.15.3:0/2509791314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838009e80 con 0x7f58440b0fb0 2023-10-06T04:53:40.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5843fff700 1 -- 172.21.15.3:0/2509791314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583800b860 con 0x7f58440b0fb0 2023-10-06T04:53:40.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f5843fff700 1 -- 172.21.15.3:0/2509791314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583800bbe0 con 0x7f58440b0fb0 2023-10-06T04:53:40.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f584b7c4700 1 -- 172.21.15.3:0/2509791314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440b0fb0 msgr2=0x7f58440b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f584b7c4700 1 --2- 172.21.15.3:0/2509791314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440b0fb0 0x7f58440b1420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f58380098e0 tx=0x7f58380091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f584b7c4700 1 -- 172.21.15.3:0/2509791314 shutdown_connections 2023-10-06T04:53:40.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f584b7c4700 1 --2- 172.21.15.3:0/2509791314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440b0fb0 0x7f58440b1420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.691+0000 7f584b7c4700 1 -- 172.21.15.3:0/2509791314 >> 172.21.15.3:0/2509791314 conn(0x7f58440ce210 msgr2=0x7f58440ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.692+0000 7f584b7c4700 1 -- 172.21.15.3:0/2509791314 shutdown_connections 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.692+0000 7f584b7c4700 1 -- 172.21.15.3:0/2509791314 wait complete. 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 Processor -- start 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 -- start start 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5844067ab0 0x7f584416d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58440b0fb0 0x7f584406e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5844140e70 0x7f584413de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584416a070 con 0x7f5844067ab0 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584406eb80 con 0x7f5844140e70 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.693+0000 7f584b7c4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584416dab0 con 0x7f58440b0fb0 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5844067ab0 0x7f584416d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:40.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5844067ab0 0x7f584416d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53020/0 (socket says 172.21.15.3:53020) 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 -- 172.21.15.3:0/239374609 learned_addr learned my addr 172.21.15.3:0/239374609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5848d5f700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58440b0fb0 0x7f584406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 -- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58440b0fb0 msgr2=0x7f584406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849d61700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5844140e70 0x7f584413de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58440b0fb0 0x7f584406e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 -- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5844140e70 msgr2=0x7f584413de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5844140e70 0x7f584413de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 -- 172.21.15.3:0/239374609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009420 con 0x7f5844067ab0 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849d61700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5844140e70 0x7f584413de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f5849560700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5844067ab0 0x7f584416d4d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f583400c8e0 tx=0x7f5834001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f58417fa700 1 -- 172.21.15.3:0/239374609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583400ce80 con 0x7f5844067ab0 2023-10-06T04:53:40.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f58417fa700 1 -- 172.21.15.3:0/239374609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583400a7e0 con 0x7f5844067ab0 2023-10-06T04:53:40.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584416dd90 con 0x7f5844067ab0 2023-10-06T04:53:40.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f58417fa700 1 -- 172.21.15.3:0/239374609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583400aba0 con 0x7f5844067ab0 2023-10-06T04:53:40.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.694+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584416e350 con 0x7f5844067ab0 2023-10-06T04:53:40.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.695+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584404c380 con 0x7f5844067ab0 2023-10-06T04:53:40.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.696+0000 7f58417fa700 1 -- 172.21.15.3:0/239374609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5834003030 con 0x7f5844067ab0 2023-10-06T04:53:40.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.696+0000 7f58417fa700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5830066f30 0x7f58300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:40.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.696+0000 7f58417fa700 1 -- 172.21.15.3:0/239374609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f58340843f0 con 0x7f5844067ab0 2023-10-06T04:53:40.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.696+0000 7f5848d5f700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5830066f30 0x7f58300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:40.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.697+0000 7f5848d5f700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5830066f30 0x7f58300693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58440b3900 tx=0x7f5838017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:40.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.700+0000 7f58417fa700 1 -- 172.21.15.3:0/239374609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5834055290 con 0x7f5844067ab0 2023-10-06T04:53:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.896+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7f584413f060 con 0x7f5844067ab0 2023-10-06T04:53:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.899+0000 7f58417fa700 1 -- 172.21.15.3:0/239374609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v13) v1 ==== 106+0+0 (secure 0 0 0) 0x7f5834054e20 con 0x7f5844067ab0 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5830066f30 msgr2=0x7f58300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5830066f30 0x7f58300693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58440b3900 tx=0x7f5838017000 comp rx=0 tx=0).stop 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5844067ab0 msgr2=0x7f584416d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5844067ab0 0x7f584416d4d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f583400c8e0 tx=0x7f5834001710 comp rx=0 tx=0).stop 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 shutdown_connections 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5830066f30 0x7f58300693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5844140e70 0x7f584413de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5844067ab0 0x7f584416d4d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 --2- 172.21.15.3:0/239374609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58440b0fb0 0x7f584406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 >> 172.21.15.3:0/239374609 conn(0x7f58440ce210 msgr2=0x7f58440c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 shutdown_connections 2023-10-06T04:53:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:40.902+0000 7f584b7c4700 1 -- 172.21.15.3:0/239374609 wait complete. 2023-10-06T04:53:40.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2023-10-06T04:53:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.034+0000 7fd6f0b2a700 1 Processor -- start 2023-10-06T04:53:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.034+0000 7fd6f0b2a700 1 -- start start 2023-10-06T04:53:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.035+0000 7fd6f0b2a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd6ec067dc0 0x7fd6ec0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.035+0000 7fd6f0b2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6ec0b1b00 0x7fd6ec15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.035+0000 7fd6f0b2a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0691c0 con 0x7fd6ec068190 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.035+0000 7fd6f0b2a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0694a0 con 0x7fd6ec0b1b00 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.035+0000 7fd6f0b2a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec069780 con 0x7fd6ec067dc0 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6ec0b1b00 0x7fd6ec15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6ead9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd6ec068190 0x7fd6ec15d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45402/0 (socket says 172.21.15.3:45402) 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6ead9d700 1 -- 172.21.15.3:0/606925679 learned_addr learned my addr 172.21.15.3:0/606925679 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6ea59c700 1 --2- 172.21.15.3:0/606925679 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec067dc0 0x7fd6ec0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e9d9b700 1 -- 172.21.15.3:0/606925679 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec067dc0 msgr2=0x7fd6ec0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e9d9b700 1 --2- 172.21.15.3:0/606925679 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec067dc0 0x7fd6ec0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e9d9b700 1 -- 172.21.15.3:0/606925679 >> v1:172.21.15.3:6789/0 conn(0x7fd6ec068190 legacy=0x7fd6ec15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e9d9b700 1 -- 172.21.15.3:0/606925679 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec069a60 con 0x7fd6ec0b1b00 2023-10-06T04:53:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e959a700 1 -- 172.21.15.3:0/606925679 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956798867 0 0) 0x7fd6ec0691c0 con 0x7fd6ec068190 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6ea59c700 1 --2- 172.21.15.3:0/606925679 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec067dc0 0x7fd6ec0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e9d9b700 1 --2- 172.21.15.3:0/606925679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec0b1b00 0x7fd6ec15c800 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06b5f0 tx=0x7fd6d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ef6fc5256ad972 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e959a700 1 -- 172.21.15.3:0/606925679 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400b6b0 con 0x7fd6ec0b1b00 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e959a700 1 -- 172.21.15.3:0/606925679 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d400b810 con 0x7fd6ec0b1b00 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.036+0000 7fd6e959a700 1 -- 172.21.15.3:0/606925679 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400bbb0 con 0x7fd6ec0b1b00 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/606925679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec0b1b00 msgr2=0x7fd6ec15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/606925679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec0b1b00 0x7fd6ec15c800 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06b5f0 tx=0x7fd6d40091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/606925679 shutdown_connections 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/606925679 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec067dc0 0x7fd6ec0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/606925679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec0b1b00 0x7fd6ec15c800 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/606925679 >> 172.21.15.3:0/606925679 conn(0x7fd6ec0ce270 msgr2=0x7fd6ec0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/606925679 shutdown_connections 2023-10-06T04:53:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.037+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/606925679 wait complete. 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.038+0000 7fd6f0b2a700 1 Processor -- start 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.038+0000 7fd6f0b2a700 1 -- start start 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6f0b2a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec067dc0 0x7fd6ec153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6f0b2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec068190 0x7fd6ec0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6f0b2a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b1b00 0x7fd6ec0adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6f0b2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1545d0 con 0x7fd6ec068190 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6f0b2a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec155150 con 0x7fd6ec0b1b00 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6f0b2a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b0810 con 0x7fd6ec067dc0 2023-10-06T04:53:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec068190 0x7fd6ec0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec068190 0x7fd6ec0b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53044/0 (socket says 172.21.15.3:53044) 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 -- 172.21.15.3:0/4166111961 learned_addr learned my addr 172.21.15.3:0/4166111961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6ea59c700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec067dc0 0x7fd6ec153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6ead9d700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b1b00 0x7fd6ec0adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 -- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec067dc0 msgr2=0x7fd6ec153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec067dc0 0x7fd6ec153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 -- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b1b00 msgr2=0x7fd6ec0adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b1b00 0x7fd6ec0adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 -- 172.21.15.3:0/4166111961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4009420 con 0x7fd6ec068190 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e9d9b700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec068190 0x7fd6ec0b02d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4013000 tx=0x7fd6d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e27fc700 1 -- 172.21.15.3:0/4166111961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400bb90 con 0x7fd6ec068190 2023-10-06T04:53:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e27fc700 1 -- 172.21.15.3:0/4166111961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d4011420 con 0x7fd6ec068190 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec0ae070 con 0x7fd6ec068190 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.039+0000 7fd6e27fc700 1 -- 172.21.15.3:0/4166111961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40117c0 con 0x7fd6ec068190 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.040+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec154b00 con 0x7fd6ec068190 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.040+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec1552c0 con 0x7fd6ec068190 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.041+0000 7fd6e27fc700 1 -- 172.21.15.3:0/4166111961 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6d4011b70 con 0x7fd6ec068190 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.041+0000 7fd6e27fc700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6cc066f30 0x7fd6cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.041+0000 7fd6e27fc700 1 -- 172.21.15.3:0/4166111961 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd6d40858f0 con 0x7fd6ec068190 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.041+0000 7fd6ea59c700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6cc066f30 0x7fd6cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.042+0000 7fd6ea59c700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6cc066f30 0x7fd6cc0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc007660 tx=0x7fd6dc0098b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.046+0000 7fd6e27fc700 1 -- 172.21.15.3:0/4166111961 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6d4052d00 con 0x7fd6ec068190 2023-10-06T04:53:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.239+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) v1 -- 0x7fd6ec04c380 con 0x7fd6ec068190 2023-10-06T04:53:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.243+0000 7fd6e27fc700 1 -- 172.21.15.3:0/4166111961 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v14) v1 ==== 119+0+0 (secure 0 0 0) 0x7fd6d4056320 con 0x7fd6ec068190 2023-10-06T04:53:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6cc066f30 msgr2=0x7fd6cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6cc066f30 0x7fd6cc0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc007660 tx=0x7fd6dc0098b0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec068190 msgr2=0x7fd6ec0b02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec068190 0x7fd6ec0b02d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4013000 tx=0x7fd6d4017000 comp rx=0 tx=0).stop 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 shutdown_connections 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6cc066f30 0x7fd6cc0693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b1b00 0x7fd6ec0adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec067dc0 0x7fd6ec153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 --2- 172.21.15.3:0/4166111961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec068190 0x7fd6ec0b02d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 >> 172.21.15.3:0/4166111961 conn(0x7fd6ec0ce270 msgr2=0x7fd6ec15a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 shutdown_connections 2023-10-06T04:53:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.245+0000 7fd6f0b2a700 1 -- 172.21.15.3:0/4166111961 wait complete. 2023-10-06T04:53:41.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:53:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.378+0000 7fa0d14b8700 1 Processor -- start 2023-10-06T04:53:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.378+0000 7fa0d14b8700 1 -- start start 2023-10-06T04:53:41.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0d14b8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0cc150690 0x7fa0cc150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0d14b8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa0cc0cba50 0x7fa0cc15d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0d14b8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc15da80 con 0x7fa0cc15a720 2023-10-06T04:53:41.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0d14b8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc15a290 con 0x7fa0cc150690 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0d14b8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc0d0020 con 0x7fa0cc0cba50 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0caffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0cc150690 0x7fa0cc150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0cb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa0cc15a720 0x7fa0cc0af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45412/0 (socket says 172.21.15.3:45412) 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0cb7fe700 1 -- 172.21.15.3:0/2784016438 learned_addr learned my addr 172.21.15.3:0/2784016438 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0ca7fc700 1 --2- 172.21.15.3:0/2784016438 >> v2:172.21.15.153:3300/0 conn(0x7fa0cc0cba50 0x7fa0cc15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0caffd700 1 -- 172.21.15.3:0/2784016438 >> v2:172.21.15.153:3300/0 conn(0x7fa0cc0cba50 msgr2=0x7fa0cc15d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0caffd700 1 --2- 172.21.15.3:0/2784016438 >> v2:172.21.15.153:3300/0 conn(0x7fa0cc0cba50 0x7fa0cc15d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0caffd700 1 -- 172.21.15.3:0/2784016438 >> v1:172.21.15.3:6789/0 conn(0x7fa0cc15a720 legacy=0x7fa0cc0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0c9ffb700 1 -- 172.21.15.3:0/2784016438 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584968959 0 0) 0x7fa0cc15da80 con 0x7fa0cc15a720 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0caffd700 1 -- 172.21.15.3:0/2784016438 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc159f40 con 0x7fa0cc150690 2023-10-06T04:53:41.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0caffd700 1 --2- 172.21.15.3:0/2784016438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc150a60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0ce1e0 tx=0x7fa0bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a2b3f4e11fc556 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0c9ffb700 1 -- 172.21.15.3:0/2784016438 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00b6b0 con 0x7fa0cc150690 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0c9ffb700 1 -- 172.21.15.3:0/2784016438 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00b810 con 0x7fa0cc150690 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.379+0000 7fa0c9ffb700 1 -- 172.21.15.3:0/2784016438 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00bbb0 con 0x7fa0cc150690 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 -- 172.21.15.3:0/2784016438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 msgr2=0x7fa0cc150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/2784016438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc150a60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0ce1e0 tx=0x7fa0bc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 -- 172.21.15.3:0/2784016438 shutdown_connections 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/2784016438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc150a60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/2784016438 >> v2:172.21.15.153:3300/0 conn(0x7fa0cc0cba50 0x7fa0cc15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 -- 172.21.15.3:0/2784016438 >> 172.21.15.3:0/2784016438 conn(0x7fa0cc070310 msgr2=0x7fa0cc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 -- 172.21.15.3:0/2784016438 shutdown_connections 2023-10-06T04:53:41.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.380+0000 7fa0d14b8700 1 -- 172.21.15.3:0/2784016438 wait complete. 2023-10-06T04:53:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.381+0000 7fa0d14b8700 1 Processor -- start 2023-10-06T04:53:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.381+0000 7fa0d14b8700 1 -- start start 2023-10-06T04:53:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0d14b8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0cc0cba50 0x7fa0cc0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0d14b8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc14fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0d14b8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc15a720 0x7fa0cc14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0d14b8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0b5f20 con 0x7fa0cc150690 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0d14b8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0b4e80 con 0x7fa0cc15a720 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0d14b8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0b5160 con 0x7fa0cc0cba50 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc14fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc14fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53060/0 (socket says 172.21.15.3:53060) 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 -- 172.21.15.3:0/736672643 learned_addr learned my addr 172.21.15.3:0/736672643 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0cb7fe700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc15a720 0x7fa0cc14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0caffd700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0cc0cba50 0x7fa0cc0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 -- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0cc0cba50 msgr2=0x7fa0cc0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0cc0cba50 0x7fa0cc0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 -- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc15a720 msgr2=0x7fa0cc14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc15a720 0x7fa0cc14ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.382+0000 7fa0ca7fc700 1 -- 172.21.15.3:0/736672643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009420 con 0x7fa0cc150690 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.383+0000 7fa0ca7fc700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc14fd90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400c7d0 tx=0x7fa0b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.383+0000 7fa0c2ffd700 1 -- 172.21.15.3:0/736672643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4011600 con 0x7fa0cc150690 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.383+0000 7fa0c2ffd700 1 -- 172.21.15.3:0/736672643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4011760 con 0x7fa0cc150690 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.383+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc0b5440 con 0x7fa0cc150690 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.383+0000 7fa0c2ffd700 1 -- 172.21.15.3:0/736672643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4011b20 con 0x7fa0cc150690 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.383+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc1502d0 con 0x7fa0cc150690 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.384+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc0b5c40 con 0x7fa0cc150690 2023-10-06T04:53:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.384+0000 7fa0c2ffd700 1 -- 172.21.15.3:0/736672643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0b40193f0 con 0x7fa0cc150690 2023-10-06T04:53:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.384+0000 7fa0c2ffd700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa0ac066f30 0x7fa0ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.385+0000 7fa0c2ffd700 1 -- 172.21.15.3:0/736672643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa0b4084770 con 0x7fa0cc150690 2023-10-06T04:53:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.385+0000 7fa0caffd700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa0ac066f30 0x7fa0ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.385+0000 7fa0caffd700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa0ac066f30 0x7fa0ac0693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0093f0 tx=0x7fa0bc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.389+0000 7fa0c2ffd700 1 -- 172.21.15.3:0/736672643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0cc0b5c40 con 0x7fa0cc150690 2023-10-06T04:53:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.598+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0cc0b5c40 con 0x7fa0cc150690 2023-10-06T04:53:41.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.603+0000 7fa0c2ffd700 1 -- 172.21.15.3:0/736672643 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0cc0b5c40 con 0x7fa0cc150690 2023-10-06T04:53:41.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.605+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa0ac066f30 msgr2=0x7fa0ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.605+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa0ac066f30 0x7fa0ac0693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0093f0 tx=0x7fa0bc017000 comp rx=0 tx=0).stop 2023-10-06T04:53:41.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.605+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 msgr2=0x7fa0cc14fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.605+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc14fd90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400c7d0 tx=0x7fa0b4008040 comp rx=0 tx=0).stop 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 shutdown_connections 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa0ac066f30 0x7fa0ac0693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc15a720 0x7fa0cc14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0cc0cba50 0x7fa0cc0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 --2- 172.21.15.3:0/736672643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0cc150690 0x7fa0cc14fd90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 >> 172.21.15.3:0/736672643 conn(0x7fa0cc070310 msgr2=0x7fa0cc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 shutdown_connections 2023-10-06T04:53:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.606+0000 7fa0d14b8700 1 -- 172.21.15.3:0/736672643 wait complete. 2023-10-06T04:53:41.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-06T04:53:41.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.738+0000 7fcece3fd700 1 Processor -- start 2023-10-06T04:53:41.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.738+0000 7fcece3fd700 1 -- start start 2023-10-06T04:53:41.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.739+0000 7fcece3fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcec80b1f10 0x7fcec815d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.739+0000 7fcece3fd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec80c93f0 con 0x7fcec815cd00 2023-10-06T04:53:41.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.739+0000 7fcece3fd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec815e0d0 con 0x7fcec815c930 2023-10-06T04:53:41.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.739+0000 7fcece3fd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec80caf50 con 0x7fcec80b1f10 2023-10-06T04:53:41.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.739+0000 7fcecc99a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcec80b1f10 0x7fcec815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.739+0000 7fcec77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcec815cd00 0x7fcec80b1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45428/0 (socket says 172.21.15.3:45428) 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.739+0000 7fcec77fe700 1 -- 172.21.15.3:0/2734855732 learned_addr learned my addr 172.21.15.3:0/2734855732 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcecc99a700 1 -- 172.21.15.3:0/2734855732 >> v1:172.21.15.153:6789/0 conn(0x7fcec815c930 legacy=0x7fcec806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcecc99a700 1 -- 172.21.15.3:0/2734855732 >> v1:172.21.15.3:6789/0 conn(0x7fcec815cd00 legacy=0x7fcec80b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcecc99a700 1 -- 172.21.15.3:0/2734855732 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec815d830 con 0x7fcec80b1f10 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcecc99a700 1 --2- 172.21.15.3:0/2734855732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec815d150 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009e60 tx=0x7fcec0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe1ca0b01a90ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcec6ffd700 1 -- 172.21.15.3:0/2734855732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec000e070 con 0x7fcec80b1f10 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcec6ffd700 1 -- 172.21.15.3:0/2734855732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec000bc30 con 0x7fcec80b1f10 2023-10-06T04:53:41.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcec6ffd700 1 -- 172.21.15.3:0/2734855732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0012430 con 0x7fcec80b1f10 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcece3fd700 1 -- 172.21.15.3:0/2734855732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 msgr2=0x7fcec815d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2734855732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec815d150 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009e60 tx=0x7fcec0009780 comp rx=0 tx=0).stop 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcece3fd700 1 -- 172.21.15.3:0/2734855732 shutdown_connections 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2734855732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec815d150 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcece3fd700 1 -- 172.21.15.3:0/2734855732 >> 172.21.15.3:0/2734855732 conn(0x7fcec8066fb0 msgr2=0x7fcec80673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcece3fd700 1 -- 172.21.15.3:0/2734855732 shutdown_connections 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.740+0000 7fcece3fd700 1 -- 172.21.15.3:0/2734855732 wait complete. 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.741+0000 7fcece3fd700 1 Processor -- start 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcece3fd700 1 -- start start 2023-10-06T04:53:41.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcece3fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec813deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcece3fd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815c930 0x7fcec80c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcece3fd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec8153d50 0x7fcec80cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcece3fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8150a00 con 0x7fcec80b1f10 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcece3fd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8151b40 con 0x7fcec8153d50 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcece3fd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8150f30 con 0x7fcec815c930 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.742+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec813deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec813deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53072/0 (socket says 172.21.15.3:53072) 2023-10-06T04:53:41.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 -- 172.21.15.3:0/2149977982 learned_addr learned my addr 172.21.15.3:0/2149977982 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:41.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec77fe700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815c930 0x7fcec80c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcecc99a700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec8153d50 0x7fcec80cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 -- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815c930 msgr2=0x7fcec80c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815c930 0x7fcec80c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 -- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec8153d50 msgr2=0x7fcec80cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:41.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec8153d50 0x7fcec80cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:41.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 -- 172.21.15.3:0/2149977982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0009a20 con 0x7fcec80b1f10 2023-10-06T04:53:41.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec77fe700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815c930 0x7fcec80c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:41.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcec7fff700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec813deb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fceb800c600 tx=0x7fceb8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcebffff700 1 -- 172.21.15.3:0/2149977982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb8011590 con 0x7fcec80b1f10 2023-10-06T04:53:41.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcebffff700 1 -- 172.21.15.3:0/2149977982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb8011710 con 0x7fcec80b1f10 2023-10-06T04:53:41.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec8151210 con 0x7fcec80b1f10 2023-10-06T04:53:41.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcebffff700 1 -- 172.21.15.3:0/2149977982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb8011ad0 con 0x7fcec80b1f10 2023-10-06T04:53:41.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.743+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec81517d0 con 0x7fcec80b1f10 2023-10-06T04:53:41.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.744+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec8071680 con 0x7fcec80b1f10 2023-10-06T04:53:41.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.745+0000 7fcebffff700 1 -- 172.21.15.3:0/2149977982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fceb801c460 con 0x7fcec80b1f10 2023-10-06T04:53:41.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.745+0000 7fcebffff700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcea8066f30 0x7fcea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:41.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.745+0000 7fcebffff700 1 -- 172.21.15.3:0/2149977982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fceb8084170 con 0x7fcec80b1f10 2023-10-06T04:53:41.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.745+0000 7fcec77fe700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcea8066f30 0x7fcea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:41.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.746+0000 7fcec77fe700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcea8066f30 0x7fcea80693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcec813e3f0 tx=0x7fceb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:41.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:41.749+0000 7fcebffff700 1 -- 172.21.15.3:0/2149977982 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fceb8051580 con 0x7fcec80b1f10 2023-10-06T04:53:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.023+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7fcec815db10 con 0x7fcec80b1f10 2023-10-06T04:53:42.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.542+0000 7fcebffff700 1 -- 172.21.15.3:0/2149977982 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v93) v1 ==== 165+0+0 (secure 0 0 0) 0x7fceb8054ba0 con 0x7fcec80b1f10 2023-10-06T04:53:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.544+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcea8066f30 msgr2=0x7fcea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.544+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcea8066f30 0x7fcea80693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcec813e3f0 tx=0x7fceb0009040 comp rx=0 tx=0).stop 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 msgr2=0x7fcec813deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec813deb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fceb800c600 tx=0x7fceb8007f60 comp rx=0 tx=0).stop 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 shutdown_connections 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcea8066f30 0x7fcea80693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec8153d50 0x7fcec80cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec80b1f10 0x7fcec813deb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 --2- 172.21.15.3:0/2149977982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815c930 0x7fcec80c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 >> 172.21.15.3:0/2149977982 conn(0x7fcec8066fb0 msgr2=0x7fcec80b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 shutdown_connections 2023-10-06T04:53:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.545+0000 7fcece3fd700 1 -- 172.21.15.3:0/2149977982 wait complete. 2023-10-06T04:53:42.563 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2023-10-06T04:53:42.563 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T04:53:42.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2023-10-06T04:53:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.684+0000 7f4d8961b700 1 Processor -- start 2023-10-06T04:53:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.684+0000 7f4d8961b700 1 -- start start 2023-10-06T04:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d8961b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d84142b20 0x7f4d840b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d8961b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d84159b30 0x7f4d840b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d8961b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8406d480 con 0x7f4d84142ef0 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d8961b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8406e000 con 0x7f4d84159b30 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d8961b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8406d6d0 con 0x7f4d84142b20 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d837fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d84159b30 0x7f4d840b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d827fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d84142ef0 0x7f4d840b12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59738/0 (socket says 172.21.15.3:59738) 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.685+0000 7f4d827fc700 1 -- 172.21.15.3:0/1925736332 learned_addr learned my addr 172.21.15.3:0/1925736332 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d837fe700 1 -- 172.21.15.3:0/1925736332 >> v2:172.21.15.153:3300/0 conn(0x7f4d84142b20 msgr2=0x7f4d840b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d82ffd700 1 --2- 172.21.15.3:0/1925736332 >> v2:172.21.15.153:3300/0 conn(0x7f4d84142b20 0x7f4d840b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d837fe700 1 --2- 172.21.15.3:0/1925736332 >> v2:172.21.15.153:3300/0 conn(0x7f4d84142b20 0x7f4d840b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d837fe700 1 -- 172.21.15.3:0/1925736332 >> v1:172.21.15.3:6789/0 conn(0x7f4d84142ef0 legacy=0x7f4d840b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d81ffb700 1 -- 172.21.15.3:0/1925736332 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939610446 0 0) 0x7f4d8406d480 con 0x7f4d84142ef0 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d837fe700 1 -- 172.21.15.3:0/1925736332 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8406d9b0 con 0x7f4d84159b30 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d82ffd700 1 --2- 172.21.15.3:0/1925736332 >> v2:172.21.15.153:3300/0 conn(0x7f4d84142b20 0x7f4d840b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d837fe700 1 --2- 172.21.15.3:0/1925736332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d840b19d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cf220 tx=0x7f4d74009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df68e2a3406f34d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d81ffb700 1 -- 172.21.15.3:0/1925736332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400e070 con 0x7f4d84159b30 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d81ffb700 1 -- 172.21.15.3:0/1925736332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7400bc30 con 0x7f4d84159b30 2023-10-06T04:53:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d81ffb700 1 -- 172.21.15.3:0/1925736332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74012430 con 0x7f4d84159b30 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 -- 172.21.15.3:0/1925736332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 msgr2=0x7f4d840b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 --2- 172.21.15.3:0/1925736332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d840b19d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cf220 tx=0x7f4d74009780 comp rx=0 tx=0).stop 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 -- 172.21.15.3:0/1925736332 shutdown_connections 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 --2- 172.21.15.3:0/1925736332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d840b19d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 --2- 172.21.15.3:0/1925736332 >> v2:172.21.15.153:3300/0 conn(0x7f4d84142b20 0x7f4d840b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 -- 172.21.15.3:0/1925736332 >> 172.21.15.3:0/1925736332 conn(0x7f4d840679d0 msgr2=0x7f4d84067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 -- 172.21.15.3:0/1925736332 shutdown_connections 2023-10-06T04:53:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.686+0000 7f4d8961b700 1 -- 172.21.15.3:0/1925736332 wait complete. 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.688+0000 7f4d8961b700 1 Processor -- start 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.688+0000 7f4d8961b700 1 -- start start 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.688+0000 7f4d8961b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d84142b20 0x7f4d840b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d8961b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d84142ef0 0x7f4d841693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d8961b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d84139fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d8961b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8413a520 con 0x7f4d84159b30 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d8961b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840aeb60 con 0x7f4d84142b20 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d8961b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84158e10 con 0x7f4d84142ef0 2023-10-06T04:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d84139fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d84139fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41676/0 (socket says 172.21.15.3:41676) 2023-10-06T04:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 -- 172.21.15.3:0/2297613260 learned_addr learned my addr 172.21.15.3:0/2297613260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d827fc700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d84142ef0 0x7f4d841693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d82ffd700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d84142b20 0x7f4d840b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 -- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d84142ef0 msgr2=0x7f4d841693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d84142ef0 0x7f4d841693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 -- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d84142b20 msgr2=0x7f4d840b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d84142b20 0x7f4d840b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 -- 172.21.15.3:0/2297613260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d74009a20 con 0x7f4d84159b30 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d827fc700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d84142ef0 0x7f4d841693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d837fe700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d84139fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d8406b1e0 tx=0x7f4d74019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d7affd700 1 -- 172.21.15.3:0/2297613260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7401d630 con 0x7f4d84159b30 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d7affd700 1 -- 172.21.15.3:0/2297613260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7401d790 con 0x7f4d84159b30 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d84169a00 con 0x7f4d84159b30 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.689+0000 7f4d7affd700 1 -- 172.21.15.3:0/2297613260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7401db30 con 0x7f4d84159b30 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.690+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d84169fc0 con 0x7f4d84159b30 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.690+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d840aed10 con 0x7f4d84159b30 2023-10-06T04:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.691+0000 7f4d7affd700 1 -- 172.21.15.3:0/2297613260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d74024070 con 0x7f4d84159b30 2023-10-06T04:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.691+0000 7f4d7affd700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d64066f30 0x7f4d640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.691+0000 7f4d7affd700 1 -- 172.21.15.3:0/2297613260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4d74086da0 con 0x7f4d84159b30 2023-10-06T04:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.691+0000 7f4d82ffd700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d64066f30 0x7f4d640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.692+0000 7f4d82ffd700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d64066f30 0x7f4d640693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c006fd0 tx=0x7f4d7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.695+0000 7f4d7affd700 1 -- 172.21.15.3:0/2297613260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d840aed10 con 0x7f4d84159b30 2023-10-06T04:53:42.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:42.967+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) v1 -- 0x7f4d8406c000 con 0x7f4d84159b30 2023-10-06T04:53:43.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.549+0000 7f4d7affd700 1 -- 172.21.15.3:0/2297613260 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v94) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4d740541b0 con 0x7f4d84159b30 2023-10-06T04:53:43.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d64066f30 msgr2=0x7f4d640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d64066f30 0x7f4d640693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c006fd0 tx=0x7f4d7c008040 comp rx=0 tx=0).stop 2023-10-06T04:53:43.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 msgr2=0x7f4d84139fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d84139fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d8406b1e0 tx=0x7f4d74019000 comp rx=0 tx=0).stop 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 shutdown_connections 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d64066f30 0x7f4d640693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d84142b20 0x7f4d840b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d84159b30 0x7f4d84139fe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 --2- 172.21.15.3:0/2297613260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d84142ef0 0x7f4d841693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 >> 172.21.15.3:0/2297613260 conn(0x7f4d840679d0 msgr2=0x7f4d8413e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 shutdown_connections 2023-10-06T04:53:43.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.551+0000 7f4d8961b700 1 -- 172.21.15.3:0/2297613260 wait complete. 2023-10-06T04:53:43.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:53:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.691+0000 7f9d0698e700 1 Processor -- start 2023-10-06T04:53:43.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.691+0000 7f9d0698e700 1 -- start start 2023-10-06T04:53:43.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.691+0000 7f9d0698e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 0x7f9d00150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9d0698e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9d0015d540 0x7f9d000af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9d0698e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0015a290 con 0x7f9d000cba50 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9d0698e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d000d0020 con 0x7f9d0015d540 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9d0698e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00159f40 con 0x7f9d00150690 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cff7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9d000cba50 0x7f9d0015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59750/0 (socket says 172.21.15.3:59750) 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cff7fe700 1 -- 172.21.15.3:0/2484368887 learned_addr learned my addr 172.21.15.3:0/2484368887 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9d04f2b700 1 --2- 172.21.15.3:0/2484368887 >> v2:172.21.15.153:3300/0 conn(0x7f9d0015d540 0x7f9d000af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfffff700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 0x7f9d00150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522922489 0 0) 0x7f9d0015a290 con 0x7f9d000cba50 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cec003570 con 0x7f9d000cba50 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 250181553 0 0) 0x7f9cec003570 con 0x7f9d000cba50 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 msgr2=0x7f9d00150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 0x7f9d00150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 >> v2:172.21.15.153:3300/0 conn(0x7f9d0015d540 msgr2=0x7f9d000af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 --2- 172.21.15.3:0/2484368887 >> v2:172.21.15.153:3300/0 conn(0x7f9d0015d540 0x7f9d000af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0015dcb0 con 0x7f9d000cba50 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f9ce8001470 con 0x7f9d000cba50 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 >> v1:172.21.15.3:6789/0 conn(0x7f9d000cba50 legacy=0x7f9d0015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec003a60 0x7f9cec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec004810 0x7f9cec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 0x7f9cec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000d0020 con 0x7f9cec003a60 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00159f40 con 0x7f9cec004810 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec00ba60 con 0x7f9d00150690 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.692+0000 7f9cfffff700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec003a60 0x7f9cec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9d04f2b700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 0x7f9cec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cff7fe700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec004810 0x7f9cec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfffff700 1 -- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 msgr2=0x7f9cec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfffff700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 0x7f9cec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfffff700 1 -- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec004810 msgr2=0x7f9cec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfffff700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec004810 0x7f9cec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfffff700 1 -- 172.21.15.3:0/2484368887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec003570 con 0x7f9cec003a60 2023-10-06T04:53:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9d04f2b700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d00150690 0x7f9cec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cff7fe700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec004810 0x7f9cec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfffff700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec003a60 0x7f9cec003e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009c30 tx=0x7f9cf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e168caebc839eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000e710 con 0x7f9cec003a60 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000e870 con 0x7f9cec003a60 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.693+0000 7f9cfeffd700 1 -- 172.21.15.3:0/2484368887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000ec30 con 0x7f9cec003a60 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.694+0000 7f9d0698e700 1 -- 172.21.15.3:0/2484368887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec003a60 msgr2=0x7f9cec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.694+0000 7f9d0698e700 1 --2- 172.21.15.3:0/2484368887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec003a60 0x7f9cec003e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009c30 tx=0x7f9cf00091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.694+0000 7f9d04f2b700 1 -- 172.21.15.3:0/2484368887 reap_dead start 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.694+0000 7f9d0698e700 1 -- 172.21.15.3:0/2484368887 shutdown_connections 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.694+0000 7f9d0698e700 1 -- 172.21.15.3:0/2484368887 >> 172.21.15.3:0/2484368887 conn(0x7f9d00070310 msgr2=0x7f9d00070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.694+0000 7f9d0698e700 1 -- 172.21.15.3:0/2484368887 shutdown_connections 2023-10-06T04:53:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.694+0000 7f9d0698e700 1 -- 172.21.15.3:0/2484368887 wait complete. 2023-10-06T04:53:43.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.695+0000 7f9d0698e700 1 Processor -- start 2023-10-06T04:53:43.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.695+0000 7f9d0698e700 1 -- start start 2023-10-06T04:53:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec003a60 0x7f9d0014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d000cd180 0x7f9d000cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d0016a210 0x7f9d0016a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000cdb30 con 0x7f9d000cd180 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0016af60 con 0x7f9d0016a210 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000b4c10 con 0x7f9cec003a60 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d000cd180 0x7f9d000cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d000cd180 0x7f9d000cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41690/0 (socket says 172.21.15.3:41690) 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 -- 172.21.15.3:0/99074448 learned_addr learned my addr 172.21.15.3:0/99074448 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cfffff700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec003a60 0x7f9d0014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d04f2b700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d0016a210 0x7f9d0016a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 -- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec003a60 msgr2=0x7f9d0014e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec003a60 0x7f9d0014e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 -- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d0016a210 msgr2=0x7f9d0016a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d0016a210 0x7f9d0016a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 -- 172.21.15.3:0/99074448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000b50a0 con 0x7f9d000cd180 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d04f2b700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d0016a210 0x7f9d0016a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cfffff700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec003a60 0x7f9d0014e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cff7fe700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d000cd180 0x7f9d000cd5f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800eb90 tx=0x7f9ce80014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cf7fff700 1 -- 172.21.15.3:0/99074448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800c7e0 con 0x7f9d000cd180 2023-10-06T04:53:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cf7fff700 1 -- 172.21.15.3:0/99074448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce800c940 con 0x7f9d000cd180 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d000b42e0 con 0x7f9d000cd180 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9cf7fff700 1 -- 172.21.15.3:0/99074448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800ccc0 con 0x7f9d000cd180 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.696+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d000b48a0 con 0x7f9d000cd180 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.697+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d000b4a60 con 0x7f9d000cd180 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.698+0000 7f9cf7fff700 1 -- 172.21.15.3:0/99074448 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ce801d460 con 0x7f9d000cd180 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.698+0000 7f9cf7fff700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ce0066f30 0x7f9ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.698+0000 7f9cf7fff700 1 -- 172.21.15.3:0/99074448 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9ce80844c0 con 0x7f9d000cd180 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.698+0000 7f9cfffff700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ce0066f30 0x7f9ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.699+0000 7f9cfffff700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ce0066f30 0x7f9ce00693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00091e0 tx=0x7f9cf001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.702+0000 7f9cf7fff700 1 -- 172.21.15.3:0/99074448 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ce80518d0 con 0x7f9d000cd180 2023-10-06T04:53:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.899+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9d0004c380 con 0x7f9d000cd180 2023-10-06T04:53:43.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.903+0000 7f9cf7fff700 1 -- 172.21.15.3:0/99074448 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9ce8054ef0 con 0x7f9d000cd180 2023-10-06T04:53:43.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ce0066f30 msgr2=0x7f9ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ce0066f30 0x7f9ce00693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00091e0 tx=0x7f9cf001a000 comp rx=0 tx=0).stop 2023-10-06T04:53:43.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d000cd180 msgr2=0x7f9d000cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:43.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d000cd180 0x7f9d000cd5f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800eb90 tx=0x7f9ce80014c0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 shutdown_connections 2023-10-06T04:53:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ce0066f30 0x7f9ce00693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d0016a210 0x7f9d0016a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec003a60 0x7f9d0014e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 --2- 172.21.15.3:0/99074448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d000cd180 0x7f9d000cd5f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 >> 172.21.15.3:0/99074448 conn(0x7f9d00070310 msgr2=0x7f9d0015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 shutdown_connections 2023-10-06T04:53:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:43.905+0000 7f9d0698e700 1 -- 172.21.15.3:0/99074448 wait complete. 2023-10-06T04:53:43.919 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:53:43.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:53:43.955 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:53:43.996 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:53:43.997 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:53:43.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.114+0000 7ff92873c700 1 Processor -- start 2023-10-06T04:53:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.114+0000 7ff92873c700 1 -- start start 2023-10-06T04:53:44.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.115+0000 7ff92873c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 0x7ff9201644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.115+0000 7ff92873c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9200cf6c0 con 0x7ff92015ccc0 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.115+0000 7ff92873c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9200cfb50 con 0x7ff92015c8f0 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.115+0000 7ff92873c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200ceab0 con 0x7ff9200d0870 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff925cd7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff92015ccc0 0x7ff9200d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59756/0 (socket says 172.21.15.3:59756) 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff925cd7700 1 -- 172.21.15.3:0/3641270673 learned_addr learned my addr 172.21.15.3:0/3641270673 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff926cd9700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 0x7ff9201644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178705243 0 0) 0x7ff9200cf6c0 con 0x7ff92015ccc0 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff908003570 con 0x7ff92015ccc0 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2911066512 0 0) 0x7ff908003570 con 0x7ff92015ccc0 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 msgr2=0x7ff9201644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 0x7ff9201644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 >> v1:172.21.15.153:6789/0 conn(0x7ff92015c8f0 legacy=0x7ff92015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9200ced90 con 0x7ff92015ccc0 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7ff914001470 con 0x7ff92015ccc0 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 >> v1:172.21.15.3:6789/0 conn(0x7ff92015ccc0 legacy=0x7ff9200d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 0x7ff908003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff908004010 0x7ff908004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908004d50 0x7ff9080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200ceab0 con 0x7ff908004d50 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90800bbf0 con 0x7ff908004010 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90800bdb0 con 0x7ff9200d0870 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff926cd9700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908004d50 0x7ff9080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff9264d8700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 0x7ff908003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.116+0000 7ff925cd7700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff908004010 0x7ff908004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff926cd9700 1 -- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 msgr2=0x7ff908003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff926cd9700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9200d0870 0x7ff908003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff926cd9700 1 -- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff908004010 msgr2=0x7ff908004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff926cd9700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff908004010 0x7ff908004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff926cd9700 1 -- 172.21.15.3:0/3641270673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908003570 con 0x7ff908004d50 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff925cd7700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff908004010 0x7ff908004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff926cd9700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908004d50 0x7ff9080051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff918006f00 tx=0x7ff9180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0205dcb595caf1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800e710 con 0x7ff908004d50 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91800e870 con 0x7ff908004d50 2023-10-06T04:53:44.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff9254d6700 1 -- 172.21.15.3:0/3641270673 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800ec30 con 0x7ff908004d50 2023-10-06T04:53:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff92873c700 1 -- 172.21.15.3:0/3641270673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908004d50 msgr2=0x7ff9080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff92873c700 1 --2- 172.21.15.3:0/3641270673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908004d50 0x7ff9080051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff918006f00 tx=0x7ff9180091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff926cd9700 1 -- 172.21.15.3:0/3641270673 reap_dead start 2023-10-06T04:53:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff92873c700 1 -- 172.21.15.3:0/3641270673 shutdown_connections 2023-10-06T04:53:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff92873c700 1 -- 172.21.15.3:0/3641270673 >> 172.21.15.3:0/3641270673 conn(0x7ff920070310 msgr2=0x7ff920070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.117+0000 7ff92873c700 1 -- 172.21.15.3:0/3641270673 shutdown_connections 2023-10-06T04:53:44.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.118+0000 7ff92873c700 1 -- 172.21.15.3:0/3641270673 wait complete. 2023-10-06T04:53:44.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 Processor -- start 2023-10-06T04:53:44.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 -- start start 2023-10-06T04:53:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff908004d50 0x7ff920154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92014f960 0x7ff92016c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff92014fdd0 0x7ff92016ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92015b690 con 0x7ff92014f960 2023-10-06T04:53:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201548c0 con 0x7ff92014fdd0 2023-10-06T04:53:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff92873c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff920150e50 con 0x7ff908004d50 2023-10-06T04:53:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff925cd7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92014f960 0x7ff92016c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff925cd7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92014f960 0x7ff92016c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41708/0 (socket says 172.21.15.3:41708) 2023-10-06T04:53:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.119+0000 7ff925cd7700 1 -- 172.21.15.3:0/2156560229 learned_addr learned my addr 172.21.15.3:0/2156560229 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff9264d8700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff908004d50 0x7ff920154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff926cd9700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff92014fdd0 0x7ff92016ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff925cd7700 1 -- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff908004d50 msgr2=0x7ff920154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff925cd7700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff908004d50 0x7ff920154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff925cd7700 1 -- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff92014fdd0 msgr2=0x7ff92016ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff925cd7700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff92014fdd0 0x7ff92016ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff925cd7700 1 -- 172.21.15.3:0/2156560229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9201502d0 con 0x7ff92014f960 2023-10-06T04:53:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff9264d8700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff908004d50 0x7ff920154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff925cd7700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92014f960 0x7ff92016c910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff91400eb60 tx=0x7ff9200cf630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff91e7fc700 1 -- 172.21.15.3:0/2156560229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914004b10 con 0x7ff92014f960 2023-10-06T04:53:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff91e7fc700 1 -- 172.21.15.3:0/2156560229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff914010040 con 0x7ff92014f960 2023-10-06T04:53:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9201505b0 con 0x7ff92014f960 2023-10-06T04:53:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff91e7fc700 1 -- 172.21.15.3:0/2156560229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914004e80 con 0x7ff92014f960 2023-10-06T04:53:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.120+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff920150b70 con 0x7ff92014f960 2023-10-06T04:53:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.121+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff920156ba0 con 0x7ff92014f960 2023-10-06T04:53:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.122+0000 7ff91e7fc700 1 -- 172.21.15.3:0/2156560229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff914005260 con 0x7ff92014f960 2023-10-06T04:53:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.122+0000 7ff91e7fc700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff90c066f60 0x7ff90c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.122+0000 7ff91e7fc700 1 -- 172.21.15.3:0/2156560229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ff914084080 con 0x7ff92014f960 2023-10-06T04:53:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.122+0000 7ff9264d8700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff90c066f60 0x7ff90c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.123+0000 7ff9264d8700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff90c066f60 0x7ff90c069410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff910002e20 tx=0x7ff910001640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:44.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.126+0000 7ff91e7fc700 1 -- 172.21.15.3:0/2156560229 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff920156ba0 con 0x7ff92014f960 2023-10-06T04:53:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.368+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff920156ba0 con 0x7ff92014f960 2023-10-06T04:53:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.368+0000 7ff91e7fc700 1 -- 172.21.15.3:0/2156560229 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3529 (secure 0 0 0) 0x7ff920156ba0 con 0x7ff92014f960 2023-10-06T04:53:44.370 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:44.370 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1757036616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1757036616},{"type":"v1","addr":"172.21.15.3:6834","nonce":1757036616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1975852497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1975852497},{"type":"v1","addr":"172.21.15.3:6835","nonce":1975852497}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3962142851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3962142851},{"type":"v1","addr":"172.21.15.153:6837","nonce":3962142851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2595201755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2595201755},{"type":"v1","addr":"172.21.15.153:6836","nonce":2595201755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[]} 2023-10-06T04:53:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff90c066f60 msgr2=0x7ff90c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff90c066f60 0x7ff90c069410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff910002e20 tx=0x7ff910001640 comp rx=0 tx=0).stop 2023-10-06T04:53:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92014f960 msgr2=0x7ff92016c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92014f960 0x7ff92016c910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff91400eb60 tx=0x7ff9200cf630 comp rx=0 tx=0).stop 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 shutdown_connections 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff90c066f60 0x7ff90c069410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff92014fdd0 0x7ff92016ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff908004d50 0x7ff920154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 --2- 172.21.15.3:0/2156560229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92014f960 0x7ff92016c910 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.370+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 >> 172.21.15.3:0/2156560229 conn(0x7ff920070310 msgr2=0x7ff9200b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.371+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 shutdown_connections 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.371+0000 7ff92873c700 1 -- 172.21.15.3:0/2156560229 wait complete. 2023-10-06T04:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2023-10-06T04:53:44.384 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:53:44.384 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:53:44.385 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:53:44.385 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:53:44.385 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:53:44.385 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:53:44.385 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:53:44.385 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:53:44.386 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:53:44.386 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:53:44.386 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:53:44.386 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:53:44.386 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:53:44.386 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:53:44.386 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:53:44.387 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:53:44.394 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:53:44.395 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:53:44.395 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:53:44.395 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:53:44.397 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:53:44.397 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:53:44.398 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:53:44.398 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:53:44.438 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:53:44.438 DEBUG:teuthology.parallel:result is None 2023-10-06T04:53:44.439 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:53:44.439 DEBUG:teuthology.parallel:result is None 2023-10-06T04:53:44.440 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:53:44.440 DEBUG:teuthology.parallel:result is None 2023-10-06T04:53:44.441 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:53:44.441 DEBUG:teuthology.parallel:result is None 2023-10-06T04:53:44.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:53:44.514 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:53:44.514 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:53:44.520 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:53:44.521 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.567+0000 7fcd85925700 1 Processor -- start 2023-10-06T04:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.567+0000 7fcd85925700 1 -- start start 2023-10-06T04:53:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd85925700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcd800cb3b0 0x7fcd800cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd85925700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd800cb780 0x7fcd80158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd85925700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd800cf130 con 0x7fcd80159500 2023-10-06T04:53:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd85925700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800ce520 con 0x7fcd800cb780 2023-10-06T04:53:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd85925700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800ce800 con 0x7fcd800cb3b0 2023-10-06T04:53:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd7ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd800cb780 0x7fcd80158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd85124700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcd80159500 0x7fcd800cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59768/0 (socket says 172.21.15.3:59768) 2023-10-06T04:53:44.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd85124700 1 -- 172.21.15.3:0/4232873982 learned_addr learned my addr 172.21.15.3:0/4232873982 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:44.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.568+0000 7fcd84923700 1 --2- 172.21.15.3:0/4232873982 >> v2:172.21.15.153:3300/0 conn(0x7fcd800cb3b0 0x7fcd800cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7ffff700 1 -- 172.21.15.3:0/4232873982 >> v2:172.21.15.153:3300/0 conn(0x7fcd800cb3b0 msgr2=0x7fcd800cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7ffff700 1 --2- 172.21.15.3:0/4232873982 >> v2:172.21.15.153:3300/0 conn(0x7fcd800cb3b0 0x7fcd800cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7ffff700 1 -- 172.21.15.3:0/4232873982 >> v1:172.21.15.3:6789/0 conn(0x7fcd80159500 legacy=0x7fcd800cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7ffff700 1 -- 172.21.15.3:0/4232873982 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd800ceae0 con 0x7fcd800cb780 2023-10-06T04:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7f7fe700 1 -- 172.21.15.3:0/4232873982 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075113386 0 0) 0x7fcd800cf130 con 0x7fcd80159500 2023-10-06T04:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7ffff700 1 --2- 172.21.15.3:0/4232873982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb780 0x7fcd80158fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcd800893a0 tx=0x7fcd6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300a4d01a862bb82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7f7fe700 1 -- 172.21.15.3:0/4232873982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c011630 con 0x7fcd800cb780 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7f7fe700 1 -- 172.21.15.3:0/4232873982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c011790 con 0x7fcd800cb780 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.569+0000 7fcd7f7fe700 1 -- 172.21.15.3:0/4232873982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c011b30 con 0x7fcd800cb780 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 -- 172.21.15.3:0/4232873982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb780 msgr2=0x7fcd80158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 --2- 172.21.15.3:0/4232873982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb780 0x7fcd80158fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcd800893a0 tx=0x7fcd6c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 -- 172.21.15.3:0/4232873982 shutdown_connections 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 --2- 172.21.15.3:0/4232873982 >> v2:172.21.15.153:3300/0 conn(0x7fcd800cb3b0 0x7fcd800cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 --2- 172.21.15.3:0/4232873982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb780 0x7fcd80158fc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 -- 172.21.15.3:0/4232873982 >> 172.21.15.3:0/4232873982 conn(0x7fcd80070310 msgr2=0x7fcd80070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 -- 172.21.15.3:0/4232873982 shutdown_connections 2023-10-06T04:53:44.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.570+0000 7fcd85925700 1 -- 172.21.15.3:0/4232873982 wait complete. 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.571+0000 7fcd85925700 1 Processor -- start 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.571+0000 7fcd85925700 1 -- start start 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85925700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb3b0 0x7fcd80151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85925700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd800cb780 0x7fcd800b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85925700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80159500 0x7fcd800b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85925700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8016b050 con 0x7fcd800cb3b0 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85925700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8016a720 con 0x7fcd80159500 2023-10-06T04:53:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85925700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8016aa00 con 0x7fcd800cb780 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb3b0 0x7fcd80151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb3b0 0x7fcd80151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41758/0 (socket says 172.21.15.3:41758) 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 -- 172.21.15.3:0/4107748420 learned_addr learned my addr 172.21.15.3:0/4107748420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd7ffff700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd800cb780 0x7fcd800b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85124700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80159500 0x7fcd800b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 -- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd800cb780 msgr2=0x7fcd800b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd800cb780 0x7fcd800b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 -- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80159500 msgr2=0x7fcd800b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80159500 0x7fcd800b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 -- 172.21.15.3:0/4107748420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c009420 con 0x7fcd800cb3b0 2023-10-06T04:53:44.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd7ffff700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd800cb780 0x7fcd800b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd85124700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80159500 0x7fcd800b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.572+0000 7fcd84923700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb3b0 0x7fcd80151440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400ec80 tx=0x7fcd740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.573+0000 7fcd7cff9700 1 -- 172.21.15.3:0/4107748420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400c9d0 con 0x7fcd800cb3b0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.573+0000 7fcd7cff9700 1 -- 172.21.15.3:0/4107748420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7400cb30 con 0x7fcd800cb3b0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.573+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8016ace0 con 0x7fcd800cb3b0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.573+0000 7fcd7cff9700 1 -- 172.21.15.3:0/4107748420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74013470 con 0x7fcd800cb3b0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.573+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd801696d0 con 0x7fcd800cb3b0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.574+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8016b330 con 0x7fcd800cb3b0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.574+0000 7fcd7cff9700 1 -- 172.21.15.3:0/4107748420 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd74013820 con 0x7fcd800cb3b0 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.574+0000 7fcd7cff9700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd64066f30 0x7fcd640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:44.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.574+0000 7fcd7cff9700 1 -- 172.21.15.3:0/4107748420 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fcd74084720 con 0x7fcd800cb3b0 2023-10-06T04:53:44.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.575+0000 7fcd7ffff700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd64066f30 0x7fcd640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:44.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.575+0000 7fcd7ffff700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd64066f30 0x7fcd640693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0093f0 tx=0x7fcd6c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:44.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.579+0000 7fcd7cff9700 1 -- 172.21.15.3:0/4107748420 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd74051b30 con 0x7fcd800cb3b0 2023-10-06T04:53:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:44.787+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcd8004c380 con 0x7fcd800cb3b0 2023-10-06T04:53:45.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.571+0000 7fcd7cff9700 1 -- 172.21.15.3:0/4107748420 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd74055150 con 0x7fcd800cb3b0 2023-10-06T04:53:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd64066f30 msgr2=0x7fcd640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd64066f30 0x7fcd640693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0093f0 tx=0x7fcd6c017000 comp rx=0 tx=0).stop 2023-10-06T04:53:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb3b0 msgr2=0x7fcd80151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb3b0 0x7fcd80151440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400ec80 tx=0x7fcd740091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 shutdown_connections 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd64066f30 0x7fcd640693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80159500 0x7fcd800b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd800cb3b0 0x7fcd80151440 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 --2- 172.21.15.3:0/4107748420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd800cb780 0x7fcd800b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.573+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 >> 172.21.15.3:0/4107748420 conn(0x7fcd80070310 msgr2=0x7fcd800c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.574+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 shutdown_connections 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.574+0000 7fcd85925700 1 -- 172.21.15.3:0/4107748420 wait complete. 2023-10-06T04:53:45.576 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T04:53:45.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:53:45.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.711+0000 7ff9e1837700 1 Processor -- start 2023-10-06T04:53:45.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.711+0000 7ff9e1837700 1 -- start start 2023-10-06T04:53:45.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9e1837700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9dc153bd0 0x7ff9dc15de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:45.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9e1837700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff9dc153fa0 0x7ff9dc159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:45.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9e1837700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9dc15d9f0 con 0x7ff9dc15a2e0 2023-10-06T04:53:45.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9e1837700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9dc15a760 con 0x7ff9dc153bd0 2023-10-06T04:53:45.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9e1837700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9dc0b0aa0 con 0x7ff9dc153fa0 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9daffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9dc153bd0 0x7ff9dc15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9daffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9dc153bd0 0x7ff9dc15de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41760/0 (socket says 172.21.15.3:41760) 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9daffd700 1 -- 172.21.15.3:0/908565258 learned_addr learned my addr 172.21.15.3:0/908565258 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9da7fc700 1 --2- 172.21.15.3:0/908565258 >> v2:172.21.15.153:3300/0 conn(0x7ff9dc153fa0 0x7ff9dc159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9daffd700 1 -- 172.21.15.3:0/908565258 >> v2:172.21.15.153:3300/0 conn(0x7ff9dc153fa0 msgr2=0x7ff9dc159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9daffd700 1 --2- 172.21.15.3:0/908565258 >> v2:172.21.15.153:3300/0 conn(0x7ff9dc153fa0 0x7ff9dc159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.712+0000 7ff9daffd700 1 -- 172.21.15.3:0/908565258 >> v1:172.21.15.153:6789/0 conn(0x7ff9dc15a2e0 legacy=0x7ff9dc06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9daffd700 1 -- 172.21.15.3:0/908565258 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc15a980 con 0x7ff9dc153bd0 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9da7fc700 1 --2- 172.21.15.3:0/908565258 >> v2:172.21.15.153:3300/0 conn(0x7ff9dc153fa0 0x7ff9dc159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:45.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9daffd700 1 --2- 172.21.15.3:0/908565258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc15de80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc15d960 tx=0x7ff9cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b61b75b8b1382bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9d9ffb700 1 -- 172.21.15.3:0/908565258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00e070 con 0x7ff9dc153bd0 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9d9ffb700 1 -- 172.21.15.3:0/908565258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc00ba10 con 0x7ff9dc153bd0 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9d9ffb700 1 -- 172.21.15.3:0/908565258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0092e0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 -- 172.21.15.3:0/908565258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 msgr2=0x7ff9dc15de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 --2- 172.21.15.3:0/908565258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc15de80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc15d960 tx=0x7ff9cc009840 comp rx=0 tx=0).stop 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 -- 172.21.15.3:0/908565258 shutdown_connections 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 --2- 172.21.15.3:0/908565258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc15de80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 --2- 172.21.15.3:0/908565258 >> v2:172.21.15.153:3300/0 conn(0x7ff9dc153fa0 0x7ff9dc159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 -- 172.21.15.3:0/908565258 >> 172.21.15.3:0/908565258 conn(0x7ff9dc0cdc30 msgr2=0x7ff9dc06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 -- 172.21.15.3:0/908565258 shutdown_connections 2023-10-06T04:53:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.713+0000 7ff9e1837700 1 -- 172.21.15.3:0/908565258 wait complete. 2023-10-06T04:53:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.715+0000 7ff9e1837700 1 Processor -- start 2023-10-06T04:53:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.715+0000 7ff9e1837700 1 -- start start 2023-10-06T04:53:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc13e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9dc153fa0 0x7ff9dc154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:45.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9dc15a2e0 0x7ff9dc16d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0ac2a0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc157bf0 con 0x7ff9dc15a2e0 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc156fb0 con 0x7ff9dc153fa0 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc13e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc13e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41766/0 (socket says 172.21.15.3:41766) 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 -- 172.21.15.3:0/2517331138 learned_addr learned my addr 172.21.15.3:0/2517331138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 -- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9dc153fa0 msgr2=0x7ff9dc154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9da7fc700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9dc153fa0 0x7ff9dc154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9dc153fa0 0x7ff9dc154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 -- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9dc15a2e0 msgr2=0x7ff9dc16d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9db7fe700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9dc15a2e0 0x7ff9dc16d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9dc15a2e0 0x7ff9dc16d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 -- 172.21.15.3:0/2517331138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc009ae0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9db7fe700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9dc15a2e0 0x7ff9dc16d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9daffd700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc13e720 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc070d80 tx=0x7ff9cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9d2ffd700 1 -- 172.21.15.3:0/2517331138 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc012aa0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9d2ffd700 1 -- 172.21.15.3:0/2517331138 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc012c00 con 0x7ff9dc153bd0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc157290 con 0x7ff9dc153bd0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9d2ffd700 1 -- 172.21.15.3:0/2517331138 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc01d420 con 0x7ff9dc153bd0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.716+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc13ec60 con 0x7ff9dc153bd0 2023-10-06T04:53:45.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.718+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc1577b0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.718+0000 7ff9d2ffd700 1 -- 172.21.15.3:0/2517331138 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9cc01d810 con 0x7ff9dc153bd0 2023-10-06T04:53:45.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.718+0000 7ff9d2ffd700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9bc066f30 0x7ff9bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:45.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.718+0000 7ff9d2ffd700 1 -- 172.21.15.3:0/2517331138 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff9cc085ac0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.719+0000 7ff9da7fc700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9bc066f30 0x7ff9bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:45.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.719+0000 7ff9da7fc700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9bc066f30 0x7ff9bc0693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc15a060 tx=0x7ff9c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:45.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.723+0000 7ff9d2ffd700 1 -- 172.21.15.3:0/2517331138 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9dc1577b0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.996+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff9dc1577b0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.996+0000 7ff9d2ffd700 1 -- 172.21.15.3:0/2517331138 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff9dc1577b0 con 0x7ff9dc153bd0 2023-10-06T04:53:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:45.998 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T04:53:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.002 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.003 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.005 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T04:53:46.007 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:46.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:53:46.009 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.011 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T04:53:46.012 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:53:46.013 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.014 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:46.014 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:53:46.014 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:53:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9bc066f30 msgr2=0x7ff9bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9bc066f30 0x7ff9bc0693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc15a060 tx=0x7ff9c4008040 comp rx=0 tx=0).stop 2023-10-06T04:53:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 msgr2=0x7ff9dc13e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc13e720 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc070d80 tx=0x7ff9cc017000 comp rx=0 tx=0).stop 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 shutdown_connections 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9bc066f30 0x7ff9bc0693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9dc15a2e0 0x7ff9dc16d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9dc153bd0 0x7ff9dc13e720 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 --2- 172.21.15.3:0/2517331138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9dc153fa0 0x7ff9dc154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 >> 172.21.15.3:0/2517331138 conn(0x7ff9dc0cdc30 msgr2=0x7ff9dc150e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 shutdown_connections 2023-10-06T04:53:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:45.999+0000 7ff9e1837700 1 -- 172.21.15.3:0/2517331138 wait complete. 2023-10-06T04:53:46.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.134+0000 7f7fa9422700 1 Processor -- start 2023-10-06T04:53:46.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.134+0000 7f7fa9422700 1 -- start start 2023-10-06T04:53:46.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa9422700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7fa40b2530 0x7f7fa40b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa9422700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa415e0d0 con 0x7f7fa40b2900 2023-10-06T04:53:46.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa9422700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40cbd70 con 0x7f7fa4159080 2023-10-06T04:53:46.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa9422700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa40b0e00 con 0x7f7fa40b2530 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7fa40b2530 0x7f7fa40b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa27fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7fa40b2900 0x7f7fa415d9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59772/0 (socket says 172.21.15.3:59772) 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa27fc700 1 -- 172.21.15.3:0/3764107430 learned_addr learned my addr 172.21.15.3:0/3764107430 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa2ffd700 1 -- 172.21.15.3:0/3764107430 >> v1:172.21.15.153:6789/0 conn(0x7f7fa4159080 legacy=0x7f7fa41594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa2ffd700 1 -- 172.21.15.3:0/3764107430 >> v1:172.21.15.3:6789/0 conn(0x7f7fa40b2900 legacy=0x7f7fa415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa1ffb700 1 -- 172.21.15.3:0/3764107430 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222477155 0 0) 0x7f7f8c001110 con 0x7f7fa40b2900 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa2ffd700 1 -- 172.21.15.3:0/3764107430 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40cb160 con 0x7f7fa40b2530 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.135+0000 7f7fa2ffd700 1 --2- 172.21.15.3:0/3764107430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa40b08c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0098e0 tx=0x7f7f9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31be4f4f439951ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa1ffb700 1 -- 172.21.15.3:0/3764107430 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c009e80 con 0x7f7fa40b2530 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa1ffb700 1 -- 172.21.15.3:0/3764107430 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c00b820 con 0x7f7fa40b2530 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa1ffb700 1 -- 172.21.15.3:0/3764107430 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c00bba0 con 0x7f7fa40b2530 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa9422700 1 -- 172.21.15.3:0/3764107430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 msgr2=0x7f7fa40b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa9422700 1 --2- 172.21.15.3:0/3764107430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa40b08c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0098e0 tx=0x7f7f9c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa9422700 1 -- 172.21.15.3:0/3764107430 shutdown_connections 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa9422700 1 --2- 172.21.15.3:0/3764107430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa40b08c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa9422700 1 -- 172.21.15.3:0/3764107430 >> 172.21.15.3:0/3764107430 conn(0x7f7fa4070310 msgr2=0x7f7fa4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa9422700 1 -- 172.21.15.3:0/3764107430 shutdown_connections 2023-10-06T04:53:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.136+0000 7f7fa9422700 1 -- 172.21.15.3:0/3764107430 wait complete. 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.137+0000 7f7fa9422700 1 Processor -- start 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.137+0000 7f7fa9422700 1 -- start start 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa9422700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa4169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa9422700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fa40b2900 0x7f7fa416ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa9422700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fa4159080 0x7f7fa40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa9422700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa415c600 con 0x7f7fa40b2530 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa9422700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4169fb0 con 0x7f7fa4159080 2023-10-06T04:53:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa9422700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa416b090 con 0x7f7fa40b2900 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa4169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa4169a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41790/0 (socket says 172.21.15.3:41790) 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa2ffd700 1 -- 172.21.15.3:0/2336008344 learned_addr learned my addr 172.21.15.3:0/2336008344 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa27fc700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fa40b2900 0x7f7fa416ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.138+0000 7f7fa37fe700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fa4159080 0x7f7fa40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa2ffd700 1 -- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fa40b2900 msgr2=0x7f7fa416ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa2ffd700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fa40b2900 0x7f7fa416ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa2ffd700 1 -- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fa4159080 msgr2=0x7f7fa40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa2ffd700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fa4159080 0x7f7fa40ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa2ffd700 1 -- 172.21.15.3:0/2336008344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c009420 con 0x7f7fa40b2530 2023-10-06T04:53:46.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa37fe700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fa4159080 0x7f7fa40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa2ffd700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa4169a70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0030c0 tx=0x7f7f9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7f9affd700 1 -- 172.21.15.3:0/2336008344 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c00bae0 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7f9affd700 1 -- 172.21.15.3:0/2336008344 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c020070 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa415ca90 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7f9affd700 1 -- 172.21.15.3:0/2336008344 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c01b6a0 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.139+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa415b9f0 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.140+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa4071680 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.140+0000 7f7f9affd700 1 -- 172.21.15.3:0/2336008344 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f9c022030 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.141+0000 7f7f9affd700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f84066f30 0x7f7f840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.141+0000 7f7f9affd700 1 -- 172.21.15.3:0/2336008344 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7f9c086300 con 0x7f7fa40b2530 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.141+0000 7f7fa27fc700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f84066f30 0x7f7f840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.142+0000 7f7fa27fc700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f84066f30 0x7f7f840693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7fa406d8b0 tx=0x7f7f8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.145+0000 7f7f9affd700 1 -- 172.21.15.3:0/2336008344 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f9c053760 con 0x7f7fa40b2530 2023-10-06T04:53:46.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.343+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fa415c8e0 con 0x7f7fa40b2530 2023-10-06T04:53:46.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.344+0000 7f7f9affd700 1 -- 172.21.15.3:0/2336008344 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13602 (secure 0 0 0) 0x7f7f9c056d80 con 0x7f7fa40b2530 2023-10-06T04:53:46.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:46.346 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:45.566816+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f84066f30 msgr2=0x7f7f840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f84066f30 0x7f7f840693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7fa406d8b0 tx=0x7f7f8c009040 comp rx=0 tx=0).stop 2023-10-06T04:53:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 msgr2=0x7f7fa4169a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa4169a70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0030c0 tx=0x7f7f9c016000 comp rx=0 tx=0).stop 2023-10-06T04:53:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 shutdown_connections 2023-10-06T04:53:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f84066f30 0x7f7f840693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fa4159080 0x7f7fa40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa40b2530 0x7f7fa4169a70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 --2- 172.21.15.3:0/2336008344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fa40b2900 0x7f7fa416ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 >> 172.21.15.3:0/2336008344 conn(0x7f7fa4070310 msgr2=0x7f7fa40cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 shutdown_connections 2023-10-06T04:53:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.346+0000 7f7fa9422700 1 -- 172.21.15.3:0/2336008344 wait complete. 2023-10-06T04:53:46.360 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:53:46.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:53:46.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.480+0000 7feb096e7700 1 Processor -- start 2023-10-06T04:53:46.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.480+0000 7feb096e7700 1 -- start start 2023-10-06T04:53:46.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb096e7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7feb04153740 0x7feb040ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb096e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0415a960 con 0x7feb04153b10 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb096e7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb040ac7e0 con 0x7feb0406ca80 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb096e7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb04159d50 con 0x7feb04153740 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb027fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feb04153b10 0x7feb0406c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59774/0 (socket says 172.21.15.3:59774) 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb027fc700 1 -- 172.21.15.3:0/3483773211 learned_addr learned my addr 172.21.15.3:0/3483773211 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192414092 0 0) 0x7feb0415a960 con 0x7feb04153b10 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf0003570 con 0x7feb04153b10 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb02ffd700 1 --2- 172.21.15.3:0/3483773211 >> v2:172.21.15.153:3300/0 conn(0x7feb04153740 0x7feb040ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.481+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3998182710 0 0) 0x7feaf0003570 con 0x7feb04153b10 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 >> v2:172.21.15.153:3300/0 conn(0x7feb04153740 msgr2=0x7feb040ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 --2- 172.21.15.3:0/3483773211 >> v2:172.21.15.153:3300/0 conn(0x7feb04153740 0x7feb040ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 >> v1:172.21.15.153:6789/0 conn(0x7feb0406ca80 legacy=0x7feb040a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0415a030 con 0x7feb04153b10 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7feaec001470 con 0x7feb04153b10 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 >> v1:172.21.15.3:6789/0 conn(0x7feb04153b10 legacy=0x7feb0406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf0003a60 0x7feaf0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feaf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf00052e0 0x7feaf000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04159d50 con 0x7feaf0004810 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf000c230 con 0x7feaf0003a60 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf000c3f0 con 0x7feaf00052e0 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb027fc700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feaf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb02ffd700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf0003a60 0x7feaf0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb037fe700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf00052e0 0x7feaf000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb027fc700 1 -- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf00052e0 msgr2=0x7feaf000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb027fc700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf00052e0 0x7feaf000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb027fc700 1 -- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf0003a60 msgr2=0x7feaf0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb027fc700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf0003a60 0x7feaf0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb027fc700 1 -- 172.21.15.3:0/3483773211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0003570 con 0x7feaf0004810 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb037fe700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf00052e0 0x7feaf000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb037fe700 1 -- 172.21.15.3:0/3483773211 reap_dead start 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb027fc700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feaf0004c80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c3c0 tx=0x7feaec001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f9dfb221b67e83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec00cc90 con 0x7feaf0004810 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec00ce10 con 0x7feaf0004810 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.482+0000 7feb01ffb700 1 -- 172.21.15.3:0/3483773211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec005b70 con 0x7feaf0004810 2023-10-06T04:53:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.483+0000 7feb096e7700 1 -- 172.21.15.3:0/3483773211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 msgr2=0x7feaf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.483+0000 7feb096e7700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feaf0004c80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c3c0 tx=0x7feaec001e50 comp rx=0 tx=0).stop 2023-10-06T04:53:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.483+0000 7feb096e7700 1 -- 172.21.15.3:0/3483773211 shutdown_connections 2023-10-06T04:53:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.483+0000 7feb096e7700 1 --2- 172.21.15.3:0/3483773211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feaf0004c80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.483+0000 7feb096e7700 1 -- 172.21.15.3:0/3483773211 >> 172.21.15.3:0/3483773211 conn(0x7feb040cdab0 msgr2=0x7feb040aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.483+0000 7feb096e7700 1 -- 172.21.15.3:0/3483773211 shutdown_connections 2023-10-06T04:53:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.483+0000 7feb096e7700 1 -- 172.21.15.3:0/3483773211 wait complete. 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.484+0000 7feb096e7700 1 Processor -- start 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.484+0000 7feb096e7700 1 -- start start 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb096e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feb0414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb096e7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0406ac80 0x7feb041406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb096e7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb0406b0f0 0x7feb04140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb096e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb040b0cd0 con 0x7feaf0004810 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb096e7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0414c060 con 0x7feb0406ac80 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb096e7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415ca40 con 0x7feb0406b0f0 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feb0414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feb0414fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41812/0 (socket says 172.21.15.3:41812) 2023-10-06T04:53:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 -- 172.21.15.3:0/3236768739 learned_addr learned my addr 172.21.15.3:0/3236768739 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb027fc700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0406ac80 0x7feb041406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 -- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb0406b0f0 msgr2=0x7feb04140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb037fe700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb0406b0f0 0x7feb04140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb0406b0f0 0x7feb04140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 -- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0406ac80 msgr2=0x7feb041406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0406ac80 0x7feb041406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 -- 172.21.15.3:0/3236768739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb040b09f0 con 0x7feaf0004810 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb027fc700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0406ac80 0x7feb041406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb037fe700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb0406b0f0 0x7feb04140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.485+0000 7feb02ffd700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feb0414fae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feafc002630 tx=0x7feafc009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.486+0000 7feafaffd700 1 -- 172.21.15.3:0/3236768739 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc014590 con 0x7feaf0004810 2023-10-06T04:53:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.486+0000 7feafaffd700 1 -- 172.21.15.3:0/3236768739 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc0146f0 con 0x7feaf0004810 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.486+0000 7feafaffd700 1 -- 172.21.15.3:0/3236768739 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc014a70 con 0x7feaf0004810 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.486+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0415c3f0 con 0x7feaf0004810 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.486+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04150020 con 0x7feaf0004810 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.487+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0415c250 con 0x7feaf0004810 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.487+0000 7feafaffd700 1 -- 172.21.15.3:0/3236768739 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feafc003040 con 0x7feaf0004810 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.488+0000 7feafaffd700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feae4066f30 0x7feae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.488+0000 7feafaffd700 1 -- 172.21.15.3:0/3236768739 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7feafc084fe0 con 0x7feaf0004810 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.488+0000 7feb027fc700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feae4066f30 0x7feae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.488+0000 7feb027fc700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feae4066f30 0x7feae40693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feaec001e90 tx=0x7feaec015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.492+0000 7feafaffd700 1 -- 172.21.15.3:0/3236768739 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb0415c250 con 0x7feaf0004810 2023-10-06T04:53:46.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.687+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feb0415c250 con 0x7feaf0004810 2023-10-06T04:53:46.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.688+0000 7feafaffd700 1 -- 172.21.15.3:0/3236768739 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7feb0415c250 con 0x7feaf0004810 2023-10-06T04:53:46.689 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feae4066f30 msgr2=0x7feae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feae4066f30 0x7feae40693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feaec001e90 tx=0x7feaec015000 comp rx=0 tx=0).stop 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 msgr2=0x7feb0414fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feb0414fae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feafc002630 tx=0x7feafc009710 comp rx=0 tx=0).stop 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 shutdown_connections 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feae4066f30 0x7feae40693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb0406b0f0 0x7feb04140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0004810 0x7feb0414fae0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 --2- 172.21.15.3:0/3236768739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0406ac80 0x7feb041406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 >> 172.21.15.3:0/3236768739 conn(0x7feb040cdab0 msgr2=0x7feb040aa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 shutdown_connections 2023-10-06T04:53:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.690+0000 7feb096e7700 1 -- 172.21.15.3:0/3236768739 wait complete. 2023-10-06T04:53:46.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:53:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.823+0000 7f6defb8e700 1 Processor -- start 2023-10-06T04:53:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.823+0000 7f6defb8e700 1 -- start start 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.824+0000 7f6defb8e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 0x7f6de80ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.824+0000 7f6defb8e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de8069f10 con 0x7f6de80cd440 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.824+0000 7f6defb8e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de80cca70 con 0x7f6de80b16c0 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.824+0000 7f6defb8e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80cc140 con 0x7f6de8140e70 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.824+0000 7f6ded129700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6de80cd440 0x7f6de80b0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59776/0 (socket says 172.21.15.3:59776) 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.824+0000 7f6ded129700 1 -- 172.21.15.3:0/4287049736 learned_addr learned my addr 172.21.15.3:0/4287049736 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404564069 0 0) 0x7f6de8069f10 con 0x7f6de80cd440 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded92a700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 0x7f6de80ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd0003570 con 0x7f6de80cd440 2023-10-06T04:53:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1392988746 0 0) 0x7f6dd0003570 con 0x7f6de80cd440 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 msgr2=0x7f6de80ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 0x7f6de80ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 >> v1:172.21.15.153:6789/0 conn(0x7f6de80b16c0 legacy=0x7f6de8067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de80cc420 con 0x7f6de80cd440 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6ddc001470 con 0x7f6de80cd440 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 >> v1:172.21.15.3:6789/0 conn(0x7f6de80cd440 legacy=0x7f6de80b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 0x7f6dd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80cc140 con 0x7f6dd0003a90 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000bb10 con 0x7f6dd0004810 2023-10-06T04:53:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000bcd0 con 0x7f6de8140e70 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded92a700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded129700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dee12b700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 0x7f6dd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded92a700 1 -- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 msgr2=0x7f6dd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded92a700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 0x7f6dd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded92a700 1 -- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd0004810 msgr2=0x7f6dd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded92a700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6ded92a700 1 -- 172.21.15.3:0/4287049736 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0003570 con 0x7f6dd0003a90 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.825+0000 7f6dee12b700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6de8140e70 0x7f6dd00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6ded92a700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003e60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e210 tx=0x7f6dd800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5286838ab06648e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800eac0 con 0x7f6dd0003a90 2023-10-06T04:53:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800f040 con 0x7f6dd0003a90 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6dec928700 1 -- 172.21.15.3:0/4287049736 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800ee70 con 0x7f6dd0003a90 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6defb8e700 1 -- 172.21.15.3:0/4287049736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0003a90 msgr2=0x7f6dd0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6defb8e700 1 --2- 172.21.15.3:0/4287049736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003e60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e210 tx=0x7f6dd800b040 comp rx=0 tx=0).stop 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6dee12b700 1 -- 172.21.15.3:0/4287049736 reap_dead start 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6defb8e700 1 -- 172.21.15.3:0/4287049736 shutdown_connections 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6defb8e700 1 -- 172.21.15.3:0/4287049736 >> 172.21.15.3:0/4287049736 conn(0x7f6de80ce210 msgr2=0x7f6de80ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6defb8e700 1 -- 172.21.15.3:0/4287049736 shutdown_connections 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.826+0000 7f6defb8e700 1 -- 172.21.15.3:0/4287049736 wait complete. 2023-10-06T04:53:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.827+0000 7f6defb8e700 1 Processor -- start 2023-10-06T04:53:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.827+0000 7f6defb8e700 1 -- start start 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6defb8e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6dd0003a90 0x7f6de80c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6defb8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6de80c71e0 0x7f6de815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6defb8e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de815ce20 0x7f6de815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6defb8e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80c7cb0 con 0x7f6de80c71e0 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6defb8e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8147b80 con 0x7f6de815ce20 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6defb8e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de815d800 con 0x7f6dd0003a90 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6ded129700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6de80c71e0 0x7f6de815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6ded129700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6de80c71e0 0x7f6de815c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41822/0 (socket says 172.21.15.3:41822) 2023-10-06T04:53:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6ded129700 1 -- 172.21.15.3:0/2177528768 learned_addr learned my addr 172.21.15.3:0/2177528768 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6dee12b700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de815ce20 0x7f6de815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.828+0000 7f6ded92a700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6dd0003a90 0x7f6de80c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6ded129700 1 -- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6dd0003a90 msgr2=0x7f6de80c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6ded129700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6dd0003a90 0x7f6de80c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6ded129700 1 -- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de815ce20 msgr2=0x7f6de815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6ded129700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de815ce20 0x7f6de815d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6ded129700 1 -- 172.21.15.3:0/2177528768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de813cfd0 con 0x7f6de80c71e0 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6dee12b700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de815ce20 0x7f6de815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6ded129700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6de80c71e0 0x7f6de815c8e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0073c0 tx=0x7f6ddc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6de5ffb700 1 -- 172.21.15.3:0/2177528768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc018030 con 0x7f6de80c71e0 2023-10-06T04:53:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6de5ffb700 1 -- 172.21.15.3:0/2177528768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00f870 con 0x7f6de80c71e0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de80c76f0 con 0x7f6de80c71e0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6de5ffb700 1 -- 172.21.15.3:0/2177528768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00fc10 con 0x7f6de80c71e0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.829+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de8147810 con 0x7f6de80c71e0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.830+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de8147670 con 0x7f6de80c71e0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.830+0000 7f6de5ffb700 1 -- 172.21.15.3:0/2177528768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ddc01b070 con 0x7f6de80c71e0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.831+0000 7f6de5ffb700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.831+0000 7f6de5ffb700 1 -- 172.21.15.3:0/2177528768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6ddc019070 con 0x7f6de80c71e0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.831+0000 7f6ded92a700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.832+0000 7f6ded92a700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800afd0 tx=0x7f6dd801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:46.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:46.835+0000 7f6de5ffb700 1 -- 172.21.15.3:0/2177528768 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6de8147670 con 0x7f6de80c71e0 2023-10-06T04:53:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.026+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6de8147670 con 0x7f6de80c71e0 2023-10-06T04:53:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.027+0000 7f6de5ffb700 1 -- 172.21.15.3:0/2177528768 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6de8147670 con 0x7f6de80c71e0 2023-10-06T04:53:47.029 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:53:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6dd4066f30 msgr2=0x7f6dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800afd0 tx=0x7f6dd801a000 comp rx=0 tx=0).stop 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6de80c71e0 msgr2=0x7f6de815c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6de80c71e0 0x7f6de815c8e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0073c0 tx=0x7f6ddc001e70 comp rx=0 tx=0).stop 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 shutdown_connections 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de815ce20 0x7f6de815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6dd0003a90 0x7f6de80c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 --2- 172.21.15.3:0/2177528768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6de80c71e0 0x7f6de815c8e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 >> 172.21.15.3:0/2177528768 conn(0x7f6de80ce210 msgr2=0x7f6de813fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 shutdown_connections 2023-10-06T04:53:47.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.029+0000 7f6defb8e700 1 -- 172.21.15.3:0/2177528768 wait complete. 2023-10-06T04:53:47.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:53:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.160+0000 7fccd976f700 1 Processor -- start 2023-10-06T04:53:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.160+0000 7fccd976f700 1 -- start start 2023-10-06T04:53:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd976f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd415ccc0 0x7fccd40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd976f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fccd40d06a0 0x7fccd40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd976f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40cf6c0 con 0x7fccd415c8f0 2023-10-06T04:53:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd976f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd40cfb50 con 0x7fccd415ccc0 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd976f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd40d0b00 con 0x7fccd40d06a0 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd415ccc0 0x7fccd40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fccd415c8f0 0x7fccd415db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59784/0 (socket says 172.21.15.3:59784) 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd2ffd700 1 -- 172.21.15.3:0/2396821754 learned_addr learned my addr 172.21.15.3:0/2396821754 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd37fe700 1 --2- 172.21.15.3:0/2396821754 >> v2:172.21.15.153:3300/0 conn(0x7fccd40d06a0 0x7fccd40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd27fc700 1 -- 172.21.15.3:0/2396821754 >> v2:172.21.15.153:3300/0 conn(0x7fccd40d06a0 msgr2=0x7fccd40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd27fc700 1 --2- 172.21.15.3:0/2396821754 >> v2:172.21.15.153:3300/0 conn(0x7fccd40d06a0 0x7fccd40b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd27fc700 1 -- 172.21.15.3:0/2396821754 >> v1:172.21.15.3:6789/0 conn(0x7fccd415c8f0 legacy=0x7fccd415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd27fc700 1 -- 172.21.15.3:0/2396821754 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40ceab0 con 0x7fccd415ccc0 2023-10-06T04:53:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd1ffb700 1 -- 172.21.15.3:0/2396821754 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922855411 0 0) 0x7fccd40cf6c0 con 0x7fccd415c8f0 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.161+0000 7fccd37fe700 1 --2- 172.21.15.3:0/2396821754 >> v2:172.21.15.153:3300/0 conn(0x7fccd40d06a0 0x7fccd40b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd27fc700 1 --2- 172.21.15.3:0/2396821754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415ccc0 0x7fccd40d0160 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fccd40b3240 tx=0x7fccbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157151f22fb11f3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd1ffb700 1 -- 172.21.15.3:0/2396821754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00b6b0 con 0x7fccd415ccc0 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd1ffb700 1 -- 172.21.15.3:0/2396821754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc00b810 con 0x7fccd415ccc0 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd1ffb700 1 -- 172.21.15.3:0/2396821754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00bbb0 con 0x7fccd415ccc0 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 -- 172.21.15.3:0/2396821754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415ccc0 msgr2=0x7fccd40d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 --2- 172.21.15.3:0/2396821754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415ccc0 0x7fccd40d0160 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fccd40b3240 tx=0x7fccbc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 -- 172.21.15.3:0/2396821754 shutdown_connections 2023-10-06T04:53:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 --2- 172.21.15.3:0/2396821754 >> v2:172.21.15.153:3300/0 conn(0x7fccd40d06a0 0x7fccd40b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 --2- 172.21.15.3:0/2396821754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415ccc0 0x7fccd40d0160 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 -- 172.21.15.3:0/2396821754 >> 172.21.15.3:0/2396821754 conn(0x7fccd4070310 msgr2=0x7fccd4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 -- 172.21.15.3:0/2396821754 shutdown_connections 2023-10-06T04:53:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.162+0000 7fccd976f700 1 -- 172.21.15.3:0/2396821754 wait complete. 2023-10-06T04:53:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.163+0000 7fccd976f700 1 Processor -- start 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd976f700 1 -- start start 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd976f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd40d06a0 0x7fccd4140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd976f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415c8f0 0x7fccd40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd976f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd415ccc0 0x7fccd40b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd976f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40c57e0 con 0x7fccd415c8f0 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd976f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd416a3b0 con 0x7fccd415ccc0 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd976f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40b5570 con 0x7fccd40d06a0 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.164+0000 7fccd27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415c8f0 0x7fccd40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415c8f0 0x7fccd40b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41840/0 (socket says 172.21.15.3:41840) 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 -- 172.21.15.3:0/828923137 learned_addr learned my addr 172.21.15.3:0/828923137 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 -- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd40d06a0 msgr2=0x7fccd4140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd37fe700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd415ccc0 0x7fccd40b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd2ffd700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd40d06a0 0x7fccd4140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd40d06a0 0x7fccd4140a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 -- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd415ccc0 msgr2=0x7fccd40b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd415ccc0 0x7fccd40b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 -- 172.21.15.3:0/828923137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc009420 con 0x7fccd415c8f0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd27fc700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415c8f0 0x7fccd40b3ba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fccd406eaa0 tx=0x7fccbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fcccaffd700 1 -- 172.21.15.3:0/828923137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc011420 con 0x7fccd415c8f0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fcccaffd700 1 -- 172.21.15.3:0/828923137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc011580 con 0x7fccd415c8f0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd416a930 con 0x7fccd415c8f0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fcccaffd700 1 -- 172.21.15.3:0/828923137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc011940 con 0x7fccd415c8f0 2023-10-06T04:53:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.165+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd40c5500 con 0x7fccd415c8f0 2023-10-06T04:53:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.166+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd416aac0 con 0x7fccd415c8f0 2023-10-06T04:53:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.167+0000 7fcccaffd700 1 -- 172.21.15.3:0/828923137 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fccbc011d30 con 0x7fccd415c8f0 2023-10-06T04:53:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.167+0000 7fcccaffd700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccb4066f60 0x7fccb4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.167+0000 7fcccaffd700 1 -- 172.21.15.3:0/828923137 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fccbc0859f0 con 0x7fccd415c8f0 2023-10-06T04:53:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.167+0000 7fccd2ffd700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccb4066f60 0x7fccb4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.168+0000 7fccd2ffd700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccb4066f60 0x7fccb4069410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fccd40b64c0 tx=0x7fccc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.171+0000 7fcccaffd700 1 -- 172.21.15.3:0/828923137 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccbc052e50 con 0x7fccd415c8f0 2023-10-06T04:53:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.368+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fccd4071680 con 0x7fccd415c8f0 2023-10-06T04:53:47.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.368+0000 7fcccaffd700 1 -- 172.21.15.3:0/828923137 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+126 (secure 0 0 0) 0x7fccbc028090 con 0x7fccd415c8f0 2023-10-06T04:53:47.370 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:47.370 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:53:47.370 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:47.370 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:53:47.371 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:53:47.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:53:47.371 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:53:47.371 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:53:47.371 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:53:47.371 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:53:47.371 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:53:47.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccb4066f60 msgr2=0x7fccb4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccb4066f60 0x7fccb4069410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fccd40b64c0 tx=0x7fccc4009040 comp rx=0 tx=0).stop 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415c8f0 msgr2=0x7fccd40b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415c8f0 0x7fccd40b3ba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fccd406eaa0 tx=0x7fccbc017000 comp rx=0 tx=0).stop 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 shutdown_connections 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccb4066f60 0x7fccb4069410 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd415ccc0 0x7fccd40b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd40d06a0 0x7fccd4140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 --2- 172.21.15.3:0/828923137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415c8f0 0x7fccd40b3ba0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 >> 172.21.15.3:0/828923137 conn(0x7fccd4070310 msgr2=0x7fccd41567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 shutdown_connections 2023-10-06T04:53:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.371+0000 7fccd976f700 1 -- 172.21.15.3:0/828923137 wait complete. 2023-10-06T04:53:47.384 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:53:47.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:53:47.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.506+0000 7fa52d671700 1 Processor -- start 2023-10-06T04:53:47.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.506+0000 7fa52d671700 1 -- start start 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa52d671700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52815acc0 0x7fa528069070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa52d671700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52806a080 con 0x7fa528068140 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa52d671700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa528069da0 con 0x7fa528067d70 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa52d671700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280b1650 con 0x7fa52815acc0 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa5267fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa528068140 0x7fa52815a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59790/0 (socket says 172.21.15.3:59790) 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa5267fc700 1 -- 172.21.15.3:0/568649807 learned_addr learned my addr 172.21.15.3:0/568649807 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa5277fe700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52815acc0 0x7fa528069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.507+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309686899 0 0) 0x7fa52806a080 con 0x7fa528068140 2023-10-06T04:53:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa514003570 con 0x7fa528068140 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316942491 0 0) 0x7fa528069da0 con 0x7fa528067d70 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa52806a080 con 0x7fa528067d70 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1139116257 0 0) 0x7fa514003570 con 0x7fa528068140 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52815acc0 msgr2=0x7fa528069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52815acc0 0x7fa528069070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 >> v1:172.21.15.153:6789/0 conn(0x7fa528067d70 legacy=0x7fa5280b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5280b1930 con 0x7fa528068140 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fa510001470 con 0x7fa528068140 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 >> v1:172.21.15.3:6789/0 conn(0x7fa528068140 legacy=0x7fa52815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa514003c20 0x7fa514004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa514004810 0x7fa514004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5140051c0 0x7fa51400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280b1650 con 0x7fa5140051c0 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51400c080 con 0x7fa514003c20 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51400c240 con 0x7fa514004810 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5277fe700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa52815acc0 0x7fa528069070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5277fe700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5140051c0 0x7fa51400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa526ffd700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa514003c20 0x7fa514004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5267fc700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa514004810 0x7fa514004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5277fe700 1 -- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa514004810 msgr2=0x7fa514004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5277fe700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa514004810 0x7fa514004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5277fe700 1 -- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa514003c20 msgr2=0x7fa514004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5277fe700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa514003c20 0x7fa514004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.508+0000 7fa5277fe700 1 -- 172.21.15.3:0/568649807 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa514003570 con 0x7fa5140051c0 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa5277fe700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5140051c0 0x7fa51400ba60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa52815b090 tx=0x7fa52000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f298af4646a68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000eaa0 con 0x7fa5140051c0 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52000f040 con 0x7fa5140051c0 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa525ffb700 1 -- 172.21.15.3:0/568649807 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000ee30 con 0x7fa5140051c0 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa52d671700 1 -- 172.21.15.3:0/568649807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5140051c0 msgr2=0x7fa51400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa52d671700 1 --2- 172.21.15.3:0/568649807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5140051c0 0x7fa51400ba60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa52815b090 tx=0x7fa52000b040 comp rx=0 tx=0).stop 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa5277fe700 1 -- 172.21.15.3:0/568649807 reap_dead start 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa52d671700 1 -- 172.21.15.3:0/568649807 shutdown_connections 2023-10-06T04:53:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa52d671700 1 -- 172.21.15.3:0/568649807 >> 172.21.15.3:0/568649807 conn(0x7fa5280ce280 msgr2=0x7fa5280ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa52d671700 1 -- 172.21.15.3:0/568649807 shutdown_connections 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.509+0000 7fa52d671700 1 -- 172.21.15.3:0/568649807 wait complete. 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.510+0000 7fa52d671700 1 Processor -- start 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.510+0000 7fa52d671700 1 -- start start 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa52d671700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5140051c0 0x7fa5281663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa52d671700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5281685c0 0x7fa52806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa52d671700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528168a30 0x7fa52806d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa52d671700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280af140 con 0x7fa528168a30 2023-10-06T04:53:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa52d671700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52813dc80 con 0x7fa5140051c0 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa52d671700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280ae530 con 0x7fa5281685c0 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa5277fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528168a30 0x7fa52806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa5277fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528168a30 0x7fa52806d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41852/0 (socket says 172.21.15.3:41852) 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa5277fe700 1 -- 172.21.15.3:0/264500874 learned_addr learned my addr 172.21.15.3:0/264500874 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa526ffd700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5140051c0 0x7fa5281663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.511+0000 7fa5267fc700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5281685c0 0x7fa52806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa5277fe700 1 -- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5281685c0 msgr2=0x7fa52806d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa5277fe700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5281685c0 0x7fa52806d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa5277fe700 1 -- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5140051c0 msgr2=0x7fa5281663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa5277fe700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5140051c0 0x7fa5281663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa5277fe700 1 -- 172.21.15.3:0/264500874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5280ae810 con 0x7fa528168a30 2023-10-06T04:53:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa5277fe700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528168a30 0x7fa52806d920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa520015000 tx=0x7fa520016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa51effd700 1 -- 172.21.15.3:0/264500874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520014d50 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa51effd700 1 -- 172.21.15.3:0/264500874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa520028470 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5280aeaf0 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa51effd700 1 -- 172.21.15.3:0/264500874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520028830 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.512+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa528166930 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.513+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5280af280 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.513+0000 7fa51effd700 1 -- 172.21.15.3:0/264500874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa520025070 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.514+0000 7fa51effd700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa508066f30 0x7fa5080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.514+0000 7fa51effd700 1 -- 172.21.15.3:0/264500874 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa520086470 con 0x7fa528168a30 2023-10-06T04:53:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.514+0000 7fa526ffd700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa508066f30 0x7fa5080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.514+0000 7fa526ffd700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa508066f30 0x7fa5080693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa518007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.518+0000 7fa51effd700 1 -- 172.21.15.3:0/264500874 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5280af280 con 0x7fa528168a30 2023-10-06T04:53:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:47.713+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa5280af280 con 0x7fa528168a30 2023-10-06T04:53:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.583+0000 7fa51effd700 1 -- 172.21.15.3:0/264500874 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa5280af280 con 0x7fa528168a30 2023-10-06T04:53:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa508066f30 msgr2=0x7fa5080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa508066f30 0x7fa5080693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa518007d10 comp rx=0 tx=0).stop 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528168a30 msgr2=0x7fa52806d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528168a30 0x7fa52806d920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa520015000 tx=0x7fa520016000 comp rx=0 tx=0).stop 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 shutdown_connections 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa508066f30 0x7fa5080693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5140051c0 0x7fa5281663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528168a30 0x7fa52806d920 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.588+0000 7fa52d671700 1 --2- 172.21.15.3:0/264500874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5281685c0 0x7fa52806d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.589+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 >> 172.21.15.3:0/264500874 conn(0x7fa5280ce280 msgr2=0x7fa5281562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.590+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 shutdown_connections 2023-10-06T04:53:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.590+0000 7fa52d671700 1 -- 172.21.15.3:0/264500874 wait complete. 2023-10-06T04:53:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:53:48.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.739+0000 7f473d58e700 1 Processor -- start 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.739+0000 7f473d58e700 1 -- start start 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.739+0000 7f473d58e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 0x7f4738150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.739+0000 7f473d58e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f47380cba50 0x7f473815d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.739+0000 7f473d58e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473815da80 con 0x7f473815a720 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.739+0000 7f473d58e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473815a290 con 0x7f47380cba50 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.739+0000 7f473d58e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380d0020 con 0x7f4738150690 2023-10-06T04:53:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f47377fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f473815a720 0x7f47380af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59804/0 (socket says 172.21.15.3:59804) 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f47377fe700 1 -- 172.21.15.3:0/178076266 learned_addr learned my addr 172.21.15.3:0/178076266 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f47367fc700 1 --2- 172.21.15.3:0/178076266 >> v2:172.21.15.153:3300/0 conn(0x7f47380cba50 0x7f473815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158518228 0 0) 0x7f473815da80 con 0x7f473815a720 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4724003570 con 0x7f473815a720 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4736ffd700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 0x7f4738150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041434327 0 0) 0x7f4724003570 con 0x7f473815a720 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 msgr2=0x7f4738150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 0x7f4738150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 >> v2:172.21.15.153:3300/0 conn(0x7f47380cba50 msgr2=0x7f473815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 --2- 172.21.15.3:0/178076266 >> v2:172.21.15.153:3300/0 conn(0x7f47380cba50 0x7f473815d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738159f40 con 0x7f473815a720 2023-10-06T04:53:48.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f47300017f0 con 0x7f473815a720 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 >> v1:172.21.15.3:6789/0 conn(0x7f473815a720 legacy=0x7f47380af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724003a60 0x7f4724003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f4724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 0x7f47240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473815a290 con 0x7f4724004810 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380d0020 con 0x7f4724003a60 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472400ba60 con 0x7f4738150690 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.740+0000 7f47367fc700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f4724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f4736ffd700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724003a60 0x7f4724003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f47377fe700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 0x7f47240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f47367fc700 1 -- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 msgr2=0x7f47240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f47367fc700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150690 0x7f47240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f47367fc700 1 -- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724003a60 msgr2=0x7f4724003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f47367fc700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724003a60 0x7f4724003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f47367fc700 1 -- 172.21.15.3:0/178076266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724003570 con 0x7f4724004810 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f47367fc700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f4724004c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4720009c30 tx=0x7f47200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0adf1e2d5127b29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000e710 con 0x7f4724004810 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472000e870 con 0x7f4724004810 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.741+0000 7f4735ffb700 1 -- 172.21.15.3:0/178076266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000ec30 con 0x7f4724004810 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.742+0000 7f473d58e700 1 -- 172.21.15.3:0/178076266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 msgr2=0x7f4724004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.742+0000 7f473d58e700 1 --2- 172.21.15.3:0/178076266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f4724004c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4720009c30 tx=0x7f47200091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.742+0000 7f47377fe700 1 -- 172.21.15.3:0/178076266 reap_dead start 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.742+0000 7f473d58e700 1 -- 172.21.15.3:0/178076266 shutdown_connections 2023-10-06T04:53:48.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.742+0000 7f473d58e700 1 -- 172.21.15.3:0/178076266 >> 172.21.15.3:0/178076266 conn(0x7f4738070310 msgr2=0x7f4738070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.742+0000 7f473d58e700 1 -- 172.21.15.3:0/178076266 shutdown_connections 2023-10-06T04:53:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.742+0000 7f473d58e700 1 -- 172.21.15.3:0/178076266 wait complete. 2023-10-06T04:53:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.743+0000 7f473d58e700 1 Processor -- start 2023-10-06T04:53:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.743+0000 7f473d58e700 1 -- start start 2023-10-06T04:53:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f47380b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150a60 0x7f47380b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4738150ed0 0x7f47381525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380b3780 con 0x7f4724004810 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381513b0 con 0x7f4738150ed0 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4738151690 con 0x7f4738150a60 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f47380b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f47380b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41868/0 (socket says 172.21.15.3:41868) 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 -- 172.21.15.3:0/3319516974 learned_addr learned my addr 172.21.15.3:0/3319516974 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f47377fe700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4738150ed0 0x7f47381525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f47367fc700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150a60 0x7f47380b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 -- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150a60 msgr2=0x7f47380b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150a60 0x7f47380b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 -- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4738150ed0 msgr2=0x7f47381525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4738150ed0 0x7f47381525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 -- 172.21.15.3:0/3319516974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738151970 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f47377fe700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4738150ed0 0x7f47381525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f4736ffd700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f47380b4ea0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4728011d40 tx=0x7f47280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f472effd700 1 -- 172.21.15.3:0/3319516974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800fac0 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f472effd700 1 -- 172.21.15.3:0/3319516974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472800fc20 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47380b3410 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f472effd700 1 -- 172.21.15.3:0/3319516974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728014440 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.744+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380af740 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.746+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473804c380 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.746+0000 7f472effd700 1 -- 172.21.15.3:0/3319516974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4728014a80 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.746+0000 7f472effd700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4718066f30 0x7f47180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.746+0000 7f472effd700 1 -- 172.21.15.3:0/3319516974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f4728018030 con 0x7f4724004810 2023-10-06T04:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.747+0000 7f47367fc700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4718066f30 0x7f47180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.747+0000 7f47367fc700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4718066f30 0x7f47180693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f47200091e0 tx=0x7f472001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.751+0000 7f472effd700 1 -- 172.21.15.3:0/3319516974 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4728052460 con 0x7f4724004810 2023-10-06T04:53:48.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:48.944+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f473815dcb0 con 0x7f4724004810 2023-10-06T04:53:49.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.588+0000 7f472effd700 1 -- 172.21.15.3:0/3319516974 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) v1 ==== 170+0+0 (secure 0 0 0) 0x7f47280220d0 con 0x7f4724004810 2023-10-06T04:53:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4718066f30 msgr2=0x7f47180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4718066f30 0x7f47180693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f47200091e0 tx=0x7f472001a000 comp rx=0 tx=0).stop 2023-10-06T04:53:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 msgr2=0x7f47380b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f47380b4ea0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4728011d40 tx=0x7f47280091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 shutdown_connections 2023-10-06T04:53:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4718066f30 0x7f47180693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4738150ed0 0x7f47381525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4724004810 0x7f47380b4ea0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 --2- 172.21.15.3:0/3319516974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4738150a60 0x7f47380b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.594+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 >> 172.21.15.3:0/3319516974 conn(0x7f4738070310 msgr2=0x7f4738070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:49.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.597+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 shutdown_connections 2023-10-06T04:53:49.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.598+0000 7f473d58e700 1 -- 172.21.15.3:0/3319516974 wait complete. 2023-10-06T04:53:49.602 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:53:49.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:53:49.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.743+0000 7fa303167700 1 Processor -- start 2023-10-06T04:53:49.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.743+0000 7fa303167700 1 -- start start 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa303167700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2fc150690 0x7fa2fc150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa303167700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc0cba50 0x7fa2fc15d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa303167700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2fc15a720 0x7fa2fc0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa303167700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2fc15da80 con 0x7fa2fc15a720 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa303167700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2fc15a290 con 0x7fa2fc150690 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa303167700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0d0020 con 0x7fa2fc0cba50 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2fc15a720 0x7fa2fc0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2fc15a720 0x7fa2fc0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41870/0 (socket says 172.21.15.3:41870) 2023-10-06T04:53:49.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 -- 172.21.15.3:0/3996959937 learned_addr learned my addr 172.21.15.3:0/3996959937 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa300f03700 1 --2- 172.21.15.3:0/3996959937 >> v2:172.21.15.153:3300/0 conn(0x7fa2fc150690 0x7fa2fc150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa2fbfff700 1 --2- 172.21.15.3:0/3996959937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc0cba50 0x7fa2fc15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 -- 172.21.15.3:0/3996959937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc0cba50 msgr2=0x7fa2fc15d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 --2- 172.21.15.3:0/3996959937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc0cba50 0x7fa2fc15d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 -- 172.21.15.3:0/3996959937 >> v2:172.21.15.153:3300/0 conn(0x7fa2fc150690 msgr2=0x7fa2fc150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 --2- 172.21.15.3:0/3996959937 >> v2:172.21.15.153:3300/0 conn(0x7fa2fc150690 0x7fa2fc150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 -- 172.21.15.3:0/3996959937 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc159f40 con 0x7fa2fc15a720 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.744+0000 7fa301704700 1 --2- 172.21.15.3:0/3996959937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc15a720 0x7fa2fc0af190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0cdf80 tx=0x7fa2ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00e38c8ab2dc53c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa2fb7fe700 1 -- 172.21.15.3:0/3996959937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00b860 con 0x7fa2fc15a720 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa2fb7fe700 1 -- 172.21.15.3:0/3996959937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec00b9c0 con 0x7fa2fc15a720 2023-10-06T04:53:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa2fb7fe700 1 -- 172.21.15.3:0/3996959937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00bd60 con 0x7fa2fc15a720 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 -- 172.21.15.3:0/3996959937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc15a720 msgr2=0x7fa2fc0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 --2- 172.21.15.3:0/3996959937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc15a720 0x7fa2fc0af190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0cdf80 tx=0x7fa2ec009250 comp rx=0 tx=0).stop 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 -- 172.21.15.3:0/3996959937 shutdown_connections 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 --2- 172.21.15.3:0/3996959937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc15a720 0x7fa2fc0af190 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 --2- 172.21.15.3:0/3996959937 >> v2:172.21.15.153:3300/0 conn(0x7fa2fc150690 0x7fa2fc150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 --2- 172.21.15.3:0/3996959937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc0cba50 0x7fa2fc15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 -- 172.21.15.3:0/3996959937 >> 172.21.15.3:0/3996959937 conn(0x7fa2fc070310 msgr2=0x7fa2fc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 -- 172.21.15.3:0/3996959937 shutdown_connections 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.745+0000 7fa303167700 1 -- 172.21.15.3:0/3996959937 wait complete. 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.746+0000 7fa303167700 1 Processor -- start 2023-10-06T04:53:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.746+0000 7fa303167700 1 -- start start 2023-10-06T04:53:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa303167700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2fc0cba50 0x7fa2fc14f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa303167700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc150690 0x7fa2fc0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa303167700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc15a720 0x7fa2fc150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa303167700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc153a10 con 0x7fa2fc150690 2023-10-06T04:53:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa303167700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc150370 con 0x7fa2fc0cba50 2023-10-06T04:53:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa303167700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc153170 con 0x7fa2fc15a720 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa2fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc150690 0x7fa2fc0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa2fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc150690 0x7fa2fc0b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41874/0 (socket says 172.21.15.3:41874) 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa2fbfff700 1 -- 172.21.15.3:0/1495306286 learned_addr learned my addr 172.21.15.3:0/1495306286 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa300f03700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2fc0cba50 0x7fa2fc14f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.747+0000 7fa301704700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc15a720 0x7fa2fc150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2fbfff700 1 -- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc15a720 msgr2=0x7fa2fc150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2fbfff700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc15a720 0x7fa2fc150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2fbfff700 1 -- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2fc0cba50 msgr2=0x7fa2fc14f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2fbfff700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2fc0cba50 0x7fa2fc14f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2fbfff700 1 -- 172.21.15.3:0/1495306286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0094f0 con 0x7fa2fc150690 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2fbfff700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc150690 0x7fa2fc0b3e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d890 tx=0x7fa2f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2f8ff9700 1 -- 172.21.15.3:0/1495306286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00125f0 con 0x7fa2fc150690 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2f8ff9700 1 -- 172.21.15.3:0/1495306286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0012750 con 0x7fa2fc150690 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa2f8ff9700 1 -- 172.21.15.3:0/1495306286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0012af0 con 0x7fa2fc150690 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc153450 con 0x7fa2fc150690 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.748+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc0b0cf0 con 0x7fa2fc150690 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.749+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc04c380 con 0x7fa2fc150690 2023-10-06T04:53:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.750+0000 7fa2f8ff9700 1 -- 172.21.15.3:0/1495306286 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2f001e460 con 0x7fa2fc150690 2023-10-06T04:53:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.750+0000 7fa2f8ff9700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa2e8066f30 0x7fa2e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.750+0000 7fa2f8ff9700 1 -- 172.21.15.3:0/1495306286 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fa2f0085630 con 0x7fa2fc150690 2023-10-06T04:53:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.750+0000 7fa300f03700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa2e8066f30 0x7fa2e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.751+0000 7fa300f03700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa2e8066f30 0x7fa2e80693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fd0 tx=0x7fa2f4009880 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.754+0000 7fa2f8ff9700 1 -- 172.21.15.3:0/1495306286 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2f0052820 con 0x7fa2fc150690 2023-10-06T04:53:49.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:49.995+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa2fc15dcb0 con 0x7fa2fc150690 2023-10-06T04:53:50.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.591+0000 7fa2f8ff9700 1 -- 172.21.15.3:0/1495306286 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa2f0055e40 con 0x7fa2fc150690 2023-10-06T04:53:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.599+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa2e8066f30 msgr2=0x7fa2e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.599+0000 7fa303167700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa2e8066f30 0x7fa2e80693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fd0 tx=0x7fa2f4009880 comp rx=0 tx=0).stop 2023-10-06T04:53:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.599+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc150690 msgr2=0x7fa2fc0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.599+0000 7fa303167700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc150690 0x7fa2fc0b3e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d890 tx=0x7fa2f0008040 comp rx=0 tx=0).stop 2023-10-06T04:53:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.605+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 shutdown_connections 2023-10-06T04:53:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.605+0000 7fa303167700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa2e8066f30 0x7fa2e80693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.605+0000 7fa303167700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2fc0cba50 0x7fa2fc14f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.605+0000 7fa303167700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2fc15a720 0x7fa2fc150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.605+0000 7fa303167700 1 --2- 172.21.15.3:0/1495306286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2fc150690 0x7fa2fc0b3e70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.605+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 >> 172.21.15.3:0/1495306286 conn(0x7fa2fc070310 msgr2=0x7fa2fc15cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.610+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 shutdown_connections 2023-10-06T04:53:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.610+0000 7fa303167700 1 -- 172.21.15.3:0/1495306286 wait complete. 2023-10-06T04:53:50.615 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 7 and data pool 8 2023-10-06T04:53:50.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:50.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.749+0000 7f3cbf4a6700 1 Processor -- start 2023-10-06T04:53:50.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.749+0000 7f3cbf4a6700 1 -- start start 2023-10-06T04:53:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbf4a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb80b39d0 0x7f3cb80cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbf4a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3cb80cde10 0x7f3cb81571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbf4a6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb815e4c0 con 0x7f3cb806d480 2023-10-06T04:53:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbf4a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb815d8b0 con 0x7f3cb80cde10 2023-10-06T04:53:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbf4a6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb815db90 con 0x7f3cb80b39d0 2023-10-06T04:53:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbca41700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3cb80cde10 0x7f3cb81571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbca41700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3cb80cde10 0x7f3cb81571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41880/0 (socket says 172.21.15.3:41880) 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.750+0000 7f3cbca41700 1 -- 172.21.15.3:0/337963062 learned_addr learned my addr 172.21.15.3:0/337963062 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbd242700 1 --2- 172.21.15.3:0/337963062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb80b39d0 0x7f3cb80cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbca41700 1 -- 172.21.15.3:0/337963062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb80b39d0 msgr2=0x7f3cb80cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbca41700 1 --2- 172.21.15.3:0/337963062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb80b39d0 0x7f3cb80cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbca41700 1 -- 172.21.15.3:0/337963062 >> v1:172.21.15.153:6789/0 conn(0x7f3cb806d480 legacy=0x7f3cb8069050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbca41700 1 -- 172.21.15.3:0/337963062 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb815de70 con 0x7f3cb80cde10 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbca41700 1 --2- 172.21.15.3:0/337963062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81571d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80099b0 tx=0x7f3ca8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fad6705a3d89e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3caffff700 1 -- 172.21.15.3:0/337963062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800b860 con 0x7f3cb80cde10 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3caffff700 1 -- 172.21.15.3:0/337963062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca800b9c0 con 0x7f3cb80cde10 2023-10-06T04:53:50.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3caffff700 1 -- 172.21.15.3:0/337963062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800bd60 con 0x7f3cb80cde10 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/337963062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 msgr2=0x7f3cb81571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.751+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/337963062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81571d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80099b0 tx=0x7f3ca8009250 comp rx=0 tx=0).stop 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.752+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/337963062 shutdown_connections 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.752+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/337963062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb80b39d0 0x7f3cb80cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.752+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/337963062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81571d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.752+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/337963062 >> 172.21.15.3:0/337963062 conn(0x7f3cb80ce2c0 msgr2=0x7f3cb80ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.752+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/337963062 shutdown_connections 2023-10-06T04:53:50.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.752+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/337963062 wait complete. 2023-10-06T04:53:50.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.753+0000 7f3cbf4a6700 1 Processor -- start 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.753+0000 7f3cbf4a6700 1 -- start start 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb806d480 0x7f3cb814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb80b39d0 0x7f3cb81581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80ae690 con 0x7f3cb80cde10 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80add60 con 0x7f3cb80b39d0 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80ae040 con 0x7f3cb806d480 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41890/0 (socket says 172.21.15.3:41890) 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 -- 172.21.15.3:0/3284921562 learned_addr learned my addr 172.21.15.3:0/3284921562 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 -- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb806d480 msgr2=0x7f3cb814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbd242700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb806d480 0x7f3cb814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbca41700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb80b39d0 0x7f3cb81581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb806d480 0x7f3cb814f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 -- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb80b39d0 msgr2=0x7f3cb81581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb80b39d0 0x7f3cb81581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 -- 172.21.15.3:0/3284921562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca80094f0 con 0x7f3cb80cde10 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbda43700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81541c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c810 tx=0x7f3cb0001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cad7fa700 1 -- 172.21.15.3:0/3284921562 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0012630 con 0x7f3cb80cde10 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cad7fa700 1 -- 172.21.15.3:0/3284921562 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb00127b0 con 0x7f3cb80cde10 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb80ae320 con 0x7f3cb80cde10 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cad7fa700 1 -- 172.21.15.3:0/3284921562 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0012b70 con 0x7f3cb80cde10 2023-10-06T04:53:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.754+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb814fd40 con 0x7f3cb80cde10 2023-10-06T04:53:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.755+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb80644c0 con 0x7f3cb80cde10 2023-10-06T04:53:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.756+0000 7f3cad7fa700 1 -- 172.21.15.3:0/3284921562 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cb0003030 con 0x7f3cb80cde10 2023-10-06T04:53:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.756+0000 7f3cad7fa700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c9c066f30 0x7f3c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.756+0000 7f3cad7fa700 1 -- 172.21.15.3:0/3284921562 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3cb0084630 con 0x7f3cb80cde10 2023-10-06T04:53:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.756+0000 7f3cbd242700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c9c066f30 0x7f3c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.757+0000 7f3cbd242700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c9c066f30 0x7f3c9c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006fd0 tx=0x7f3cb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:50.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.760+0000 7f3cad7fa700 1 -- 172.21.15.3:0/3284921562 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb0051740 con 0x7f3cb80cde10 2023-10-06T04:53:50.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.955+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cb8071680 con 0x7f3cb80cde10 2023-10-06T04:53:50.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.956+0000 7f3cad7fa700 1 -- 172.21.15.3:0/3284921562 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f3cb0054d60 con 0x7f3cb80cde10 2023-10-06T04:53:50.957 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:50.958 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:49.996758+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:47.743195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:48.974063+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:50.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.957+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c9c066f30 msgr2=0x7f3c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.957+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c9c066f30 0x7f3c9c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006fd0 tx=0x7f3cb4008040 comp rx=0 tx=0).stop 2023-10-06T04:53:50.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.957+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 msgr2=0x7f3cb81541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.957+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81541c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c810 tx=0x7f3cb0001650 comp rx=0 tx=0).stop 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 shutdown_connections 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c9c066f30 0x7f3c9c0693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb80cde10 0x7f3cb81541c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb806d480 0x7f3cb814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 --2- 172.21.15.3:0/3284921562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb80b39d0 0x7f3cb81581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 >> 172.21.15.3:0/3284921562 conn(0x7f3cb80ce2c0 msgr2=0x7f3cb806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 shutdown_connections 2023-10-06T04:53:50.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:50.958+0000 7f3cbf4a6700 1 -- 172.21.15.3:0/3284921562 wait complete. 2023-10-06T04:53:50.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:51.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.088+0000 7fa442379700 1 Processor -- start 2023-10-06T04:53:51.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.088+0000 7fa442379700 1 -- start start 2023-10-06T04:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa442379700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43c159350 0x7fa43c159720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa442379700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa43c0af050 0x7fa43c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa442379700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43c0af9d0 con 0x7fa43c0cac60 2023-10-06T04:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa442379700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c15e080 con 0x7fa43c159350 2023-10-06T04:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa442379700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c15d2c0 con 0x7fa43c0af050 2023-10-06T04:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa441377700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43c159350 0x7fa43c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa441377700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43c159350 0x7fa43c159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41900/0 (socket says 172.21.15.3:41900) 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa441377700 1 -- 172.21.15.3:0/1372015037 learned_addr learned my addr 172.21.15.3:0/1372015037 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.089+0000 7fa441b78700 1 --2- 172.21.15.3:0/1372015037 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0af050 0x7fa43c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa441377700 1 -- 172.21.15.3:0/1372015037 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0af050 msgr2=0x7fa43c0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa441377700 1 --2- 172.21.15.3:0/1372015037 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0af050 0x7fa43c0af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa441377700 1 -- 172.21.15.3:0/1372015037 >> v1:172.21.15.153:6789/0 conn(0x7fa43c0cac60 legacy=0x7fa43c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa441377700 1 -- 172.21.15.3:0/1372015037 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c15d5a0 con 0x7fa43c159350 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa441377700 1 --2- 172.21.15.3:0/1372015037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c159350 0x7fa43c159720 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa43c089340 tx=0x7fa434009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4d76bdc629a8c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa43bfff700 1 -- 172.21.15.3:0/1372015037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400b820 con 0x7fa43c159350 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa43bfff700 1 -- 172.21.15.3:0/1372015037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43400b980 con 0x7fa43c159350 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa43bfff700 1 -- 172.21.15.3:0/1372015037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400bd20 con 0x7fa43c159350 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa442379700 1 -- 172.21.15.3:0/1372015037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c159350 msgr2=0x7fa43c159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa442379700 1 --2- 172.21.15.3:0/1372015037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c159350 0x7fa43c159720 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa43c089340 tx=0x7fa434009250 comp rx=0 tx=0).stop 2023-10-06T04:53:51.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa442379700 1 -- 172.21.15.3:0/1372015037 shutdown_connections 2023-10-06T04:53:51.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa442379700 1 --2- 172.21.15.3:0/1372015037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c159350 0x7fa43c159720 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa442379700 1 --2- 172.21.15.3:0/1372015037 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0af050 0x7fa43c0af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa442379700 1 -- 172.21.15.3:0/1372015037 >> 172.21.15.3:0/1372015037 conn(0x7fa43c06ecc0 msgr2=0x7fa43c06f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:51.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.090+0000 7fa442379700 1 -- 172.21.15.3:0/1372015037 shutdown_connections 2023-10-06T04:53:51.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.091+0000 7fa442379700 1 -- 172.21.15.3:0/1372015037 wait complete. 2023-10-06T04:53:51.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 Processor -- start 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 -- start start 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0af050 0x7fa43c0b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c0cac60 0x7fa43c15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c159350 0x7fa43c148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c15ae00 con 0x7fa43c0af050 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c169e20 con 0x7fa43c159350 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.092+0000 7fa442379700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c16a100 con 0x7fa43c0cac60 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0af050 0x7fa43c0b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0af050 0x7fa43c0b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41910/0 (socket says 172.21.15.3:41910) 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 -- 172.21.15.3:0/1765230429 learned_addr learned my addr 172.21.15.3:0/1765230429 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa440b76700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c0cac60 0x7fa43c15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441b78700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c159350 0x7fa43c148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 -- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c0cac60 msgr2=0x7fa43c15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c0cac60 0x7fa43c15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 -- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c159350 msgr2=0x7fa43c148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c159350 0x7fa43c148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 -- 172.21.15.3:0/1765230429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340094f0 con 0x7fa43c0af050 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa441377700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0af050 0x7fa43c0b3070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa434009290 tx=0x7fa434016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa4397fa700 1 -- 172.21.15.3:0/1765230429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434011e20 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa4397fa700 1 -- 172.21.15.3:0/1765230429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43401b680 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c16a3e0 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa4397fa700 1 -- 172.21.15.3:0/1765230429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43401ba40 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.093+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c1493e0 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.094+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c04c380 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.095+0000 7fa4397fa700 1 -- 172.21.15.3:0/1765230429 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa43401c450 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.095+0000 7fa4397fa700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.095+0000 7fa4397fa700 1 -- 172.21.15.3:0/1765230429 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4340862d0 con 0x7fa43c0af050 2023-10-06T04:53:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.095+0000 7fa440b76700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.096+0000 7fa440b76700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0cf220 tx=0x7fa430009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.099+0000 7fa4397fa700 1 -- 172.21.15.3:0/1765230429 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4340533e0 con 0x7fa43c0af050 2023-10-06T04:53:51.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.296+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa43c15d880 con 0x7fa43c0af050 2023-10-06T04:53:51.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.297+0000 7fa4397fa700 1 -- 172.21.15.3:0/1765230429 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16831 (secure 0 0 0) 0x7fa434056a00 con 0x7fa43c0af050 2023-10-06T04:53:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:51.299 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:49.996758+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:47.743195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:48.974063+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 msgr2=0x7fa4280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0cf220 tx=0x7fa430009040 comp rx=0 tx=0).stop 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0af050 msgr2=0x7fa43c0b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0af050 0x7fa43c0b3070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa434009290 tx=0x7fa434016000 comp rx=0 tx=0).stop 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 shutdown_connections 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c159350 0x7fa43c148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0af050 0x7fa43c0b3070 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 --2- 172.21.15.3:0/1765230429 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c0cac60 0x7fa43c15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 >> 172.21.15.3:0/1765230429 conn(0x7fa43c06ecc0 msgr2=0x7fa43c156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:51.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 shutdown_connections 2023-10-06T04:53:51.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.299+0000 7fa442379700 1 -- 172.21.15.3:0/1765230429 wait complete. 2023-10-06T04:53:51.313 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.433+0000 7fc7225ad700 1 Processor -- start 2023-10-06T04:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.433+0000 7fc7225ad700 1 -- start start 2023-10-06T04:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.433+0000 7fc7225ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc71c15cb30 0x7fc71c0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.433+0000 7fc7225ad700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0cf0f0 con 0x7fc71c15c760 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.433+0000 7fc7225ad700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0ce7c0 con 0x7fc71c0afec0 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.433+0000 7fc7225ad700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71c15cf80 con 0x7fc71c15cb30 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc71c15cb30 0x7fc71c0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc71c15c760 0x7fc71c15db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59820/0 (socket says 172.21.15.3:59820) 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71bfff700 1 -- 172.21.15.3:0/1329270447 learned_addr learned my addr 172.21.15.3:0/1329270447 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71b7fe700 1 -- 172.21.15.3:0/1329270447 >> v1:172.21.15.153:6789/0 conn(0x7fc71c0afec0 legacy=0x7fc71c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71b7fe700 1 -- 172.21.15.3:0/1329270447 >> v1:172.21.15.3:6789/0 conn(0x7fc71c15c760 legacy=0x7fc71c15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71b7fe700 1 -- 172.21.15.3:0/1329270447 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0ceaa0 con 0x7fc71c15cb30 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71b7fe700 1 --2- 172.21.15.3:0/1329270447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c0cb3c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc7100098e0 tx=0x7fc7100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33be78676cd856b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71affd700 1 -- 172.21.15.3:0/1329270447 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710009e80 con 0x7fc71c15cb30 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71affd700 1 -- 172.21.15.3:0/1329270447 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71000b860 con 0x7fc71c15cb30 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.434+0000 7fc71affd700 1 -- 172.21.15.3:0/1329270447 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000bbe0 con 0x7fc71c15cb30 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.435+0000 7fc7225ad700 1 -- 172.21.15.3:0/1329270447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 msgr2=0x7fc71c0cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.435+0000 7fc7225ad700 1 --2- 172.21.15.3:0/1329270447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c0cb3c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc7100098e0 tx=0x7fc7100091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.435+0000 7fc7225ad700 1 -- 172.21.15.3:0/1329270447 shutdown_connections 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.435+0000 7fc7225ad700 1 --2- 172.21.15.3:0/1329270447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c0cb3c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.435+0000 7fc7225ad700 1 -- 172.21.15.3:0/1329270447 >> 172.21.15.3:0/1329270447 conn(0x7fc71c06f940 msgr2=0x7fc71c070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.435+0000 7fc7225ad700 1 -- 172.21.15.3:0/1329270447 shutdown_connections 2023-10-06T04:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.435+0000 7fc7225ad700 1 -- 172.21.15.3:0/1329270447 wait complete. 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.436+0000 7fc7225ad700 1 Processor -- start 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.436+0000 7fc7225ad700 1 -- start start 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc7225ad700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc71c0afec0 0x7fc71c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc7225ad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71c15c760 0x7fc71c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc7225ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc7225ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0c8400 con 0x7fc71c15cb30 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc7225ad700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0c7810 con 0x7fc71c0afec0 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc7225ad700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0c7ad0 con 0x7fc71c15c760 2023-10-06T04:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41936/0 (socket says 172.21.15.3:41936) 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 -- 172.21.15.3:0/4145253731 learned_addr learned my addr 172.21.15.3:0/4145253731 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc71b7fe700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71c15c760 0x7fc71c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc71bfff700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc71c0afec0 0x7fc71c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 -- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71c15c760 msgr2=0x7fc71c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71c15c760 0x7fc71c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 -- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc71c0afec0 msgr2=0x7fc71c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc71c0afec0 0x7fc71c158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 -- 172.21.15.3:0/4145253731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710009420 con 0x7fc71c15cb30 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.437+0000 7fc720b4a700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c15bbf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c840 tx=0x7fc714001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.438+0000 7fc703fff700 1 -- 172.21.15.3:0/4145253731 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714012630 con 0x7fc71c15cb30 2023-10-06T04:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.438+0000 7fc703fff700 1 -- 172.21.15.3:0/4145253731 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7140127b0 con 0x7fc71c15cb30 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.438+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c0c7db0 con 0x7fc71c15cb30 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.438+0000 7fc703fff700 1 -- 172.21.15.3:0/4145253731 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714012b70 con 0x7fc71c15cb30 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.438+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c0c7500 con 0x7fc71c15cb30 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.439+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c0c8570 con 0x7fc71c15cb30 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.439+0000 7fc703fff700 1 -- 172.21.15.3:0/4145253731 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc714003030 con 0x7fc71c15cb30 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.439+0000 7fc703fff700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc708066f30 0x7fc7080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.439+0000 7fc703fff700 1 -- 172.21.15.3:0/4145253731 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc714084650 con 0x7fc71c15cb30 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.440+0000 7fc71bfff700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc708066f30 0x7fc7080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.440+0000 7fc71bfff700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc708066f30 0x7fc7080693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc70c001820 tx=0x7fc70c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.444+0000 7fc703fff700 1 -- 172.21.15.3:0/4145253731 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc714051760 con 0x7fc71c15cb30 2023-10-06T04:53:51.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:51.684+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc71c04c380 con 0x7fc71c15cb30 2023-10-06T04:53:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.625+0000 7fc703fff700 1 -- 172.21.15.3:0/4145253731 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc714054d80 con 0x7fc71c15cb30 2023-10-06T04:53:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc708066f30 msgr2=0x7fc7080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc708066f30 0x7fc7080693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc70c001820 tx=0x7fc70c009040 comp rx=0 tx=0).stop 2023-10-06T04:53:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 msgr2=0x7fc71c15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c15bbf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c840 tx=0x7fc714001670 comp rx=0 tx=0).stop 2023-10-06T04:53:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 shutdown_connections 2023-10-06T04:53:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc708066f30 0x7fc7080693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc71c0afec0 0x7fc71c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71c15cb30 0x7fc71c15bbf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 --2- 172.21.15.3:0/4145253731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71c15c760 0x7fc71c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 >> 172.21.15.3:0/4145253731 conn(0x7fc71c06f940 msgr2=0x7fc71c0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:52.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 shutdown_connections 2023-10-06T04:53:52.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.628+0000 7fc7225ad700 1 -- 172.21.15.3:0/4145253731 wait complete. 2023-10-06T04:53:52.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:53:52.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.771+0000 7fc5265d5700 1 Processor -- start 2023-10-06T04:53:52.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc5265d5700 1 -- start start 2023-10-06T04:53:52.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc5265d5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc520150690 0x7fc520150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:52.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc5265d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:52.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc5265d5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52015da80 con 0x7fc52015a720 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc5265d5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52015a290 con 0x7fc520150690 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc5265d5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5200d0020 con 0x7fc5200cba50 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc51ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc520150690 0x7fc520150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.772+0000 7fc524b72700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc52015a720 0x7fc5200af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48988/0 (socket says 172.21.15.3:48988) 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc524b72700 1 -- 172.21.15.3:0/2349849281 learned_addr learned my addr 172.21.15.3:0/2349849281 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51f7fe700 1 --2- 172.21.15.3:0/2349849281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51ffff700 1 -- 172.21.15.3:0/2349849281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 msgr2=0x7fc52015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51ffff700 1 --2- 172.21.15.3:0/2349849281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51ffff700 1 -- 172.21.15.3:0/2349849281 >> v1:172.21.15.3:6789/0 conn(0x7fc52015a720 legacy=0x7fc5200af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51effd700 1 -- 172.21.15.3:0/2349849281 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144832622 0 0) 0x7fc52015da80 con 0x7fc52015a720 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51ffff700 1 -- 172.21.15.3:0/2349849281 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc520159f40 con 0x7fc520150690 2023-10-06T04:53:52.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51ffff700 1 --2- 172.21.15.3:0/2349849281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc520150a60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098e0 tx=0x7fc5100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f3de76fb568e0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51effd700 1 -- 172.21.15.3:0/2349849281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000b640 con 0x7fc520150690 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51effd700 1 -- 172.21.15.3:0/2349849281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000b7a0 con 0x7fc520150690 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.773+0000 7fc51effd700 1 -- 172.21.15.3:0/2349849281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000bb40 con 0x7fc520150690 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 -- 172.21.15.3:0/2349849281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 msgr2=0x7fc520150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 --2- 172.21.15.3:0/2349849281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc520150a60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098e0 tx=0x7fc5100091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 -- 172.21.15.3:0/2349849281 shutdown_connections 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 --2- 172.21.15.3:0/2349849281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc520150a60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 --2- 172.21.15.3:0/2349849281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 -- 172.21.15.3:0/2349849281 >> 172.21.15.3:0/2349849281 conn(0x7fc520070310 msgr2=0x7fc520070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 -- 172.21.15.3:0/2349849281 shutdown_connections 2023-10-06T04:53:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.774+0000 7fc5265d5700 1 -- 172.21.15.3:0/2349849281 wait complete. 2023-10-06T04:53:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.775+0000 7fc5265d5700 1 Processor -- start 2023-10-06T04:53:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.775+0000 7fc5265d5700 1 -- start start 2023-10-06T04:53:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc52014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc52015a720 0x7fc520158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5201533d0 con 0x7fc520150690 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5201536b0 con 0x7fc52015a720 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc520153990 con 0x7fc5200cba50 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc52014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc52014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40458/0 (socket says 172.21.15.3:40458) 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 -- 172.21.15.3:0/3893293808 learned_addr learned my addr 172.21.15.3:0/3893293808 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51ffff700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc524b72700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc52015a720 0x7fc520158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 -- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 msgr2=0x7fc52015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 -- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc52015a720 msgr2=0x7fc520158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc52015a720 0x7fc520158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 -- 172.21.15.3:0/3893293808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510009420 con 0x7fc520150690 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51ffff700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc524b72700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc52015a720 0x7fc520158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc51f7fe700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc52014fe10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc50800ec80 tx=0x7fc5080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc517fff700 1 -- 172.21.15.3:0/3893293808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50800c9d0 con 0x7fc520150690 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc517fff700 1 -- 172.21.15.3:0/3893293808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50800cb30 con 0x7fc520150690 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52016b200 con 0x7fc520150690 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc517fff700 1 -- 172.21.15.3:0/3893293808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508013470 con 0x7fc520150690 2023-10-06T04:53:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.776+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52015d330 con 0x7fc520150690 2023-10-06T04:53:52.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.777+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52004c380 con 0x7fc520150690 2023-10-06T04:53:52.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.778+0000 7fc517fff700 1 -- 172.21.15.3:0/3893293808 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc508013820 con 0x7fc520150690 2023-10-06T04:53:52.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.778+0000 7fc517fff700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc500066f30 0x7fc5000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:52.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.778+0000 7fc517fff700 1 -- 172.21.15.3:0/3893293808 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc508084a80 con 0x7fc520150690 2023-10-06T04:53:52.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.778+0000 7fc51ffff700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc500066f30 0x7fc5000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:52.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.779+0000 7fc51ffff700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc500066f30 0x7fc5000693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc510003080 tx=0x7fc510017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:52.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:52.782+0000 7fc517fff700 1 -- 172.21.15.3:0/3893293808 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc508051b90 con 0x7fc520150690 2023-10-06T04:53:53.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.027+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc52015dcb0 con 0x7fc520150690 2023-10-06T04:53:53.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.632+0000 7fc517fff700 1 -- 172.21.15.3:0/3893293808 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc5080170c0 con 0x7fc520150690 2023-10-06T04:53:53.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc500066f30 msgr2=0x7fc5000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:53.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc500066f30 0x7fc5000693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc510003080 tx=0x7fc510017000 comp rx=0 tx=0).stop 2023-10-06T04:53:53.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 msgr2=0x7fc52014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:53.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc52014fe10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc50800ec80 tx=0x7fc5080091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 shutdown_connections 2023-10-06T04:53:53.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc500066f30 0x7fc5000693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc52015a720 0x7fc520158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5200cba50 0x7fc52015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 --2- 172.21.15.3:0/3893293808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc520150690 0x7fc52014fe10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 >> 172.21.15.3:0/3893293808 conn(0x7fc520070310 msgr2=0x7fc5200cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:53.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 shutdown_connections 2023-10-06T04:53:53.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.635+0000 7fc5265d5700 1 -- 172.21.15.3:0/3893293808 wait complete. 2023-10-06T04:53:53.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:53:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.778+0000 7fe3a12d9700 1 Processor -- start 2023-10-06T04:53:53.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.778+0000 7fe3a12d9700 1 -- start start 2023-10-06T04:53:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.779+0000 7fe3a12d9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe39c15c8f0 0x7fe39c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.779+0000 7fe3a12d9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c0d0870 0x7fe39c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.779+0000 7fe3a12d9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39c0cf6c0 con 0x7fe39c15ccc0 2023-10-06T04:53:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.779+0000 7fe3a12d9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39c0cfb50 con 0x7fe39c15c8f0 2023-10-06T04:53:53.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.779+0000 7fe3a12d9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c15e050 con 0x7fe39c0d0870 2023-10-06T04:53:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.779+0000 7fe39a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe39c15ccc0 0x7fe39c0d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48992/0 (socket says 172.21.15.3:48992) 2023-10-06T04:53:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.779+0000 7fe39a7fc700 1 -- 172.21.15.3:0/2132354558 learned_addr learned my addr 172.21.15.3:0/2132354558 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe39b7fe700 1 --2- 172.21.15.3:0/2132354558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c0d0870 0x7fe39c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe39affd700 1 --2- 172.21.15.3:0/2132354558 >> v2:172.21.15.153:3300/0 conn(0x7fe39c15c8f0 0x7fe39c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe399ffb700 1 -- 172.21.15.3:0/2132354558 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687205304 0 0) 0x7fe39c0cf6c0 con 0x7fe39c15ccc0 2023-10-06T04:53:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe399ffb700 1 -- 172.21.15.3:0/2132354558 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe388003570 con 0x7fe39c15ccc0 2023-10-06T04:53:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe399ffb700 1 -- 172.21.15.3:0/2132354558 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 765577092 0 0) 0x7fe388003570 con 0x7fe39c15ccc0 2023-10-06T04:53:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe39b7fe700 1 -- 172.21.15.3:0/2132354558 >> v2:172.21.15.153:3300/0 conn(0x7fe39c15c8f0 msgr2=0x7fe39c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe39b7fe700 1 --2- 172.21.15.3:0/2132354558 >> v2:172.21.15.153:3300/0 conn(0x7fe39c15c8f0 0x7fe39c15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe39b7fe700 1 -- 172.21.15.3:0/2132354558 >> v1:172.21.15.3:6789/0 conn(0x7fe39c15ccc0 legacy=0x7fe39c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe39b7fe700 1 -- 172.21.15.3:0/2132354558 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c0ceab0 con 0x7fe39c0d0870 2023-10-06T04:53:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe39b7fe700 1 --2- 172.21.15.3:0/2132354558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c0d0870 0x7fe39c0b3ba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009e60 tx=0x7fe38c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be59e91988487ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe399ffb700 1 -- 172.21.15.3:0/2132354558 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c00e070 con 0x7fe39c0d0870 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe399ffb700 1 -- 172.21.15.3:0/2132354558 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38c00b7a0 con 0x7fe39c0d0870 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.780+0000 7fe399ffb700 1 -- 172.21.15.3:0/2132354558 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384001470 con 0x7fe39c0d0870 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2132354558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c0d0870 msgr2=0x7fe39c0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2132354558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c0d0870 0x7fe39c0b3ba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009e60 tx=0x7fe38c009780 comp rx=0 tx=0).stop 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2132354558 shutdown_connections 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2132354558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c0d0870 0x7fe39c0b3ba0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2132354558 >> v2:172.21.15.153:3300/0 conn(0x7fe39c15c8f0 0x7fe39c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2132354558 >> 172.21.15.3:0/2132354558 conn(0x7fe39c070310 msgr2=0x7fe39c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:53.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2132354558 shutdown_connections 2023-10-06T04:53:53.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.781+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2132354558 wait complete. 2023-10-06T04:53:53.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.782+0000 7fe3a12d9700 1 Processor -- start 2023-10-06T04:53:53.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.782+0000 7fe3a12d9700 1 -- start start 2023-10-06T04:53:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe3a12d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c0d0870 0x7fe39c150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe3a12d9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c15c8f0 0x7fe39c158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe3a12d9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe39c15ccc0 0x7fe39c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe3a12d9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c15bf90 con 0x7fe39c0d0870 2023-10-06T04:53:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe3a12d9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c1536e0 con 0x7fe39c15ccc0 2023-10-06T04:53:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe3a12d9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c153250 con 0x7fe39c15c8f0 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c0d0870 0x7fe39c150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c0d0870 0x7fe39c150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40464/0 (socket says 172.21.15.3:40464) 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 -- 172.21.15.3:0/2739296740 learned_addr learned my addr 172.21.15.3:0/2739296740 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 -- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c15c8f0 msgr2=0x7fe39c158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39a7fc700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c15c8f0 0x7fe39c158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39b7fe700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe39c15ccc0 0x7fe39c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c15c8f0 0x7fe39c158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 -- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe39c15ccc0 msgr2=0x7fe39c1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe39c15ccc0 0x7fe39c1593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39affd700 1 -- 172.21.15.3:0/2739296740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c009a20 con 0x7fe39c0d0870 2023-10-06T04:53:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.783+0000 7fe39a7fc700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c15c8f0 0x7fe39c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.784+0000 7fe39affd700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c0d0870 0x7fe39c150140 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe39400ecb0 tx=0x7fe3940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.784+0000 7fe392ffd700 1 -- 172.21.15.3:0/2739296740 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400c9d0 con 0x7fe39c0d0870 2023-10-06T04:53:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.784+0000 7fe392ffd700 1 -- 172.21.15.3:0/2739296740 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39400cb30 con 0x7fe39c0d0870 2023-10-06T04:53:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.784+0000 7fe392ffd700 1 -- 172.21.15.3:0/2739296740 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400ced0 con 0x7fe39c0d0870 2023-10-06T04:53:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.784+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c152640 con 0x7fe39c0d0870 2023-10-06T04:53:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.784+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c152c00 con 0x7fe39c0d0870 2023-10-06T04:53:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.785+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c153530 con 0x7fe39c0d0870 2023-10-06T04:53:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.785+0000 7fe392ffd700 1 -- 172.21.15.3:0/2739296740 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe394013680 con 0x7fe39c0d0870 2023-10-06T04:53:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.785+0000 7fe392ffd700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe37c066f30 0x7fe37c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.786+0000 7fe392ffd700 1 -- 172.21.15.3:0/2739296740 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe394084a50 con 0x7fe39c0d0870 2023-10-06T04:53:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.786+0000 7fe39a7fc700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe37c066f30 0x7fe37c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.786+0000 7fe39a7fc700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe37c066f30 0x7fe37c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe384001620 tx=0x7fe384009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:53.790+0000 7fe392ffd700 1 -- 172.21.15.3:0/2739296740 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe394051b60 con 0x7fe39c0d0870 2023-10-06T04:53:54.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:54.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.025+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe39c071680 con 0x7fe39c0d0870 2023-10-06T04:53:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.026+0000 7fe392ffd700 1 -- 172.21.15.3:0/2739296740 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fe394055180 con 0x7fe39c0d0870 2023-10-06T04:53:54.029 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:54.029 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/596033972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":596033972},{"type":"v1","addr":"172.21.15.153:6837","nonce":596033972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1359588707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1359588707},{"type":"v1","addr":"172.21.15.3:6834","nonce":1359588707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/616658407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":616658407},{"type":"v1","addr":"172.21.15.3:6835","nonce":616658407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:49.997154+0000","modified":"2023-10-06T04:53:53.627991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/4017437319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":4017437319},{"type":"v1","addr":"172.21.15.153:6836","nonce":4017437319}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe37c066f30 msgr2=0x7fe37c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe37c066f30 0x7fe37c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe384001620 tx=0x7fe384009040 comp rx=0 tx=0).stop 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c0d0870 msgr2=0x7fe39c150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c0d0870 0x7fe39c150140 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe39400ecb0 tx=0x7fe3940091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 shutdown_connections 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe37c066f30 0x7fe37c0693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe39c15ccc0 0x7fe39c1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c0d0870 0x7fe39c150140 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 --2- 172.21.15.3:0/2739296740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe39c15c8f0 0x7fe39c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 >> 172.21.15.3:0/2739296740 conn(0x7fe39c070310 msgr2=0x7fe39c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:54.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 shutdown_connections 2023-10-06T04:53:54.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.028+0000 7fe3a12d9700 1 -- 172.21.15.3:0/2739296740 wait complete. 2023-10-06T04:53:54.032 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2023-10-06T04:53:54.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:53:54.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.200+0000 7f70cf594700 1 Processor -- start 2023-10-06T04:53:54.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.200+0000 7f70cf594700 1 -- start start 2023-10-06T04:53:54.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cf594700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815c8f0 0x7f70c815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cf594700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f70c815ccc0 0x7f70c80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cf594700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70c80d06a0 0x7f70c806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cf594700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c80cf6c0 con 0x7f70c80d06a0 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cf594700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c80cfb50 con 0x7f70c815ccc0 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cf594700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c815e050 con 0x7f70c815c8f0 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cdb31700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70c80d06a0 0x7f70c806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cdb31700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70c80d06a0 0x7f70c806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40470/0 (socket says 172.21.15.3:40470) 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cdb31700 1 -- 172.21.15.3:0/1798335041 learned_addr learned my addr 172.21.15.3:0/1798335041 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70cd330700 1 --2- 172.21.15.3:0/1798335041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815c8f0 0x7f70c815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.201+0000 7f70ccb2f700 1 --2- 172.21.15.3:0/1798335041 >> v2:172.21.15.153:3300/0 conn(0x7f70c815ccc0 0x7f70c80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cdb31700 1 -- 172.21.15.3:0/1798335041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815c8f0 msgr2=0x7f70c815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cdb31700 1 --2- 172.21.15.3:0/1798335041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815c8f0 0x7f70c815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cdb31700 1 -- 172.21.15.3:0/1798335041 >> v2:172.21.15.153:3300/0 conn(0x7f70c815ccc0 msgr2=0x7f70c80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cdb31700 1 --2- 172.21.15.3:0/1798335041 >> v2:172.21.15.153:3300/0 conn(0x7f70c815ccc0 0x7f70c80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cdb31700 1 -- 172.21.15.3:0/1798335041 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c80d0af0 con 0x7f70c80d06a0 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70ccb2f700 1 --2- 172.21.15.3:0/1798335041 >> v2:172.21.15.153:3300/0 conn(0x7f70c815ccc0 0x7f70c80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cdb31700 1 --2- 172.21.15.3:0/1798335041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c80d06a0 0x7f70c806d950 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f70c806bed0 tx=0x7f70c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f088384d3c5821e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70c7fff700 1 -- 172.21.15.3:0/1798335041 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c000e070 con 0x7f70c80d06a0 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70c7fff700 1 -- 172.21.15.3:0/1798335041 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c000ba20 con 0x7f70c80d06a0 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70c7fff700 1 -- 172.21.15.3:0/1798335041 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00092e0 con 0x7f70c80d06a0 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cf594700 1 -- 172.21.15.3:0/1798335041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c80d06a0 msgr2=0x7f70c806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cf594700 1 --2- 172.21.15.3:0/1798335041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c80d06a0 0x7f70c806d950 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f70c806bed0 tx=0x7f70c0009840 comp rx=0 tx=0).stop 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cf594700 1 -- 172.21.15.3:0/1798335041 shutdown_connections 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cf594700 1 --2- 172.21.15.3:0/1798335041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c80d06a0 0x7f70c806d950 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cf594700 1 --2- 172.21.15.3:0/1798335041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815c8f0 0x7f70c815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cf594700 1 --2- 172.21.15.3:0/1798335041 >> v2:172.21.15.153:3300/0 conn(0x7f70c815ccc0 0x7f70c80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.202+0000 7f70cf594700 1 -- 172.21.15.3:0/1798335041 >> 172.21.15.3:0/1798335041 conn(0x7f70c8070310 msgr2=0x7f70c8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:54.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.203+0000 7f70cf594700 1 -- 172.21.15.3:0/1798335041 shutdown_connections 2023-10-06T04:53:54.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.203+0000 7f70cf594700 1 -- 172.21.15.3:0/1798335041 wait complete. 2023-10-06T04:53:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.204+0000 7f70cf594700 1 Processor -- start 2023-10-06T04:53:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.204+0000 7f70cf594700 1 -- start start 2023-10-06T04:53:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.204+0000 7f70cf594700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70c80d06a0 0x7f70c8146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.204+0000 7f70cf594700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c815c8f0 0x7f70c81689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cf594700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815ccc0 0x7f70c80c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cf594700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c815a830 con 0x7f70c815c8f0 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cf594700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c8168ef0 con 0x7f70c80d06a0 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cf594700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c8159bb0 con 0x7f70c815ccc0 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c815c8f0 0x7f70c81689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c815c8f0 0x7f70c81689b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40486/0 (socket says 172.21.15.3:40486) 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 -- 172.21.15.3:0/874879902 learned_addr learned my addr 172.21.15.3:0/874879902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cdb31700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815ccc0 0x7f70c80c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cd330700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70c80d06a0 0x7f70c8146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 -- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815ccc0 msgr2=0x7f70c80c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815ccc0 0x7f70c80c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 -- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70c80d06a0 msgr2=0x7f70c8146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70c80d06a0 0x7f70c8146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 -- 172.21.15.3:0/874879902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c0009ae0 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70ccb2f700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c815c8f0 0x7f70c81689b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e8a0 tx=0x7f70bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70c57fa700 1 -- 172.21.15.3:0/874879902 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc013410 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70c57fa700 1 -- 172.21.15.3:0/874879902 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc013570 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70c57fa700 1 -- 172.21.15.3:0/874879902 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc013930 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c8159e90 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.205+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c815a450 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.206+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c8071680 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.207+0000 7f70c57fa700 1 -- 172.21.15.3:0/874879902 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70bc013ce0 con 0x7f70c815c8f0 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.207+0000 7f70c57fa700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70b4066f30 0x7f70b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.207+0000 7f70c57fa700 1 -- 172.21.15.3:0/874879902 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f70bc0842b0 con 0x7f70c815c8f0 2023-10-06T04:53:54.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.207+0000 7f70cd330700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70b4066f30 0x7f70b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.208+0000 7f70cd330700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70b4066f30 0x7f70b40693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009db0 tx=0x7f70b80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.211+0000 7f70c57fa700 1 -- 172.21.15.3:0/874879902 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70bc0513c0 con 0x7f70c815c8f0 2023-10-06T04:53:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.406+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70c80b35a0 con 0x7f70c815c8f0 2023-10-06T04:53:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.407+0000 7f70c57fa700 1 -- 172.21.15.3:0/874879902 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16818 (secure 0 0 0) 0x7f70bc0549e0 con 0x7f70c815c8f0 2023-10-06T04:53:54.409 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:54.409 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":44,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:53:51.587265+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:47.743195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:48.974063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:53:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.408+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70b4066f30 msgr2=0x7f70b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.408+0000 7f70cf594700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70b4066f30 0x7f70b40693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009db0 tx=0x7f70b80091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.408+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c815c8f0 msgr2=0x7f70c81689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.408+0000 7f70cf594700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c815c8f0 0x7f70c81689b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e8a0 tx=0x7f70bc008040 comp rx=0 tx=0).stop 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 shutdown_connections 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70b4066f30 0x7f70b40693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70c80d06a0 0x7f70c8146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70c815ccc0 0x7f70c80c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 --2- 172.21.15.3:0/874879902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70c815c8f0 0x7f70c81689b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 >> 172.21.15.3:0/874879902 conn(0x7f70c8070310 msgr2=0x7f70c80b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 shutdown_connections 2023-10-06T04:53:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.409+0000 7f70cf594700 1 -- 172.21.15.3:0/874879902 wait complete. 2023-10-06T04:53:54.423 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:53:54.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:53:54.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.540+0000 7f922d5ce700 1 Processor -- start 2023-10-06T04:53:54.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.540+0000 7f922d5ce700 1 -- start start 2023-10-06T04:53:54.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922d5ce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f92280cb780 0x7f9228158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922d5ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9228159500 0x7f92280cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922d5ce700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92280cf130 con 0x7f92280cb3b0 2023-10-06T04:53:54.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922d5ce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280ce520 con 0x7f92280cb780 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922d5ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280ce800 con 0x7f9228159500 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922cdcd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9228159500 0x7f92280cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f92277fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f92280cb780 0x7f9228158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922cdcd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9228159500 0x7f92280cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:51934/0 (socket says 172.21.15.3:51934) 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.541+0000 7f922cdcd700 1 -- 172.21.15.3:0/2831186652 learned_addr learned my addr 172.21.15.3:0/2831186652 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f922cdcd700 1 -- 172.21.15.3:0/2831186652 >> v2:172.21.15.153:3300/0 conn(0x7f92280cb780 msgr2=0x7f9228158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f922cdcd700 1 --2- 172.21.15.3:0/2831186652 >> v2:172.21.15.153:3300/0 conn(0x7f92280cb780 0x7f9228158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f922cdcd700 1 -- 172.21.15.3:0/2831186652 >> v1:172.21.15.153:6789/0 conn(0x7f92280cb3b0 legacy=0x7f92280cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f922cdcd700 1 -- 172.21.15.3:0/2831186652 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280ceae0 con 0x7f9228159500 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f92277fe700 1 --2- 172.21.15.3:0/2831186652 >> v2:172.21.15.153:3300/0 conn(0x7f92280cb780 0x7f9228158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:53:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f922cdcd700 1 --2- 172.21.15.3:0/2831186652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9228159500 0x7f92280cccc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9220009f20 tx=0x7f9220009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b5bb8f1984275f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f9226ffd700 1 -- 172.21.15.3:0/2831186652 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922000e070 con 0x7f9228159500 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f9226ffd700 1 -- 172.21.15.3:0/2831186652 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922000b9d0 con 0x7f9228159500 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.542+0000 7f9226ffd700 1 -- 172.21.15.3:0/2831186652 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200092e0 con 0x7f9228159500 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 -- 172.21.15.3:0/2831186652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9228159500 msgr2=0x7f92280cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2831186652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9228159500 0x7f92280cccc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9220009f20 tx=0x7f9220009840 comp rx=0 tx=0).stop 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 -- 172.21.15.3:0/2831186652 shutdown_connections 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2831186652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9228159500 0x7f92280cccc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2831186652 >> v2:172.21.15.153:3300/0 conn(0x7f92280cb780 0x7f9228158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 -- 172.21.15.3:0/2831186652 >> 172.21.15.3:0/2831186652 conn(0x7f9228070310 msgr2=0x7f9228070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 -- 172.21.15.3:0/2831186652 shutdown_connections 2023-10-06T04:53:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.543+0000 7f922d5ce700 1 -- 172.21.15.3:0/2831186652 wait complete. 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.544+0000 7f922d5ce700 1 Processor -- start 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.544+0000 7f922d5ce700 1 -- start start 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922d5ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92280cb3b0 0x7f92280c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922d5ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92280cb780 0x7f922815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922d5ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9228159500 0x7f92280b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922d5ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922815bb50 con 0x7f9228159500 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922d5ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922815c380 con 0x7f92280cb3b0 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922d5ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922815e4f0 con 0x7f92280cb780 2023-10-06T04:53:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9228159500 0x7f92280b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9228159500 0x7f92280b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40496/0 (socket says 172.21.15.3:40496) 2023-10-06T04:53:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 -- 172.21.15.3:0/2120951702 learned_addr learned my addr 172.21.15.3:0/2120951702 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f92277fe700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92280cb780 0x7f922815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f9227fff700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92280cb3b0 0x7f92280c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 -- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92280cb780 msgr2=0x7f922815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92280cb780 0x7f922815dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 -- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92280cb3b0 msgr2=0x7f92280c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92280cb3b0 0x7f92280c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 -- 172.21.15.3:0/2120951702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9218009420 con 0x7f9228159500 2023-10-06T04:53:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.545+0000 7f922cdcd700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9228159500 0x7f92280b4470 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f92280ce940 tx=0x7f9220018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.546+0000 7f921ffff700 1 -- 172.21.15.3:0/2120951702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200094e0 con 0x7f9228159500 2023-10-06T04:53:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.546+0000 7f921ffff700 1 -- 172.21.15.3:0/2120951702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9220008bc0 con 0x7f9228159500 2023-10-06T04:53:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.546+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9220009ae0 con 0x7f9228159500 2023-10-06T04:53:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.546+0000 7f921ffff700 1 -- 172.21.15.3:0/2120951702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220025460 con 0x7f9228159500 2023-10-06T04:53:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.546+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922815e790 con 0x7f9228159500 2023-10-06T04:53:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.547+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922815c530 con 0x7f9228159500 2023-10-06T04:53:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.547+0000 7f921ffff700 1 -- 172.21.15.3:0/2120951702 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9220003040 con 0x7f9228159500 2023-10-06T04:53:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.547+0000 7f921ffff700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9208066f30 0x7f92080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.547+0000 7f921ffff700 1 -- 172.21.15.3:0/2120951702 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f92200862c0 con 0x7f9228159500 2023-10-06T04:53:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.547+0000 7f9227fff700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9208066f30 0x7f92080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.548+0000 7f9227fff700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9208066f30 0x7f92080693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9210003860 tx=0x7f9210009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.552+0000 7f921ffff700 1 -- 172.21.15.3:0/2120951702 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f922815c530 con 0x7f9228159500 2023-10-06T04:53:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.743+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f922815c4f0 con 0x7f9208066f30 2023-10-06T04:53:54.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.752+0000 7f921ffff700 1 -- 172.21.15.3:0/2120951702 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308294 (secure 0 0 0) 0x7f922815c4f0 con 0x7f9208066f30 2023-10-06T04:53:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9208066f30 msgr2=0x7f92080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9208066f30 0x7f92080693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9210003860 tx=0x7f9210009040 comp rx=0 tx=0).stop 2023-10-06T04:53:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9228159500 msgr2=0x7f92280b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9228159500 0x7f92280b4470 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f92280ce940 tx=0x7f9220018000 comp rx=0 tx=0).stop 2023-10-06T04:53:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 shutdown_connections 2023-10-06T04:53:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9208066f30 0x7f92080693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92280cb3b0 0x7f92280c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9228159500 0x7f92280b4470 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 --2- 172.21.15.3:0/2120951702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92280cb780 0x7f922815dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.755+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 >> 172.21.15.3:0/2120951702 conn(0x7f9228070310 msgr2=0x7f92280b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.756+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 shutdown_connections 2023-10-06T04:53:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.756+0000 7f922d5ce700 1 -- 172.21.15.3:0/2120951702 wait complete. 2023-10-06T04:53:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:53:54.770 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-10-06T04:53:53.445077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411964,"kb_used_data":4988,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752562756,"statfs":{"total":773094113280,"available":770624262144,"internally_reserved":0,"allocated":5107712,"data_stored":2821805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.862377"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593046+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:26.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594750+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598556+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592337+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593553+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592931+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:24.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593327+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590816+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595430+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:10.975006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591645+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.589759+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:04.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590440+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595968+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594915+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596102+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591659+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.599110+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594214+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595173+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594900+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596007+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590728+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590223+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:34.162265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597780+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590762+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591744+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593808+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594471+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596034+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594500+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592863+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601799+0000","last_change":"2023-10-06T04:53:49.601799+0000","last_active":"2023-10-06T04:53:49.601799+0000","last_peered":"2023-10-06T04:53:49.601799+0000","last_clean":"2023-10-06T04:53:49.601799+0000","last_became_active":"2023-10-06T04:53:49.601428+0000","last_became_peered":"2023-10-06T04:53:49.601428+0000","last_unstale":"2023-10-06T04:53:49.601799+0000","last_undegraded":"2023-10-06T04:53:49.601799+0000","last_fullsized":"2023-10-06T04:53:49.601799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:55.184468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593805+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607471+0000","last_change":"2023-10-06T04:53:49.607471+0000","last_active":"2023-10-06T04:53:49.607471+0000","last_peered":"2023-10-06T04:53:49.607471+0000","last_clean":"2023-10-06T04:53:49.607471+0000","last_became_active":"2023-10-06T04:53:49.607158+0000","last_became_peered":"2023-10-06T04:53:49.607158+0000","last_unstale":"2023-10-06T04:53:49.607471+0000","last_undegraded":"2023-10-06T04:53:49.607471+0000","last_fullsized":"2023-10-06T04:53:49.607471+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:41.756942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595565+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605024+0000","last_change":"2023-10-06T04:53:49.605024+0000","last_active":"2023-10-06T04:53:49.605024+0000","last_peered":"2023-10-06T04:53:49.605024+0000","last_clean":"2023-10-06T04:53:49.605024+0000","last_became_active":"2023-10-06T04:53:49.604790+0000","last_became_peered":"2023-10-06T04:53:49.604790+0000","last_unstale":"2023-10-06T04:53:49.605024+0000","last_undegraded":"2023-10-06T04:53:49.605024+0000","last_fullsized":"2023-10-06T04:53:49.605024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:44.358080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595049+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:50.391853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607504+0000","last_change":"2023-10-06T04:53:49.607504+0000","last_active":"2023-10-06T04:53:49.607504+0000","last_peered":"2023-10-06T04:53:49.607504+0000","last_clean":"2023-10-06T04:53:49.607504+0000","last_became_active":"2023-10-06T04:53:49.607263+0000","last_became_peered":"2023-10-06T04:53:49.607263+0000","last_unstale":"2023-10-06T04:53:49.607504+0000","last_undegraded":"2023-10-06T04:53:49.607504+0000","last_fullsized":"2023-10-06T04:53:49.607504+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596208+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606739+0000","last_change":"2023-10-06T04:53:49.606739+0000","last_active":"2023-10-06T04:53:49.606739+0000","last_peered":"2023-10-06T04:53:49.606739+0000","last_clean":"2023-10-06T04:53:49.606739+0000","last_became_active":"2023-10-06T04:53:49.606408+0000","last_became_peered":"2023-10-06T04:53:49.606408+0000","last_unstale":"2023-10-06T04:53:49.606739+0000","last_undegraded":"2023-10-06T04:53:49.606739+0000","last_fullsized":"2023-10-06T04:53:49.606739+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592689+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602410+0000","last_change":"2023-10-06T04:53:49.602410+0000","last_active":"2023-10-06T04:53:49.602410+0000","last_peered":"2023-10-06T04:53:49.602410+0000","last_clean":"2023-10-06T04:53:49.602410+0000","last_became_active":"2023-10-06T04:53:49.602130+0000","last_became_peered":"2023-10-06T04:53:49.602130+0000","last_unstale":"2023-10-06T04:53:49.602410+0000","last_undegraded":"2023-10-06T04:53:49.602410+0000","last_fullsized":"2023-10-06T04:53:49.602410+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594116+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602126+0000","last_change":"2023-10-06T04:53:49.602126+0000","last_active":"2023-10-06T04:53:49.602126+0000","last_peered":"2023-10-06T04:53:49.602126+0000","last_clean":"2023-10-06T04:53:49.602126+0000","last_became_active":"2023-10-06T04:53:49.601062+0000","last_became_peered":"2023-10-06T04:53:49.601062+0000","last_unstale":"2023-10-06T04:53:49.602126+0000","last_undegraded":"2023-10-06T04:53:49.602126+0000","last_fullsized":"2023-10-06T04:53:49.602126+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:54.807253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595466+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610421+0000","last_change":"2023-10-06T04:53:49.610421+0000","last_active":"2023-10-06T04:53:49.610421+0000","last_peered":"2023-10-06T04:53:49.610421+0000","last_clean":"2023-10-06T04:53:49.610421+0000","last_became_active":"2023-10-06T04:53:49.610133+0000","last_became_peered":"2023-10-06T04:53:49.610133+0000","last_unstale":"2023-10-06T04:53:49.610421+0000","last_undegraded":"2023-10-06T04:53:49.610421+0000","last_fullsized":"2023-10-06T04:53:49.610421+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:25.048345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592520+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602037+0000","last_change":"2023-10-06T04:53:49.602037+0000","last_active":"2023-10-06T04:53:49.602037+0000","last_peered":"2023-10-06T04:53:49.602037+0000","last_clean":"2023-10-06T04:53:49.602037+0000","last_became_active":"2023-10-06T04:53:49.601746+0000","last_became_peered":"2023-10-06T04:53:49.601746+0000","last_unstale":"2023-10-06T04:53:49.602037+0000","last_undegraded":"2023-10-06T04:53:49.602037+0000","last_fullsized":"2023-10-06T04:53:49.602037+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594303+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608823+0000","last_change":"2023-10-06T04:53:49.608823+0000","last_active":"2023-10-06T04:53:49.608823+0000","last_peered":"2023-10-06T04:53:49.608823+0000","last_clean":"2023-10-06T04:53:49.608823+0000","last_became_active":"2023-10-06T04:53:49.608542+0000","last_became_peered":"2023-10-06T04:53:49.608542+0000","last_unstale":"2023-10-06T04:53:49.608823+0000","last_undegraded":"2023-10-06T04:53:49.608823+0000","last_fullsized":"2023-10-06T04:53:49.608823+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594373+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610243+0000","last_change":"2023-10-06T04:53:49.610243+0000","last_active":"2023-10-06T04:53:49.610243+0000","last_peered":"2023-10-06T04:53:49.610243+0000","last_clean":"2023-10-06T04:53:49.610243+0000","last_became_active":"2023-10-06T04:53:49.609974+0000","last_became_peered":"2023-10-06T04:53:49.609974+0000","last_unstale":"2023-10-06T04:53:49.610243+0000","last_undegraded":"2023-10-06T04:53:49.610243+0000","last_fullsized":"2023-10-06T04:53:49.610243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592536+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599507+0000","last_change":"2023-10-06T04:53:49.599507+0000","last_active":"2023-10-06T04:53:49.599507+0000","last_peered":"2023-10-06T04:53:49.599507+0000","last_clean":"2023-10-06T04:53:49.599507+0000","last_became_active":"2023-10-06T04:53:49.599244+0000","last_became_peered":"2023-10-06T04:53:49.599244+0000","last_unstale":"2023-10-06T04:53:49.599507+0000","last_undegraded":"2023-10-06T04:53:49.599507+0000","last_fullsized":"2023-10-06T04:53:49.599507+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593289+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602521+0000","last_change":"2023-10-06T04:53:49.602521+0000","last_active":"2023-10-06T04:53:49.602521+0000","last_peered":"2023-10-06T04:53:49.602521+0000","last_clean":"2023-10-06T04:53:49.602521+0000","last_became_active":"2023-10-06T04:53:49.602150+0000","last_became_peered":"2023-10-06T04:53:49.602150+0000","last_unstale":"2023-10-06T04:53:49.602521+0000","last_undegraded":"2023-10-06T04:53:49.602521+0000","last_fullsized":"2023-10-06T04:53:49.602521+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597828+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608700+0000","last_change":"2023-10-06T04:53:49.608700+0000","last_active":"2023-10-06T04:53:49.608700+0000","last_peered":"2023-10-06T04:53:49.608700+0000","last_clean":"2023-10-06T04:53:49.608700+0000","last_became_active":"2023-10-06T04:53:49.608420+0000","last_became_peered":"2023-10-06T04:53:49.608420+0000","last_unstale":"2023-10-06T04:53:49.608700+0000","last_undegraded":"2023-10-06T04:53:49.608700+0000","last_fullsized":"2023-10-06T04:53:49.608700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:24.697185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594521+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606724+0000","last_change":"2023-10-06T04:53:49.606724+0000","last_active":"2023-10-06T04:53:49.606724+0000","last_peered":"2023-10-06T04:53:49.606724+0000","last_clean":"2023-10-06T04:53:49.606724+0000","last_became_active":"2023-10-06T04:53:49.606477+0000","last_became_peered":"2023-10-06T04:53:49.606477+0000","last_unstale":"2023-10-06T04:53:49.606724+0000","last_undegraded":"2023-10-06T04:53:49.606724+0000","last_fullsized":"2023-10-06T04:53:49.606724+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597267+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.598387+0000","last_change":"2023-10-06T04:53:49.598387+0000","last_active":"2023-10-06T04:53:49.598387+0000","last_peered":"2023-10-06T04:53:49.598387+0000","last_clean":"2023-10-06T04:53:49.598387+0000","last_became_active":"2023-10-06T04:53:49.598184+0000","last_became_peered":"2023-10-06T04:53:49.598184+0000","last_unstale":"2023-10-06T04:53:49.598387+0000","last_undegraded":"2023-10-06T04:53:49.598387+0000","last_fullsized":"2023-10-06T04:53:49.598387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608228+0000","last_change":"2023-10-06T04:53:49.608228+0000","last_active":"2023-10-06T04:53:49.608228+0000","last_peered":"2023-10-06T04:53:49.608228+0000","last_clean":"2023-10-06T04:53:49.608228+0000","last_became_active":"2023-10-06T04:53:49.607857+0000","last_became_peered":"2023-10-06T04:53:49.607857+0000","last_unstale":"2023-10-06T04:53:49.608228+0000","last_undegraded":"2023-10-06T04:53:49.608228+0000","last_fullsized":"2023-10-06T04:53:49.608228+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:22.996368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591635+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.600834+0000","last_change":"2023-10-06T04:53:49.600834+0000","last_active":"2023-10-06T04:53:49.600834+0000","last_peered":"2023-10-06T04:53:49.600834+0000","last_clean":"2023-10-06T04:53:49.600834+0000","last_became_active":"2023-10-06T04:53:49.600540+0000","last_became_peered":"2023-10-06T04:53:49.600540+0000","last_unstale":"2023-10-06T04:53:49.600834+0000","last_undegraded":"2023-10-06T04:53:49.600834+0000","last_fullsized":"2023-10-06T04:53:49.600834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:06.484647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593411+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613344+0000","last_change":"2023-10-06T04:53:49.613344+0000","last_active":"2023-10-06T04:53:49.613344+0000","last_peered":"2023-10-06T04:53:49.613344+0000","last_clean":"2023-10-06T04:53:49.613344+0000","last_became_active":"2023-10-06T04:53:49.613045+0000","last_became_peered":"2023-10-06T04:53:49.613045+0000","last_unstale":"2023-10-06T04:53:49.613344+0000","last_undegraded":"2023-10-06T04:53:49.613344+0000","last_fullsized":"2023-10-06T04:53:49.613344+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:29.920075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591953+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.047782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601346+0000","last_change":"2023-10-06T04:53:49.601346+0000","last_active":"2023-10-06T04:53:49.601346+0000","last_peered":"2023-10-06T04:53:49.601346+0000","last_clean":"2023-10-06T04:53:49.601346+0000","last_became_active":"2023-10-06T04:53:49.600993+0000","last_became_peered":"2023-10-06T04:53:49.600993+0000","last_unstale":"2023-10-06T04:53:49.601346+0000","last_undegraded":"2023-10-06T04:53:49.601346+0000","last_fullsized":"2023-10-06T04:53:49.601346+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:58.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592361+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:26.053487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606428+0000","last_change":"2023-10-06T04:53:49.606428+0000","last_active":"2023-10-06T04:53:49.606428+0000","last_peered":"2023-10-06T04:53:49.606428+0000","last_clean":"2023-10-06T04:53:49.606428+0000","last_became_active":"2023-10-06T04:53:49.606178+0000","last_became_peered":"2023-10-06T04:53:49.606178+0000","last_unstale":"2023-10-06T04:53:49.606428+0000","last_undegraded":"2023-10-06T04:53:49.606428+0000","last_fullsized":"2023-10-06T04:53:49.606428+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595074+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599873+0000","last_change":"2023-10-06T04:53:49.599873+0000","last_active":"2023-10-06T04:53:49.599873+0000","last_peered":"2023-10-06T04:53:49.599873+0000","last_clean":"2023-10-06T04:53:49.599873+0000","last_became_active":"2023-10-06T04:53:49.599582+0000","last_became_peered":"2023-10-06T04:53:49.599582+0000","last_unstale":"2023-10-06T04:53:49.599873+0000","last_undegraded":"2023-10-06T04:53:49.599873+0000","last_fullsized":"2023-10-06T04:53:49.599873+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598198+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601241+0000","last_change":"2023-10-06T04:53:49.601241+0000","last_active":"2023-10-06T04:53:49.601241+0000","last_peered":"2023-10-06T04:53:49.601241+0000","last_clean":"2023-10-06T04:53:49.601241+0000","last_became_active":"2023-10-06T04:53:49.600982+0000","last_became_peered":"2023-10-06T04:53:49.600982+0000","last_unstale":"2023-10-06T04:53:49.601241+0000","last_undegraded":"2023-10-06T04:53:49.601241+0000","last_fullsized":"2023-10-06T04:53:49.601241+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:26.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592554+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:27.741668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599636+0000","last_change":"2023-10-06T04:53:49.599636+0000","last_active":"2023-10-06T04:53:49.599636+0000","last_peered":"2023-10-06T04:53:49.599636+0000","last_clean":"2023-10-06T04:53:49.599636+0000","last_became_active":"2023-10-06T04:53:49.599356+0000","last_became_peered":"2023-10-06T04:53:49.599356+0000","last_unstale":"2023-10-06T04:53:49.599636+0000","last_undegraded":"2023-10-06T04:53:49.599636+0000","last_fullsized":"2023-10-06T04:53:49.599636+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:50.733425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593469+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608771+0000","last_change":"2023-10-06T04:53:49.608771+0000","last_active":"2023-10-06T04:53:49.608771+0000","last_peered":"2023-10-06T04:53:49.608771+0000","last_clean":"2023-10-06T04:53:49.608771+0000","last_became_active":"2023-10-06T04:53:49.608472+0000","last_became_peered":"2023-10-06T04:53:49.608472+0000","last_unstale":"2023-10-06T04:53:49.608771+0000","last_undegraded":"2023-10-06T04:53:49.608771+0000","last_fullsized":"2023-10-06T04:53:49.608771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:41.942599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590905+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:53:48.590905+0000","last_peered":"2023-10-06T04:53:48.590905+0000","last_clean":"2023-10-06T04:53:48.590905+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:53:48.590905+0000","last_undegraded":"2023-10-06T04:53:48.590905+0000","last_fullsized":"2023-10-06T04:53:48.590905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594775+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.597803+0000","last_change":"2023-10-06T04:53:49.597803+0000","last_active":"2023-10-06T04:53:49.597803+0000","last_peered":"2023-10-06T04:53:49.597803+0000","last_clean":"2023-10-06T04:53:49.597803+0000","last_became_active":"2023-10-06T04:53:49.597467+0000","last_became_peered":"2023-10-06T04:53:49.597467+0000","last_unstale":"2023-10-06T04:53:49.597803+0000","last_undegraded":"2023-10-06T04:53:49.597803+0000","last_fullsized":"2023-10-06T04:53:49.597803+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:42.301398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590629+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:53:48.590629+0000","last_peered":"2023-10-06T04:53:48.590629+0000","last_clean":"2023-10-06T04:53:48.590629+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:53:48.590629+0000","last_undegraded":"2023-10-06T04:53:48.590629+0000","last_fullsized":"2023-10-06T04:53:48.590629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608060+0000","last_change":"2023-10-06T04:53:49.608060+0000","last_active":"2023-10-06T04:53:49.608060+0000","last_peered":"2023-10-06T04:53:49.608060+0000","last_clean":"2023-10-06T04:53:49.608060+0000","last_became_active":"2023-10-06T04:53:49.607815+0000","last_became_peered":"2023-10-06T04:53:49.607815+0000","last_unstale":"2023-10-06T04:53:49.608060+0000","last_undegraded":"2023-10-06T04:53:49.608060+0000","last_fullsized":"2023-10-06T04:53:49.608060+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.584714+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:53:48.584714+0000","last_peered":"2023-10-06T04:53:48.584714+0000","last_clean":"2023-10-06T04:53:48.584714+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:53:48.584714+0000","last_undegraded":"2023-10-06T04:53:48.584714+0000","last_fullsized":"2023-10-06T04:53:48.584714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":179,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594422+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.611927+0000","last_change":"2023-10-06T04:53:49.611927+0000","last_active":"2023-10-06T04:53:49.611927+0000","last_peered":"2023-10-06T04:53:49.611927+0000","last_clean":"2023-10-06T04:53:49.611927+0000","last_became_active":"2023-10-06T04:53:49.611624+0000","last_became_peered":"2023-10-06T04:53:49.611624+0000","last_unstale":"2023-10-06T04:53:49.611927+0000","last_undegraded":"2023-10-06T04:53:49.611927+0000","last_fullsized":"2023-10-06T04:53:49.611927+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585519+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:53:48.585519+0000","last_peered":"2023-10-06T04:53:48.585519+0000","last_clean":"2023-10-06T04:53:48.585519+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:53:48.585519+0000","last_undegraded":"2023-10-06T04:53:48.585519+0000","last_fullsized":"2023-10-06T04:53:48.585519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592939+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:14.972969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607924+0000","last_change":"2023-10-06T04:53:49.607924+0000","last_active":"2023-10-06T04:53:49.607924+0000","last_peered":"2023-10-06T04:53:49.607924+0000","last_clean":"2023-10-06T04:53:49.607924+0000","last_became_active":"2023-10-06T04:53:49.607341+0000","last_became_peered":"2023-10-06T04:53:49.607341+0000","last_unstale":"2023-10-06T04:53:49.607924+0000","last_undegraded":"2023-10-06T04:53:49.607924+0000","last_fullsized":"2023-10-06T04:53:49.607924+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592777+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608707+0000","last_change":"2023-10-06T04:53:49.608707+0000","last_active":"2023-10-06T04:53:49.608707+0000","last_peered":"2023-10-06T04:53:49.608707+0000","last_clean":"2023-10-06T04:53:49.608707+0000","last_became_active":"2023-10-06T04:53:49.608433+0000","last_became_peered":"2023-10-06T04:53:49.608433+0000","last_unstale":"2023-10-06T04:53:49.608707+0000","last_undegraded":"2023-10-06T04:53:49.608707+0000","last_fullsized":"2023-10-06T04:53:49.608707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.587417+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:53:48.587417+0000","last_peered":"2023-10-06T04:53:48.587417+0000","last_clean":"2023-10-06T04:53:48.587417+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:53:48.587417+0000","last_undegraded":"2023-10-06T04:53:48.587417+0000","last_fullsized":"2023-10-06T04:53:48.587417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594963+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613283+0000","last_change":"2023-10-06T04:53:49.613283+0000","last_active":"2023-10-06T04:53:49.613283+0000","last_peered":"2023-10-06T04:53:49.613283+0000","last_clean":"2023-10-06T04:53:49.613283+0000","last_became_active":"2023-10-06T04:53:49.612982+0000","last_became_peered":"2023-10-06T04:53:49.612982+0000","last_unstale":"2023-10-06T04:53:49.613283+0000","last_undegraded":"2023-10-06T04:53:49.613283+0000","last_fullsized":"2023-10-06T04:53:49.613283+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588728+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:53:48.588728+0000","last_peered":"2023-10-06T04:53:48.588728+0000","last_clean":"2023-10-06T04:53:48.588728+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:53:48.588728+0000","last_undegraded":"2023-10-06T04:53:48.588728+0000","last_fullsized":"2023-10-06T04:53:48.588728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598678+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605704+0000","last_change":"2023-10-06T04:53:49.605704+0000","last_active":"2023-10-06T04:53:49.605704+0000","last_peered":"2023-10-06T04:53:49.605704+0000","last_clean":"2023-10-06T04:53:49.605704+0000","last_became_active":"2023-10-06T04:53:49.605449+0000","last_became_peered":"2023-10-06T04:53:49.605449+0000","last_unstale":"2023-10-06T04:53:49.605704+0000","last_undegraded":"2023-10-06T04:53:49.605704+0000","last_fullsized":"2023-10-06T04:53:49.605704+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588915+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:53:48.588915+0000","last_peered":"2023-10-06T04:53:48.588915+0000","last_clean":"2023-10-06T04:53:48.588915+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:53:48.588915+0000","last_undegraded":"2023-10-06T04:53:48.588915+0000","last_fullsized":"2023-10-06T04:53:48.588915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":7.1779999999999999,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.504},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.375},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.376},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.495}]},{"osd":6,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.51800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.25600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.438}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.503}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.372}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.27300000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.069,"5min":5.069,"15min":5.069},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":6.4210000000000003,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4009999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.367,"5min":6.367,"15min":6.367},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":5,"5min":5,"15min":5},"last":0.51600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.501},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":5.4569999999999999,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":5.5510000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.415,"5min":5.415,"15min":5.415},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.202,"5min":4.202,"15min":4.202},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.628}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301860,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069980,"statfs":{"total":96636764160,"available":96327659520,"internally_reserved":0,"allocated":487424,"data_stored":204288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.7949999999999999,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.8170000000000002,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.7210000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:53:54.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:53:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.927+0000 7f8aec622700 1 Processor -- start 2023-10-06T04:53:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.927+0000 7f8aec622700 1 -- start start 2023-10-06T04:53:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8aec622700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8ae415cb50 0x7f8ae40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8aec622700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8aec622700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae40b0ce0 con 0x7f8ae40cd950 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8aec622700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae40cccc0 con 0x7f8ae415cb50 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8aec622700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40cb820 con 0x7f8ae40b0590 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8ae9bbd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8aea3be700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8ae415cb50 0x7f8ae40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8ae9bbd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae40b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:51956/0 (socket says 172.21.15.3:51956) 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.928+0000 7f8ae9bbd700 1 -- 172.21.15.3:0/309251165 learned_addr learned my addr 172.21.15.3:0/309251165 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae9bbd700 1 -- 172.21.15.3:0/309251165 >> v2:172.21.15.153:3300/0 conn(0x7f8ae415cb50 msgr2=0x7f8ae40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae9bbd700 1 --2- 172.21.15.3:0/309251165 >> v2:172.21.15.153:3300/0 conn(0x7f8ae415cb50 0x7f8ae40b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae9bbd700 1 -- 172.21.15.3:0/309251165 >> v1:172.21.15.153:6789/0 conn(0x7f8ae40cd950 legacy=0x7f8ae406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae9bbd700 1 -- 172.21.15.3:0/309251165 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40b0a00 con 0x7f8ae40b0590 2023-10-06T04:53:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae9bbd700 1 --2- 172.21.15.3:0/309251165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae40b2200 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80099b0 tx=0x7f8ad8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114850510000fdcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae93bc700 1 -- 172.21.15.3:0/309251165 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8009ea0 con 0x7f8ae40b0590 2023-10-06T04:53:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae93bc700 1 -- 172.21.15.3:0/309251165 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad800b860 con 0x7f8ae40b0590 2023-10-06T04:53:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.929+0000 7f8ae93bc700 1 -- 172.21.15.3:0/309251165 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800bc00 con 0x7f8ae40b0590 2023-10-06T04:53:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 -- 172.21.15.3:0/309251165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 msgr2=0x7f8ae40b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 --2- 172.21.15.3:0/309251165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae40b2200 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80099b0 tx=0x7f8ad8009250 comp rx=0 tx=0).stop 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 -- 172.21.15.3:0/309251165 shutdown_connections 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 --2- 172.21.15.3:0/309251165 >> v2:172.21.15.153:3300/0 conn(0x7f8ae415cb50 0x7f8ae40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 --2- 172.21.15.3:0/309251165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae40b2200 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 -- 172.21.15.3:0/309251165 >> 172.21.15.3:0/309251165 conn(0x7f8ae4070310 msgr2=0x7f8ae4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 -- 172.21.15.3:0/309251165 shutdown_connections 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.930+0000 7f8aec622700 1 -- 172.21.15.3:0/309251165 wait complete. 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.931+0000 7f8aec622700 1 Processor -- start 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.931+0000 7f8aec622700 1 -- start start 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.931+0000 7f8aec622700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae416ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aec622700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae40cd950 0x7f8ae415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aec622700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae415cb50 0x7f8ae40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aec622700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40ced30 con 0x7f8ae415cb50 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aec622700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40ce120 con 0x7f8ae40cd950 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aec622700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40ce400 con 0x7f8ae40b0590 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae415cb50 0x7f8ae40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae415cb50 0x7f8ae40c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40512/0 (socket says 172.21.15.3:40512) 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 -- 172.21.15.3:0/1364473435 learned_addr learned my addr 172.21.15.3:0/1364473435 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8ae9bbd700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae40cd950 0x7f8ae415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aea3be700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae416ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 -- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 msgr2=0x7f8ae416ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae416ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 -- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae40cd950 msgr2=0x7f8ae415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae40cd950 0x7f8ae415a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 -- 172.21.15.3:0/1364473435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad80094f0 con 0x7f8ae415cb50 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aea3be700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae416ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aeabbf700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae415cb50 0x7f8ae40c9620 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c8e0 tx=0x7f8ad4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8ae27fc700 1 -- 172.21.15.3:0/1364473435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400ce80 con 0x7f8ae415cb50 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8ae27fc700 1 -- 172.21.15.3:0/1364473435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400a7e0 con 0x7f8ae415cb50 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae40ce6e0 con 0x7f8ae415cb50 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8ae27fc700 1 -- 172.21.15.3:0/1364473435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400ab80 con 0x7f8ae415cb50 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.932+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae415a840 con 0x7f8ae415cb50 2023-10-06T04:53:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.933+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae4071680 con 0x7f8ae415cb50 2023-10-06T04:53:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.934+0000 7f8ae27fc700 1 -- 172.21.15.3:0/1364473435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ad4012420 con 0x7f8ae415cb50 2023-10-06T04:53:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.934+0000 7f8ae27fc700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ad0066f30 0x7f8ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.934+0000 7f8ae27fc700 1 -- 172.21.15.3:0/1364473435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ad40849a0 con 0x7f8ae415cb50 2023-10-06T04:53:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.934+0000 7f8aea3be700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ad0066f30 0x7f8ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:54.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.935+0000 7f8aea3be700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ad0066f30 0x7f8ad00693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8adc006ea0 tx=0x7f8adc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:54.938+0000 7f8ae27fc700 1 -- 172.21.15.3:0/1364473435 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ad4051ab0 con 0x7f8ae415cb50 2023-10-06T04:53:55.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.126+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ae40cb1d0 con 0x7f8ad0066f30 2023-10-06T04:53:55.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.135+0000 7f8ae27fc700 1 -- 172.21.15.3:0/1364473435 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308294 (secure 0 0 0) 0x7f8ae40cb1d0 con 0x7f8ad0066f30 2023-10-06T04:53:55.141 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ad0066f30 msgr2=0x7f8ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ad0066f30 0x7f8ad00693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8adc006ea0 tx=0x7f8adc008040 comp rx=0 tx=0).stop 2023-10-06T04:53:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae415cb50 msgr2=0x7f8ae40c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae415cb50 0x7f8ae40c9620 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c8e0 tx=0x7f8ad4001710 comp rx=0 tx=0).stop 2023-10-06T04:53:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 shutdown_connections 2023-10-06T04:53:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ad0066f30 0x7f8ad00693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae415cb50 0x7f8ae40c9620 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae40b0590 0x7f8ae416ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 --2- 172.21.15.3:0/1364473435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae40cd950 0x7f8ae415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 >> 172.21.15.3:0/1364473435 conn(0x7f8ae4070310 msgr2=0x7f8ae40cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 shutdown_connections 2023-10-06T04:53:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.139+0000 7f8aec622700 1 -- 172.21.15.3:0/1364473435 wait complete. 2023-10-06T04:53:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:53:55.153 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-10-06T04:53:53.445077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411964,"kb_used_data":4988,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752562756,"statfs":{"total":773094113280,"available":770624262144,"internally_reserved":0,"allocated":5107712,"data_stored":2821805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.862377"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593046+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:26.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594750+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598556+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592337+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593553+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592931+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:24.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593327+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590816+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595430+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:10.975006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591645+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.589759+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:04.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590440+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595968+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594915+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596102+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591659+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.599110+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594214+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595173+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594900+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596007+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590728+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590223+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:34.162265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597780+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590762+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591744+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593808+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594471+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596034+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594500+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592863+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601799+0000","last_change":"2023-10-06T04:53:49.601799+0000","last_active":"2023-10-06T04:53:49.601799+0000","last_peered":"2023-10-06T04:53:49.601799+0000","last_clean":"2023-10-06T04:53:49.601799+0000","last_became_active":"2023-10-06T04:53:49.601428+0000","last_became_peered":"2023-10-06T04:53:49.601428+0000","last_unstale":"2023-10-06T04:53:49.601799+0000","last_undegraded":"2023-10-06T04:53:49.601799+0000","last_fullsized":"2023-10-06T04:53:49.601799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:55.184468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593805+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607471+0000","last_change":"2023-10-06T04:53:49.607471+0000","last_active":"2023-10-06T04:53:49.607471+0000","last_peered":"2023-10-06T04:53:49.607471+0000","last_clean":"2023-10-06T04:53:49.607471+0000","last_became_active":"2023-10-06T04:53:49.607158+0000","last_became_peered":"2023-10-06T04:53:49.607158+0000","last_unstale":"2023-10-06T04:53:49.607471+0000","last_undegraded":"2023-10-06T04:53:49.607471+0000","last_fullsized":"2023-10-06T04:53:49.607471+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:41.756942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595565+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605024+0000","last_change":"2023-10-06T04:53:49.605024+0000","last_active":"2023-10-06T04:53:49.605024+0000","last_peered":"2023-10-06T04:53:49.605024+0000","last_clean":"2023-10-06T04:53:49.605024+0000","last_became_active":"2023-10-06T04:53:49.604790+0000","last_became_peered":"2023-10-06T04:53:49.604790+0000","last_unstale":"2023-10-06T04:53:49.605024+0000","last_undegraded":"2023-10-06T04:53:49.605024+0000","last_fullsized":"2023-10-06T04:53:49.605024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:44.358080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595049+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:50.391853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607504+0000","last_change":"2023-10-06T04:53:49.607504+0000","last_active":"2023-10-06T04:53:49.607504+0000","last_peered":"2023-10-06T04:53:49.607504+0000","last_clean":"2023-10-06T04:53:49.607504+0000","last_became_active":"2023-10-06T04:53:49.607263+0000","last_became_peered":"2023-10-06T04:53:49.607263+0000","last_unstale":"2023-10-06T04:53:49.607504+0000","last_undegraded":"2023-10-06T04:53:49.607504+0000","last_fullsized":"2023-10-06T04:53:49.607504+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596208+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606739+0000","last_change":"2023-10-06T04:53:49.606739+0000","last_active":"2023-10-06T04:53:49.606739+0000","last_peered":"2023-10-06T04:53:49.606739+0000","last_clean":"2023-10-06T04:53:49.606739+0000","last_became_active":"2023-10-06T04:53:49.606408+0000","last_became_peered":"2023-10-06T04:53:49.606408+0000","last_unstale":"2023-10-06T04:53:49.606739+0000","last_undegraded":"2023-10-06T04:53:49.606739+0000","last_fullsized":"2023-10-06T04:53:49.606739+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592689+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602410+0000","last_change":"2023-10-06T04:53:49.602410+0000","last_active":"2023-10-06T04:53:49.602410+0000","last_peered":"2023-10-06T04:53:49.602410+0000","last_clean":"2023-10-06T04:53:49.602410+0000","last_became_active":"2023-10-06T04:53:49.602130+0000","last_became_peered":"2023-10-06T04:53:49.602130+0000","last_unstale":"2023-10-06T04:53:49.602410+0000","last_undegraded":"2023-10-06T04:53:49.602410+0000","last_fullsized":"2023-10-06T04:53:49.602410+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594116+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602126+0000","last_change":"2023-10-06T04:53:49.602126+0000","last_active":"2023-10-06T04:53:49.602126+0000","last_peered":"2023-10-06T04:53:49.602126+0000","last_clean":"2023-10-06T04:53:49.602126+0000","last_became_active":"2023-10-06T04:53:49.601062+0000","last_became_peered":"2023-10-06T04:53:49.601062+0000","last_unstale":"2023-10-06T04:53:49.602126+0000","last_undegraded":"2023-10-06T04:53:49.602126+0000","last_fullsized":"2023-10-06T04:53:49.602126+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:54.807253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595466+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610421+0000","last_change":"2023-10-06T04:53:49.610421+0000","last_active":"2023-10-06T04:53:49.610421+0000","last_peered":"2023-10-06T04:53:49.610421+0000","last_clean":"2023-10-06T04:53:49.610421+0000","last_became_active":"2023-10-06T04:53:49.610133+0000","last_became_peered":"2023-10-06T04:53:49.610133+0000","last_unstale":"2023-10-06T04:53:49.610421+0000","last_undegraded":"2023-10-06T04:53:49.610421+0000","last_fullsized":"2023-10-06T04:53:49.610421+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:25.048345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592520+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602037+0000","last_change":"2023-10-06T04:53:49.602037+0000","last_active":"2023-10-06T04:53:49.602037+0000","last_peered":"2023-10-06T04:53:49.602037+0000","last_clean":"2023-10-06T04:53:49.602037+0000","last_became_active":"2023-10-06T04:53:49.601746+0000","last_became_peered":"2023-10-06T04:53:49.601746+0000","last_unstale":"2023-10-06T04:53:49.602037+0000","last_undegraded":"2023-10-06T04:53:49.602037+0000","last_fullsized":"2023-10-06T04:53:49.602037+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594303+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608823+0000","last_change":"2023-10-06T04:53:49.608823+0000","last_active":"2023-10-06T04:53:49.608823+0000","last_peered":"2023-10-06T04:53:49.608823+0000","last_clean":"2023-10-06T04:53:49.608823+0000","last_became_active":"2023-10-06T04:53:49.608542+0000","last_became_peered":"2023-10-06T04:53:49.608542+0000","last_unstale":"2023-10-06T04:53:49.608823+0000","last_undegraded":"2023-10-06T04:53:49.608823+0000","last_fullsized":"2023-10-06T04:53:49.608823+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594373+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610243+0000","last_change":"2023-10-06T04:53:49.610243+0000","last_active":"2023-10-06T04:53:49.610243+0000","last_peered":"2023-10-06T04:53:49.610243+0000","last_clean":"2023-10-06T04:53:49.610243+0000","last_became_active":"2023-10-06T04:53:49.609974+0000","last_became_peered":"2023-10-06T04:53:49.609974+0000","last_unstale":"2023-10-06T04:53:49.610243+0000","last_undegraded":"2023-10-06T04:53:49.610243+0000","last_fullsized":"2023-10-06T04:53:49.610243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592536+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599507+0000","last_change":"2023-10-06T04:53:49.599507+0000","last_active":"2023-10-06T04:53:49.599507+0000","last_peered":"2023-10-06T04:53:49.599507+0000","last_clean":"2023-10-06T04:53:49.599507+0000","last_became_active":"2023-10-06T04:53:49.599244+0000","last_became_peered":"2023-10-06T04:53:49.599244+0000","last_unstale":"2023-10-06T04:53:49.599507+0000","last_undegraded":"2023-10-06T04:53:49.599507+0000","last_fullsized":"2023-10-06T04:53:49.599507+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593289+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602521+0000","last_change":"2023-10-06T04:53:49.602521+0000","last_active":"2023-10-06T04:53:49.602521+0000","last_peered":"2023-10-06T04:53:49.602521+0000","last_clean":"2023-10-06T04:53:49.602521+0000","last_became_active":"2023-10-06T04:53:49.602150+0000","last_became_peered":"2023-10-06T04:53:49.602150+0000","last_unstale":"2023-10-06T04:53:49.602521+0000","last_undegraded":"2023-10-06T04:53:49.602521+0000","last_fullsized":"2023-10-06T04:53:49.602521+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597828+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608700+0000","last_change":"2023-10-06T04:53:49.608700+0000","last_active":"2023-10-06T04:53:49.608700+0000","last_peered":"2023-10-06T04:53:49.608700+0000","last_clean":"2023-10-06T04:53:49.608700+0000","last_became_active":"2023-10-06T04:53:49.608420+0000","last_became_peered":"2023-10-06T04:53:49.608420+0000","last_unstale":"2023-10-06T04:53:49.608700+0000","last_undegraded":"2023-10-06T04:53:49.608700+0000","last_fullsized":"2023-10-06T04:53:49.608700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:24.697185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594521+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606724+0000","last_change":"2023-10-06T04:53:49.606724+0000","last_active":"2023-10-06T04:53:49.606724+0000","last_peered":"2023-10-06T04:53:49.606724+0000","last_clean":"2023-10-06T04:53:49.606724+0000","last_became_active":"2023-10-06T04:53:49.606477+0000","last_became_peered":"2023-10-06T04:53:49.606477+0000","last_unstale":"2023-10-06T04:53:49.606724+0000","last_undegraded":"2023-10-06T04:53:49.606724+0000","last_fullsized":"2023-10-06T04:53:49.606724+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597267+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.598387+0000","last_change":"2023-10-06T04:53:49.598387+0000","last_active":"2023-10-06T04:53:49.598387+0000","last_peered":"2023-10-06T04:53:49.598387+0000","last_clean":"2023-10-06T04:53:49.598387+0000","last_became_active":"2023-10-06T04:53:49.598184+0000","last_became_peered":"2023-10-06T04:53:49.598184+0000","last_unstale":"2023-10-06T04:53:49.598387+0000","last_undegraded":"2023-10-06T04:53:49.598387+0000","last_fullsized":"2023-10-06T04:53:49.598387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608228+0000","last_change":"2023-10-06T04:53:49.608228+0000","last_active":"2023-10-06T04:53:49.608228+0000","last_peered":"2023-10-06T04:53:49.608228+0000","last_clean":"2023-10-06T04:53:49.608228+0000","last_became_active":"2023-10-06T04:53:49.607857+0000","last_became_peered":"2023-10-06T04:53:49.607857+0000","last_unstale":"2023-10-06T04:53:49.608228+0000","last_undegraded":"2023-10-06T04:53:49.608228+0000","last_fullsized":"2023-10-06T04:53:49.608228+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:22.996368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591635+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.600834+0000","last_change":"2023-10-06T04:53:49.600834+0000","last_active":"2023-10-06T04:53:49.600834+0000","last_peered":"2023-10-06T04:53:49.600834+0000","last_clean":"2023-10-06T04:53:49.600834+0000","last_became_active":"2023-10-06T04:53:49.600540+0000","last_became_peered":"2023-10-06T04:53:49.600540+0000","last_unstale":"2023-10-06T04:53:49.600834+0000","last_undegraded":"2023-10-06T04:53:49.600834+0000","last_fullsized":"2023-10-06T04:53:49.600834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:06.484647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593411+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613344+0000","last_change":"2023-10-06T04:53:49.613344+0000","last_active":"2023-10-06T04:53:49.613344+0000","last_peered":"2023-10-06T04:53:49.613344+0000","last_clean":"2023-10-06T04:53:49.613344+0000","last_became_active":"2023-10-06T04:53:49.613045+0000","last_became_peered":"2023-10-06T04:53:49.613045+0000","last_unstale":"2023-10-06T04:53:49.613344+0000","last_undegraded":"2023-10-06T04:53:49.613344+0000","last_fullsized":"2023-10-06T04:53:49.613344+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:29.920075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591953+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.047782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601346+0000","last_change":"2023-10-06T04:53:49.601346+0000","last_active":"2023-10-06T04:53:49.601346+0000","last_peered":"2023-10-06T04:53:49.601346+0000","last_clean":"2023-10-06T04:53:49.601346+0000","last_became_active":"2023-10-06T04:53:49.600993+0000","last_became_peered":"2023-10-06T04:53:49.600993+0000","last_unstale":"2023-10-06T04:53:49.601346+0000","last_undegraded":"2023-10-06T04:53:49.601346+0000","last_fullsized":"2023-10-06T04:53:49.601346+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:58.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592361+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:26.053487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606428+0000","last_change":"2023-10-06T04:53:49.606428+0000","last_active":"2023-10-06T04:53:49.606428+0000","last_peered":"2023-10-06T04:53:49.606428+0000","last_clean":"2023-10-06T04:53:49.606428+0000","last_became_active":"2023-10-06T04:53:49.606178+0000","last_became_peered":"2023-10-06T04:53:49.606178+0000","last_unstale":"2023-10-06T04:53:49.606428+0000","last_undegraded":"2023-10-06T04:53:49.606428+0000","last_fullsized":"2023-10-06T04:53:49.606428+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595074+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599873+0000","last_change":"2023-10-06T04:53:49.599873+0000","last_active":"2023-10-06T04:53:49.599873+0000","last_peered":"2023-10-06T04:53:49.599873+0000","last_clean":"2023-10-06T04:53:49.599873+0000","last_became_active":"2023-10-06T04:53:49.599582+0000","last_became_peered":"2023-10-06T04:53:49.599582+0000","last_unstale":"2023-10-06T04:53:49.599873+0000","last_undegraded":"2023-10-06T04:53:49.599873+0000","last_fullsized":"2023-10-06T04:53:49.599873+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598198+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601241+0000","last_change":"2023-10-06T04:53:49.601241+0000","last_active":"2023-10-06T04:53:49.601241+0000","last_peered":"2023-10-06T04:53:49.601241+0000","last_clean":"2023-10-06T04:53:49.601241+0000","last_became_active":"2023-10-06T04:53:49.600982+0000","last_became_peered":"2023-10-06T04:53:49.600982+0000","last_unstale":"2023-10-06T04:53:49.601241+0000","last_undegraded":"2023-10-06T04:53:49.601241+0000","last_fullsized":"2023-10-06T04:53:49.601241+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:26.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592554+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:27.741668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599636+0000","last_change":"2023-10-06T04:53:49.599636+0000","last_active":"2023-10-06T04:53:49.599636+0000","last_peered":"2023-10-06T04:53:49.599636+0000","last_clean":"2023-10-06T04:53:49.599636+0000","last_became_active":"2023-10-06T04:53:49.599356+0000","last_became_peered":"2023-10-06T04:53:49.599356+0000","last_unstale":"2023-10-06T04:53:49.599636+0000","last_undegraded":"2023-10-06T04:53:49.599636+0000","last_fullsized":"2023-10-06T04:53:49.599636+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:50.733425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593469+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608771+0000","last_change":"2023-10-06T04:53:49.608771+0000","last_active":"2023-10-06T04:53:49.608771+0000","last_peered":"2023-10-06T04:53:49.608771+0000","last_clean":"2023-10-06T04:53:49.608771+0000","last_became_active":"2023-10-06T04:53:49.608472+0000","last_became_peered":"2023-10-06T04:53:49.608472+0000","last_unstale":"2023-10-06T04:53:49.608771+0000","last_undegraded":"2023-10-06T04:53:49.608771+0000","last_fullsized":"2023-10-06T04:53:49.608771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:41.942599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590905+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:53:48.590905+0000","last_peered":"2023-10-06T04:53:48.590905+0000","last_clean":"2023-10-06T04:53:48.590905+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:53:48.590905+0000","last_undegraded":"2023-10-06T04:53:48.590905+0000","last_fullsized":"2023-10-06T04:53:48.590905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594775+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.597803+0000","last_change":"2023-10-06T04:53:49.597803+0000","last_active":"2023-10-06T04:53:49.597803+0000","last_peered":"2023-10-06T04:53:49.597803+0000","last_clean":"2023-10-06T04:53:49.597803+0000","last_became_active":"2023-10-06T04:53:49.597467+0000","last_became_peered":"2023-10-06T04:53:49.597467+0000","last_unstale":"2023-10-06T04:53:49.597803+0000","last_undegraded":"2023-10-06T04:53:49.597803+0000","last_fullsized":"2023-10-06T04:53:49.597803+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:42.301398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590629+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:53:48.590629+0000","last_peered":"2023-10-06T04:53:48.590629+0000","last_clean":"2023-10-06T04:53:48.590629+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:53:48.590629+0000","last_undegraded":"2023-10-06T04:53:48.590629+0000","last_fullsized":"2023-10-06T04:53:48.590629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608060+0000","last_change":"2023-10-06T04:53:49.608060+0000","last_active":"2023-10-06T04:53:49.608060+0000","last_peered":"2023-10-06T04:53:49.608060+0000","last_clean":"2023-10-06T04:53:49.608060+0000","last_became_active":"2023-10-06T04:53:49.607815+0000","last_became_peered":"2023-10-06T04:53:49.607815+0000","last_unstale":"2023-10-06T04:53:49.608060+0000","last_undegraded":"2023-10-06T04:53:49.608060+0000","last_fullsized":"2023-10-06T04:53:49.608060+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.584714+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:53:48.584714+0000","last_peered":"2023-10-06T04:53:48.584714+0000","last_clean":"2023-10-06T04:53:48.584714+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:53:48.584714+0000","last_undegraded":"2023-10-06T04:53:48.584714+0000","last_fullsized":"2023-10-06T04:53:48.584714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":179,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594422+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.611927+0000","last_change":"2023-10-06T04:53:49.611927+0000","last_active":"2023-10-06T04:53:49.611927+0000","last_peered":"2023-10-06T04:53:49.611927+0000","last_clean":"2023-10-06T04:53:49.611927+0000","last_became_active":"2023-10-06T04:53:49.611624+0000","last_became_peered":"2023-10-06T04:53:49.611624+0000","last_unstale":"2023-10-06T04:53:49.611927+0000","last_undegraded":"2023-10-06T04:53:49.611927+0000","last_fullsized":"2023-10-06T04:53:49.611927+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585519+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:53:48.585519+0000","last_peered":"2023-10-06T04:53:48.585519+0000","last_clean":"2023-10-06T04:53:48.585519+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:53:48.585519+0000","last_undegraded":"2023-10-06T04:53:48.585519+0000","last_fullsized":"2023-10-06T04:53:48.585519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592939+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:14.972969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607924+0000","last_change":"2023-10-06T04:53:49.607924+0000","last_active":"2023-10-06T04:53:49.607924+0000","last_peered":"2023-10-06T04:53:49.607924+0000","last_clean":"2023-10-06T04:53:49.607924+0000","last_became_active":"2023-10-06T04:53:49.607341+0000","last_became_peered":"2023-10-06T04:53:49.607341+0000","last_unstale":"2023-10-06T04:53:49.607924+0000","last_undegraded":"2023-10-06T04:53:49.607924+0000","last_fullsized":"2023-10-06T04:53:49.607924+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592777+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608707+0000","last_change":"2023-10-06T04:53:49.608707+0000","last_active":"2023-10-06T04:53:49.608707+0000","last_peered":"2023-10-06T04:53:49.608707+0000","last_clean":"2023-10-06T04:53:49.608707+0000","last_became_active":"2023-10-06T04:53:49.608433+0000","last_became_peered":"2023-10-06T04:53:49.608433+0000","last_unstale":"2023-10-06T04:53:49.608707+0000","last_undegraded":"2023-10-06T04:53:49.608707+0000","last_fullsized":"2023-10-06T04:53:49.608707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.587417+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:53:48.587417+0000","last_peered":"2023-10-06T04:53:48.587417+0000","last_clean":"2023-10-06T04:53:48.587417+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:53:48.587417+0000","last_undegraded":"2023-10-06T04:53:48.587417+0000","last_fullsized":"2023-10-06T04:53:48.587417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594963+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613283+0000","last_change":"2023-10-06T04:53:49.613283+0000","last_active":"2023-10-06T04:53:49.613283+0000","last_peered":"2023-10-06T04:53:49.613283+0000","last_clean":"2023-10-06T04:53:49.613283+0000","last_became_active":"2023-10-06T04:53:49.612982+0000","last_became_peered":"2023-10-06T04:53:49.612982+0000","last_unstale":"2023-10-06T04:53:49.613283+0000","last_undegraded":"2023-10-06T04:53:49.613283+0000","last_fullsized":"2023-10-06T04:53:49.613283+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588728+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:53:48.588728+0000","last_peered":"2023-10-06T04:53:48.588728+0000","last_clean":"2023-10-06T04:53:48.588728+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:53:48.588728+0000","last_undegraded":"2023-10-06T04:53:48.588728+0000","last_fullsized":"2023-10-06T04:53:48.588728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598678+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605704+0000","last_change":"2023-10-06T04:53:49.605704+0000","last_active":"2023-10-06T04:53:49.605704+0000","last_peered":"2023-10-06T04:53:49.605704+0000","last_clean":"2023-10-06T04:53:49.605704+0000","last_became_active":"2023-10-06T04:53:49.605449+0000","last_became_peered":"2023-10-06T04:53:49.605449+0000","last_unstale":"2023-10-06T04:53:49.605704+0000","last_undegraded":"2023-10-06T04:53:49.605704+0000","last_fullsized":"2023-10-06T04:53:49.605704+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588915+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:53:48.588915+0000","last_peered":"2023-10-06T04:53:48.588915+0000","last_clean":"2023-10-06T04:53:48.588915+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:53:48.588915+0000","last_undegraded":"2023-10-06T04:53:48.588915+0000","last_fullsized":"2023-10-06T04:53:48.588915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":7.1779999999999999,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.504},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.375},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.376},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.495}]},{"osd":6,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.51800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.25600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.438}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.503}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.372}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.27300000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.069,"5min":5.069,"15min":5.069},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":6.4210000000000003,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4009999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.367,"5min":6.367,"15min":6.367},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":5,"5min":5,"15min":5},"last":0.51600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.501},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":5.4569999999999999,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":5.5510000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.415,"5min":5.415,"15min":5.415},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.202,"5min":4.202,"15min":4.202},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.628}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301860,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069980,"statfs":{"total":96636764160,"available":96327659520,"internally_reserved":0,"allocated":487424,"data_stored":204288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.7949999999999999,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.8170000000000002,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.7210000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:53:55.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:53:55.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.311+0000 7f8e08bcc700 1 Processor -- start 2023-10-06T04:53:55.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.311+0000 7f8e08bcc700 1 -- start start 2023-10-06T04:53:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e08bcc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e04159500 0x7f8e040cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e08bcc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e08bcc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e040b2220 0x7f8e040b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e08bcc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e040cb7b0 con 0x7f8e040b2220 2023-10-06T04:53:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e08bcc700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e040cc5c0 con 0x7f8e04159500 2023-10-06T04:53:55.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e08bcc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040cb9b0 con 0x7f8e041598d0 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e040b2220 0x7f8e040b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e040b2220 0x7f8e040b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40528/0 (socket says 172.21.15.3:40528) 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 -- 172.21.15.3:0/2365037243 learned_addr learned my addr 172.21.15.3:0/2365037243 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e0259c700 1 --2- 172.21.15.3:0/2365037243 >> v2:172.21.15.153:3300/0 conn(0x7f8e04159500 0x7f8e040cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e01d9b700 1 --2- 172.21.15.3:0/2365037243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 -- 172.21.15.3:0/2365037243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 msgr2=0x7f8e040cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 --2- 172.21.15.3:0/2365037243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 -- 172.21.15.3:0/2365037243 >> v2:172.21.15.153:3300/0 conn(0x7f8e04159500 msgr2=0x7f8e040cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 --2- 172.21.15.3:0/2365037243 >> v2:172.21.15.153:3300/0 conn(0x7f8e04159500 0x7f8e040cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e02d9d700 1 -- 172.21.15.3:0/2365037243 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e040cbc90 con 0x7f8e040b2220 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.312+0000 7f8e01d9b700 1 --2- 172.21.15.3:0/2365037243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:55.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e02d9d700 1 --2- 172.21.15.3:0/2365037243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e040b2660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8e04089390 tx=0x7f8dfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f370440bd1499b50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e0159a700 1 -- 172.21.15.3:0/2365037243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00e070 con 0x7f8e040b2220 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e0159a700 1 -- 172.21.15.3:0/2365037243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc00ba50 con 0x7f8e040b2220 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e0159a700 1 -- 172.21.15.3:0/2365037243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0092e0 con 0x7f8e040b2220 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 -- 172.21.15.3:0/2365037243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 msgr2=0x7f8e040b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/2365037243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e040b2660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8e04089390 tx=0x7f8dfc009840 comp rx=0 tx=0).stop 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 -- 172.21.15.3:0/2365037243 shutdown_connections 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/2365037243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e040b2660 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/2365037243 >> v2:172.21.15.153:3300/0 conn(0x7f8e04159500 0x7f8e040cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/2365037243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 -- 172.21.15.3:0/2365037243 >> 172.21.15.3:0/2365037243 conn(0x7f8e04070310 msgr2=0x7f8e04070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 -- 172.21.15.3:0/2365037243 shutdown_connections 2023-10-06T04:53:55.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.313+0000 7f8e08bcc700 1 -- 172.21.15.3:0/2365037243 wait complete. 2023-10-06T04:53:55.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.314+0000 7f8e08bcc700 1 Processor -- start 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e08bcc700 1 -- start start 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e08bcc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e041573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e08bcc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e04159500 0x7f8e04157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e08bcc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e08bcc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e04158c90 con 0x7f8e040b2220 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e08bcc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e04169120 con 0x7f8e04159500 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e08bcc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040b55e0 con 0x7f8e041598d0 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e041573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e041573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40538/0 (socket says 172.21.15.3:40538) 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.315+0000 7f8e0259c700 1 -- 172.21.15.3:0/1050242873 learned_addr learned my addr 172.21.15.3:0/1050242873 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e01d9b700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e04159500 0x7f8e04157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e02d9d700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e0259c700 1 -- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 msgr2=0x7f8e040b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e0259c700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e0259c700 1 -- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e04159500 msgr2=0x7f8e04157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e0259c700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e04159500 0x7f8e04157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e0259c700 1 -- 172.21.15.3:0/1050242873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc009ae0 con 0x7f8e040b2220 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e02d9d700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e01d9b700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e04159500 0x7f8e04157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e0259c700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e041573f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8df400d8c0 tx=0x7f8df4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:55.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8dfa7fc700 1 -- 172.21.15.3:0/1050242873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df400de60 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8dfa7fc700 1 -- 172.21.15.3:0/1050242873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df400b780 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e04158010 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8dfa7fc700 1 -- 172.21.15.3:0/1050242873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df400bb20 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.316+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e041585d0 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.317+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e04071680 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.318+0000 7f8dfa7fc700 1 -- 172.21.15.3:0/1050242873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8df401e460 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.318+0000 7f8dfa7fc700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8de4066f30 0x7f8de40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.318+0000 7f8dfa7fc700 1 -- 172.21.15.3:0/1050242873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8df4084d20 con 0x7f8e040b2220 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.318+0000 7f8e01d9b700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8de4066f30 0x7f8de40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.319+0000 7f8e01d9b700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8de4066f30 0x7f8de40693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e040ce9d0 tx=0x7f8dec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:55.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.322+0000 7f8dfa7fc700 1 -- 172.21.15.3:0/1050242873 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8df4051e30 con 0x7f8e040b2220 2023-10-06T04:53:55.512 INFO:tasks.ceph.mds.d.smithi153.stderr:2023-10-06T04:53:55.511+0000 7f70e07b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:53:55.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.513+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e040cbf70 con 0x7f8de4066f30 2023-10-06T04:53:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.523+0000 7f8dfa7fc700 1 -- 172.21.15.3:0/1050242873 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308294 (secure 0 0 0) 0x7f8e040cbf70 con 0x7f8de4066f30 2023-10-06T04:53:55.528 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8de4066f30 msgr2=0x7f8de40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8de4066f30 0x7f8de40693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e040ce9d0 tx=0x7f8dec008040 comp rx=0 tx=0).stop 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 msgr2=0x7f8e041573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e041573f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8df400d8c0 tx=0x7f8df4008040 comp rx=0 tx=0).stop 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 shutdown_connections 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8de4066f30 0x7f8de40693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e041598d0 0x7f8e040b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e040b2220 0x7f8e041573f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 --2- 172.21.15.3:0/1050242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e04159500 0x7f8e04157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 >> 172.21.15.3:0/1050242873 conn(0x7f8e04070310 msgr2=0x7f8e040cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:55.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 shutdown_connections 2023-10-06T04:53:55.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.526+0000 7f8e08bcc700 1 -- 172.21.15.3:0/1050242873 wait complete. 2023-10-06T04:53:55.532 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:53:55.540 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-10-06T04:53:55.445598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411964,"kb_used_data":4988,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752562756,"statfs":{"total":773094113280,"available":770624262144,"internally_reserved":0,"allocated":5107712,"data_stored":2821805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001476"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593046+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:26.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594750+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598556+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592337+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593553+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592931+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:24.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593327+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590816+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595430+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:10.975006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591645+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.589759+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:04.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590440+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595968+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594915+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596102+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591659+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.599110+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594214+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595173+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594900+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596007+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590728+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590223+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:34.162265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597780+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590762+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591744+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593808+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594471+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596034+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594500+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592863+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601799+0000","last_change":"2023-10-06T04:53:49.601799+0000","last_active":"2023-10-06T04:53:49.601799+0000","last_peered":"2023-10-06T04:53:49.601799+0000","last_clean":"2023-10-06T04:53:49.601799+0000","last_became_active":"2023-10-06T04:53:49.601428+0000","last_became_peered":"2023-10-06T04:53:49.601428+0000","last_unstale":"2023-10-06T04:53:49.601799+0000","last_undegraded":"2023-10-06T04:53:49.601799+0000","last_fullsized":"2023-10-06T04:53:49.601799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:55.184468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593805+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607471+0000","last_change":"2023-10-06T04:53:49.607471+0000","last_active":"2023-10-06T04:53:49.607471+0000","last_peered":"2023-10-06T04:53:49.607471+0000","last_clean":"2023-10-06T04:53:49.607471+0000","last_became_active":"2023-10-06T04:53:49.607158+0000","last_became_peered":"2023-10-06T04:53:49.607158+0000","last_unstale":"2023-10-06T04:53:49.607471+0000","last_undegraded":"2023-10-06T04:53:49.607471+0000","last_fullsized":"2023-10-06T04:53:49.607471+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:41.756942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595565+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605024+0000","last_change":"2023-10-06T04:53:49.605024+0000","last_active":"2023-10-06T04:53:49.605024+0000","last_peered":"2023-10-06T04:53:49.605024+0000","last_clean":"2023-10-06T04:53:49.605024+0000","last_became_active":"2023-10-06T04:53:49.604790+0000","last_became_peered":"2023-10-06T04:53:49.604790+0000","last_unstale":"2023-10-06T04:53:49.605024+0000","last_undegraded":"2023-10-06T04:53:49.605024+0000","last_fullsized":"2023-10-06T04:53:49.605024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:44.358080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595049+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:50.391853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607504+0000","last_change":"2023-10-06T04:53:49.607504+0000","last_active":"2023-10-06T04:53:49.607504+0000","last_peered":"2023-10-06T04:53:49.607504+0000","last_clean":"2023-10-06T04:53:49.607504+0000","last_became_active":"2023-10-06T04:53:49.607263+0000","last_became_peered":"2023-10-06T04:53:49.607263+0000","last_unstale":"2023-10-06T04:53:49.607504+0000","last_undegraded":"2023-10-06T04:53:49.607504+0000","last_fullsized":"2023-10-06T04:53:49.607504+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596208+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606739+0000","last_change":"2023-10-06T04:53:49.606739+0000","last_active":"2023-10-06T04:53:49.606739+0000","last_peered":"2023-10-06T04:53:49.606739+0000","last_clean":"2023-10-06T04:53:49.606739+0000","last_became_active":"2023-10-06T04:53:49.606408+0000","last_became_peered":"2023-10-06T04:53:49.606408+0000","last_unstale":"2023-10-06T04:53:49.606739+0000","last_undegraded":"2023-10-06T04:53:49.606739+0000","last_fullsized":"2023-10-06T04:53:49.606739+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592689+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602410+0000","last_change":"2023-10-06T04:53:49.602410+0000","last_active":"2023-10-06T04:53:49.602410+0000","last_peered":"2023-10-06T04:53:49.602410+0000","last_clean":"2023-10-06T04:53:49.602410+0000","last_became_active":"2023-10-06T04:53:49.602130+0000","last_became_peered":"2023-10-06T04:53:49.602130+0000","last_unstale":"2023-10-06T04:53:49.602410+0000","last_undegraded":"2023-10-06T04:53:49.602410+0000","last_fullsized":"2023-10-06T04:53:49.602410+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594116+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602126+0000","last_change":"2023-10-06T04:53:49.602126+0000","last_active":"2023-10-06T04:53:49.602126+0000","last_peered":"2023-10-06T04:53:49.602126+0000","last_clean":"2023-10-06T04:53:49.602126+0000","last_became_active":"2023-10-06T04:53:49.601062+0000","last_became_peered":"2023-10-06T04:53:49.601062+0000","last_unstale":"2023-10-06T04:53:49.602126+0000","last_undegraded":"2023-10-06T04:53:49.602126+0000","last_fullsized":"2023-10-06T04:53:49.602126+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:54.807253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595466+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610421+0000","last_change":"2023-10-06T04:53:49.610421+0000","last_active":"2023-10-06T04:53:49.610421+0000","last_peered":"2023-10-06T04:53:49.610421+0000","last_clean":"2023-10-06T04:53:49.610421+0000","last_became_active":"2023-10-06T04:53:49.610133+0000","last_became_peered":"2023-10-06T04:53:49.610133+0000","last_unstale":"2023-10-06T04:53:49.610421+0000","last_undegraded":"2023-10-06T04:53:49.610421+0000","last_fullsized":"2023-10-06T04:53:49.610421+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:25.048345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592520+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602037+0000","last_change":"2023-10-06T04:53:49.602037+0000","last_active":"2023-10-06T04:53:49.602037+0000","last_peered":"2023-10-06T04:53:49.602037+0000","last_clean":"2023-10-06T04:53:49.602037+0000","last_became_active":"2023-10-06T04:53:49.601746+0000","last_became_peered":"2023-10-06T04:53:49.601746+0000","last_unstale":"2023-10-06T04:53:49.602037+0000","last_undegraded":"2023-10-06T04:53:49.602037+0000","last_fullsized":"2023-10-06T04:53:49.602037+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594303+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608823+0000","last_change":"2023-10-06T04:53:49.608823+0000","last_active":"2023-10-06T04:53:49.608823+0000","last_peered":"2023-10-06T04:53:49.608823+0000","last_clean":"2023-10-06T04:53:49.608823+0000","last_became_active":"2023-10-06T04:53:49.608542+0000","last_became_peered":"2023-10-06T04:53:49.608542+0000","last_unstale":"2023-10-06T04:53:49.608823+0000","last_undegraded":"2023-10-06T04:53:49.608823+0000","last_fullsized":"2023-10-06T04:53:49.608823+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594373+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610243+0000","last_change":"2023-10-06T04:53:49.610243+0000","last_active":"2023-10-06T04:53:49.610243+0000","last_peered":"2023-10-06T04:53:49.610243+0000","last_clean":"2023-10-06T04:53:49.610243+0000","last_became_active":"2023-10-06T04:53:49.609974+0000","last_became_peered":"2023-10-06T04:53:49.609974+0000","last_unstale":"2023-10-06T04:53:49.610243+0000","last_undegraded":"2023-10-06T04:53:49.610243+0000","last_fullsized":"2023-10-06T04:53:49.610243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592536+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599507+0000","last_change":"2023-10-06T04:53:49.599507+0000","last_active":"2023-10-06T04:53:49.599507+0000","last_peered":"2023-10-06T04:53:49.599507+0000","last_clean":"2023-10-06T04:53:49.599507+0000","last_became_active":"2023-10-06T04:53:49.599244+0000","last_became_peered":"2023-10-06T04:53:49.599244+0000","last_unstale":"2023-10-06T04:53:49.599507+0000","last_undegraded":"2023-10-06T04:53:49.599507+0000","last_fullsized":"2023-10-06T04:53:49.599507+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593289+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602521+0000","last_change":"2023-10-06T04:53:49.602521+0000","last_active":"2023-10-06T04:53:49.602521+0000","last_peered":"2023-10-06T04:53:49.602521+0000","last_clean":"2023-10-06T04:53:49.602521+0000","last_became_active":"2023-10-06T04:53:49.602150+0000","last_became_peered":"2023-10-06T04:53:49.602150+0000","last_unstale":"2023-10-06T04:53:49.602521+0000","last_undegraded":"2023-10-06T04:53:49.602521+0000","last_fullsized":"2023-10-06T04:53:49.602521+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597828+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608700+0000","last_change":"2023-10-06T04:53:49.608700+0000","last_active":"2023-10-06T04:53:49.608700+0000","last_peered":"2023-10-06T04:53:49.608700+0000","last_clean":"2023-10-06T04:53:49.608700+0000","last_became_active":"2023-10-06T04:53:49.608420+0000","last_became_peered":"2023-10-06T04:53:49.608420+0000","last_unstale":"2023-10-06T04:53:49.608700+0000","last_undegraded":"2023-10-06T04:53:49.608700+0000","last_fullsized":"2023-10-06T04:53:49.608700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:24.697185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594521+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606724+0000","last_change":"2023-10-06T04:53:49.606724+0000","last_active":"2023-10-06T04:53:49.606724+0000","last_peered":"2023-10-06T04:53:49.606724+0000","last_clean":"2023-10-06T04:53:49.606724+0000","last_became_active":"2023-10-06T04:53:49.606477+0000","last_became_peered":"2023-10-06T04:53:49.606477+0000","last_unstale":"2023-10-06T04:53:49.606724+0000","last_undegraded":"2023-10-06T04:53:49.606724+0000","last_fullsized":"2023-10-06T04:53:49.606724+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597267+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.598387+0000","last_change":"2023-10-06T04:53:49.598387+0000","last_active":"2023-10-06T04:53:49.598387+0000","last_peered":"2023-10-06T04:53:49.598387+0000","last_clean":"2023-10-06T04:53:49.598387+0000","last_became_active":"2023-10-06T04:53:49.598184+0000","last_became_peered":"2023-10-06T04:53:49.598184+0000","last_unstale":"2023-10-06T04:53:49.598387+0000","last_undegraded":"2023-10-06T04:53:49.598387+0000","last_fullsized":"2023-10-06T04:53:49.598387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608228+0000","last_change":"2023-10-06T04:53:49.608228+0000","last_active":"2023-10-06T04:53:49.608228+0000","last_peered":"2023-10-06T04:53:49.608228+0000","last_clean":"2023-10-06T04:53:49.608228+0000","last_became_active":"2023-10-06T04:53:49.607857+0000","last_became_peered":"2023-10-06T04:53:49.607857+0000","last_unstale":"2023-10-06T04:53:49.608228+0000","last_undegraded":"2023-10-06T04:53:49.608228+0000","last_fullsized":"2023-10-06T04:53:49.608228+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:22.996368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591635+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.600834+0000","last_change":"2023-10-06T04:53:49.600834+0000","last_active":"2023-10-06T04:53:49.600834+0000","last_peered":"2023-10-06T04:53:49.600834+0000","last_clean":"2023-10-06T04:53:49.600834+0000","last_became_active":"2023-10-06T04:53:49.600540+0000","last_became_peered":"2023-10-06T04:53:49.600540+0000","last_unstale":"2023-10-06T04:53:49.600834+0000","last_undegraded":"2023-10-06T04:53:49.600834+0000","last_fullsized":"2023-10-06T04:53:49.600834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:06.484647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593411+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613344+0000","last_change":"2023-10-06T04:53:49.613344+0000","last_active":"2023-10-06T04:53:49.613344+0000","last_peered":"2023-10-06T04:53:49.613344+0000","last_clean":"2023-10-06T04:53:49.613344+0000","last_became_active":"2023-10-06T04:53:49.613045+0000","last_became_peered":"2023-10-06T04:53:49.613045+0000","last_unstale":"2023-10-06T04:53:49.613344+0000","last_undegraded":"2023-10-06T04:53:49.613344+0000","last_fullsized":"2023-10-06T04:53:49.613344+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:29.920075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591953+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.047782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601346+0000","last_change":"2023-10-06T04:53:49.601346+0000","last_active":"2023-10-06T04:53:49.601346+0000","last_peered":"2023-10-06T04:53:49.601346+0000","last_clean":"2023-10-06T04:53:49.601346+0000","last_became_active":"2023-10-06T04:53:49.600993+0000","last_became_peered":"2023-10-06T04:53:49.600993+0000","last_unstale":"2023-10-06T04:53:49.601346+0000","last_undegraded":"2023-10-06T04:53:49.601346+0000","last_fullsized":"2023-10-06T04:53:49.601346+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:58.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592361+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:26.053487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606428+0000","last_change":"2023-10-06T04:53:49.606428+0000","last_active":"2023-10-06T04:53:49.606428+0000","last_peered":"2023-10-06T04:53:49.606428+0000","last_clean":"2023-10-06T04:53:49.606428+0000","last_became_active":"2023-10-06T04:53:49.606178+0000","last_became_peered":"2023-10-06T04:53:49.606178+0000","last_unstale":"2023-10-06T04:53:49.606428+0000","last_undegraded":"2023-10-06T04:53:49.606428+0000","last_fullsized":"2023-10-06T04:53:49.606428+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595074+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599873+0000","last_change":"2023-10-06T04:53:49.599873+0000","last_active":"2023-10-06T04:53:49.599873+0000","last_peered":"2023-10-06T04:53:49.599873+0000","last_clean":"2023-10-06T04:53:49.599873+0000","last_became_active":"2023-10-06T04:53:49.599582+0000","last_became_peered":"2023-10-06T04:53:49.599582+0000","last_unstale":"2023-10-06T04:53:49.599873+0000","last_undegraded":"2023-10-06T04:53:49.599873+0000","last_fullsized":"2023-10-06T04:53:49.599873+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598198+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601241+0000","last_change":"2023-10-06T04:53:49.601241+0000","last_active":"2023-10-06T04:53:49.601241+0000","last_peered":"2023-10-06T04:53:49.601241+0000","last_clean":"2023-10-06T04:53:49.601241+0000","last_became_active":"2023-10-06T04:53:49.600982+0000","last_became_peered":"2023-10-06T04:53:49.600982+0000","last_unstale":"2023-10-06T04:53:49.601241+0000","last_undegraded":"2023-10-06T04:53:49.601241+0000","last_fullsized":"2023-10-06T04:53:49.601241+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:26.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592554+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:27.741668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599636+0000","last_change":"2023-10-06T04:53:49.599636+0000","last_active":"2023-10-06T04:53:49.599636+0000","last_peered":"2023-10-06T04:53:49.599636+0000","last_clean":"2023-10-06T04:53:49.599636+0000","last_became_active":"2023-10-06T04:53:49.599356+0000","last_became_peered":"2023-10-06T04:53:49.599356+0000","last_unstale":"2023-10-06T04:53:49.599636+0000","last_undegraded":"2023-10-06T04:53:49.599636+0000","last_fullsized":"2023-10-06T04:53:49.599636+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:50.733425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593469+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608771+0000","last_change":"2023-10-06T04:53:49.608771+0000","last_active":"2023-10-06T04:53:49.608771+0000","last_peered":"2023-10-06T04:53:49.608771+0000","last_clean":"2023-10-06T04:53:49.608771+0000","last_became_active":"2023-10-06T04:53:49.608472+0000","last_became_peered":"2023-10-06T04:53:49.608472+0000","last_unstale":"2023-10-06T04:53:49.608771+0000","last_undegraded":"2023-10-06T04:53:49.608771+0000","last_fullsized":"2023-10-06T04:53:49.608771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:41.942599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590905+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:53:48.590905+0000","last_peered":"2023-10-06T04:53:48.590905+0000","last_clean":"2023-10-06T04:53:48.590905+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:53:48.590905+0000","last_undegraded":"2023-10-06T04:53:48.590905+0000","last_fullsized":"2023-10-06T04:53:48.590905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594775+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.597803+0000","last_change":"2023-10-06T04:53:49.597803+0000","last_active":"2023-10-06T04:53:49.597803+0000","last_peered":"2023-10-06T04:53:49.597803+0000","last_clean":"2023-10-06T04:53:49.597803+0000","last_became_active":"2023-10-06T04:53:49.597467+0000","last_became_peered":"2023-10-06T04:53:49.597467+0000","last_unstale":"2023-10-06T04:53:49.597803+0000","last_undegraded":"2023-10-06T04:53:49.597803+0000","last_fullsized":"2023-10-06T04:53:49.597803+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:42.301398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590629+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:53:48.590629+0000","last_peered":"2023-10-06T04:53:48.590629+0000","last_clean":"2023-10-06T04:53:48.590629+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:53:48.590629+0000","last_undegraded":"2023-10-06T04:53:48.590629+0000","last_fullsized":"2023-10-06T04:53:48.590629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608060+0000","last_change":"2023-10-06T04:53:49.608060+0000","last_active":"2023-10-06T04:53:49.608060+0000","last_peered":"2023-10-06T04:53:49.608060+0000","last_clean":"2023-10-06T04:53:49.608060+0000","last_became_active":"2023-10-06T04:53:49.607815+0000","last_became_peered":"2023-10-06T04:53:49.607815+0000","last_unstale":"2023-10-06T04:53:49.608060+0000","last_undegraded":"2023-10-06T04:53:49.608060+0000","last_fullsized":"2023-10-06T04:53:49.608060+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.584714+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:53:48.584714+0000","last_peered":"2023-10-06T04:53:48.584714+0000","last_clean":"2023-10-06T04:53:48.584714+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:53:48.584714+0000","last_undegraded":"2023-10-06T04:53:48.584714+0000","last_fullsized":"2023-10-06T04:53:48.584714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":179,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594422+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.611927+0000","last_change":"2023-10-06T04:53:49.611927+0000","last_active":"2023-10-06T04:53:49.611927+0000","last_peered":"2023-10-06T04:53:49.611927+0000","last_clean":"2023-10-06T04:53:49.611927+0000","last_became_active":"2023-10-06T04:53:49.611624+0000","last_became_peered":"2023-10-06T04:53:49.611624+0000","last_unstale":"2023-10-06T04:53:49.611927+0000","last_undegraded":"2023-10-06T04:53:49.611927+0000","last_fullsized":"2023-10-06T04:53:49.611927+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585519+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:53:48.585519+0000","last_peered":"2023-10-06T04:53:48.585519+0000","last_clean":"2023-10-06T04:53:48.585519+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:53:48.585519+0000","last_undegraded":"2023-10-06T04:53:48.585519+0000","last_fullsized":"2023-10-06T04:53:48.585519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592939+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:14.972969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607924+0000","last_change":"2023-10-06T04:53:49.607924+0000","last_active":"2023-10-06T04:53:49.607924+0000","last_peered":"2023-10-06T04:53:49.607924+0000","last_clean":"2023-10-06T04:53:49.607924+0000","last_became_active":"2023-10-06T04:53:49.607341+0000","last_became_peered":"2023-10-06T04:53:49.607341+0000","last_unstale":"2023-10-06T04:53:49.607924+0000","last_undegraded":"2023-10-06T04:53:49.607924+0000","last_fullsized":"2023-10-06T04:53:49.607924+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592777+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608707+0000","last_change":"2023-10-06T04:53:49.608707+0000","last_active":"2023-10-06T04:53:49.608707+0000","last_peered":"2023-10-06T04:53:49.608707+0000","last_clean":"2023-10-06T04:53:49.608707+0000","last_became_active":"2023-10-06T04:53:49.608433+0000","last_became_peered":"2023-10-06T04:53:49.608433+0000","last_unstale":"2023-10-06T04:53:49.608707+0000","last_undegraded":"2023-10-06T04:53:49.608707+0000","last_fullsized":"2023-10-06T04:53:49.608707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.587417+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:53:48.587417+0000","last_peered":"2023-10-06T04:53:48.587417+0000","last_clean":"2023-10-06T04:53:48.587417+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:53:48.587417+0000","last_undegraded":"2023-10-06T04:53:48.587417+0000","last_fullsized":"2023-10-06T04:53:48.587417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594963+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613283+0000","last_change":"2023-10-06T04:53:49.613283+0000","last_active":"2023-10-06T04:53:49.613283+0000","last_peered":"2023-10-06T04:53:49.613283+0000","last_clean":"2023-10-06T04:53:49.613283+0000","last_became_active":"2023-10-06T04:53:49.612982+0000","last_became_peered":"2023-10-06T04:53:49.612982+0000","last_unstale":"2023-10-06T04:53:49.613283+0000","last_undegraded":"2023-10-06T04:53:49.613283+0000","last_fullsized":"2023-10-06T04:53:49.613283+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588728+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:53:48.588728+0000","last_peered":"2023-10-06T04:53:48.588728+0000","last_clean":"2023-10-06T04:53:48.588728+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:53:48.588728+0000","last_undegraded":"2023-10-06T04:53:48.588728+0000","last_fullsized":"2023-10-06T04:53:48.588728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598678+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605704+0000","last_change":"2023-10-06T04:53:49.605704+0000","last_active":"2023-10-06T04:53:49.605704+0000","last_peered":"2023-10-06T04:53:49.605704+0000","last_clean":"2023-10-06T04:53:49.605704+0000","last_became_active":"2023-10-06T04:53:49.605449+0000","last_became_peered":"2023-10-06T04:53:49.605449+0000","last_unstale":"2023-10-06T04:53:49.605704+0000","last_undegraded":"2023-10-06T04:53:49.605704+0000","last_fullsized":"2023-10-06T04:53:49.605704+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588915+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:53:48.588915+0000","last_peered":"2023-10-06T04:53:48.588915+0000","last_clean":"2023-10-06T04:53:48.588915+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:53:48.588915+0000","last_undegraded":"2023-10-06T04:53:48.588915+0000","last_fullsized":"2023-10-06T04:53:48.588915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":7.1779999999999999,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.504},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.375},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.376},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.495}]},{"osd":6,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.51800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.25600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.438}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.503}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.372}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.27300000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.069,"5min":5.069,"15min":5.069},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":6.4210000000000003,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4009999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.367,"5min":6.367,"15min":6.367},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":5,"5min":5,"15min":5},"last":0.51600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.501},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":5.4569999999999999,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":5.5510000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.415,"5min":5.415,"15min":5.415},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.202,"5min":4.202,"15min":4.202},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.628}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301860,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069980,"statfs":{"total":96636764160,"available":96327659520,"internally_reserved":0,"allocated":487424,"data_stored":204288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.7949999999999999,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.8170000000000002,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.7210000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:53:55.547 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2023-10-06T04:53:55.547 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593046+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:47:26.211024+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.547 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2023-10-06T04:53:55.547 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594750+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.547 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2023-10-06T04:53:55.547 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.598556+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592337+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593553+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592931+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:32:24.001510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593327+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2023-10-06T04:53:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.590816+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595430+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:17:10.975006+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.591645+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.589759+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:17:04.678555+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.590440+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.549 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595968+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594915+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.596102+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.591659+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2023-10-06T04:53:55.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.599110+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594214+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595173+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594900+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2023-10-06T04:53:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.596007+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.590728+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.590223+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:10:34.162265+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595325+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.597780+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2023-10-06T04:53:55.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.590762+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.591744+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593808+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594471+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.596034+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594500+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593281+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592863+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593805+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-10-06T04:53:55.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595565+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595049+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:14:50.391853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.596208+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592689+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594116+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595466+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592520+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594303+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594373+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-10-06T04:53:55.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592536+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593289+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.597828+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594521+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-10-06T04:53:55.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.597267+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.590325+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.591635+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593411+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.591953+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:35:47.047782+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592361+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:49:26.053487+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.595074+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.598198+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592554+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:11:27.741668+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-10-06T04:53:55.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.593469+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594775+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592281+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594422+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592939+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:29:14.972969+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:53:55.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-10-06T04:53:55.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.592777+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:53:55.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-10-06T04:53:55.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.594963+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:53:55.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-10-06T04:53:55.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:53:49.582035+0000', 'last_change': '2023-10-06T04:53:49.598678+0000', 'last_active': '2023-10-06T04:53:49.582035+0000', 'last_peered': '2023-10-06T04:53:49.582035+0000', 'last_clean': '2023-10-06T04:53:49.582035+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:53:49.582035+0000', 'last_undegraded': '2023-10-06T04:53:49.582035+0000', 'last_fullsized': '2023-10-06T04:53:49.582035+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'last_clean_scrub_stamp': '2023-10-06T04:53:49.582035+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:53:55.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T04:53:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.697+0000 7f8013b6a700 1 Processor -- start 2023-10-06T04:53:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.697+0000 7f8013b6a700 1 -- start start 2023-10-06T04:53:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8013b6a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f800c15d240 0x7f800c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8013b6a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f800c0cd320 0x7f800c0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8013b6a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c15d6d0 con 0x7f800c0b27b0 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8013b6a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c15aba0 con 0x7f800c0cd320 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8013b6a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c150b30 con 0x7f800c15d240 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8012107700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f800c0cd320 0x7f800c0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8012107700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f800c0cd320 0x7f800c0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40550/0 (socket says 172.21.15.3:40550) 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8012107700 1 -- 172.21.15.3:0/4275857358 learned_addr learned my addr 172.21.15.3:0/4275857358 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8011105700 1 --2- 172.21.15.3:0/4275857358 >> v2:172.21.15.153:3300/0 conn(0x7f800c15d240 0x7f800c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8012107700 1 -- 172.21.15.3:0/4275857358 >> v2:172.21.15.153:3300/0 conn(0x7f800c15d240 msgr2=0x7f800c15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8012107700 1 --2- 172.21.15.3:0/4275857358 >> v2:172.21.15.153:3300/0 conn(0x7f800c15d240 0x7f800c15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8012107700 1 -- 172.21.15.3:0/4275857358 >> v1:172.21.15.153:6789/0 conn(0x7f800c0b27b0 legacy=0x7f800c0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.698+0000 7f8012107700 1 -- 172.21.15.3:0/4275857358 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c15d9b0 con 0x7f800c0cd320 2023-10-06T04:53:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8012107700 1 --2- 172.21.15.3:0/4275857358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0cd320 0x7f800c0aeef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f800c0cc930 tx=0x7f8004009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df30b5f67158541 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:55.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8010904700 1 -- 172.21.15.3:0/4275857358 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400e070 con 0x7f800c0cd320 2023-10-06T04:53:55.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8010904700 1 -- 172.21.15.3:0/4275857358 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800400ba10 con 0x7f800c0cd320 2023-10-06T04:53:55.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8010904700 1 -- 172.21.15.3:0/4275857358 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040092e0 con 0x7f800c0cd320 2023-10-06T04:53:55.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 -- 172.21.15.3:0/4275857358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0cd320 msgr2=0x7f800c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 --2- 172.21.15.3:0/4275857358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0cd320 0x7f800c0aeef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f800c0cc930 tx=0x7f8004009840 comp rx=0 tx=0).stop 2023-10-06T04:53:55.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 -- 172.21.15.3:0/4275857358 shutdown_connections 2023-10-06T04:53:55.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 --2- 172.21.15.3:0/4275857358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0cd320 0x7f800c0aeef0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 --2- 172.21.15.3:0/4275857358 >> v2:172.21.15.153:3300/0 conn(0x7f800c15d240 0x7f800c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 -- 172.21.15.3:0/4275857358 >> 172.21.15.3:0/4275857358 conn(0x7f800c070310 msgr2=0x7f800c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 -- 172.21.15.3:0/4275857358 shutdown_connections 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.699+0000 7f8013b6a700 1 -- 172.21.15.3:0/4275857358 wait complete. 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.700+0000 7f8013b6a700 1 Processor -- start 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8013b6a700 1 -- start start 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8013b6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0b27b0 0x7f800c14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8013b6a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f800c0cd320 0x7f800c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8013b6a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f800c15d240 0x7f800c150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8013b6a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0b4210 con 0x7f800c0b27b0 2023-10-06T04:53:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8013b6a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0b3600 con 0x7f800c0cd320 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8013b6a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0b38e0 con 0x7f800c15d240 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8011906700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0b27b0 0x7f800c14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8011906700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0b27b0 0x7f800c14fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40556/0 (socket says 172.21.15.3:40556) 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8011906700 1 -- 172.21.15.3:0/2829339139 learned_addr learned my addr 172.21.15.3:0/2829339139 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8011105700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f800c0cd320 0x7f800c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.701+0000 7f8012107700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f800c15d240 0x7f800c150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8011906700 1 -- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f800c15d240 msgr2=0x7f800c150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8011906700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f800c15d240 0x7f800c150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8011906700 1 -- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f800c0cd320 msgr2=0x7f800c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8011906700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f800c0cd320 0x7f800c15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8011906700 1 -- 172.21.15.3:0/2829339139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004009ae0 con 0x7f800c0b27b0 2023-10-06T04:53:55.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8011906700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0b27b0 0x7f800c14fe70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c5d0 tx=0x7f7ffc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8009ffb700 1 -- 172.21.15.3:0/2829339139 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011590 con 0x7f800c0b27b0 2023-10-06T04:53:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8009ffb700 1 -- 172.21.15.3:0/2829339139 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc011710 con 0x7f800c0b27b0 2023-10-06T04:53:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c0b3bc0 con 0x7f800c0b27b0 2023-10-06T04:53:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8009ffb700 1 -- 172.21.15.3:0/2829339139 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011ad0 con 0x7f800c0b27b0 2023-10-06T04:53:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.702+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c0c7ab0 con 0x7f800c0b27b0 2023-10-06T04:53:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.703+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c071680 con 0x7f800c0b27b0 2023-10-06T04:53:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.704+0000 7f8009ffb700 1 -- 172.21.15.3:0/2829339139 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ffc01c460 con 0x7f800c0b27b0 2023-10-06T04:53:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.704+0000 7f8009ffb700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ff8066f30 0x7f7ff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.704+0000 7f8009ffb700 1 -- 172.21.15.3:0/2829339139 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7ffc0843d0 con 0x7f800c0b27b0 2023-10-06T04:53:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.704+0000 7f8011105700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ff8066f30 0x7f7ff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.705+0000 7f8011105700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ff8066f30 0x7f7ff80693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f800c0cb310 tx=0x7f8000008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:55.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.708+0000 7f8009ffb700 1 -- 172.21.15.3:0/2829339139 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ffc0514e0 con 0x7f800c0b27b0 2023-10-06T04:53:55.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.982+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f800c0cb950 con 0x7f800c0b27b0 2023-10-06T04:53:55.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.983+0000 7f8009ffb700 1 -- 172.21.15.3:0/2829339139 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1456 (secure 0 0 0) 0x7f7ffc054b00 con 0x7f800c0b27b0 2023-10-06T04:53:55.985 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:55.985 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":116,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1696567930,"num_in_osds":8,"osd_in_since":1696567922,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2469851136,"bytes_avail":770624262144,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5361}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-06T04:53:37.441716+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T04:53:55.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ff8066f30 msgr2=0x7f7ff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ff8066f30 0x7f7ff80693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f800c0cb310 tx=0x7f8000008040 comp rx=0 tx=0).stop 2023-10-06T04:53:55.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0b27b0 msgr2=0x7f800c14fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0b27b0 0x7f800c14fe70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c5d0 tx=0x7f7ffc007f60 comp rx=0 tx=0).stop 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 shutdown_connections 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ff8066f30 0x7f7ff80693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f800c15d240 0x7f800c150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800c0b27b0 0x7f800c14fe70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 --2- 172.21.15.3:0/2829339139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f800c0cd320 0x7f800c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 >> 172.21.15.3:0/2829339139 conn(0x7f800c070310 msgr2=0x7f800c0b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 shutdown_connections 2023-10-06T04:53:55.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:55.985+0000 7f8013b6a700 1 -- 172.21.15.3:0/2829339139 wait complete. 2023-10-06T04:53:55.999 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T04:53:55.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:53:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.115+0000 7f3d9c9ae700 1 Processor -- start 2023-10-06T04:53:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.115+0000 7f3d9c9ae700 1 -- start start 2023-10-06T04:53:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.116+0000 7f3d9c9ae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d9815c8f0 0x7f3d9815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.116+0000 7f3d9c9ae700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d980d0870 0x7f3d980b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.116+0000 7f3d9c9ae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d980cf6c0 con 0x7f3d9815ccc0 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.116+0000 7f3d9c9ae700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d980cfb50 con 0x7f3d9815c8f0 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.116+0000 7f3d9c9ae700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9815e050 con 0x7f3d980d0870 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d9815c8f0 0x7f3d9815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d95d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3d9815ccc0 0x7f3d980d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48994/0 (socket says 172.21.15.3:48994) 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d95d9b700 1 -- 172.21.15.3:0/1437279330 learned_addr learned my addr 172.21.15.3:0/1437279330 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d96d9d700 1 --2- 172.21.15.3:0/1437279330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d980d0870 0x7f3d980b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9659c700 1 -- 172.21.15.3:0/1437279330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d980d0870 msgr2=0x7f3d980b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9659c700 1 --2- 172.21.15.3:0/1437279330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d980d0870 0x7f3d980b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9659c700 1 -- 172.21.15.3:0/1437279330 >> v1:172.21.15.3:6789/0 conn(0x7f3d9815ccc0 legacy=0x7f3d980d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9659c700 1 -- 172.21.15.3:0/1437279330 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d980ceab0 con 0x7f3d9815c8f0 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9659c700 1 --2- 172.21.15.3:0/1437279330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815c8f0 0x7f3d9815db10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3d880098e0 tx=0x7f3d880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5575d305ff42cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9559a700 1 -- 172.21.15.3:0/1437279330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88009e80 con 0x7f3d9815c8f0 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9559a700 1 -- 172.21.15.3:0/1437279330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8800b860 con 0x7f3d9815c8f0 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.117+0000 7f3d9559a700 1 -- 172.21.15.3:0/1437279330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8800bbe0 con 0x7f3d9815c8f0 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/1437279330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815c8f0 msgr2=0x7f3d9815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/1437279330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815c8f0 0x7f3d9815db10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3d880098e0 tx=0x7f3d880091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/1437279330 shutdown_connections 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/1437279330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d980d0870 0x7f3d980b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/1437279330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815c8f0 0x7f3d9815db10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/1437279330 >> 172.21.15.3:0/1437279330 conn(0x7f3d98070310 msgr2=0x7f3d98070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/1437279330 shutdown_connections 2023-10-06T04:53:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.118+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/1437279330 wait complete. 2023-10-06T04:53:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.119+0000 7f3d9c9ae700 1 Processor -- start 2023-10-06T04:53:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.119+0000 7f3d9c9ae700 1 -- start start 2023-10-06T04:53:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d9c9ae700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d980d0870 0x7f3d981502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d9c9ae700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d9815c8f0 0x7f3d98153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d9c9ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815ccc0 0x7f3d98153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d9c9ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d981541c0 con 0x7f3d9815ccc0 2023-10-06T04:53:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d9c9ae700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9814ed80 con 0x7f3d980d0870 2023-10-06T04:53:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d9c9ae700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98169ae0 con 0x7f3d9815c8f0 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815ccc0 0x7f3d98153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815ccc0 0x7f3d98153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40578/0 (socket says 172.21.15.3:40578) 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 -- 172.21.15.3:0/871969201 learned_addr learned my addr 172.21.15.3:0/871969201 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d9659c700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d980d0870 0x7f3d981502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d95d9b700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d9815c8f0 0x7f3d98153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 -- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d9815c8f0 msgr2=0x7f3d98153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d9815c8f0 0x7f3d98153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 -- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d980d0870 msgr2=0x7f3d981502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d980d0870 0x7f3d981502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 -- 172.21.15.3:0/871969201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d88009420 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d95d9b700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d9815c8f0 0x7f3d98153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.120+0000 7f3d96d9d700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815ccc0 0x7f3d98153c80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007ef0 tx=0x7f3d90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.121+0000 7f3d8e7fc700 1 -- 172.21.15.3:0/871969201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000ebe0 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.121+0000 7f3d8e7fc700 1 -- 172.21.15.3:0/871969201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9000ed40 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.121+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d98169dc0 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.121+0000 7f3d8e7fc700 1 -- 172.21.15.3:0/871969201 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90014670 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.121+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9816a400 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.122+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d98071680 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.122+0000 7f3d8e7fc700 1 -- 172.21.15.3:0/871969201 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d90014a60 con 0x7f3d9815ccc0 2023-10-06T04:53:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.122+0000 7f3d8e7fc700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.123+0000 7f3d8e7fc700 1 -- 172.21.15.3:0/871969201 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d90084cd0 con 0x7f3d9815ccc0 2023-10-06T04:53:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.123+0000 7f3d9659c700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.123+0000 7f3d9659c700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d880091e0 tx=0x7f3d88017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.127+0000 7f3d8e7fc700 1 -- 172.21.15.3:0/871969201 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d90051de0 con 0x7f3d9815ccc0 2023-10-06T04:53:56.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.316+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d981516d0 con 0x7f3d78066f30 2023-10-06T04:53:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.325+0000 7f3d8e7fc700 1 -- 172.21.15.3:0/871969201 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308294 (secure 0 0 0) 0x7f3d981516d0 con 0x7f3d78066f30 2023-10-06T04:53:56.331 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 msgr2=0x7f3d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d880091e0 tx=0x7f3d88017000 comp rx=0 tx=0).stop 2023-10-06T04:53:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815ccc0 msgr2=0x7f3d98153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815ccc0 0x7f3d98153c80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007ef0 tx=0x7f3d90009840 comp rx=0 tx=0).stop 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 shutdown_connections 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d980d0870 0x7f3d981502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9815ccc0 0x7f3d98153c80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 --2- 172.21.15.3:0/871969201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d9815c8f0 0x7f3d98153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 >> 172.21.15.3:0/871969201 conn(0x7f3d98070310 msgr2=0x7f3d9815f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 shutdown_connections 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:56.328+0000 7f3d9c9ae700 1 -- 172.21.15.3:0/871969201 wait complete. 2023-10-06T04:53:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:53:56.342 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-10-06T04:53:55.445598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411964,"kb_used_data":4988,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752562756,"statfs":{"total":773094113280,"available":770624262144,"internally_reserved":0,"allocated":5107712,"data_stored":2821805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001476"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593046+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:26.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594750+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598556+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592337+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593553+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592931+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:24.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593327+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590816+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595430+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:10.975006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591645+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.589759+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:04.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590440+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595968+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594915+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596102+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591659+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.599110+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594214+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595173+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594900+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596007+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590728+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590223+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:34.162265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597780+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590762+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591744+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:47.600972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593808+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594471+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596034+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594500+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592863+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601799+0000","last_change":"2023-10-06T04:53:49.601799+0000","last_active":"2023-10-06T04:53:49.601799+0000","last_peered":"2023-10-06T04:53:49.601799+0000","last_clean":"2023-10-06T04:53:49.601799+0000","last_became_active":"2023-10-06T04:53:49.601428+0000","last_became_peered":"2023-10-06T04:53:49.601428+0000","last_unstale":"2023-10-06T04:53:49.601799+0000","last_undegraded":"2023-10-06T04:53:49.601799+0000","last_fullsized":"2023-10-06T04:53:49.601799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:55.184468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593805+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607471+0000","last_change":"2023-10-06T04:53:49.607471+0000","last_active":"2023-10-06T04:53:49.607471+0000","last_peered":"2023-10-06T04:53:49.607471+0000","last_clean":"2023-10-06T04:53:49.607471+0000","last_became_active":"2023-10-06T04:53:49.607158+0000","last_became_peered":"2023-10-06T04:53:49.607158+0000","last_unstale":"2023-10-06T04:53:49.607471+0000","last_undegraded":"2023-10-06T04:53:49.607471+0000","last_fullsized":"2023-10-06T04:53:49.607471+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:41.756942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595565+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:03.100998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605024+0000","last_change":"2023-10-06T04:53:49.605024+0000","last_active":"2023-10-06T04:53:49.605024+0000","last_peered":"2023-10-06T04:53:49.605024+0000","last_clean":"2023-10-06T04:53:49.605024+0000","last_became_active":"2023-10-06T04:53:49.604790+0000","last_became_peered":"2023-10-06T04:53:49.604790+0000","last_unstale":"2023-10-06T04:53:49.605024+0000","last_undegraded":"2023-10-06T04:53:49.605024+0000","last_fullsized":"2023-10-06T04:53:49.605024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:44.358080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595049+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:50.391853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607504+0000","last_change":"2023-10-06T04:53:49.607504+0000","last_active":"2023-10-06T04:53:49.607504+0000","last_peered":"2023-10-06T04:53:49.607504+0000","last_clean":"2023-10-06T04:53:49.607504+0000","last_became_active":"2023-10-06T04:53:49.607263+0000","last_became_peered":"2023-10-06T04:53:49.607263+0000","last_unstale":"2023-10-06T04:53:49.607504+0000","last_undegraded":"2023-10-06T04:53:49.607504+0000","last_fullsized":"2023-10-06T04:53:49.607504+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.596208+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606739+0000","last_change":"2023-10-06T04:53:49.606739+0000","last_active":"2023-10-06T04:53:49.606739+0000","last_peered":"2023-10-06T04:53:49.606739+0000","last_clean":"2023-10-06T04:53:49.606739+0000","last_became_active":"2023-10-06T04:53:49.606408+0000","last_became_peered":"2023-10-06T04:53:49.606408+0000","last_unstale":"2023-10-06T04:53:49.606739+0000","last_undegraded":"2023-10-06T04:53:49.606739+0000","last_fullsized":"2023-10-06T04:53:49.606739+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592689+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602410+0000","last_change":"2023-10-06T04:53:49.602410+0000","last_active":"2023-10-06T04:53:49.602410+0000","last_peered":"2023-10-06T04:53:49.602410+0000","last_clean":"2023-10-06T04:53:49.602410+0000","last_became_active":"2023-10-06T04:53:49.602130+0000","last_became_peered":"2023-10-06T04:53:49.602130+0000","last_unstale":"2023-10-06T04:53:49.602410+0000","last_undegraded":"2023-10-06T04:53:49.602410+0000","last_fullsized":"2023-10-06T04:53:49.602410+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594116+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602126+0000","last_change":"2023-10-06T04:53:49.602126+0000","last_active":"2023-10-06T04:53:49.602126+0000","last_peered":"2023-10-06T04:53:49.602126+0000","last_clean":"2023-10-06T04:53:49.602126+0000","last_became_active":"2023-10-06T04:53:49.601062+0000","last_became_peered":"2023-10-06T04:53:49.601062+0000","last_unstale":"2023-10-06T04:53:49.602126+0000","last_undegraded":"2023-10-06T04:53:49.602126+0000","last_fullsized":"2023-10-06T04:53:49.602126+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:54.807253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595466+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610421+0000","last_change":"2023-10-06T04:53:49.610421+0000","last_active":"2023-10-06T04:53:49.610421+0000","last_peered":"2023-10-06T04:53:49.610421+0000","last_clean":"2023-10-06T04:53:49.610421+0000","last_became_active":"2023-10-06T04:53:49.610133+0000","last_became_peered":"2023-10-06T04:53:49.610133+0000","last_unstale":"2023-10-06T04:53:49.610421+0000","last_undegraded":"2023-10-06T04:53:49.610421+0000","last_fullsized":"2023-10-06T04:53:49.610421+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:25.048345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592520+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602037+0000","last_change":"2023-10-06T04:53:49.602037+0000","last_active":"2023-10-06T04:53:49.602037+0000","last_peered":"2023-10-06T04:53:49.602037+0000","last_clean":"2023-10-06T04:53:49.602037+0000","last_became_active":"2023-10-06T04:53:49.601746+0000","last_became_peered":"2023-10-06T04:53:49.601746+0000","last_unstale":"2023-10-06T04:53:49.602037+0000","last_undegraded":"2023-10-06T04:53:49.602037+0000","last_fullsized":"2023-10-06T04:53:49.602037+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594303+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608823+0000","last_change":"2023-10-06T04:53:49.608823+0000","last_active":"2023-10-06T04:53:49.608823+0000","last_peered":"2023-10-06T04:53:49.608823+0000","last_clean":"2023-10-06T04:53:49.608823+0000","last_became_active":"2023-10-06T04:53:49.608542+0000","last_became_peered":"2023-10-06T04:53:49.608542+0000","last_unstale":"2023-10-06T04:53:49.608823+0000","last_undegraded":"2023-10-06T04:53:49.608823+0000","last_fullsized":"2023-10-06T04:53:49.608823+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594373+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.610243+0000","last_change":"2023-10-06T04:53:49.610243+0000","last_active":"2023-10-06T04:53:49.610243+0000","last_peered":"2023-10-06T04:53:49.610243+0000","last_clean":"2023-10-06T04:53:49.610243+0000","last_became_active":"2023-10-06T04:53:49.609974+0000","last_became_peered":"2023-10-06T04:53:49.609974+0000","last_unstale":"2023-10-06T04:53:49.610243+0000","last_undegraded":"2023-10-06T04:53:49.610243+0000","last_fullsized":"2023-10-06T04:53:49.610243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592536+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599507+0000","last_change":"2023-10-06T04:53:49.599507+0000","last_active":"2023-10-06T04:53:49.599507+0000","last_peered":"2023-10-06T04:53:49.599507+0000","last_clean":"2023-10-06T04:53:49.599507+0000","last_became_active":"2023-10-06T04:53:49.599244+0000","last_became_peered":"2023-10-06T04:53:49.599244+0000","last_unstale":"2023-10-06T04:53:49.599507+0000","last_undegraded":"2023-10-06T04:53:49.599507+0000","last_fullsized":"2023-10-06T04:53:49.599507+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593289+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.602521+0000","last_change":"2023-10-06T04:53:49.602521+0000","last_active":"2023-10-06T04:53:49.602521+0000","last_peered":"2023-10-06T04:53:49.602521+0000","last_clean":"2023-10-06T04:53:49.602521+0000","last_became_active":"2023-10-06T04:53:49.602150+0000","last_became_peered":"2023-10-06T04:53:49.602150+0000","last_unstale":"2023-10-06T04:53:49.602521+0000","last_undegraded":"2023-10-06T04:53:49.602521+0000","last_fullsized":"2023-10-06T04:53:49.602521+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597828+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608700+0000","last_change":"2023-10-06T04:53:49.608700+0000","last_active":"2023-10-06T04:53:49.608700+0000","last_peered":"2023-10-06T04:53:49.608700+0000","last_clean":"2023-10-06T04:53:49.608700+0000","last_became_active":"2023-10-06T04:53:49.608420+0000","last_became_peered":"2023-10-06T04:53:49.608420+0000","last_unstale":"2023-10-06T04:53:49.608700+0000","last_undegraded":"2023-10-06T04:53:49.608700+0000","last_fullsized":"2023-10-06T04:53:49.608700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:24.697185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594521+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606724+0000","last_change":"2023-10-06T04:53:49.606724+0000","last_active":"2023-10-06T04:53:49.606724+0000","last_peered":"2023-10-06T04:53:49.606724+0000","last_clean":"2023-10-06T04:53:49.606724+0000","last_became_active":"2023-10-06T04:53:49.606477+0000","last_became_peered":"2023-10-06T04:53:49.606477+0000","last_unstale":"2023-10-06T04:53:49.606724+0000","last_undegraded":"2023-10-06T04:53:49.606724+0000","last_fullsized":"2023-10-06T04:53:49.606724+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.597267+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.598387+0000","last_change":"2023-10-06T04:53:49.598387+0000","last_active":"2023-10-06T04:53:49.598387+0000","last_peered":"2023-10-06T04:53:49.598387+0000","last_clean":"2023-10-06T04:53:49.598387+0000","last_became_active":"2023-10-06T04:53:49.598184+0000","last_became_peered":"2023-10-06T04:53:49.598184+0000","last_unstale":"2023-10-06T04:53:49.598387+0000","last_undegraded":"2023-10-06T04:53:49.598387+0000","last_fullsized":"2023-10-06T04:53:49.598387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.590325+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608228+0000","last_change":"2023-10-06T04:53:49.608228+0000","last_active":"2023-10-06T04:53:49.608228+0000","last_peered":"2023-10-06T04:53:49.608228+0000","last_clean":"2023-10-06T04:53:49.608228+0000","last_became_active":"2023-10-06T04:53:49.607857+0000","last_became_peered":"2023-10-06T04:53:49.607857+0000","last_unstale":"2023-10-06T04:53:49.608228+0000","last_undegraded":"2023-10-06T04:53:49.608228+0000","last_fullsized":"2023-10-06T04:53:49.608228+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:22.996368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591635+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:16.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.600834+0000","last_change":"2023-10-06T04:53:49.600834+0000","last_active":"2023-10-06T04:53:49.600834+0000","last_peered":"2023-10-06T04:53:49.600834+0000","last_clean":"2023-10-06T04:53:49.600834+0000","last_became_active":"2023-10-06T04:53:49.600540+0000","last_became_peered":"2023-10-06T04:53:49.600540+0000","last_unstale":"2023-10-06T04:53:49.600834+0000","last_undegraded":"2023-10-06T04:53:49.600834+0000","last_fullsized":"2023-10-06T04:53:49.600834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:06.484647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593411+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613344+0000","last_change":"2023-10-06T04:53:49.613344+0000","last_active":"2023-10-06T04:53:49.613344+0000","last_peered":"2023-10-06T04:53:49.613344+0000","last_clean":"2023-10-06T04:53:49.613344+0000","last_became_active":"2023-10-06T04:53:49.613045+0000","last_became_peered":"2023-10-06T04:53:49.613045+0000","last_unstale":"2023-10-06T04:53:49.613344+0000","last_undegraded":"2023-10-06T04:53:49.613344+0000","last_fullsized":"2023-10-06T04:53:49.613344+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:29.920075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.591953+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.047782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601346+0000","last_change":"2023-10-06T04:53:49.601346+0000","last_active":"2023-10-06T04:53:49.601346+0000","last_peered":"2023-10-06T04:53:49.601346+0000","last_clean":"2023-10-06T04:53:49.601346+0000","last_became_active":"2023-10-06T04:53:49.600993+0000","last_became_peered":"2023-10-06T04:53:49.600993+0000","last_unstale":"2023-10-06T04:53:49.601346+0000","last_undegraded":"2023-10-06T04:53:49.601346+0000","last_fullsized":"2023-10-06T04:53:49.601346+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:58.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592361+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:26.053487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.606428+0000","last_change":"2023-10-06T04:53:49.606428+0000","last_active":"2023-10-06T04:53:49.606428+0000","last_peered":"2023-10-06T04:53:49.606428+0000","last_clean":"2023-10-06T04:53:49.606428+0000","last_became_active":"2023-10-06T04:53:49.606178+0000","last_became_peered":"2023-10-06T04:53:49.606178+0000","last_unstale":"2023-10-06T04:53:49.606428+0000","last_undegraded":"2023-10-06T04:53:49.606428+0000","last_fullsized":"2023-10-06T04:53:49.606428+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.595074+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:40.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599873+0000","last_change":"2023-10-06T04:53:49.599873+0000","last_active":"2023-10-06T04:53:49.599873+0000","last_peered":"2023-10-06T04:53:49.599873+0000","last_clean":"2023-10-06T04:53:49.599873+0000","last_became_active":"2023-10-06T04:53:49.599582+0000","last_became_peered":"2023-10-06T04:53:49.599582+0000","last_unstale":"2023-10-06T04:53:49.599873+0000","last_undegraded":"2023-10-06T04:53:49.599873+0000","last_fullsized":"2023-10-06T04:53:49.599873+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598198+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.601241+0000","last_change":"2023-10-06T04:53:49.601241+0000","last_active":"2023-10-06T04:53:49.601241+0000","last_peered":"2023-10-06T04:53:49.601241+0000","last_clean":"2023-10-06T04:53:49.601241+0000","last_became_active":"2023-10-06T04:53:49.600982+0000","last_became_peered":"2023-10-06T04:53:49.600982+0000","last_unstale":"2023-10-06T04:53:49.601241+0000","last_undegraded":"2023-10-06T04:53:49.601241+0000","last_fullsized":"2023-10-06T04:53:49.601241+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:26.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592554+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:27.741668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.599636+0000","last_change":"2023-10-06T04:53:49.599636+0000","last_active":"2023-10-06T04:53:49.599636+0000","last_peered":"2023-10-06T04:53:49.599636+0000","last_clean":"2023-10-06T04:53:49.599636+0000","last_became_active":"2023-10-06T04:53:49.599356+0000","last_became_peered":"2023-10-06T04:53:49.599356+0000","last_unstale":"2023-10-06T04:53:49.599636+0000","last_undegraded":"2023-10-06T04:53:49.599636+0000","last_fullsized":"2023-10-06T04:53:49.599636+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:50.733425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.593469+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608771+0000","last_change":"2023-10-06T04:53:49.608771+0000","last_active":"2023-10-06T04:53:49.608771+0000","last_peered":"2023-10-06T04:53:49.608771+0000","last_clean":"2023-10-06T04:53:49.608771+0000","last_became_active":"2023-10-06T04:53:49.608472+0000","last_became_peered":"2023-10-06T04:53:49.608472+0000","last_unstale":"2023-10-06T04:53:49.608771+0000","last_undegraded":"2023-10-06T04:53:49.608771+0000","last_fullsized":"2023-10-06T04:53:49.608771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:41.942599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590905+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:53:48.590905+0000","last_peered":"2023-10-06T04:53:48.590905+0000","last_clean":"2023-10-06T04:53:48.590905+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:53:48.590905+0000","last_undegraded":"2023-10-06T04:53:48.590905+0000","last_fullsized":"2023-10-06T04:53:48.590905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594775+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.597803+0000","last_change":"2023-10-06T04:53:49.597803+0000","last_active":"2023-10-06T04:53:49.597803+0000","last_peered":"2023-10-06T04:53:49.597803+0000","last_clean":"2023-10-06T04:53:49.597803+0000","last_became_active":"2023-10-06T04:53:49.597467+0000","last_became_peered":"2023-10-06T04:53:49.597467+0000","last_unstale":"2023-10-06T04:53:49.597803+0000","last_undegraded":"2023-10-06T04:53:49.597803+0000","last_fullsized":"2023-10-06T04:53:49.597803+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:42.301398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.590629+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:53:48.590629+0000","last_peered":"2023-10-06T04:53:48.590629+0000","last_clean":"2023-10-06T04:53:48.590629+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:53:48.590629+0000","last_undegraded":"2023-10-06T04:53:48.590629+0000","last_fullsized":"2023-10-06T04:53:48.590629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592281+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:21.453482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608060+0000","last_change":"2023-10-06T04:53:49.608060+0000","last_active":"2023-10-06T04:53:49.608060+0000","last_peered":"2023-10-06T04:53:49.608060+0000","last_clean":"2023-10-06T04:53:49.608060+0000","last_became_active":"2023-10-06T04:53:49.607815+0000","last_became_peered":"2023-10-06T04:53:49.607815+0000","last_unstale":"2023-10-06T04:53:49.608060+0000","last_undegraded":"2023-10-06T04:53:49.608060+0000","last_fullsized":"2023-10-06T04:53:49.608060+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.584714+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:53:48.584714+0000","last_peered":"2023-10-06T04:53:48.584714+0000","last_clean":"2023-10-06T04:53:48.584714+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:53:48.584714+0000","last_undegraded":"2023-10-06T04:53:48.584714+0000","last_fullsized":"2023-10-06T04:53:48.584714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":179,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594422+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.611927+0000","last_change":"2023-10-06T04:53:49.611927+0000","last_active":"2023-10-06T04:53:49.611927+0000","last_peered":"2023-10-06T04:53:49.611927+0000","last_clean":"2023-10-06T04:53:49.611927+0000","last_became_active":"2023-10-06T04:53:49.611624+0000","last_became_peered":"2023-10-06T04:53:49.611624+0000","last_unstale":"2023-10-06T04:53:49.611927+0000","last_undegraded":"2023-10-06T04:53:49.611927+0000","last_fullsized":"2023-10-06T04:53:49.611927+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585519+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:53:48.585519+0000","last_peered":"2023-10-06T04:53:48.585519+0000","last_clean":"2023-10-06T04:53:48.585519+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:53:48.585519+0000","last_undegraded":"2023-10-06T04:53:48.585519+0000","last_fullsized":"2023-10-06T04:53:48.585519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592939+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:14.972969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.607924+0000","last_change":"2023-10-06T04:53:49.607924+0000","last_active":"2023-10-06T04:53:49.607924+0000","last_peered":"2023-10-06T04:53:49.607924+0000","last_clean":"2023-10-06T04:53:49.607924+0000","last_became_active":"2023-10-06T04:53:49.607341+0000","last_became_peered":"2023-10-06T04:53:49.607341+0000","last_unstale":"2023-10-06T04:53:49.607924+0000","last_undegraded":"2023-10-06T04:53:49.607924+0000","last_fullsized":"2023-10-06T04:53:49.607924+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.585502+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:53:48.585502+0000","last_peered":"2023-10-06T04:53:48.585502+0000","last_clean":"2023-10-06T04:53:48.585502+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:53:48.585502+0000","last_undegraded":"2023-10-06T04:53:48.585502+0000","last_fullsized":"2023-10-06T04:53:48.585502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.592777+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.608707+0000","last_change":"2023-10-06T04:53:49.608707+0000","last_active":"2023-10-06T04:53:49.608707+0000","last_peered":"2023-10-06T04:53:49.608707+0000","last_clean":"2023-10-06T04:53:49.608707+0000","last_became_active":"2023-10-06T04:53:49.608433+0000","last_became_peered":"2023-10-06T04:53:49.608433+0000","last_unstale":"2023-10-06T04:53:49.608707+0000","last_undegraded":"2023-10-06T04:53:49.608707+0000","last_fullsized":"2023-10-06T04:53:49.608707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.587417+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:53:48.587417+0000","last_peered":"2023-10-06T04:53:48.587417+0000","last_clean":"2023-10-06T04:53:48.587417+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:53:48.587417+0000","last_undegraded":"2023-10-06T04:53:48.587417+0000","last_fullsized":"2023-10-06T04:53:48.587417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.594963+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:49.091547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.613283+0000","last_change":"2023-10-06T04:53:49.613283+0000","last_active":"2023-10-06T04:53:49.613283+0000","last_peered":"2023-10-06T04:53:49.613283+0000","last_clean":"2023-10-06T04:53:49.613283+0000","last_became_active":"2023-10-06T04:53:49.612982+0000","last_became_peered":"2023-10-06T04:53:49.612982+0000","last_unstale":"2023-10-06T04:53:49.613283+0000","last_undegraded":"2023-10-06T04:53:49.613283+0000","last_fullsized":"2023-10-06T04:53:49.613283+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588728+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:53:48.588728+0000","last_peered":"2023-10-06T04:53:48.588728+0000","last_clean":"2023-10-06T04:53:48.588728+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:53:48.588728+0000","last_undegraded":"2023-10-06T04:53:48.588728+0000","last_fullsized":"2023-10-06T04:53:48.588728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:53:49.582035+0000","last_change":"2023-10-06T04:53:49.598678+0000","last_active":"2023-10-06T04:53:49.582035+0000","last_peered":"2023-10-06T04:53:49.582035+0000","last_clean":"2023-10-06T04:53:49.582035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:53:49.582035+0000","last_undegraded":"2023-10-06T04:53:49.582035+0000","last_fullsized":"2023-10-06T04:53:49.582035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T04:53:49.605704+0000","last_change":"2023-10-06T04:53:49.605704+0000","last_active":"2023-10-06T04:53:49.605704+0000","last_peered":"2023-10-06T04:53:49.605704+0000","last_clean":"2023-10-06T04:53:49.605704+0000","last_became_active":"2023-10-06T04:53:49.605449+0000","last_became_peered":"2023-10-06T04:53:49.605449+0000","last_unstale":"2023-10-06T04:53:49.605704+0000","last_undegraded":"2023-10-06T04:53:49.605704+0000","last_fullsized":"2023-10-06T04:53:49.605704+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:53:48.588915+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:53:48.588915+0000","last_peered":"2023-10-06T04:53:48.588915+0000","last_clean":"2023-10-06T04:53:48.588915+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:53:48.588915+0000","last_undegraded":"2023-10-06T04:53:48.588915+0000","last_fullsized":"2023-10-06T04:53:48.588915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":7.1779999999999999,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.504},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.375},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":1081344,"data_stored":794656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.376},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.495}]},{"osd":6,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.51800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.25600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.438}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.503}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.372}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.27300000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.069,"5min":5.069,"15min":5.069},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":6.4210000000000003,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4009999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.367,"5min":6.367,"15min":6.367},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":5,"5min":5,"15min":5},"last":0.51600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301864,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069976,"statfs":{"total":96636764160,"available":96327655424,"internally_reserved":0,"allocated":491520,"data_stored":205641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.501},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":5.4569999999999999,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":5.5510000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.415,"5min":5.415,"15min":5.415},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.202,"5min":4.202,"15min":4.202},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.628}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301860,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069980,"statfs":{"total":96636764160,"available":96327659520,"internally_reserved":0,"allocated":487424,"data_stored":204288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.7949999999999999,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.8170000000000002,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.7210000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:53:59.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:53:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.479+0000 7f15d62e1700 1 Processor -- start 2023-10-06T04:53:59.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.479+0000 7f15d62e1700 1 -- start start 2023-10-06T04:53:59.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.480+0000 7f15d62e1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f15d015c8f0 0x7f15d015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.480+0000 7f15d62e1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 0x7f15d00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.480+0000 7f15d62e1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d00cf6c0 con 0x7f15d015ccc0 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.480+0000 7f15d62e1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d00cfb50 con 0x7f15d015c8f0 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.480+0000 7f15d62e1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d015e050 con 0x7f15d00d0870 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.480+0000 7f15cf7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f15d015ccc0 0x7f15d00d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48996/0 (socket says 172.21.15.3:48996) 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.480+0000 7f15cf7fe700 1 -- 172.21.15.3:0/2052926091 learned_addr learned my addr 172.21.15.3:0/2052926091 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15cffff700 1 --2- 172.21.15.3:0/2052926091 >> v2:172.21.15.153:3300/0 conn(0x7f15d015c8f0 0x7f15d015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15d487e700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 0x7f15d00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232777367 0 0) 0x7f15d00cf6c0 con 0x7f15d015ccc0 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15bc003570 con 0x7f15d015ccc0 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590130692 0 0) 0x7f15bc003570 con 0x7f15d015ccc0 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 msgr2=0x7f15d00b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:59.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 0x7f15d00b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 >> v2:172.21.15.153:3300/0 conn(0x7f15d015c8f0 msgr2=0x7f15d015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 --2- 172.21.15.3:0/2052926091 >> v2:172.21.15.153:3300/0 conn(0x7f15d015c8f0 0x7f15d015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00ceab0 con 0x7f15d015ccc0 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f15b8001470 con 0x7f15d015ccc0 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 >> v1:172.21.15.3:6789/0 conn(0x7f15d015ccc0 legacy=0x7f15d00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc003a60 0x7f15bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 0x7f15bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15bc004d50 0x7f15bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00cfb50 con 0x7f15bc004d50 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d015e050 con 0x7f15bc003a60 2023-10-06T04:53:59.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00ba60 con 0x7f15d00d0870 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15d487e700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15bc004d50 0x7f15bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15cffff700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc003a60 0x7f15bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15cf7fe700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 0x7f15bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15d487e700 1 -- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 msgr2=0x7f15bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15d487e700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 0x7f15bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15d487e700 1 -- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc003a60 msgr2=0x7f15bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15d487e700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc003a60 0x7f15bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15d487e700 1 -- 172.21.15.3:0/2052926091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc003570 con 0x7f15bc004d50 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.481+0000 7f15cf7fe700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d00d0870 0x7f15bc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d487e700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15bc004d50 0x7f15bc0051c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f15d00d0cd0 tx=0x7f15c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ff29099bc364fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800ea30 con 0x7f15bc004d50 2023-10-06T04:53:59.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c800f040 con 0x7f15bc004d50 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15ceffd700 1 -- 172.21.15.3:0/2052926091 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800ede0 con 0x7f15bc004d50 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d62e1700 1 -- 172.21.15.3:0/2052926091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15bc004d50 msgr2=0x7f15bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d62e1700 1 --2- 172.21.15.3:0/2052926091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15bc004d50 0x7f15bc0051c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f15d00d0cd0 tx=0x7f15c800b040 comp rx=0 tx=0).stop 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d487e700 1 -- 172.21.15.3:0/2052926091 reap_dead start 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d62e1700 1 -- 172.21.15.3:0/2052926091 shutdown_connections 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d62e1700 1 -- 172.21.15.3:0/2052926091 >> 172.21.15.3:0/2052926091 conn(0x7f15d0070310 msgr2=0x7f15d0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d62e1700 1 -- 172.21.15.3:0/2052926091 shutdown_connections 2023-10-06T04:53:59.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.482+0000 7f15d62e1700 1 -- 172.21.15.3:0/2052926091 wait complete. 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.484+0000 7f15d62e1700 1 Processor -- start 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.484+0000 7f15d62e1700 1 -- start start 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.484+0000 7f15d62e1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15d014c070 0x7f15d0150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d62e1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d014c440 0x7f15d01550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d62e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d0151290 0x7f15d01555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d62e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d016aa70 con 0x7f15d0151290 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d62e1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0153b90 con 0x7f15d014c070 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d62e1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0169e60 con 0x7f15d014c440 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d0151290 0x7f15d01555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d0151290 0x7f15d01555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40598/0 (socket says 172.21.15.3:40598) 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 -- 172.21.15.3:0/356115363 learned_addr learned my addr 172.21.15.3:0/356115363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15cffff700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15d014c070 0x7f15d0150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15cf7fe700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d014c440 0x7f15d01550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 -- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d014c440 msgr2=0x7f15d01550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d014c440 0x7f15d01550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 -- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15d014c070 msgr2=0x7f15d0150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15d014c070 0x7f15d0150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 -- 172.21.15.3:0/356115363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d016a140 con 0x7f15d0151290 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d487e700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d0151290 0x7f15d01555e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f15c800afd0 tx=0x7f15c801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15c7fff700 1 -- 172.21.15.3:0/356115363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8014d50 con 0x7f15d0151290 2023-10-06T04:53:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15c7fff700 1 -- 172.21.15.3:0/356115363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8022070 con 0x7f15d0151290 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15c7fff700 1 -- 172.21.15.3:0/356115363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c80296d0 con 0x7f15d0151290 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.485+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d016a420 con 0x7f15d0151290 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.486+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d0169b50 con 0x7f15d0151290 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.487+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d0071680 con 0x7f15d0151290 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.487+0000 7f15c7fff700 1 -- 172.21.15.3:0/356115363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15c8003040 con 0x7f15d0151290 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.487+0000 7f15c7fff700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f15b0066f30 0x7f15b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.487+0000 7f15c7fff700 1 -- 172.21.15.3:0/356115363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f15c8024070 con 0x7f15d0151290 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.487+0000 7f15cffff700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f15b0066f30 0x7f15b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.488+0000 7f15cffff700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f15b0066f30 0x7f15b00693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15d00cc920 tx=0x7f15c00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:59.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.492+0000 7f15c7fff700 1 -- 172.21.15.3:0/356115363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15c8053510 con 0x7f15d0151290 2023-10-06T04:53:59.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.682+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15d00b23d0 con 0x7f15b0066f30 2023-10-06T04:53:59.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.691+0000 7f15c7fff700 1 -- 172.21.15.3:0/356115363 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311136 (secure 0 0 0) 0x7f15d00b23d0 con 0x7f15b0066f30 2023-10-06T04:53:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f15b0066f30 msgr2=0x7f15b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f15b0066f30 0x7f15b00693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15d00cc920 tx=0x7f15c00091a0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d0151290 msgr2=0x7f15d01555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d0151290 0x7f15d01555e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f15c800afd0 tx=0x7f15c801a000 comp rx=0 tx=0).stop 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 shutdown_connections 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f15b0066f30 0x7f15b00693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15d014c070 0x7f15d0150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d0151290 0x7f15d01555e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 --2- 172.21.15.3:0/356115363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15d014c440 0x7f15d01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.694+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 >> 172.21.15.3:0/356115363 conn(0x7f15d0070310 msgr2=0x7f15d0156dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:59.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.695+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 shutdown_connections 2023-10-06T04:53:59.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.695+0000 7f15d62e1700 1 -- 172.21.15.3:0/356115363 wait complete. 2023-10-06T04:53:59.701 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:53:59.709 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-10-06T04:53:59.447039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2416316,"kb_used_data":5244,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752558404,"statfs":{"total":773094113280,"available":770619805696,"internally_reserved":0,"allocated":5369856,"data_stored":2936473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.858485"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618041+0000","last_change":"2023-10-06T04:53:50.610894+0000","last_active":"2023-10-06T04:53:50.618041+0000","last_peered":"2023-10-06T04:53:50.618041+0000","last_clean":"2023-10-06T04:53:50.618041+0000","last_became_active":"2023-10-06T04:53:50.610627+0000","last_became_peered":"2023-10-06T04:53:50.610627+0000","last_unstale":"2023-10-06T04:53:50.618041+0000","last_undegraded":"2023-10-06T04:53:50.618041+0000","last_fullsized":"2023-10-06T04:53:50.618041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.306540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.609067+0000","last_change":"2023-10-06T04:53:50.604867+0000","last_active":"2023-10-06T04:53:50.609067+0000","last_peered":"2023-10-06T04:53:50.609067+0000","last_clean":"2023-10-06T04:53:50.609067+0000","last_became_active":"2023-10-06T04:53:50.604589+0000","last_became_peered":"2023-10-06T04:53:50.604589+0000","last_unstale":"2023-10-06T04:53:50.609067+0000","last_undegraded":"2023-10-06T04:53:50.609067+0000","last_fullsized":"2023-10-06T04:53:50.609067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.612655+0000","last_change":"2023-10-06T04:53:50.609024+0000","last_active":"2023-10-06T04:53:50.612655+0000","last_peered":"2023-10-06T04:53:50.612655+0000","last_clean":"2023-10-06T04:53:50.612655+0000","last_became_active":"2023-10-06T04:53:50.608757+0000","last_became_peered":"2023-10-06T04:53:50.608757+0000","last_unstale":"2023-10-06T04:53:50.612655+0000","last_undegraded":"2023-10-06T04:53:50.612655+0000","last_fullsized":"2023-10-06T04:53:50.612655+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:02.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.608476+0000","last_change":"2023-10-06T04:53:50.602420+0000","last_active":"2023-10-06T04:53:50.608476+0000","last_peered":"2023-10-06T04:53:50.608476+0000","last_clean":"2023-10-06T04:53:50.608476+0000","last_became_active":"2023-10-06T04:53:50.602151+0000","last_became_peered":"2023-10-06T04:53:50.602151+0000","last_unstale":"2023-10-06T04:53:50.608476+0000","last_undegraded":"2023-10-06T04:53:50.608476+0000","last_fullsized":"2023-10-06T04:53:50.608476+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615432+0000","last_change":"2023-10-06T04:53:50.608745+0000","last_active":"2023-10-06T04:53:50.615432+0000","last_peered":"2023-10-06T04:53:50.615432+0000","last_clean":"2023-10-06T04:53:50.615432+0000","last_became_active":"2023-10-06T04:53:50.608513+0000","last_became_peered":"2023-10-06T04:53:50.608513+0000","last_unstale":"2023-10-06T04:53:50.615432+0000","last_undegraded":"2023-10-06T04:53:50.615432+0000","last_fullsized":"2023-10-06T04:53:50.615432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618214+0000","last_change":"2023-10-06T04:53:50.611249+0000","last_active":"2023-10-06T04:53:50.618214+0000","last_peered":"2023-10-06T04:53:50.618214+0000","last_clean":"2023-10-06T04:53:50.618214+0000","last_became_active":"2023-10-06T04:53:50.608843+0000","last_became_peered":"2023-10-06T04:53:50.608843+0000","last_unstale":"2023-10-06T04:53:50.618214+0000","last_undegraded":"2023-10-06T04:53:50.618214+0000","last_fullsized":"2023-10-06T04:53:50.618214+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.606455+0000","last_change":"2023-10-06T04:53:50.600577+0000","last_active":"2023-10-06T04:53:50.606455+0000","last_peered":"2023-10-06T04:53:50.606455+0000","last_clean":"2023-10-06T04:53:50.606455+0000","last_became_active":"2023-10-06T04:53:50.600293+0000","last_became_peered":"2023-10-06T04:53:50.600293+0000","last_unstale":"2023-10-06T04:53:50.606455+0000","last_undegraded":"2023-10-06T04:53:50.606455+0000","last_fullsized":"2023-10-06T04:53:50.606455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:16.106274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618697+0000","last_change":"2023-10-06T04:53:50.613561+0000","last_active":"2023-10-06T04:53:50.618697+0000","last_peered":"2023-10-06T04:53:50.618697+0000","last_clean":"2023-10-06T04:53:50.618697+0000","last_became_active":"2023-10-06T04:53:50.612476+0000","last_became_peered":"2023-10-06T04:53:50.612476+0000","last_unstale":"2023-10-06T04:53:50.618697+0000","last_undegraded":"2023-10-06T04:53:50.618697+0000","last_fullsized":"2023-10-06T04:53:50.618697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:51.236747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618325+0000","last_change":"2023-10-06T04:53:50.611221+0000","last_active":"2023-10-06T04:53:50.618325+0000","last_peered":"2023-10-06T04:53:50.618325+0000","last_clean":"2023-10-06T04:53:50.618325+0000","last_became_active":"2023-10-06T04:53:50.610593+0000","last_became_peered":"2023-10-06T04:53:50.610593+0000","last_unstale":"2023-10-06T04:53:50.618325+0000","last_undegraded":"2023-10-06T04:53:50.618325+0000","last_fullsized":"2023-10-06T04:53:50.618325+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:12.570445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.608928+0000","last_change":"2023-10-06T04:53:50.601434+0000","last_active":"2023-10-06T04:53:50.608928+0000","last_peered":"2023-10-06T04:53:50.608928+0000","last_clean":"2023-10-06T04:53:50.608928+0000","last_became_active":"2023-10-06T04:53:50.601140+0000","last_became_peered":"2023-10-06T04:53:50.601140+0000","last_unstale":"2023-10-06T04:53:50.608928+0000","last_undegraded":"2023-10-06T04:53:50.608928+0000","last_fullsized":"2023-10-06T04:53:50.608928+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:26.053487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.612341+0000","last_change":"2023-10-06T04:53:50.609728+0000","last_active":"2023-10-06T04:53:50.612341+0000","last_peered":"2023-10-06T04:53:50.612341+0000","last_clean":"2023-10-06T04:53:50.612341+0000","last_became_active":"2023-10-06T04:53:50.609350+0000","last_became_peered":"2023-10-06T04:53:50.609350+0000","last_unstale":"2023-10-06T04:53:50.612341+0000","last_undegraded":"2023-10-06T04:53:50.612341+0000","last_fullsized":"2023-10-06T04:53:50.612341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:55.812395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610153+0000","last_change":"2023-10-06T04:53:50.603994+0000","last_active":"2023-10-06T04:53:50.610153+0000","last_peered":"2023-10-06T04:53:50.610153+0000","last_clean":"2023-10-06T04:53:50.610153+0000","last_became_active":"2023-10-06T04:53:50.603634+0000","last_became_peered":"2023-10-06T04:53:50.603634+0000","last_unstale":"2023-10-06T04:53:50.610153+0000","last_undegraded":"2023-10-06T04:53:50.610153+0000","last_fullsized":"2023-10-06T04:53:50.610153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:24.493969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610363+0000","last_change":"2023-10-06T04:53:50.604230+0000","last_active":"2023-10-06T04:53:50.610363+0000","last_peered":"2023-10-06T04:53:50.610363+0000","last_clean":"2023-10-06T04:53:50.610363+0000","last_became_active":"2023-10-06T04:53:50.603921+0000","last_became_peered":"2023-10-06T04:53:50.603921+0000","last_unstale":"2023-10-06T04:53:50.610363+0000","last_undegraded":"2023-10-06T04:53:50.610363+0000","last_fullsized":"2023-10-06T04:53:50.610363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:26.390192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613205+0000","last_change":"2023-10-06T04:53:50.609101+0000","last_active":"2023-10-06T04:53:50.613205+0000","last_peered":"2023-10-06T04:53:50.613205+0000","last_clean":"2023-10-06T04:53:50.613205+0000","last_became_active":"2023-10-06T04:53:50.608821+0000","last_became_peered":"2023-10-06T04:53:50.608821+0000","last_unstale":"2023-10-06T04:53:50.613205+0000","last_undegraded":"2023-10-06T04:53:50.613205+0000","last_fullsized":"2023-10-06T04:53:50.613205+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:38.723594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615145+0000","last_change":"2023-10-06T04:53:50.611909+0000","last_active":"2023-10-06T04:53:50.615145+0000","last_peered":"2023-10-06T04:53:50.615145+0000","last_clean":"2023-10-06T04:53:50.615145+0000","last_became_active":"2023-10-06T04:53:50.608952+0000","last_became_peered":"2023-10-06T04:53:50.608952+0000","last_unstale":"2023-10-06T04:53:50.615145+0000","last_undegraded":"2023-10-06T04:53:50.615145+0000","last_fullsized":"2023-10-06T04:53:50.615145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.609035+0000","last_change":"2023-10-06T04:53:50.606007+0000","last_active":"2023-10-06T04:53:50.609035+0000","last_peered":"2023-10-06T04:53:50.609035+0000","last_clean":"2023-10-06T04:53:50.609035+0000","last_became_active":"2023-10-06T04:53:50.605765+0000","last_became_peered":"2023-10-06T04:53:50.605765+0000","last_unstale":"2023-10-06T04:53:50.609035+0000","last_undegraded":"2023-10-06T04:53:50.609035+0000","last_fullsized":"2023-10-06T04:53:50.609035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:46.799897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.611039+0000","last_change":"2023-10-06T04:53:50.607382+0000","last_active":"2023-10-06T04:53:50.611039+0000","last_peered":"2023-10-06T04:53:50.611039+0000","last_clean":"2023-10-06T04:53:50.611039+0000","last_became_active":"2023-10-06T04:53:50.604809+0000","last_became_peered":"2023-10-06T04:53:50.604809+0000","last_unstale":"2023-10-06T04:53:50.611039+0000","last_undegraded":"2023-10-06T04:53:50.611039+0000","last_fullsized":"2023-10-06T04:53:50.611039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:48.544686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615578+0000","last_change":"2023-10-06T04:53:50.612938+0000","last_active":"2023-10-06T04:53:50.615578+0000","last_peered":"2023-10-06T04:53:50.615578+0000","last_clean":"2023-10-06T04:53:50.615578+0000","last_became_active":"2023-10-06T04:53:50.608899+0000","last_became_peered":"2023-10-06T04:53:50.608899+0000","last_unstale":"2023-10-06T04:53:50.615578+0000","last_undegraded":"2023-10-06T04:53:50.615578+0000","last_fullsized":"2023-10-06T04:53:50.615578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.607117+0000","last_change":"2023-10-06T04:53:50.603382+0000","last_active":"2023-10-06T04:53:50.607117+0000","last_peered":"2023-10-06T04:53:50.607117+0000","last_clean":"2023-10-06T04:53:50.607117+0000","last_became_active":"2023-10-06T04:53:50.603111+0000","last_became_peered":"2023-10-06T04:53:50.603111+0000","last_unstale":"2023-10-06T04:53:50.607117+0000","last_undegraded":"2023-10-06T04:53:50.607117+0000","last_fullsized":"2023-10-06T04:53:50.607117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:15.352583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.614169+0000","last_change":"2023-10-06T04:53:50.608763+0000","last_active":"2023-10-06T04:53:50.614169+0000","last_peered":"2023-10-06T04:53:50.614169+0000","last_clean":"2023-10-06T04:53:50.614169+0000","last_became_active":"2023-10-06T04:53:50.608505+0000","last_became_peered":"2023-10-06T04:53:50.608505+0000","last_unstale":"2023-10-06T04:53:50.614169+0000","last_undegraded":"2023-10-06T04:53:50.614169+0000","last_fullsized":"2023-10-06T04:53:50.614169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:24.001510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.614331+0000","last_change":"2023-10-06T04:53:50.610700+0000","last_active":"2023-10-06T04:53:50.614331+0000","last_peered":"2023-10-06T04:53:50.614331+0000","last_clean":"2023-10-06T04:53:50.614331+0000","last_became_active":"2023-10-06T04:53:50.609281+0000","last_became_peered":"2023-10-06T04:53:50.609281+0000","last_unstale":"2023-10-06T04:53:50.614331+0000","last_undegraded":"2023-10-06T04:53:50.614331+0000","last_fullsized":"2023-10-06T04:53:50.614331+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:27.253859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.616172+0000","last_change":"2023-10-06T04:53:50.606578+0000","last_active":"2023-10-06T04:53:50.616172+0000","last_peered":"2023-10-06T04:53:50.616172+0000","last_clean":"2023-10-06T04:53:50.616172+0000","last_became_active":"2023-10-06T04:53:50.606295+0000","last_became_peered":"2023-10-06T04:53:50.606295+0000","last_unstale":"2023-10-06T04:53:50.616172+0000","last_undegraded":"2023-10-06T04:53:50.616172+0000","last_fullsized":"2023-10-06T04:53:50.616172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:26.216510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.608255+0000","last_change":"2023-10-06T04:53:50.603772+0000","last_active":"2023-10-06T04:53:50.608255+0000","last_peered":"2023-10-06T04:53:50.608255+0000","last_clean":"2023-10-06T04:53:50.608255+0000","last_became_active":"2023-10-06T04:53:50.603493+0000","last_became_peered":"2023-10-06T04:53:50.603493+0000","last_unstale":"2023-10-06T04:53:50.608255+0000","last_undegraded":"2023-10-06T04:53:50.608255+0000","last_fullsized":"2023-10-06T04:53:50.608255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:59.245595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.607136+0000","last_change":"2023-10-06T04:53:50.602510+0000","last_active":"2023-10-06T04:53:50.607136+0000","last_peered":"2023-10-06T04:53:50.607136+0000","last_clean":"2023-10-06T04:53:50.607136+0000","last_became_active":"2023-10-06T04:53:50.602214+0000","last_became_peered":"2023-10-06T04:53:50.602214+0000","last_unstale":"2023-10-06T04:53:50.607136+0000","last_undegraded":"2023-10-06T04:53:50.607136+0000","last_fullsized":"2023-10-06T04:53:50.607136+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:10.975006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.609185+0000","last_change":"2023-10-06T04:53:50.602277+0000","last_active":"2023-10-06T04:53:50.609185+0000","last_peered":"2023-10-06T04:53:50.609185+0000","last_clean":"2023-10-06T04:53:50.609185+0000","last_became_active":"2023-10-06T04:53:50.601995+0000","last_became_peered":"2023-10-06T04:53:50.601995+0000","last_unstale":"2023-10-06T04:53:50.609185+0000","last_undegraded":"2023-10-06T04:53:50.609185+0000","last_fullsized":"2023-10-06T04:53:50.609185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:24.493969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618094+0000","last_change":"2023-10-06T04:53:50.609478+0000","last_active":"2023-10-06T04:53:50.618094+0000","last_peered":"2023-10-06T04:53:50.618094+0000","last_clean":"2023-10-06T04:53:50.618094+0000","last_became_active":"2023-10-06T04:53:50.607479+0000","last_became_peered":"2023-10-06T04:53:50.607479+0000","last_unstale":"2023-10-06T04:53:50.618094+0000","last_undegraded":"2023-10-06T04:53:50.618094+0000","last_fullsized":"2023-10-06T04:53:50.618094+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:16.106274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.607645+0000","last_change":"2023-10-06T04:53:50.600905+0000","last_active":"2023-10-06T04:53:50.607645+0000","last_peered":"2023-10-06T04:53:50.607645+0000","last_clean":"2023-10-06T04:53:50.607645+0000","last_became_active":"2023-10-06T04:53:50.600655+0000","last_became_peered":"2023-10-06T04:53:50.600655+0000","last_unstale":"2023-10-06T04:53:50.607645+0000","last_undegraded":"2023-10-06T04:53:50.607645+0000","last_fullsized":"2023-10-06T04:53:50.607645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.614093+0000","last_change":"2023-10-06T04:53:50.606478+0000","last_active":"2023-10-06T04:53:50.614093+0000","last_peered":"2023-10-06T04:53:50.614093+0000","last_clean":"2023-10-06T04:53:50.614093+0000","last_became_active":"2023-10-06T04:53:50.606170+0000","last_became_peered":"2023-10-06T04:53:50.606170+0000","last_unstale":"2023-10-06T04:53:50.614093+0000","last_undegraded":"2023-10-06T04:53:50.614093+0000","last_fullsized":"2023-10-06T04:53:50.614093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:27.741668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.607602+0000","last_change":"2023-10-06T04:53:50.602843+0000","last_active":"2023-10-06T04:53:50.607602+0000","last_peered":"2023-10-06T04:53:50.607602+0000","last_clean":"2023-10-06T04:53:50.607602+0000","last_became_active":"2023-10-06T04:53:50.602567+0000","last_became_peered":"2023-10-06T04:53:50.602567+0000","last_unstale":"2023-10-06T04:53:50.607602+0000","last_undegraded":"2023-10-06T04:53:50.607602+0000","last_fullsized":"2023-10-06T04:53:50.607602+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:12.570445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.612330+0000","last_change":"2023-10-06T04:53:50.608090+0000","last_active":"2023-10-06T04:53:50.612330+0000","last_peered":"2023-10-06T04:53:50.612330+0000","last_clean":"2023-10-06T04:53:50.612330+0000","last_became_active":"2023-10-06T04:53:50.607840+0000","last_became_peered":"2023-10-06T04:53:50.607840+0000","last_unstale":"2023-10-06T04:53:50.612330+0000","last_undegraded":"2023-10-06T04:53:50.612330+0000","last_fullsized":"2023-10-06T04:53:50.612330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610763+0000","last_change":"2023-10-06T04:53:50.606889+0000","last_active":"2023-10-06T04:53:50.610763+0000","last_peered":"2023-10-06T04:53:50.610763+0000","last_clean":"2023-10-06T04:53:50.610763+0000","last_became_active":"2023-10-06T04:53:50.606253+0000","last_became_peered":"2023-10-06T04:53:50.606253+0000","last_unstale":"2023-10-06T04:53:50.610763+0000","last_undegraded":"2023-10-06T04:53:50.610763+0000","last_fullsized":"2023-10-06T04:53:50.610763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:41.257847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610397+0000","last_change":"2023-10-06T04:53:50.608936+0000","last_active":"2023-10-06T04:53:50.610397+0000","last_peered":"2023-10-06T04:53:50.610397+0000","last_clean":"2023-10-06T04:53:50.610397+0000","last_became_active":"2023-10-06T04:53:50.608684+0000","last_became_peered":"2023-10-06T04:53:50.608684+0000","last_unstale":"2023-10-06T04:53:50.610397+0000","last_undegraded":"2023-10-06T04:53:50.610397+0000","last_fullsized":"2023-10-06T04:53:50.610397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615707+0000","last_change":"2023-10-06T04:53:50.613185+0000","last_active":"2023-10-06T04:53:50.615707+0000","last_peered":"2023-10-06T04:53:50.615707+0000","last_clean":"2023-10-06T04:53:50.615707+0000","last_became_active":"2023-10-06T04:53:50.612909+0000","last_became_peered":"2023-10-06T04:53:50.612909+0000","last_unstale":"2023-10-06T04:53:50.615707+0000","last_undegraded":"2023-10-06T04:53:50.615707+0000","last_fullsized":"2023-10-06T04:53:50.615707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"43'4","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613629+0000","last_change":"2023-10-06T04:53:49.601799+0000","last_active":"2023-10-06T04:53:50.613629+0000","last_peered":"2023-10-06T04:53:50.613629+0000","last_clean":"2023-10-06T04:53:50.613629+0000","last_became_active":"2023-10-06T04:53:49.601428+0000","last_became_peered":"2023-10-06T04:53:49.601428+0000","last_unstale":"2023-10-06T04:53:50.613629+0000","last_undegraded":"2023-10-06T04:53:50.613629+0000","last_fullsized":"2023-10-06T04:53:50.613629+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:55.184468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.612532+0000","last_change":"2023-10-06T04:53:50.609889+0000","last_active":"2023-10-06T04:53:50.612532+0000","last_peered":"2023-10-06T04:53:50.612532+0000","last_clean":"2023-10-06T04:53:50.612532+0000","last_became_active":"2023-10-06T04:53:50.609614+0000","last_became_peered":"2023-10-06T04:53:50.609614+0000","last_unstale":"2023-10-06T04:53:50.612532+0000","last_undegraded":"2023-10-06T04:53:50.612532+0000","last_fullsized":"2023-10-06T04:53:50.612532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:43.322412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618290+0000","last_change":"2023-10-06T04:53:49.607471+0000","last_active":"2023-10-06T04:53:50.618290+0000","last_peered":"2023-10-06T04:53:50.618290+0000","last_clean":"2023-10-06T04:53:50.618290+0000","last_became_active":"2023-10-06T04:53:49.607158+0000","last_became_peered":"2023-10-06T04:53:49.607158+0000","last_unstale":"2023-10-06T04:53:50.618290+0000","last_undegraded":"2023-10-06T04:53:50.618290+0000","last_fullsized":"2023-10-06T04:53:50.618290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:41.756942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613337+0000","last_change":"2023-10-06T04:53:50.604271+0000","last_active":"2023-10-06T04:53:50.613337+0000","last_peered":"2023-10-06T04:53:50.613337+0000","last_clean":"2023-10-06T04:53:50.613337+0000","last_became_active":"2023-10-06T04:53:50.603986+0000","last_became_peered":"2023-10-06T04:53:50.603986+0000","last_unstale":"2023-10-06T04:53:50.613337+0000","last_undegraded":"2023-10-06T04:53:50.613337+0000","last_fullsized":"2023-10-06T04:53:50.613337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:38.165159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618031+0000","last_change":"2023-10-06T04:53:49.605024+0000","last_active":"2023-10-06T04:53:50.618031+0000","last_peered":"2023-10-06T04:53:50.618031+0000","last_clean":"2023-10-06T04:53:50.618031+0000","last_became_active":"2023-10-06T04:53:49.604790+0000","last_became_peered":"2023-10-06T04:53:49.604790+0000","last_unstale":"2023-10-06T04:53:50.618031+0000","last_undegraded":"2023-10-06T04:53:50.618031+0000","last_fullsized":"2023-10-06T04:53:50.618031+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:44.358080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.601068+0000","last_change":"2023-10-06T04:53:50.599603+0000","last_active":"2023-10-06T04:53:50.601068+0000","last_peered":"2023-10-06T04:53:50.601068+0000","last_clean":"2023-10-06T04:53:50.601068+0000","last_became_active":"2023-10-06T04:53:50.599311+0000","last_became_peered":"2023-10-06T04:53:50.599311+0000","last_unstale":"2023-10-06T04:53:50.601068+0000","last_undegraded":"2023-10-06T04:53:50.601068+0000","last_fullsized":"2023-10-06T04:53:50.601068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:48.544686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.616826+0000","last_change":"2023-10-06T04:53:49.607504+0000","last_active":"2023-10-06T04:53:50.616826+0000","last_peered":"2023-10-06T04:53:50.616826+0000","last_clean":"2023-10-06T04:53:50.616826+0000","last_became_active":"2023-10-06T04:53:49.607263+0000","last_became_peered":"2023-10-06T04:53:49.607263+0000","last_unstale":"2023-10-06T04:53:50.616826+0000","last_undegraded":"2023-10-06T04:53:50.616826+0000","last_fullsized":"2023-10-06T04:53:50.616826+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615790+0000","last_change":"2023-10-06T04:53:50.611475+0000","last_active":"2023-10-06T04:53:50.615790+0000","last_peered":"2023-10-06T04:53:50.615790+0000","last_clean":"2023-10-06T04:53:50.615790+0000","last_became_active":"2023-10-06T04:53:50.611202+0000","last_became_peered":"2023-10-06T04:53:50.611202+0000","last_unstale":"2023-10-06T04:53:50.615790+0000","last_undegraded":"2023-10-06T04:53:50.615790+0000","last_fullsized":"2023-10-06T04:53:50.615790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:25.000940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.612765+0000","last_change":"2023-10-06T04:53:49.606739+0000","last_active":"2023-10-06T04:53:50.612765+0000","last_peered":"2023-10-06T04:53:50.612765+0000","last_clean":"2023-10-06T04:53:50.612765+0000","last_became_active":"2023-10-06T04:53:49.606408+0000","last_became_peered":"2023-10-06T04:53:49.606408+0000","last_unstale":"2023-10-06T04:53:50.612765+0000","last_undegraded":"2023-10-06T04:53:50.612765+0000","last_fullsized":"2023-10-06T04:53:50.612765+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615238+0000","last_change":"2023-10-06T04:53:50.612488+0000","last_active":"2023-10-06T04:53:50.615238+0000","last_peered":"2023-10-06T04:53:50.615238+0000","last_clean":"2023-10-06T04:53:50.615238+0000","last_became_active":"2023-10-06T04:53:50.612168+0000","last_became_peered":"2023-10-06T04:53:50.612168+0000","last_unstale":"2023-10-06T04:53:50.615238+0000","last_undegraded":"2023-10-06T04:53:50.615238+0000","last_fullsized":"2023-10-06T04:53:50.615238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:36.714837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.614722+0000","last_change":"2023-10-06T04:53:49.602410+0000","last_active":"2023-10-06T04:53:50.614722+0000","last_peered":"2023-10-06T04:53:50.614722+0000","last_clean":"2023-10-06T04:53:50.614722+0000","last_became_active":"2023-10-06T04:53:49.602130+0000","last_became_peered":"2023-10-06T04:53:49.602130+0000","last_unstale":"2023-10-06T04:53:50.614722+0000","last_undegraded":"2023-10-06T04:53:50.614722+0000","last_fullsized":"2023-10-06T04:53:50.614722+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618142+0000","last_change":"2023-10-06T04:53:50.610004+0000","last_active":"2023-10-06T04:53:50.618142+0000","last_peered":"2023-10-06T04:53:50.618142+0000","last_clean":"2023-10-06T04:53:50.618142+0000","last_became_active":"2023-10-06T04:53:50.609450+0000","last_became_peered":"2023-10-06T04:53:50.609450+0000","last_unstale":"2023-10-06T04:53:50.618142+0000","last_undegraded":"2023-10-06T04:53:50.618142+0000","last_fullsized":"2023-10-06T04:53:50.618142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:02.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.597850+0000","last_change":"2023-10-06T04:53:49.602126+0000","last_active":"2023-10-06T04:53:50.597850+0000","last_peered":"2023-10-06T04:53:50.597850+0000","last_clean":"2023-10-06T04:53:50.597850+0000","last_became_active":"2023-10-06T04:53:49.601062+0000","last_became_peered":"2023-10-06T04:53:49.601062+0000","last_unstale":"2023-10-06T04:53:50.597850+0000","last_undegraded":"2023-10-06T04:53:50.597850+0000","last_fullsized":"2023-10-06T04:53:50.597850+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:54.807253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615217+0000","last_change":"2023-10-06T04:53:50.612438+0000","last_active":"2023-10-06T04:53:50.615217+0000","last_peered":"2023-10-06T04:53:50.615217+0000","last_clean":"2023-10-06T04:53:50.615217+0000","last_became_active":"2023-10-06T04:53:50.612156+0000","last_became_peered":"2023-10-06T04:53:50.612156+0000","last_unstale":"2023-10-06T04:53:50.615217+0000","last_undegraded":"2023-10-06T04:53:50.615217+0000","last_fullsized":"2023-10-06T04:53:50.615217+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.589251+0000","last_change":"2023-10-06T04:53:49.610421+0000","last_active":"2023-10-06T04:53:50.589251+0000","last_peered":"2023-10-06T04:53:50.589251+0000","last_clean":"2023-10-06T04:53:50.589251+0000","last_became_active":"2023-10-06T04:53:49.610133+0000","last_became_peered":"2023-10-06T04:53:49.610133+0000","last_unstale":"2023-10-06T04:53:50.589251+0000","last_undegraded":"2023-10-06T04:53:50.589251+0000","last_fullsized":"2023-10-06T04:53:50.589251+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:25.048345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.606583+0000","last_change":"2023-10-06T04:53:50.600771+0000","last_active":"2023-10-06T04:53:50.606583+0000","last_peered":"2023-10-06T04:53:50.606583+0000","last_clean":"2023-10-06T04:53:50.606583+0000","last_became_active":"2023-10-06T04:53:50.600504+0000","last_became_peered":"2023-10-06T04:53:50.600504+0000","last_unstale":"2023-10-06T04:53:50.606583+0000","last_undegraded":"2023-10-06T04:53:50.606583+0000","last_fullsized":"2023-10-06T04:53:50.606583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.593910+0000","last_change":"2023-10-06T04:53:49.602037+0000","last_active":"2023-10-06T04:53:50.593910+0000","last_peered":"2023-10-06T04:53:50.593910+0000","last_clean":"2023-10-06T04:53:50.593910+0000","last_became_active":"2023-10-06T04:53:49.601746+0000","last_became_peered":"2023-10-06T04:53:49.601746+0000","last_unstale":"2023-10-06T04:53:50.593910+0000","last_undegraded":"2023-10-06T04:53:50.593910+0000","last_fullsized":"2023-10-06T04:53:50.593910+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610800+0000","last_change":"2023-10-06T04:53:50.605433+0000","last_active":"2023-10-06T04:53:50.610800+0000","last_peered":"2023-10-06T04:53:50.610800+0000","last_clean":"2023-10-06T04:53:50.610800+0000","last_became_active":"2023-10-06T04:53:50.603833+0000","last_became_peered":"2023-10-06T04:53:50.603833+0000","last_unstale":"2023-10-06T04:53:50.610800+0000","last_undegraded":"2023-10-06T04:53:50.610800+0000","last_fullsized":"2023-10-06T04:53:50.610800+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:38.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.595518+0000","last_change":"2023-10-06T04:53:49.608823+0000","last_active":"2023-10-06T04:53:50.595518+0000","last_peered":"2023-10-06T04:53:50.595518+0000","last_clean":"2023-10-06T04:53:50.595518+0000","last_became_active":"2023-10-06T04:53:49.608542+0000","last_became_peered":"2023-10-06T04:53:49.608542+0000","last_unstale":"2023-10-06T04:53:50.595518+0000","last_undegraded":"2023-10-06T04:53:50.595518+0000","last_fullsized":"2023-10-06T04:53:50.595518+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.607631+0000","last_change":"2023-10-06T04:53:50.602506+0000","last_active":"2023-10-06T04:53:50.607631+0000","last_peered":"2023-10-06T04:53:50.607631+0000","last_clean":"2023-10-06T04:53:50.607631+0000","last_became_active":"2023-10-06T04:53:50.602214+0000","last_became_peered":"2023-10-06T04:53:50.602214+0000","last_unstale":"2023-10-06T04:53:50.607631+0000","last_undegraded":"2023-10-06T04:53:50.607631+0000","last_fullsized":"2023-10-06T04:53:50.607631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:26.216510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"43'3","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.616137+0000","last_change":"2023-10-06T04:53:49.610243+0000","last_active":"2023-10-06T04:53:50.616137+0000","last_peered":"2023-10-06T04:53:50.616137+0000","last_clean":"2023-10-06T04:53:50.616137+0000","last_became_active":"2023-10-06T04:53:49.609974+0000","last_became_peered":"2023-10-06T04:53:49.609974+0000","last_unstale":"2023-10-06T04:53:50.616137+0000","last_undegraded":"2023-10-06T04:53:50.616137+0000","last_fullsized":"2023-10-06T04:53:50.616137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.205881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.611627+0000","last_change":"2023-10-06T04:53:50.607918+0000","last_active":"2023-10-06T04:53:50.611627+0000","last_peered":"2023-10-06T04:53:50.611627+0000","last_clean":"2023-10-06T04:53:50.611627+0000","last_became_active":"2023-10-06T04:53:50.607657+0000","last_became_peered":"2023-10-06T04:53:50.607657+0000","last_unstale":"2023-10-06T04:53:50.611627+0000","last_undegraded":"2023-10-06T04:53:50.611627+0000","last_fullsized":"2023-10-06T04:53:50.611627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:55.911519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.626032+0000","last_change":"2023-10-06T04:53:49.599507+0000","last_active":"2023-10-06T04:53:50.626032+0000","last_peered":"2023-10-06T04:53:50.626032+0000","last_clean":"2023-10-06T04:53:50.626032+0000","last_became_active":"2023-10-06T04:53:49.599244+0000","last_became_peered":"2023-10-06T04:53:49.599244+0000","last_unstale":"2023-10-06T04:53:50.626032+0000","last_undegraded":"2023-10-06T04:53:50.626032+0000","last_fullsized":"2023-10-06T04:53:50.626032+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613030+0000","last_change":"2023-10-06T04:53:50.610204+0000","last_active":"2023-10-06T04:53:50.613030+0000","last_peered":"2023-10-06T04:53:50.613030+0000","last_clean":"2023-10-06T04:53:50.613030+0000","last_became_active":"2023-10-06T04:53:50.609968+0000","last_became_peered":"2023-10-06T04:53:50.609968+0000","last_unstale":"2023-10-06T04:53:50.613030+0000","last_undegraded":"2023-10-06T04:53:50.613030+0000","last_fullsized":"2023-10-06T04:53:50.613030+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:19.791226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613017+0000","last_change":"2023-10-06T04:53:49.602521+0000","last_active":"2023-10-06T04:53:50.613017+0000","last_peered":"2023-10-06T04:53:50.613017+0000","last_clean":"2023-10-06T04:53:50.613017+0000","last_became_active":"2023-10-06T04:53:49.602150+0000","last_became_peered":"2023-10-06T04:53:49.602150+0000","last_unstale":"2023-10-06T04:53:50.613017+0000","last_undegraded":"2023-10-06T04:53:50.613017+0000","last_fullsized":"2023-10-06T04:53:50.613017+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610789+0000","last_change":"2023-10-06T04:53:50.607523+0000","last_active":"2023-10-06T04:53:50.610789+0000","last_peered":"2023-10-06T04:53:50.610789+0000","last_clean":"2023-10-06T04:53:50.610789+0000","last_became_active":"2023-10-06T04:53:50.607232+0000","last_became_peered":"2023-10-06T04:53:50.607232+0000","last_unstale":"2023-10-06T04:53:50.610789+0000","last_undegraded":"2023-10-06T04:53:50.610789+0000","last_fullsized":"2023-10-06T04:53:50.610789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:17.094191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.598593+0000","last_change":"2023-10-06T04:53:49.608700+0000","last_active":"2023-10-06T04:53:50.598593+0000","last_peered":"2023-10-06T04:53:50.598593+0000","last_clean":"2023-10-06T04:53:50.598593+0000","last_became_active":"2023-10-06T04:53:49.608420+0000","last_became_peered":"2023-10-06T04:53:49.608420+0000","last_unstale":"2023-10-06T04:53:50.598593+0000","last_undegraded":"2023-10-06T04:53:50.598593+0000","last_fullsized":"2023-10-06T04:53:50.598593+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:24.697185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618162+0000","last_change":"2023-10-06T04:53:50.610799+0000","last_active":"2023-10-06T04:53:50.618162+0000","last_peered":"2023-10-06T04:53:50.618162+0000","last_clean":"2023-10-06T04:53:50.618162+0000","last_became_active":"2023-10-06T04:53:50.610532+0000","last_became_peered":"2023-10-06T04:53:50.610532+0000","last_unstale":"2023-10-06T04:53:50.618162+0000","last_undegraded":"2023-10-06T04:53:50.618162+0000","last_fullsized":"2023-10-06T04:53:50.618162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:38.723594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.595635+0000","last_change":"2023-10-06T04:53:49.606724+0000","last_active":"2023-10-06T04:53:50.595635+0000","last_peered":"2023-10-06T04:53:50.595635+0000","last_clean":"2023-10-06T04:53:50.595635+0000","last_became_active":"2023-10-06T04:53:49.606477+0000","last_became_peered":"2023-10-06T04:53:49.606477+0000","last_unstale":"2023-10-06T04:53:50.595635+0000","last_undegraded":"2023-10-06T04:53:50.595635+0000","last_fullsized":"2023-10-06T04:53:50.595635+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.609700+0000","last_change":"2023-10-06T04:53:50.605070+0000","last_active":"2023-10-06T04:53:50.609700+0000","last_peered":"2023-10-06T04:53:50.609700+0000","last_clean":"2023-10-06T04:53:50.609700+0000","last_became_active":"2023-10-06T04:53:50.604369+0000","last_became_peered":"2023-10-06T04:53:50.604369+0000","last_unstale":"2023-10-06T04:53:50.609700+0000","last_undegraded":"2023-10-06T04:53:50.609700+0000","last_fullsized":"2023-10-06T04:53:50.609700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:50.391853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"43'8","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.625693+0000","last_change":"2023-10-06T04:53:49.598387+0000","last_active":"2023-10-06T04:53:50.625693+0000","last_peered":"2023-10-06T04:53:50.625693+0000","last_clean":"2023-10-06T04:53:50.625693+0000","last_became_active":"2023-10-06T04:53:49.598184+0000","last_became_peered":"2023-10-06T04:53:49.598184+0000","last_unstale":"2023-10-06T04:53:50.625693+0000","last_undegraded":"2023-10-06T04:53:50.625693+0000","last_fullsized":"2023-10-06T04:53:50.625693+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.204877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.617651+0000","last_change":"2023-10-06T04:53:50.606849+0000","last_active":"2023-10-06T04:53:50.617651+0000","last_peered":"2023-10-06T04:53:50.617651+0000","last_clean":"2023-10-06T04:53:50.617651+0000","last_became_active":"2023-10-06T04:53:50.606568+0000","last_became_peered":"2023-10-06T04:53:50.606568+0000","last_unstale":"2023-10-06T04:53:50.617651+0000","last_undegraded":"2023-10-06T04:53:50.617651+0000","last_fullsized":"2023-10-06T04:53:50.617651+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:14.125211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618614+0000","last_change":"2023-10-06T04:53:49.608228+0000","last_active":"2023-10-06T04:53:50.618614+0000","last_peered":"2023-10-06T04:53:50.618614+0000","last_clean":"2023-10-06T04:53:50.618614+0000","last_became_active":"2023-10-06T04:53:49.607857+0000","last_became_peered":"2023-10-06T04:53:49.607857+0000","last_unstale":"2023-10-06T04:53:50.618614+0000","last_undegraded":"2023-10-06T04:53:50.618614+0000","last_fullsized":"2023-10-06T04:53:50.618614+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:22.996368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.607573+0000","last_change":"2023-10-06T04:53:50.600449+0000","last_active":"2023-10-06T04:53:50.607573+0000","last_peered":"2023-10-06T04:53:50.607573+0000","last_clean":"2023-10-06T04:53:50.607573+0000","last_became_active":"2023-10-06T04:53:50.600169+0000","last_became_peered":"2023-10-06T04:53:50.600169+0000","last_unstale":"2023-10-06T04:53:50.607573+0000","last_undegraded":"2023-10-06T04:53:50.607573+0000","last_fullsized":"2023-10-06T04:53:50.607573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:27.253859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.611780+0000","last_change":"2023-10-06T04:53:49.600834+0000","last_active":"2023-10-06T04:53:50.611780+0000","last_peered":"2023-10-06T04:53:50.611780+0000","last_clean":"2023-10-06T04:53:50.611780+0000","last_became_active":"2023-10-06T04:53:49.600540+0000","last_became_peered":"2023-10-06T04:53:49.600540+0000","last_unstale":"2023-10-06T04:53:50.611780+0000","last_undegraded":"2023-10-06T04:53:50.611780+0000","last_fullsized":"2023-10-06T04:53:50.611780+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:06.484647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.615508+0000","last_change":"2023-10-06T04:53:50.610939+0000","last_active":"2023-10-06T04:53:50.615508+0000","last_peered":"2023-10-06T04:53:50.615508+0000","last_clean":"2023-10-06T04:53:50.615508+0000","last_became_active":"2023-10-06T04:53:50.610685+0000","last_became_peered":"2023-10-06T04:53:50.610685+0000","last_unstale":"2023-10-06T04:53:50.615508+0000","last_undegraded":"2023-10-06T04:53:50.615508+0000","last_fullsized":"2023-10-06T04:53:50.615508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:39.103579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613417+0000","last_change":"2023-10-06T04:53:49.613344+0000","last_active":"2023-10-06T04:53:50.613417+0000","last_peered":"2023-10-06T04:53:50.613417+0000","last_clean":"2023-10-06T04:53:50.613417+0000","last_became_active":"2023-10-06T04:53:49.613045+0000","last_became_peered":"2023-10-06T04:53:49.613045+0000","last_unstale":"2023-10-06T04:53:50.613417+0000","last_undegraded":"2023-10-06T04:53:50.613417+0000","last_fullsized":"2023-10-06T04:53:50.613417+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:29.920075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610381+0000","last_change":"2023-10-06T04:53:50.606585+0000","last_active":"2023-10-06T04:53:50.610381+0000","last_peered":"2023-10-06T04:53:50.610381+0000","last_clean":"2023-10-06T04:53:50.610381+0000","last_became_active":"2023-10-06T04:53:50.606325+0000","last_became_peered":"2023-10-06T04:53:50.606325+0000","last_unstale":"2023-10-06T04:53:50.610381+0000","last_undegraded":"2023-10-06T04:53:50.610381+0000","last_fullsized":"2023-10-06T04:53:50.610381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:32.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.597700+0000","last_change":"2023-10-06T04:53:49.601346+0000","last_active":"2023-10-06T04:53:50.597700+0000","last_peered":"2023-10-06T04:53:50.597700+0000","last_clean":"2023-10-06T04:53:50.597700+0000","last_became_active":"2023-10-06T04:53:49.600993+0000","last_became_peered":"2023-10-06T04:53:49.600993+0000","last_unstale":"2023-10-06T04:53:50.597700+0000","last_undegraded":"2023-10-06T04:53:50.597700+0000","last_fullsized":"2023-10-06T04:53:50.597700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:58.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613074+0000","last_change":"2023-10-06T04:53:50.611198+0000","last_active":"2023-10-06T04:53:50.613074+0000","last_peered":"2023-10-06T04:53:50.613074+0000","last_clean":"2023-10-06T04:53:50.613074+0000","last_became_active":"2023-10-06T04:53:50.610808+0000","last_became_peered":"2023-10-06T04:53:50.610808+0000","last_unstale":"2023-10-06T04:53:50.613074+0000","last_undegraded":"2023-10-06T04:53:50.613074+0000","last_fullsized":"2023-10-06T04:53:50.613074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618027+0000","last_change":"2023-10-06T04:53:49.606428+0000","last_active":"2023-10-06T04:53:50.618027+0000","last_peered":"2023-10-06T04:53:50.618027+0000","last_clean":"2023-10-06T04:53:50.618027+0000","last_became_active":"2023-10-06T04:53:49.606178+0000","last_became_peered":"2023-10-06T04:53:49.606178+0000","last_unstale":"2023-10-06T04:53:50.618027+0000","last_undegraded":"2023-10-06T04:53:50.618027+0000","last_fullsized":"2023-10-06T04:53:50.618027+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.614128+0000","last_change":"2023-10-06T04:53:50.608150+0000","last_active":"2023-10-06T04:53:50.614128+0000","last_peered":"2023-10-06T04:53:50.614128+0000","last_clean":"2023-10-06T04:53:50.614128+0000","last_became_active":"2023-10-06T04:53:50.607900+0000","last_became_peered":"2023-10-06T04:53:50.607900+0000","last_unstale":"2023-10-06T04:53:50.614128+0000","last_undegraded":"2023-10-06T04:53:50.614128+0000","last_fullsized":"2023-10-06T04:53:50.614128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:26.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.591486+0000","last_change":"2023-10-06T04:53:49.599873+0000","last_active":"2023-10-06T04:53:50.591486+0000","last_peered":"2023-10-06T04:53:50.591486+0000","last_clean":"2023-10-06T04:53:50.591486+0000","last_became_active":"2023-10-06T04:53:49.599582+0000","last_became_peered":"2023-10-06T04:53:49.599582+0000","last_unstale":"2023-10-06T04:53:50.591486+0000","last_undegraded":"2023-10-06T04:53:50.591486+0000","last_fullsized":"2023-10-06T04:53:50.591486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:31.212541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.611481+0000","last_change":"2023-10-06T04:53:50.606936+0000","last_active":"2023-10-06T04:53:50.611481+0000","last_peered":"2023-10-06T04:53:50.611481+0000","last_clean":"2023-10-06T04:53:50.611481+0000","last_became_active":"2023-10-06T04:53:50.606648+0000","last_became_peered":"2023-10-06T04:53:50.606648+0000","last_unstale":"2023-10-06T04:53:50.611481+0000","last_undegraded":"2023-10-06T04:53:50.611481+0000","last_fullsized":"2023-10-06T04:53:50.611481+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:27.708179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.590111+0000","last_change":"2023-10-06T04:53:49.601241+0000","last_active":"2023-10-06T04:53:50.590111+0000","last_peered":"2023-10-06T04:53:50.590111+0000","last_clean":"2023-10-06T04:53:50.590111+0000","last_became_active":"2023-10-06T04:53:49.600982+0000","last_became_peered":"2023-10-06T04:53:49.600982+0000","last_unstale":"2023-10-06T04:53:50.590111+0000","last_undegraded":"2023-10-06T04:53:50.590111+0000","last_fullsized":"2023-10-06T04:53:50.590111+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:26.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.608902+0000","last_change":"2023-10-06T04:53:50.601564+0000","last_active":"2023-10-06T04:53:50.608902+0000","last_peered":"2023-10-06T04:53:50.608902+0000","last_clean":"2023-10-06T04:53:50.608902+0000","last_became_active":"2023-10-06T04:53:50.601273+0000","last_became_peered":"2023-10-06T04:53:50.601273+0000","last_unstale":"2023-10-06T04:53:50.608902+0000","last_undegraded":"2023-10-06T04:53:50.608902+0000","last_fullsized":"2023-10-06T04:53:50.608902+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:58.367511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.599679+0000","last_change":"2023-10-06T04:53:49.599636+0000","last_active":"2023-10-06T04:53:50.599679+0000","last_peered":"2023-10-06T04:53:50.599679+0000","last_clean":"2023-10-06T04:53:50.599679+0000","last_became_active":"2023-10-06T04:53:49.599356+0000","last_became_peered":"2023-10-06T04:53:49.599356+0000","last_unstale":"2023-10-06T04:53:50.599679+0000","last_undegraded":"2023-10-06T04:53:50.599679+0000","last_fullsized":"2023-10-06T04:53:50.599679+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:50.733425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610241+0000","last_change":"2023-10-06T04:53:50.605337+0000","last_active":"2023-10-06T04:53:50.610241+0000","last_peered":"2023-10-06T04:53:50.610241+0000","last_clean":"2023-10-06T04:53:50.610241+0000","last_became_active":"2023-10-06T04:53:50.599264+0000","last_became_peered":"2023-10-06T04:53:50.599264+0000","last_unstale":"2023-10-06T04:53:50.610241+0000","last_undegraded":"2023-10-06T04:53:50.610241+0000","last_fullsized":"2023-10-06T04:53:50.610241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:14.125211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.592369+0000","last_change":"2023-10-06T04:53:49.608771+0000","last_active":"2023-10-06T04:53:50.592369+0000","last_peered":"2023-10-06T04:53:50.592369+0000","last_clean":"2023-10-06T04:53:50.592369+0000","last_became_active":"2023-10-06T04:53:49.608472+0000","last_became_peered":"2023-10-06T04:53:49.608472+0000","last_unstale":"2023-10-06T04:53:50.592369+0000","last_undegraded":"2023-10-06T04:53:50.592369+0000","last_fullsized":"2023-10-06T04:53:50.592369+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:41.942599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.596385+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:53:50.596385+0000","last_peered":"2023-10-06T04:53:50.596385+0000","last_clean":"2023-10-06T04:53:50.596385+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:53:50.596385+0000","last_undegraded":"2023-10-06T04:53:50.596385+0000","last_fullsized":"2023-10-06T04:53:50.596385+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.612790+0000","last_change":"2023-10-06T04:53:50.610102+0000","last_active":"2023-10-06T04:53:50.612790+0000","last_peered":"2023-10-06T04:53:50.612790+0000","last_clean":"2023-10-06T04:53:50.612790+0000","last_became_active":"2023-10-06T04:53:50.609763+0000","last_became_peered":"2023-10-06T04:53:50.609763+0000","last_unstale":"2023-10-06T04:53:50.612790+0000","last_undegraded":"2023-10-06T04:53:50.612790+0000","last_fullsized":"2023-10-06T04:53:50.612790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:02.954769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.591571+0000","last_change":"2023-10-06T04:53:49.597803+0000","last_active":"2023-10-06T04:53:50.591571+0000","last_peered":"2023-10-06T04:53:50.591571+0000","last_clean":"2023-10-06T04:53:50.591571+0000","last_became_active":"2023-10-06T04:53:49.597467+0000","last_became_peered":"2023-10-06T04:53:49.597467+0000","last_unstale":"2023-10-06T04:53:50.591571+0000","last_undegraded":"2023-10-06T04:53:50.591571+0000","last_fullsized":"2023-10-06T04:53:50.591571+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:42.301398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.593501+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:53:50.593501+0000","last_peered":"2023-10-06T04:53:50.593501+0000","last_clean":"2023-10-06T04:53:50.593501+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:53:50.593501+0000","last_undegraded":"2023-10-06T04:53:50.593501+0000","last_fullsized":"2023-10-06T04:53:50.593501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618194+0000","last_change":"2023-10-06T04:53:50.611100+0000","last_active":"2023-10-06T04:53:50.618194+0000","last_peered":"2023-10-06T04:53:50.618194+0000","last_clean":"2023-10-06T04:53:50.618194+0000","last_became_active":"2023-10-06T04:53:50.610854+0000","last_became_peered":"2023-10-06T04:53:50.610854+0000","last_unstale":"2023-10-06T04:53:50.618194+0000","last_undegraded":"2023-10-06T04:53:50.618194+0000","last_fullsized":"2023-10-06T04:53:50.618194+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:38.165159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.595284+0000","last_change":"2023-10-06T04:53:49.608060+0000","last_active":"2023-10-06T04:53:50.595284+0000","last_peered":"2023-10-06T04:53:50.595284+0000","last_clean":"2023-10-06T04:53:50.595284+0000","last_became_active":"2023-10-06T04:53:49.607815+0000","last_became_peered":"2023-10-06T04:53:49.607815+0000","last_unstale":"2023-10-06T04:53:50.595284+0000","last_undegraded":"2023-10-06T04:53:50.595284+0000","last_fullsized":"2023-10-06T04:53:50.595284+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:45.794754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.596945+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:53:50.596945+0000","last_peered":"2023-10-06T04:53:50.596945+0000","last_clean":"2023-10-06T04:53:50.596945+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:53:50.596945+0000","last_undegraded":"2023-10-06T04:53:50.596945+0000","last_fullsized":"2023-10-06T04:53:50.596945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.594057+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:53:50.594057+0000","last_peered":"2023-10-06T04:53:50.594057+0000","last_clean":"2023-10-06T04:53:50.594057+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:53:50.594057+0000","last_undegraded":"2023-10-06T04:53:50.594057+0000","last_fullsized":"2023-10-06T04:53:50.594057+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610099+0000","last_change":"2023-10-06T04:53:50.604914+0000","last_active":"2023-10-06T04:53:50.610099+0000","last_peered":"2023-10-06T04:53:50.610099+0000","last_clean":"2023-10-06T04:53:50.610099+0000","last_became_active":"2023-10-06T04:53:50.604632+0000","last_became_peered":"2023-10-06T04:53:50.604632+0000","last_unstale":"2023-10-06T04:53:50.610099+0000","last_undegraded":"2023-10-06T04:53:50.610099+0000","last_fullsized":"2023-10-06T04:53:50.610099+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:41.257847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.589440+0000","last_change":"2023-10-06T04:53:49.611927+0000","last_active":"2023-10-06T04:53:50.589440+0000","last_peered":"2023-10-06T04:53:50.589440+0000","last_clean":"2023-10-06T04:53:50.589440+0000","last_became_active":"2023-10-06T04:53:49.611624+0000","last_became_peered":"2023-10-06T04:53:49.611624+0000","last_unstale":"2023-10-06T04:53:50.589440+0000","last_undegraded":"2023-10-06T04:53:50.589440+0000","last_fullsized":"2023-10-06T04:53:50.589440+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:26.248716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.594411+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:53:50.594411+0000","last_peered":"2023-10-06T04:53:50.594411+0000","last_clean":"2023-10-06T04:53:50.594411+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:53:50.594411+0000","last_undegraded":"2023-10-06T04:53:50.594411+0000","last_fullsized":"2023-10-06T04:53:50.594411+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.614301+0000","last_change":"2023-10-06T04:53:50.610630+0000","last_active":"2023-10-06T04:53:50.614301+0000","last_peered":"2023-10-06T04:53:50.614301+0000","last_clean":"2023-10-06T04:53:50.614301+0000","last_became_active":"2023-10-06T04:53:50.610295+0000","last_became_peered":"2023-10-06T04:53:50.610295+0000","last_unstale":"2023-10-06T04:53:50.614301+0000","last_undegraded":"2023-10-06T04:53:50.614301+0000","last_fullsized":"2023-10-06T04:53:50.614301+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.047782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:53:51.590732+0000","last_change":"2023-10-06T04:53:49.607924+0000","last_active":"2023-10-06T04:53:51.590732+0000","last_peered":"2023-10-06T04:53:51.590732+0000","last_clean":"2023-10-06T04:53:51.590732+0000","last_became_active":"2023-10-06T04:53:49.607341+0000","last_became_peered":"2023-10-06T04:53:49.607341+0000","last_unstale":"2023-10-06T04:53:51.590732+0000","last_undegraded":"2023-10-06T04:53:51.590732+0000","last_fullsized":"2023-10-06T04:53:51.590732+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.593966+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:53:50.593966+0000","last_peered":"2023-10-06T04:53:50.593966+0000","last_clean":"2023-10-06T04:53:50.593966+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:53:50.593966+0000","last_undegraded":"2023-10-06T04:53:50.593966+0000","last_fullsized":"2023-10-06T04:53:50.593966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.610378+0000","last_change":"2023-10-06T04:53:50.608484+0000","last_active":"2023-10-06T04:53:50.610378+0000","last_peered":"2023-10-06T04:53:50.610378+0000","last_clean":"2023-10-06T04:53:50.610378+0000","last_became_active":"2023-10-06T04:53:50.608045+0000","last_became_peered":"2023-10-06T04:53:50.608045+0000","last_unstale":"2023-10-06T04:53:50.610378+0000","last_undegraded":"2023-10-06T04:53:50.610378+0000","last_fullsized":"2023-10-06T04:53:50.610378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:52.970100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.613500+0000","last_change":"2023-10-06T04:53:49.608707+0000","last_active":"2023-10-06T04:53:50.613500+0000","last_peered":"2023-10-06T04:53:50.613500+0000","last_clean":"2023-10-06T04:53:50.613500+0000","last_became_active":"2023-10-06T04:53:49.608433+0000","last_became_peered":"2023-10-06T04:53:49.608433+0000","last_unstale":"2023-10-06T04:53:50.613500+0000","last_undegraded":"2023-10-06T04:53:50.613500+0000","last_fullsized":"2023-10-06T04:53:50.613500+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:10.623083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.594229+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:53:50.594229+0000","last_peered":"2023-10-06T04:53:50.594229+0000","last_clean":"2023-10-06T04:53:50.594229+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:53:50.594229+0000","last_undegraded":"2023-10-06T04:53:50.594229+0000","last_fullsized":"2023-10-06T04:53:50.594229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.618351+0000","last_change":"2023-10-06T04:53:50.611396+0000","last_active":"2023-10-06T04:53:50.618351+0000","last_peered":"2023-10-06T04:53:50.618351+0000","last_clean":"2023-10-06T04:53:50.618351+0000","last_became_active":"2023-10-06T04:53:50.610949+0000","last_became_peered":"2023-10-06T04:53:50.610949+0000","last_unstale":"2023-10-06T04:53:50.618351+0000","last_undegraded":"2023-10-06T04:53:50.618351+0000","last_fullsized":"2023-10-06T04:53:50.618351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:15.352583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.603252+0000","last_change":"2023-10-06T04:53:49.613283+0000","last_active":"2023-10-06T04:53:50.603252+0000","last_peered":"2023-10-06T04:53:50.603252+0000","last_clean":"2023-10-06T04:53:50.603252+0000","last_became_active":"2023-10-06T04:53:49.612982+0000","last_became_peered":"2023-10-06T04:53:49.612982+0000","last_unstale":"2023-10-06T04:53:50.603252+0000","last_undegraded":"2023-10-06T04:53:50.603252+0000","last_fullsized":"2023-10-06T04:53:50.603252+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:37.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.598511+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:53:50.598511+0000","last_peered":"2023-10-06T04:53:50.598511+0000","last_clean":"2023-10-06T04:53:50.598511+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:53:50.598511+0000","last_undegraded":"2023-10-06T04:53:50.598511+0000","last_fullsized":"2023-10-06T04:53:50.598511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.606442+0000","last_change":"2023-10-06T04:53:50.602637+0000","last_active":"2023-10-06T04:53:50.606442+0000","last_peered":"2023-10-06T04:53:50.606442+0000","last_clean":"2023-10-06T04:53:50.606442+0000","last_became_active":"2023-10-06T04:53:50.602265+0000","last_became_peered":"2023-10-06T04:53:50.602265+0000","last_unstale":"2023-10-06T04:53:50.606442+0000","last_undegraded":"2023-10-06T04:53:50.606442+0000","last_fullsized":"2023-10-06T04:53:50.606442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:49.582035+0000","last_clean_scrub_stamp":"2023-10-06T04:53:49.582035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:51.236747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.597971+0000","last_change":"2023-10-06T04:53:49.605704+0000","last_active":"2023-10-06T04:53:50.597971+0000","last_peered":"2023-10-06T04:53:50.597971+0000","last_clean":"2023-10-06T04:53:50.597971+0000","last_became_active":"2023-10-06T04:53:49.605449+0000","last_became_peered":"2023-10-06T04:53:49.605449+0000","last_unstale":"2023-10-06T04:53:50.597971+0000","last_undegraded":"2023-10-06T04:53:50.597971+0000","last_fullsized":"2023-10-06T04:53:50.597971+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:53:48.576893+0000","last_clean_scrub_stamp":"2023-10-06T04:53:48.576893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:57.362369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:53:50.598319+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:53:50.598319+0000","last_peered":"2023-10-06T04:53:50.598319+0000","last_clean":"2023-10-06T04:53:50.598319+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:53:50.598319+0000","last_undegraded":"2023-10-06T04:53:50.598319+0000","last_fullsized":"2023-10-06T04:53:50.598319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306568,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065272,"statfs":{"total":96636764160,"available":96322838528,"internally_reserved":0,"allocated":1114112,"data_stored":808537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":7.1779999999999999,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.44400000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.35899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.498}]},{"osd":3,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":512,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":524288,"data_stored":220022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.64400000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.38}]},{"osd":5,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.498},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.437}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":808573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.314},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.29399999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.375}]},{"osd":4,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.52000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301896,"kb_used_data":512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069944,"statfs":{"total":96636764160,"available":96327622656,"internally_reserved":0,"allocated":524288,"data_stored":219454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.38},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.505}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51300000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.627}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":528384,"data_stored":220934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.34899999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.40100000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41599999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.432},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.44},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301904,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069936,"statfs":{"total":96636764160,"available":96327614464,"internally_reserved":0,"allocated":532480,"data_stored":220956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.372}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.069,"5min":5.069,"15min":5.069},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.439}]},{"osd":4,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":6.4210000000000003,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4009999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.749},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.367,"5min":6.367,"15min":6.367},"last":0.629},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":5,"5min":5,"15min":5},"last":0.70799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":219952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":5.4569999999999999,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":5.5510000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.415,"5min":5.415,"15min":5.415},"last":0.48799999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.202,"5min":4.202,"15min":4.202},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.68300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":512000,"data_stored":218045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.625},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.7949999999999999,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.8170000000000002,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.7210000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:53:59.717 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:53:59.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:53:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.873+0000 7fdf62ed2700 1 Processor -- start 2023-10-06T04:53:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.873+0000 7fdf62ed2700 1 -- start start 2023-10-06T04:53:59.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.874+0000 7fdf62ed2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156050 0x7fdf5c06f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.874+0000 7fdf62ed2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf5c156420 0x7fdf5c06f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.874+0000 7fdf62ed2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0c7f60 con 0x7fdf5c0c6660 2023-10-06T04:53:59.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.874+0000 7fdf62ed2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0c7350 con 0x7fdf5c156420 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.874+0000 7fdf62ed2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0c7630 con 0x7fdf5c156050 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.874+0000 7fdf5bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf5c156420 0x7fdf5c06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.874+0000 7fdf5bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf5c156420 0x7fdf5c06f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40614/0 (socket says 172.21.15.3:40614) 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5bfff700 1 -- 172.21.15.3:0/134522383 learned_addr learned my addr 172.21.15.3:0/134522383 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5bfff700 1 -- 172.21.15.3:0/134522383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156050 msgr2=0x7fdf5c06f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf60c6e700 1 --2- 172.21.15.3:0/134522383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156050 0x7fdf5c06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5bfff700 1 --2- 172.21.15.3:0/134522383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156050 0x7fdf5c06f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5bfff700 1 -- 172.21.15.3:0/134522383 >> v1:172.21.15.153:6789/0 conn(0x7fdf5c0c6660 legacy=0x7fdf5c0c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5bfff700 1 -- 172.21.15.3:0/134522383 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0c7910 con 0x7fdf5c156420 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf60c6e700 1 --2- 172.21.15.3:0/134522383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156050 0x7fdf5c06f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:53:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5bfff700 1 --2- 172.21.15.3:0/134522383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156420 0x7fdf5c06f770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000bc50 tx=0x7fdf5000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd4d8563e3b3f38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5b7fe700 1 -- 172.21.15.3:0/134522383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000be60 con 0x7fdf5c156420 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5b7fe700 1 -- 172.21.15.3:0/134522383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5000ddc0 con 0x7fdf5c156420 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.875+0000 7fdf5b7fe700 1 -- 172.21.15.3:0/134522383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500106a0 con 0x7fdf5c156420 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 -- 172.21.15.3:0/134522383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156420 msgr2=0x7fdf5c06f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/134522383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156420 0x7fdf5c06f770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000bc50 tx=0x7fdf5000b4f0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 -- 172.21.15.3:0/134522383 shutdown_connections 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/134522383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156050 0x7fdf5c06f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/134522383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156420 0x7fdf5c06f770 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 -- 172.21.15.3:0/134522383 >> 172.21.15.3:0/134522383 conn(0x7fdf5c06ce40 msgr2=0x7fdf5c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 -- 172.21.15.3:0/134522383 shutdown_connections 2023-10-06T04:53:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.876+0000 7fdf62ed2700 1 -- 172.21.15.3:0/134522383 wait complete. 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.877+0000 7fdf62ed2700 1 Processor -- start 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.877+0000 7fdf62ed2700 1 -- start start 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf5c0c6660 0x7fdf5c13aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156050 0x7fdf5c0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156420 0x7fdf5c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c16a630 con 0x7fdf5c156050 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c169a20 con 0x7fdf5c0c6660 2023-10-06T04:53:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c169d00 con 0x7fdf5c156420 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156050 0x7fdf5c0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156050 0x7fdf5c0b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40616/0 (socket says 172.21.15.3:40616) 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 -- 172.21.15.3:0/1356438925 learned_addr learned my addr 172.21.15.3:0/1356438925 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf60c6e700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf5c0c6660 0x7fdf5c13aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf6146f700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156420 0x7fdf5c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 -- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156420 msgr2=0x7fdf5c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156420 0x7fdf5c0b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 -- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf5c0c6660 msgr2=0x7fdf5c13aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf5c0c6660 0x7fdf5c13aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 -- 172.21.15.3:0/1356438925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5000b790 con 0x7fdf5c156050 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf5bfff700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156050 0x7fdf5c0b1270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000b530 tx=0x7fdf5001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf58ff9700 1 -- 172.21.15.3:0/1356438925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500188a0 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf58ff9700 1 -- 172.21.15.3:0/1356438925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50018a20 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c169fe0 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf58ff9700 1 -- 172.21.15.3:0/1356438925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50018de0 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.878+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c169710 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.879+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c04c380 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.880+0000 7fdf58ff9700 1 -- 172.21.15.3:0/1356438925 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf5002b450 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.880+0000 7fdf58ff9700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdf48066f30 0x7fdf480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.880+0000 7fdf58ff9700 1 -- 172.21.15.3:0/1356438925 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdf50014070 con 0x7fdf5c156050 2023-10-06T04:53:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.880+0000 7fdf60c6e700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdf48066f30 0x7fdf480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:53:59.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.881+0000 7fdf60c6e700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdf48066f30 0x7fdf480693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00ac00 tx=0x7fdf4c006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:53:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:53:59.885+0000 7fdf58ff9700 1 -- 172.21.15.3:0/1356438925 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf50053370 con 0x7fdf5c156050 2023-10-06T04:54:00.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.159+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdf5c16a770 con 0x7fdf5c156050 2023-10-06T04:54:00.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.162+0000 7fdf58ff9700 1 -- 172.21.15.3:0/1356438925 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdf5c16a770 con 0x7fdf5c156050 2023-10-06T04:54:00.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdf48066f30 msgr2=0x7fdf480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdf48066f30 0x7fdf480693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00ac00 tx=0x7fdf4c006f90 comp rx=0 tx=0).stop 2023-10-06T04:54:00.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156050 msgr2=0x7fdf5c0b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156050 0x7fdf5c0b1270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000b530 tx=0x7fdf5001a000 comp rx=0 tx=0).stop 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 shutdown_connections 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdf48066f30 0x7fdf480693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf5c0c6660 0x7fdf5c13aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf5c156420 0x7fdf5c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 --2- 172.21.15.3:0/1356438925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c156050 0x7fdf5c0b1270 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 >> 172.21.15.3:0/1356438925 conn(0x7fdf5c06ce40 msgr2=0x7fdf5c06f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 shutdown_connections 2023-10-06T04:54:00.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.164+0000 7fdf62ed2700 1 -- 172.21.15.3:0/1356438925 wait complete. 2023-10-06T04:54:00.169 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T04:54:00.179 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.296+0000 7f7a53c38700 1 Processor -- start 2023-10-06T04:54:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.296+0000 7f7a53c38700 1 -- start start 2023-10-06T04:54:00.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a53c38700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 0x7f7a4c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a53c38700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c15e0d0 con 0x7f7a4c159080 2023-10-06T04:54:00.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a53c38700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0cbd70 con 0x7f7a4c0b2900 2023-10-06T04:54:00.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a53c38700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0b0e00 con 0x7f7a4c0b2530 2023-10-06T04:54:00.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a521d5700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7a4c159080 0x7f7a4c159500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49012/0 (socket says 172.21.15.3:49012) 2023-10-06T04:54:00.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a521d5700 1 -- 172.21.15.3:0/258362349 learned_addr learned my addr 172.21.15.3:0/258362349 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a519d4700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 0x7f7a4c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824263108 0 0) 0x7f7a44001490 con 0x7f7a4c159080 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a34003570 con 0x7f7a4c159080 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241007563 0 0) 0x7f7a4c0cbd70 con 0x7f7a4c0b2900 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.297+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a44001490 con 0x7f7a4c0b2900 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3793072513 0 0) 0x7f7a34003570 con 0x7f7a4c159080 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 msgr2=0x7f7a4c0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 0x7f7a4c0b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 >> v1:172.21.15.153:6789/0 conn(0x7f7a4c0b2900 legacy=0x7f7a4c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0cb160 con 0x7f7a4c159080 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f7a44001980 con 0x7f7a4c159080 2023-10-06T04:54:00.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a519d4700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 0x7f7a4c0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 >> v1:172.21.15.3:6789/0 conn(0x7f7a4c159080 legacy=0x7f7a4c159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 0x7f7a34003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a34004160 0x7f7a34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0b0e00 con 0x7f7a34004d50 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3400bba0 con 0x7f7a34004160 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3400bd60 con 0x7f7a4c0b2530 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a521d5700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a519d4700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 0x7f7a34003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a511d3700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a34004160 0x7f7a34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a521d5700 1 -- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 msgr2=0x7f7a34003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a521d5700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c0b2530 0x7f7a34003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a521d5700 1 -- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a34004160 msgr2=0x7f7a34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a521d5700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a34004160 0x7f7a34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a521d5700 1 -- 172.21.15.3:0/258362349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c009420 con 0x7f7a34004d50 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a521d5700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a340051c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400aa60 tx=0x7f7a4400a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6e8460d9b14e80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.298+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400a3f0 con 0x7f7a34004d50 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4400c5d0 con 0x7f7a34004d50 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a509d2700 1 -- 172.21.15.3:0/258362349 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400c950 con 0x7f7a34004d50 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a53c38700 1 -- 172.21.15.3:0/258362349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 msgr2=0x7f7a340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a53c38700 1 --2- 172.21.15.3:0/258362349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a340051c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400aa60 tx=0x7f7a4400a120 comp rx=0 tx=0).stop 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a521d5700 1 -- 172.21.15.3:0/258362349 reap_dead start 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a53c38700 1 -- 172.21.15.3:0/258362349 shutdown_connections 2023-10-06T04:54:00.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a53c38700 1 -- 172.21.15.3:0/258362349 >> 172.21.15.3:0/258362349 conn(0x7f7a4c070310 msgr2=0x7f7a4c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:00.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a53c38700 1 -- 172.21.15.3:0/258362349 shutdown_connections 2023-10-06T04:54:00.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.299+0000 7f7a53c38700 1 -- 172.21.15.3:0/258362349 wait complete. 2023-10-06T04:54:00.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.300+0000 7f7a53c38700 1 Processor -- start 2023-10-06T04:54:00.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.300+0000 7f7a53c38700 1 -- start start 2023-10-06T04:54:00.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a53c38700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a4c0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a53c38700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c159d30 0x7f7a4c168c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a53c38700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c156ad0 0x7f7a4c156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a53c38700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c15a1a0 con 0x7f7a34004d50 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a53c38700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c15b640 con 0x7f7a4c159d30 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a53c38700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c157460 con 0x7f7a4c156ad0 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a4c0c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a4c0c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40630/0 (socket says 172.21.15.3:40630) 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 -- 172.21.15.3:0/865773931 learned_addr learned my addr 172.21.15.3:0/865773931 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a511d3700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c159d30 0x7f7a4c168c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a521d5700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c156ad0 0x7f7a4c156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 -- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c156ad0 msgr2=0x7f7a4c156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c156ad0 0x7f7a4c156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 -- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c159d30 msgr2=0x7f7a4c168c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c159d30 0x7f7a4c168c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.301+0000 7f7a519d4700 1 -- 172.21.15.3:0/865773931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c15a3c0 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.302+0000 7f7a519d4700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a4c0c5f40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c011da0 tx=0x7f7a3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.302+0000 7f7a49ffb700 1 -- 172.21.15.3:0/865773931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c011ea0 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.302+0000 7f7a49ffb700 1 -- 172.21.15.3:0/865773931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00fc20 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.302+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c15a6a0 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.302+0000 7f7a49ffb700 1 -- 172.21.15.3:0/865773931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c014440 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.302+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c15ac60 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.303+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c071680 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.303+0000 7f7a49ffb700 1 -- 172.21.15.3:0/865773931 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a3c0147f0 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.304+0000 7f7a49ffb700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a38066f30 0x7f7a380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.304+0000 7f7a49ffb700 1 -- 172.21.15.3:0/865773931 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7a3c018030 con 0x7f7a34004d50 2023-10-06T04:54:00.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.304+0000 7f7a511d3700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a38066f30 0x7f7a380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:00.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.304+0000 7f7a511d3700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a38066f30 0x7f7a380693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a40007e40 tx=0x7f7a40007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:00.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.308+0000 7f7a49ffb700 1 -- 172.21.15.3:0/865773931 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a3c052290 con 0x7f7a34004d50 2023-10-06T04:54:00.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.547+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a4c0caad0 con 0x7f7a34004d50 2023-10-06T04:54:00.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.547+0000 7f7a49ffb700 1 -- 172.21.15.3:0/865773931 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f7a3c0558b0 con 0x7f7a34004d50 2023-10-06T04:54:00.550 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:00.551 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/596033972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":596033972},{"type":"v1","addr":"172.21.15.153:6837","nonce":596033972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1359588707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1359588707},{"type":"v1","addr":"172.21.15.3:6834","nonce":1359588707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/616658407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":616658407},{"type":"v1","addr":"172.21.15.3:6835","nonce":616658407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:49.997154+0000","modified":"2023-10-06T04:53:53.627991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/4017437319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":4017437319},{"type":"v1","addr":"172.21.15.153:6836","nonce":4017437319}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:54:00.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a38066f30 msgr2=0x7f7a380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a38066f30 0x7f7a380693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a40007e40 tx=0x7f7a40007d30 comp rx=0 tx=0).stop 2023-10-06T04:54:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 msgr2=0x7f7a4c0c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a4c0c5f40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c011da0 tx=0x7f7a3c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 shutdown_connections 2023-10-06T04:54:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a38066f30 0x7f7a380693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a4c156ad0 0x7f7a4c156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a34004d50 0x7f7a4c0c5f40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 --2- 172.21.15.3:0/865773931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c159d30 0x7f7a4c168c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 >> 172.21.15.3:0/865773931 conn(0x7f7a4c070310 msgr2=0x7f7a4c0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 shutdown_connections 2023-10-06T04:54:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:00.550+0000 7f7a53c38700 1 -- 172.21.15.3:0/865773931 wait complete. 2023-10-06T04:54:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2023-10-06T04:54:00.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:53:49.997154+0000', 'modified': '2023-10-06T04:53:53.627991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5361}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5361': {'gid': 5361, 'name': 'd', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6836/4017437319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 4017437319}, {'type': 'v1', 'addr': '172.21.15.153:6836', 'nonce': 4017437319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:54:00.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:54:00.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T04:54:01.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.088+0000 7f805417b700 1 Processor -- start 2023-10-06T04:54:01.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.089+0000 7f805417b700 1 -- start start 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.089+0000 7f805417b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f804c15c8f0 0x7f804c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.089+0000 7f805417b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c0d0870 0x7f804c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.089+0000 7f805417b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c0cf6c0 con 0x7f804c15ccc0 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.089+0000 7f805417b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c0cfb50 con 0x7f804c15c8f0 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.089+0000 7f805417b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c15e050 con 0x7f804c0d0870 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051f17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f804c15c8f0 0x7f804c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051716700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f804c15ccc0 0x7f804c0d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49026/0 (socket says 172.21.15.3:49026) 2023-10-06T04:54:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051716700 1 -- 172.21.15.3:0/2560918388 learned_addr learned my addr 172.21.15.3:0/2560918388 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8052718700 1 --2- 172.21.15.3:0/2560918388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c0d0870 0x7f804c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8050f15700 1 -- 172.21.15.3:0/2560918388 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897814541 0 0) 0x7f804c0cf6c0 con 0x7f804c15ccc0 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8050f15700 1 -- 172.21.15.3:0/2560918388 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8034003570 con 0x7f804c15ccc0 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051f17700 1 -- 172.21.15.3:0/2560918388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c0d0870 msgr2=0x7f804c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051f17700 1 --2- 172.21.15.3:0/2560918388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c0d0870 0x7f804c0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051f17700 1 -- 172.21.15.3:0/2560918388 >> v1:172.21.15.3:6789/0 conn(0x7f804c15ccc0 legacy=0x7f804c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051f17700 1 -- 172.21.15.3:0/2560918388 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0ceab0 con 0x7f804c15c8f0 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8051f17700 1 --2- 172.21.15.3:0/2560918388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c15c8f0 0x7f804c15db10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2610c98e17fe9764 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8050f15700 1 -- 172.21.15.3:0/2560918388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c011630 con 0x7f804c15c8f0 2023-10-06T04:54:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8050f15700 1 -- 172.21.15.3:0/2560918388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c011790 con 0x7f804c15c8f0 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.090+0000 7f8050f15700 1 -- 172.21.15.3:0/2560918388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c011b30 con 0x7f804c15c8f0 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 -- 172.21.15.3:0/2560918388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c15c8f0 msgr2=0x7f804c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 --2- 172.21.15.3:0/2560918388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c15c8f0 0x7f804c15db10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 -- 172.21.15.3:0/2560918388 shutdown_connections 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 --2- 172.21.15.3:0/2560918388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c0d0870 0x7f804c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 --2- 172.21.15.3:0/2560918388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c15c8f0 0x7f804c15db10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 -- 172.21.15.3:0/2560918388 >> 172.21.15.3:0/2560918388 conn(0x7f804c070310 msgr2=0x7f804c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 -- 172.21.15.3:0/2560918388 shutdown_connections 2023-10-06T04:54:01.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.091+0000 7f805417b700 1 -- 172.21.15.3:0/2560918388 wait complete. 2023-10-06T04:54:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.092+0000 7f805417b700 1 Processor -- start 2023-10-06T04:54:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.092+0000 7f805417b700 1 -- start start 2023-10-06T04:54:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f805417b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c0d0870 0x7f804c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f805417b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f804c15c8f0 0x7f804c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f805417b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c15ccc0 0x7f804c15aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f805417b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c1543f0 con 0x7f804c0d0870 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f805417b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c158880 con 0x7f804c15c8f0 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f805417b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c159250 con 0x7f804c15ccc0 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c0d0870 0x7f804c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c0d0870 0x7f804c153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40640/0 (socket says 172.21.15.3:40640) 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 -- 172.21.15.3:0/1323241356 learned_addr learned my addr 172.21.15.3:0/1323241356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 -- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c15ccc0 msgr2=0x7f804c15aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c15ccc0 0x7f804c15aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 -- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f804c15c8f0 msgr2=0x7f804c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f804c15c8f0 0x7f804c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 -- 172.21.15.3:0/1323241356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c009420 con 0x7f804c0d0870 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8051f17700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c0d0870 0x7f804c153eb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f804c06eaa0 tx=0x7f803c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.093+0000 7f8049ffb700 1 -- 172.21.15.3:0/1323241356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c011b10 con 0x7f804c0d0870 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.094+0000 7f8049ffb700 1 -- 172.21.15.3:0/1323241356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c0253f0 con 0x7f804c0d0870 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.094+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c157c70 con 0x7f804c0d0870 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.094+0000 7f8049ffb700 1 -- 172.21.15.3:0/1323241356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0257b0 con 0x7f804c0d0870 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.094+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c158230 con 0x7f804c0d0870 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.095+0000 7f8049ffb700 1 -- 172.21.15.3:0/1323241356 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f803c025ba0 con 0x7f804c0d0870 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.096+0000 7f8049ffb700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8038066f30 0x7f80380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.096+0000 7f8049ffb700 1 -- 172.21.15.3:0/1323241356 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f803c085fc0 con 0x7f804c0d0870 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.096+0000 7f8051716700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8038066f30 0x7f80380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.097+0000 7f8051716700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8038066f30 0x7f80380693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80400058d0 tx=0x7f80400079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 Processor -- start 2023-10-06T04:54:01.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 -- start start 2023-10-06T04:54:01.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a629c80 con 0x55774a6ac9a0 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a600a70 con 0x55774a6aabd0 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774a5fe270 con 0x55774a6f0a90 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.098+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6a9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40654/0 (socket says 172.21.15.3:40654) 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 -- 172.21.15.3:0/3546765771 learned_addr learned my addr 172.21.15.3:0/3546765771 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051f17700 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8052718700 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 -- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 msgr2=0x55774a6f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 -- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 msgr2=0x55774a6a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 -- 172.21.15.3:0/3546765771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774a494270 con 0x55774a6ac9a0 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8052718700 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8051716700 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6a9580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8040005c50 tx=0x7f8040011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772b18367fdaa5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8032ffd700 1 -- 172.21.15.3:0/3546765771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804001b070 con 0x55774a6ac9a0 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8032ffd700 1 -- 172.21.15.3:0/3546765771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040012440 con 0x55774a6ac9a0 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.099+0000 7f8032ffd700 1 -- 172.21.15.3:0/3546765771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040012800 con 0x55774a6ac9a0 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 -- 172.21.15.3:0/3546765771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 msgr2=0x55774a6a9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6a9580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8040005c50 tx=0x7f8040011000 comp rx=0 tx=0).stop 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 -- 172.21.15.3:0/3546765771 shutdown_connections 2023-10-06T04:54:01.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 --2- 172.21.15.3:0/3546765771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6a9580 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 -- 172.21.15.3:0/3546765771 >> 172.21.15.3:0/3546765771 conn(0x55774a6ab070 msgr2=0x55774a6ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 -- 172.21.15.3:0/3546765771 shutdown_connections 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 -- 172.21.15.3:0/3546765771 wait complete. 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 -- start start 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.100+0000 7f805d473b80 1 Processor -- start 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6aabd0 0x55774a6ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f805d473b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6f0a90 0x55774a6d3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f805d473b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a5fa060 con 0x55774a6ac9a0 2023-10-06T04:54:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a5a5890 con 0x55774a6f0a90 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774a5c0ea0 con 0x55774a6aabd0 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f805d473b80 10 client.0 _ll_register_callbacks cb 0x55774a496f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6eb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40662/0 (socket says 172.21.15.3:40662) 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.101+0000 7f8051716700 1 -- 172.21.15.3:0/1510266848 learned_addr learned my addr 172.21.15.3:0/1510266848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8051f17700 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6aabd0 0x55774a6ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8051716700 1 -- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6aabd0 msgr2=0x55774a6ed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8051716700 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6aabd0 0x55774a6ed1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8051716700 1 -- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6f0a90 msgr2=0x55774a6d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8052718700 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6f0a90 0x55774a6d3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8051716700 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6f0a90 0x55774a6d3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8051716700 1 -- 172.21.15.3:0/1510266848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8034003570 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f8051716700 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6eb890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8040007dc0 tx=0x7f8040015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b94585c22369af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f802bfff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f802bfff700 1 -- 172.21.15.3:0/1510266848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040027070 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f802bfff700 1 -- 172.21.15.3:0/1510266848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f80400125b0 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55774a69e690 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f802bfff700 1 -- 172.21.15.3:0/1510266848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8040021430 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f805d473b80 10 client.5529 fetch_fsmap learned FSMap version 23 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55774a436fe0 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f802bfff700 1 -- 172.21.15.3:0/1510266848 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55774a69e690 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.102+0000 7f802bfff700 1 -- 172.21.15.3:0/1510266848 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8040021590 con 0x55774a6ac9a0 2023-10-06T04:54:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f805d473b80 10 client.5529 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f805d473b80 10 client.5529 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f805d473b80 20 client.5529 populate_metadata read hostname 'smithi003' 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f805d473b80 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6d1b10 0x55774a6d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f805d473b80 4 client.5529 mds_command: new command op to 5361 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 --> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55774a67b5c0 con 0x55774a6d1b10 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f8051f17700 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6d1b10 0x55774a6d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f802bfff700 10 client.5529 ms_handle_connect on v2:172.21.15.153:6834/4017437319 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.103+0000 7f8051f17700 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6d1b10 0x55774a6d1ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8deedcd0c20ce2e6 server_cookie=ae42a7fdfb3e495c in_seq=0 out_seq=0 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f802bfff700 1 -- 172.21.15.3:0/1510266848 <== mds.0 v2:172.21.15.153:6834/4017437319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55774a67b5c0 con 0x55774a6d1b10 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f802bfff700 10 client.5529 handle_command_reply: tid=0 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 client.5529 shutdown 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 msgr2=0x55774a6eb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6eb890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8040007dc0 tx=0x7f8040015000 comp rx=0 tx=0).stop 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 shutdown_connections 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6f0a90 0x55774a6d3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6d1b10 0x55774a6d1ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6aabd0 0x55774a6ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f802bfff700 0 client.5529 ms_handle_reset on v2:172.21.15.153:6834/4017437319 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 --2- 172.21.15.3:0/1510266848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6eb890 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 >> 172.21.15.3:0/1510266848 conn(0x55774a6ab070 msgr2=0x55774a6eff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 shutdown_connections 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.104+0000 7f805d473b80 1 -- 172.21.15.3:0/1510266848 wait complete. 2023-10-06T04:54:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.105+0000 7f805d473b80 20 client.5529 trim_cache size 0 max 16384 2023-10-06T04:54:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.121+0000 7f805d473b80 1 Processor -- start 2023-10-06T04:54:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f805d473b80 1 -- start start 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f805d473b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f805d473b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a47ce30 con 0x55774a6ac9a0 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a712770 con 0x55774a6aabd0 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774a6d4e10 con 0x55774a6f0a90 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6ef280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40666/0 (socket says 172.21.15.3:40666) 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f8051716700 1 -- 172.21.15.3:0/3136717888 learned_addr learned my addr 172.21.15.3:0/3136717888 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:01.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f8051f17700 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.122+0000 7f8052718700 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f8051716700 1 -- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 msgr2=0x55774a6f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f8051716700 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f8051716700 1 -- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 msgr2=0x55774a6a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f8051716700 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f8051716700 1 -- 172.21.15.3:0/3136717888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774a5c8780 con 0x55774a6ac9a0 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f8052718700 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f8051716700 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6ef280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8040013f50 tx=0x7f8040027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167e495520080a4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f80337fe700 1 -- 172.21.15.3:0/3136717888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040021c80 con 0x55774a6ac9a0 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f80337fe700 1 -- 172.21.15.3:0/3136717888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040021a50 con 0x55774a6ac9a0 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f80337fe700 1 -- 172.21.15.3:0/3136717888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400125b0 con 0x55774a6ac9a0 2023-10-06T04:54:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 -- 172.21.15.3:0/3136717888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 msgr2=0x55774a6ef280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6ef280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8040013f50 tx=0x7f8040027000 comp rx=0 tx=0).stop 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 -- 172.21.15.3:0/3136717888 shutdown_connections 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a6f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 --2- 172.21.15.3:0/3136717888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6ef280 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 -- 172.21.15.3:0/3136717888 >> 172.21.15.3:0/3136717888 conn(0x55774a6ab070 msgr2=0x55774a6eff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 -- 172.21.15.3:0/3136717888 shutdown_connections 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.123+0000 7f805d473b80 1 -- 172.21.15.3:0/3136717888 wait complete. 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.124+0000 7f805d473b80 1 -- start start 2023-10-06T04:54:01.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.124+0000 7f805d473b80 1 Processor -- start 2023-10-06T04:54:01.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f805d473b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6d87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f805d473b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a758390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f805d473b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a756940 con 0x55774a6ac9a0 2023-10-06T04:54:01.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774a6eb790 con 0x55774a6aabd0 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f805d473b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774a7588d0 con 0x55774a6f0a90 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f805d473b80 10 client.0 _ll_register_callbacks cb 0x55774a496f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6d87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6d87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40670/0 (socket says 172.21.15.3:40670) 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 -- 172.21.15.3:0/708455471 learned_addr learned my addr 172.21.15.3:0/708455471 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8052718700 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a758390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051f17700 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 -- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 msgr2=0x55774a758390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a758390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 -- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 msgr2=0x55774a6a6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 -- 172.21.15.3:0/708455471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55774a436fe0 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f8051716700 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6d87b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f804000c200 tx=0x7f8040023070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3756236313c271 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.125+0000 7f802bfff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f802bfff700 1 -- 172.21.15.3:0/708455471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040027070 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f802bfff700 1 -- 172.21.15.3:0/708455471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f80400125b0 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8040012440 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f802bfff700 1 -- 172.21.15.3:0/708455471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8040014640 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 10 client.5535 fetch_fsmap learned FSMap version 23 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55774a5af470 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f802bfff700 1 -- 172.21.15.3:0/708455471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040012440 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f802bfff700 1 -- 172.21.15.3:0/708455471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f80400147a0 con 0x55774a6ac9a0 2023-10-06T04:54:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 10 client.5535 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 10 client.5535 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 20 client.5535 populate_metadata read hostname 'smithi003' 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6a9b50 0x55774a6a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 4 client.5535 mds_command: new command op to 5361 tid=0[{"prefix": "status"}] 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 --> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55774a59ce00 con 0x55774a6a9b50 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.126+0000 7f8051f17700 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6a9b50 0x55774a6a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f802bfff700 10 client.5535 ms_handle_connect on v2:172.21.15.153:6834/4017437319 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f8051f17700 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6a9b50 0x55774a6a9f20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac103200065aaf0 server_cookie=5c7bce4e965d754d in_seq=0 out_seq=0 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f802bfff700 1 -- 172.21.15.3:0/708455471 <== mds.0 v2:172.21.15.153:6834/4017437319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55774a59ce00 con 0x55774a6a9b50 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f802bfff700 10 client.5535 handle_command_reply: tid=0 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 client.5535 shutdown 2023-10-06T04:54:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 msgr2=0x55774a6d87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6d87b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f804000c200 tx=0x7f8040023070 comp rx=0 tx=0).stop 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 shutdown_connections 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774a6aabd0 0x55774a6a6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:6834/4017437319,v1:172.21.15.153:6836/4017437319] conn(0x55774a6a9b50 0x55774a6a9f20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55774a6f0a90 0x55774a758390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f802bfff700 0 client.5535 ms_handle_reset on v2:172.21.15.153:6834/4017437319 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 --2- 172.21.15.3:0/708455471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55774a6ac9a0 0x55774a6d87b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 >> 172.21.15.3:0/708455471 conn(0x55774a6ab070 msgr2=0x55774a6a5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 shutdown_connections 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 1 -- 172.21.15.3:0/708455471 wait complete. 2023-10-06T04:54:01.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.127+0000 7f805d473b80 20 client.5535 trim_cache size 0 max 16384 2023-10-06T04:54:01.138 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:01.138 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T04:54:01.138 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T04:54:01.138 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5361, 2023-10-06T04:54:01.138 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T04:54:01.138 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T04:54:01.138 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T04:54:01.139 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.541070056000001, 2023-10-06T04:54:01.139 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 23, 2023-10-06T04:54:01.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 44, 2023-10-06T04:54:01.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 44, 2023-10-06T04:54:01.139 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.607453942999999 2023-10-06T04:54:01.139 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8038066f30 msgr2=0x7f80380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8038066f30 0x7f80380693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80400058d0 tx=0x7f80400079d0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c0d0870 msgr2=0x7f804c153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c0d0870 0x7f804c153eb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f804c06eaa0 tx=0x7f803c017000 comp rx=0 tx=0).stop 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 shutdown_connections 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8038066f30 0x7f80380693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f804c15ccc0 0x7f804c15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c0d0870 0x7f804c153eb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 --2- 172.21.15.3:0/1323241356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f804c15c8f0 0x7f804c158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 >> 172.21.15.3:0/1323241356 conn(0x7f804c070310 msgr2=0x7f804c0cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 shutdown_connections 2023-10-06T04:54:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.128+0000 7f805417b700 1 -- 172.21.15.3:0/1323241356 wait complete. 2023-10-06T04:54:01.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:54:01.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.258+0000 7f70fc0f2700 1 Processor -- start 2023-10-06T04:54:01.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.259+0000 7f70fc0f2700 1 -- start start 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.259+0000 7f70fc0f2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f70f41578e0 0x7f70f4164910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.259+0000 7f70fc0f2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f415e4c0 con 0x7f70f406d480 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.259+0000 7f70fc0f2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f415d8b0 con 0x7f70f40b39d0 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.259+0000 7f70fc0f2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f415db90 con 0x7f70f41578e0 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70f968d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f70f406d480 0x7f70f41571d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49034/0 (socket says 172.21.15.3:49034) 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70f968d700 1 -- 172.21.15.3:0/646746050 learned_addr learned my addr 172.21.15.3:0/646746050 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70fa68f700 1 --2- 172.21.15.3:0/646746050 >> v2:172.21.15.153:3300/0 conn(0x7f70f41578e0 0x7f70f4164910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70f8e8c700 1 -- 172.21.15.3:0/646746050 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681050113 0 0) 0x7f70f415e4c0 con 0x7f70f406d480 2023-10-06T04:54:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70f8e8c700 1 -- 172.21.15.3:0/646746050 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70dc003570 con 0x7f70f406d480 2023-10-06T04:54:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70fa68f700 1 -- 172.21.15.3:0/646746050 >> v1:172.21.15.153:6789/0 conn(0x7f70f40b39d0 legacy=0x7f70f40cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70fa68f700 1 -- 172.21.15.3:0/646746050 >> v1:172.21.15.3:6789/0 conn(0x7f70f406d480 legacy=0x7f70f41571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70fa68f700 1 -- 172.21.15.3:0/646746050 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f415de70 con 0x7f70f41578e0 2023-10-06T04:54:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70fa68f700 1 --2- 172.21.15.3:0/646746050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 0x7f70f4164910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009e60 tx=0x7f70e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=680d5b58e99ca5b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.260+0000 7f70f8e8c700 1 -- 172.21.15.3:0/646746050 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400e070 con 0x7f70f41578e0 2023-10-06T04:54:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70f8e8c700 1 -- 172.21.15.3:0/646746050 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e400bb60 con 0x7f70f41578e0 2023-10-06T04:54:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70f8e8c700 1 -- 172.21.15.3:0/646746050 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4012430 con 0x7f70f41578e0 2023-10-06T04:54:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70fc0f2700 1 -- 172.21.15.3:0/646746050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 msgr2=0x7f70f4164910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/646746050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 0x7f70f4164910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009e60 tx=0x7f70e4009780 comp rx=0 tx=0).stop 2023-10-06T04:54:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70fc0f2700 1 -- 172.21.15.3:0/646746050 shutdown_connections 2023-10-06T04:54:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/646746050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 0x7f70f4164910 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70fc0f2700 1 -- 172.21.15.3:0/646746050 >> 172.21.15.3:0/646746050 conn(0x7f70f40ce2c0 msgr2=0x7f70f40ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70fc0f2700 1 -- 172.21.15.3:0/646746050 shutdown_connections 2023-10-06T04:54:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.261+0000 7f70fc0f2700 1 -- 172.21.15.3:0/646746050 wait complete. 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.262+0000 7f70fc0f2700 1 Processor -- start 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.262+0000 7f70fc0f2700 1 -- start start 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70fc0f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f406d480 0x7f70f40af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70fc0f2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70f40b39d0 0x7f70f41501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70fc0f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 0x7f70f414cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70fc0f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f406d970 con 0x7f70f406d480 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70fc0f2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f414fcb0 con 0x7f70f41578e0 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70fc0f2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40afe60 con 0x7f70f40b39d0 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f406d480 0x7f70f40af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f406d480 0x7f70f40af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40674/0 (socket says 172.21.15.3:40674) 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 -- 172.21.15.3:0/1429678407 learned_addr learned my addr 172.21.15.3:0/1429678407 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:01.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f968d700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70f40b39d0 0x7f70f41501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70fa68f700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 0x7f70f414cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 -- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70f40b39d0 msgr2=0x7f70f41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70f40b39d0 0x7f70f41501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 -- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 msgr2=0x7f70f414cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 0x7f70f414cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 -- 172.21.15.3:0/1429678407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4009a20 con 0x7f70f406d480 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.263+0000 7f70f9e8e700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f406d480 0x7f70f40af7d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c590 tx=0x7f70ec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.264+0000 7f70f1ffb700 1 -- 172.21.15.3:0/1429678407 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec011590 con 0x7f70f406d480 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.264+0000 7f70f1ffb700 1 -- 172.21.15.3:0/1429678407 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec0116f0 con 0x7f70f406d480 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.264+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f406dc10 con 0x7f70f406d480 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.264+0000 7f70f1ffb700 1 -- 172.21.15.3:0/1429678407 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec011ab0 con 0x7f70f406d480 2023-10-06T04:54:01.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.264+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f40b0140 con 0x7f70f406d480 2023-10-06T04:54:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.265+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f40644c0 con 0x7f70f406d480 2023-10-06T04:54:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.265+0000 7f70f1ffb700 1 -- 172.21.15.3:0/1429678407 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70ec01b3b0 con 0x7f70f406d480 2023-10-06T04:54:01.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.265+0000 7f70f1ffb700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70e0066f30 0x7f70e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:01.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.266+0000 7f70f1ffb700 1 -- 172.21.15.3:0/1429678407 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f70ec083c40 con 0x7f70f406d480 2023-10-06T04:54:01.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.266+0000 7f70f968d700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70e0066f30 0x7f70e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:01.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.266+0000 7f70f968d700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70e0066f30 0x7f70e00693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f70f4159ea0 tx=0x7f70e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:01.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.270+0000 7f70f1ffb700 1 -- 172.21.15.3:0/1429678407 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70ec050d50 con 0x7f70f406d480 2023-10-06T04:54:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.511+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70f4071680 con 0x7f70f406d480 2023-10-06T04:54:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.511+0000 7f70f1ffb700 1 -- 172.21.15.3:0/1429678407 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70ec017070 con 0x7f70f406d480 2023-10-06T04:54:01.514 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:54:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70e0066f30 msgr2=0x7f70e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70e0066f30 0x7f70e00693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f70f4159ea0 tx=0x7f70e8009040 comp rx=0 tx=0).stop 2023-10-06T04:54:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f406d480 msgr2=0x7f70f40af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f406d480 0x7f70f40af7d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c590 tx=0x7f70ec00c950 comp rx=0 tx=0).stop 2023-10-06T04:54:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 shutdown_connections 2023-10-06T04:54:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f70e0066f30 0x7f70e00693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f41578e0 0x7f70f414cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f406d480 0x7f70f40af7d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 --2- 172.21.15.3:0/1429678407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f70f40b39d0 0x7f70f41501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 >> 172.21.15.3:0/1429678407 conn(0x7f70f40ce2c0 msgr2=0x7f70f4069050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 shutdown_connections 2023-10-06T04:54:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:01.513+0000 7f70fc0f2700 1 -- 172.21.15.3:0/1429678407 wait complete. 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:54:01.528 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:01.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:01.529 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:01.544 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42389sec preferred_lft 42389sec 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:54:01.545 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:54:01.546 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:01.547 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:01.547 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:54:01.547 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:54:01.547 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:01.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:01.547 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:54:01.608 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:54:01.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:01.610 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:54:01.666 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:54:01.666 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:54:01.666 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:54:01.666 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:54:01.666 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:54:01.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:01.667 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:54:01.667 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:54:01.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:54:01.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:54:01.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:54:01.667 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:54:01.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:01.769 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:54:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:54:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:54:01.784 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:01.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:01.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:54:01.849 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:54:01.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:01.850 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:01.904 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:01.905 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:01.905 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:01.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:54:02.025 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:02.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:02.089 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:02.090 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:02.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:02.091 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:54:02.146 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:54:02.147 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:54:02.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:54:02.214 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:02.215 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:02.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:54:02.269+0000 7ff0ee41d540 -1 init, newargv = 0x562eaa343870 newargc=15 2023-10-06T04:54:02.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[106632]: starting ceph client 2023-10-06T04:54:02.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[106632]: starting fuse 2023-10-06T04:54:02.281 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:02.282 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:02.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:02.283 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:54:02.337 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:54:02.338 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:02.338 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:02.339 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:02.340 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:02.340 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:02.340 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:02.340 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:02.340 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:02.340 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:02.462 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:02.644 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:02.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:02.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106632.asok status 2023-10-06T04:54:02.806 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "106632", 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:54:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:54:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5550, 2023-10-06T04:54:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5550 2023-10-06T04:54:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5550 192.168.0.1:0/2158008368", 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2158008368", 2023-10-06T04:54:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:54:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 44, 2023-10-06T04:54:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:54:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:02.811 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:02.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:02.816 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:54:02.816 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:02.832 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:54:02.833 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:02.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:02.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:02.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:54:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.058+0000 7f0b35724700 1 Processor -- start 2023-10-06T04:54:03.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.058+0000 7f0b35724700 1 -- start start 2023-10-06T04:54:03.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b35724700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3015c8f0 0x7f0b3015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:03.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b35724700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0b3015ccc0 0x7f0b300d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:03.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b35724700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300cf6c0 con 0x7f0b300d06a0 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b35724700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b300cfb50 con 0x7f0b3015ccc0 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b35724700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3015e050 con 0x7f0b3015c8f0 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b2e7fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0b3015ccc0 0x7f0b300d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b2effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3015c8f0 0x7f0b3015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b2effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3015c8f0 0x7f0b3015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:57006/0 (socket says 172.21.15.3:57006) 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b2e7fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0b3015ccc0 0x7f0b300d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:40394/0 (socket says 172.21.15.3:40394) 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.059+0000 7f0b2effd700 1 -- 172.21.15.3:0/1616443219 learned_addr learned my addr 172.21.15.3:0/1616443219 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2e7fc700 1 -- 172.21.15.3:0/1616443219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3015c8f0 msgr2=0x7f0b3015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2e7fc700 1 --2- 172.21.15.3:0/1616443219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3015c8f0 0x7f0b3015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2e7fc700 1 -- 172.21.15.3:0/1616443219 >> v1:172.21.15.153:6789/0 conn(0x7f0b300d06a0 legacy=0x7f0b3006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2e7fc700 1 -- 172.21.15.3:0/1616443219 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300d0af0 con 0x7f0b3015ccc0 2023-10-06T04:54:03.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2dffb700 1 -- 172.21.15.3:0/1616443219 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325624440 0 0) 0x7f0b300cf6c0 con 0x7f0b300d06a0 2023-10-06T04:54:03.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2e7fc700 1 --2- 172.21.15.3:0/1616443219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 0x7f0b300d0160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b180098e0 tx=0x7f0b180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360a4ac5633d6852 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:03.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2dffb700 1 -- 172.21.15.3:0/1616443219 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800b640 con 0x7f0b3015ccc0 2023-10-06T04:54:03.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2dffb700 1 -- 172.21.15.3:0/1616443219 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1800b7a0 con 0x7f0b3015ccc0 2023-10-06T04:54:03.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b2dffb700 1 -- 172.21.15.3:0/1616443219 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800bb40 con 0x7f0b3015ccc0 2023-10-06T04:54:03.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b35724700 1 -- 172.21.15.3:0/1616443219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 msgr2=0x7f0b300d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:03.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.060+0000 7f0b35724700 1 --2- 172.21.15.3:0/1616443219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 0x7f0b300d0160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b180098e0 tx=0x7f0b180091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.061+0000 7f0b35724700 1 -- 172.21.15.3:0/1616443219 shutdown_connections 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.061+0000 7f0b35724700 1 --2- 172.21.15.3:0/1616443219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3015c8f0 0x7f0b3015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.061+0000 7f0b35724700 1 --2- 172.21.15.3:0/1616443219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 0x7f0b300d0160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.061+0000 7f0b35724700 1 -- 172.21.15.3:0/1616443219 >> 172.21.15.3:0/1616443219 conn(0x7f0b30070310 msgr2=0x7f0b30070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.061+0000 7f0b35724700 1 -- 172.21.15.3:0/1616443219 shutdown_connections 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.061+0000 7f0b35724700 1 -- 172.21.15.3:0/1616443219 wait complete. 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.062+0000 7f0b35724700 1 Processor -- start 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.062+0000 7f0b35724700 1 -- start start 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b35724700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b300d06a0 0x7f0b3014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b35724700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b3015c8f0 0x7f0b3016d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b35724700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 0x7f0b30158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:03.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b35724700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3016ce00 con 0x7f0b3015c8f0 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b35724700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3016d0a0 con 0x7f0b3015ccc0 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b35724700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30154640 con 0x7f0b300d06a0 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b3015c8f0 0x7f0b3016d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b3015c8f0 0x7f0b3016d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43924/0 (socket says 172.21.15.3:43924) 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 -- 172.21.15.3:0/454705871 learned_addr learned my addr 172.21.15.3:0/454705871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2effd700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b300d06a0 0x7f0b3014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2f7fe700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 0x7f0b30158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 -- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b300d06a0 msgr2=0x7f0b3014e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b300d06a0 0x7f0b3014e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 -- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 msgr2=0x7f0b30158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 0x7f0b30158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 -- 172.21.15.3:0/454705871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b18009420 con 0x7f0b3015c8f0 2023-10-06T04:54:03.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b2e7fc700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b3015c8f0 0x7f0b3016d530 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0b180093f0 tx=0x7f0b18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:03.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.063+0000 7f0b26ffd700 1 -- 172.21.15.3:0/454705871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800bb20 con 0x7f0b3015c8f0 2023-10-06T04:54:03.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.064+0000 7f0b26ffd700 1 -- 172.21.15.3:0/454705871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18011420 con 0x7f0b3015c8f0 2023-10-06T04:54:03.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.064+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b30154920 con 0x7f0b3015c8f0 2023-10-06T04:54:03.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.064+0000 7f0b26ffd700 1 -- 172.21.15.3:0/454705871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b180117c0 con 0x7f0b3015c8f0 2023-10-06T04:54:03.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.064+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3014f650 con 0x7f0b3015c8f0 2023-10-06T04:54:03.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.065+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3016d210 con 0x7f0b3015c8f0 2023-10-06T04:54:03.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.065+0000 7f0b26ffd700 1 -- 172.21.15.3:0/454705871 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b18011bf0 con 0x7f0b3015c8f0 2023-10-06T04:54:03.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.065+0000 7f0b26ffd700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0b10066f30 0x7f0b100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:03.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.066+0000 7f0b26ffd700 1 -- 172.21.15.3:0/454705871 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0b18086040 con 0x7f0b3015c8f0 2023-10-06T04:54:03.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.066+0000 7f0b2effd700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0b10066f30 0x7f0b100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:03.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.066+0000 7f0b2effd700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0b10066f30 0x7f0b100693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0b28006fd0 tx=0x7f0b28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:03.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.070+0000 7f0b26ffd700 1 -- 172.21.15.3:0/454705871 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b18053150 con 0x7f0b3015c8f0 2023-10-06T04:54:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.308+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b30071680 con 0x7f0b3015c8f0 2023-10-06T04:54:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.308+0000 7f0b26ffd700 1 -- 172.21.15.3:0/454705871 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0b18056770 con 0x7f0b3015c8f0 2023-10-06T04:54:03.311 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:54:03.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.310+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0b10066f30 msgr2=0x7f0b100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.310+0000 7f0b35724700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0b10066f30 0x7f0b100693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0b28006fd0 tx=0x7f0b28008040 comp rx=0 tx=0).stop 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b3015c8f0 msgr2=0x7f0b3016d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b3015c8f0 0x7f0b3016d530 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0b180093f0 tx=0x7f0b18017000 comp rx=0 tx=0).stop 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 shutdown_connections 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0b10066f30 0x7f0b100693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b3015ccc0 0x7f0b30158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b300d06a0 0x7f0b3014e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 --2- 172.21.15.3:0/454705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b3015c8f0 0x7f0b3016d530 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 >> 172.21.15.3:0/454705871 conn(0x7f0b30070310 msgr2=0x7f0b301567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 shutdown_connections 2023-10-06T04:54:03.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:03.311+0000 7f0b35724700 1 -- 172.21.15.3:0/454705871 wait complete. 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:54:03.325 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:54:03.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:03.326 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:54:03.344 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:54:03.344 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:54:03.344 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:03.345 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42389sec preferred_lft 42389sec 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T04:54:03.346 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:03.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:03.347 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:54:03.412 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:54:03.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:03.414 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:54:03.472 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:54:03.472 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:54:03.472 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:54:03.472 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:54:03.472 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:54:03.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:03.473 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:54:03.473 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:54:03.473 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:54:03.474 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:54:03.474 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:54:03.474 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:54:03.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:03.576 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:54:03.591 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T04:54:03.592 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:54:03.592 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:54:03.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:03.592 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T04:54:03.657 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:54:03.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:03.657 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:03.712 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:54:03.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:03.712 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:03.766 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:54:03.831 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:03.831 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:03.895 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:03.896 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:03.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:03.897 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:54:03.951 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:54:03.952 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:54:03.994 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:54:04.019 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:04.019 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:04.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T04:54:04.073+0000 7fc7c01ce540 -1 init, newargv = 0x561f2e6b5b70 newargc=15 2023-10-06T04:54:04.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[98725]: starting ceph client 2023-10-06T04:54:04.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[98725]: starting fuse 2023-10-06T04:54:04.084 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:04.085 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:04.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:04.085 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:54:04.140 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:54:04.141 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:54:04.141 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:04.141 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:54:04.141 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:54:04.141 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:54:04.141 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:54:04.141 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:04.141 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:04.142 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T04:54:04.143 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:54:04.261 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.98725.asok 2023-10-06T04:54:04.443 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.98725.asok 2023-10-06T04:54:04.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:04.443 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.98725.asok status 2023-10-06T04:54:04.594 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "98725", 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:54:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 5422, 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 5422 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:54:04.596 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2349374817 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2349374817 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:04.597 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.5422 192.168.0.1:0/2349374817", 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/2349374817", 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 23, 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 44, 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:54:04.598 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:54:04.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:04.605 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:54:04.605 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:04.620 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:54:04.620 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:04.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:04.620 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:04.684 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:04.684 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:04.685 DEBUG:teuthology.orchestra.run.smithi003:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2023-10-06T04:54:04.685 DEBUG:teuthology.orchestra.run.smithi003:> for n in range(0, 1000): 2023-10-06T04:54:04.685 DEBUG:teuthology.orchestra.run.smithi003:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2023-10-06T04:54:04.685 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:06.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:06.149 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2023-10-06T04:54:06.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:06.582 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:06.582 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:06.582 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:06.583 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:06.584 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:06.675 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:06.854 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:06.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:06.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106632.asok status 2023-10-06T04:54:07.011 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:07.011 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "106632", 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 1001, 2023-10-06T04:54:07.012 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 1001, 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5550, 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5550 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:07.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5550 192.168.0.1:0/2158008368", 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2158008368", 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1002, 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 44, 2023-10-06T04:54:07.014 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 44, 2023-10-06T04:54:07.015 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:07.015 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:07.015 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:07.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:07.022 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:07.022 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:07.022 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:07.022 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:07.022 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:07.023 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:07.024 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:07.107 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:07.289 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:07.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:07.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106632.asok config set client_cache_size 100 2023-10-06T04:54:07.442 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:07.442 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "client_cache_size = '100' " 2023-10-06T04:54:07.442 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:07.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:07.452 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:07.452 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:07.452 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:07.452 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:07.453 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:07.454 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:07.537 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:07.719 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:07.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:07.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106632.asok status 2023-10-06T04:54:07.874 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:07.874 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:07.874 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:07.874 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "106632", 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 1001, 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 1001, 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5550, 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:07.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5550 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:07.876 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5550 192.168.0.1:0/2158008368", 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2158008368", 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1002, 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 44, 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 44, 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:07.877 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:07.884 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 1001 2023-10-06T04:54:07.884 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-10-06T04:54:12.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:12.885 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:12.886 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:12.887 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:12.888 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:12.984 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:13.165 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106632.asok 2023-10-06T04:54:13.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:13.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106632.asok status 2023-10-06T04:54:13.328 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "106632", 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 100, 2023-10-06T04:54:13.329 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 1, 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5550, 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5550 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:13.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2158008368 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5550 192.168.0.1:0/2158008368", 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2158008368", 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 101, 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 44, 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 44, 2023-10-06T04:54:13.331 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:13.332 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:13.332 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:13.338 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2023-10-06T04:54:13.338 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T04:54:13.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:13.339 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-10-06T04:54:13.359 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:54:13.359 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:54:13.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:13.436 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[106632]: fuse finished with error 0 and tester_r 0 2023-10-06T04:54:13.437 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:54:13.480 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:54:13.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:13.480 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:54:13.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:54:13.582 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:54:13.625 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-06T04:54:13.625 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-06T04:54:13.625 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:54:13.625 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-06T04:54:13.625 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.625 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:54:13.626 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:54:13.627 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:54:13.628 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.629 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.630 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.631 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-06T04:54:13.632 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-06T04:54:13.633 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:13.634 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:54:13.635 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:54:13.636 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:13.637 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:54:13.638 INFO:teuthology.orchestra.run.smithi003.stdout: client mountpoint = /subdir 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-06T04:54:13.639 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:54:13.640 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-10-06T04:54:13.641 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.642 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:54:13.643 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:54:13.643 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.643 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-10-06T04:54:13.643 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.644 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-10-06T04:54:13.644 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-10-06T04:54:13.644 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:54:13.644 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:54:13.644 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-10-06T04:54:13.645 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-10-06T04:54:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.647 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-10-06T04:54:13.648 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:54:13.649 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:54:13.650 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:13.651 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-10-06T04:54:13.652 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-10-06T04:54:13.653 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout: client mountpoint = /subdir 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-10-06T04:54:13.654 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-10-06T04:54:13.655 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-10-06T04:54:13.656 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-10-06T04:54:13.656 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:54:13.656 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:54:13.656 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:54:13.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:54:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.814+0000 7fa8dcd7a700 1 Processor -- start 2023-10-06T04:54:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.814+0000 7fa8dcd7a700 1 -- start start 2023-10-06T04:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8dcd7a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa8d806eb60 0x7fa8d806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8dcd7a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8d8155e00 0x7fa8d8156240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8dcd7a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d80aad40 con 0x7fa8d806e790 2023-10-06T04:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8dcd7a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d80aa130 con 0x7fa8d8155e00 2023-10-06T04:54:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8dcd7a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d80aa410 con 0x7fa8d806eb60 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8d6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8d8155e00 0x7fa8d8156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8d6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8d8155e00 0x7fa8d8156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52212/0 (socket says 172.21.15.3:52212) 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.815+0000 7fa8d6d9d700 1 -- 172.21.15.3:0/3065681357 learned_addr learned my addr 172.21.15.3:0/3065681357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d5d9b700 1 --2- 172.21.15.3:0/3065681357 >> v2:172.21.15.153:3300/0 conn(0x7fa8d806eb60 0x7fa8d806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d6d9d700 1 -- 172.21.15.3:0/3065681357 >> v2:172.21.15.153:3300/0 conn(0x7fa8d806eb60 msgr2=0x7fa8d806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d6d9d700 1 --2- 172.21.15.3:0/3065681357 >> v2:172.21.15.153:3300/0 conn(0x7fa8d806eb60 0x7fa8d806df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d6d9d700 1 -- 172.21.15.3:0/3065681357 >> v1:172.21.15.153:6789/0 conn(0x7fa8d806e790 legacy=0x7fa8d80a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d6d9d700 1 -- 172.21.15.3:0/3065681357 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d80aa6f0 con 0x7fa8d8155e00 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d6d9d700 1 --2- 172.21.15.3:0/3065681357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8155e00 0x7fa8d8156240 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa8d806fbd0 tx=0x7fa8c800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c5d20719aab23f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d559a700 1 -- 172.21.15.3:0/3065681357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8010070 con 0x7fa8d8155e00 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d559a700 1 -- 172.21.15.3:0/3065681357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c800de50 con 0x7fa8d8155e00 2023-10-06T04:54:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8d559a700 1 -- 172.21.15.3:0/3065681357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800b320 con 0x7fa8d8155e00 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3065681357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8155e00 msgr2=0x7fa8d8156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.816+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3065681357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8155e00 0x7fa8d8156240 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa8d806fbd0 tx=0x7fa8c800ba50 comp rx=0 tx=0).stop 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.817+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3065681357 shutdown_connections 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.817+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3065681357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8155e00 0x7fa8d8156240 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.817+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3065681357 >> v2:172.21.15.153:3300/0 conn(0x7fa8d806eb60 0x7fa8d806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.817+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3065681357 >> 172.21.15.3:0/3065681357 conn(0x7fa8d8068e70 msgr2=0x7fa8d8069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.817+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3065681357 shutdown_connections 2023-10-06T04:54:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.817+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3065681357 wait complete. 2023-10-06T04:54:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.818+0000 7fa8dcd7a700 1 Processor -- start 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.818+0000 7fa8dcd7a700 1 -- start start 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.818+0000 7fa8dcd7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d806e790 0x7fa8d80ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.818+0000 7fa8dcd7a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8d806eb60 0x7fa8d814d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8dcd7a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8d8155e00 0x7fa8d8153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8dcd7a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80adc30 con 0x7fa8d806e790 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8dcd7a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81571d0 con 0x7fa8d8155e00 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8dcd7a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81565c0 con 0x7fa8d806eb60 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d806e790 0x7fa8d80ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d806e790 0x7fa8d80ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52224/0 (socket says 172.21.15.3:52224) 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 -- 172.21.15.3:0/3180144964 learned_addr learned my addr 172.21.15.3:0/3180144964 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 -- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8d806eb60 msgr2=0x7fa8d814d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8d806eb60 0x7fa8d814d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d6d9d700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8d8155e00 0x7fa8d8153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 -- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8d8155e00 msgr2=0x7fa8d8153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8d8155e00 0x7fa8d8153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 -- 172.21.15.3:0/3180144964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c800bcf0 con 0x7fa8d806e790 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8d659c700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d806e790 0x7fa8d80ab9d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000c5d0 tx=0x7fa8d0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8ce7fc700 1 -- 172.21.15.3:0/3180144964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0011590 con 0x7fa8d806e790 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8ce7fc700 1 -- 172.21.15.3:0/3180144964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d0011710 con 0x7fa8d806e790 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d81568a0 con 0x7fa8d806e790 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8ce7fc700 1 -- 172.21.15.3:0/3180144964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0011ad0 con 0x7fa8d806e790 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.819+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d8156e60 con 0x7fa8d806e790 2023-10-06T04:54:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.820+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d806c6f0 con 0x7fa8d806e790 2023-10-06T04:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.821+0000 7fa8ce7fc700 1 -- 172.21.15.3:0/3180144964 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8d001c460 con 0x7fa8d806e790 2023-10-06T04:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.821+0000 7fa8ce7fc700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8b8066f30 0x7fa8b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.821+0000 7fa8ce7fc700 1 -- 172.21.15.3:0/3180144964 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa8d0084510 con 0x7fa8d806e790 2023-10-06T04:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.821+0000 7fa8d5d9b700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8b8066f30 0x7fa8b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.822+0000 7fa8d5d9b700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8b8066f30 0x7fa8b80693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa8d81549f0 tx=0x7fa8c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:13.825+0000 7fa8ce7fc700 1 -- 172.21.15.3:0/3180144964 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8d00515e0 con 0x7fa8d806e790 2023-10-06T04:54:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.063+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa8d80aa9d0 con 0x7fa8d806e790 2023-10-06T04:54:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.064+0000 7fa8ce7fc700 1 -- 172.21.15.3:0/3180144964 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa8d0054c00 con 0x7fa8d806e790 2023-10-06T04:54:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8b8066f30 msgr2=0x7fa8b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8b8066f30 0x7fa8b80693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa8d81549f0 tx=0x7fa8c0008040 comp rx=0 tx=0).stop 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d806e790 msgr2=0x7fa8d80ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d806e790 0x7fa8d80ab9d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000c5d0 tx=0x7fa8d0007f60 comp rx=0 tx=0).stop 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 shutdown_connections 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8b8066f30 0x7fa8b80693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8d8155e00 0x7fa8d8153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d806e790 0x7fa8d80ab9d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 --2- 172.21.15.3:0/3180144964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8d806eb60 0x7fa8d814d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 >> 172.21.15.3:0/3180144964 conn(0x7fa8d8068e70 msgr2=0x7fa8d80c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 shutdown_connections 2023-10-06T04:54:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:14.066+0000 7fa8dcd7a700 1 -- 172.21.15.3:0/3180144964 wait complete. 2023-10-06T04:54:14.080 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:54:14.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:54:14.081 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:54:14.081 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:54:14.081 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:14.081 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:54:14.081 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:54:14.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:54:14.081 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:14.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:14.081 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:54:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:54:14.097 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42376sec preferred_lft 42376sec 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:14.098 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:54:14.099 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:54:14.100 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:14.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:14.100 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:54:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:54:14.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:14.162 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:54:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:54:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:54:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:54:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:54:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:54:14.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:14.220 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:54:14.220 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:54:14.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:54:14.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:54:14.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:54:14.220 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:54:14.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:14.322 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:54:14.337 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:54:14.337 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:54:14.337 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:14.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:14.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:54:14.402 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:54:14.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:14.403 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:14.457 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:14.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:14.458 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:14.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:54:14.578 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:14.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:14.643 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:14.643 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:14.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:14.644 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:54:14.700 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:54:14.700 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:54:14.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:54:14.768 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:14.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:14.826 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:54:14.824+0000 7f8375a5e540 -1 init, newargv = 0x563d4c27fa40 newargc=15 2023-10-06T04:54:14.827 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[107347]: starting ceph client 2023-10-06T04:54:14.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[107347]: starting fuse 2023-10-06T04:54:14.835 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:14.837 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:14.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:14.837 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:54:14.892 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:54:14.892 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:14.893 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:14.894 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:14.895 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:14.895 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:14.895 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:15.015 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:15.196 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:15.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:15.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107347.asok status 2023-10-06T04:54:15.351 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:15.351 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:15.351 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "107347", 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/subdir" 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5571, 2023-10-06T04:54:15.352 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5571 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:15.353 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5571 192.168.0.1:0/231427151", 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/231427151", 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 2, 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 46, 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:15.354 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:15.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:15.361 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:54:15.361 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:15.377 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:54:15.377 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:15.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:15.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:15.443 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:15.443 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:15.443 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2023-10-06T04:54:15.443 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:15.443 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/.").st_ino) 2023-10-06T04:54:15.444 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:15.550 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:54:15.738 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:15.738 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:15.738 DEBUG:teuthology.orchestra.run.smithi003:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2023-10-06T04:54:15.738 DEBUG:teuthology.orchestra.run.smithi003:> for n in range(0, 1000): 2023-10-06T04:54:15.738 DEBUG:teuthology.orchestra.run.smithi003:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2023-10-06T04:54:15.739 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:17.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:17.234 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:17.628 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:17.629 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:17.630 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:17.630 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:17.630 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:17.630 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:17.630 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:17.722 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:17.904 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:17.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:17.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107347.asok status 2023-10-06T04:54:18.062 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:18.062 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "107347", 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/subdir" 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 1001, 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 1001, 2023-10-06T04:54:18.063 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5571, 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5571 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:18.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5571 192.168.0.1:0/231427151", 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/231427151", 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1003, 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 46, 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 44, 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:18.065 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:18.066 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:18.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.073 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:18.074 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.075 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:18.075 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:18.159 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:18.341 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:18.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:18.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107347.asok config set client_cache_size 100 2023-10-06T04:54:18.494 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:18.494 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "client_cache_size = '100' " 2023-10-06T04:54:18.494 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:18.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:18.504 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:18.504 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:18.505 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:18.506 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:18.507 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:18.589 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:18.771 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:18.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:18.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107347.asok status 2023-10-06T04:54:18.931 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "107347", 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/subdir" 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 1001, 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 135, 2023-10-06T04:54:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5571, 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5571 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5571 192.168.0.1:0/231427151", 2023-10-06T04:54:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/231427151", 2023-10-06T04:54:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1003, 2023-10-06T04:54:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 46, 2023-10-06T04:54:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 44, 2023-10-06T04:54:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:18.935 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:18.941 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 135 2023-10-06T04:54:18.941 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-10-06T04:54:23.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:23.943 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:23.944 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:24.040 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:24.220 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107347.asok 2023-10-06T04:54:24.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:24.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107347.asok status 2023-10-06T04:54:24.381 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "107347", 2023-10-06T04:54:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/subdir" 2023-10-06T04:54:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 100, 2023-10-06T04:54:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 1, 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5571, 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5571 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 231427151 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5571 192.168.0.1:0/231427151", 2023-10-06T04:54:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/231427151", 2023-10-06T04:54:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 102, 2023-10-06T04:54:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2023-10-06T04:54:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 46, 2023-10-06T04:54:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 44, 2023-10-06T04:54:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:24.385 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:24.391 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2023-10-06T04:54:24.391 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T04:54:24.391 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:54:24.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:54:24.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:54:24.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:24.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:24.484 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:54:24.484 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:54:24.484 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:54:24.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:24.553 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[107347]: fuse finished with error 0 and tester_r 0 2023-10-06T04:54:24.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:54:24.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:24.555 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:54:24.721 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:54:24.722 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:54:24.722 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:24.759 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[98725]: fuse finished with error 0 and tester_r 0 2023-10-06T04:54:24.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:54:24.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:24.761 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:54:24.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:54:24.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:24.896 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:54:24.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:54:24.912 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T04:54:24.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:54:24.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:24.912 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:54:24.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:54:24.928 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T04:54:24.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.071+0000 7f096d79b700 1 Processor -- start 2023-10-06T04:54:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.071+0000 7f096d79b700 1 -- start start 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.071+0000 7f096d79b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f096806e790 0x7f096806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.071+0000 7f096d79b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f096806eb60 0x7f09680a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.071+0000 7f096d79b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09680aad40 con 0x7f0968155e00 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.071+0000 7f096d79b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680aa130 con 0x7f096806eb60 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.071+0000 7f096d79b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680aa410 con 0x7f096806e790 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f096806eb60 0x7f09680a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f096806eb60 0x7f09680a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54478/0 (socket says 172.21.15.3:54478) 2023-10-06T04:54:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 -- 172.21.15.3:0/3841179544 learned_addr learned my addr 172.21.15.3:0/3841179544 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f0966ffd700 1 --2- 172.21.15.3:0/3841179544 >> v2:172.21.15.153:3300/0 conn(0x7f096806e790 0x7f096806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 -- 172.21.15.3:0/3841179544 >> v2:172.21.15.153:3300/0 conn(0x7f096806e790 msgr2=0x7f096806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 --2- 172.21.15.3:0/3841179544 >> v2:172.21.15.153:3300/0 conn(0x7f096806e790 0x7f096806df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 -- 172.21.15.3:0/3841179544 >> v1:172.21.15.153:6789/0 conn(0x7f0968155e00 legacy=0x7f0968156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 -- 172.21.15.3:0/3841179544 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680aa6f0 con 0x7f096806eb60 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f09667fc700 1 --2- 172.21.15.3:0/3841179544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806eb60 0x7f09680a98c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f096806e510 tx=0x7f0950009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8ba15f7df2f140 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f0965ffb700 1 -- 172.21.15.3:0/3841179544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000b860 con 0x7f096806eb60 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f0965ffb700 1 -- 172.21.15.3:0/3841179544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095000b9c0 con 0x7f096806eb60 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.072+0000 7f0965ffb700 1 -- 172.21.15.3:0/3841179544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000bd60 con 0x7f096806eb60 2023-10-06T04:54:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 -- 172.21.15.3:0/3841179544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806eb60 msgr2=0x7f09680a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 --2- 172.21.15.3:0/3841179544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806eb60 0x7f09680a98c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f096806e510 tx=0x7f0950009250 comp rx=0 tx=0).stop 2023-10-06T04:54:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 -- 172.21.15.3:0/3841179544 shutdown_connections 2023-10-06T04:54:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 --2- 172.21.15.3:0/3841179544 >> v2:172.21.15.153:3300/0 conn(0x7f096806e790 0x7f096806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 --2- 172.21.15.3:0/3841179544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806eb60 0x7f09680a98c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 -- 172.21.15.3:0/3841179544 >> 172.21.15.3:0/3841179544 conn(0x7f0968068e70 msgr2=0x7f0968069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 -- 172.21.15.3:0/3841179544 shutdown_connections 2023-10-06T04:54:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.073+0000 7f096d79b700 1 -- 172.21.15.3:0/3841179544 wait complete. 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.074+0000 7f096d79b700 1 Processor -- start 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.074+0000 7f096d79b700 1 -- start start 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f096d79b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806e790 0x7f09680aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f096d79b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f096806eb60 0x7f0968149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f096d79b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968155e00 0x7f09680ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f096d79b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c32d0 con 0x7f096806e790 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f096d79b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680ad610 con 0x7f0968155e00 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f096d79b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680ab4f0 con 0x7f096806eb60 2023-10-06T04:54:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806e790 0x7f09680aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806e790 0x7f09680aafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54484/0 (socket says 172.21.15.3:54484) 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 -- 172.21.15.3:0/4097931890 learned_addr learned my addr 172.21.15.3:0/4097931890 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 -- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f096806eb60 msgr2=0x7f0968149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f096806eb60 0x7f0968149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 -- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968155e00 msgr2=0x7f09680ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968155e00 0x7f09680ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 -- 172.21.15.3:0/4097931890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500094f0 con 0x7f096806e790 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f0966ffd700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806e790 0x7f09680aafb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f096000d890 tx=0x7f0960008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f095effd700 1 -- 172.21.15.3:0/4097931890 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600125f0 con 0x7f096806e790 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f095effd700 1 -- 172.21.15.3:0/4097931890 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0960012770 con 0x7f096806e790 2023-10-06T04:54:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09680ad8b0 con 0x7f096806e790 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.075+0000 7f095effd700 1 -- 172.21.15.3:0/4097931890 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960012b30 con 0x7f096806e790 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.076+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09680ade70 con 0x7f096806e790 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.076+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09680c3440 con 0x7f096806e790 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.077+0000 7f095effd700 1 -- 172.21.15.3:0/4097931890 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f096001e460 con 0x7f096806e790 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.077+0000 7f095effd700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0948066f30 0x7f09480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.077+0000 7f095effd700 1 -- 172.21.15.3:0/4097931890 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f0960084dd0 con 0x7f096806e790 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.077+0000 7f09667fc700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0948066f30 0x7f09480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.078+0000 7f09667fc700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0948066f30 0x7f09480693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f096806e510 tx=0x7f0950017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.082+0000 7f095effd700 1 -- 172.21.15.3:0/4097931890 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09680c3440 con 0x7f096806e790 2023-10-06T04:54:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.323+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09680c3410 con 0x7f096806e790 2023-10-06T04:54:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.323+0000 7f095effd700 1 -- 172.21.15.3:0/4097931890 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f0960086090 con 0x7f096806e790 2023-10-06T04:54:25.325 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:25.325 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/596033972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":596033972},{"type":"v1","addr":"172.21.15.153:6837","nonce":596033972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1359588707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1359588707},{"type":"v1","addr":"172.21.15.3:6834","nonce":1359588707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/616658407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":616658407},{"type":"v1","addr":"172.21.15.3:6835","nonce":616658407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:49.997154+0000","modified":"2023-10-06T04:53:53.627991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/4017437319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":4017437319},{"type":"v1","addr":"172.21.15.153:6836","nonce":4017437319}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0948066f30 msgr2=0x7f09480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0948066f30 0x7f09480693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f096806e510 tx=0x7f0950017000 comp rx=0 tx=0).stop 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806e790 msgr2=0x7f09680aafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806e790 0x7f09680aafb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f096000d890 tx=0x7f0960008040 comp rx=0 tx=0).stop 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 shutdown_connections 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0948066f30 0x7f09480693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968155e00 0x7f09680ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096806e790 0x7f09680aafb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 --2- 172.21.15.3:0/4097931890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f096806eb60 0x7f0968149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 >> 172.21.15.3:0/4097931890 conn(0x7f0968068e70 msgr2=0x7f096806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:25.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 shutdown_connections 2023-10-06T04:54:25.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.325+0000 7f096d79b700 1 -- 172.21.15.3:0/4097931890 wait complete. 2023-10-06T04:54:25.328 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2023-10-06T04:54:25.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:54:25.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.456+0000 7f6b25f0e700 1 Processor -- start 2023-10-06T04:54:25.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.457+0000 7f6b25f0e700 1 -- start start 2023-10-06T04:54:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.457+0000 7f6b25f0e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b20140100 0x7f6b200c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.457+0000 7f6b25f0e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b2006a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.457+0000 7f6b25f0e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b200c9910 con 0x7f6b2013fd30 2023-10-06T04:54:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.457+0000 7f6b25f0e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b200c8d00 con 0x7f6b20140100 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.457+0000 7f6b25f0e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200c8fe0 con 0x7f6b200ca2e0 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.457+0000 7f6b1effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b20140100 0x7f6b200c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b20140100 0x7f6b200c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54486/0 (socket says 172.21.15.3:54486) 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1effd700 1 -- 172.21.15.3:0/4192277805 learned_addr learned my addr 172.21.15.3:0/4192277805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1ffff700 1 --2- 172.21.15.3:0/4192277805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b2006a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1effd700 1 -- 172.21.15.3:0/4192277805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 msgr2=0x7f6b2006a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1effd700 1 --2- 172.21.15.3:0/4192277805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b2006a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1effd700 1 -- 172.21.15.3:0/4192277805 >> v1:172.21.15.153:6789/0 conn(0x7f6b2013fd30 legacy=0x7f6b200cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1effd700 1 -- 172.21.15.3:0/4192277805 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200c92c0 con 0x7f6b20140100 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1effd700 1 --2- 172.21.15.3:0/4192277805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b20140100 0x7f6b200c9da0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6b140099b0 tx=0x7f6b14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a6496ceda45a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1e7fc700 1 -- 172.21.15.3:0/4192277805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400b8b0 con 0x7f6b20140100 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1e7fc700 1 -- 172.21.15.3:0/4192277805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1400ba10 con 0x7f6b20140100 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.458+0000 7f6b1e7fc700 1 -- 172.21.15.3:0/4192277805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400bdb0 con 0x7f6b20140100 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 -- 172.21.15.3:0/4192277805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b20140100 msgr2=0x7f6b200c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/4192277805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b20140100 0x7f6b200c9da0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6b140099b0 tx=0x7f6b14009250 comp rx=0 tx=0).stop 2023-10-06T04:54:25.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 -- 172.21.15.3:0/4192277805 shutdown_connections 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/4192277805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b2006a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/4192277805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b20140100 0x7f6b200c9da0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 -- 172.21.15.3:0/4192277805 >> 172.21.15.3:0/4192277805 conn(0x7f6b2006c4c0 msgr2=0x7f6b2006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 -- 172.21.15.3:0/4192277805 shutdown_connections 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.459+0000 7f6b25f0e700 1 -- 172.21.15.3:0/4192277805 wait complete. 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.460+0000 7f6b25f0e700 1 Processor -- start 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.460+0000 7f6b25f0e700 1 -- start start 2023-10-06T04:54:25.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b25f0e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b20158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b25f0e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b2013fd30 0x7f6b200b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b25f0e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b200b23b0 0x7f6b200b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b25f0e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200af3a0 con 0x7f6b200b23b0 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b25f0e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200afc40 con 0x7f6b2013fd30 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b25f0e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200b2830 con 0x7f6b200ca2e0 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b200b23b0 0x7f6b200b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b200b23b0 0x7f6b200b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54498/0 (socket says 172.21.15.3:54498) 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 -- 172.21.15.3:0/1896748897 learned_addr learned my addr 172.21.15.3:0/1896748897 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:25.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1f7fe700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b20158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1effd700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b2013fd30 0x7f6b200b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 -- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 msgr2=0x7f6b20158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b20158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 -- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b2013fd30 msgr2=0x7f6b200b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b2013fd30 0x7f6b200b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 -- 172.21.15.3:0/1896748897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b140094f0 con 0x7f6b200b23b0 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1effd700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b2013fd30 0x7f6b200b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:25.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b1ffff700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b200b23b0 0x7f6b200b00d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009fd0 tx=0x7f6b18009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b077fe700 1 -- 172.21.15.3:0/1896748897 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800eca0 con 0x7f6b200b23b0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b077fe700 1 -- 172.21.15.3:0/1896748897 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1800ee00 con 0x7f6b200b23b0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b200af5f0 con 0x7f6b200b23b0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.461+0000 7f6b077fe700 1 -- 172.21.15.3:0/1896748897 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18014670 con 0x7f6b200b23b0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.462+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b200aef90 con 0x7f6b200b23b0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.463+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2006e8c0 con 0x7f6b200b23b0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.463+0000 7f6b077fe700 1 -- 172.21.15.3:0/1896748897 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b180149e0 con 0x7f6b200b23b0 2023-10-06T04:54:25.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.463+0000 7f6b077fe700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b0c066f30 0x7f6b0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.463+0000 7f6b077fe700 1 -- 172.21.15.3:0/1896748897 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6b18084e00 con 0x7f6b200b23b0 2023-10-06T04:54:25.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.464+0000 7f6b1f7fe700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b0c066f30 0x7f6b0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.464+0000 7f6b1f7fe700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b0c066f30 0x7f6b0c0693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6b10003a30 tx=0x7f6b10007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.468+0000 7f6b077fe700 1 -- 172.21.15.3:0/1896748897 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b18051ed0 con 0x7f6b200b23b0 2023-10-06T04:54:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.664+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b200afd80 con 0x7f6b200b23b0 2023-10-06T04:54:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.665+0000 7f6b077fe700 1 -- 172.21.15.3:0/1896748897 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f6b180554f0 con 0x7f6b200b23b0 2023-10-06T04:54:25.670 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:25.670 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":46,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:54:06.663887+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:47.743195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:48.974063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.25","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:54:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b0c066f30 msgr2=0x7f6b0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b0c066f30 0x7f6b0c0693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6b10003a30 tx=0x7f6b10007ea0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b200b23b0 msgr2=0x7f6b200b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b200b23b0 0x7f6b200b00d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009fd0 tx=0x7f6b18009900 comp rx=0 tx=0).stop 2023-10-06T04:54:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 shutdown_connections 2023-10-06T04:54:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b0c066f30 0x7f6b0c0693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b200b23b0 0x7f6b200b00d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b200ca2e0 0x7f6b20158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 --2- 172.21.15.3:0/1896748897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b2013fd30 0x7f6b200b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 >> 172.21.15.3:0/1896748897 conn(0x7f6b2006c4c0 msgr2=0x7f6b200c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 shutdown_connections 2023-10-06T04:54:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.667+0000 7f6b25f0e700 1 -- 172.21.15.3:0/1896748897 wait complete. 2023-10-06T04:54:25.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:53:47.743195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:53:48.974063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:54:25.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:54:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.803+0000 7fb9f1da7700 1 Processor -- start 2023-10-06T04:54:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.803+0000 7fb9f1da7700 1 -- start start 2023-10-06T04:54:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9f1da7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb9ec159720 0x7fb9ec15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9f1da7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 0x7fb9ec06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9f1da7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec06dd50 con 0x7fb9ec159af0 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9f1da7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec0ae160 con 0x7fb9ec159720 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9f1da7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0cb650 con 0x7fb9ec06d3b0 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9eaffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb9ec159af0 0x7fb9ec0caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57478/0 (socket says 172.21.15.3:57478) 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9eaffd700 1 -- 172.21.15.3:0/1872084633 learned_addr learned my addr 172.21.15.3:0/1872084633 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266860766 0 0) 0x7fb9e0001110 con 0x7fb9ec159af0 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9eb7fe700 1 --2- 172.21.15.3:0/1872084633 >> v2:172.21.15.153:3300/0 conn(0x7fb9ec159720 0x7fb9ec15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ebfff700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 0x7fb9ec06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d4003570 con 0x7fb9ec159af0 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3133838536 0 0) 0x7fb9d4003570 con 0x7fb9ec159af0 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 msgr2=0x7fb9ec06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 0x7fb9ec06d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 >> v2:172.21.15.153:3300/0 conn(0x7fb9ec159720 msgr2=0x7fb9ec15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 --2- 172.21.15.3:0/1872084633 >> v2:172.21.15.153:3300/0 conn(0x7fb9ec159720 0x7fb9ec15de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0af920 con 0x7fb9ec159af0 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fb9e0001600 con 0x7fb9ec159af0 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 >> v1:172.21.15.3:6789/0 conn(0x7fb9ec159af0 legacy=0x7fb9ec0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.804+0000 7fb9ea7fc700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 0x7fb9d4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9d4004d50 0x7fb9d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0ae160 con 0x7fb9d4003fe0 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0cb650 con 0x7fb9d4004d50 2023-10-06T04:54:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d400ba90 con 0x7fb9ec06d3b0 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eaffd700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eb7fe700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 0x7fb9d4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ebfff700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9d4004d50 0x7fb9d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eaffd700 1 -- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 msgr2=0x7fb9d4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eaffd700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 0x7fb9d4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eaffd700 1 -- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9d4004d50 msgr2=0x7fb9d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eaffd700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9d4004d50 0x7fb9d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eaffd700 1 -- 172.21.15.3:0/1872084633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4003570 con 0x7fb9d4003fe0 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eb7fe700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec06d3b0 0x7fb9d4003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9eaffd700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9d4004810 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000c5f0 tx=0x7fb9e0007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7712a4ce75bb1193 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e000cd90 con 0x7fb9d4003fe0 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e0005920 con 0x7fb9d4003fe0 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.805+0000 7fb9ea7fc700 1 -- 172.21.15.3:0/1872084633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0005ce0 con 0x7fb9d4003fe0 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.806+0000 7fb9f1da7700 1 -- 172.21.15.3:0/1872084633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 msgr2=0x7fb9d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.806+0000 7fb9f1da7700 1 --2- 172.21.15.3:0/1872084633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9d4004810 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000c5f0 tx=0x7fb9e0007a80 comp rx=0 tx=0).stop 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.806+0000 7fb9ebfff700 1 -- 172.21.15.3:0/1872084633 reap_dead start 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.806+0000 7fb9f1da7700 1 -- 172.21.15.3:0/1872084633 shutdown_connections 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.806+0000 7fb9f1da7700 1 -- 172.21.15.3:0/1872084633 >> 172.21.15.3:0/1872084633 conn(0x7fb9ec068e70 msgr2=0x7fb9ec069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.806+0000 7fb9f1da7700 1 -- 172.21.15.3:0/1872084633 shutdown_connections 2023-10-06T04:54:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.806+0000 7fb9f1da7700 1 -- 172.21.15.3:0/1872084633 wait complete. 2023-10-06T04:54:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.807+0000 7fb9f1da7700 1 Processor -- start 2023-10-06T04:54:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.807+0000 7fb9f1da7700 1 -- start start 2023-10-06T04:54:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9f1da7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9ec158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9f1da7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec16b670 0x7fb9ec0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9f1da7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9ec16bac0 0x7fb9ec16cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9f1da7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0b2c70 con 0x7fb9d4003fe0 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9f1da7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0b0bc0 con 0x7fb9ec16bac0 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9f1da7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0affb0 con 0x7fb9ec16b670 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9ec158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9ec158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54510/0 (socket says 172.21.15.3:54510) 2023-10-06T04:54:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 -- 172.21.15.3:0/3256248625 learned_addr learned my addr 172.21.15.3:0/3256248625 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eaffd700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec16b670 0x7fb9ec0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 -- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec16b670 msgr2=0x7fb9ec0b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9ebfff700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9ec16bac0 0x7fb9ec16cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec16b670 0x7fb9ec0b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 -- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9ec16bac0 msgr2=0x7fb9ec16cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9ec16bac0 0x7fb9ec16cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 -- 172.21.15.3:0/3256248625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0b0290 con 0x7fb9d4003fe0 2023-10-06T04:54:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9eb7fe700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9ec158420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0118e0 tx=0x7fb9dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.808+0000 7fb9d37fe700 1 -- 172.21.15.3:0/3256248625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc011e80 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.809+0000 7fb9d37fe700 1 -- 172.21.15.3:0/3256248625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc00f860 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.809+0000 7fb9d37fe700 1 -- 172.21.15.3:0/3256248625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00fc00 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.809+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec0b0570 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.809+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec158960 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.810+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec06c410 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.810+0000 7fb9d37fe700 1 -- 172.21.15.3:0/3256248625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9dc015460 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.811+0000 7fb9d37fe700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb9d8066f30 0x7fb9d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.811+0000 7fb9d37fe700 1 -- 172.21.15.3:0/3256248625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fb9dc084ee0 con 0x7fb9d4003fe0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.811+0000 7fb9eaffd700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb9d8066f30 0x7fb9d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.811+0000 7fb9eaffd700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb9d8066f30 0x7fb9d80693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0008fd0 tx=0x7fb9e0015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:25.815+0000 7fb9d37fe700 1 -- 172.21.15.3:0/3256248625 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9dc055a40 con 0x7fb9d4003fe0 2023-10-06T04:54:26.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.009+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb9ec04bb40 con 0x7fb9d4003fe0 2023-10-06T04:54:26.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.009+0000 7fb9d37fe700 1 -- 172.21.15.3:0/3256248625 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v46) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb9dc0555d0 con 0x7fb9d4003fe0 2023-10-06T04:54:26.011 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:54:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.011+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb9d8066f30 msgr2=0x7fb9d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.011+0000 7fb9f1da7700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb9d8066f30 0x7fb9d80693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0008fd0 tx=0x7fb9e0015000 comp rx=0 tx=0).stop 2023-10-06T04:54:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 msgr2=0x7fb9ec158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9ec158420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0118e0 tx=0x7fb9dc008040 comp rx=0 tx=0).stop 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 shutdown_connections 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb9d8066f30 0x7fb9d80693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9ec16bac0 0x7fb9ec16cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9d4003fe0 0x7fb9ec158420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 --2- 172.21.15.3:0/3256248625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9ec16b670 0x7fb9ec0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 >> 172.21.15.3:0/3256248625 conn(0x7fb9ec068e70 msgr2=0x7fb9ec0d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 shutdown_connections 2023-10-06T04:54:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.012+0000 7fb9f1da7700 1 -- 172.21.15.3:0/3256248625 wait complete. 2023-10-06T04:54:26.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:54:26.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.146+0000 7fa462cde700 1 Processor -- start 2023-10-06T04:54:26.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.146+0000 7fa462cde700 1 -- start start 2023-10-06T04:54:26.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.146+0000 7fa462cde700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa462cde700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa45c0cae70 0x7fa45c0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa462cde700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa45c0c9170 0x7fa45c0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa462cde700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45c0c9b30 con 0x7fa45c0c9170 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa462cde700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45c15d640 con 0x7fa45c0cae70 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa462cde700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0cb9a0 con 0x7fa45c159790 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa45c0c9170 0x7fa45c0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa45c0c9170 0x7fa45c0c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54516/0 (socket says 172.21.15.3:54516) 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 -- 172.21.15.3:0/2490837406 learned_addr learned my addr 172.21.15.3:0/2490837406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa45bfff700 1 --2- 172.21.15.3:0/2490837406 >> v2:172.21.15.153:3300/0 conn(0x7fa45c0cae70 0x7fa45c0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 -- 172.21.15.3:0/2490837406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 msgr2=0x7fa45c159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa460a7a700 1 --2- 172.21.15.3:0/2490837406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 --2- 172.21.15.3:0/2490837406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 -- 172.21.15.3:0/2490837406 >> v2:172.21.15.153:3300/0 conn(0x7fa45c0cae70 msgr2=0x7fa45c0cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 --2- 172.21.15.3:0/2490837406 >> v2:172.21.15.153:3300/0 conn(0x7fa45c0cae70 0x7fa45c0cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 -- 172.21.15.3:0/2490837406 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c15d890 con 0x7fa45c0c9170 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa45bfff700 1 --2- 172.21.15.3:0/2490837406 >> v2:172.21.15.153:3300/0 conn(0x7fa45c0cae70 0x7fa45c0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa46127b700 1 --2- 172.21.15.3:0/2490837406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0c9170 0x7fa45c0c95f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa45c06b700 tx=0x7fa454009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0c56d2c000db88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa45b7fe700 1 -- 172.21.15.3:0/2490837406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45400e070 con 0x7fa45c0c9170 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa45b7fe700 1 -- 172.21.15.3:0/2490837406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45400ba50 con 0x7fa45c0c9170 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.147+0000 7fa45b7fe700 1 -- 172.21.15.3:0/2490837406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4540092e0 con 0x7fa45c0c9170 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 -- 172.21.15.3:0/2490837406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0c9170 msgr2=0x7fa45c0c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 --2- 172.21.15.3:0/2490837406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0c9170 0x7fa45c0c95f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa45c06b700 tx=0x7fa454009840 comp rx=0 tx=0).stop 2023-10-06T04:54:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 -- 172.21.15.3:0/2490837406 shutdown_connections 2023-10-06T04:54:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 --2- 172.21.15.3:0/2490837406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0c9170 0x7fa45c0c95f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 --2- 172.21.15.3:0/2490837406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 --2- 172.21.15.3:0/2490837406 >> v2:172.21.15.153:3300/0 conn(0x7fa45c0cae70 0x7fa45c0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 -- 172.21.15.3:0/2490837406 >> 172.21.15.3:0/2490837406 conn(0x7fa45c0cd2a0 msgr2=0x7fa45c0cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 -- 172.21.15.3:0/2490837406 shutdown_connections 2023-10-06T04:54:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.148+0000 7fa462cde700 1 -- 172.21.15.3:0/2490837406 wait complete. 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.149+0000 7fa462cde700 1 Processor -- start 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.149+0000 7fa462cde700 1 -- start start 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa462cde700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa45c0c9170 0x7fa45c0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa462cde700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0cae70 0x7fa45c0c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa462cde700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa462cde700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0b10e0 con 0x7fa45c0cae70 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa462cde700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0b1330 con 0x7fa45c0c9170 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa462cde700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0b1610 con 0x7fa45c159790 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0cae70 0x7fa45c0c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0cae70 0x7fa45c0c28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54524/0 (socket says 172.21.15.3:54524) 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 -- 172.21.15.3:0/1350352592 learned_addr learned my addr 172.21.15.3:0/1350352592 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa460a7a700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa45c0c9170 0x7fa45c0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa46127b700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 -- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 msgr2=0x7fa45c156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 -- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa45c0c9170 msgr2=0x7fa45c0ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa45c0c9170 0x7fa45c0ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 -- 172.21.15.3:0/1350352592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa454009ae0 con 0x7fa45c0cae70 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.150+0000 7fa45bfff700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0cae70 0x7fa45c0c28c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa44800c830 tx=0x7fa448008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.151+0000 7fa458ff9700 1 -- 172.21.15.3:0/1350352592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448011600 con 0x7fa45c0cae70 2023-10-06T04:54:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.151+0000 7fa458ff9700 1 -- 172.21.15.3:0/1350352592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa448011760 con 0x7fa45c0cae70 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.151+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c0b18f0 con 0x7fa45c0cae70 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.151+0000 7fa458ff9700 1 -- 172.21.15.3:0/1350352592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448011b20 con 0x7fa45c0cae70 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.151+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c0ad8c0 con 0x7fa45c0cae70 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.152+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45c0b1e10 con 0x7fa45c0cae70 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.152+0000 7fa458ff9700 1 -- 172.21.15.3:0/1350352592 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa44800a3e0 con 0x7fa45c0cae70 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.152+0000 7fa458ff9700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa440066f30 0x7fa4400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.152+0000 7fa458ff9700 1 -- 172.21.15.3:0/1350352592 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa448084b90 con 0x7fa45c0cae70 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.153+0000 7fa460a7a700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa440066f30 0x7fa4400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.153+0000 7fa460a7a700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa440066f30 0x7fa4400693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa450006fd0 tx=0x7fa450008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.157+0000 7fa458ff9700 1 -- 172.21.15.3:0/1350352592 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa45c0b1e10 con 0x7fa45c0cae70 2023-10-06T04:54:26.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.349+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa45c0b1e10 con 0x7fa45c0cae70 2023-10-06T04:54:26.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.349+0000 7fa458ff9700 1 -- 172.21.15.3:0/1350352592 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa45c0b1e10 con 0x7fa45c0cae70 2023-10-06T04:54:26.351 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:54:26.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa440066f30 msgr2=0x7fa4400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa440066f30 0x7fa4400693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa450006fd0 tx=0x7fa450008040 comp rx=0 tx=0).stop 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0cae70 msgr2=0x7fa45c0c28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0cae70 0x7fa45c0c28c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa44800c830 tx=0x7fa448008040 comp rx=0 tx=0).stop 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 shutdown_connections 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa440066f30 0x7fa4400693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa45c0c9170 0x7fa45c0ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa45c159790 0x7fa45c156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 --2- 172.21.15.3:0/1350352592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa45c0cae70 0x7fa45c0c28c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 >> 172.21.15.3:0/1350352592 conn(0x7fa45c0cd2a0 msgr2=0x7fa45c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 shutdown_connections 2023-10-06T04:54:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.352+0000 7fa462cde700 1 -- 172.21.15.3:0/1350352592 wait complete. 2023-10-06T04:54:26.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:54:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.485+0000 7f2a13047700 1 Processor -- start 2023-10-06T04:54:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.485+0000 7f2a13047700 1 -- start start 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.486+0000 7f2a13047700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2a0c159720 0x7f2a0c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.486+0000 7f2a13047700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c0ae160 con 0x7f2a0c159af0 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.486+0000 7f2a13047700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c0cb650 con 0x7f2a0c06d3b0 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.486+0000 7f2a13047700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c0af920 con 0x7f2a0c159720 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.486+0000 7f2a0bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2a0c159af0 0x7f2a0c0caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57484/0 (socket says 172.21.15.3:57484) 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.486+0000 7f2a0bfff700 1 -- 172.21.15.3:0/3582420653 learned_addr learned my addr 172.21.15.3:0/3582420653 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a10de3700 1 --2- 172.21.15.3:0/3582420653 >> v2:172.21.15.153:3300/0 conn(0x7f2a0c159720 0x7f2a0c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854904610 0 0) 0x7f2a0c0ae160 con 0x7f2a0c159af0 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29f4003570 con 0x7f2a0c159af0 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1400776443 0 0) 0x7f29f4003570 con 0x7f2a0c159af0 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 >> v2:172.21.15.153:3300/0 conn(0x7f2a0c159720 msgr2=0x7f2a0c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 --2- 172.21.15.3:0/3582420653 >> v2:172.21.15.153:3300/0 conn(0x7f2a0c159720 0x7f2a0c15de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 >> v1:172.21.15.153:6789/0 conn(0x7f2a0c06d3b0 legacy=0x7f2a0c0ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c15e3b0 con 0x7f2a0c159af0 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f2a00001470 con 0x7f2a0c159af0 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 >> v1:172.21.15.3:6789/0 conn(0x7f2a0c159af0 legacy=0x7f2a0c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4003a60 0x7f29f4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29f4004810 0x7f29f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f40052e0 0x7f29f400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0af920 con 0x7f29f40052e0 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f400c2a0 con 0x7f29f4003a60 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f400c460 con 0x7f29f4004810 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a115e4700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f40052e0 0x7f29f400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a10de3700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4003a60 0x7f29f4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.487+0000 7f2a0bfff700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29f4004810 0x7f29f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a115e4700 1 -- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29f4004810 msgr2=0x7f29f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a115e4700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29f4004810 0x7f29f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a115e4700 1 -- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4003a60 msgr2=0x7f29f4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a115e4700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4003a60 0x7f29f4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a115e4700 1 -- 172.21.15.3:0/3582420653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f4003570 con 0x7f29f40052e0 2023-10-06T04:54:26.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a115e4700 1 -- 172.21.15.3:0/3582420653 reap_dead start 2023-10-06T04:54:26.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a115e4700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f40052e0 0x7f29f400ba60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0018b0 tx=0x7f29fc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1194886c16c0d28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc012420 con 0x7f29f40052e0 2023-10-06T04:54:26.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc0125a0 con 0x7f29f40052e0 2023-10-06T04:54:26.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a0b7fe700 1 -- 172.21.15.3:0/3582420653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc012960 con 0x7f29f40052e0 2023-10-06T04:54:26.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a13047700 1 -- 172.21.15.3:0/3582420653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f40052e0 msgr2=0x7f29f400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a13047700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f40052e0 0x7f29f400ba60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0018b0 tx=0x7f29fc007d30 comp rx=0 tx=0).stop 2023-10-06T04:54:26.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a13047700 1 -- 172.21.15.3:0/3582420653 shutdown_connections 2023-10-06T04:54:26.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a13047700 1 --2- 172.21.15.3:0/3582420653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f40052e0 0x7f29f400ba60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a13047700 1 -- 172.21.15.3:0/3582420653 >> 172.21.15.3:0/3582420653 conn(0x7f2a0c068e70 msgr2=0x7f2a0c069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:26.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.488+0000 7f2a13047700 1 -- 172.21.15.3:0/3582420653 shutdown_connections 2023-10-06T04:54:26.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.489+0000 7f2a13047700 1 -- 172.21.15.3:0/3582420653 wait complete. 2023-10-06T04:54:26.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 Processor -- start 2023-10-06T04:54:26.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 -- start start 2023-10-06T04:54:26.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f40052e0 0x7f2a0c0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a0c155c80 0x7f2a0c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a0c155640 0x7f2a0c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0b5bc0 con 0x7f2a0c155640 2023-10-06T04:54:26.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0b5730 con 0x7f29f40052e0 2023-10-06T04:54:26.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.490+0000 7f2a13047700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0b4b20 con 0x7f2a0c155c80 2023-10-06T04:54:26.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a0c155640 0x7f2a0c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a0c155640 0x7f2a0c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54546/0 (socket says 172.21.15.3:54546) 2023-10-06T04:54:26.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 -- 172.21.15.3:0/1558611511 learned_addr learned my addr 172.21.15.3:0/1558611511 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:26.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a0bfff700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a0c155c80 0x7f2a0c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a10de3700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f40052e0 0x7f2a0c0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 -- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a0c155c80 msgr2=0x7f2a0c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a0c155c80 0x7f2a0c0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 -- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f40052e0 msgr2=0x7f2a0c0b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f40052e0 0x7f2a0c0b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 -- 172.21.15.3:0/1558611511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0b4e00 con 0x7f2a0c155640 2023-10-06T04:54:26.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a10de3700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f40052e0 0x7f2a0c0b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:26.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a0bfff700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a0c155c80 0x7f2a0c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:26.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a115e4700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a0c155640 0x7f2a0c0cf7b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c06e390 tx=0x7f29fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a08ff9700 1 -- 172.21.15.3:0/1558611511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc012420 con 0x7f2a0c155640 2023-10-06T04:54:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a08ff9700 1 -- 172.21.15.3:0/1558611511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc001200 con 0x7f2a0c155640 2023-10-06T04:54:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c0b50e0 con 0x7f2a0c155640 2023-10-06T04:54:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a08ff9700 1 -- 172.21.15.3:0/1558611511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00ce50 con 0x7f2a0c155640 2023-10-06T04:54:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.491+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c0b1cc0 con 0x7f2a0c155640 2023-10-06T04:54:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.492+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c06c410 con 0x7f2a0c155640 2023-10-06T04:54:26.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.493+0000 7f2a08ff9700 1 -- 172.21.15.3:0/1558611511 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29fc019070 con 0x7f2a0c155640 2023-10-06T04:54:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.493+0000 7f2a08ff9700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29f8066f30 0x7f29f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.493+0000 7f2a08ff9700 1 -- 172.21.15.3:0/1558611511 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f29fc085b90 con 0x7f2a0c155640 2023-10-06T04:54:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.493+0000 7f2a10de3700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29f8066f30 0x7f29f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.494+0000 7f2a10de3700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29f8066f30 0x7f29f80693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a040094a0 tx=0x7f2a0400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.497+0000 7f2a08ff9700 1 -- 172.21.15.3:0/1558611511 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29fc052c60 con 0x7f2a0c155640 2023-10-06T04:54:26.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.687+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a0c0b5870 con 0x7f2a0c155640 2023-10-06T04:54:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.688+0000 7f2a08ff9700 1 -- 172.21.15.3:0/1558611511 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29fc017070 con 0x7f2a0c155640 2023-10-06T04:54:26.690 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:54:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29f8066f30 msgr2=0x7f29f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29f8066f30 0x7f29f80693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a040094a0 tx=0x7f2a0400b040 comp rx=0 tx=0).stop 2023-10-06T04:54:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a0c155640 msgr2=0x7f2a0c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a0c155640 0x7f2a0c0cf7b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c06e390 tx=0x7f29fc015000 comp rx=0 tx=0).stop 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 shutdown_connections 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f29f8066f30 0x7f29f80693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f40052e0 0x7f2a0c0b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a0c155640 0x7f2a0c0cf7b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 --2- 172.21.15.3:0/1558611511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a0c155c80 0x7f2a0c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 >> 172.21.15.3:0/1558611511 conn(0x7f2a0c068e70 msgr2=0x7f2a0c0cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 shutdown_connections 2023-10-06T04:54:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.690+0000 7f2a13047700 1 -- 172.21.15.3:0/1558611511 wait complete. 2023-10-06T04:54:26.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:54:26.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.824+0000 7fe8c97db700 1 Processor -- start 2023-10-06T04:54:26.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.824+0000 7fe8c97db700 1 -- start start 2023-10-06T04:54:26.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c97db700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8c415cc30 0x7fe8c40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c97db700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8c40ca650 0x7fe8c40ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c97db700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c40cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c97db700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c40ccd90 con 0x7fe8c415cc30 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c97db700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c40caaf0 con 0x7fe8c40ca650 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c97db700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c406a760 con 0x7fe8c40ce9c0 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8c415cc30 0x7fe8c40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.825+0000 7fe8c2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8c415cc30 0x7fe8c40ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54552/0 (socket says 172.21.15.3:54552) 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/3116984193 learned_addr learned my addr 172.21.15.3:0/3116984193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c27fc700 1 --2- 172.21.15.3:0/3116984193 >> v2:172.21.15.153:3300/0 conn(0x7fe8c40ca650 0x7fe8c40ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/3116984193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 msgr2=0x7fe8c40cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:26.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c37fe700 1 --2- 172.21.15.3:0/3116984193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c40cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c2ffd700 1 --2- 172.21.15.3:0/3116984193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c40cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/3116984193 >> v2:172.21.15.153:3300/0 conn(0x7fe8c40ca650 msgr2=0x7fe8c40ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c2ffd700 1 --2- 172.21.15.3:0/3116984193 >> v2:172.21.15.153:3300/0 conn(0x7fe8c40ca650 0x7fe8c40ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/3116984193 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c415e0a0 con 0x7fe8c415cc30 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c27fc700 1 --2- 172.21.15.3:0/3116984193 >> v2:172.21.15.153:3300/0 conn(0x7fe8c40ca650 0x7fe8c40ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c37fe700 1 --2- 172.21.15.3:0/3116984193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c40cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c2ffd700 1 --2- 172.21.15.3:0/3116984193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c415cc30 0x7fe8c40ca110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4064d10 tx=0x7fe8b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6682dba098734a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c1ffb700 1 -- 172.21.15.3:0/3116984193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400b860 con 0x7fe8c415cc30 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c1ffb700 1 -- 172.21.15.3:0/3116984193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b400b9c0 con 0x7fe8c415cc30 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.826+0000 7fe8c1ffb700 1 -- 172.21.15.3:0/3116984193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400bd60 con 0x7fe8c415cc30 2023-10-06T04:54:26.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 -- 172.21.15.3:0/3116984193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c415cc30 msgr2=0x7fe8c40ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 --2- 172.21.15.3:0/3116984193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c415cc30 0x7fe8c40ca110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4064d10 tx=0x7fe8b4009250 comp rx=0 tx=0).stop 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 -- 172.21.15.3:0/3116984193 shutdown_connections 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 --2- 172.21.15.3:0/3116984193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c40cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 --2- 172.21.15.3:0/3116984193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c415cc30 0x7fe8c40ca110 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 --2- 172.21.15.3:0/3116984193 >> v2:172.21.15.153:3300/0 conn(0x7fe8c40ca650 0x7fe8c40ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 -- 172.21.15.3:0/3116984193 >> 172.21.15.3:0/3116984193 conn(0x7fe8c406c4c0 msgr2=0x7fe8c406c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 -- 172.21.15.3:0/3116984193 shutdown_connections 2023-10-06T04:54:26.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.827+0000 7fe8c97db700 1 -- 172.21.15.3:0/3116984193 wait complete. 2023-10-06T04:54:26.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.828+0000 7fe8c97db700 1 Processor -- start 2023-10-06T04:54:26.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.828+0000 7fe8c97db700 1 -- start start 2023-10-06T04:54:26.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c97db700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c40ca650 0x7fe8c416ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c97db700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c97db700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c415cc30 0x7fe8c40b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c97db700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40b1940 con 0x7fe8c40ca650 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c97db700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c415a9c0 con 0x7fe8c415cc30 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c97db700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c415ac90 con 0x7fe8c40ce9c0 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c40ca650 0x7fe8c416ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c40ca650 0x7fe8c416ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54554/0 (socket says 172.21.15.3:54554) 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/872242024 learned_addr learned my addr 172.21.15.3:0/872242024 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c27fc700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c37fe700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c415cc30 0x7fe8c40b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 msgr2=0x7fe8c415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c415cc30 msgr2=0x7fe8c40b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c415cc30 0x7fe8c40b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 -- 172.21.15.3:0/872242024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40094f0 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c37fe700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c415cc30 0x7fe8c40b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.829+0000 7fe8c2ffd700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c40ca650 0x7fe8c416ad00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4068c20 tx=0x7fe8b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.830+0000 7fe8aaffd700 1 -- 172.21.15.3:0/872242024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4011e70 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.830+0000 7fe8aaffd700 1 -- 172.21.15.3:0/872242024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b4024440 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.830+0000 7fe8aaffd700 1 -- 172.21.15.3:0/872242024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4024800 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.830+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c40b12f0 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.830+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c416b2e0 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.830+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c406e620 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.831+0000 7fe8aaffd700 1 -- 172.21.15.3:0/872242024 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8b401a420 con 0x7fe8c40ca650 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.831+0000 7fe8aaffd700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe8b0066f30 0x7fe8b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.831+0000 7fe8aaffd700 1 -- 172.21.15.3:0/872242024 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe8b4086510 con 0x7fe8c40ca650 2023-10-06T04:54:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.832+0000 7fe8c27fc700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe8b0066f30 0x7fe8b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.832+0000 7fe8c27fc700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe8b0066f30 0x7fe8b00693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800acc0 tx=0x7fe8b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:26.836+0000 7fe8aaffd700 1 -- 172.21.15.3:0/872242024 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8b4053660 con 0x7fe8c40ca650 2023-10-06T04:54:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.026+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8c415d490 con 0x7fe8c40ca650 2023-10-06T04:54:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.027+0000 7fe8aaffd700 1 -- 172.21.15.3:0/872242024 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8b4056c80 con 0x7fe8c40ca650 2023-10-06T04:54:27.029 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:54:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe8b0066f30 msgr2=0x7fe8b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe8b0066f30 0x7fe8b00693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800acc0 tx=0x7fe8b8008040 comp rx=0 tx=0).stop 2023-10-06T04:54:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c40ca650 msgr2=0x7fe8c416ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c40ca650 0x7fe8c416ad00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4068c20 tx=0x7fe8b4017000 comp rx=0 tx=0).stop 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 shutdown_connections 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe8b0066f30 0x7fe8b00693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c415cc30 0x7fe8c40b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c40ca650 0x7fe8c416ad00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 --2- 172.21.15.3:0/872242024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8c40ce9c0 0x7fe8c415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 >> 172.21.15.3:0/872242024 conn(0x7fe8c406c4c0 msgr2=0x7fe8c414d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 shutdown_connections 2023-10-06T04:54:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.029+0000 7fe8c97db700 1 -- 172.21.15.3:0/872242024 wait complete. 2023-10-06T04:54:27.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:27.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 Processor -- start 2023-10-06T04:54:27.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 -- start start 2023-10-06T04:54:27.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69fc159720 0x7f69fc15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f69fc159af0 0x7f69fc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc06d3b0 0x7f69fc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc06dd50 con 0x7f69fc159720 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc0ae160 con 0x7f69fc159af0 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.160+0000 7f6a03a6b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0af920 con 0x7f69fc06d3b0 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69fc159720 0x7f69fc15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69fc159720 0x7f69fc15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54564/0 (socket says 172.21.15.3:54564) 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 -- 172.21.15.3:0/4083438913 learned_addr learned my addr 172.21.15.3:0/4083438913 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01006700 1 --2- 172.21.15.3:0/4083438913 >> v2:172.21.15.153:3300/0 conn(0x7f69fc159af0 0x7f69fc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a02008700 1 --2- 172.21.15.3:0/4083438913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc06d3b0 0x7f69fc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 -- 172.21.15.3:0/4083438913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc06d3b0 msgr2=0x7f69fc06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 --2- 172.21.15.3:0/4083438913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc06d3b0 0x7f69fc06d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 -- 172.21.15.3:0/4083438913 >> v2:172.21.15.153:3300/0 conn(0x7f69fc159af0 msgr2=0x7f69fc0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 --2- 172.21.15.3:0/4083438913 >> v2:172.21.15.153:3300/0 conn(0x7f69fc159af0 0x7f69fc0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 -- 172.21.15.3:0/4083438913 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc15e3b0 con 0x7f69fc159720 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01006700 1 --2- 172.21.15.3:0/4083438913 >> v2:172.21.15.153:3300/0 conn(0x7f69fc159af0 0x7f69fc0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a02008700 1 --2- 172.21.15.3:0/4083438913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc06d3b0 0x7f69fc06d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a01807700 1 --2- 172.21.15.3:0/4083438913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159720 0x7f69fc15de70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0c7550 tx=0x7f69f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fa7bb1bfb4e015 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a00805700 1 -- 172.21.15.3:0/4083438913 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400e070 con 0x7f69fc159720 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a00805700 1 -- 172.21.15.3:0/4083438913 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f400ba50 con 0x7f69fc159720 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.161+0000 7f6a00805700 1 -- 172.21.15.3:0/4083438913 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f40092e0 con 0x7f69fc159720 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 -- 172.21.15.3:0/4083438913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159720 msgr2=0x7f69fc15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/4083438913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159720 0x7f69fc15de70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0c7550 tx=0x7f69f4009840 comp rx=0 tx=0).stop 2023-10-06T04:54:27.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 -- 172.21.15.3:0/4083438913 shutdown_connections 2023-10-06T04:54:27.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/4083438913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc06d3b0 0x7f69fc06d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/4083438913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc159720 0x7f69fc15de70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/4083438913 >> v2:172.21.15.153:3300/0 conn(0x7f69fc159af0 0x7f69fc0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 -- 172.21.15.3:0/4083438913 >> 172.21.15.3:0/4083438913 conn(0x7f69fc068e70 msgr2=0x7f69fc069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:27.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 -- 172.21.15.3:0/4083438913 shutdown_connections 2023-10-06T04:54:27.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.162+0000 7f6a03a6b700 1 -- 172.21.15.3:0/4083438913 wait complete. 2023-10-06T04:54:27.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.163+0000 7f6a03a6b700 1 Processor -- start 2023-10-06T04:54:27.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.163+0000 7f6a03a6b700 1 -- start start 2023-10-06T04:54:27.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a03a6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc06d3b0 0x7f69fc0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a03a6b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc159720 0x7f69fc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a03a6b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69fc159af0 0x7f69fc15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a03a6b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc15b390 con 0x7f69fc06d3b0 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a03a6b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0b3950 con 0x7f69fc159af0 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a03a6b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc15c700 con 0x7f69fc159720 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc06d3b0 0x7f69fc0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc06d3b0 0x7f69fc0b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54576/0 (socket says 172.21.15.3:54576) 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 -- 172.21.15.3:0/449774811 learned_addr learned my addr 172.21.15.3:0/449774811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01006700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc159720 0x7f69fc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a02008700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69fc159af0 0x7f69fc15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 -- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc159720 msgr2=0x7f69fc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc159720 0x7f69fc0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 -- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69fc159af0 msgr2=0x7f69fc15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69fc159af0 0x7f69fc15bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 -- 172.21.15.3:0/449774811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4009ae0 con 0x7f69fc06d3b0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01006700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc159720 0x7f69fc0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a02008700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69fc159af0 0x7f69fc15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.164+0000 7f6a01807700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc06d3b0 0x7f69fc0b5f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009880 tx=0x7f69f4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.165+0000 7f69f9ffb700 1 -- 172.21.15.3:0/449774811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4008ae0 con 0x7f69fc06d3b0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.165+0000 7f69f9ffb700 1 -- 172.21.15.3:0/449774811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4008c60 con 0x7f69fc06d3b0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.165+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc15c9e0 con 0x7f69fc06d3b0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.165+0000 7f69f9ffb700 1 -- 172.21.15.3:0/449774811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4026460 con 0x7f69fc06d3b0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.165+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc15cfa0 con 0x7f69fc06d3b0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.166+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc15d190 con 0x7f69fc06d3b0 2023-10-06T04:54:27.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.166+0000 7f69f9ffb700 1 -- 172.21.15.3:0/449774811 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69f4003040 con 0x7f69fc06d3b0 2023-10-06T04:54:27.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.166+0000 7f69f9ffb700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69e8066f30 0x7f69e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.167+0000 7f69f9ffb700 1 -- 172.21.15.3:0/449774811 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f69f4086550 con 0x7f69fc06d3b0 2023-10-06T04:54:27.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.167+0000 7f6a01006700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69e8066f30 0x7f69e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.167+0000 7f6a01006700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69e8066f30 0x7f69e80693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.171+0000 7f69f9ffb700 1 -- 172.21.15.3:0/449774811 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69f40536d0 con 0x7f69fc06d3b0 2023-10-06T04:54:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.409+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69fc0cad00 con 0x7f69fc06d3b0 2023-10-06T04:54:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.410+0000 7f69f9ffb700 1 -- 172.21.15.3:0/449774811 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f69f4056cf0 con 0x7f69fc06d3b0 2023-10-06T04:54:27.412 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:27.412 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/596033972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":596033972},{"type":"v1","addr":"172.21.15.153:6837","nonce":596033972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1359588707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1359588707},{"type":"v1","addr":"172.21.15.3:6834","nonce":1359588707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/616658407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":616658407},{"type":"v1","addr":"172.21.15.3:6835","nonce":616658407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:49.997154+0000","modified":"2023-10-06T04:53:53.627991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/4017437319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":4017437319},{"type":"v1","addr":"172.21.15.153:6836","nonce":4017437319}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:54:27.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69e8066f30 msgr2=0x7f69e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69e8066f30 0x7f69e80693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0008040 comp rx=0 tx=0).stop 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc06d3b0 msgr2=0x7f69fc0b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc06d3b0 0x7f69fc0b5f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009880 tx=0x7f69f4018000 comp rx=0 tx=0).stop 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 shutdown_connections 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69e8066f30 0x7f69e80693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69fc159af0 0x7f69fc15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69fc06d3b0 0x7f69fc0b5f10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 --2- 172.21.15.3:0/449774811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69fc159720 0x7f69fc0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 >> 172.21.15.3:0/449774811 conn(0x7f69fc068e70 msgr2=0x7f69fc15d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 shutdown_connections 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.412+0000 7f6a03a6b700 1 -- 172.21.15.3:0/449774811 wait complete. 2023-10-06T04:54:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2023-10-06T04:54:27.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:54:27.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.545+0000 7f9103af7700 1 Processor -- start 2023-10-06T04:54:27.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.545+0000 7f9103af7700 1 -- start start 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9103af7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9103af7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90fc159af0 0x7f90fc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9103af7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90fc0ae160 con 0x7f90fc06d3b0 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9103af7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc0af920 con 0x7f90fc159af0 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9103af7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc15e3b0 con 0x7f90fc159720 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90fc159af0 0x7f90fc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90fc159af0 0x7f90fc0caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54586/0 (socket says 172.21.15.3:54586) 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 -- 172.21.15.3:0/669648778 learned_addr learned my addr 172.21.15.3:0/669648778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101893700 1 --2- 172.21.15.3:0/669648778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 -- 172.21.15.3:0/669648778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 msgr2=0x7f90fc15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 --2- 172.21.15.3:0/669648778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc15de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 -- 172.21.15.3:0/669648778 >> v1:172.21.15.153:6789/0 conn(0x7f90fc06d3b0 legacy=0x7f90fc0ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 -- 172.21.15.3:0/669648778 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc0ad550 con 0x7f90fc159af0 2023-10-06T04:54:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.546+0000 7f9101092700 1 --2- 172.21.15.3:0/669648778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc159af0 0x7f90fc0caf40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f90f00099b0 tx=0x7f90f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6823d947728e183c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9100891700 1 -- 172.21.15.3:0/669648778 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0009ea0 con 0x7f90fc159af0 2023-10-06T04:54:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9100891700 1 -- 172.21.15.3:0/669648778 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f000b860 con 0x7f90fc159af0 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9100891700 1 -- 172.21.15.3:0/669648778 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000bc00 con 0x7f90fc159af0 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 -- 172.21.15.3:0/669648778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc159af0 msgr2=0x7f90fc0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 --2- 172.21.15.3:0/669648778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc159af0 0x7f90fc0caf40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f90f00099b0 tx=0x7f90f0009250 comp rx=0 tx=0).stop 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 -- 172.21.15.3:0/669648778 shutdown_connections 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 --2- 172.21.15.3:0/669648778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc15de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 --2- 172.21.15.3:0/669648778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc159af0 0x7f90fc0caf40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 -- 172.21.15.3:0/669648778 >> 172.21.15.3:0/669648778 conn(0x7f90fc068e70 msgr2=0x7f90fc069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 -- 172.21.15.3:0/669648778 shutdown_connections 2023-10-06T04:54:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.547+0000 7f9103af7700 1 -- 172.21.15.3:0/669648778 wait complete. 2023-10-06T04:54:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.548+0000 7f9103af7700 1 Processor -- start 2023-10-06T04:54:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.548+0000 7f9103af7700 1 -- start start 2023-10-06T04:54:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9103af7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc06d3b0 0x7f90fc157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9103af7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc09ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9103af7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90fc159af0 0x7f90fc0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9103af7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc09bec0 con 0x7f90fc06d3b0 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9103af7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc0b3e70 con 0x7f90fc159af0 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9103af7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc09b900 con 0x7f90fc159720 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc06d3b0 0x7f90fc157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc06d3b0 0x7f90fc157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54602/0 (socket says 172.21.15.3:54602) 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 -- 172.21.15.3:0/2843527005 learned_addr learned my addr 172.21.15.3:0/2843527005 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9102094700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90fc159af0 0x7f90fc0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101092700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc09ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 -- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 msgr2=0x7f90fc09ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc09ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 -- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90fc159af0 msgr2=0x7f90fc0c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90fc159af0 0x7f90fc0c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.549+0000 7f9101893700 1 -- 172.21.15.3:0/2843527005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00094f0 con 0x7f90fc06d3b0 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.550+0000 7f9101092700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc09ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.550+0000 7f9101893700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc06d3b0 0x7f90fc157e80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009ef0 tx=0x7f90ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.550+0000 7f90f9ffb700 1 -- 172.21.15.3:0/2843527005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00cc50 con 0x7f90fc06d3b0 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.550+0000 7f90f9ffb700 1 -- 172.21.15.3:0/2843527005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec00f040 con 0x7f90fc06d3b0 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.550+0000 7f90f9ffb700 1 -- 172.21.15.3:0/2843527005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec014480 con 0x7f90fc06d3b0 2023-10-06T04:54:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.550+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc09bbe0 con 0x7f90fc06d3b0 2023-10-06T04:54:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.550+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc0c4420 con 0x7f90fc06d3b0 2023-10-06T04:54:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.551+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc06c410 con 0x7f90fc06d3b0 2023-10-06T04:54:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.551+0000 7f90f9ffb700 1 -- 172.21.15.3:0/2843527005 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90ec0147f0 con 0x7f90fc06d3b0 2023-10-06T04:54:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.552+0000 7f90f9ffb700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e8066f30 0x7f90e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.552+0000 7f90f9ffb700 1 -- 172.21.15.3:0/2843527005 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f90ec085220 con 0x7f90fc06d3b0 2023-10-06T04:54:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.552+0000 7f9101092700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e8066f30 0x7f90e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.552+0000 7f9101092700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e8066f30 0x7f90e80693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90f00099b0 tx=0x7f90f0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.556+0000 7f90f9ffb700 1 -- 172.21.15.3:0/2843527005 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90ec052250 con 0x7f90fc06d3b0 2023-10-06T04:54:27.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.750+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90fc0ce180 con 0x7f90fc06d3b0 2023-10-06T04:54:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.751+0000 7f90f9ffb700 1 -- 172.21.15.3:0/2843527005 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f90ec055870 con 0x7f90fc06d3b0 2023-10-06T04:54:27.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:27.753 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":46,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:54:06.663887+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:47.743195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:48.974063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.25","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e8066f30 msgr2=0x7f90e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e8066f30 0x7f90e80693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90f00099b0 tx=0x7f90f0017000 comp rx=0 tx=0).stop 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc06d3b0 msgr2=0x7f90fc157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc06d3b0 0x7f90fc157e80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009ef0 tx=0x7f90ec00b040 comp rx=0 tx=0).stop 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 shutdown_connections 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e8066f30 0x7f90e80693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90fc159af0 0x7f90fc0c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90fc06d3b0 0x7f90fc157e80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 --2- 172.21.15.3:0/2843527005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90fc159720 0x7f90fc09ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 >> 172.21.15.3:0/2843527005 conn(0x7f90fc068e70 msgr2=0x7f90fc0cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:27.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 shutdown_connections 2023-10-06T04:54:27.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.753+0000 7f9103af7700 1 -- 172.21.15.3:0/2843527005 wait complete. 2023-10-06T04:54:27.767 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:54:27.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:27.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.884+0000 7fc00098b700 1 Processor -- start 2023-10-06T04:54:27.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.884+0000 7fc00098b700 1 -- start start 2023-10-06T04:54:27.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fc00098b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbffc06a9c0 0x7fbffc1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fc00098b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc06e2d0 con 0x7fbffc0b1ad0 2023-10-06T04:54:27.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fc00098b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc06d6c0 con 0x7fbffc158210 2023-10-06T04:54:27.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fc00098b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc06d9a0 con 0x7fbffc06a9c0 2023-10-06T04:54:27.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fbffad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbffc0b1ad0 0x7fbffc157760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57486/0 (socket says 172.21.15.3:57486) 2023-10-06T04:54:27.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fbffad9d700 1 -- 172.21.15.3:0/1783026252 learned_addr learned my addr 172.21.15.3:0/1783026252 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:27.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fbffb59e700 1 --2- 172.21.15.3:0/1783026252 >> v2:172.21.15.153:3300/0 conn(0x7fbffc06a9c0 0x7fbffc1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354319021 0 0) 0x7fbffc06e2d0 con 0x7fbffc0b1ad0 2023-10-06T04:54:27.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe8003570 con 0x7fbffc0b1ad0 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457001449 0 0) 0x7fbffc06d6c0 con 0x7fbffc158210 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.885+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbffc06e2d0 con 0x7fbffc158210 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 731468798 0 0) 0x7fbfe8003570 con 0x7fbffc0b1ad0 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbffb59e700 1 -- 172.21.15.3:0/1783026252 >> v1:172.21.15.153:6789/0 conn(0x7fbffc158210 legacy=0x7fbffc06a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbffb59e700 1 -- 172.21.15.3:0/1783026252 >> v1:172.21.15.3:6789/0 conn(0x7fbffc0b1ad0 legacy=0x7fbffc157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbffb59e700 1 -- 172.21.15.3:0/1783026252 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc06dc80 con 0x7fbffc06a9c0 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbffb59e700 1 --2- 172.21.15.3:0/1783026252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc06a9c0 0x7fbffc1616b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009910 tx=0x7fbff40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ed682514ecb76a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0017f0 con 0x7fbffc06a9c0 2023-10-06T04:54:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4011420 con 0x7fbffc06a9c0 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fbff9d9b700 1 -- 172.21.15.3:0/1783026252 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff40117c0 con 0x7fbffc06a9c0 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fc00098b700 1 -- 172.21.15.3:0/1783026252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc06a9c0 msgr2=0x7fbffc1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fc00098b700 1 --2- 172.21.15.3:0/1783026252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc06a9c0 0x7fbffc1616b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009910 tx=0x7fbff40091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fc00098b700 1 -- 172.21.15.3:0/1783026252 shutdown_connections 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fc00098b700 1 --2- 172.21.15.3:0/1783026252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc06a9c0 0x7fbffc1616b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.886+0000 7fc00098b700 1 -- 172.21.15.3:0/1783026252 >> 172.21.15.3:0/1783026252 conn(0x7fbffc0675c0 msgr2=0x7fbffc0679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.887+0000 7fc00098b700 1 -- 172.21.15.3:0/1783026252 shutdown_connections 2023-10-06T04:54:27.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.887+0000 7fc00098b700 1 -- 172.21.15.3:0/1783026252 wait complete. 2023-10-06T04:54:27.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 Processor -- start 2023-10-06T04:54:27.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 -- start start 2023-10-06T04:54:27.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc06a9c0 0x7fbffc167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0b1ad0 0x7fbffc13c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc158210 0x7fbffc0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc168780 con 0x7fbffc06a9c0 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc168a60 con 0x7fbffc158210 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.888+0000 7fc00098b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc168d40 con 0x7fbffc0b1ad0 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc06a9c0 0x7fbffc167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc06a9c0 0x7fbffc167ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54604/0 (socket says 172.21.15.3:54604) 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 -- 172.21.15.3:0/1116489007 learned_addr learned my addr 172.21.15.3:0/1116489007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffb59e700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc158210 0x7fbffc0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffa59c700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0b1ad0 0x7fbffc13c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 -- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0b1ad0 msgr2=0x7fbffc13c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0b1ad0 0x7fbffc13c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 -- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc158210 msgr2=0x7fbffc0cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc158210 0x7fbffc0cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 -- 172.21.15.3:0/1116489007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009420 con 0x7fbffc06a9c0 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbffad9d700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc06a9c0 0x7fbffc167ee0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00cd50 tx=0x7fbfec007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbff2ffd700 1 -- 172.21.15.3:0/1116489007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec004dc0 con 0x7fbffc06a9c0 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbff2ffd700 1 -- 172.21.15.3:0/1116489007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00a920 con 0x7fbffc06a9c0 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fbff2ffd700 1 -- 172.21.15.3:0/1116489007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00ace0 con 0x7fbffc06a9c0 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc0aa700 con 0x7fbffc06a9c0 2023-10-06T04:54:27.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.889+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc168420 con 0x7fbffc06a9c0 2023-10-06T04:54:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.890+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc0aa570 con 0x7fbffc06a9c0 2023-10-06T04:54:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.891+0000 7fbff2ffd700 1 -- 172.21.15.3:0/1116489007 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfec007360 con 0x7fbffc06a9c0 2023-10-06T04:54:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.891+0000 7fbff2ffd700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbfdc066f30 0x7fbfdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.891+0000 7fbff2ffd700 1 -- 172.21.15.3:0/1116489007 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fbfec084030 con 0x7fbffc06a9c0 2023-10-06T04:54:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.891+0000 7fbffa59c700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbfdc066f30 0x7fbfdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.892+0000 7fbffa59c700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbfdc066f30 0x7fbfdc0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbffc15aa60 tx=0x7fbfe4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:27.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:27.895+0000 7fbff2ffd700 1 -- 172.21.15.3:0/1116489007 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfec051180 con 0x7fbffc06a9c0 2023-10-06T04:54:28.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.135+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbffc157f20 con 0x7fbffc06a9c0 2023-10-06T04:54:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.135+0000 7fbff2ffd700 1 -- 172.21.15.3:0/1116489007 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fbfec0547a0 con 0x7fbffc06a9c0 2023-10-06T04:54:28.137 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:28.137 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/596033972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":596033972},{"type":"v1","addr":"172.21.15.153:6837","nonce":596033972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1359588707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1359588707},{"type":"v1","addr":"172.21.15.3:6834","nonce":1359588707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/616658407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":616658407},{"type":"v1","addr":"172.21.15.3:6835","nonce":616658407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:49.997154+0000","modified":"2023-10-06T04:53:53.627991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/4017437319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":4017437319},{"type":"v1","addr":"172.21.15.153:6836","nonce":4017437319}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:54:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbfdc066f30 msgr2=0x7fbfdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbfdc066f30 0x7fbfdc0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbffc15aa60 tx=0x7fbfe4009040 comp rx=0 tx=0).stop 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc06a9c0 msgr2=0x7fbffc167ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc06a9c0 0x7fbffc167ee0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00cd50 tx=0x7fbfec007dd0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 shutdown_connections 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbfdc066f30 0x7fbfdc0693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc158210 0x7fbffc0cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc06a9c0 0x7fbffc167ee0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 --2- 172.21.15.3:0/1116489007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0b1ad0 0x7fbffc13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 >> 172.21.15.3:0/1116489007 conn(0x7fbffc0675c0 msgr2=0x7fbffc13e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 shutdown_connections 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.137+0000 7fc00098b700 1 -- 172.21.15.3:0/1116489007 wait complete. 2023-10-06T04:54:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2023-10-06T04:54:28.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.268+0000 7fa4435b5700 1 Processor -- start 2023-10-06T04:54:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.268+0000 7fa4435b5700 1 -- start start 2023-10-06T04:54:28.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa4435b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa4435b5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43c0b33d0 0x7fa43c0d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa4435b5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa43c0b3840 0x7fa43c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa4435b5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c0b4290 con 0x7fa43c0b33d0 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa4435b5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c0b2470 con 0x7fa43c0b3840 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa4435b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c0b1580 con 0x7fa43c1595b0 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa440b50700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43c0b33d0 0x7fa43c0d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa440b50700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43c0b33d0 0x7fa43c0d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54606/0 (socket says 172.21.15.3:54606) 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.269+0000 7fa440b50700 1 -- 172.21.15.3:0/43031443 learned_addr learned my addr 172.21.15.3:0/43031443 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa441b52700 1 --2- 172.21.15.3:0/43031443 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0b3840 0x7fa43c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa441351700 1 --2- 172.21.15.3:0/43031443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa440b50700 1 -- 172.21.15.3:0/43031443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 msgr2=0x7fa43c0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa440b50700 1 --2- 172.21.15.3:0/43031443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa440b50700 1 -- 172.21.15.3:0/43031443 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0b3840 msgr2=0x7fa43c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa440b50700 1 --2- 172.21.15.3:0/43031443 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0b3840 0x7fa43c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa440b50700 1 -- 172.21.15.3:0/43031443 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c0b1860 con 0x7fa43c0b33d0 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa441351700 1 --2- 172.21.15.3:0/43031443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa440b50700 1 --2- 172.21.15.3:0/43031443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c0d05d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0d12e0 tx=0x7fa430009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31d65d6b19ee802 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa43bfff700 1 -- 172.21.15.3:0/43031443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43000b860 con 0x7fa43c0b33d0 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa43bfff700 1 -- 172.21.15.3:0/43031443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43000b9c0 con 0x7fa43c0b33d0 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa43bfff700 1 -- 172.21.15.3:0/43031443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43000bd60 con 0x7fa43c0b33d0 2023-10-06T04:54:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa4435b5700 1 -- 172.21.15.3:0/43031443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 msgr2=0x7fa43c0d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.270+0000 7fa4435b5700 1 --2- 172.21.15.3:0/43031443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c0d05d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0d12e0 tx=0x7fa430009250 comp rx=0 tx=0).stop 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.271+0000 7fa4435b5700 1 -- 172.21.15.3:0/43031443 shutdown_connections 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.271+0000 7fa4435b5700 1 --2- 172.21.15.3:0/43031443 >> v2:172.21.15.153:3300/0 conn(0x7fa43c0b3840 0x7fa43c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.271+0000 7fa4435b5700 1 --2- 172.21.15.3:0/43031443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.271+0000 7fa4435b5700 1 --2- 172.21.15.3:0/43031443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c0d05d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.271+0000 7fa4435b5700 1 -- 172.21.15.3:0/43031443 >> 172.21.15.3:0/43031443 conn(0x7fa43c0675c0 msgr2=0x7fa43c0679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.271+0000 7fa4435b5700 1 -- 172.21.15.3:0/43031443 shutdown_connections 2023-10-06T04:54:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.271+0000 7fa4435b5700 1 -- 172.21.15.3:0/43031443 wait complete. 2023-10-06T04:54:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.272+0000 7fa4435b5700 1 Processor -- start 2023-10-06T04:54:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.272+0000 7fa4435b5700 1 -- start start 2023-10-06T04:54:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c0b3840 0x7fa43c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c15e930 con 0x7fa43c0b33d0 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c06da20 con 0x7fa43c0b3840 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c06dd00 con 0x7fa43c1595b0 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c15ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54620/0 (socket says 172.21.15.3:54620) 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 -- 172.21.15.3:0/459699148 learned_addr learned my addr 172.21.15.3:0/459699148 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa440b50700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c0b3840 0x7fa43c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441b52700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 -- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 msgr2=0x7fa43c1576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c1576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 -- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c0b3840 msgr2=0x7fa43c14b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c0b3840 0x7fa43c14b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 -- 172.21.15.3:0/459699148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4300094f0 con 0x7fa43c0b33d0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa440b50700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c0b3840 0x7fa43c14b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441b52700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c1576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa441351700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c15ec10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa43400c850 tx=0x7fa434008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4397fa700 1 -- 172.21.15.3:0/459699148 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434011600 con 0x7fa43c0b33d0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4397fa700 1 -- 172.21.15.3:0/459699148 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa434011760 con 0x7fa43c0b33d0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4397fa700 1 -- 172.21.15.3:0/459699148 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434011b00 con 0x7fa43c0b33d0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c06dfe0 con 0x7fa43c0b33d0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.273+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c14b650 con 0x7fa43c0b33d0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.274+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c150680 con 0x7fa43c0b33d0 2023-10-06T04:54:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.275+0000 7fa4397fa700 1 -- 172.21.15.3:0/459699148 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4340073b0 con 0x7fa43c0b33d0 2023-10-06T04:54:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.275+0000 7fa4397fa700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.275+0000 7fa4397fa700 1 -- 172.21.15.3:0/459699148 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa4340846d0 con 0x7fa43c0b33d0 2023-10-06T04:54:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.275+0000 7fa440b50700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.276+0000 7fa440b50700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0d12e0 tx=0x7fa430017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.280+0000 7fa4397fa700 1 -- 172.21.15.3:0/459699148 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa434055230 con 0x7fa43c0b33d0 2023-10-06T04:54:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.515+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa43c0b1b40 con 0x7fa43c0b33d0 2023-10-06T04:54:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.515+0000 7fa4397fa700 1 -- 172.21.15.3:0/459699148 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fa434054dc0 con 0x7fa43c0b33d0 2023-10-06T04:54:28.517 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:28.518 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/596033972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":596033972},{"type":"v1","addr":"172.21.15.153:6837","nonce":596033972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1359588707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1359588707},{"type":"v1","addr":"172.21.15.3:6834","nonce":1359588707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/616658407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":616658407},{"type":"v1","addr":"172.21.15.3:6835","nonce":616658407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:53:49.997154+0000","modified":"2023-10-06T04:53:53.627991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/4017437319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":4017437319},{"type":"v1","addr":"172.21.15.153:6836","nonce":4017437319}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:54:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 msgr2=0x7fa4280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0d12e0 tx=0x7fa430017000 comp rx=0 tx=0).stop 2023-10-06T04:54:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 msgr2=0x7fa43c15ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c15ec10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa43400c850 tx=0x7fa434008040 comp rx=0 tx=0).stop 2023-10-06T04:54:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 shutdown_connections 2023-10-06T04:54:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa428066f30 0x7fa4280693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa43c1595b0 0x7fa43c1576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43c0b33d0 0x7fa43c15ec10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 --2- 172.21.15.3:0/459699148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c0b3840 0x7fa43c14b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 >> 172.21.15.3:0/459699148 conn(0x7fa43c0675c0 msgr2=0x7fa43c156130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 shutdown_connections 2023-10-06T04:54:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.518+0000 7fa4435b5700 1 -- 172.21.15.3:0/459699148 wait complete. 2023-10-06T04:54:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2023-10-06T04:54:28.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:54:28.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.663+0000 7fd296665700 1 Processor -- start 2023-10-06T04:54:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.663+0000 7fd296665700 1 -- start start 2023-10-06T04:54:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd296665700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd29006c9f0 0x7fd290157980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd296665700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29015c070 con 0x7fd290067cd0 2023-10-06T04:54:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd296665700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29015b4d0 con 0x7fd2901575b0 2023-10-06T04:54:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd296665700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29015b740 con 0x7fd29006c9f0 2023-10-06T04:54:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd28f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd290067cd0 0x7fd2900af150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57490/0 (socket says 172.21.15.3:57490) 2023-10-06T04:54:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd28f7fe700 1 -- 172.21.15.3:0/108561135 learned_addr learned my addr 172.21.15.3:0/108561135 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd294c02700 1 --2- 172.21.15.3:0/108561135 >> v2:172.21.15.3:3300/0 conn(0x7fd29006c9f0 0x7fd290157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038676857 0 0) 0x7fd29015c070 con 0x7fd290067cd0 2023-10-06T04:54:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd278003570 con 0x7fd290067cd0 2023-10-06T04:54:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814691474 0 0) 0x7fd29015b4d0 con 0x7fd2901575b0 2023-10-06T04:54:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29015c070 con 0x7fd2901575b0 2023-10-06T04:54:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd294c02700 1 -- 172.21.15.3:0/108561135 >> v1:172.21.15.153:6789/0 conn(0x7fd2901575b0 legacy=0x7fd2900675c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd294c02700 1 -- 172.21.15.3:0/108561135 >> v1:172.21.15.3:6789/0 conn(0x7fd290067cd0 legacy=0x7fd2900af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16096259 0 0) 0x7fd278003570 con 0x7fd290067cd0 2023-10-06T04:54:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.664+0000 7fd294c02700 1 -- 172.21.15.3:0/108561135 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29015ba20 con 0x7fd29006c9f0 2023-10-06T04:54:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd294c02700 1 --2- 172.21.15.3:0/108561135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29006c9f0 0x7fd290157980 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd288009e60 tx=0x7fd288009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c56424f94e9a19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28800e070 con 0x7fd29006c9f0 2023-10-06T04:54:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28800b7a0 con 0x7fd29006c9f0 2023-10-06T04:54:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd28effd700 1 -- 172.21.15.3:0/108561135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284001470 con 0x7fd29006c9f0 2023-10-06T04:54:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd296665700 1 -- 172.21.15.3:0/108561135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29006c9f0 msgr2=0x7fd290157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd296665700 1 --2- 172.21.15.3:0/108561135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29006c9f0 0x7fd290157980 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd288009e60 tx=0x7fd288009780 comp rx=0 tx=0).stop 2023-10-06T04:54:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd296665700 1 -- 172.21.15.3:0/108561135 shutdown_connections 2023-10-06T04:54:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd296665700 1 --2- 172.21.15.3:0/108561135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd29006c9f0 0x7fd290157980 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd296665700 1 -- 172.21.15.3:0/108561135 >> 172.21.15.3:0/108561135 conn(0x7fd2900cd2a0 msgr2=0x7fd2900cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd296665700 1 -- 172.21.15.3:0/108561135 shutdown_connections 2023-10-06T04:54:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.665+0000 7fd296665700 1 -- 172.21.15.3:0/108561135 wait complete. 2023-10-06T04:54:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.666+0000 7fd296665700 1 Processor -- start 2023-10-06T04:54:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd296665700 1 -- start start 2023-10-06T04:54:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd296665700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd290067cd0 0x7fd290167a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd296665700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd29006c9f0 0x7fd29016a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd296665700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2901575b0 0x7fd29014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd296665700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29014c7c0 con 0x7fd290067cd0 2023-10-06T04:54:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd296665700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29016ad90 con 0x7fd2901575b0 2023-10-06T04:54:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd296665700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd290154540 con 0x7fd29006c9f0 2023-10-06T04:54:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd28ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd290067cd0 0x7fd290167a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd28ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd290067cd0 0x7fd290167a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54634/0 (socket says 172.21.15.3:54634) 2023-10-06T04:54:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.667+0000 7fd28ffff700 1 -- 172.21.15.3:0/1216902341 learned_addr learned my addr 172.21.15.3:0/1216902341 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:28.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd294c02700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2901575b0 0x7fd29014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd28ffff700 1 -- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd29006c9f0 msgr2=0x7fd29016a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd28ffff700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd29006c9f0 0x7fd29016a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd28ffff700 1 -- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2901575b0 msgr2=0x7fd29014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd28ffff700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2901575b0 0x7fd29014bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd28ffff700 1 -- 172.21.15.3:0/1216902341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288009a20 con 0x7fd290067cd0 2023-10-06T04:54:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd28ffff700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd290067cd0 0x7fd290167a90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd28000c580 tx=0x7fd280007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd277fff700 1 -- 172.21.15.3:0/1216902341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280010510 con 0x7fd290067cd0 2023-10-06T04:54:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd277fff700 1 -- 172.21.15.3:0/1216902341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280010670 con 0x7fd290067cd0 2023-10-06T04:54:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd277fff700 1 -- 172.21.15.3:0/1216902341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280010a30 con 0x7fd290067cd0 2023-10-06T04:54:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd290154820 con 0x7fd290067cd0 2023-10-06T04:54:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.668+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd290154de0 con 0x7fd290067cd0 2023-10-06T04:54:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.669+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd290155000 con 0x7fd290067cd0 2023-10-06T04:54:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.669+0000 7fd277fff700 1 -- 172.21.15.3:0/1216902341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd28001b400 con 0x7fd290067cd0 2023-10-06T04:54:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.670+0000 7fd277fff700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd27c066f30 0x7fd27c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.670+0000 7fd277fff700 1 -- 172.21.15.3:0/1216902341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fd280084380 con 0x7fd290067cd0 2023-10-06T04:54:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.670+0000 7fd28f7fe700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd27c066f30 0x7fd27c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.670+0000 7fd28f7fe700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd27c066f30 0x7fd27c0693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd2840011e0 tx=0x7fd284001320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.674+0000 7fd277fff700 1 -- 172.21.15.3:0/1216902341 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd280051450 con 0x7fd290067cd0 2023-10-06T04:54:28.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.871+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd29015bd00 con 0x7fd290067cd0 2023-10-06T04:54:28.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.871+0000 7fd277fff700 1 -- 172.21.15.3:0/1216902341 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fd280054a70 con 0x7fd290067cd0 2023-10-06T04:54:28.874 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:28.874 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":46,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:54:06.663887+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:53:47.743195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T04:53:48.974063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.25","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd27c066f30 msgr2=0x7fd27c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd27c066f30 0x7fd27c0693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd2840011e0 tx=0x7fd284001320 comp rx=0 tx=0).stop 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd290067cd0 msgr2=0x7fd290167a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd290067cd0 0x7fd290167a90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd28000c580 tx=0x7fd280007e10 comp rx=0 tx=0).stop 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 shutdown_connections 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd27c066f30 0x7fd27c0693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2901575b0 0x7fd29014bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd290067cd0 0x7fd290167a90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 --2- 172.21.15.3:0/1216902341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd29006c9f0 0x7fd29016a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:28.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 >> 172.21.15.3:0/1216902341 conn(0x7fd2900cd2a0 msgr2=0x7fd2900c2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:28.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 shutdown_connections 2023-10-06T04:54:28.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:28.874+0000 7fd296665700 1 -- 172.21.15.3:0/1216902341 wait complete. 2023-10-06T04:54:28.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:54:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.007+0000 7f56a38d2700 1 Processor -- start 2023-10-06T04:54:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.007+0000 7f56a38d2700 1 -- start start 2023-10-06T04:54:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.008+0000 7f56a38d2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 0x7f569c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.008+0000 7f56a38d2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c0ae160 con 0x7f569c159720 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.008+0000 7f56a38d2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c0af920 con 0x7f569c06d3b0 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.008+0000 7f56a38d2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0ad550 con 0x7f569c159af0 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.008+0000 7f56a166e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f569c159720 0x7f569c15de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57492/0 (socket says 172.21.15.3:57492) 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.008+0000 7f56a166e700 1 -- 172.21.15.3:0/4045135563 learned_addr learned my addr 172.21.15.3:0/4045135563 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407055037 0 0) 0x7f569c0ae160 con 0x7f569c159720 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 0x7f569c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5684003570 con 0x7f569c159720 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 457963851 0 0) 0x7f5684003570 con 0x7f569c159720 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 msgr2=0x7f569c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 0x7f569c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 >> v1:172.21.15.153:6789/0 conn(0x7f569c06d3b0 legacy=0x7f569c0ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0ad830 con 0x7f569c159720 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f5694001470 con 0x7f569c159720 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 0x7f569c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 >> v1:172.21.15.3:6789/0 conn(0x7f569c159720 legacy=0x7f569c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 0x7f5684003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5684004080 0x7f5684004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004d50 0x7f56840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0ad550 con 0x7f5684004080 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568400bca0 con 0x7f5684004d50 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568400be60 con 0x7f569c159af0 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5684004080 0x7f5684004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a166e700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 0x7f5684003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a1e6f700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004d50 0x7f56840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 -- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 msgr2=0x7f5684003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c159af0 0x7f5684003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 -- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004d50 msgr2=0x7f56840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004d50 0x7f56840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a0e6d700 1 -- 172.21.15.3:0/4045135563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684003570 con 0x7f5684004080 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.009+0000 7f56a1e6f700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004d50 0x7f56840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5684004080 0x7f5684004810 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5690006fd0 tx=0x7f5690008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b4c6cf85fffe40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569000c070 con 0x7f5684004080 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569000e5e0 con 0x7f5684004080 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f569bfff700 1 -- 172.21.15.3:0/4045135563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690009510 con 0x7f5684004080 2023-10-06T04:54:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a38d2700 1 -- 172.21.15.3:0/4045135563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5684004080 msgr2=0x7f5684004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a38d2700 1 --2- 172.21.15.3:0/4045135563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5684004080 0x7f5684004810 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5690006fd0 tx=0x7f5690008040 comp rx=0 tx=0).stop 2023-10-06T04:54:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a1e6f700 1 -- 172.21.15.3:0/4045135563 reap_dead start 2023-10-06T04:54:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a38d2700 1 -- 172.21.15.3:0/4045135563 shutdown_connections 2023-10-06T04:54:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a38d2700 1 -- 172.21.15.3:0/4045135563 >> 172.21.15.3:0/4045135563 conn(0x7f569c068e70 msgr2=0x7f569c069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a38d2700 1 -- 172.21.15.3:0/4045135563 shutdown_connections 2023-10-06T04:54:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.010+0000 7f56a38d2700 1 -- 172.21.15.3:0/4045135563 wait complete. 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.011+0000 7f56a38d2700 1 Processor -- start 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.011+0000 7f56a38d2700 1 -- start start 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a38d2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004080 0x7f569c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a38d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569c09d3a0 0x7f569c0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a38d2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c09d810 0x7f569c0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a38d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0b0810 con 0x7f569c09d3a0 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a38d2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0aff20 con 0x7f5684004080 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a38d2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0b01c0 con 0x7f569c09d810 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a0e6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569c09d3a0 0x7f569c0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a0e6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569c09d3a0 0x7f569c0cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54652/0 (socket says 172.21.15.3:54652) 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.012+0000 7f56a0e6d700 1 -- 172.21.15.3:0/244737709 learned_addr learned my addr 172.21.15.3:0/244737709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a166e700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004080 0x7f569c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a1e6f700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c09d810 0x7f569c0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a0e6d700 1 -- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c09d810 msgr2=0x7f569c0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c09d810 0x7f569c0cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a0e6d700 1 -- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004080 msgr2=0x7f569c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004080 0x7f569c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a0e6d700 1 -- 172.21.15.3:0/244737709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0b04a0 con 0x7f569c09d3a0 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a166e700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004080 0x7f569c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a0e6d700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569c09d3a0 0x7f569c0cf4f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f568400bc70 tx=0x7f569001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56997fa700 1 -- 172.21.15.3:0/244737709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569000eec0 con 0x7f569c09d3a0 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56997fa700 1 -- 172.21.15.3:0/244737709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5690013cb0 con 0x7f569c09d3a0 2023-10-06T04:54:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c09dd80 con 0x7f569c09d3a0 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56997fa700 1 -- 172.21.15.3:0/244737709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569001c670 con 0x7f569c09d3a0 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.013+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c09c5d0 con 0x7f569c09d3a0 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.014+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c0b0630 con 0x7f569c09d3a0 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.014+0000 7f56997fa700 1 -- 172.21.15.3:0/244737709 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f569001c9e0 con 0x7f569c09d3a0 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.015+0000 7f56997fa700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5688066f00 0x7f56880693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.015+0000 7f56997fa700 1 -- 172.21.15.3:0/244737709 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5690086070 con 0x7f569c09d3a0 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.015+0000 7f56a166e700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5688066f00 0x7f56880693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.016+0000 7f56a166e700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5688066f00 0x7f56880693b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5694001120 tx=0x7f5694009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.019+0000 7f56997fa700 1 -- 172.21.15.3:0/244737709 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5690053140 con 0x7f569c09d3a0 2023-10-06T04:54:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.258+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f569c0cad00 con 0x7f569c09d3a0 2023-10-06T04:54:29.260 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T04:54:29.258+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:54:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.262+0000 7f56997fa700 1 -- 172.21.15.3:0/244737709 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5690056760 con 0x7f569c09d3a0 2023-10-06T04:54:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5688066f00 msgr2=0x7f56880693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5688066f00 0x7f56880693b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5694001120 tx=0x7f5694009040 comp rx=0 tx=0).stop 2023-10-06T04:54:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569c09d3a0 msgr2=0x7f569c0cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569c09d3a0 0x7f569c0cf4f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f568400bc70 tx=0x7f569001a000 comp rx=0 tx=0).stop 2023-10-06T04:54:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 shutdown_connections 2023-10-06T04:54:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5688066f00 0x7f56880693b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5684004080 0x7f569c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f569c09d810 0x7f569c0cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 --2- 172.21.15.3:0/244737709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569c09d3a0 0x7f569c0cf4f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 >> 172.21.15.3:0/244737709 conn(0x7f569c068e70 msgr2=0x7f569c0cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 shutdown_connections 2023-10-06T04:54:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.264+0000 7f56a38d2700 1 -- 172.21.15.3:0/244737709 wait complete. 2023-10-06T04:54:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:54:29.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:54:29.300 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:54:29.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.399+0000 7ff13780e700 1 Processor -- start 2023-10-06T04:54:29.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.400+0000 7ff13780e700 1 -- start start 2023-10-06T04:54:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.400+0000 7ff13780e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff13013fd30 0x7ff1300cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.400+0000 7ff13780e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff130140100 0x7ff1300c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.400+0000 7ff13780e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff1300ca2e0 0x7ff13006a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.400+0000 7ff13780e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300c9910 con 0x7ff13013fd30 2023-10-06T04:54:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.400+0000 7ff13780e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300c8d00 con 0x7ff1300ca2e0 2023-10-06T04:54:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.400+0000 7ff13780e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300c8fe0 con 0x7ff130140100 2023-10-06T04:54:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff13013fd30 0x7ff1300cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff13013fd30 0x7ff1300cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54660/0 (socket says 172.21.15.3:54660) 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 -- 172.21.15.3:0/2470914235 learned_addr learned my addr 172.21.15.3:0/2470914235 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff134da9700 1 --2- 172.21.15.3:0/2470914235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff130140100 0x7ff1300c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 -- 172.21.15.3:0/2470914235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff130140100 msgr2=0x7ff1300c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff135dab700 1 --2- 172.21.15.3:0/2470914235 >> v2:172.21.15.153:3300/0 conn(0x7ff1300ca2e0 0x7ff13006a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 --2- 172.21.15.3:0/2470914235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff130140100 0x7ff1300c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 -- 172.21.15.3:0/2470914235 >> v2:172.21.15.153:3300/0 conn(0x7ff1300ca2e0 msgr2=0x7ff13006a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 --2- 172.21.15.3:0/2470914235 >> v2:172.21.15.153:3300/0 conn(0x7ff1300ca2e0 0x7ff13006a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 -- 172.21.15.3:0/2470914235 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300c92c0 con 0x7ff13013fd30 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff1355aa700 1 --2- 172.21.15.3:0/2470914235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300cb610 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff1300cdb50 tx=0x7ff120009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b7133e17e6bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff12ffff700 1 -- 172.21.15.3:0/2470914235 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000e070 con 0x7ff13013fd30 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff12ffff700 1 -- 172.21.15.3:0/2470914235 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12000bae0 con 0x7ff13013fd30 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.401+0000 7ff12ffff700 1 -- 172.21.15.3:0/2470914235 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200092e0 con 0x7ff13013fd30 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 -- 172.21.15.3:0/2470914235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 msgr2=0x7ff1300cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 --2- 172.21.15.3:0/2470914235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300cb610 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff1300cdb50 tx=0x7ff120009840 comp rx=0 tx=0).stop 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 -- 172.21.15.3:0/2470914235 shutdown_connections 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 --2- 172.21.15.3:0/2470914235 >> v2:172.21.15.153:3300/0 conn(0x7ff1300ca2e0 0x7ff13006a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 --2- 172.21.15.3:0/2470914235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300cb610 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 --2- 172.21.15.3:0/2470914235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff130140100 0x7ff1300c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 -- 172.21.15.3:0/2470914235 >> 172.21.15.3:0/2470914235 conn(0x7ff13006c4c0 msgr2=0x7ff13006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 -- 172.21.15.3:0/2470914235 shutdown_connections 2023-10-06T04:54:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.402+0000 7ff13780e700 1 -- 172.21.15.3:0/2470914235 wait complete. 2023-10-06T04:54:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.403+0000 7ff13780e700 1 Processor -- start 2023-10-06T04:54:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.403+0000 7ff13780e700 1 -- start start 2023-10-06T04:54:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff13780e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1300ca2e0 0x7ff13013de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff13780e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff13780e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1300c5960 0x7ff1300c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff13780e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300c6af0 con 0x7ff13013fd30 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff13780e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13013e340 con 0x7ff1300c5960 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff13780e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300c5ee0 con 0x7ff1300ca2e0 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54666/0 (socket says 172.21.15.3:54666) 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 -- 172.21.15.3:0/209198893 learned_addr learned my addr 172.21.15.3:0/209198893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff1355aa700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1300ca2e0 0x7ff13013de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff135dab700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1300c5960 0x7ff1300c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 -- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1300ca2e0 msgr2=0x7ff13013de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1300ca2e0 0x7ff13013de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 -- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1300c5960 msgr2=0x7ff1300c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1300c5960 0x7ff1300c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 -- 172.21.15.3:0/209198893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120009ae0 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.404+0000 7ff134da9700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300c5420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c880 tx=0x7ff124008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.405+0000 7ff12d7fa700 1 -- 172.21.15.3:0/209198893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124011600 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.405+0000 7ff12d7fa700 1 -- 172.21.15.3:0/209198893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff124011780 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.405+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1300c61c0 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.405+0000 7ff12d7fa700 1 -- 172.21.15.3:0/209198893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124011b40 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.405+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1300c6780 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.405+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1300c6e80 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.406+0000 7ff12d7fa700 1 -- 172.21.15.3:0/209198893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff124018360 con 0x7ff13013fd30 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.406+0000 7ff12d7fa700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff11c066f30 0x7ff11c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.406+0000 7ff12d7fa700 1 -- 172.21.15.3:0/209198893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6202+0+0 (secure 0 0 0) 0x7ff124084420 con 0x7ff13013fd30 2023-10-06T04:54:29.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.407+0000 7ff1355aa700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff11c066f30 0x7ff11c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:29.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.407+0000 7ff1355aa700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff11c066f30 0x7ff11c0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff13006fb20 tx=0x7ff120018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:29.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.411+0000 7ff12d7fa700 1 -- 172.21.15.3:0/209198893 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1300c6e80 con 0x7ff13013fd30 2023-10-06T04:54:29.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:29.660+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1300c6e80 con 0x7ff13013fd30 2023-10-06T04:54:30.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.267+0000 7ff12d7fa700 1 -- 172.21.15.3:0/209198893 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff1300c6e80 con 0x7ff13013fd30 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff11c066f30 msgr2=0x7ff11c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff11c066f30 0x7ff11c0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff13006fb20 tx=0x7ff120018000 comp rx=0 tx=0).stop 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 msgr2=0x7ff1300c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300c5420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c880 tx=0x7ff124008040 comp rx=0 tx=0).stop 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 shutdown_connections 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff11c066f30 0x7ff11c0693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1300c5960 0x7ff1300c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1300ca2e0 0x7ff13013de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 --2- 172.21.15.3:0/209198893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13013fd30 0x7ff1300c5420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 >> 172.21.15.3:0/209198893 conn(0x7ff13006c4c0 msgr2=0x7ff13006ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:30.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 shutdown_connections 2023-10-06T04:54:30.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.270+0000 7ff13780e700 1 -- 172.21.15.3:0/209198893 wait complete. 2023-10-06T04:54:30.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:54:30.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.408+0000 7fd44dd72700 1 Processor -- start 2023-10-06T04:54:30.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.408+0000 7fd44dd72700 1 -- start start 2023-10-06T04:54:30.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.409+0000 7fd44dd72700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 0x7fd4480cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.409+0000 7fd44dd72700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4480cddc0 con 0x7fd448143150 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.409+0000 7fd44dd72700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4480ce750 con 0x7fd4480cd270 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.409+0000 7fd44dd72700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480cf2d0 con 0x7fd4480d0c60 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.409+0000 7fd4477fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd448143150 0x7fd4480d0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57494/0 (socket says 172.21.15.3:57494) 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.409+0000 7fd4477fe700 1 -- 172.21.15.3:0/1411670795 learned_addr learned my addr 172.21.15.3:0/1411670795 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd446ffd700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 0x7fd4480cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079144403 0 0) 0x7fd4480cddc0 con 0x7fd448143150 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd430003570 con 0x7fd448143150 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454662719 0 0) 0x7fd430003570 con 0x7fd448143150 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 msgr2=0x7fd4480cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 0x7fd4480cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 >> v1:172.21.15.153:6789/0 conn(0x7fd4480cd270 legacy=0x7fd4480cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480ce9a0 con 0x7fd448143150 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fd438001470 con 0x7fd448143150 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 >> v1:172.21.15.3:6789/0 conn(0x7fd448143150 legacy=0x7fd4480d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd430003a90 0x7fd430003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 0x7fd430004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd430004d50 0x7fd4300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480cf2d0 con 0x7fd430004d50 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43000bb70 con 0x7fd430003a90 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43000bd30 con 0x7fd4480d0c60 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd447fff700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd430004d50 0x7fd4300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd4477fe700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd430003a90 0x7fd430003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd446ffd700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 0x7fd430004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd447fff700 1 -- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 msgr2=0x7fd430004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd447fff700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 0x7fd430004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd447fff700 1 -- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd430003a90 msgr2=0x7fd430003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd447fff700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd430003a90 0x7fd430003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd447fff700 1 -- 172.21.15.3:0/1411670795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd430003570 con 0x7fd430004d50 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.410+0000 7fd446ffd700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4480d0c60 0x7fd430004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd447fff700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd430004d50 0x7fd4300051c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd440001fc0 tx=0x7fd440001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7127deb21c7d31c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440001950 con 0x7fd430004d50 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44000c8b0 con 0x7fd430004d50 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd4467fc700 1 -- 172.21.15.3:0/1411670795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44000cc50 con 0x7fd430004d50 2023-10-06T04:54:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd44dd72700 1 -- 172.21.15.3:0/1411670795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd430004d50 msgr2=0x7fd4300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd44dd72700 1 --2- 172.21.15.3:0/1411670795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd430004d50 0x7fd4300051c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd440001fc0 tx=0x7fd440001640 comp rx=0 tx=0).stop 2023-10-06T04:54:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd447fff700 1 -- 172.21.15.3:0/1411670795 reap_dead start 2023-10-06T04:54:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd44dd72700 1 -- 172.21.15.3:0/1411670795 shutdown_connections 2023-10-06T04:54:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd44dd72700 1 -- 172.21.15.3:0/1411670795 >> 172.21.15.3:0/1411670795 conn(0x7fd4480675c0 msgr2=0x7fd4480679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd44dd72700 1 -- 172.21.15.3:0/1411670795 shutdown_connections 2023-10-06T04:54:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.411+0000 7fd44dd72700 1 -- 172.21.15.3:0/1411670795 wait complete. 2023-10-06T04:54:30.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 Processor -- start 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 -- start start 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd448159c30 0x7fd4480c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd44815a000 0x7fd4480c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44815aaf0 0x7fd4480c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480ca5a0 con 0x7fd448159c30 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480c8be0 con 0x7fd44815a000 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd44dd72700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44815aff0 con 0x7fd44815aaf0 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd4477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd448159c30 0x7fd4480c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.413+0000 7fd4477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd448159c30 0x7fd4480c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54686/0 (socket says 172.21.15.3:54686) 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd4477fe700 1 -- 172.21.15.3:0/190794907 learned_addr learned my addr 172.21.15.3:0/190794907 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:30.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd4477fe700 1 -- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44815aaf0 msgr2=0x7fd4480c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd447fff700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44815aaf0 0x7fd4480c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd446ffd700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd44815a000 0x7fd4480c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd4477fe700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44815aaf0 0x7fd4480c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd4477fe700 1 -- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd44815a000 msgr2=0x7fd4480c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd4477fe700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd44815a000 0x7fd4480c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd4477fe700 1 -- 172.21.15.3:0/190794907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44815b2d0 con 0x7fd448159c30 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd4477fe700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd448159c30 0x7fd4480c86a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd4380073c0 tx=0x7fd438001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd42f7fe700 1 -- 172.21.15.3:0/190794907 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438018030 con 0x7fd448159c30 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd42f7fe700 1 -- 172.21.15.3:0/190794907 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438000c70 con 0x7fd448159c30 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44815b5b0 con 0x7fd448159c30 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd42f7fe700 1 -- 172.21.15.3:0/190794907 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43800fb00 con 0x7fd448159c30 2023-10-06T04:54:30.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.414+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44815bb70 con 0x7fd448159c30 2023-10-06T04:54:30.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.415+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44806c760 con 0x7fd448159c30 2023-10-06T04:54:30.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.416+0000 7fd42f7fe700 1 -- 172.21.15.3:0/190794907 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd43801b070 con 0x7fd448159c30 2023-10-06T04:54:30.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.416+0000 7fd42f7fe700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd434066f30 0x7fd4340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:30.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.416+0000 7fd42f7fe700 1 -- 172.21.15.3:0/190794907 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fd438019070 con 0x7fd448159c30 2023-10-06T04:54:30.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.416+0000 7fd446ffd700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd434066f30 0x7fd4340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:30.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.417+0000 7fd446ffd700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd434066f30 0x7fd4340693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd43c006f00 tx=0x7fd43c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:30.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.420+0000 7fd42f7fe700 1 -- 172.21.15.3:0/190794907 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4380515c0 con 0x7fd448159c30 2023-10-06T04:54:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:30.617+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4480cec80 con 0x7fd448159c30 2023-10-06T04:54:31.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.270+0000 7fd42f7fe700 1 -- 172.21.15.3:0/190794907 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd438054be0 con 0x7fd448159c30 2023-10-06T04:54:31.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd434066f30 msgr2=0x7fd4340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:31.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd434066f30 0x7fd4340693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd43c006f00 tx=0x7fd43c008040 comp rx=0 tx=0).stop 2023-10-06T04:54:31.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd448159c30 msgr2=0x7fd4480c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:31.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd448159c30 0x7fd4480c86a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd4380073c0 tx=0x7fd438001e50 comp rx=0 tx=0).stop 2023-10-06T04:54:31.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 shutdown_connections 2023-10-06T04:54:31.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd434066f30 0x7fd4340693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44815aaf0 0x7fd4480c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd448159c30 0x7fd4480c86a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 --2- 172.21.15.3:0/190794907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd44815a000 0x7fd4480c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.275+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 >> 172.21.15.3:0/190794907 conn(0x7fd4480675c0 msgr2=0x7fd44815c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:31.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.276+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 shutdown_connections 2023-10-06T04:54:31.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.276+0000 7fd44dd72700 1 -- 172.21.15.3:0/190794907 wait complete. 2023-10-06T04:54:31.279 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:54:31.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:54:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.423+0000 7f8b6de11700 1 Processor -- start 2023-10-06T04:54:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.423+0000 7f8b6de11700 1 -- start start 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.423+0000 7f8b6de11700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b68159590 0x7f8b680cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.423+0000 7f8b6de11700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b680cc550 con 0x7f8b680ca820 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.423+0000 7f8b6de11700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6815e320 con 0x7f8b68159960 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.423+0000 7f8b6de11700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b680c8bf0 con 0x7f8b68159590 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b677fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b68159590 0x7f8b680cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b67fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8b680ca820 0x7f8b680c9080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57506/0 (socket says 172.21.15.3:57506) 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b67fff700 1 -- 172.21.15.3:0/3765831629 learned_addr learned my addr 172.21.15.3:0/3765831629 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b677fe700 1 -- 172.21.15.3:0/3765831629 >> v1:172.21.15.153:6789/0 conn(0x7f8b68159960 legacy=0x7f8b680ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b677fe700 1 -- 172.21.15.3:0/3765831629 >> v1:172.21.15.3:6789/0 conn(0x7f8b680ca820 legacy=0x7f8b680c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b677fe700 1 -- 172.21.15.3:0/3765831629 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6815da80 con 0x7f8b68159590 2023-10-06T04:54:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b677fe700 1 --2- 172.21.15.3:0/3765831629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b68159590 0x7f8b680cc990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8b580098e0 tx=0x7f8b580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660cf1241f24d154 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b667fc700 1 -- 172.21.15.3:0/3765831629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58009e80 con 0x7f8b68159590 2023-10-06T04:54:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b667fc700 1 -- 172.21.15.3:0/3765831629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5800b820 con 0x7f8b68159590 2023-10-06T04:54:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.424+0000 7f8b667fc700 1 -- 172.21.15.3:0/3765831629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800bba0 con 0x7f8b68159590 2023-10-06T04:54:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.425+0000 7f8b6de11700 1 -- 172.21.15.3:0/3765831629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b68159590 msgr2=0x7f8b680cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.425+0000 7f8b6de11700 1 --2- 172.21.15.3:0/3765831629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b68159590 0x7f8b680cc990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8b580098e0 tx=0x7f8b580091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.425+0000 7f8b6de11700 1 -- 172.21.15.3:0/3765831629 shutdown_connections 2023-10-06T04:54:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.425+0000 7f8b6de11700 1 --2- 172.21.15.3:0/3765831629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b68159590 0x7f8b680cc990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.425+0000 7f8b6de11700 1 -- 172.21.15.3:0/3765831629 >> 172.21.15.3:0/3765831629 conn(0x7f8b680675c0 msgr2=0x7f8b680679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.425+0000 7f8b6de11700 1 -- 172.21.15.3:0/3765831629 shutdown_connections 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.425+0000 7f8b6de11700 1 -- 172.21.15.3:0/3765831629 wait complete. 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.426+0000 7f8b6de11700 1 Processor -- start 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.426+0000 7f8b6de11700 1 -- start start 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b6de11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b680ca820 0x7f8b6815c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b6de11700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b68159590 0x7f8b681509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b6de11700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b68159960 0x7f8b68150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b6de11700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68150800 con 0x7f8b680ca820 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b6de11700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6815cc00 con 0x7f8b68159590 2023-10-06T04:54:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b6de11700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68152140 con 0x7f8b68159960 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b680ca820 0x7f8b6815c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b680ca820 0x7f8b6815c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54708/0 (socket says 172.21.15.3:54708) 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 -- 172.21.15.3:0/634641479 learned_addr learned my addr 172.21.15.3:0/634641479 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 -- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b68159960 msgr2=0x7f8b68150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b68159960 0x7f8b68150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 -- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b68159590 msgr2=0x7f8b681509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b68159590 0x7f8b681509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 -- 172.21.15.3:0/634641479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b58009420 con 0x7f8b680ca820 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.427+0000 7f8b677fe700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b680ca820 0x7f8b6815c6c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8b68068ef0 tx=0x7f8b58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.428+0000 7f8b4f7fe700 1 -- 172.21.15.3:0/634641479 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800bae0 con 0x7f8b680ca820 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.428+0000 7f8b4f7fe700 1 -- 172.21.15.3:0/634641479 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b58020070 con 0x7f8b680ca820 2023-10-06T04:54:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.428+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6816a850 con 0x7f8b680ca820 2023-10-06T04:54:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.428+0000 7f8b4f7fe700 1 -- 172.21.15.3:0/634641479 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5801b6a0 con 0x7f8b680ca820 2023-10-06T04:54:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.428+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b68151810 con 0x7f8b680ca820 2023-10-06T04:54:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.429+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6806e970 con 0x7f8b680ca820 2023-10-06T04:54:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.429+0000 7f8b4f7fe700 1 -- 172.21.15.3:0/634641479 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b58022030 con 0x7f8b680ca820 2023-10-06T04:54:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.429+0000 7f8b4f7fe700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b54066f30 0x7f8b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.430+0000 7f8b4f7fe700 1 -- 172.21.15.3:0/634641479 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f8b58086540 con 0x7f8b680ca820 2023-10-06T04:54:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.430+0000 7f8b66ffd700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b54066f30 0x7f8b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.431+0000 7f8b66ffd700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b54066f30 0x7f8b540693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8b68152de0 tx=0x7f8b5c006c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:31.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.434+0000 7f8b4f7fe700 1 -- 172.21.15.3:0/634641479 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b58053770 con 0x7f8b680ca820 2023-10-06T04:54:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:31.629+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b6806e970 con 0x7f8b680ca820 2023-10-06T04:54:32.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.277+0000 7f8b4f7fe700 1 -- 172.21.15.3:0/634641479 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b58056d90 con 0x7f8b680ca820 2023-10-06T04:54:32.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.281+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b54066f30 msgr2=0x7f8b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.281+0000 7f8b6de11700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b54066f30 0x7f8b540693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8b68152de0 tx=0x7f8b5c006c60 comp rx=0 tx=0).stop 2023-10-06T04:54:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.281+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b680ca820 msgr2=0x7f8b6815c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.281+0000 7f8b6de11700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b680ca820 0x7f8b6815c6c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8b68068ef0 tx=0x7f8b58016000 comp rx=0 tx=0).stop 2023-10-06T04:54:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.282+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 shutdown_connections 2023-10-06T04:54:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.282+0000 7f8b6de11700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b54066f30 0x7f8b540693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.282+0000 7f8b6de11700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b68159960 0x7f8b68150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.282+0000 7f8b6de11700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b680ca820 0x7f8b6815c6c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.282+0000 7f8b6de11700 1 --2- 172.21.15.3:0/634641479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b68159590 0x7f8b681509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.282+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 >> 172.21.15.3:0/634641479 conn(0x7f8b680675c0 msgr2=0x7f8b6806a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.283+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 shutdown_connections 2023-10-06T04:54:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.283+0000 7f8b6de11700 1 -- 172.21.15.3:0/634641479 wait complete. 2023-10-06T04:54:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T04:54:32.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:54:32.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:54:32.347 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-06T04:54:32.348 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-06T04:54:32.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-06T04:54:32.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:54:32.351 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-06T04:54:32.352 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.353 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-06T04:54:32.354 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-06T04:54:32.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-06T04:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:54:32.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:54:32.358 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T04:54:32.359 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:54:32.360 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:54:32.361 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-10-06T04:54:32.361 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-10-06T04:54:32.361 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:54:32.361 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-10-06T04:54:32.361 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.361 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:54:32.362 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:54:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-10-06T04:54:32.364 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.365 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.366 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-10-06T04:54:32.367 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-10-06T04:54:32.368 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-10-06T04:54:32.369 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:54:32.370 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-10-06T04:54:32.371 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:54:32.372 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-10-06T04:54:32.373 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-10-06T04:54:32.374 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-10-06T04:54:32.374 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:54:32.374 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:54:32.374 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:54:32.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:54:32.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.497+0000 7fc24f378700 1 Processor -- start 2023-10-06T04:54:32.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.497+0000 7fc24f378700 1 -- start start 2023-10-06T04:54:32.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24f378700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480af290 0x7fc2480ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24f378700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc2480af660 0x7fc2480ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24f378700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2480cb190 con 0x7fc2480cd190 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24f378700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc248159950 con 0x7fc2480af660 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24f378700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480cd6a0 con 0x7fc2480af290 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24eb77700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc2480cd190 0x7fc2480c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56686/0 (socket says 172.21.15.3:56686) 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24eb77700 1 -- 172.21.15.3:0/229605131 learned_addr learned my addr 172.21.15.3:0/229605131 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24db75700 1 --2- 172.21.15.3:0/229605131 >> v2:172.21.15.153:3300/0 conn(0x7fc2480af660 0x7fc2480ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24e376700 1 --2- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480af290 0x7fc2480ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24d374700 1 -- 172.21.15.3:0/229605131 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658034876 0 0) 0x7fc2480cb190 con 0x7fc2480cd190 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.498+0000 7fc24d374700 1 -- 172.21.15.3:0/229605131 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc230003570 con 0x7fc2480cd190 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24db75700 1 -- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480af290 msgr2=0x7fc2480ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24db75700 1 --2- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480af290 0x7fc2480ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24db75700 1 -- 172.21.15.3:0/229605131 >> v1:172.21.15.3:6789/0 conn(0x7fc2480cd190 legacy=0x7fc2480c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24db75700 1 -- 172.21.15.3:0/229605131 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480cb3f0 con 0x7fc2480af660 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24db75700 1 --2- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480ccc50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0098d0 tx=0x7fc23c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dba61e3d7fa54d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24d374700 1 -- 172.21.15.3:0/229605131 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c005c10 con 0x7fc2480af660 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24d374700 1 -- 172.21.15.3:0/229605131 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c005d70 con 0x7fc2480af660 2023-10-06T04:54:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24d374700 1 -- 172.21.15.3:0/229605131 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c006240 con 0x7fc2480af660 2023-10-06T04:54:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24f378700 1 -- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 msgr2=0x7fc2480ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.499+0000 7fc24f378700 1 --2- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480ccc50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0098d0 tx=0x7fc23c009120 comp rx=0 tx=0).stop 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.500+0000 7fc24f378700 1 -- 172.21.15.3:0/229605131 shutdown_connections 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.500+0000 7fc24f378700 1 --2- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480af290 0x7fc2480ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.500+0000 7fc24f378700 1 --2- 172.21.15.3:0/229605131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480ccc50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.500+0000 7fc24f378700 1 -- 172.21.15.3:0/229605131 >> 172.21.15.3:0/229605131 conn(0x7fc248089dd0 msgr2=0x7fc2480996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.500+0000 7fc24f378700 1 -- 172.21.15.3:0/229605131 shutdown_connections 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.500+0000 7fc24f378700 1 -- 172.21.15.3:0/229605131 wait complete. 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 Processor -- start 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 -- start start 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2480af290 0x7fc2480ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480cd190 0x7fc24814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480cf2c0 con 0x7fc2480af290 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480ce6b0 con 0x7fc2480af660 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.501+0000 7fc24f378700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480ce990 con 0x7fc2480cd190 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2480af290 0x7fc2480ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2480af290 0x7fc2480ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36696/0 (socket says 172.21.15.3:36696) 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 -- 172.21.15.3:0/3072693894 learned_addr learned my addr 172.21.15.3:0/3072693894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24db75700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24eb77700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480cd190 0x7fc24814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 -- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480cd190 msgr2=0x7fc24814f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480cd190 0x7fc24814f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 -- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 msgr2=0x7fc2480b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 -- 172.21.15.3:0/3072693894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0093c0 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24db75700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24e376700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2480af290 0x7fc2480ca0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc23800ec80 tx=0x7fc238009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc2467fc700 1 -- 172.21.15.3:0/3072693894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800c9d0 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc2467fc700 1 -- 172.21.15.3:0/3072693894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800cb30 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2480cec70 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc2467fc700 1 -- 172.21.15.3:0/3072693894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238013470 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.502+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2480ca600 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.503+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24804c380 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.504+0000 7fc2467fc700 1 -- 172.21.15.3:0/3072693894 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc238013860 con 0x7fc2480af290 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.504+0000 7fc2467fc700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc234066f30 0x7fc2340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.504+0000 7fc2467fc700 1 -- 172.21.15.3:0/3072693894 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc238084ae0 con 0x7fc2480af290 2023-10-06T04:54:32.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.504+0000 7fc24db75700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc234066f30 0x7fc2340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.505+0000 7fc24db75700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc234066f30 0x7fc2340693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2480d1240 tx=0x7fc23c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:32.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.508+0000 7fc2467fc700 1 -- 172.21.15.3:0/3072693894 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc238051ef0 con 0x7fc2480af290 2023-10-06T04:54:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.704+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc248159da0 con 0x7fc2480af290 2023-10-06T04:54:32.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.707+0000 7fc2467fc700 1 -- 172.21.15.3:0/3072693894 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc2380170a0 con 0x7fc2480af290 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc234066f30 msgr2=0x7fc2340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc234066f30 0x7fc2340693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2480d1240 tx=0x7fc23c016000 comp rx=0 tx=0).stop 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2480af290 msgr2=0x7fc2480ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2480af290 0x7fc2480ca0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc23800ec80 tx=0x7fc238009120 comp rx=0 tx=0).stop 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 shutdown_connections 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc234066f30 0x7fc2340693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2480cd190 0x7fc24814f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2480af290 0x7fc2480ca0c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 --2- 172.21.15.3:0/3072693894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2480af660 0x7fc2480b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 >> 172.21.15.3:0/3072693894 conn(0x7fc248089dd0 msgr2=0x7fc248158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 shutdown_connections 2023-10-06T04:54:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.709+0000 7fc24f378700 1 -- 172.21.15.3:0/3072693894 wait complete. 2023-10-06T04:54:32.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2023-10-06T04:54:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.840+0000 7f992e48a700 1 Processor -- start 2023-10-06T04:54:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.841+0000 7f992e48a700 1 -- start start 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.841+0000 7f992e48a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99280d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.841+0000 7f992e48a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99280d06a0 0x7f99280b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.841+0000 7f992e48a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99280cf6c0 con 0x7f992815c8f0 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.841+0000 7f992e48a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99280cfb50 con 0x7f99280d06a0 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.841+0000 7f992e48a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99280d0b20 con 0x7f992815ccc0 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99280d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f992ca27700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99280d06a0 0x7f99280b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99280d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:41374/0 (socket says 172.21.15.3:41374) 2023-10-06T04:54:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 -- 172.21.15.3:0/1906430919 learned_addr learned my addr 172.21.15.3:0/1906430919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 -- 172.21.15.3:0/1906430919 >> v2:172.21.15.153:3300/0 conn(0x7f99280d06a0 msgr2=0x7f99280b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 --2- 172.21.15.3:0/1906430919 >> v2:172.21.15.153:3300/0 conn(0x7f99280d06a0 0x7f99280b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 -- 172.21.15.3:0/1906430919 >> v1:172.21.15.153:6789/0 conn(0x7f992815c8f0 legacy=0x7f992815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 -- 172.21.15.3:0/1906430919 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99280ceab0 con 0x7f992815ccc0 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f99277fe700 1 --2- 172.21.15.3:0/1906430919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99280d0160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f99180099b0 tx=0x7f9918009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade5d38e5fb3d242 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f9926ffd700 1 -- 172.21.15.3:0/1906430919 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918009ea0 con 0x7f992815ccc0 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f9926ffd700 1 -- 172.21.15.3:0/1906430919 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991800b860 con 0x7f992815ccc0 2023-10-06T04:54:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.842+0000 7f9926ffd700 1 -- 172.21.15.3:0/1906430919 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800bc00 con 0x7f992815ccc0 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 -- 172.21.15.3:0/1906430919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 msgr2=0x7f99280d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 --2- 172.21.15.3:0/1906430919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99280d0160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f99180099b0 tx=0x7f9918009250 comp rx=0 tx=0).stop 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 -- 172.21.15.3:0/1906430919 shutdown_connections 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 --2- 172.21.15.3:0/1906430919 >> v2:172.21.15.153:3300/0 conn(0x7f99280d06a0 0x7f99280b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 --2- 172.21.15.3:0/1906430919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99280d0160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 -- 172.21.15.3:0/1906430919 >> 172.21.15.3:0/1906430919 conn(0x7f9928070310 msgr2=0x7f9928070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 -- 172.21.15.3:0/1906430919 shutdown_connections 2023-10-06T04:54:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.843+0000 7f992e48a700 1 -- 172.21.15.3:0/1906430919 wait complete. 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.844+0000 7f992e48a700 1 Processor -- start 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.844+0000 7f992e48a700 1 -- start start 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f992e48a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99280d06a0 0x7f99280c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f992e48a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f992815c8f0 0x7f99280c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f992e48a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99281588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f992e48a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281595a0 con 0x7f99280d06a0 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f992e48a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280c52c0 con 0x7f992815c8f0 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f992e48a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99280c55a0 con 0x7f992815ccc0 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99280d06a0 0x7f99280c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99280d06a0 0x7f99280c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36706/0 (socket says 172.21.15.3:36706) 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 -- 172.21.15.3:0/3182376965 learned_addr learned my addr 172.21.15.3:0/3182376965 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f992ca27700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99281588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f99277fe700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f992815c8f0 0x7f99280c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 -- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 msgr2=0x7f99281588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99281588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 -- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f992815c8f0 msgr2=0x7f99280c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f992815c8f0 0x7f99280c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.845+0000 7f9927fff700 1 -- 172.21.15.3:0/3182376965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180094f0 con 0x7f99280d06a0 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.846+0000 7f9927fff700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99280d06a0 0x7f99280c7190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f991000c590 tx=0x7f991000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.846+0000 7f991ffff700 1 -- 172.21.15.3:0/3182376965 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910011590 con 0x7f99280d06a0 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.846+0000 7f991ffff700 1 -- 172.21.15.3:0/3182376965 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910011710 con 0x7f99280d06a0 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.846+0000 7f991ffff700 1 -- 172.21.15.3:0/3182376965 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910011ad0 con 0x7f99280d06a0 2023-10-06T04:54:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.846+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99280c5880 con 0x7f99280d06a0 2023-10-06T04:54:32.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.846+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9928159880 con 0x7f99280d06a0 2023-10-06T04:54:32.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.847+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9928071680 con 0x7f99280d06a0 2023-10-06T04:54:32.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.847+0000 7f991ffff700 1 -- 172.21.15.3:0/3182376965 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f991001b3b0 con 0x7f99280d06a0 2023-10-06T04:54:32.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.847+0000 7f991ffff700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9908066f30 0x7f99080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:32.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.848+0000 7f991ffff700 1 -- 172.21.15.3:0/3182376965 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9910083800 con 0x7f99280d06a0 2023-10-06T04:54:32.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.848+0000 7f99277fe700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9908066f30 0x7f99080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:32.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.848+0000 7f99277fe700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9908066f30 0x7f99080693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f99180099b0 tx=0x7f9918017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:32.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:32.852+0000 7f991ffff700 1 -- 172.21.15.3:0/3182376965 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9910050c10 con 0x7f99280d06a0 2023-10-06T04:54:33.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.124+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) v1 -- 0x7f99280b35a0 con 0x7f99280d06a0 2023-10-06T04:54:33.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.279+0000 7f991ffff700 1 -- 172.21.15.3:0/3182376965 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v134) v1 ==== 150+0+0 (secure 0 0 0) 0x7f9910017070 con 0x7f99280d06a0 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9908066f30 msgr2=0x7f99080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9908066f30 0x7f99080693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f99180099b0 tx=0x7f9918017000 comp rx=0 tx=0).stop 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99280d06a0 msgr2=0x7f99280c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99280d06a0 0x7f99280c7190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f991000c590 tx=0x7f991000c8a0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 shutdown_connections 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9908066f30 0x7f99080693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f992815ccc0 0x7f99281588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99280d06a0 0x7f99280c7190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 --2- 172.21.15.3:0/3182376965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f992815c8f0 0x7f99280c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 >> 172.21.15.3:0/3182376965 conn(0x7f9928070310 msgr2=0x7f9928155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 shutdown_connections 2023-10-06T04:54:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.281+0000 7f992e48a700 1 -- 172.21.15.3:0/3182376965 wait complete. 2023-10-06T04:54:33.295 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-10-06T04:54:33.295 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) That the MDS will not let a client sit above mds_max_caps_per_client caps. 2023-10-06T04:54:33.295 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T04:54:33.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2023-10-06T04:54:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.413+0000 7f853aa81700 1 Processor -- start 2023-10-06T04:54:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.413+0000 7f853aa81700 1 -- start start 2023-10-06T04:54:33.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.413+0000 7f853aa81700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8534156110 0x7f853406aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.413+0000 7f853aa81700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85341564e0 0x7f853406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.413+0000 7f853aa81700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85340c7410 con 0x7f85340c68d0 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.413+0000 7f853aa81700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340acbb0 con 0x7f85341564e0 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.413+0000 7f853aa81700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340c7640 con 0x7f8534156110 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f8533fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85341564e0 0x7f853406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f853901e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f85340c68d0 0x7f85340c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56696/0 (socket says 172.21.15.3:56696) 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f853901e700 1 -- 172.21.15.3:0/1603878868 learned_addr learned my addr 172.21.15.3:0/1603878868 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f853881d700 1 --2- 172.21.15.3:0/1603878868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8534156110 0x7f853406aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f8533fff700 1 -- 172.21.15.3:0/1603878868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8534156110 msgr2=0x7f853406aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f8533fff700 1 --2- 172.21.15.3:0/1603878868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8534156110 0x7f853406aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f8533fff700 1 -- 172.21.15.3:0/1603878868 >> v1:172.21.15.3:6789/0 conn(0x7f85340c68d0 legacy=0x7f85340c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f8533fff700 1 -- 172.21.15.3:0/1603878868 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340c7920 con 0x7f85341564e0 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f85337fe700 1 -- 172.21.15.3:0/1603878868 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663133331 0 0) 0x7f85340c7410 con 0x7f85340c68d0 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f8533fff700 1 --2- 172.21.15.3:0/1603878868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f853406afd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f85200098e0 tx=0x7f85200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d603000b693551 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f85337fe700 1 -- 172.21.15.3:0/1603878868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000b640 con 0x7f85341564e0 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f85337fe700 1 -- 172.21.15.3:0/1603878868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852000b7a0 con 0x7f85341564e0 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.414+0000 7f85337fe700 1 -- 172.21.15.3:0/1603878868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000bb40 con 0x7f85341564e0 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 -- 172.21.15.3:0/1603878868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 msgr2=0x7f853406afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 --2- 172.21.15.3:0/1603878868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f853406afd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f85200098e0 tx=0x7f85200091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 -- 172.21.15.3:0/1603878868 shutdown_connections 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 --2- 172.21.15.3:0/1603878868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8534156110 0x7f853406aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 --2- 172.21.15.3:0/1603878868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f853406afd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 -- 172.21.15.3:0/1603878868 >> 172.21.15.3:0/1603878868 conn(0x7f853406c370 msgr2=0x7f853406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:33.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 -- 172.21.15.3:0/1603878868 shutdown_connections 2023-10-06T04:54:33.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.415+0000 7f853aa81700 1 -- 172.21.15.3:0/1603878868 wait complete. 2023-10-06T04:54:33.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.416+0000 7f853aa81700 1 Processor -- start 2023-10-06T04:54:33.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.416+0000 7f853aa81700 1 -- start start 2023-10-06T04:54:33.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853aa81700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85340c68d0 0x7f85340b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:33.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853aa81700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8534156110 0x7f85340afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853aa81700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f8534151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853aa81700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340b01e0 con 0x7f85341564e0 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853aa81700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534151e80 con 0x7f8534156110 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853aa81700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853415b200 con 0x7f85340c68d0 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f8534151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f8534151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36722/0 (socket says 172.21.15.3:36722) 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 -- 172.21.15.3:0/1414011919 learned_addr learned my addr 172.21.15.3:0/1414011919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853881d700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85340c68d0 0x7f85340b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f8533fff700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8534156110 0x7f85340afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 -- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85340c68d0 msgr2=0x7f85340b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:33.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85340c68d0 0x7f85340b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 -- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8534156110 msgr2=0x7f85340afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8534156110 0x7f85340afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 -- 172.21.15.3:0/1414011919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8520009420 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.417+0000 7f853901e700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f8534151940 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f852c00ccc0 tx=0x7f852c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.418+0000 7f8530ff9700 1 -- 172.21.15.3:0/1414011919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00aad0 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.418+0000 7f8530ff9700 1 -- 172.21.15.3:0/1414011919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c00ac30 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.418+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85340b0dc0 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.418+0000 7f8530ff9700 1 -- 172.21.15.3:0/1414011919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c0070f0 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.418+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85340b1400 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.419+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534071680 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.419+0000 7f8530ff9700 1 -- 172.21.15.3:0/1414011919 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f852c0074c0 con 0x7f85341564e0 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.419+0000 7f8530ff9700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8518066f30 0x7f85180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:33.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.419+0000 7f8530ff9700 1 -- 172.21.15.3:0/1414011919 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f852c0847a0 con 0x7f85341564e0 2023-10-06T04:54:33.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.420+0000 7f853881d700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8518066f30 0x7f85180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:33.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.420+0000 7f853881d700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8518066f30 0x7f85180693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8528009d40 tx=0x7f8528008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:33.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.424+0000 7f8530ff9700 1 -- 172.21.15.3:0/1414011919 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f852c051c30 con 0x7f85341564e0 2023-10-06T04:54:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:33.695+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) v1 -- 0x7f85340c7c00 con 0x7f85341564e0 2023-10-06T04:54:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.285+0000 7f8530ff9700 1 -- 172.21.15.3:0/1414011919 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v135) v1 ==== 151+0+0 (secure 0 0 0) 0x7f852c055250 con 0x7f85341564e0 2023-10-06T04:54:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8518066f30 msgr2=0x7f85180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8518066f30 0x7f85180693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8528009d40 tx=0x7f8528008040 comp rx=0 tx=0).stop 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 msgr2=0x7f8534151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f8534151940 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f852c00ccc0 tx=0x7f852c001a80 comp rx=0 tx=0).stop 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 shutdown_connections 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8518066f30 0x7f85180693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85341564e0 0x7f8534151940 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85340c68d0 0x7f85340b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 --2- 172.21.15.3:0/1414011919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8534156110 0x7f85340afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 >> 172.21.15.3:0/1414011919 conn(0x7f853406c370 msgr2=0x7f853406ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 shutdown_connections 2023-10-06T04:54:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.288+0000 7f853aa81700 1 -- 172.21.15.3:0/1414011919 wait complete. 2023-10-06T04:54:34.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:54:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.427+0000 7f69fd511700 1 Processor -- start 2023-10-06T04:54:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.427+0000 7f69fd511700 1 -- start start 2023-10-06T04:54:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69fd511700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f80b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69fd511700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f69f80caa10 0x7f69f815a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69fd511700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69f80cb2b0 0x7f69f80cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69fd511700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f8159a80 con 0x7f69f80cb2b0 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69fd511700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f815a000 con 0x7f69f80caa10 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69fd511700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f815a9d0 con 0x7f69f80b2450 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69f77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69f80cb2b0 0x7f69f80cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69f77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69f80cb2b0 0x7f69f80cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36730/0 (socket says 172.21.15.3:36730) 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69f77fe700 1 -- 172.21.15.3:0/2549533473 learned_addr learned my addr 172.21.15.3:0/2549533473 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69f6ffd700 1 --2- 172.21.15.3:0/2549533473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f80b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.428+0000 7f69f67fc700 1 --2- 172.21.15.3:0/2549533473 >> v2:172.21.15.153:3300/0 conn(0x7f69f80caa10 0x7f69f815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f77fe700 1 -- 172.21.15.3:0/2549533473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 msgr2=0x7f69f80b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f77fe700 1 --2- 172.21.15.3:0/2549533473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f80b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f77fe700 1 -- 172.21.15.3:0/2549533473 >> v2:172.21.15.153:3300/0 conn(0x7f69f80caa10 msgr2=0x7f69f815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f77fe700 1 --2- 172.21.15.3:0/2549533473 >> v2:172.21.15.153:3300/0 conn(0x7f69f80caa10 0x7f69f815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f77fe700 1 -- 172.21.15.3:0/2549533473 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80cfbb0 con 0x7f69f80cb2b0 2023-10-06T04:54:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f77fe700 1 --2- 172.21.15.3:0/2549533473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f80cccd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f69f80894c0 tx=0x7f69e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99510d404b383ef4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f5ffb700 1 -- 172.21.15.3:0/2549533473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800e070 con 0x7f69f80cb2b0 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f5ffb700 1 -- 172.21.15.3:0/2549533473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e800ba50 con 0x7f69f80cb2b0 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69f5ffb700 1 -- 172.21.15.3:0/2549533473 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80092e0 con 0x7f69f80cb2b0 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 -- 172.21.15.3:0/2549533473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 msgr2=0x7f69f80cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 --2- 172.21.15.3:0/2549533473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f80cccd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f69f80894c0 tx=0x7f69e8009840 comp rx=0 tx=0).stop 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 -- 172.21.15.3:0/2549533473 shutdown_connections 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 --2- 172.21.15.3:0/2549533473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f80cccd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 --2- 172.21.15.3:0/2549533473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f80b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 --2- 172.21.15.3:0/2549533473 >> v2:172.21.15.153:3300/0 conn(0x7f69f80caa10 0x7f69f815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 -- 172.21.15.3:0/2549533473 >> 172.21.15.3:0/2549533473 conn(0x7f69f806ecc0 msgr2=0x7f69f806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 -- 172.21.15.3:0/2549533473 shutdown_connections 2023-10-06T04:54:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.429+0000 7f69fd511700 1 -- 172.21.15.3:0/2549533473 wait complete. 2023-10-06T04:54:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 Processor -- start 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 -- start start 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f80b3070 0x7f69f814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f814f070 con 0x7f69f80cb2b0 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80b6010 con 0x7f69f80b3070 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69fd511700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f815e630 con 0x7f69f80b2450 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69f67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69f67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36738/0 (socket says 172.21.15.3:36738) 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.431+0000 7f69f67fc700 1 -- 172.21.15.3:0/4250100393 learned_addr learned my addr 172.21.15.3:0/4250100393 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f6ffd700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f77fe700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f80b3070 0x7f69f814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f67fc700 1 -- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 msgr2=0x7f69f815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f67fc700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f67fc700 1 -- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f80b3070 msgr2=0x7f69f814eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f67fc700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f80b3070 0x7f69f814eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f67fc700 1 -- 172.21.15.3:0/4250100393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e8009ae0 con 0x7f69f80cb2b0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69f67fc700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f815bb90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f69e000d860 tx=0x7f69e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69eeffd700 1 -- 172.21.15.3:0/4250100393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0011590 con 0x7f69f80cb2b0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69eeffd700 1 -- 172.21.15.3:0/4250100393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e00116f0 con 0x7f69f80cb2b0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69eeffd700 1 -- 172.21.15.3:0/4250100393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0011a90 con 0x7f69f80cb2b0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f80b5d30 con 0x7f69f80cb2b0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.432+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f815e2c0 con 0x7f69f80cb2b0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.433+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f804c380 con 0x7f69f80cb2b0 2023-10-06T04:54:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.433+0000 7f69eeffd700 1 -- 172.21.15.3:0/4250100393 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69e001c460 con 0x7f69f80cb2b0 2023-10-06T04:54:34.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.434+0000 7f69eeffd700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69d8066f30 0x7f69d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.434+0000 7f69eeffd700 1 -- 172.21.15.3:0/4250100393 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f69e0084330 con 0x7f69f80cb2b0 2023-10-06T04:54:34.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.434+0000 7f69f6ffd700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69d8066f30 0x7f69d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.435+0000 7f69f6ffd700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69d8066f30 0x7f69d80693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69f00061f0 tx=0x7f69f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.438+0000 7f69eeffd700 1 -- 172.21.15.3:0/4250100393 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69e00551d0 con 0x7f69f80cb2b0 2023-10-06T04:54:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.633+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f69f815e770 con 0x7f69f80cb2b0 2023-10-06T04:54:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.637+0000 7f69eeffd700 1 -- 172.21.15.3:0/4250100393 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f69e0054d60 con 0x7f69f80cb2b0 2023-10-06T04:54:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69d8066f30 msgr2=0x7f69d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69d8066f30 0x7f69d80693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69f00061f0 tx=0x7f69f0008040 comp rx=0 tx=0).stop 2023-10-06T04:54:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 msgr2=0x7f69f815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f815bb90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f69e000d860 tx=0x7f69e0008040 comp rx=0 tx=0).stop 2023-10-06T04:54:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 shutdown_connections 2023-10-06T04:54:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f69d8066f30 0x7f69d80693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f80b3070 0x7f69f814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69f80b2450 0x7f69f815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 --2- 172.21.15.3:0/4250100393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69f80cb2b0 0x7f69f815bb90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 >> 172.21.15.3:0/4250100393 conn(0x7f69f806ecc0 msgr2=0x7f69f80b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 shutdown_connections 2023-10-06T04:54:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.639+0000 7f69fd511700 1 -- 172.21.15.3:0/4250100393 wait complete. 2023-10-06T04:54:34.653 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:54:34.653 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:54:34.690 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:54:34.737 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:54:34.737 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:54:34.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:34.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.854+0000 7fbf65b01700 1 Processor -- start 2023-10-06T04:54:34.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.854+0000 7fbf65b01700 1 -- start start 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf65b01700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf600cb780 0x7fbf60158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf65b01700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600cf130 con 0x7fbf600cb3b0 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf65b01700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600ce520 con 0x7fbf60159500 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf65b01700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf600ce800 con 0x7fbf600cb780 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf5ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf600cb780 0x7fbf60158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf64aff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbf600cb3b0 0x7fbf600cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56704/0 (socket says 172.21.15.3:56704) 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf64aff700 1 -- 172.21.15.3:0/679577982 learned_addr learned my addr 172.21.15.3:0/679577982 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf5ffff700 1 -- 172.21.15.3:0/679577982 >> v1:172.21.15.153:6789/0 conn(0x7fbf60159500 legacy=0x7fbf600cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf5ffff700 1 -- 172.21.15.3:0/679577982 >> v1:172.21.15.3:6789/0 conn(0x7fbf600cb3b0 legacy=0x7fbf600cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf5ffff700 1 -- 172.21.15.3:0/679577982 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf600ceae0 con 0x7fbf600cb780 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.855+0000 7fbf5f7fe700 1 -- 172.21.15.3:0/679577982 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24300702 0 0) 0x7fbf600cf130 con 0x7fbf600cb3b0 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf5ffff700 1 --2- 172.21.15.3:0/679577982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb780 0x7fbf60158fc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbf540098e0 tx=0x7fbf540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ade732b993d2bef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf5f7fe700 1 -- 172.21.15.3:0/679577982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54009e80 con 0x7fbf600cb780 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf5f7fe700 1 -- 172.21.15.3:0/679577982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400b820 con 0x7fbf600cb780 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf5f7fe700 1 -- 172.21.15.3:0/679577982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400bba0 con 0x7fbf600cb780 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf65b01700 1 -- 172.21.15.3:0/679577982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb780 msgr2=0x7fbf60158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf65b01700 1 --2- 172.21.15.3:0/679577982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb780 0x7fbf60158fc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbf540098e0 tx=0x7fbf540091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf65b01700 1 -- 172.21.15.3:0/679577982 shutdown_connections 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf65b01700 1 --2- 172.21.15.3:0/679577982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb780 0x7fbf60158fc0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf65b01700 1 -- 172.21.15.3:0/679577982 >> 172.21.15.3:0/679577982 conn(0x7fbf60070310 msgr2=0x7fbf60070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:34.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf65b01700 1 -- 172.21.15.3:0/679577982 shutdown_connections 2023-10-06T04:54:34.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.856+0000 7fbf65b01700 1 -- 172.21.15.3:0/679577982 wait complete. 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.857+0000 7fbf65b01700 1 Processor -- start 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf65b01700 1 -- start start 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf65b01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb3b0 0x7fbf6016cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf65b01700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf600cb780 0x7fbf600d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf65b01700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf60159500 0x7fbf600d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf65b01700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6016a9a0 con 0x7fbf600cb3b0 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf65b01700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6016a070 con 0x7fbf600cb780 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf65b01700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6016a350 con 0x7fbf60159500 2023-10-06T04:54:34.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf64aff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb3b0 0x7fbf6016cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf64aff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb3b0 0x7fbf6016cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36752/0 (socket says 172.21.15.3:36752) 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf64aff700 1 -- 172.21.15.3:0/318145932 learned_addr learned my addr 172.21.15.3:0/318145932 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.858+0000 7fbf5ffff700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf600cb780 0x7fbf600d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf65300700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf60159500 0x7fbf600d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf64aff700 1 -- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf60159500 msgr2=0x7fbf600d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf64aff700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf60159500 0x7fbf600d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf64aff700 1 -- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf600cb780 msgr2=0x7fbf600d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf64aff700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf600cb780 0x7fbf600d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf64aff700 1 -- 172.21.15.3:0/318145932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf54009420 con 0x7fbf600cb3b0 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf5ffff700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf600cb780 0x7fbf600d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf64aff700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb3b0 0x7fbf6016cd40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800cd80 tx=0x7fbf58001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf5cff9700 1 -- 172.21.15.3:0/318145932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800ab10 con 0x7fbf600cb3b0 2023-10-06T04:54:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf5cff9700 1 -- 172.21.15.3:0/318145932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5800ac70 con 0x7fbf600cb3b0 2023-10-06T04:54:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6016a630 con 0x7fbf600cb3b0 2023-10-06T04:54:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf5cff9700 1 -- 172.21.15.3:0/318145932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580070f0 con 0x7fbf600cb3b0 2023-10-06T04:54:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.859+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6016b200 con 0x7fbf600cb3b0 2023-10-06T04:54:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.860+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6016ac80 con 0x7fbf600cb3b0 2023-10-06T04:54:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.860+0000 7fbf5cff9700 1 -- 172.21.15.3:0/318145932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf580074e0 con 0x7fbf600cb3b0 2023-10-06T04:54:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.861+0000 7fbf5cff9700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf4c066f30 0x7fbf4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.861+0000 7fbf5cff9700 1 -- 172.21.15.3:0/318145932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fbf580848c0 con 0x7fbf600cb3b0 2023-10-06T04:54:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.861+0000 7fbf5ffff700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf4c066f30 0x7fbf4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.862+0000 7fbf5ffff700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf4c066f30 0x7fbf4c0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf540030c0 tx=0x7fbf54016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:34.865+0000 7fbf5cff9700 1 -- 172.21.15.3:0/318145932 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf58051d50 con 0x7fbf600cb3b0 2023-10-06T04:54:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.102+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf6004c380 con 0x7fbf600cb3b0 2023-10-06T04:54:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.102+0000 7fbf5cff9700 1 -- 172.21.15.3:0/318145932 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fbf58055370 con 0x7fbf600cb3b0 2023-10-06T04:54:35.104 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:35.104 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/596033972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":596033972},{"type":"v1","addr":"172.21.15.153:6837","nonce":596033972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1359588707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1359588707},{"type":"v1","addr":"172.21.15.3:6834","nonce":1359588707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/616658407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":616658407},{"type":"v1","addr":"172.21.15.3:6835","nonce":616658407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3913286660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3913286660},{"type":"v1","addr":"172.21.15.153:6836","nonce":3913286660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf4c066f30 msgr2=0x7fbf4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf4c066f30 0x7fbf4c0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf540030c0 tx=0x7fbf54016000 comp rx=0 tx=0).stop 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb3b0 msgr2=0x7fbf6016cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb3b0 0x7fbf6016cd40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800cd80 tx=0x7fbf58001ae0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 shutdown_connections 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf4c066f30 0x7fbf4c0693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf60159500 0x7fbf600d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf600cb3b0 0x7fbf6016cd40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 --2- 172.21.15.3:0/318145932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf600cb780 0x7fbf600d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 >> 172.21.15.3:0/318145932 conn(0x7fbf60070310 msgr2=0x7fbf600b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 shutdown_connections 2023-10-06T04:54:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.104+0000 7fbf65b01700 1 -- 172.21.15.3:0/318145932 wait complete. 2023-10-06T04:54:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2023-10-06T04:54:35.118 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:54:35.118 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:54:35.118 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:54:35.118 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:54:35.118 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:54:35.118 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:54:35.119 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:54:35.119 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:54:35.119 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:54:35.119 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:54:35.119 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:54:35.119 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:54:35.119 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:54:35.120 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:54:35.120 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:54:35.120 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:54:35.127 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:54:35.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:54:35.128 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:54:35.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:54:35.131 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:54:35.132 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:54:35.132 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:54:35.132 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:54:35.170 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:54:35.170 DEBUG:teuthology.parallel:result is None 2023-10-06T04:54:35.170 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:54:35.171 DEBUG:teuthology.parallel:result is None 2023-10-06T04:54:35.174 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:54:35.175 DEBUG:teuthology.parallel:result is None 2023-10-06T04:54:35.175 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:54:35.175 DEBUG:teuthology.parallel:result is None 2023-10-06T04:54:35.176 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:54:35.246 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:54:35.247 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:54:35.259 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:54:35.259 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:54:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.300+0000 7f63d19e9700 1 Processor -- start 2023-10-06T04:54:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.300+0000 7f63d19e9700 1 -- start start 2023-10-06T04:54:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63d19e9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f63cc06d480 0x7f63cc1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63d19e9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63cc0b3da0 0x7f63cc157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63d19e9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc15e4c0 con 0x7f63cc0b39d0 2023-10-06T04:54:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63d19e9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc157c50 con 0x7f63cc0b3da0 2023-10-06T04:54:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63d19e9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc15d8b0 con 0x7f63cc06d480 2023-10-06T04:54:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63cb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63cc0b3da0 0x7f63cc157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63caffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f63cc0b39d0 0x7f63cc0cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56732/0 (socket says 172.21.15.3:56732) 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63caffd700 1 -- 172.21.15.3:0/1177845929 learned_addr learned my addr 172.21.15.3:0/1177845929 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63cb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63cc0b3da0 0x7f63cc157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36798/0 (socket says 172.21.15.3:36798) 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63ca7fc700 1 --2- 172.21.15.3:0/1177845929 >> v2:172.21.15.153:3300/0 conn(0x7f63cc06d480 0x7f63cc1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63cb7fe700 1 -- 172.21.15.3:0/1177845929 >> v2:172.21.15.153:3300/0 conn(0x7f63cc06d480 msgr2=0x7f63cc1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63cb7fe700 1 --2- 172.21.15.3:0/1177845929 >> v2:172.21.15.153:3300/0 conn(0x7f63cc06d480 0x7f63cc1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63cb7fe700 1 -- 172.21.15.3:0/1177845929 >> v1:172.21.15.3:6789/0 conn(0x7f63cc0b39d0 legacy=0x7f63cc0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63cb7fe700 1 -- 172.21.15.3:0/1177845929 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc15db90 con 0x7f63cc0b3da0 2023-10-06T04:54:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.301+0000 7f63cb7fe700 1 --2- 172.21.15.3:0/1177845929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc157710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f63cc15a180 tx=0x7f63c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5992c9e101bde084 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.302+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1177845929 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c400e070 con 0x7f63cc0b3da0 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.302+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1177845929 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c400bb60 con 0x7f63cc0b3da0 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.302+0000 7f63c9ffb700 1 -- 172.21.15.3:0/1177845929 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4012430 con 0x7f63cc0b3da0 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 -- 172.21.15.3:0/1177845929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 msgr2=0x7f63cc157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 --2- 172.21.15.3:0/1177845929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc157710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f63cc15a180 tx=0x7f63c4009780 comp rx=0 tx=0).stop 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 -- 172.21.15.3:0/1177845929 shutdown_connections 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 --2- 172.21.15.3:0/1177845929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc157710 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 --2- 172.21.15.3:0/1177845929 >> v2:172.21.15.153:3300/0 conn(0x7f63cc06d480 0x7f63cc1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 -- 172.21.15.3:0/1177845929 >> 172.21.15.3:0/1177845929 conn(0x7f63cc0ce2c0 msgr2=0x7f63cc0ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 -- 172.21.15.3:0/1177845929 shutdown_connections 2023-10-06T04:54:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.303+0000 7f63d19e9700 1 -- 172.21.15.3:0/1177845929 wait complete. 2023-10-06T04:54:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.304+0000 7f63d19e9700 1 Processor -- start 2023-10-06T04:54:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.304+0000 7f63d19e9700 1 -- start start 2023-10-06T04:54:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63d19e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc06d480 0x7f63cc14f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63d19e9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63cc0b39d0 0x7f63cc153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63d19e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63d19e9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc156280 con 0x7f63cc0b3da0 2023-10-06T04:54:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63d19e9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc15bb10 con 0x7f63cc06d480 2023-10-06T04:54:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63d19e9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc15af00 con 0x7f63cc0b39d0 2023-10-06T04:54:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc1543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36804/0 (socket says 172.21.15.3:36804) 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 -- 172.21.15.3:0/2409691421 learned_addr learned my addr 172.21.15.3:0/2409691421 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63caffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc06d480 0x7f63cc14f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63ca7fc700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63cc0b39d0 0x7f63cc153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 -- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63cc0b39d0 msgr2=0x7f63cc153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63cc0b39d0 0x7f63cc153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 -- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc06d480 msgr2=0x7f63cc14f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc06d480 0x7f63cc14f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.305+0000 7f63cb7fe700 1 -- 172.21.15.3:0/2409691421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c4009a20 con 0x7f63cc0b3da0 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.306+0000 7f63cb7fe700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc1543d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009fd0 tx=0x7f63c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.306+0000 7f63b2ffd700 1 -- 172.21.15.3:0/2409691421 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4027470 con 0x7f63cc0b3da0 2023-10-06T04:54:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.306+0000 7f63b2ffd700 1 -- 172.21.15.3:0/2409691421 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c40275f0 con 0x7f63cc0b3da0 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.306+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc15b1e0 con 0x7f63cc0b3da0 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.306+0000 7f63b2ffd700 1 -- 172.21.15.3:0/2409691421 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40279b0 con 0x7f63cc0b3da0 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.306+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc15b7a0 con 0x7f63cc0b3da0 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.307+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc0644c0 con 0x7f63cc0b3da0 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.307+0000 7f63b2ffd700 1 -- 172.21.15.3:0/2409691421 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63c4024070 con 0x7f63cc0b3da0 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.308+0000 7f63b2ffd700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.308+0000 7f63b2ffd700 1 -- 172.21.15.3:0/2409691421 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f63c4086be0 con 0x7f63cc0b3da0 2023-10-06T04:54:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.308+0000 7f63caffd700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:35.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.308+0000 7f63caffd700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b8066f30 0x7f63b80693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f63bc007130 tx=0x7f63bc007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.312+0000 7f63b2ffd700 1 -- 172.21.15.3:0/2409691421 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63c4053ff0 con 0x7f63cc0b3da0 2023-10-06T04:54:35.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:35.524+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f63cc15bc80 con 0x7f63cc0b3da0 2023-10-06T04:54:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.308+0000 7f63b2ffd700 1 -- 172.21.15.3:0/2409691421 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7f63c4057610 con 0x7f63cc0b3da0 2023-10-06T04:54:36.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b8066f30 msgr2=0x7f63b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b8066f30 0x7f63b80693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f63bc007130 tx=0x7f63bc007eb0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 msgr2=0x7f63cc1543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc1543d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009fd0 tx=0x7f63c4019000 comp rx=0 tx=0).stop 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 shutdown_connections 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc06d480 0x7f63cc14f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc0b3da0 0x7f63cc1543d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 --2- 172.21.15.3:0/2409691421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63cc0b39d0 0x7f63cc153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 >> 172.21.15.3:0/2409691421 conn(0x7f63cc0ce2c0 msgr2=0x7f63cc1588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 shutdown_connections 2023-10-06T04:54:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.311+0000 7f63d19e9700 1 -- 172.21.15.3:0/2409691421 wait complete. 2023-10-06T04:54:36.314 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T04:54:36.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:54:36.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.452+0000 7ffae450b700 1 Processor -- start 2023-10-06T04:54:36.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.452+0000 7ffae450b700 1 -- start start 2023-10-06T04:54:36.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.452+0000 7ffae450b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffadc159500 0x7ffadc0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae450b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc0b2930 0x7ffadc0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae450b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc0cc5c0 con 0x7ffadc1598d0 2023-10-06T04:54:36.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae450b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc0cb7b0 con 0x7ffadc159500 2023-10-06T04:54:36.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae450b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0cda60 con 0x7ffadc0b2930 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae22a7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffadc159500 0x7ffadc0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae2aa8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc0b2930 0x7ffadc0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae1aa6700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ffadc1598d0 0x7ffadc0b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:46346/0 (socket says 172.21.15.3:46346) 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae1aa6700 1 -- 172.21.15.3:0/2726626759 learned_addr learned my addr 172.21.15.3:0/2726626759 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae2aa8700 1 -- 172.21.15.3:0/2726626759 >> v2:172.21.15.153:3300/0 conn(0x7ffadc159500 msgr2=0x7ffadc0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae2aa8700 1 --2- 172.21.15.3:0/2726626759 >> v2:172.21.15.153:3300/0 conn(0x7ffadc159500 0x7ffadc0cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae2aa8700 1 -- 172.21.15.3:0/2726626759 >> v1:172.21.15.153:6789/0 conn(0x7ffadc1598d0 legacy=0x7ffadc0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae12a5700 1 -- 172.21.15.3:0/2726626759 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071691611 0 0) 0x7ffadc0cc5c0 con 0x7ffadc1598d0 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae2aa8700 1 -- 172.21.15.3:0/2726626759 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc0cb9b0 con 0x7ffadc0b2930 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae22a7700 1 --2- 172.21.15.3:0/2726626759 >> v2:172.21.15.153:3300/0 conn(0x7ffadc159500 0x7ffadc0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.453+0000 7ffae2aa8700 1 --2- 172.21.15.3:0/2726626759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc0b2930 0x7ffadc0cb270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0098e0 tx=0x7ffacc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9969a180662fd98d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:36.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae12a5700 1 -- 172.21.15.3:0/2726626759 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00b640 con 0x7ffadc0b2930 2023-10-06T04:54:36.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae12a5700 1 -- 172.21.15.3:0/2726626759 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc00b7a0 con 0x7ffadc0b2930 2023-10-06T04:54:36.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae12a5700 1 -- 172.21.15.3:0/2726626759 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00bb40 con 0x7ffadc0b2930 2023-10-06T04:54:36.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 -- 172.21.15.3:0/2726626759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc0b2930 msgr2=0x7ffadc0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 --2- 172.21.15.3:0/2726626759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc0b2930 0x7ffadc0cb270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0098e0 tx=0x7ffacc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 -- 172.21.15.3:0/2726626759 shutdown_connections 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 --2- 172.21.15.3:0/2726626759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc0b2930 0x7ffadc0cb270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 --2- 172.21.15.3:0/2726626759 >> v2:172.21.15.153:3300/0 conn(0x7ffadc159500 0x7ffadc0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 -- 172.21.15.3:0/2726626759 >> 172.21.15.3:0/2726626759 conn(0x7ffadc070310 msgr2=0x7ffadc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 -- 172.21.15.3:0/2726626759 shutdown_connections 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.454+0000 7ffae450b700 1 -- 172.21.15.3:0/2726626759 wait complete. 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.455+0000 7ffae450b700 1 Processor -- start 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.455+0000 7ffae450b700 1 -- start start 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae450b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffadc0b2930 0x7ffadc16a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae450b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc159500 0x7ffadc16c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae450b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffadc1598d0 0x7ffadc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae450b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc16ab10 con 0x7ffadc1598d0 2023-10-06T04:54:36.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae450b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc15e2d0 con 0x7ffadc0b2930 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae450b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc15eb30 con 0x7ffadc159500 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae2aa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffadc1598d0 0x7ffadc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae2aa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffadc1598d0 0x7ffadc0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36806/0 (socket says 172.21.15.3:36806) 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae2aa8700 1 -- 172.21.15.3:0/871265010 learned_addr learned my addr 172.21.15.3:0/871265010 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae22a7700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffadc0b2930 0x7ffadc16a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.456+0000 7ffae1aa6700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc159500 0x7ffadc16c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae2aa8700 1 -- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc159500 msgr2=0x7ffadc16c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae2aa8700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc159500 0x7ffadc16c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae2aa8700 1 -- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffadc0b2930 msgr2=0x7ffadc16a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae2aa8700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffadc0b2930 0x7ffadc16a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae2aa8700 1 -- 172.21.15.3:0/871265010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4009a20 con 0x7ffadc1598d0 2023-10-06T04:54:36.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae2aa8700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffadc1598d0 0x7ffadc0b5520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0cdba0 tx=0x7ffacc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:36.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffada7fc700 1 -- 172.21.15.3:0/871265010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00bb20 con 0x7ffadc1598d0 2023-10-06T04:54:36.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffada7fc700 1 -- 172.21.15.3:0/871265010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc011420 con 0x7ffadc1598d0 2023-10-06T04:54:36.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc009420 con 0x7ffadc1598d0 2023-10-06T04:54:36.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffada7fc700 1 -- 172.21.15.3:0/871265010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0117a0 con 0x7ffadc1598d0 2023-10-06T04:54:36.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.457+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc15e560 con 0x7ffadc1598d0 2023-10-06T04:54:36.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.458+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc071680 con 0x7ffadc1598d0 2023-10-06T04:54:36.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.458+0000 7ffada7fc700 1 -- 172.21.15.3:0/871265010 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffacc011b10 con 0x7ffadc1598d0 2023-10-06T04:54:36.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.459+0000 7ffada7fc700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffac8066f30 0x7ffac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.459+0000 7ffada7fc700 1 -- 172.21.15.3:0/871265010 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ffacc085e00 con 0x7ffadc1598d0 2023-10-06T04:54:36.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.459+0000 7ffae22a7700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffac8066f30 0x7ffac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.460+0000 7ffae22a7700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffac8066f30 0x7ffac80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffadc15ccc0 tx=0x7ffad400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.463+0000 7ffada7fc700 1 -- 172.21.15.3:0/871265010 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffacc053260 con 0x7ffadc1598d0 2023-10-06T04:54:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.735+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffadc15ec70 con 0x7ffadc1598d0 2023-10-06T04:54:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.736+0000 7ffada7fc700 1 -- 172.21.15.3:0/871265010 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ffacc056880 con 0x7ffadc1598d0 2023-10-06T04:54:36.737 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:36.737 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.739 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T04:54:36.740 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T04:54:36.740 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.740 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.740 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:54:36.740 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:54:36.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.741 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.741 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.741 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T04:54:36.741 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T04:54:36.741 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.741 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.742 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:54:36.742 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:54:36.742 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.742 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.742 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.743 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T04:54:36.743 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T04:54:36.743 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.743 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.743 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.743 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.744 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.744 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.744 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T04:54:36.744 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T04:54:36.744 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.745 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.745 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.745 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.745 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.746 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.746 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T04:54:36.746 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T04:54:36.746 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.746 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.746 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.747 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.747 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.747 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T04:54:36.747 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T04:54:36.747 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.748 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.748 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.748 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.748 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.748 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.749 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T04:54:36.749 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T04:54:36.749 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.749 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.749 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.749 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.750 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.750 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.750 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T04:54:36.750 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T04:54:36.750 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.751 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.751 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.751 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.751 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.751 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.751 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T04:54:36.752 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T04:54:36.753 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:54:36.754 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.755 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T04:54:36.756 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T04:54:36.757 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:54:36.758 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:54:36.759 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:54:36.759 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.759 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:36.759 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:54:36.759 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffac8066f30 msgr2=0x7ffac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffac8066f30 0x7ffac80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffadc15ccc0 tx=0x7ffad400b040 comp rx=0 tx=0).stop 2023-10-06T04:54:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffadc1598d0 msgr2=0x7ffadc0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffadc1598d0 0x7ffadc0b5520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0cdba0 tx=0x7ffacc017000 comp rx=0 tx=0).stop 2023-10-06T04:54:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 shutdown_connections 2023-10-06T04:54:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffac8066f30 0x7ffac80693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffadc0b2930 0x7ffadc16a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffadc1598d0 0x7ffadc0b5520 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 --2- 172.21.15.3:0/871265010 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffadc159500 0x7ffadc16c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 >> 172.21.15.3:0/871265010 conn(0x7ffadc070310 msgr2=0x7ffadc0ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 shutdown_connections 2023-10-06T04:54:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.738+0000 7ffae450b700 1 -- 172.21.15.3:0/871265010 wait complete. 2023-10-06T04:54:36.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:54:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.880+0000 7fb217fff700 1 Processor -- start 2023-10-06T04:54:36.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.880+0000 7fb217fff700 1 -- start start 2023-10-06T04:54:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.881+0000 7fb217fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb780 0x7fb218158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.881+0000 7fb217fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb218159500 0x7fb2180cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.881+0000 7fb217fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2180cf130 con 0x7fb2180cb3b0 2023-10-06T04:54:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.881+0000 7fb217fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2180ce520 con 0x7fb218159500 2023-10-06T04:54:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.881+0000 7fb217fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180ce800 con 0x7fb2180cb780 2023-10-06T04:54:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2167fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb780 0x7fb218158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2177fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb218159500 0x7fb2180cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb216ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb2180cb3b0 0x7fb2180cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:46356/0 (socket says 172.21.15.3:46356) 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2177fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb218159500 0x7fb2180cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:55340/0 (socket says 172.21.15.3:55340) 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2167fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb780 0x7fb218158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:41458/0 (socket says 172.21.15.3:41458) 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb216ffd700 1 -- 172.21.15.3:0/330645355 learned_addr learned my addr 172.21.15.3:0/330645355 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2177fe700 1 -- 172.21.15.3:0/330645355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb780 msgr2=0x7fb218158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2177fe700 1 --2- 172.21.15.3:0/330645355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb780 0x7fb218158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2177fe700 1 -- 172.21.15.3:0/330645355 >> v1:172.21.15.153:6789/0 conn(0x7fb2180cb3b0 legacy=0x7fb2180cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2177fe700 1 -- 172.21.15.3:0/330645355 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180ceae0 con 0x7fb218159500 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb215ffb700 1 -- 172.21.15.3:0/330645355 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746532794 0 0) 0x7fb2180cf130 con 0x7fb2180cb3b0 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb2177fe700 1 --2- 172.21.15.3:0/330645355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 0x7fb2180cccc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb210009e60 tx=0x7fb210009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd764334c8cb712 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb215ffb700 1 -- 172.21.15.3:0/330645355 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000e070 con 0x7fb218159500 2023-10-06T04:54:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb215ffb700 1 -- 172.21.15.3:0/330645355 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21000bb60 con 0x7fb218159500 2023-10-06T04:54:36.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.882+0000 7fb215ffb700 1 -- 172.21.15.3:0/330645355 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210012430 con 0x7fb218159500 2023-10-06T04:54:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 -- 172.21.15.3:0/330645355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 msgr2=0x7fb2180cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 --2- 172.21.15.3:0/330645355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 0x7fb2180cccc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb210009e60 tx=0x7fb210009780 comp rx=0 tx=0).stop 2023-10-06T04:54:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 -- 172.21.15.3:0/330645355 shutdown_connections 2023-10-06T04:54:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 --2- 172.21.15.3:0/330645355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 0x7fb2180cccc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 --2- 172.21.15.3:0/330645355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb780 0x7fb218158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 -- 172.21.15.3:0/330645355 >> 172.21.15.3:0/330645355 conn(0x7fb218070310 msgr2=0x7fb218070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 -- 172.21.15.3:0/330645355 shutdown_connections 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.883+0000 7fb217fff700 1 -- 172.21.15.3:0/330645355 wait complete. 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.884+0000 7fb217fff700 1 Processor -- start 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.884+0000 7fb217fff700 1 -- start start 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb217fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb3b0 0x7fb21815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb217fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2180cb780 0x7fb2180b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb217fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 0x7fb2180b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb217fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b0470 con 0x7fb2180cb780 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb217fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b12d0 con 0x7fb218159500 2023-10-06T04:54:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb217fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b5470 con 0x7fb2180cb3b0 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2180cb780 0x7fb2180b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2180cb780 0x7fb2180b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36812/0 (socket says 172.21.15.3:36812) 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 -- 172.21.15.3:0/947068046 learned_addr learned my addr 172.21.15.3:0/947068046 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb216ffd700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb3b0 0x7fb21815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2177fe700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 0x7fb2180b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 -- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb3b0 msgr2=0x7fb21815c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb3b0 0x7fb21815c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 -- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 msgr2=0x7fb2180b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 0x7fb2180b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.885+0000 7fb2167fc700 1 -- 172.21.15.3:0/947068046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210009a20 con 0x7fb2180cb780 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.886+0000 7fb2167fc700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2180cb780 0x7fb2180b4f30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb20000ec80 tx=0x7fb2000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.886+0000 7fb20effd700 1 -- 172.21.15.3:0/947068046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000c850 con 0x7fb2180cb780 2023-10-06T04:54:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.886+0000 7fb20effd700 1 -- 172.21.15.3:0/947068046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20000c9b0 con 0x7fb2180cb780 2023-10-06T04:54:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.886+0000 7fb20effd700 1 -- 172.21.15.3:0/947068046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000cd50 con 0x7fb2180cb780 2023-10-06T04:54:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.886+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2180b06c0 con 0x7fb2180cb780 2023-10-06T04:54:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.886+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2180b0c80 con 0x7fb2180cb780 2023-10-06T04:54:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.887+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21804c380 con 0x7fb2180cb780 2023-10-06T04:54:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.887+0000 7fb20effd700 1 -- 172.21.15.3:0/947068046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb200013680 con 0x7fb2180cb780 2023-10-06T04:54:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.888+0000 7fb20effd700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb1f8066f30 0x7fb1f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.888+0000 7fb20effd700 1 -- 172.21.15.3:0/947068046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb200084660 con 0x7fb2180cb780 2023-10-06T04:54:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.888+0000 7fb216ffd700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb1f8066f30 0x7fb1f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.888+0000 7fb216ffd700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb1f8066f30 0x7fb1f80693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb21816cae0 tx=0x7fb208007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:36.892+0000 7fb20effd700 1 -- 172.21.15.3:0/947068046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb200051a20 con 0x7fb2180cb780 2023-10-06T04:54:37.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.086+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2180b1410 con 0x7fb2180cb780 2023-10-06T04:54:37.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.087+0000 7fb20effd700 1 -- 172.21.15.3:0/947068046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+13602 (secure 0 0 0) 0x7fb200055040 con 0x7fb2180cb780 2023-10-06T04:54:37.089 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:37.089 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:54:36.300964+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:54:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb1f8066f30 msgr2=0x7fb1f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb1f8066f30 0x7fb1f80693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb21816cae0 tx=0x7fb208007d30 comp rx=0 tx=0).stop 2023-10-06T04:54:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2180cb780 msgr2=0x7fb2180b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2180cb780 0x7fb2180b4f30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb20000ec80 tx=0x7fb2000091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 shutdown_connections 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb1f8066f30 0x7fb1f80693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb218159500 0x7fb2180b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2180cb3b0 0x7fb21815c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 --2- 172.21.15.3:0/947068046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2180cb780 0x7fb2180b4f30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 >> 172.21.15.3:0/947068046 conn(0x7fb218070310 msgr2=0x7fb2180c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 shutdown_connections 2023-10-06T04:54:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.089+0000 7fb217fff700 1 -- 172.21.15.3:0/947068046 wait complete. 2023-10-06T04:54:37.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:54:37.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:54:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.224+0000 7f033dec6700 1 Processor -- start 2023-10-06T04:54:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.224+0000 7f033dec6700 1 -- start start 2023-10-06T04:54:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f033dec6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0338159eb0 0x7f03380ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f033dec6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380afca0 con 0x7f03380cef10 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f033dec6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033815d510 con 0x7f03380cfb80 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f033dec6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033815a480 con 0x7f0338159eb0 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f0337fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f03380cef10 0x7f033815da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56740/0 (socket says 172.21.15.3:56740) 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f0337fff700 1 -- 172.21.15.3:0/2063035353 learned_addr learned my addr 172.21.15.3:0/2063035353 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f0336ffd700 1 --2- 172.21.15.3:0/2063035353 >> v2:172.21.15.153:3300/0 conn(0x7f0338159eb0 0x7f03380ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f03367fc700 1 -- 172.21.15.3:0/2063035353 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173661917 0 0) 0x7f03380afca0 con 0x7f03380cef10 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f03367fc700 1 -- 172.21.15.3:0/2063035353 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0320003570 con 0x7f03380cef10 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f03367fc700 1 -- 172.21.15.3:0/2063035353 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312191342 0 0) 0x7f033815d510 con 0x7f03380cfb80 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f03367fc700 1 -- 172.21.15.3:0/2063035353 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03380afca0 con 0x7f03380cfb80 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f0336ffd700 1 -- 172.21.15.3:0/2063035353 >> v1:172.21.15.153:6789/0 conn(0x7f03380cfb80 legacy=0x7f03380af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f0336ffd700 1 -- 172.21.15.3:0/2063035353 >> v1:172.21.15.3:6789/0 conn(0x7f03380cef10 legacy=0x7f033815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f0336ffd700 1 -- 172.21.15.3:0/2063035353 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033815a760 con 0x7f0338159eb0 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.225+0000 7f0336ffd700 1 --2- 172.21.15.3:0/2063035353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0338159eb0 0x7f03380ce9d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098e0 tx=0x7f032c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25465fa54949be3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f03367fc700 1 -- 172.21.15.3:0/2063035353 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c011630 con 0x7f0338159eb0 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f03367fc700 1 -- 172.21.15.3:0/2063035353 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c011790 con 0x7f0338159eb0 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f03367fc700 1 -- 172.21.15.3:0/2063035353 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c011b30 con 0x7f0338159eb0 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f033dec6700 1 -- 172.21.15.3:0/2063035353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0338159eb0 msgr2=0x7f03380ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f033dec6700 1 --2- 172.21.15.3:0/2063035353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0338159eb0 0x7f03380ce9d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098e0 tx=0x7f032c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f033dec6700 1 -- 172.21.15.3:0/2063035353 shutdown_connections 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f033dec6700 1 --2- 172.21.15.3:0/2063035353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0338159eb0 0x7f03380ce9d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f033dec6700 1 -- 172.21.15.3:0/2063035353 >> 172.21.15.3:0/2063035353 conn(0x7f0338070310 msgr2=0x7f0338070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f033dec6700 1 -- 172.21.15.3:0/2063035353 shutdown_connections 2023-10-06T04:54:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.226+0000 7f033dec6700 1 -- 172.21.15.3:0/2063035353 wait complete. 2023-10-06T04:54:37.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.227+0000 7f033dec6700 1 Processor -- start 2023-10-06T04:54:37.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.227+0000 7f033dec6700 1 -- start start 2023-10-06T04:54:37.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f033dec6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03380cef10 0x7f03380af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f033dec6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03380cfb80 0x7f03380c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f033dec6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0338159eb0 0x7f03380c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f033dec6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b38d0 con 0x7f0338159eb0 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f033dec6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b1320 con 0x7f03380cef10 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f033dec6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380ca010 con 0x7f03380cfb80 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0338159eb0 0x7f03380c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0338159eb0 0x7f03380c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36826/0 (socket says 172.21.15.3:36826) 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 -- 172.21.15.3:0/2870097178 learned_addr learned my addr 172.21.15.3:0/2870097178 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f03377fe700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03380cef10 0x7f03380af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0336ffd700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03380cfb80 0x7f03380c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 -- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03380cfb80 msgr2=0x7f03380c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03380cfb80 0x7f03380c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 -- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03380cef10 msgr2=0x7f03380af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03380cef10 0x7f03380af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.228+0000 7f0337fff700 1 -- 172.21.15.3:0/2870097178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009420 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.229+0000 7f0337fff700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0338159eb0 0x7f03380c74f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0330001460 tx=0x7f0330001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.229+0000 7f031f7fe700 1 -- 172.21.15.3:0/2870097178 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033000a8b0 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.229+0000 7f031f7fe700 1 -- 172.21.15.3:0/2870097178 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033000aa10 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.229+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03380ca2f0 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.229+0000 7f031f7fe700 1 -- 172.21.15.3:0/2870097178 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033000adb0 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.229+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03380b15c0 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.230+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033804c380 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.230+0000 7f031f7fe700 1 -- 172.21.15.3:0/2870097178 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0330007300 con 0x7f0338159eb0 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.231+0000 7f031f7fe700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0324066f30 0x7f03240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.231+0000 7f031f7fe700 1 -- 172.21.15.3:0/2870097178 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f03300845f0 con 0x7f0338159eb0 2023-10-06T04:54:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.231+0000 7f03377fe700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0324066f30 0x7f03240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.231+0000 7f03377fe700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0324066f30 0x7f03240693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0328007fd0 tx=0x7f0328007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.235+0000 7f031f7fe700 1 -- 172.21.15.3:0/2870097178 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0330051ad0 con 0x7f0338159eb0 2023-10-06T04:54:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.426+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f03380b17b0 con 0x7f0338159eb0 2023-10-06T04:54:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.427+0000 7f031f7fe700 1 -- 172.21.15.3:0/2870097178 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f03300550f0 con 0x7f0338159eb0 2023-10-06T04:54:37.428 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:54:37.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.428+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0324066f30 msgr2=0x7f03240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.428+0000 7f033dec6700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0324066f30 0x7f03240693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0328007fd0 tx=0x7f0328007d50 comp rx=0 tx=0).stop 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.428+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0338159eb0 msgr2=0x7f03380c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.428+0000 7f033dec6700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0338159eb0 0x7f03380c74f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0330001460 tx=0x7f0330001ac0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 shutdown_connections 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0324066f30 0x7f03240693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03380cef10 0x7f03380af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0338159eb0 0x7f03380c74f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 --2- 172.21.15.3:0/2870097178 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03380cfb80 0x7f03380c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 >> 172.21.15.3:0/2870097178 conn(0x7f0338070310 msgr2=0x7f033815cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 shutdown_connections 2023-10-06T04:54:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.429+0000 7f033dec6700 1 -- 172.21.15.3:0/2870097178 wait complete. 2023-10-06T04:54:37.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:54:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.563+0000 7f74b56b6700 1 Processor -- start 2023-10-06T04:54:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.563+0000 7f74b56b6700 1 -- start start 2023-10-06T04:54:37.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74b56b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 0x7f74b01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74b56b6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b006a9d0 con 0x7f74b01597e0 2023-10-06T04:54:37.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74b56b6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b006acb0 con 0x7f74b0159bb0 2023-10-06T04:54:37.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74b56b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b006af90 con 0x7f74b0150260 2023-10-06T04:54:37.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74aeffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f74b01597e0 0x7f74b0069f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56744/0 (socket says 172.21.15.3:56744) 2023-10-06T04:54:37.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74aeffd700 1 -- 172.21.15.3:0/1805281233 learned_addr learned my addr 172.21.15.3:0/1805281233 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070472948 0 0) 0x7f74b006a9d0 con 0x7f74b01597e0 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74af7fe700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 0x7f74b01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f749c005780 con 0x7f74b01597e0 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2088123037 0 0) 0x7f749c005780 con 0x7f74b01597e0 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.564+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 msgr2=0x7f74b01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 0x7f74b01609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 >> v1:172.21.15.153:6789/0 conn(0x7f74b0159bb0 legacy=0x7f74b014fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b006b270 con 0x7f74b01597e0 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f74a00017f0 con 0x7f74b01597e0 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 >> v1:172.21.15.3:6789/0 conn(0x7f74b01597e0 legacy=0x7f74b0069f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 0x7f749c005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f749c006220 0x7f749c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c006f60 0x7f749c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b006af90 con 0x7f749c006f60 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c00de00 con 0x7f749c006220 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c00dfc0 con 0x7f74b0150260 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74af7fe700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c006f60 0x7f749c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74aeffd700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 0x7f749c005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74ae7fc700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f749c006220 0x7f749c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74af7fe700 1 -- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 msgr2=0x7f749c005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74af7fe700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b0150260 0x7f749c005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74af7fe700 1 -- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f749c006220 msgr2=0x7f749c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74af7fe700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f749c006220 0x7f749c006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74af7fe700 1 -- 172.21.15.3:0/1805281233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c005780 con 0x7f749c006f60 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74af7fe700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c006f60 0x7f749c0073d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f74a8006f00 tx=0x7f74a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4434b5feb2c864 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800e710 con 0x7f749c006f60 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a800e870 con 0x7f749c006f60 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.565+0000 7f74adffb700 1 -- 172.21.15.3:0/1805281233 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800ec30 con 0x7f749c006f60 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.566+0000 7f74b56b6700 1 -- 172.21.15.3:0/1805281233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c006f60 msgr2=0x7f749c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.566+0000 7f74b56b6700 1 --2- 172.21.15.3:0/1805281233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c006f60 0x7f749c0073d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f74a8006f00 tx=0x7f74a80091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.566+0000 7f74af7fe700 1 -- 172.21.15.3:0/1805281233 reap_dead start 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.566+0000 7f74b56b6700 1 -- 172.21.15.3:0/1805281233 shutdown_connections 2023-10-06T04:54:37.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.566+0000 7f74b56b6700 1 -- 172.21.15.3:0/1805281233 >> 172.21.15.3:0/1805281233 conn(0x7f74b006ce40 msgr2=0x7f74b00cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:37.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.566+0000 7f74b56b6700 1 -- 172.21.15.3:0/1805281233 shutdown_connections 2023-10-06T04:54:37.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.566+0000 7f74b56b6700 1 -- 172.21.15.3:0/1805281233 wait complete. 2023-10-06T04:54:37.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.567+0000 7f74b56b6700 1 Processor -- start 2023-10-06T04:54:37.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.567+0000 7f74b56b6700 1 -- start start 2023-10-06T04:54:37.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74b56b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f749c006f60 0x7f74b014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74b56b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b01466e0 0x7f74b0149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74b56b6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b006f150 0x7f74b006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74b56b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b006fb00 con 0x7f74b01466e0 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74b56b6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b014d0d0 con 0x7f74b006f150 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74b56b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01453a0 con 0x7f749c006f60 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b01466e0 0x7f74b0149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b01466e0 0x7f74b0149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36842/0 (socket says 172.21.15.3:36842) 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 -- 172.21.15.3:0/434653295 learned_addr learned my addr 172.21.15.3:0/434653295 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74aeffd700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f749c006f60 0x7f74b014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74af7fe700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b006f150 0x7f74b006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 -- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f749c006f60 msgr2=0x7f74b014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f749c006f60 0x7f74b014cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 -- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b006f150 msgr2=0x7f74b006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b006f150 0x7f74b006f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 -- 172.21.15.3:0/434653295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b0145640 con 0x7f74b01466e0 2023-10-06T04:54:37.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74af7fe700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b006f150 0x7f74b006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.568+0000 7f74ae7fc700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b01466e0 0x7f74b0149080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7498002950 tx=0x7f7498007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.569+0000 7f74a6ffd700 1 -- 172.21.15.3:0/434653295 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800f670 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.569+0000 7f74a6ffd700 1 -- 172.21.15.3:0/434653295 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749800f7d0 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.569+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b0145920 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.569+0000 7f74a6ffd700 1 -- 172.21.15.3:0/434653295 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800fb50 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.569+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b0145ee0 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.570+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b004c380 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.570+0000 7f74a6ffd700 1 -- 172.21.15.3:0/434653295 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f749801c400 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.570+0000 7f74a6ffd700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7490066f30 0x7f74900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.570+0000 7f74a6ffd700 1 -- 172.21.15.3:0/434653295 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f74980847a0 con 0x7f74b01466e0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.571+0000 7f74aeffd700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7490066f30 0x7f74900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.571+0000 7f74aeffd700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7490066f30 0x7f74900693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f74a0001460 tx=0x7f74b0157030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.575+0000 7f74a6ffd700 1 -- 172.21.15.3:0/434653295 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7498051cb0 con 0x7f74b01466e0 2023-10-06T04:54:37.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.769+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74b0148450 con 0x7f74b01466e0 2023-10-06T04:54:37.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.770+0000 7f74a6ffd700 1 -- 172.21.15.3:0/434653295 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74980552d0 con 0x7f74b01466e0 2023-10-06T04:54:37.772 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:54:37.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7490066f30 msgr2=0x7f74900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7490066f30 0x7f74900693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f74a0001460 tx=0x7f74b0157030 comp rx=0 tx=0).stop 2023-10-06T04:54:37.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b01466e0 msgr2=0x7f74b0149080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b01466e0 0x7f74b0149080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7498002950 tx=0x7f7498007d30 comp rx=0 tx=0).stop 2023-10-06T04:54:37.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 shutdown_connections 2023-10-06T04:54:37.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7490066f30 0x7f74900693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b006f150 0x7f74b006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f749c006f60 0x7f74b014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 --2- 172.21.15.3:0/434653295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b01466e0 0x7f74b0149080 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 >> 172.21.15.3:0/434653295 conn(0x7f74b006ce40 msgr2=0x7f74b00cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:37.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 shutdown_connections 2023-10-06T04:54:37.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.772+0000 7f74b56b6700 1 -- 172.21.15.3:0/434653295 wait complete. 2023-10-06T04:54:37.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:54:37.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.904+0000 7fc0b28ce700 1 Processor -- start 2023-10-06T04:54:37.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.904+0000 7fc0b28ce700 1 -- start start 2023-10-06T04:54:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.905+0000 7fc0b28ce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc0ac0b2910 0x7fc0ac1642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.905+0000 7fc0b28ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ac0b0ce0 con 0x7fc0ac0cd950 2023-10-06T04:54:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.905+0000 7fc0b28ce700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ac0cb820 con 0x7fc0ac15cb50 2023-10-06T04:54:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.905+0000 7fc0b28ce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cccc0 con 0x7fc0ac0b2910 2023-10-06T04:54:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.905+0000 7fc0ab7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc0ac0cd950 0x7fc0ac0b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56746/0 (socket says 172.21.15.3:56746) 2023-10-06T04:54:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.905+0000 7fc0ab7fe700 1 -- 172.21.15.3:0/402017606 learned_addr learned my addr 172.21.15.3:0/402017606 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0b0e6b700 1 --2- 172.21.15.3:0/402017606 >> v2:172.21.15.153:3300/0 conn(0x7fc0ac0b2910 0x7fc0ac1642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120826940 0 0) 0x7fc0ac0b0ce0 con 0x7fc0ac0cd950 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc098003570 con 0x7fc0ac0cd950 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221391627 0 0) 0x7fc098003570 con 0x7fc0ac0cd950 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 >> v2:172.21.15.153:3300/0 conn(0x7fc0ac0b2910 msgr2=0x7fc0ac1642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 --2- 172.21.15.3:0/402017606 >> v2:172.21.15.153:3300/0 conn(0x7fc0ac0b2910 0x7fc0ac1642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 >> v1:172.21.15.153:6789/0 conn(0x7fc0ac15cb50 legacy=0x7fc0ac0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0b0760 con 0x7fc0ac0cd950 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc094001470 con 0x7fc0ac0cd950 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0b0e6b700 1 --2- 172.21.15.3:0/402017606 >> v2:172.21.15.153:3300/0 conn(0x7fc0ac0b2910 0x7fc0ac1642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 >> v1:172.21.15.3:6789/0 conn(0x7fc0ac0cd950 legacy=0x7fc0ac0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc098003a60 0x7fc098003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098004810 0x7fc098004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0980051c0 0x7fc09800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cccc0 con 0x7fc098003a60 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09800c0e0 con 0x7fc0980051c0 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09800c2a0 con 0x7fc098004810 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0abfff700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc098003a60 0x7fc098003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0ab7fe700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098004810 0x7fc098004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0b0e6b700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0980051c0 0x7fc09800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0abfff700 1 -- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098004810 msgr2=0x7fc098004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0abfff700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098004810 0x7fc098004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0abfff700 1 -- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0980051c0 msgr2=0x7fc09800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0abfff700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0980051c0 0x7fc09800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0abfff700 1 -- 172.21.15.3:0/402017606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098003570 con 0x7fc098003a60 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.906+0000 7fc0b0e6b700 1 -- 172.21.15.3:0/402017606 reap_dead start 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0abfff700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc098003a60 0x7fc098003e30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0019c0 tx=0x7fc09c007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a737e2a1ef24283 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c001330 con 0x7fc098003a60 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c00c5d0 con 0x7fc098003a60 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0aaffd700 1 -- 172.21.15.3:0/402017606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00c970 con 0x7fc098003a60 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0b28ce700 1 -- 172.21.15.3:0/402017606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc098003a60 msgr2=0x7fc098003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc098003a60 0x7fc098003e30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0019c0 tx=0x7fc09c007d60 comp rx=0 tx=0).stop 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0b28ce700 1 -- 172.21.15.3:0/402017606 shutdown_connections 2023-10-06T04:54:37.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/402017606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc098003a60 0x7fc098003e30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0b28ce700 1 -- 172.21.15.3:0/402017606 >> 172.21.15.3:0/402017606 conn(0x7fc0ac070310 msgr2=0x7fc0ac070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0b28ce700 1 -- 172.21.15.3:0/402017606 shutdown_connections 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.907+0000 7fc0b28ce700 1 -- 172.21.15.3:0/402017606 wait complete. 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.908+0000 7fc0b28ce700 1 Processor -- start 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b28ce700 1 -- start start 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b28ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098003a60 0x7fc0ac16a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b28ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0ac0d0330 0x7fc0ac0c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b28ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0ac0c61c0 0x7fc0ac0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b28ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0d0af0 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b28ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac16a700 con 0x7fc0ac0d0330 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b28ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac159e70 con 0x7fc098003a60 2023-10-06T04:54:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b0e6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0ac0c61c0 0x7fc0ac0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b0e6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0ac0c61c0 0x7fc0ac0c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36862/0 (socket says 172.21.15.3:36862) 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.909+0000 7fc0b0e6b700 1 -- 172.21.15.3:0/378116027 learned_addr learned my addr 172.21.15.3:0/378116027 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0abfff700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098003a60 0x7fc0ac16a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0ab7fe700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0ac0d0330 0x7fc0ac0c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b0e6b700 1 -- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098003a60 msgr2=0x7fc0ac16a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b0e6b700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098003a60 0x7fc0ac16a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b0e6b700 1 -- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0ac0d0330 msgr2=0x7fc0ac0c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b0e6b700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0ac0d0330 0x7fc0ac0c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b0e6b700 1 -- 172.21.15.3:0/378116027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0d0810 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b0e6b700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0ac0c61c0 0x7fc0ac0c4ab0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0b5c30 tx=0x7fc0a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0a3fff700 1 -- 172.21.15.3:0/378116027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a400e070 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0a3fff700 1 -- 172.21.15.3:0/378116027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4011b50 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac0c3fd0 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0a3fff700 1 -- 172.21.15.3:0/378116027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4016470 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.910+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac0c4590 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.911+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac0c3e30 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.912+0000 7fc0a3fff700 1 -- 172.21.15.3:0/378116027 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0a4016820 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.912+0000 7fc0a3fff700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc08c066f30 0x7fc08c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.912+0000 7fc0a3fff700 1 -- 172.21.15.3:0/378116027 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc0a4084df0 con 0x7fc0ac0c61c0 2023-10-06T04:54:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.912+0000 7fc0abfff700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc08c066f30 0x7fc08c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.913+0000 7fc0abfff700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc08c066f30 0x7fc08c0693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc09c001020 tx=0x7fc09c015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:37.916+0000 7fc0a3fff700 1 -- 172.21.15.3:0/378116027 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0ac0c3e30 con 0x7fc0ac0c61c0 2023-10-06T04:54:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.111+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc0ac0c3e30 con 0x7fc0ac0c61c0 2023-10-06T04:54:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.111+0000 7fc0a3fff700 1 -- 172.21.15.3:0/378116027 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc0ac0c3e30 con 0x7fc0ac0c61c0 2023-10-06T04:54:38.113 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:38.113 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:54:38.113 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:38.113 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:54:38.114 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:54:38.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:38.114 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:38.114 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:54:38.114 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:54:38.114 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:38.114 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:54:38.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc08c066f30 msgr2=0x7fc08c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:38.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc08c066f30 0x7fc08c0693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc09c001020 tx=0x7fc09c015000 comp rx=0 tx=0).stop 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0ac0c61c0 msgr2=0x7fc0ac0c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0ac0c61c0 0x7fc0ac0c4ab0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0b5c30 tx=0x7fc0a400b040 comp rx=0 tx=0).stop 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 shutdown_connections 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc08c066f30 0x7fc08c0693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0ac0c61c0 0x7fc0ac0c4ab0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc098003a60 0x7fc0ac16a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 --2- 172.21.15.3:0/378116027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0ac0d0330 0x7fc0ac0c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 >> 172.21.15.3:0/378116027 conn(0x7fc0ac070310 msgr2=0x7fc0ac15b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 shutdown_connections 2023-10-06T04:54:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.113+0000 7fc0b28ce700 1 -- 172.21.15.3:0/378116027 wait complete. 2023-10-06T04:54:38.127 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:54:38.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:54:38.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.248+0000 7f916bfff700 1 Processor -- start 2023-10-06T04:54:38.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.248+0000 7f916bfff700 1 -- start start 2023-10-06T04:54:38.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.249+0000 7f916bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f916c0af150 0x7f916c0c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:38.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.249+0000 7f916bfff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c159c00 con 0x7f916c0cd390 2023-10-06T04:54:38.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.249+0000 7f916bfff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c15e000 con 0x7f916c0c6de0 2023-10-06T04:54:38.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.249+0000 7f916bfff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0ce0f0 con 0x7f916c0af150 2023-10-06T04:54:38.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.249+0000 7f916b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f916c0cd390 0x7f916c0cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56756/0 (socket says 172.21.15.3:56756) 2023-10-06T04:54:38.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.249+0000 7f916b7fe700 1 -- 172.21.15.3:0/22957395 learned_addr learned my addr 172.21.15.3:0/22957395 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.249+0000 7f916affd700 1 --2- 172.21.15.3:0/22957395 >> v2:172.21.15.153:3300/0 conn(0x7f916c0af150 0x7f916c0c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924375302 0 0) 0x7f916c159c00 con 0x7f916c0cd390 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9158003570 con 0x7f916c0cd390 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359218156 0 0) 0x7f916c15e000 con 0x7f916c0c6de0 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916c159c00 con 0x7f916c0c6de0 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 888650557 0 0) 0x7f9158003570 con 0x7f916c0cd390 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f916affd700 1 -- 172.21.15.3:0/22957395 >> v1:172.21.15.153:6789/0 conn(0x7f916c0c6de0 legacy=0x7f916c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f916affd700 1 -- 172.21.15.3:0/22957395 >> v1:172.21.15.3:6789/0 conn(0x7f916c0cd390 legacy=0x7f916c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:38.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f916affd700 1 -- 172.21.15.3:0/22957395 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c15d3f0 con 0x7f916c0af150 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f916affd700 1 --2- 172.21.15.3:0/22957395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0af150 0x7f916c0c68a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f915c009e60 tx=0x7f915c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37973c5e75d430e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00e070 con 0x7f916c0af150 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c00b7a0 con 0x7f916c0af150 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.250+0000 7f9169ffb700 1 -- 172.21.15.3:0/22957395 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164001470 con 0x7f916c0af150 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.251+0000 7f916bfff700 1 -- 172.21.15.3:0/22957395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0af150 msgr2=0x7f916c0c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.251+0000 7f916bfff700 1 --2- 172.21.15.3:0/22957395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0af150 0x7f916c0c68a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f915c009e60 tx=0x7f915c009780 comp rx=0 tx=0).stop 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.251+0000 7f916bfff700 1 -- 172.21.15.3:0/22957395 shutdown_connections 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.251+0000 7f916bfff700 1 --2- 172.21.15.3:0/22957395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0af150 0x7f916c0c68a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.251+0000 7f916bfff700 1 -- 172.21.15.3:0/22957395 >> 172.21.15.3:0/22957395 conn(0x7f916c070310 msgr2=0x7f916c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.251+0000 7f916bfff700 1 -- 172.21.15.3:0/22957395 shutdown_connections 2023-10-06T04:54:38.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.251+0000 7f916bfff700 1 -- 172.21.15.3:0/22957395 wait complete. 2023-10-06T04:54:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.252+0000 7f916bfff700 1 Processor -- start 2023-10-06T04:54:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.252+0000 7f916bfff700 1 -- start start 2023-10-06T04:54:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f916c0af150 0x7f916c150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0c6de0 0x7f916c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c0cd390 0x7f916c0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916bfff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c16db80 con 0x7f916c0cd390 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916bfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c15cc50 con 0x7f916c0c6de0 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916bfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c16cf00 con 0x7f916c0af150 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c0cd390 0x7f916c0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c0cd390 0x7f916c0cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36870/0 (socket says 172.21.15.3:36870) 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 -- 172.21.15.3:0/2681119934 learned_addr learned my addr 172.21.15.3:0/2681119934 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916affd700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f916c0af150 0x7f916c150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916a7fc700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0c6de0 0x7f916c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 -- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f916c0af150 msgr2=0x7f916c150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f916c0af150 0x7f916c150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 -- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0c6de0 msgr2=0x7f916c0b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0c6de0 0x7f916c0b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 -- 172.21.15.3:0/2681119934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c009a20 con 0x7f916c0cd390 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916affd700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f916c0af150 0x7f916c150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916a7fc700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0c6de0 0x7f916c0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.253+0000 7f916b7fe700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c0cd390 0x7f916c0cf730 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f916400c810 tx=0x7f9164001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.254+0000 7f9162ffd700 1 -- 172.21.15.3:0/2681119934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164007360 con 0x7f916c0cd390 2023-10-06T04:54:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.254+0000 7f9162ffd700 1 -- 172.21.15.3:0/2681119934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640074e0 con 0x7f916c0cd390 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.254+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c16d1e0 con 0x7f916c0cd390 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.254+0000 7f9162ffd700 1 -- 172.21.15.3:0/2681119934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164004830 con 0x7f916c0cd390 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.254+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c16d7a0 con 0x7f916c0cd390 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.255+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c16dcc0 con 0x7f916c0cd390 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.255+0000 7f9162ffd700 1 -- 172.21.15.3:0/2681119934 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9164004be0 con 0x7f916c0cd390 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.255+0000 7f9162ffd700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f914c066f30 0x7f914c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.255+0000 7f9162ffd700 1 -- 172.21.15.3:0/2681119934 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9164083b40 con 0x7f916c0cd390 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.256+0000 7f916affd700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f914c066f30 0x7f914c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.256+0000 7f916affd700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f914c066f30 0x7f914c0693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f916c0d0410 tx=0x7f915c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.260+0000 7f9162ffd700 1 -- 172.21.15.3:0/2681119934 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f916c16dcc0 con 0x7f916c0cd390 2023-10-06T04:54:38.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:38.455+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f916c16dcc0 con 0x7f916c0cd390 2023-10-06T04:54:39.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.318+0000 7f9162ffd700 1 -- 172.21.15.3:0/2681119934 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7f916c16dcc0 con 0x7f916c0cd390 2023-10-06T04:54:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.323+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f914c066f30 msgr2=0x7f914c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.323+0000 7f916bfff700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f914c066f30 0x7f914c0693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f916c0d0410 tx=0x7f915c017000 comp rx=0 tx=0).stop 2023-10-06T04:54:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.323+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c0cd390 msgr2=0x7f916c0cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.323+0000 7f916bfff700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c0cd390 0x7f916c0cf730 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f916400c810 tx=0x7f9164001a00 comp rx=0 tx=0).stop 2023-10-06T04:54:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.324+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 shutdown_connections 2023-10-06T04:54:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.324+0000 7f916bfff700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f914c066f30 0x7f914c0693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.324+0000 7f916bfff700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c0cd390 0x7f916c0cf730 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.324+0000 7f916bfff700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f916c0af150 0x7f916c150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.324+0000 7f916bfff700 1 --2- 172.21.15.3:0/2681119934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f916c0c6de0 0x7f916c0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.324+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 >> 172.21.15.3:0/2681119934 conn(0x7f916c070310 msgr2=0x7f916c06e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:39.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.325+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 shutdown_connections 2023-10-06T04:54:39.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.325+0000 7f916bfff700 1 -- 172.21.15.3:0/2681119934 wait complete. 2023-10-06T04:54:39.329 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:54:39.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:54:39.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.473+0000 7efbf8e78700 1 Processor -- start 2023-10-06T04:54:39.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.473+0000 7efbf8e78700 1 -- start start 2023-10-06T04:54:39.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf8e78700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efbf40cba50 0x7efbf4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf8e78700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf415a290 con 0x7efbf415d540 2023-10-06T04:54:39.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf8e78700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf40d0020 con 0x7efbf4150690 2023-10-06T04:54:39.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf8e78700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf4159f40 con 0x7efbf40cba50 2023-10-06T04:54:39.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efbf415d540 0x7efbf40af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56766/0 (socket says 172.21.15.3:56766) 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf2d9d700 1 -- 172.21.15.3:0/2797229949 learned_addr learned my addr 172.21.15.3:0/2797229949 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf1d9b700 1 --2- 172.21.15.3:0/2797229949 >> v2:172.21.15.153:3300/0 conn(0x7efbf40cba50 0x7efbf4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574874083 0 0) 0x7efbf415a290 con 0x7efbf415d540 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe0003570 con 0x7efbf415d540 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4211779549 0 0) 0x7efbe0003570 con 0x7efbf415d540 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 >> v2:172.21.15.153:3300/0 conn(0x7efbf40cba50 msgr2=0x7efbf4150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 --2- 172.21.15.3:0/2797229949 >> v2:172.21.15.153:3300/0 conn(0x7efbf40cba50 0x7efbf4150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 >> v1:172.21.15.153:6789/0 conn(0x7efbf4150690 legacy=0x7efbf415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf415dcb0 con 0x7efbf415d540 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7efbec001470 con 0x7efbf415d540 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 >> v1:172.21.15.3:6789/0 conn(0x7efbf415d540 legacy=0x7efbf40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbe0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbe0004810 0x7efbe0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbe00051c0 0x7efbe000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4159f40 con 0x7efbe0003a60 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbe000c0e0 con 0x7efbe00051c0 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbe000c2a0 con 0x7efbe0004810 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.474+0000 7efbf259c700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbe0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf2d9d700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbe00051c0 0x7efbe000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf1d9b700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbe0004810 0x7efbe0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf259c700 1 -- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbe0004810 msgr2=0x7efbe0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf259c700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbe0004810 0x7efbe0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf259c700 1 -- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbe00051c0 msgr2=0x7efbe000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf259c700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbe00051c0 0x7efbe000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf259c700 1 -- 172.21.15.3:0/2797229949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe0003570 con 0x7efbe0003a60 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf2d9d700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbe00051c0 0x7efbe000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf2d9d700 1 -- 172.21.15.3:0/2797229949 reap_dead start 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf1d9b700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbe0004810 0x7efbe0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf259c700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbe0003e30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efbe40020a0 tx=0x7efbe40016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b483000937029fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe40019e0 con 0x7efbe0003a60 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe400c8b0 con 0x7efbe0003a60 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.475+0000 7efbf159a700 1 -- 172.21.15.3:0/2797229949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe400cc50 con 0x7efbe0003a60 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.476+0000 7efbf8e78700 1 -- 172.21.15.3:0/2797229949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 msgr2=0x7efbe0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.476+0000 7efbf8e78700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbe0003e30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efbe40020a0 tx=0x7efbe40016d0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.476+0000 7efbf8e78700 1 -- 172.21.15.3:0/2797229949 shutdown_connections 2023-10-06T04:54:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.476+0000 7efbf8e78700 1 --2- 172.21.15.3:0/2797229949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbe0003e30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.476+0000 7efbf8e78700 1 -- 172.21.15.3:0/2797229949 >> 172.21.15.3:0/2797229949 conn(0x7efbf4070310 msgr2=0x7efbf4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.476+0000 7efbf8e78700 1 -- 172.21.15.3:0/2797229949 shutdown_connections 2023-10-06T04:54:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.476+0000 7efbf8e78700 1 -- 172.21.15.3:0/2797229949 wait complete. 2023-10-06T04:54:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.477+0000 7efbf8e78700 1 Processor -- start 2023-10-06T04:54:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.477+0000 7efbf8e78700 1 -- start start 2023-10-06T04:54:39.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.477+0000 7efbf8e78700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbf414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf8e78700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf4154510 0x7efbf4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf8e78700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf4154980 0x7efbf41592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf8e78700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf40b45c0 con 0x7efbe0003a60 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf8e78700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4151890 con 0x7efbf4154980 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf8e78700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4152410 con 0x7efbf4154510 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbf414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbf414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36884/0 (socket says 172.21.15.3:36884) 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 -- 172.21.15.3:0/669345193 learned_addr learned my addr 172.21.15.3:0/669345193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf1d9b700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf4154510 0x7efbf4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf2d9d700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf4154980 0x7efbf41592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 -- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf4154510 msgr2=0x7efbf4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf4154510 0x7efbf4158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 -- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf4154980 msgr2=0x7efbf41592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf4154980 0x7efbf41592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 -- 172.21.15.3:0/669345193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4151ae0 con 0x7efbe0003a60 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf2d9d700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf4154980 0x7efbf41592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf259c700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbf414f840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efbe400c6a0 tx=0x7efbe4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbea7fc700 1 -- 172.21.15.3:0/669345193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4012820 con 0x7efbe0003a60 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbea7fc700 1 -- 172.21.15.3:0/669345193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe4012980 con 0x7efbe0003a60 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf4151dc0 con 0x7efbe0003a60 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbea7fc700 1 -- 172.21.15.3:0/669345193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4012d40 con 0x7efbe0003a60 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.478+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf41528a0 con 0x7efbe0003a60 2023-10-06T04:54:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.479+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf404c380 con 0x7efbe0003a60 2023-10-06T04:54:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.480+0000 7efbea7fc700 1 -- 172.21.15.3:0/669345193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efbe401e640 con 0x7efbe0003a60 2023-10-06T04:54:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.480+0000 7efbea7fc700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efbd4066f30 0x7efbd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.480+0000 7efbea7fc700 1 -- 172.21.15.3:0/669345193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5576+0+0 (secure 0 0 0) 0x7efbe4086590 con 0x7efbe0003a60 2023-10-06T04:54:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.480+0000 7efbf1d9b700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efbd4066f30 0x7efbd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.481+0000 7efbf1d9b700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efbd4066f30 0x7efbd40693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efbf4159940 tx=0x7efbdc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.485+0000 7efbea7fc700 1 -- 172.21.15.3:0/669345193 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efbe4053880 con 0x7efbe0003a60 2023-10-06T04:54:39.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:39.677+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efbf415df90 con 0x7efbe0003a60 2023-10-06T04:54:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.325+0000 7efbea7fc700 1 -- 172.21.15.3:0/669345193 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7efbe4056ea0 con 0x7efbe0003a60 2023-10-06T04:54:40.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efbd4066f30 msgr2=0x7efbd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:40.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efbd4066f30 0x7efbd40693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efbf4159940 tx=0x7efbdc008040 comp rx=0 tx=0).stop 2023-10-06T04:54:40.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 msgr2=0x7efbf414f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:40.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbf414f840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efbe400c6a0 tx=0x7efbe4016000 comp rx=0 tx=0).stop 2023-10-06T04:54:40.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 shutdown_connections 2023-10-06T04:54:40.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efbd4066f30 0x7efbd40693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf4154980 0x7efbf41592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbe0003a60 0x7efbf414f840 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 --2- 172.21.15.3:0/669345193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf4154510 0x7efbf4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.330+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 >> 172.21.15.3:0/669345193 conn(0x7efbf4070310 msgr2=0x7efbf40c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:40.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.332+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 shutdown_connections 2023-10-06T04:54:40.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.333+0000 7efbf8e78700 1 -- 172.21.15.3:0/669345193 wait complete. 2023-10-06T04:54:40.336 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:54:40.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:54:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.480+0000 7f8bc6b07700 1 Processor -- start 2023-10-06T04:54:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.480+0000 7f8bc6b07700 1 -- start start 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.480+0000 7f8bc6b07700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc015c760 0x7f8bc015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.480+0000 7f8bc6b07700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8bc00afec0 0x7f8bc00cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.480+0000 7f8bc6b07700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc00cb810 con 0x7f8bc00cb3c0 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.480+0000 7f8bc6b07700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc00cf0f0 con 0x7f8bc00afec0 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.480+0000 7f8bc6b07700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00ce7c0 con 0x7f8bc015c760 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bc48a3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc015c760 0x7f8bc015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bc50a4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8bc00afec0 0x7f8bc00cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bbffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8bc00cb3c0 0x7f8bc015db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:46414/0 (socket says 172.21.15.3:46414) 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bbffff700 1 -- 172.21.15.3:0/3076517255 learned_addr learned my addr 172.21.15.3:0/3076517255 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bc48a3700 1 -- 172.21.15.3:0/3076517255 >> v2:172.21.15.153:3300/0 conn(0x7f8bc00afec0 msgr2=0x7f8bc00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bc48a3700 1 --2- 172.21.15.3:0/3076517255 >> v2:172.21.15.153:3300/0 conn(0x7f8bc00afec0 0x7f8bc00cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bc48a3700 1 -- 172.21.15.3:0/3076517255 >> v1:172.21.15.153:6789/0 conn(0x7f8bc00cb3c0 legacy=0x7f8bc015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bc48a3700 1 -- 172.21.15.3:0/3076517255 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00ceaa0 con 0x7f8bc015c760 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bc48a3700 1 --2- 172.21.15.3:0/3076517255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc015c760 0x7f8bc015cb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00098e0 tx=0x7f8bb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68cd8ddd7b41935 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.481+0000 7f8bbf7fe700 1 -- 172.21.15.3:0/3076517255 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000b640 con 0x7f8bc015c760 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bbf7fe700 1 -- 172.21.15.3:0/3076517255 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb000b7a0 con 0x7f8bc015c760 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bbf7fe700 1 -- 172.21.15.3:0/3076517255 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000bb40 con 0x7f8bc015c760 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 -- 172.21.15.3:0/3076517255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc015c760 msgr2=0x7f8bc015cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/3076517255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc015c760 0x7f8bc015cb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00098e0 tx=0x7f8bb00091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 -- 172.21.15.3:0/3076517255 shutdown_connections 2023-10-06T04:54:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/3076517255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc015c760 0x7f8bc015cb30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/3076517255 >> v2:172.21.15.153:3300/0 conn(0x7f8bc00afec0 0x7f8bc00cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 -- 172.21.15.3:0/3076517255 >> 172.21.15.3:0/3076517255 conn(0x7f8bc006f940 msgr2=0x7f8bc0070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 -- 172.21.15.3:0/3076517255 shutdown_connections 2023-10-06T04:54:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.482+0000 7f8bc6b07700 1 -- 172.21.15.3:0/3076517255 wait complete. 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.483+0000 7f8bc6b07700 1 Processor -- start 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.483+0000 7f8bc6b07700 1 -- start start 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc6b07700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc00afec0 0x7f8bc00c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc6b07700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc00cb3c0 0x7f8bc0155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc6b07700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc015c760 0x7f8bc0155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc6b07700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00c8610 con 0x7f8bc00afec0 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc6b07700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00c7a00 con 0x7f8bc015c760 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc6b07700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00c7ce0 con 0x7f8bc00cb3c0 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc48a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc00afec0 0x7f8bc00c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc48a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc00afec0 0x7f8bc00c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36898/0 (socket says 172.21.15.3:36898) 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc48a3700 1 -- 172.21.15.3:0/1316770497 learned_addr learned my addr 172.21.15.3:0/1316770497 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bc50a4700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc015c760 0x7f8bc0155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.484+0000 7f8bbffff700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc00cb3c0 0x7f8bc0155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc48a3700 1 -- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc00cb3c0 msgr2=0x7f8bc0155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc48a3700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc00cb3c0 0x7f8bc0155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc48a3700 1 -- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc015c760 msgr2=0x7f8bc0155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc48a3700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc015c760 0x7f8bc0155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc48a3700 1 -- 172.21.15.3:0/1316770497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0009420 con 0x7f8bc00afec0 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bbffff700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc00cb3c0 0x7f8bc0155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc48a3700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc00afec0 0x7f8bc00c9930 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00ce900 tx=0x7f8bb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bbcff9700 1 -- 172.21.15.3:0/1316770497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb001bc40 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc00c7fc0 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bbcff9700 1 -- 172.21.15.3:0/1316770497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb001bdc0 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bbcff9700 1 -- 172.21.15.3:0/1316770497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0011670 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.485+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc00c9e70 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.486+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc00ca060 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.486+0000 7f8bbcff9700 1 -- 172.21.15.3:0/1316770497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bb0011a20 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.487+0000 7f8bbcff9700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bac066f30 0x7f8bac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.487+0000 7f8bbcff9700 1 -- 172.21.15.3:0/1316770497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f8bb0085bb0 con 0x7f8bc00afec0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.487+0000 7f8bbffff700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bac066f30 0x7f8bac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.487+0000 7f8bbffff700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bac066f30 0x7f8bac0693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00b5c40 tx=0x7f8bb4007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.491+0000 7f8bbcff9700 1 -- 172.21.15.3:0/1316770497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bb0052d20 con 0x7f8bc00afec0 2023-10-06T04:54:40.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:40.733+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8bc004c380 con 0x7f8bc00afec0 2023-10-06T04:54:41.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.329+0000 7f8bbcff9700 1 -- 172.21.15.3:0/1316770497 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8bb00150c0 con 0x7f8bc00afec0 2023-10-06T04:54:41.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.336+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bac066f30 msgr2=0x7f8bac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.336+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bac066f30 0x7f8bac0693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00b5c40 tx=0x7f8bb4007eb0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.336+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc00afec0 msgr2=0x7f8bc00c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.336+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc00afec0 0x7f8bc00c9930 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00ce900 tx=0x7f8bb0017000 comp rx=0 tx=0).stop 2023-10-06T04:54:41.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.342+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 shutdown_connections 2023-10-06T04:54:41.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.342+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bac066f30 0x7f8bac0693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.342+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc015c760 0x7f8bc0155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.342+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc00afec0 0x7f8bc00c9930 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.342+0000 7f8bc6b07700 1 --2- 172.21.15.3:0/1316770497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc00cb3c0 0x7f8bc0155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.342+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 >> 172.21.15.3:0/1316770497 conn(0x7f8bc006f940 msgr2=0x7f8bc00b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:41.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.343+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 shutdown_connections 2023-10-06T04:54:41.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.349+0000 7f8bc6b07700 1 -- 172.21.15.3:0/1316770497 wait complete. 2023-10-06T04:54:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 9 and data pool 10 2023-10-06T04:54:41.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:54:41.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.490+0000 7f01aff8e700 1 Processor -- start 2023-10-06T04:54:41.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.490+0000 7f01aff8e700 1 -- start start 2023-10-06T04:54:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.490+0000 7f01aff8e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01a815cb50 0x7f01a80b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.490+0000 7f01aff8e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f01a80b0590 0x7f01a80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.490+0000 7f01aff8e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80cd950 0x7f01a80b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01aff8e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80b0ce0 con 0x7f01a815cb50 2023-10-06T04:54:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01aff8e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80cb820 con 0x7f01a80b0590 2023-10-06T04:54:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01aff8e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80b2c80 con 0x7f01a80cd950 2023-10-06T04:54:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01a815cb50 0x7f01a80b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01a815cb50 0x7f01a80b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36912/0 (socket says 172.21.15.3:36912) 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 -- 172.21.15.3:0/4082512376 learned_addr learned my addr 172.21.15.3:0/4082512376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01ad529700 1 --2- 172.21.15.3:0/4082512376 >> v2:172.21.15.153:3300/0 conn(0x7f01a80b0590 0x7f01a80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01ae52b700 1 --2- 172.21.15.3:0/4082512376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80cd950 0x7f01a80b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 -- 172.21.15.3:0/4082512376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80cd950 msgr2=0x7f01a80b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 --2- 172.21.15.3:0/4082512376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80cd950 0x7f01a80b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 -- 172.21.15.3:0/4082512376 >> v2:172.21.15.153:3300/0 conn(0x7f01a80b0590 msgr2=0x7f01a80b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 --2- 172.21.15.3:0/4082512376 >> v2:172.21.15.153:3300/0 conn(0x7f01a80b0590 0x7f01a80b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 -- 172.21.15.3:0/4082512376 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80b0a00 con 0x7f01a815cb50 2023-10-06T04:54:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01ad529700 1 --2- 172.21.15.3:0/4082512376 >> v2:172.21.15.153:3300/0 conn(0x7f01a80b0590 0x7f01a80b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01add2a700 1 --2- 172.21.15.3:0/4082512376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a815cb50 0x7f01a80b0050 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f01a80c73f0 tx=0x7f0198009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a174436f2a44680 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01acd28700 1 -- 172.21.15.3:0/4082512376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800e070 con 0x7f01a815cb50 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01acd28700 1 -- 172.21.15.3:0/4082512376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019800ba20 con 0x7f01a815cb50 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.491+0000 7f01acd28700 1 -- 172.21.15.3:0/4082512376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980092e0 con 0x7f01a815cb50 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 -- 172.21.15.3:0/4082512376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a815cb50 msgr2=0x7f01a80b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 --2- 172.21.15.3:0/4082512376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a815cb50 0x7f01a80b0050 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f01a80c73f0 tx=0x7f0198009840 comp rx=0 tx=0).stop 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 -- 172.21.15.3:0/4082512376 shutdown_connections 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 --2- 172.21.15.3:0/4082512376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80cd950 0x7f01a80b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 --2- 172.21.15.3:0/4082512376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a815cb50 0x7f01a80b0050 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 --2- 172.21.15.3:0/4082512376 >> v2:172.21.15.153:3300/0 conn(0x7f01a80b0590 0x7f01a80b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 -- 172.21.15.3:0/4082512376 >> 172.21.15.3:0/4082512376 conn(0x7f01a8070310 msgr2=0x7f01a8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:41.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 -- 172.21.15.3:0/4082512376 shutdown_connections 2023-10-06T04:54:41.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.492+0000 7f01aff8e700 1 -- 172.21.15.3:0/4082512376 wait complete. 2023-10-06T04:54:41.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.493+0000 7f01aff8e700 1 Processor -- start 2023-10-06T04:54:41.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.493+0000 7f01aff8e700 1 -- start start 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01aff8e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80b0590 0x7f01a81585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01aff8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a80cd950 0x7f01a80ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01aff8e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01a815cb50 0x7f01a80cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01aff8e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8158af0 con 0x7f01a80cd950 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01aff8e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80c3e90 con 0x7f01a815cb50 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01aff8e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80c4130 con 0x7f01a80b0590 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a80cd950 0x7f01a80ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a80cd950 0x7f01a80ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36914/0 (socket says 172.21.15.3:36914) 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 -- 172.21.15.3:0/307101944 learned_addr learned my addr 172.21.15.3:0/307101944 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:41.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01add2a700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80b0590 0x7f01a81585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ae52b700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01a815cb50 0x7f01a80cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 -- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80b0590 msgr2=0x7f01a81585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80b0590 0x7f01a81585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 -- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01a815cb50 msgr2=0x7f01a80cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01a815cb50 0x7f01a80cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 -- 172.21.15.3:0/307101944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198009ae0 con 0x7f01a80cd950 2023-10-06T04:54:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.494+0000 7f01ad529700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a80cd950 0x7f01a80ce6f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f019c00d7e0 tx=0x7f019c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.495+0000 7f01a5ffb700 1 -- 172.21.15.3:0/307101944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c0125f0 con 0x7f01a80cd950 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.495+0000 7f01a5ffb700 1 -- 172.21.15.3:0/307101944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c012750 con 0x7f01a80cd950 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.495+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a80c4410 con 0x7f01a80cd950 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.495+0000 7f01a5ffb700 1 -- 172.21.15.3:0/307101944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c012b10 con 0x7f01a80cd950 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.495+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80c49d0 con 0x7f01a80cd950 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.496+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a8071680 con 0x7f01a80cd950 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.496+0000 7f01a5ffb700 1 -- 172.21.15.3:0/307101944 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f019c013480 con 0x7f01a80cd950 2023-10-06T04:54:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.496+0000 7f01a5ffb700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0194066f30 0x7f01940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.497+0000 7f01a5ffb700 1 -- 172.21.15.3:0/307101944 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f019c0849e0 con 0x7f01a80cd950 2023-10-06T04:54:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.497+0000 7f01add2a700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0194066f30 0x7f01940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.497+0000 7f01add2a700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0194066f30 0x7f01940693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0198009fd0 tx=0x7f0198018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.501+0000 7f01a5ffb700 1 -- 172.21.15.3:0/307101944 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f019c051af0 con 0x7f01a80cd950 2023-10-06T04:54:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.696+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01a80cb1d0 con 0x7f01a80cd950 2023-10-06T04:54:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.697+0000 7f01a5ffb700 1 -- 172.21.15.3:0/307101944 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16833 (secure 0 0 0) 0x7f019c055110 con 0x7f01a80cd950 2023-10-06T04:54:41.699 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:41.699 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":57,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:54:40.734907+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:54:38.486656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T04:54:39.707870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:54:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0194066f30 msgr2=0x7f01940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0194066f30 0x7f01940693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0198009fd0 tx=0x7f0198018000 comp rx=0 tx=0).stop 2023-10-06T04:54:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a80cd950 msgr2=0x7f01a80ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a80cd950 0x7f01a80ce6f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f019c00d7e0 tx=0x7f019c008040 comp rx=0 tx=0).stop 2023-10-06T04:54:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 shutdown_connections 2023-10-06T04:54:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0194066f30 0x7f01940693e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01a815cb50 0x7f01a80cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01a80b0590 0x7f01a81585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 --2- 172.21.15.3:0/307101944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01a80cd950 0x7f01a80ce6f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 >> 172.21.15.3:0/307101944 conn(0x7f01a8070310 msgr2=0x7f01a815a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 shutdown_connections 2023-10-06T04:54:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.699+0000 7f01aff8e700 1 -- 172.21.15.3:0/307101944 wait complete. 2023-10-06T04:54:41.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:54:41.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.832+0000 7f8424828700 1 Processor -- start 2023-10-06T04:54:41.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.832+0000 7f8424828700 1 -- start start 2023-10-06T04:54:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f8424828700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 0x7f842015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f8424828700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f84200ce020 0x7f84200ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f8424828700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842015abc0 con 0x7f84200c8630 2023-10-06T04:54:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f8424828700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842015e140 con 0x7f84200ce020 2023-10-06T04:54:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f8424828700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200ce9e0 con 0x7f8420157160 2023-10-06T04:54:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f84200c8630 0x7f842015a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56770/0 (socket says 172.21.15.3:56770) 2023-10-06T04:54:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841e59c700 1 -- 172.21.15.3:0/2497004066 learned_addr learned my addr 172.21.15.3:0/2497004066 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841f59e700 1 --2- 172.21.15.3:0/2497004066 >> v2:172.21.15.153:3300/0 conn(0x7f84200ce020 0x7f84200ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841ed9d700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 0x7f842015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963254243 0 0) 0x7f8408001110 con 0x7f84200c8630 2023-10-06T04:54:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f840c003570 con 0x7f84200c8630 2023-10-06T04:54:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 149478723 0 0) 0x7f840c003570 con 0x7f84200c8630 2023-10-06T04:54:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 msgr2=0x7f842015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 0x7f842015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 >> v2:172.21.15.153:3300/0 conn(0x7f84200ce020 msgr2=0x7f84200ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 --2- 172.21.15.3:0/2497004066 >> v2:172.21.15.153:3300/0 conn(0x7f84200ce020 0x7f84200ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842015d1b0 con 0x7f84200c8630 2023-10-06T04:54:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f8408001600 con 0x7f84200c8630 2023-10-06T04:54:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.833+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 >> v1:172.21.15.3:6789/0 conn(0x7f84200c8630 legacy=0x7f842015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 0x7f840c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c004010 0x7f840c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004d50 0x7f840c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842015e140 con 0x7f840c004010 2023-10-06T04:54:41.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200ce9e0 con 0x7f840c004d50 2023-10-06T04:54:41.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c00bab0 con 0x7f8420157160 2023-10-06T04:54:41.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841e59c700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c004010 0x7f840c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841ed9d700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 0x7f840c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841f59e700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004d50 0x7f840c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841e59c700 1 -- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 msgr2=0x7f840c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:41.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841e59c700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420157160 0x7f840c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841e59c700 1 -- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004d50 msgr2=0x7f840c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841e59c700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004d50 0x7f840c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841e59c700 1 -- 172.21.15.3:0/2497004066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c003570 con 0x7f840c004010 2023-10-06T04:54:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841e59c700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c004010 0x7f840c004810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f840800c5f0 tx=0x7f842015d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc459206c5d93d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800cdf0 con 0x7f840c004010 2023-10-06T04:54:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408005920 con 0x7f840c004010 2023-10-06T04:54:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.834+0000 7f841dd9b700 1 -- 172.21.15.3:0/2497004066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408005ce0 con 0x7f840c004010 2023-10-06T04:54:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.835+0000 7f8424828700 1 -- 172.21.15.3:0/2497004066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c004010 msgr2=0x7f840c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.835+0000 7f8424828700 1 --2- 172.21.15.3:0/2497004066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c004010 0x7f840c004810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f840800c5f0 tx=0x7f842015d6e0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.835+0000 7f841f59e700 1 -- 172.21.15.3:0/2497004066 reap_dead start 2023-10-06T04:54:41.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.835+0000 7f8424828700 1 -- 172.21.15.3:0/2497004066 shutdown_connections 2023-10-06T04:54:41.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.835+0000 7f8424828700 1 -- 172.21.15.3:0/2497004066 >> 172.21.15.3:0/2497004066 conn(0x7f8420070310 msgr2=0x7f8420070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:41.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.835+0000 7f8424828700 1 -- 172.21.15.3:0/2497004066 shutdown_connections 2023-10-06T04:54:41.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.835+0000 7f8424828700 1 -- 172.21.15.3:0/2497004066 wait complete. 2023-10-06T04:54:41.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.836+0000 7f8424828700 1 Processor -- start 2023-10-06T04:54:41.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.836+0000 7f8424828700 1 -- start start 2023-10-06T04:54:41.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f8424828700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004010 0x7f84200b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f8424828700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84200c6d80 0x7f84200c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f8424828700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420154460 0x7f84200ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f8424828700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200c71f0 con 0x7f84200c6d80 2023-10-06T04:54:41.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f8424828700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200b3570 con 0x7f840c004010 2023-10-06T04:54:41.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f8424828700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200c62c0 con 0x7f8420154460 2023-10-06T04:54:41.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84200c6d80 0x7f84200c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84200c6d80 0x7f84200c5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36946/0 (socket says 172.21.15.3:36946) 2023-10-06T04:54:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 -- 172.21.15.3:0/1026836887 learned_addr learned my addr 172.21.15.3:0/1026836887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841ed9d700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004010 0x7f84200b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841f59e700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420154460 0x7f84200ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 -- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420154460 msgr2=0x7f84200ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420154460 0x7f84200ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 -- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004010 msgr2=0x7f84200b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004010 0x7f84200b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 -- 172.21.15.3:0/1026836887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200c5ff0 con 0x7f84200c6d80 2023-10-06T04:54:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841ed9d700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004010 0x7f84200b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.837+0000 7f841e59c700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84200c6d80 0x7f84200c5ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f840800cbe0 tx=0x7f8408015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:41.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.838+0000 7f8416ffd700 1 -- 172.21.15.3:0/1026836887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080010f0 con 0x7f84200c6d80 2023-10-06T04:54:41.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.838+0000 7f8416ffd700 1 -- 172.21.15.3:0/1026836887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84080072e0 con 0x7f84200c6d80 2023-10-06T04:54:41.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.838+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84200b2c40 con 0x7f84200c6d80 2023-10-06T04:54:41.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.838+0000 7f8416ffd700 1 -- 172.21.15.3:0/1026836887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408007660 con 0x7f84200c6d80 2023-10-06T04:54:41.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.838+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84200b3200 con 0x7f84200c6d80 2023-10-06T04:54:41.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.839+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84200b3700 con 0x7f84200c6d80 2023-10-06T04:54:41.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.840+0000 7f8416ffd700 1 -- 172.21.15.3:0/1026836887 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8408019070 con 0x7f84200c6d80 2023-10-06T04:54:41.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.841+0000 7f8416ffd700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8400066f30 0x7f84000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:41.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.841+0000 7f8416ffd700 1 -- 172.21.15.3:0/1026836887 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f84080854a0 con 0x7f84200c6d80 2023-10-06T04:54:41.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.841+0000 7f841ed9d700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8400066f30 0x7f84000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:41.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.841+0000 7f841ed9d700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8400066f30 0x7f84000693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f84100099f0 tx=0x7f841000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:41.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:41.844+0000 7f8416ffd700 1 -- 172.21.15.3:0/1026836887 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8408052640 con 0x7f84200c6d80 2023-10-06T04:54:42.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.040+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f842015b390 con 0x7f84200c6d80 2023-10-06T04:54:42.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.041+0000 7f8416ffd700 1 -- 172.21.15.3:0/1026836887 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16833 (secure 0 0 0) 0x7f842015b390 con 0x7f84200c6d80 2023-10-06T04:54:42.043 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:42.043 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":57,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:54:40.734907+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:54:38.486656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T04:54:39.707870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:54:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.043+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8400066f30 msgr2=0x7f84000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.043+0000 7f8424828700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8400066f30 0x7f84000693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f84100099f0 tx=0x7f841000b040 comp rx=0 tx=0).stop 2023-10-06T04:54:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.043+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84200c6d80 msgr2=0x7f84200c5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.043+0000 7f8424828700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84200c6d80 0x7f84200c5ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f840800cbe0 tx=0x7f8408015000 comp rx=0 tx=0).stop 2023-10-06T04:54:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 shutdown_connections 2023-10-06T04:54:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8400066f30 0x7f84000693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f840c004010 0x7f84200b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8420154460 0x7f84200ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 --2- 172.21.15.3:0/1026836887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84200c6d80 0x7f84200c5ab0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 >> 172.21.15.3:0/1026836887 conn(0x7f8420070310 msgr2=0x7f8420156720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 shutdown_connections 2023-10-06T04:54:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.044+0000 7f8424828700 1 -- 172.21.15.3:0/1026836887 wait complete. 2023-10-06T04:54:42.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:54:42.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.178+0000 7f4231947700 1 Processor -- start 2023-10-06T04:54:42.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.178+0000 7f4231947700 1 -- start start 2023-10-06T04:54:42.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f4231947700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:42.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f4231947700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f422c150260 0x7f422c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f4231947700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c06a9d0 con 0x7f422c069f60 2023-10-06T04:54:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f4231947700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c06acb0 con 0x7f422c150260 2023-10-06T04:54:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f4231947700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c06af90 con 0x7f422c1597e0 2023-10-06T04:54:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f422b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f422c150260 0x7f422c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f422b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f422c150260 0x7f422c06a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36948/0 (socket says 172.21.15.3:36948) 2023-10-06T04:54:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.179+0000 7f422b7fe700 1 -- 172.21.15.3:0/1506196682 learned_addr learned my addr 172.21.15.3:0/1506196682 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f422affd700 1 --2- 172.21.15.3:0/1506196682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f422b7fe700 1 -- 172.21.15.3:0/1506196682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 msgr2=0x7f422c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f422b7fe700 1 --2- 172.21.15.3:0/1506196682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f422b7fe700 1 -- 172.21.15.3:0/1506196682 >> v1:172.21.15.153:6789/0 conn(0x7f422c069f60 legacy=0x7f422c14fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f422b7fe700 1 -- 172.21.15.3:0/1506196682 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c06b270 con 0x7f422c150260 2023-10-06T04:54:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f422affd700 1 --2- 172.21.15.3:0/1506196682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f422b7fe700 1 --2- 172.21.15.3:0/1506196682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c06a3d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f421c009f20 tx=0x7f421c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833d5d04d65882f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:42.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f4229ffb700 1 -- 172.21.15.3:0/1506196682 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c00e070 con 0x7f422c150260 2023-10-06T04:54:42.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f4229ffb700 1 -- 172.21.15.3:0/1506196682 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c00ba40 con 0x7f422c150260 2023-10-06T04:54:42.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f4229ffb700 1 -- 172.21.15.3:0/1506196682 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0092e0 con 0x7f422c150260 2023-10-06T04:54:42.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f4231947700 1 -- 172.21.15.3:0/1506196682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 msgr2=0x7f422c06a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:42.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.180+0000 7f4231947700 1 --2- 172.21.15.3:0/1506196682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c06a3d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f421c009f20 tx=0x7f421c009840 comp rx=0 tx=0).stop 2023-10-06T04:54:42.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.181+0000 7f4231947700 1 -- 172.21.15.3:0/1506196682 shutdown_connections 2023-10-06T04:54:42.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.181+0000 7f4231947700 1 --2- 172.21.15.3:0/1506196682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c06a3d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.181+0000 7f4231947700 1 --2- 172.21.15.3:0/1506196682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.181+0000 7f4231947700 1 -- 172.21.15.3:0/1506196682 >> 172.21.15.3:0/1506196682 conn(0x7f422c06ce40 msgr2=0x7f422c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:42.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.181+0000 7f4231947700 1 -- 172.21.15.3:0/1506196682 shutdown_connections 2023-10-06T04:54:42.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.181+0000 7f4231947700 1 -- 172.21.15.3:0/1506196682 wait complete. 2023-10-06T04:54:42.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.182+0000 7f4231947700 1 Processor -- start 2023-10-06T04:54:42.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.182+0000 7f4231947700 1 -- start start 2023-10-06T04:54:42.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f422c069f60 0x7f422c0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:42.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c14a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:42.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:42.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0ac4c0 con 0x7f422c150260 2023-10-06T04:54:42.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0c4330 con 0x7f422c069f60 2023-10-06T04:54:42.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0c4610 con 0x7f422c1597e0 2023-10-06T04:54:42.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c14a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:42.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c14a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36960/0 (socket says 172.21.15.3:36960) 2023-10-06T04:54:42.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 -- 172.21.15.3:0/1215378491 learned_addr learned my addr 172.21.15.3:0/1215378491 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:42.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422affd700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f422c069f60 0x7f422c0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:42.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422b7fe700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:42.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 -- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 msgr2=0x7f422c14ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:42.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c14ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 -- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f422c069f60 msgr2=0x7f422c0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:42.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f422c069f60 0x7f422c0affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:42.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 -- 172.21.15.3:0/1215378491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c009ae0 con 0x7f422c150260 2023-10-06T04:54:42.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422affd700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f422c069f60 0x7f422c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:42.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422b7fe700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:42.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f422a7fc700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c14a720 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f421400b5e0 tx=0x7f4214009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:42.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4222ffd700 1 -- 172.21.15.3:0/1215378491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421400f510 con 0x7f422c150260 2023-10-06T04:54:42.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4222ffd700 1 -- 172.21.15.3:0/1215378491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421400f670 con 0x7f422c150260 2023-10-06T04:54:42.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c0c48f0 con 0x7f422c150260 2023-10-06T04:54:42.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4222ffd700 1 -- 172.21.15.3:0/1215378491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421400fa30 con 0x7f422c150260 2023-10-06T04:54:42.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.183+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c0b0520 con 0x7f422c150260 2023-10-06T04:54:42.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.184+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c04c380 con 0x7f422c150260 2023-10-06T04:54:42.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.185+0000 7f4222ffd700 1 -- 172.21.15.3:0/1215378491 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f421401a400 con 0x7f422c150260 2023-10-06T04:54:42.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.185+0000 7f4222ffd700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f420c066f30 0x7f420c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:42.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.185+0000 7f4222ffd700 1 -- 172.21.15.3:0/1215378491 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f42140841a0 con 0x7f422c150260 2023-10-06T04:54:42.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.185+0000 7f422affd700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f420c066f30 0x7f420c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:42.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.186+0000 7f422affd700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f420c066f30 0x7f420c0693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4224008b20 tx=0x7f4224008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:42.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.189+0000 7f4222ffd700 1 -- 172.21.15.3:0/1215378491 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4214051330 con 0x7f422c150260 2023-10-06T04:54:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:42.430+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f422c0ca210 con 0x7f422c150260 2023-10-06T04:54:43.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.357+0000 7f4222ffd700 1 -- 172.21.15.3:0/1215378491 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4214054950 con 0x7f422c150260 2023-10-06T04:54:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f420c066f30 msgr2=0x7f420c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f420c066f30 0x7f420c0693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4224008b20 tx=0x7f4224008040 comp rx=0 tx=0).stop 2023-10-06T04:54:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 msgr2=0x7f422c14a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c14a720 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f421400b5e0 tx=0x7f4214009120 comp rx=0 tx=0).stop 2023-10-06T04:54:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 shutdown_connections 2023-10-06T04:54:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f420c066f30 0x7f420c0693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:43.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f422c069f60 0x7f422c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f422c1597e0 0x7f422c14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 --2- 172.21.15.3:0/1215378491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f422c150260 0x7f422c14a720 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 >> 172.21.15.3:0/1215378491 conn(0x7f422c06ce40 msgr2=0x7f422c06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 shutdown_connections 2023-10-06T04:54:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.360+0000 7f4231947700 1 -- 172.21.15.3:0/1215378491 wait complete. 2023-10-06T04:54:43.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:54:43.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.504+0000 7f3f756b6700 1 Processor -- start 2023-10-06T04:54:43.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.504+0000 7f3f756b6700 1 -- start start 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f756b6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f70150690 0x7f3f70150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f756b6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7015a290 con 0x7f3f700cba50 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f756b6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f700d0020 con 0x7f3f7015d540 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f756b6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f70159f40 con 0x7f3f70150690 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f6effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f70150690 0x7f3f70150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f6e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3f700cba50 0x7f3f7015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48200/0 (socket says 172.21.15.3:48200) 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f6e7fc700 1 -- 172.21.15.3:0/2538976473 learned_addr learned my addr 172.21.15.3:0/2538976473 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f6effd700 1 -- 172.21.15.3:0/2538976473 >> v1:172.21.15.153:6789/0 conn(0x7f3f7015d540 legacy=0x7f3f700af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f6effd700 1 -- 172.21.15.3:0/2538976473 >> v1:172.21.15.3:6789/0 conn(0x7f3f700cba50 legacy=0x7f3f7015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f6effd700 1 -- 172.21.15.3:0/2538976473 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7015dcb0 con 0x7f3f70150690 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.505+0000 7f3f6dffb700 1 -- 172.21.15.3:0/2538976473 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843019206 0 0) 0x7f3f7015a290 con 0x7f3f700cba50 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f6effd700 1 --2- 172.21.15.3:0/2538976473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f70150690 0x7f3f70150a60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009e60 tx=0x7f3f60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80633e0fe5a7fb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f6dffb700 1 -- 172.21.15.3:0/2538976473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000e070 con 0x7f3f70150690 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f6dffb700 1 -- 172.21.15.3:0/2538976473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6000bc30 con 0x7f3f70150690 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f6dffb700 1 -- 172.21.15.3:0/2538976473 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60012430 con 0x7f3f70150690 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f756b6700 1 -- 172.21.15.3:0/2538976473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f70150690 msgr2=0x7f3f70150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f756b6700 1 --2- 172.21.15.3:0/2538976473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f70150690 0x7f3f70150a60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009e60 tx=0x7f3f60009780 comp rx=0 tx=0).stop 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f756b6700 1 -- 172.21.15.3:0/2538976473 shutdown_connections 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f756b6700 1 --2- 172.21.15.3:0/2538976473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f70150690 0x7f3f70150a60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f756b6700 1 -- 172.21.15.3:0/2538976473 >> 172.21.15.3:0/2538976473 conn(0x7f3f70070310 msgr2=0x7f3f70070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f756b6700 1 -- 172.21.15.3:0/2538976473 shutdown_connections 2023-10-06T04:54:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.506+0000 7f3f756b6700 1 -- 172.21.15.3:0/2538976473 wait complete. 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 Processor -- start 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 -- start start 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f700cba50 0x7f3f700c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70150690 0x7f3f700b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f7015d540 0x7f3f700b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701537c0 con 0x7f3f700cba50 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700c8530 con 0x7f3f70150690 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f756b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700c8740 con 0x7f3f7015d540 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f6effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f700cba50 0x7f3f700c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f6effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f700cba50 0x7f3f700c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44904/0 (socket says 172.21.15.3:44904) 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.508+0000 7f3f6effd700 1 -- 172.21.15.3:0/892415694 learned_addr learned my addr 172.21.15.3:0/892415694 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6e7fc700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70150690 0x7f3f700b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6f7fe700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f7015d540 0x7f3f700b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6effd700 1 -- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f7015d540 msgr2=0x7f3f700b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6effd700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f7015d540 0x7f3f700b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6effd700 1 -- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70150690 msgr2=0x7f3f700b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6effd700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70150690 0x7f3f700b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6effd700 1 -- 172.21.15.3:0/892415694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f60009a20 con 0x7f3f700cba50 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f6effd700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f700cba50 0x7f3f700c9500 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f600099f0 tx=0x7f3f60019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f66ffd700 1 -- 172.21.15.3:0/892415694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60012db0 con 0x7f3f700cba50 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f66ffd700 1 -- 172.21.15.3:0/892415694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6001b420 con 0x7f3f700cba50 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f66ffd700 1 -- 172.21.15.3:0/892415694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6001b7e0 con 0x7f3f700cba50 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f700c8a20 con 0x7f3f700cba50 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.509+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f700c8fe0 con 0x7f3f700cba50 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.510+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f700c91d0 con 0x7f3f700cba50 2023-10-06T04:54:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.510+0000 7f3f66ffd700 1 -- 172.21.15.3:0/892415694 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f6001bb90 con 0x7f3f700cba50 2023-10-06T04:54:43.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.511+0000 7f3f66ffd700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:43.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.511+0000 7f3f66ffd700 1 -- 172.21.15.3:0/892415694 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f60085f50 con 0x7f3f700cba50 2023-10-06T04:54:43.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.511+0000 7f3f6e7fc700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:43.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.512+0000 7f3f6e7fc700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f700b6460 tx=0x7f3f58007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:43.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.515+0000 7f3f66ffd700 1 -- 172.21.15.3:0/892415694 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f60053060 con 0x7f3f700cba50 2023-10-06T04:54:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:43.751+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3f7004c380 con 0x7f3f700cba50 2023-10-06T04:54:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.362+0000 7f3f66ffd700 1 -- 172.21.15.3:0/892415694 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3f6002a020 con 0x7f3f700cba50 2023-10-06T04:54:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.364+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 msgr2=0x7f3f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.364+0000 7f3f756b6700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f700b6460 tx=0x7f3f58007d30 comp rx=0 tx=0).stop 2023-10-06T04:54:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.364+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f700cba50 msgr2=0x7f3f700c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.364+0000 7f3f756b6700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f700cba50 0x7f3f700c9500 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f600099f0 tx=0x7f3f60019000 comp rx=0 tx=0).stop 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.364+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 shutdown_connections 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.364+0000 7f3f756b6700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.364+0000 7f3f756b6700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f7015d540 0x7f3f700b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.365+0000 7f3f756b6700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f700cba50 0x7f3f700c9500 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.365+0000 7f3f756b6700 1 --2- 172.21.15.3:0/892415694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70150690 0x7f3f700b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.365+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 >> 172.21.15.3:0/892415694 conn(0x7f3f70070310 msgr2=0x7f3f7015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.365+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 shutdown_connections 2023-10-06T04:54:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.365+0000 7f3f756b6700 1 -- 172.21.15.3:0/892415694 wait complete. 2023-10-06T04:54:44.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:44.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.511+0000 7f8aed6dd700 1 Processor -- start 2023-10-06T04:54:44.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.511+0000 7f8aed6dd700 1 -- start start 2023-10-06T04:54:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8aed6dd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8ae8156330 0x7f8ae80cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8aed6dd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ae80cd8f0 0x7f8ae80b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8aed6dd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae806b430 con 0x7f8ae8155f60 2023-10-06T04:54:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8aed6dd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80cdd70 con 0x7f8ae80cd8f0 2023-10-06T04:54:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8aed6dd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae806b710 con 0x7f8ae8156330 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8ae77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ae80cd8f0 0x7f8ae80b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8ae6ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ae8155f60 0x7f8ae806a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48206/0 (socket says 172.21.15.3:48206) 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/2263385826 learned_addr learned my addr 172.21.15.3:0/2263385826 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8ae67fc700 1 --2- 172.21.15.3:0/2263385826 >> v2:172.21.15.153:3300/0 conn(0x7f8ae8156330 0x7f8ae80cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8ae77fe700 1 -- 172.21.15.3:0/2263385826 >> v2:172.21.15.153:3300/0 conn(0x7f8ae8156330 msgr2=0x7f8ae80cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8ae77fe700 1 --2- 172.21.15.3:0/2263385826 >> v2:172.21.15.153:3300/0 conn(0x7f8ae8156330 0x7f8ae80cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.512+0000 7f8ae77fe700 1 -- 172.21.15.3:0/2263385826 >> v1:172.21.15.3:6789/0 conn(0x7f8ae8155f60 legacy=0x7f8ae806a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8ae77fe700 1 -- 172.21.15.3:0/2263385826 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae806b9f0 con 0x7f8ae80cd8f0 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8ae5ffb700 1 -- 172.21.15.3:0/2263385826 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305784794 0 0) 0x7f8ae806b430 con 0x7f8ae8155f60 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8ae67fc700 1 --2- 172.21.15.3:0/2263385826 >> v2:172.21.15.153:3300/0 conn(0x7f8ae8156330 0x7f8ae80cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8ae77fe700 1 --2- 172.21.15.3:0/2263385826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae80cd8f0 0x7f8ae80b06d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80b2aa0 tx=0x7f8ad80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95947b4d3806b1e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8ae5ffb700 1 -- 172.21.15.3:0/2263385826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800b6b0 con 0x7f8ae80cd8f0 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8ae5ffb700 1 -- 172.21.15.3:0/2263385826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad800b810 con 0x7f8ae80cd8f0 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8ae5ffb700 1 -- 172.21.15.3:0/2263385826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800bbb0 con 0x7f8ae80cd8f0 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8aed6dd700 1 -- 172.21.15.3:0/2263385826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae80cd8f0 msgr2=0x7f8ae80b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/2263385826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae80cd8f0 0x7f8ae80b06d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80b2aa0 tx=0x7f8ad80091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8aed6dd700 1 -- 172.21.15.3:0/2263385826 shutdown_connections 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/2263385826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae80cd8f0 0x7f8ae80b06d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.513+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/2263385826 >> v2:172.21.15.153:3300/0 conn(0x7f8ae8156330 0x7f8ae80cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.514+0000 7f8aed6dd700 1 -- 172.21.15.3:0/2263385826 >> 172.21.15.3:0/2263385826 conn(0x7f8ae80687d0 msgr2=0x7f8ae8068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.514+0000 7f8aed6dd700 1 -- 172.21.15.3:0/2263385826 shutdown_connections 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.514+0000 7f8aed6dd700 1 -- 172.21.15.3:0/2263385826 wait complete. 2023-10-06T04:54:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.515+0000 7f8aed6dd700 1 Processor -- start 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.515+0000 7f8aed6dd700 1 -- start start 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8aed6dd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae80cd8f0 0x7f8ae814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8aed6dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8155f60 0x7f8ae8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8aed6dd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae8156330 0x7f8ae80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8aed6dd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae816b560 con 0x7f8ae8155f60 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8aed6dd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae816a8e0 con 0x7f8ae80cd8f0 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8aed6dd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae816abc0 con 0x7f8ae8156330 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8155f60 0x7f8ae8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8155f60 0x7f8ae8158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44918/0 (socket says 172.21.15.3:44918) 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 -- 172.21.15.3:0/1115139541 learned_addr learned my addr 172.21.15.3:0/1115139541 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae6ffd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae80cd8f0 0x7f8ae814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae77fe700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae8156330 0x7f8ae80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 -- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae8156330 msgr2=0x7f8ae80a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae8156330 0x7f8ae80a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 -- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae80cd8f0 msgr2=0x7f8ae814fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae80cd8f0 0x7f8ae814fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 -- 172.21.15.3:0/1115139541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad8009420 con 0x7f8ae8155f60 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8ae67fc700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8155f60 0x7f8ae8158a30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c830 tx=0x7f8ad0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8adeffd700 1 -- 172.21.15.3:0/1115139541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0011600 con 0x7f8ae8155f60 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8adeffd700 1 -- 172.21.15.3:0/1115139541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0011760 con 0x7f8ae8155f60 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae816aea0 con 0x7f8ae8155f60 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.516+0000 7f8adeffd700 1 -- 172.21.15.3:0/1115139541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0011b20 con 0x7f8ae8155f60 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.517+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae80a5ad0 con 0x7f8ae8155f60 2023-10-06T04:54:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.517+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae804c380 con 0x7f8ae8155f60 2023-10-06T04:54:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.518+0000 7f8adeffd700 1 -- 172.21.15.3:0/1115139541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ad0018360 con 0x7f8ae8155f60 2023-10-06T04:54:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.518+0000 7f8adeffd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ac8066f30 0x7f8ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.518+0000 7f8adeffd700 1 -- 172.21.15.3:0/1115139541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ad00845a0 con 0x7f8ae8155f60 2023-10-06T04:54:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.518+0000 7f8ae6ffd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ac8066f30 0x7f8ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.519+0000 7f8ae6ffd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ac8066f30 0x7f8ac80693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0002250 tx=0x7f8ae0001a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.523+0000 7f8adeffd700 1 -- 172.21.15.3:0/1115139541 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ad00516b0 con 0x7f8ae8155f60 2023-10-06T04:54:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.761+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ae80afed0 con 0x7f8ae8155f60 2023-10-06T04:54:44.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.761+0000 7f8adeffd700 1 -- 172.21.15.3:0/1115139541 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7f8ad0054cd0 con 0x7f8ae8155f60 2023-10-06T04:54:44.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:44.763 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:54:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ac8066f30 msgr2=0x7f8ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ac8066f30 0x7f8ac80693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0002250 tx=0x7f8ae0001a20 comp rx=0 tx=0).stop 2023-10-06T04:54:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8155f60 msgr2=0x7f8ae8158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8155f60 0x7f8ae8158a30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c830 tx=0x7f8ad0008040 comp rx=0 tx=0).stop 2023-10-06T04:54:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 shutdown_connections 2023-10-06T04:54:44.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ac8066f30 0x7f8ac80693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ae80cd8f0 0x7f8ae814fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ae8156330 0x7f8ae80a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 --2- 172.21.15.3:0/1115139541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8155f60 0x7f8ae8158a30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 >> 172.21.15.3:0/1115139541 conn(0x7f8ae80687d0 msgr2=0x7f8ae8151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:44.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 shutdown_connections 2023-10-06T04:54:44.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.763+0000 7f8aed6dd700 1 -- 172.21.15.3:0/1115139541 wait complete. 2023-10-06T04:54:44.767 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:54:44.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:54:44.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.895+0000 7f0d9d4d6700 1 Processor -- start 2023-10-06T04:54:44.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.895+0000 7f0d9d4d6700 1 -- start start 2023-10-06T04:54:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9d4d6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0d98142de0 0x7f0d9806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9d4d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d98068160 0x7f0d980b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9d4d6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9815a9b0 con 0x7f0d980b15c0 2023-10-06T04:54:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9d4d6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d981431b0 con 0x7f0d98068160 2023-10-06T04:54:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9d4d6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d98159ae0 con 0x7f0d98142de0 2023-10-06T04:54:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9ccd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d98068160 0x7f0d980b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9ccd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d98068160 0x7f0d980b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44928/0 (socket says 172.21.15.3:44928) 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9ccd5700 1 -- 172.21.15.3:0/1906991886 learned_addr learned my addr 172.21.15.3:0/1906991886 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d97fff700 1 --2- 172.21.15.3:0/1906991886 >> v2:172.21.15.153:3300/0 conn(0x7f0d98142de0 0x7f0d9806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9ccd5700 1 -- 172.21.15.3:0/1906991886 >> v2:172.21.15.153:3300/0 conn(0x7f0d98142de0 msgr2=0x7f0d9806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9ccd5700 1 --2- 172.21.15.3:0/1906991886 >> v2:172.21.15.153:3300/0 conn(0x7f0d98142de0 0x7f0d9806d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9ccd5700 1 -- 172.21.15.3:0/1906991886 >> v1:172.21.15.153:6789/0 conn(0x7f0d980b15c0 legacy=0x7f0d98067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.896+0000 7f0d9ccd5700 1 -- 172.21.15.3:0/1906991886 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98159da0 con 0x7f0d98068160 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9ccd5700 1 --2- 172.21.15.3:0/1906991886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d98068160 0x7f0d980b3330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806d2b0 tx=0x7f0d90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c6129f7eca5b15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d96ffd700 1 -- 172.21.15.3:0/1906991886 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000e070 con 0x7f0d98068160 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d96ffd700 1 -- 172.21.15.3:0/1906991886 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9000ba20 con 0x7f0d98068160 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d96ffd700 1 -- 172.21.15.3:0/1906991886 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900092e0 con 0x7f0d98068160 2023-10-06T04:54:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/1906991886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d98068160 msgr2=0x7f0d980b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/1906991886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d98068160 0x7f0d980b3330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806d2b0 tx=0x7f0d90009840 comp rx=0 tx=0).stop 2023-10-06T04:54:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/1906991886 shutdown_connections 2023-10-06T04:54:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/1906991886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d98068160 0x7f0d980b3330 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/1906991886 >> v2:172.21.15.153:3300/0 conn(0x7f0d98142de0 0x7f0d9806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/1906991886 >> 172.21.15.3:0/1906991886 conn(0x7f0d9806ce40 msgr2=0x7f0d980afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/1906991886 shutdown_connections 2023-10-06T04:54:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.897+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/1906991886 wait complete. 2023-10-06T04:54:44.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.898+0000 7f0d9d4d6700 1 Processor -- start 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d9d4d6700 1 -- start start 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d9d4d6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d98068160 0x7f0d9816d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d9d4d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d980b15c0 0x7f0d98169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d9d4d6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d98142de0 0x7f0d98140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d9d4d6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9816dd50 con 0x7f0d980b15c0 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d9d4d6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980cd4f0 con 0x7f0d98142de0 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d9d4d6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980cab50 con 0x7f0d98068160 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d980b15c0 0x7f0d98169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d980b15c0 0x7f0d98169880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44942/0 (socket says 172.21.15.3:44942) 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.899+0000 7f0d977fe700 1 -- 172.21.15.3:0/2384274307 learned_addr learned my addr 172.21.15.3:0/2384274307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d9ccd5700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d98142de0 0x7f0d98140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d97fff700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d98068160 0x7f0d9816d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d977fe700 1 -- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d98068160 msgr2=0x7f0d9816d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d977fe700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d98068160 0x7f0d9816d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d977fe700 1 -- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d98142de0 msgr2=0x7f0d98140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d977fe700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d98142de0 0x7f0d98140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d977fe700 1 -- 172.21.15.3:0/2384274307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d90009ae0 con 0x7f0d980b15c0 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d9ccd5700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d98142de0 0x7f0d98140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d977fe700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d980b15c0 0x7f0d98169880 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800c590 tx=0x7f0d8800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d8ffff700 1 -- 172.21.15.3:0/2384274307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88011590 con 0x7f0d980b15c0 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d8ffff700 1 -- 172.21.15.3:0/2384274307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d880116f0 con 0x7f0d980b15c0 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d980ccf30 con 0x7f0d980b15c0 2023-10-06T04:54:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d8ffff700 1 -- 172.21.15.3:0/2384274307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88011ab0 con 0x7f0d980b15c0 2023-10-06T04:54:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.900+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d980cad60 con 0x7f0d980b15c0 2023-10-06T04:54:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.901+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9804c380 con 0x7f0d980b15c0 2023-10-06T04:54:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.901+0000 7f0d8ffff700 1 -- 172.21.15.3:0/2384274307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d8801b3b0 con 0x7f0d980b15c0 2023-10-06T04:54:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.902+0000 7f0d8ffff700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d78066f30 0x7f0d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.902+0000 7f0d8ffff700 1 -- 172.21.15.3:0/2384274307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d88083c40 con 0x7f0d980b15c0 2023-10-06T04:54:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.902+0000 7f0d97fff700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d78066f30 0x7f0d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.903+0000 7f0d97fff700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d78066f30 0x7f0d780693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006f00 tx=0x7f0d80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:44.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:44.906+0000 7f0d8ffff700 1 -- 172.21.15.3:0/2384274307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d88050d50 con 0x7f0d980b15c0 2023-10-06T04:54:45.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.102+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d9815a080 con 0x7f0d980b15c0 2023-10-06T04:54:45.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.103+0000 7f0d8ffff700 1 -- 172.21.15.3:0/2384274307 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16820 (secure 0 0 0) 0x7f0d88017070 con 0x7f0d980b15c0 2023-10-06T04:54:45.104 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:45.105 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":58,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:54:42.323969+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:54:38.486656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T04:54:39.707870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:54:45.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d78066f30 msgr2=0x7f0d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d78066f30 0x7f0d780693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006f00 tx=0x7f0d80008040 comp rx=0 tx=0).stop 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d980b15c0 msgr2=0x7f0d98169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d980b15c0 0x7f0d98169880 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800c590 tx=0x7f0d8800c950 comp rx=0 tx=0).stop 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 shutdown_connections 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0d78066f30 0x7f0d780693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d98142de0 0x7f0d98140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d98068160 0x7f0d9816d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 --2- 172.21.15.3:0/2384274307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d980b15c0 0x7f0d98169880 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 >> 172.21.15.3:0/2384274307 conn(0x7f0d9806ce40 msgr2=0x7f0d9815e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 shutdown_connections 2023-10-06T04:54:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.105+0000 7f0d9d4d6700 1 -- 172.21.15.3:0/2384274307 wait complete. 2023-10-06T04:54:45.119 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:54:45.119 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:54:45.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.236+0000 7fbb2220c700 1 Processor -- start 2023-10-06T04:54:45.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.236+0000 7fbb2220c700 1 -- start start 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.237+0000 7fbb2220c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbb1c157030 0x7fbb1c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.237+0000 7fbb2220c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.237+0000 7fbb2220c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c15aeb0 con 0x7fbb1c15a220 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.237+0000 7fbb2220c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c15a7a0 con 0x7fbb1c157030 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.237+0000 7fbb2220c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c15aa80 con 0x7fbb1c157400 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbb1c157030 0x7fbb1c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1affd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbb1c15a220 0x7fbb1c0b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:37254/0 (socket says 172.21.15.3:37254) 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1affd700 1 -- 172.21.15.3:0/2192221306 learned_addr learned my addr 172.21.15.3:0/2192221306 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1b7fe700 1 --2- 172.21.15.3:0/2192221306 >> v2:172.21.15.153:3300/0 conn(0x7fbb1c157030 0x7fbb1c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:43130/0 (socket says 172.21.15.3:43130) 2023-10-06T04:54:45.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1b7fe700 1 -- 172.21.15.3:0/2192221306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 msgr2=0x7fbb1c06e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1b7fe700 1 --2- 172.21.15.3:0/2192221306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c06e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1b7fe700 1 -- 172.21.15.3:0/2192221306 >> v1:172.21.15.153:6789/0 conn(0x7fbb1c15a220 legacy=0x7fbb1c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1a7fc700 1 -- 172.21.15.3:0/2192221306 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238749323 0 0) 0x7fbb1c15aeb0 con 0x7fbb1c15a220 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1b7fe700 1 -- 172.21.15.3:0/2192221306 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c06dba0 con 0x7fbb1c157030 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1b7fe700 1 --2- 172.21.15.3:0/2192221306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c157030 0x7fbb1c159ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0098e0 tx=0x7fbb0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fea05437a597f96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1a7fc700 1 -- 172.21.15.3:0/2192221306 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00b640 con 0x7fbb1c157030 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1a7fc700 1 -- 172.21.15.3:0/2192221306 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00b7a0 con 0x7fbb1c157030 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.238+0000 7fbb1a7fc700 1 -- 172.21.15.3:0/2192221306 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00bb40 con 0x7fbb1c157030 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 -- 172.21.15.3:0/2192221306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c157030 msgr2=0x7fbb1c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 --2- 172.21.15.3:0/2192221306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c157030 0x7fbb1c159ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0098e0 tx=0x7fbb0c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 -- 172.21.15.3:0/2192221306 shutdown_connections 2023-10-06T04:54:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 --2- 172.21.15.3:0/2192221306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c06e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 --2- 172.21.15.3:0/2192221306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c157030 0x7fbb1c159ce0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 -- 172.21.15.3:0/2192221306 >> 172.21.15.3:0/2192221306 conn(0x7fbb1c0ce270 msgr2=0x7fbb1c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 -- 172.21.15.3:0/2192221306 shutdown_connections 2023-10-06T04:54:45.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.239+0000 7fbb2220c700 1 -- 172.21.15.3:0/2192221306 wait complete. 2023-10-06T04:54:45.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.240+0000 7fbb2220c700 1 Processor -- start 2023-10-06T04:54:45.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.240+0000 7fbb2220c700 1 -- start start 2023-10-06T04:54:45.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb2220c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb1c157030 0x7fbb1c14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb2220c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb2220c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c15a220 0x7fbb1c15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb2220c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1510e0 con 0x7fbb1c157030 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb2220c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c151be0 con 0x7fbb1c15a220 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb2220c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c15ef80 con 0x7fbb1c157400 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb1c157030 0x7fbb1c14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb1c157030 0x7fbb1c14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44958/0 (socket says 172.21.15.3:44958) 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 -- 172.21.15.3:0/1475092144 learned_addr learned my addr 172.21.15.3:0/1475092144 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1bfff700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c15a220 0x7fbb1c15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1affd700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 -- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 msgr2=0x7fbb1c15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 -- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c15a220 msgr2=0x7fbb1c15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c15a220 0x7fbb1c15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.241+0000 7fbb1b7fe700 1 -- 172.21.15.3:0/1475092144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c009420 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.242+0000 7fbb1b7fe700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb1c157030 0x7fbb1c14fe20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009f80 tx=0x7fbb0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.242+0000 7fbb037fe700 1 -- 172.21.15.3:0/1475092144 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00bb20 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.242+0000 7fbb037fe700 1 -- 172.21.15.3:0/1475092144 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c000c90 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.242+0000 7fbb037fe700 1 -- 172.21.15.3:0/1475092144 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c011650 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.242+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1504b0 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.242+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c151590 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.243+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c0644c0 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.243+0000 7fbb037fe700 1 -- 172.21.15.3:0/1475092144 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb0c011a40 con 0x7fbb1c157030 2023-10-06T04:54:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.244+0000 7fbb037fe700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb08066fb0 0x7fbb08069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.244+0000 7fbb037fe700 1 -- 172.21.15.3:0/1475092144 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbb0c0860c0 con 0x7fbb1c157030 2023-10-06T04:54:45.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.244+0000 7fbb1affd700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb08066fb0 0x7fbb08069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.244+0000 7fbb1affd700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb08066fb0 0x7fbb08069460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0cef80 tx=0x7fbb10007f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:45.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.248+0000 7fbb037fe700 1 -- 172.21.15.3:0/1475092144 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb0c053130 con 0x7fbb1c157030 2023-10-06T04:54:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.438+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb1c071680 con 0x7fbb08066fb0 2023-10-06T04:54:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.448+0000 7fbb037fe700 1 -- 172.21.15.3:0/1475092144 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308672 (secure 0 0 0) 0x7fbb1c071680 con 0x7fbb08066fb0 2023-10-06T04:54:45.453 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb08066fb0 msgr2=0x7fbb08069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb08066fb0 0x7fbb08069460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0cef80 tx=0x7fbb10007f50 comp rx=0 tx=0).stop 2023-10-06T04:54:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb1c157030 msgr2=0x7fbb1c14fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb1c157030 0x7fbb1c14fe20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009f80 tx=0x7fbb0c017000 comp rx=0 tx=0).stop 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 shutdown_connections 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb08066fb0 0x7fbb08069460 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb1c15a220 0x7fbb1c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb1c157030 0x7fbb1c14fe20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 --2- 172.21.15.3:0/1475092144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb1c157400 0x7fbb1c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 >> 172.21.15.3:0/1475092144 conn(0x7fbb1c0ce270 msgr2=0x7fbb1c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 shutdown_connections 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.451+0000 7fbb2220c700 1 -- 172.21.15.3:0/1475092144 wait complete. 2023-10-06T04:54:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:54:45.464 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2023-10-06T04:54:43.457961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425588,"kb_used_data":6324,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752549132,"statfs":{"total":773094113280,"available":770610311168,"internally_reserved":0,"allocated":6475776,"data_stored":3465669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.139757"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333786+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331685+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.335105+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329702+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:55.591082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330610+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327617+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:01.768927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330876+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329686+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328812+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:33.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332622+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:54.642506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331759+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333352+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328323+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328695+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:29.292684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327879+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:14.188545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331666+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328892+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330778+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:47.801935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325842+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332133+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326112+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326304+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328233+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331026+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331474+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329585+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332529+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331609+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332943+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:05.163312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331203+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.065580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332535+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:26.889286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339794+0000","last_change":"2023-10-06T04:54:40.339794+0000","last_active":"2023-10-06T04:54:40.339794+0000","last_peered":"2023-10-06T04:54:40.339794+0000","last_clean":"2023-10-06T04:54:40.339794+0000","last_became_active":"2023-10-06T04:54:40.339482+0000","last_became_peered":"2023-10-06T04:54:40.339482+0000","last_unstale":"2023-10-06T04:54:40.339794+0000","last_undegraded":"2023-10-06T04:54:40.339794+0000","last_fullsized":"2023-10-06T04:54:40.339794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328444+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340091+0000","last_change":"2023-10-06T04:54:40.340091+0000","last_active":"2023-10-06T04:54:40.340091+0000","last_peered":"2023-10-06T04:54:40.340091+0000","last_clean":"2023-10-06T04:54:40.340091+0000","last_became_active":"2023-10-06T04:54:40.339729+0000","last_became_peered":"2023-10-06T04:54:40.339729+0000","last_unstale":"2023-10-06T04:54:40.340091+0000","last_undegraded":"2023-10-06T04:54:40.340091+0000","last_fullsized":"2023-10-06T04:54:40.340091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330525+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:05.603150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344126+0000","last_change":"2023-10-06T04:54:40.344126+0000","last_active":"2023-10-06T04:54:40.344126+0000","last_peered":"2023-10-06T04:54:40.344126+0000","last_clean":"2023-10-06T04:54:40.344126+0000","last_became_active":"2023-10-06T04:54:40.343824+0000","last_became_peered":"2023-10-06T04:54:40.343824+0000","last_unstale":"2023-10-06T04:54:40.344126+0000","last_undegraded":"2023-10-06T04:54:40.344126+0000","last_fullsized":"2023-10-06T04:54:40.344126+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:32.239178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329189+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:33.176469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338831+0000","last_change":"2023-10-06T04:54:40.338831+0000","last_active":"2023-10-06T04:54:40.338831+0000","last_peered":"2023-10-06T04:54:40.338831+0000","last_clean":"2023-10-06T04:54:40.338831+0000","last_became_active":"2023-10-06T04:54:40.338570+0000","last_became_peered":"2023-10-06T04:54:40.338570+0000","last_unstale":"2023-10-06T04:54:40.338831+0000","last_undegraded":"2023-10-06T04:54:40.338831+0000","last_fullsized":"2023-10-06T04:54:40.338831+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:32.170845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327374+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:43.786031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.332965+0000","last_change":"2023-10-06T04:54:40.332965+0000","last_active":"2023-10-06T04:54:40.332965+0000","last_peered":"2023-10-06T04:54:40.332965+0000","last_clean":"2023-10-06T04:54:40.332965+0000","last_became_active":"2023-10-06T04:54:40.332661+0000","last_became_peered":"2023-10-06T04:54:40.332661+0000","last_unstale":"2023-10-06T04:54:40.332965+0000","last_undegraded":"2023-10-06T04:54:40.332965+0000","last_fullsized":"2023-10-06T04:54:40.332965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.328723+0000","last_change":"2023-10-06T04:54:40.328723+0000","last_active":"2023-10-06T04:54:40.328723+0000","last_peered":"2023-10-06T04:54:40.328723+0000","last_clean":"2023-10-06T04:54:40.328723+0000","last_became_active":"2023-10-06T04:54:40.328424+0000","last_became_peered":"2023-10-06T04:54:40.328424+0000","last_unstale":"2023-10-06T04:54:40.328723+0000","last_undegraded":"2023-10-06T04:54:40.328723+0000","last_fullsized":"2023-10-06T04:54:40.328723+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334363+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:37.872096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344255+0000","last_change":"2023-10-06T04:54:40.344255+0000","last_active":"2023-10-06T04:54:40.344255+0000","last_peered":"2023-10-06T04:54:40.344255+0000","last_clean":"2023-10-06T04:54:40.344255+0000","last_became_active":"2023-10-06T04:54:40.343918+0000","last_became_peered":"2023-10-06T04:54:40.343918+0000","last_unstale":"2023-10-06T04:54:40.344255+0000","last_undegraded":"2023-10-06T04:54:40.344255+0000","last_fullsized":"2023-10-06T04:54:40.344255+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326195+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338640+0000","last_change":"2023-10-06T04:54:40.338640+0000","last_active":"2023-10-06T04:54:40.338640+0000","last_peered":"2023-10-06T04:54:40.338640+0000","last_clean":"2023-10-06T04:54:40.338640+0000","last_became_active":"2023-10-06T04:54:40.338385+0000","last_became_peered":"2023-10-06T04:54:40.338385+0000","last_unstale":"2023-10-06T04:54:40.338640+0000","last_undegraded":"2023-10-06T04:54:40.338640+0000","last_fullsized":"2023-10-06T04:54:40.338640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:40.835360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332028+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340720+0000","last_change":"2023-10-06T04:54:40.340720+0000","last_active":"2023-10-06T04:54:40.340720+0000","last_peered":"2023-10-06T04:54:40.340720+0000","last_clean":"2023-10-06T04:54:40.340720+0000","last_became_active":"2023-10-06T04:54:40.340447+0000","last_became_peered":"2023-10-06T04:54:40.340447+0000","last_unstale":"2023-10-06T04:54:40.340720+0000","last_undegraded":"2023-10-06T04:54:40.340720+0000","last_fullsized":"2023-10-06T04:54:40.340720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:38.363715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325883+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339697+0000","last_change":"2023-10-06T04:54:40.339697+0000","last_active":"2023-10-06T04:54:40.339697+0000","last_peered":"2023-10-06T04:54:40.339697+0000","last_clean":"2023-10-06T04:54:40.339697+0000","last_became_active":"2023-10-06T04:54:40.339299+0000","last_became_peered":"2023-10-06T04:54:40.339299+0000","last_unstale":"2023-10-06T04:54:40.339697+0000","last_undegraded":"2023-10-06T04:54:40.339697+0000","last_fullsized":"2023-10-06T04:54:40.339697+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:12.235281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334708+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:36.233559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339769+0000","last_change":"2023-10-06T04:54:40.339769+0000","last_active":"2023-10-06T04:54:40.339769+0000","last_peered":"2023-10-06T04:54:40.339769+0000","last_clean":"2023-10-06T04:54:40.339769+0000","last_became_active":"2023-10-06T04:54:40.339511+0000","last_became_peered":"2023-10-06T04:54:40.339511+0000","last_unstale":"2023-10-06T04:54:40.339769+0000","last_undegraded":"2023-10-06T04:54:40.339769+0000","last_fullsized":"2023-10-06T04:54:40.339769+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:25.589985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328269+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341385+0000","last_change":"2023-10-06T04:54:40.341385+0000","last_active":"2023-10-06T04:54:40.341385+0000","last_peered":"2023-10-06T04:54:40.341385+0000","last_clean":"2023-10-06T04:54:40.341385+0000","last_became_active":"2023-10-06T04:54:40.341078+0000","last_became_peered":"2023-10-06T04:54:40.341078+0000","last_unstale":"2023-10-06T04:54:40.341385+0000","last_undegraded":"2023-10-06T04:54:40.341385+0000","last_fullsized":"2023-10-06T04:54:40.341385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328102+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344621+0000","last_change":"2023-10-06T04:54:40.344621+0000","last_active":"2023-10-06T04:54:40.344621+0000","last_peered":"2023-10-06T04:54:40.344621+0000","last_clean":"2023-10-06T04:54:40.344621+0000","last_became_active":"2023-10-06T04:54:40.344347+0000","last_became_peered":"2023-10-06T04:54:40.344347+0000","last_unstale":"2023-10-06T04:54:40.344621+0000","last_undegraded":"2023-10-06T04:54:40.344621+0000","last_fullsized":"2023-10-06T04:54:40.344621+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:41.740059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330179+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342909+0000","last_change":"2023-10-06T04:54:40.342909+0000","last_active":"2023-10-06T04:54:40.342909+0000","last_peered":"2023-10-06T04:54:40.342909+0000","last_clean":"2023-10-06T04:54:40.342909+0000","last_became_active":"2023-10-06T04:54:40.342316+0000","last_became_peered":"2023-10-06T04:54:40.342316+0000","last_unstale":"2023-10-06T04:54:40.342909+0000","last_undegraded":"2023-10-06T04:54:40.342909+0000","last_fullsized":"2023-10-06T04:54:40.342909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333878+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:04.598076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341225+0000","last_change":"2023-10-06T04:54:40.341225+0000","last_active":"2023-10-06T04:54:40.341225+0000","last_peered":"2023-10-06T04:54:40.341225+0000","last_clean":"2023-10-06T04:54:40.341225+0000","last_became_active":"2023-10-06T04:54:40.340018+0000","last_became_peered":"2023-10-06T04:54:40.340018+0000","last_unstale":"2023-10-06T04:54:40.341225+0000","last_undegraded":"2023-10-06T04:54:40.341225+0000","last_fullsized":"2023-10-06T04:54:40.341225+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331646+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.345606+0000","last_change":"2023-10-06T04:54:40.345606+0000","last_active":"2023-10-06T04:54:40.345606+0000","last_peered":"2023-10-06T04:54:40.345606+0000","last_clean":"2023-10-06T04:54:40.345606+0000","last_became_active":"2023-10-06T04:54:40.345364+0000","last_became_peered":"2023-10-06T04:54:40.345364+0000","last_unstale":"2023-10-06T04:54:40.345606+0000","last_undegraded":"2023-10-06T04:54:40.345606+0000","last_fullsized":"2023-10-06T04:54:40.345606+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:32.478317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330958+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:35.749980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340969+0000","last_change":"2023-10-06T04:54:40.340969+0000","last_active":"2023-10-06T04:54:40.340969+0000","last_peered":"2023-10-06T04:54:40.340969+0000","last_clean":"2023-10-06T04:54:40.340969+0000","last_became_active":"2023-10-06T04:54:40.340716+0000","last_became_peered":"2023-10-06T04:54:40.340716+0000","last_unstale":"2023-10-06T04:54:40.340969+0000","last_undegraded":"2023-10-06T04:54:40.340969+0000","last_fullsized":"2023-10-06T04:54:40.340969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327245+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.375936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342698+0000","last_change":"2023-10-06T04:54:40.342698+0000","last_active":"2023-10-06T04:54:40.342698+0000","last_peered":"2023-10-06T04:54:40.342698+0000","last_clean":"2023-10-06T04:54:40.342698+0000","last_became_active":"2023-10-06T04:54:40.342336+0000","last_became_peered":"2023-10-06T04:54:40.342336+0000","last_unstale":"2023-10-06T04:54:40.342698+0000","last_undegraded":"2023-10-06T04:54:40.342698+0000","last_fullsized":"2023-10-06T04:54:40.342698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:35.638003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330725+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343535+0000","last_change":"2023-10-06T04:54:40.343535+0000","last_active":"2023-10-06T04:54:40.343535+0000","last_peered":"2023-10-06T04:54:40.343535+0000","last_clean":"2023-10-06T04:54:40.343535+0000","last_became_active":"2023-10-06T04:54:40.343213+0000","last_became_peered":"2023-10-06T04:54:40.343213+0000","last_unstale":"2023-10-06T04:54:40.343535+0000","last_undegraded":"2023-10-06T04:54:40.343535+0000","last_fullsized":"2023-10-06T04:54:40.343535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:24.202745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333947+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:28.396681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344767+0000","last_change":"2023-10-06T04:54:40.344767+0000","last_active":"2023-10-06T04:54:40.344767+0000","last_peered":"2023-10-06T04:54:40.344767+0000","last_clean":"2023-10-06T04:54:40.344767+0000","last_became_active":"2023-10-06T04:54:40.344507+0000","last_became_peered":"2023-10-06T04:54:40.344507+0000","last_unstale":"2023-10-06T04:54:40.344767+0000","last_undegraded":"2023-10-06T04:54:40.344767+0000","last_fullsized":"2023-10-06T04:54:40.344767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329365+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:18.138617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341127+0000","last_change":"2023-10-06T04:54:40.341127+0000","last_active":"2023-10-06T04:54:40.341127+0000","last_peered":"2023-10-06T04:54:40.341127+0000","last_clean":"2023-10-06T04:54:40.341127+0000","last_became_active":"2023-10-06T04:54:40.339641+0000","last_became_peered":"2023-10-06T04:54:40.339641+0000","last_unstale":"2023-10-06T04:54:40.341127+0000","last_undegraded":"2023-10-06T04:54:40.341127+0000","last_fullsized":"2023-10-06T04:54:40.341127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.636882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.333730+0000","last_change":"2023-10-06T04:54:40.333730+0000","last_active":"2023-10-06T04:54:40.333730+0000","last_peered":"2023-10-06T04:54:40.333730+0000","last_clean":"2023-10-06T04:54:40.333730+0000","last_became_active":"2023-10-06T04:54:40.333468+0000","last_became_peered":"2023-10-06T04:54:40.333468+0000","last_unstale":"2023-10-06T04:54:40.333730+0000","last_undegraded":"2023-10-06T04:54:40.333730+0000","last_fullsized":"2023-10-06T04:54:40.333730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:34.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332948+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344570+0000","last_change":"2023-10-06T04:54:40.344570+0000","last_active":"2023-10-06T04:54:40.344570+0000","last_peered":"2023-10-06T04:54:40.344570+0000","last_clean":"2023-10-06T04:54:40.344570+0000","last_became_active":"2023-10-06T04:54:40.344295+0000","last_became_peered":"2023-10-06T04:54:40.344295+0000","last_unstale":"2023-10-06T04:54:40.344570+0000","last_undegraded":"2023-10-06T04:54:40.344570+0000","last_fullsized":"2023-10-06T04:54:40.344570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:40.208274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327281+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:47.664286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344877+0000","last_change":"2023-10-06T04:54:40.344877+0000","last_active":"2023-10-06T04:54:40.344877+0000","last_peered":"2023-10-06T04:54:40.344877+0000","last_clean":"2023-10-06T04:54:40.344877+0000","last_became_active":"2023-10-06T04:54:40.344580+0000","last_became_peered":"2023-10-06T04:54:40.344580+0000","last_unstale":"2023-10-06T04:54:40.344877+0000","last_undegraded":"2023-10-06T04:54:40.344877+0000","last_fullsized":"2023-10-06T04:54:40.344877+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328360+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340640+0000","last_change":"2023-10-06T04:54:40.340640+0000","last_active":"2023-10-06T04:54:40.340640+0000","last_peered":"2023-10-06T04:54:40.340640+0000","last_clean":"2023-10-06T04:54:40.340640+0000","last_became_active":"2023-10-06T04:54:40.340384+0000","last_became_peered":"2023-10-06T04:54:40.340384+0000","last_unstale":"2023-10-06T04:54:40.340640+0000","last_undegraded":"2023-10-06T04:54:40.340640+0000","last_fullsized":"2023-10-06T04:54:40.340640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.769263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323560+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:54:39.323560+0000","last_peered":"2023-10-06T04:54:39.323560+0000","last_clean":"2023-10-06T04:54:39.323560+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:54:39.323560+0000","last_undegraded":"2023-10-06T04:54:39.323560+0000","last_fullsized":"2023-10-06T04:54:39.323560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334410+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:15.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343623+0000","last_change":"2023-10-06T04:54:40.343623+0000","last_active":"2023-10-06T04:54:40.343623+0000","last_peered":"2023-10-06T04:54:40.343623+0000","last_clean":"2023-10-06T04:54:40.343623+0000","last_became_active":"2023-10-06T04:54:40.343366+0000","last_became_peered":"2023-10-06T04:54:40.343366+0000","last_unstale":"2023-10-06T04:54:40.343623+0000","last_undegraded":"2023-10-06T04:54:40.343623+0000","last_fullsized":"2023-10-06T04:54:40.343623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:33.074026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.325164+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:54:39.325164+0000","last_peered":"2023-10-06T04:54:39.325164+0000","last_clean":"2023-10-06T04:54:39.325164+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:54:39.325164+0000","last_undegraded":"2023-10-06T04:54:39.325164+0000","last_fullsized":"2023-10-06T04:54:39.325164+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330526+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:04.376901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344112+0000","last_change":"2023-10-06T04:54:40.344112+0000","last_active":"2023-10-06T04:54:40.344112+0000","last_peered":"2023-10-06T04:54:40.344112+0000","last_clean":"2023-10-06T04:54:40.344112+0000","last_became_active":"2023-10-06T04:54:40.343905+0000","last_became_peered":"2023-10-06T04:54:40.343905+0000","last_unstale":"2023-10-06T04:54:40.344112+0000","last_undegraded":"2023-10-06T04:54:40.344112+0000","last_fullsized":"2023-10-06T04:54:40.344112+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.324421+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:54:39.324421+0000","last_peered":"2023-10-06T04:54:39.324421+0000","last_clean":"2023-10-06T04:54:39.324421+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:54:39.324421+0000","last_undegraded":"2023-10-06T04:54:39.324421+0000","last_fullsized":"2023-10-06T04:54:39.324421+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":190,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321139+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:54:39.321139+0000","last_peered":"2023-10-06T04:54:39.321139+0000","last_clean":"2023-10-06T04:54:39.321139+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:54:39.321139+0000","last_undegraded":"2023-10-06T04:54:39.321139+0000","last_fullsized":"2023-10-06T04:54:39.321139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329601+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343032+0000","last_change":"2023-10-06T04:54:40.343032+0000","last_active":"2023-10-06T04:54:40.343032+0000","last_peered":"2023-10-06T04:54:40.343032+0000","last_clean":"2023-10-06T04:54:40.343032+0000","last_became_active":"2023-10-06T04:54:40.342591+0000","last_became_peered":"2023-10-06T04:54:40.342591+0000","last_unstale":"2023-10-06T04:54:40.343032+0000","last_undegraded":"2023-10-06T04:54:40.343032+0000","last_fullsized":"2023-10-06T04:54:40.343032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321070+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:54:39.321070+0000","last_peered":"2023-10-06T04:54:39.321070+0000","last_clean":"2023-10-06T04:54:39.321070+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:54:39.321070+0000","last_undegraded":"2023-10-06T04:54:39.321070+0000","last_fullsized":"2023-10-06T04:54:39.321070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329145+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:46.287413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.334344+0000","last_change":"2023-10-06T04:54:40.334344+0000","last_active":"2023-10-06T04:54:40.334344+0000","last_peered":"2023-10-06T04:54:40.334344+0000","last_clean":"2023-10-06T04:54:40.334344+0000","last_became_active":"2023-10-06T04:54:40.334079+0000","last_became_peered":"2023-10-06T04:54:40.334079+0000","last_unstale":"2023-10-06T04:54:40.334344+0000","last_undegraded":"2023-10-06T04:54:40.334344+0000","last_fullsized":"2023-10-06T04:54:40.334344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:28.287060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321036+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:54:39.321036+0000","last_peered":"2023-10-06T04:54:39.321036+0000","last_clean":"2023-10-06T04:54:39.321036+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:54:39.321036+0000","last_undegraded":"2023-10-06T04:54:39.321036+0000","last_fullsized":"2023-10-06T04:54:39.321036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333407+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:10.632704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340001+0000","last_change":"2023-10-06T04:54:40.340001+0000","last_active":"2023-10-06T04:54:40.340001+0000","last_peered":"2023-10-06T04:54:40.340001+0000","last_clean":"2023-10-06T04:54:40.340001+0000","last_became_active":"2023-10-06T04:54:40.339665+0000","last_became_peered":"2023-10-06T04:54:40.339665+0000","last_unstale":"2023-10-06T04:54:40.340001+0000","last_undegraded":"2023-10-06T04:54:40.340001+0000","last_fullsized":"2023-10-06T04:54:40.340001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:07.045545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.328478+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:54:39.328478+0000","last_peered":"2023-10-06T04:54:39.328478+0000","last_clean":"2023-10-06T04:54:39.328478+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:54:39.328478+0000","last_undegraded":"2023-10-06T04:54:39.328478+0000","last_fullsized":"2023-10-06T04:54:39.328478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330328+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:56.766026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343461+0000","last_change":"2023-10-06T04:54:40.343461+0000","last_active":"2023-10-06T04:54:40.343461+0000","last_peered":"2023-10-06T04:54:40.343461+0000","last_clean":"2023-10-06T04:54:40.343461+0000","last_became_active":"2023-10-06T04:54:40.343184+0000","last_became_peered":"2023-10-06T04:54:40.343184+0000","last_unstale":"2023-10-06T04:54:40.343461+0000","last_undegraded":"2023-10-06T04:54:40.343461+0000","last_fullsized":"2023-10-06T04:54:40.343461+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:47.709557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323296+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:54:39.323296+0000","last_peered":"2023-10-06T04:54:39.323296+0000","last_clean":"2023-10-06T04:54:39.323296+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:54:39.323296+0000","last_undegraded":"2023-10-06T04:54:39.323296+0000","last_fullsized":"2023-10-06T04:54:39.323296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331007+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342683+0000","last_change":"2023-10-06T04:54:40.342683+0000","last_active":"2023-10-06T04:54:40.342683+0000","last_peered":"2023-10-06T04:54:40.342683+0000","last_clean":"2023-10-06T04:54:40.342683+0000","last_became_active":"2023-10-06T04:54:40.342261+0000","last_became_peered":"2023-10-06T04:54:40.342261+0000","last_unstale":"2023-10-06T04:54:40.342683+0000","last_undegraded":"2023-10-06T04:54:40.342683+0000","last_fullsized":"2023-10-06T04:54:40.342683+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323303+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:54:39.323303+0000","last_peered":"2023-10-06T04:54:39.323303+0000","last_clean":"2023-10-06T04:54:39.323303+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:54:39.323303+0000","last_undegraded":"2023-10-06T04:54:39.323303+0000","last_fullsized":"2023-10-06T04:54:39.323303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306700,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065140,"statfs":{"total":96636764160,"available":96322703360,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.751,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.66400000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.80200000000000005,"15min":0.84499999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70099999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":1.111,"15min":1.111},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.874,"15min":0.874},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77500000000000002,"5min":1.214,"15min":1.214},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72599999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.751,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63300000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302604,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069236,"statfs":{"total":96636764160,"available":96326897664,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.48399999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51000000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.379,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.42399999999999999,"15min":0.42899999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54100000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.376,"5min":0.42799999999999999,"15min":0.437},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57499999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.63400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73699999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":1.026,"15min":1.026},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.46200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.38},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36499999999999999,"15min":0.36599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64400000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.747,"5min":1.048,"15min":1.048},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.22700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.57999999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.24399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68700000000000006,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85399999999999998,"5min":1.246,"15min":1.246},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.40699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.877,"15min":0.877},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306136,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065704,"statfs":{"total":96636764160,"available":96323280896,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44500000000000001,"5min":1.415,"15min":1.415},"last":0.25},{"interface":"front","average":{"1min":0.37,"5min":0.58599999999999997,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51400000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.44600000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.45700000000000002,"15min":0.47099999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53600000000000003,"5min":1.482,"15min":1.482},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":5.069,"15min":5.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57799999999999996,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.71299999999999997,"15min":0.73799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94799999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80200000000000005,"15min":0.83999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.504}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.77300000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":1.452,"15min":1.452},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.81399999999999995,"15min":0.85399999999999998},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":6.367,"15min":6.367},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":5,"15min":5},"last":0.84199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.81999999999999995,"15min":0.876},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63200000000000001,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.85299999999999998,"15min":0.90300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67900000000000005,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.503},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.85799999999999998,"15min":0.91400000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.68400000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64300000000000002,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.75},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.70299999999999996,"15min":0.73899999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61599999999999999,"5min":5.415,"15min":5.415},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67200000000000004,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66200000000000003,"5min":4.202,"15min":4.202},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74099999999999999,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":0.872},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":655360,"data_stored":283119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70699999999999996,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63700000000000001,"15min":0.65500000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72399999999999998,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.69299999999999995,"15min":0.71599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.746,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.44}]},{"osd":3,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.0049999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.499},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.008,"15min":1.0760000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.751},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76500000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:54:45.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:54:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.624+0000 7f9807729700 1 Processor -- start 2023-10-06T04:54:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.624+0000 7f9807729700 1 -- start start 2023-10-06T04:54:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9807729700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9800153bd0 0x7f980015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9807729700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9800153fa0 0x7f9800159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9807729700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980015d9f0 con 0x7f980015a2e0 2023-10-06T04:54:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9807729700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980015a710 con 0x7f9800153bd0 2023-10-06T04:54:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9807729700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98000b0aa0 con 0x7f9800153fa0 2023-10-06T04:54:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9805cc6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f980015a2e0 0x7f980006b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48216/0 (socket says 172.21.15.3:48216) 2023-10-06T04:54:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9805cc6700 1 -- 172.21.15.3:0/3260895561 learned_addr learned my addr 172.21.15.3:0/3260895561 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9804cc4700 1 --2- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9800153fa0 0x7f9800159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f98054c5700 1 --2- 172.21.15.3:0/3260895561 >> v2:172.21.15.153:3300/0 conn(0x7f9800153bd0 0x7f980015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f97f7fff700 1 -- 172.21.15.3:0/3260895561 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492547119 0 0) 0x7f980015d9f0 con 0x7f980015a2e0 2023-10-06T04:54:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f97f7fff700 1 -- 172.21.15.3:0/3260895561 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97ec003570 con 0x7f980015a2e0 2023-10-06T04:54:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f98054c5700 1 -- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9800153fa0 msgr2=0x7f9800159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f98054c5700 1 --2- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9800153fa0 0x7f9800159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f98054c5700 1 -- 172.21.15.3:0/3260895561 >> v1:172.21.15.3:6789/0 conn(0x7f980015a2e0 legacy=0x7f980006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f98054c5700 1 -- 172.21.15.3:0/3260895561 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980015a980 con 0x7f9800153bd0 2023-10-06T04:54:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.625+0000 7f9804cc4700 1 --2- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9800153fa0 0x7f9800159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:54:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f98054c5700 1 --2- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153bd0 0x7f980015de80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0098e0 tx=0x7f97fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db9f9aefa86f0b25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f97f7fff700 1 -- 172.21.15.3:0/3260895561 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc011630 con 0x7f9800153bd0 2023-10-06T04:54:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f97f7fff700 1 -- 172.21.15.3:0/3260895561 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc011790 con 0x7f9800153bd0 2023-10-06T04:54:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f97f7fff700 1 -- 172.21.15.3:0/3260895561 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc011b10 con 0x7f9800153bd0 2023-10-06T04:54:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 -- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153bd0 msgr2=0x7f980015de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 --2- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153bd0 0x7f980015de80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0098e0 tx=0x7f97fc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 -- 172.21.15.3:0/3260895561 shutdown_connections 2023-10-06T04:54:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 --2- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153bd0 0x7f980015de80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 --2- 172.21.15.3:0/3260895561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9800153fa0 0x7f9800159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 -- 172.21.15.3:0/3260895561 >> 172.21.15.3:0/3260895561 conn(0x7f98000cdc30 msgr2=0x7f980006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 -- 172.21.15.3:0/3260895561 shutdown_connections 2023-10-06T04:54:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.626+0000 7f9807729700 1 -- 172.21.15.3:0/3260895561 wait complete. 2023-10-06T04:54:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.627+0000 7f9807729700 1 Processor -- start 2023-10-06T04:54:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f9807729700 1 -- start start 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f9807729700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9800153bd0 0x7f9800151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f9807729700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153fa0 0x7f9800152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f9807729700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f980015a2e0 0x7f9800152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f9807729700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800169660 con 0x7f9800153bd0 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f9807729700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800169940 con 0x7f9800153fa0 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f9807729700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9800169c20 con 0x7f980015a2e0 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f98054c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9800153bd0 0x7f9800151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f98054c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9800153bd0 0x7f9800151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44968/0 (socket says 172.21.15.3:44968) 2023-10-06T04:54:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.628+0000 7f98054c5700 1 -- 172.21.15.3:0/1555897812 learned_addr learned my addr 172.21.15.3:0/1555897812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f9804cc4700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153fa0 0x7f9800152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f9805cc6700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f980015a2e0 0x7f9800152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f98054c5700 1 -- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f980015a2e0 msgr2=0x7f9800152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f98054c5700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f980015a2e0 0x7f9800152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f98054c5700 1 -- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153fa0 msgr2=0x7f9800152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f98054c5700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153fa0 0x7f9800152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f98054c5700 1 -- 172.21.15.3:0/1555897812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0009420 con 0x7f9800153bd0 2023-10-06T04:54:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f98054c5700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9800153bd0 0x7f9800151760 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9800070d80 tx=0x7f97fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f97f57fa700 1 -- 172.21.15.3:0/1555897812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc011900 con 0x7f9800153bd0 2023-10-06T04:54:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f97f57fa700 1 -- 172.21.15.3:0/1555897812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc011a60 con 0x7f9800153bd0 2023-10-06T04:54:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f97f57fa700 1 -- 172.21.15.3:0/1555897812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc01a680 con 0x7f9800153bd0 2023-10-06T04:54:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc009420 con 0x7f9800153bd0 2023-10-06T04:54:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.629+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980016a1e0 con 0x7f9800153bd0 2023-10-06T04:54:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.630+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9800071680 con 0x7f9800153bd0 2023-10-06T04:54:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.631+0000 7f97f57fa700 1 -- 172.21.15.3:0/1555897812 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97fc01aa70 con 0x7f9800153bd0 2023-10-06T04:54:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.631+0000 7f97f57fa700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.631+0000 7f97f57fa700 1 -- 172.21.15.3:0/1555897812 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f97fc0860b0 con 0x7f9800153bd0 2023-10-06T04:54:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.631+0000 7f9804cc4700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.632+0000 7f9804cc4700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f97e4066f30 0x7f97e40693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f98000b0210 tx=0x7f97f00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.635+0000 7f97f57fa700 1 -- 172.21.15.3:0/1555897812 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97fc0531c0 con 0x7f9800153bd0 2023-10-06T04:54:45.643 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-10-06T04:54:45.642+0000 7f18c1764700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:54:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.824+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98000692d0 con 0x7f97e4066f30 2023-10-06T04:54:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.834+0000 7f97f57fa700 1 -- 172.21.15.3:0/1555897812 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308672 (secure 0 0 0) 0x7f98000692d0 con 0x7f97e4066f30 2023-10-06T04:54:45.839 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:45.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f97e4066f30 msgr2=0x7f97e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f97e4066f30 0x7f97e40693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f98000b0210 tx=0x7f97f00091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9800153bd0 msgr2=0x7f9800151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9800153bd0 0x7f9800151760 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9800070d80 tx=0x7f97fc017000 comp rx=0 tx=0).stop 2023-10-06T04:54:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 shutdown_connections 2023-10-06T04:54:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f980015a2e0 0x7f9800152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9800153bd0 0x7f9800151760 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 --2- 172.21.15.3:0/1555897812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800153fa0 0x7f9800152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 >> 172.21.15.3:0/1555897812 conn(0x7f98000cdc30 msgr2=0x7f98001520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 shutdown_connections 2023-10-06T04:54:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:45.837+0000 7f9807729700 1 -- 172.21.15.3:0/1555897812 wait complete. 2023-10-06T04:54:45.843 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:54:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-10-06T04:54:45.458470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425588,"kb_used_data":6324,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752549132,"statfs":{"total":773094113280,"available":770610311168,"internally_reserved":0,"allocated":6475776,"data_stored":3465669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001559"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333786+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331685+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.335105+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329702+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:55.591082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330610+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327617+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:01.768927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330876+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329686+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328812+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:33.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332622+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:54.642506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331759+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333352+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328323+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328695+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:29.292684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327879+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:14.188545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331666+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328892+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330778+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:47.801935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325842+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332133+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326112+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326304+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328233+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331026+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331474+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329585+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332529+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331609+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332943+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:05.163312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331203+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.065580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332535+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:26.889286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339794+0000","last_change":"2023-10-06T04:54:40.339794+0000","last_active":"2023-10-06T04:54:40.339794+0000","last_peered":"2023-10-06T04:54:40.339794+0000","last_clean":"2023-10-06T04:54:40.339794+0000","last_became_active":"2023-10-06T04:54:40.339482+0000","last_became_peered":"2023-10-06T04:54:40.339482+0000","last_unstale":"2023-10-06T04:54:40.339794+0000","last_undegraded":"2023-10-06T04:54:40.339794+0000","last_fullsized":"2023-10-06T04:54:40.339794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328444+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340091+0000","last_change":"2023-10-06T04:54:40.340091+0000","last_active":"2023-10-06T04:54:40.340091+0000","last_peered":"2023-10-06T04:54:40.340091+0000","last_clean":"2023-10-06T04:54:40.340091+0000","last_became_active":"2023-10-06T04:54:40.339729+0000","last_became_peered":"2023-10-06T04:54:40.339729+0000","last_unstale":"2023-10-06T04:54:40.340091+0000","last_undegraded":"2023-10-06T04:54:40.340091+0000","last_fullsized":"2023-10-06T04:54:40.340091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330525+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:05.603150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344126+0000","last_change":"2023-10-06T04:54:40.344126+0000","last_active":"2023-10-06T04:54:40.344126+0000","last_peered":"2023-10-06T04:54:40.344126+0000","last_clean":"2023-10-06T04:54:40.344126+0000","last_became_active":"2023-10-06T04:54:40.343824+0000","last_became_peered":"2023-10-06T04:54:40.343824+0000","last_unstale":"2023-10-06T04:54:40.344126+0000","last_undegraded":"2023-10-06T04:54:40.344126+0000","last_fullsized":"2023-10-06T04:54:40.344126+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:32.239178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329189+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:33.176469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338831+0000","last_change":"2023-10-06T04:54:40.338831+0000","last_active":"2023-10-06T04:54:40.338831+0000","last_peered":"2023-10-06T04:54:40.338831+0000","last_clean":"2023-10-06T04:54:40.338831+0000","last_became_active":"2023-10-06T04:54:40.338570+0000","last_became_peered":"2023-10-06T04:54:40.338570+0000","last_unstale":"2023-10-06T04:54:40.338831+0000","last_undegraded":"2023-10-06T04:54:40.338831+0000","last_fullsized":"2023-10-06T04:54:40.338831+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:32.170845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327374+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:43.786031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.332965+0000","last_change":"2023-10-06T04:54:40.332965+0000","last_active":"2023-10-06T04:54:40.332965+0000","last_peered":"2023-10-06T04:54:40.332965+0000","last_clean":"2023-10-06T04:54:40.332965+0000","last_became_active":"2023-10-06T04:54:40.332661+0000","last_became_peered":"2023-10-06T04:54:40.332661+0000","last_unstale":"2023-10-06T04:54:40.332965+0000","last_undegraded":"2023-10-06T04:54:40.332965+0000","last_fullsized":"2023-10-06T04:54:40.332965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.328723+0000","last_change":"2023-10-06T04:54:40.328723+0000","last_active":"2023-10-06T04:54:40.328723+0000","last_peered":"2023-10-06T04:54:40.328723+0000","last_clean":"2023-10-06T04:54:40.328723+0000","last_became_active":"2023-10-06T04:54:40.328424+0000","last_became_peered":"2023-10-06T04:54:40.328424+0000","last_unstale":"2023-10-06T04:54:40.328723+0000","last_undegraded":"2023-10-06T04:54:40.328723+0000","last_fullsized":"2023-10-06T04:54:40.328723+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334363+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:37.872096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344255+0000","last_change":"2023-10-06T04:54:40.344255+0000","last_active":"2023-10-06T04:54:40.344255+0000","last_peered":"2023-10-06T04:54:40.344255+0000","last_clean":"2023-10-06T04:54:40.344255+0000","last_became_active":"2023-10-06T04:54:40.343918+0000","last_became_peered":"2023-10-06T04:54:40.343918+0000","last_unstale":"2023-10-06T04:54:40.344255+0000","last_undegraded":"2023-10-06T04:54:40.344255+0000","last_fullsized":"2023-10-06T04:54:40.344255+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326195+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338640+0000","last_change":"2023-10-06T04:54:40.338640+0000","last_active":"2023-10-06T04:54:40.338640+0000","last_peered":"2023-10-06T04:54:40.338640+0000","last_clean":"2023-10-06T04:54:40.338640+0000","last_became_active":"2023-10-06T04:54:40.338385+0000","last_became_peered":"2023-10-06T04:54:40.338385+0000","last_unstale":"2023-10-06T04:54:40.338640+0000","last_undegraded":"2023-10-06T04:54:40.338640+0000","last_fullsized":"2023-10-06T04:54:40.338640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:40.835360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332028+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340720+0000","last_change":"2023-10-06T04:54:40.340720+0000","last_active":"2023-10-06T04:54:40.340720+0000","last_peered":"2023-10-06T04:54:40.340720+0000","last_clean":"2023-10-06T04:54:40.340720+0000","last_became_active":"2023-10-06T04:54:40.340447+0000","last_became_peered":"2023-10-06T04:54:40.340447+0000","last_unstale":"2023-10-06T04:54:40.340720+0000","last_undegraded":"2023-10-06T04:54:40.340720+0000","last_fullsized":"2023-10-06T04:54:40.340720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:38.363715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325883+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339697+0000","last_change":"2023-10-06T04:54:40.339697+0000","last_active":"2023-10-06T04:54:40.339697+0000","last_peered":"2023-10-06T04:54:40.339697+0000","last_clean":"2023-10-06T04:54:40.339697+0000","last_became_active":"2023-10-06T04:54:40.339299+0000","last_became_peered":"2023-10-06T04:54:40.339299+0000","last_unstale":"2023-10-06T04:54:40.339697+0000","last_undegraded":"2023-10-06T04:54:40.339697+0000","last_fullsized":"2023-10-06T04:54:40.339697+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:12.235281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334708+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:36.233559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339769+0000","last_change":"2023-10-06T04:54:40.339769+0000","last_active":"2023-10-06T04:54:40.339769+0000","last_peered":"2023-10-06T04:54:40.339769+0000","last_clean":"2023-10-06T04:54:40.339769+0000","last_became_active":"2023-10-06T04:54:40.339511+0000","last_became_peered":"2023-10-06T04:54:40.339511+0000","last_unstale":"2023-10-06T04:54:40.339769+0000","last_undegraded":"2023-10-06T04:54:40.339769+0000","last_fullsized":"2023-10-06T04:54:40.339769+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:25.589985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328269+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341385+0000","last_change":"2023-10-06T04:54:40.341385+0000","last_active":"2023-10-06T04:54:40.341385+0000","last_peered":"2023-10-06T04:54:40.341385+0000","last_clean":"2023-10-06T04:54:40.341385+0000","last_became_active":"2023-10-06T04:54:40.341078+0000","last_became_peered":"2023-10-06T04:54:40.341078+0000","last_unstale":"2023-10-06T04:54:40.341385+0000","last_undegraded":"2023-10-06T04:54:40.341385+0000","last_fullsized":"2023-10-06T04:54:40.341385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328102+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344621+0000","last_change":"2023-10-06T04:54:40.344621+0000","last_active":"2023-10-06T04:54:40.344621+0000","last_peered":"2023-10-06T04:54:40.344621+0000","last_clean":"2023-10-06T04:54:40.344621+0000","last_became_active":"2023-10-06T04:54:40.344347+0000","last_became_peered":"2023-10-06T04:54:40.344347+0000","last_unstale":"2023-10-06T04:54:40.344621+0000","last_undegraded":"2023-10-06T04:54:40.344621+0000","last_fullsized":"2023-10-06T04:54:40.344621+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:41.740059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330179+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342909+0000","last_change":"2023-10-06T04:54:40.342909+0000","last_active":"2023-10-06T04:54:40.342909+0000","last_peered":"2023-10-06T04:54:40.342909+0000","last_clean":"2023-10-06T04:54:40.342909+0000","last_became_active":"2023-10-06T04:54:40.342316+0000","last_became_peered":"2023-10-06T04:54:40.342316+0000","last_unstale":"2023-10-06T04:54:40.342909+0000","last_undegraded":"2023-10-06T04:54:40.342909+0000","last_fullsized":"2023-10-06T04:54:40.342909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333878+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:04.598076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341225+0000","last_change":"2023-10-06T04:54:40.341225+0000","last_active":"2023-10-06T04:54:40.341225+0000","last_peered":"2023-10-06T04:54:40.341225+0000","last_clean":"2023-10-06T04:54:40.341225+0000","last_became_active":"2023-10-06T04:54:40.340018+0000","last_became_peered":"2023-10-06T04:54:40.340018+0000","last_unstale":"2023-10-06T04:54:40.341225+0000","last_undegraded":"2023-10-06T04:54:40.341225+0000","last_fullsized":"2023-10-06T04:54:40.341225+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331646+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.345606+0000","last_change":"2023-10-06T04:54:40.345606+0000","last_active":"2023-10-06T04:54:40.345606+0000","last_peered":"2023-10-06T04:54:40.345606+0000","last_clean":"2023-10-06T04:54:40.345606+0000","last_became_active":"2023-10-06T04:54:40.345364+0000","last_became_peered":"2023-10-06T04:54:40.345364+0000","last_unstale":"2023-10-06T04:54:40.345606+0000","last_undegraded":"2023-10-06T04:54:40.345606+0000","last_fullsized":"2023-10-06T04:54:40.345606+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:32.478317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330958+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:35.749980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340969+0000","last_change":"2023-10-06T04:54:40.340969+0000","last_active":"2023-10-06T04:54:40.340969+0000","last_peered":"2023-10-06T04:54:40.340969+0000","last_clean":"2023-10-06T04:54:40.340969+0000","last_became_active":"2023-10-06T04:54:40.340716+0000","last_became_peered":"2023-10-06T04:54:40.340716+0000","last_unstale":"2023-10-06T04:54:40.340969+0000","last_undegraded":"2023-10-06T04:54:40.340969+0000","last_fullsized":"2023-10-06T04:54:40.340969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327245+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.375936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342698+0000","last_change":"2023-10-06T04:54:40.342698+0000","last_active":"2023-10-06T04:54:40.342698+0000","last_peered":"2023-10-06T04:54:40.342698+0000","last_clean":"2023-10-06T04:54:40.342698+0000","last_became_active":"2023-10-06T04:54:40.342336+0000","last_became_peered":"2023-10-06T04:54:40.342336+0000","last_unstale":"2023-10-06T04:54:40.342698+0000","last_undegraded":"2023-10-06T04:54:40.342698+0000","last_fullsized":"2023-10-06T04:54:40.342698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:35.638003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330725+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343535+0000","last_change":"2023-10-06T04:54:40.343535+0000","last_active":"2023-10-06T04:54:40.343535+0000","last_peered":"2023-10-06T04:54:40.343535+0000","last_clean":"2023-10-06T04:54:40.343535+0000","last_became_active":"2023-10-06T04:54:40.343213+0000","last_became_peered":"2023-10-06T04:54:40.343213+0000","last_unstale":"2023-10-06T04:54:40.343535+0000","last_undegraded":"2023-10-06T04:54:40.343535+0000","last_fullsized":"2023-10-06T04:54:40.343535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:24.202745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333947+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:28.396681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344767+0000","last_change":"2023-10-06T04:54:40.344767+0000","last_active":"2023-10-06T04:54:40.344767+0000","last_peered":"2023-10-06T04:54:40.344767+0000","last_clean":"2023-10-06T04:54:40.344767+0000","last_became_active":"2023-10-06T04:54:40.344507+0000","last_became_peered":"2023-10-06T04:54:40.344507+0000","last_unstale":"2023-10-06T04:54:40.344767+0000","last_undegraded":"2023-10-06T04:54:40.344767+0000","last_fullsized":"2023-10-06T04:54:40.344767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329365+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:18.138617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341127+0000","last_change":"2023-10-06T04:54:40.341127+0000","last_active":"2023-10-06T04:54:40.341127+0000","last_peered":"2023-10-06T04:54:40.341127+0000","last_clean":"2023-10-06T04:54:40.341127+0000","last_became_active":"2023-10-06T04:54:40.339641+0000","last_became_peered":"2023-10-06T04:54:40.339641+0000","last_unstale":"2023-10-06T04:54:40.341127+0000","last_undegraded":"2023-10-06T04:54:40.341127+0000","last_fullsized":"2023-10-06T04:54:40.341127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.636882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.333730+0000","last_change":"2023-10-06T04:54:40.333730+0000","last_active":"2023-10-06T04:54:40.333730+0000","last_peered":"2023-10-06T04:54:40.333730+0000","last_clean":"2023-10-06T04:54:40.333730+0000","last_became_active":"2023-10-06T04:54:40.333468+0000","last_became_peered":"2023-10-06T04:54:40.333468+0000","last_unstale":"2023-10-06T04:54:40.333730+0000","last_undegraded":"2023-10-06T04:54:40.333730+0000","last_fullsized":"2023-10-06T04:54:40.333730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:34.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332948+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344570+0000","last_change":"2023-10-06T04:54:40.344570+0000","last_active":"2023-10-06T04:54:40.344570+0000","last_peered":"2023-10-06T04:54:40.344570+0000","last_clean":"2023-10-06T04:54:40.344570+0000","last_became_active":"2023-10-06T04:54:40.344295+0000","last_became_peered":"2023-10-06T04:54:40.344295+0000","last_unstale":"2023-10-06T04:54:40.344570+0000","last_undegraded":"2023-10-06T04:54:40.344570+0000","last_fullsized":"2023-10-06T04:54:40.344570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:40.208274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327281+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:47.664286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344877+0000","last_change":"2023-10-06T04:54:40.344877+0000","last_active":"2023-10-06T04:54:40.344877+0000","last_peered":"2023-10-06T04:54:40.344877+0000","last_clean":"2023-10-06T04:54:40.344877+0000","last_became_active":"2023-10-06T04:54:40.344580+0000","last_became_peered":"2023-10-06T04:54:40.344580+0000","last_unstale":"2023-10-06T04:54:40.344877+0000","last_undegraded":"2023-10-06T04:54:40.344877+0000","last_fullsized":"2023-10-06T04:54:40.344877+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328360+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340640+0000","last_change":"2023-10-06T04:54:40.340640+0000","last_active":"2023-10-06T04:54:40.340640+0000","last_peered":"2023-10-06T04:54:40.340640+0000","last_clean":"2023-10-06T04:54:40.340640+0000","last_became_active":"2023-10-06T04:54:40.340384+0000","last_became_peered":"2023-10-06T04:54:40.340384+0000","last_unstale":"2023-10-06T04:54:40.340640+0000","last_undegraded":"2023-10-06T04:54:40.340640+0000","last_fullsized":"2023-10-06T04:54:40.340640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.769263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323560+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:54:39.323560+0000","last_peered":"2023-10-06T04:54:39.323560+0000","last_clean":"2023-10-06T04:54:39.323560+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:54:39.323560+0000","last_undegraded":"2023-10-06T04:54:39.323560+0000","last_fullsized":"2023-10-06T04:54:39.323560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334410+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:15.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343623+0000","last_change":"2023-10-06T04:54:40.343623+0000","last_active":"2023-10-06T04:54:40.343623+0000","last_peered":"2023-10-06T04:54:40.343623+0000","last_clean":"2023-10-06T04:54:40.343623+0000","last_became_active":"2023-10-06T04:54:40.343366+0000","last_became_peered":"2023-10-06T04:54:40.343366+0000","last_unstale":"2023-10-06T04:54:40.343623+0000","last_undegraded":"2023-10-06T04:54:40.343623+0000","last_fullsized":"2023-10-06T04:54:40.343623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:33.074026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.325164+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:54:39.325164+0000","last_peered":"2023-10-06T04:54:39.325164+0000","last_clean":"2023-10-06T04:54:39.325164+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:54:39.325164+0000","last_undegraded":"2023-10-06T04:54:39.325164+0000","last_fullsized":"2023-10-06T04:54:39.325164+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330526+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:04.376901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344112+0000","last_change":"2023-10-06T04:54:40.344112+0000","last_active":"2023-10-06T04:54:40.344112+0000","last_peered":"2023-10-06T04:54:40.344112+0000","last_clean":"2023-10-06T04:54:40.344112+0000","last_became_active":"2023-10-06T04:54:40.343905+0000","last_became_peered":"2023-10-06T04:54:40.343905+0000","last_unstale":"2023-10-06T04:54:40.344112+0000","last_undegraded":"2023-10-06T04:54:40.344112+0000","last_fullsized":"2023-10-06T04:54:40.344112+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.324421+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:54:39.324421+0000","last_peered":"2023-10-06T04:54:39.324421+0000","last_clean":"2023-10-06T04:54:39.324421+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:54:39.324421+0000","last_undegraded":"2023-10-06T04:54:39.324421+0000","last_fullsized":"2023-10-06T04:54:39.324421+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":190,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321139+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:54:39.321139+0000","last_peered":"2023-10-06T04:54:39.321139+0000","last_clean":"2023-10-06T04:54:39.321139+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:54:39.321139+0000","last_undegraded":"2023-10-06T04:54:39.321139+0000","last_fullsized":"2023-10-06T04:54:39.321139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329601+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343032+0000","last_change":"2023-10-06T04:54:40.343032+0000","last_active":"2023-10-06T04:54:40.343032+0000","last_peered":"2023-10-06T04:54:40.343032+0000","last_clean":"2023-10-06T04:54:40.343032+0000","last_became_active":"2023-10-06T04:54:40.342591+0000","last_became_peered":"2023-10-06T04:54:40.342591+0000","last_unstale":"2023-10-06T04:54:40.343032+0000","last_undegraded":"2023-10-06T04:54:40.343032+0000","last_fullsized":"2023-10-06T04:54:40.343032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321070+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:54:39.321070+0000","last_peered":"2023-10-06T04:54:39.321070+0000","last_clean":"2023-10-06T04:54:39.321070+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:54:39.321070+0000","last_undegraded":"2023-10-06T04:54:39.321070+0000","last_fullsized":"2023-10-06T04:54:39.321070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329145+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:46.287413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.334344+0000","last_change":"2023-10-06T04:54:40.334344+0000","last_active":"2023-10-06T04:54:40.334344+0000","last_peered":"2023-10-06T04:54:40.334344+0000","last_clean":"2023-10-06T04:54:40.334344+0000","last_became_active":"2023-10-06T04:54:40.334079+0000","last_became_peered":"2023-10-06T04:54:40.334079+0000","last_unstale":"2023-10-06T04:54:40.334344+0000","last_undegraded":"2023-10-06T04:54:40.334344+0000","last_fullsized":"2023-10-06T04:54:40.334344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:28.287060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321036+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:54:39.321036+0000","last_peered":"2023-10-06T04:54:39.321036+0000","last_clean":"2023-10-06T04:54:39.321036+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:54:39.321036+0000","last_undegraded":"2023-10-06T04:54:39.321036+0000","last_fullsized":"2023-10-06T04:54:39.321036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333407+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:10.632704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340001+0000","last_change":"2023-10-06T04:54:40.340001+0000","last_active":"2023-10-06T04:54:40.340001+0000","last_peered":"2023-10-06T04:54:40.340001+0000","last_clean":"2023-10-06T04:54:40.340001+0000","last_became_active":"2023-10-06T04:54:40.339665+0000","last_became_peered":"2023-10-06T04:54:40.339665+0000","last_unstale":"2023-10-06T04:54:40.340001+0000","last_undegraded":"2023-10-06T04:54:40.340001+0000","last_fullsized":"2023-10-06T04:54:40.340001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:07.045545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.328478+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:54:39.328478+0000","last_peered":"2023-10-06T04:54:39.328478+0000","last_clean":"2023-10-06T04:54:39.328478+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:54:39.328478+0000","last_undegraded":"2023-10-06T04:54:39.328478+0000","last_fullsized":"2023-10-06T04:54:39.328478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330328+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:56.766026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343461+0000","last_change":"2023-10-06T04:54:40.343461+0000","last_active":"2023-10-06T04:54:40.343461+0000","last_peered":"2023-10-06T04:54:40.343461+0000","last_clean":"2023-10-06T04:54:40.343461+0000","last_became_active":"2023-10-06T04:54:40.343184+0000","last_became_peered":"2023-10-06T04:54:40.343184+0000","last_unstale":"2023-10-06T04:54:40.343461+0000","last_undegraded":"2023-10-06T04:54:40.343461+0000","last_fullsized":"2023-10-06T04:54:40.343461+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:47.709557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323296+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:54:39.323296+0000","last_peered":"2023-10-06T04:54:39.323296+0000","last_clean":"2023-10-06T04:54:39.323296+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:54:39.323296+0000","last_undegraded":"2023-10-06T04:54:39.323296+0000","last_fullsized":"2023-10-06T04:54:39.323296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331007+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342683+0000","last_change":"2023-10-06T04:54:40.342683+0000","last_active":"2023-10-06T04:54:40.342683+0000","last_peered":"2023-10-06T04:54:40.342683+0000","last_clean":"2023-10-06T04:54:40.342683+0000","last_became_active":"2023-10-06T04:54:40.342261+0000","last_became_peered":"2023-10-06T04:54:40.342261+0000","last_unstale":"2023-10-06T04:54:40.342683+0000","last_undegraded":"2023-10-06T04:54:40.342683+0000","last_fullsized":"2023-10-06T04:54:40.342683+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323303+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:54:39.323303+0000","last_peered":"2023-10-06T04:54:39.323303+0000","last_clean":"2023-10-06T04:54:39.323303+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:54:39.323303+0000","last_undegraded":"2023-10-06T04:54:39.323303+0000","last_fullsized":"2023-10-06T04:54:39.323303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306700,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065140,"statfs":{"total":96636764160,"available":96322703360,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.751,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.66400000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.80200000000000005,"15min":0.84499999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70099999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":1.111,"15min":1.111},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.874,"15min":0.874},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77500000000000002,"5min":1.214,"15min":1.214},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72599999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.751,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63300000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302604,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069236,"statfs":{"total":96636764160,"available":96326897664,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.48399999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51000000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.379,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.42399999999999999,"15min":0.42899999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54100000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.376,"5min":0.42799999999999999,"15min":0.437},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57499999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.63400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73699999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":1.026,"15min":1.026},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.46200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.38},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36499999999999999,"15min":0.36599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64400000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.747,"5min":1.048,"15min":1.048},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.22700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.57999999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.24399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68700000000000006,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85399999999999998,"5min":1.246,"15min":1.246},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.40699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.877,"15min":0.877},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306136,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065704,"statfs":{"total":96636764160,"available":96323280896,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44500000000000001,"5min":1.415,"15min":1.415},"last":0.25},{"interface":"front","average":{"1min":0.37,"5min":0.58599999999999997,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51400000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.44600000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.45700000000000002,"15min":0.47099999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53600000000000003,"5min":1.482,"15min":1.482},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":5.069,"15min":5.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57799999999999996,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.71299999999999997,"15min":0.73799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94799999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80200000000000005,"15min":0.83999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.504}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.77300000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":1.452,"15min":1.452},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.81399999999999995,"15min":0.85399999999999998},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":6.367,"15min":6.367},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":5,"15min":5},"last":0.84199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.81999999999999995,"15min":0.876},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63200000000000001,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.85299999999999998,"15min":0.90300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67900000000000005,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.503},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.85799999999999998,"15min":0.91400000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.68400000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64300000000000002,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.75},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.70299999999999996,"15min":0.73899999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61599999999999999,"5min":5.415,"15min":5.415},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67200000000000004,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66200000000000003,"5min":4.202,"15min":4.202},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74099999999999999,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":0.872},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":655360,"data_stored":283119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70699999999999996,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63700000000000001,"15min":0.65500000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72399999999999998,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.69299999999999995,"15min":0.71599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.746,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.44}]},{"osd":3,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.0049999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.499},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.008,"15min":1.0760000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.751},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76500000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:54:45.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:54:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.010+0000 7f8cc92c7700 1 Processor -- start 2023-10-06T04:54:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.010+0000 7f8cc92c7700 1 -- start start 2023-10-06T04:54:46.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc92c7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 0x7f8cc41642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc92c7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc40b0ce0 con 0x7f8cc415cb50 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc92c7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc40cb820 con 0x7f8cc40cd950 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc92c7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40cccc0 con 0x7f8cc40b2910 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8cc415cb50 0x7f8cc40b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48218/0 (socket says 172.21.15.3:48218) 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc2ffd700 1 -- 172.21.15.3:0/3725231074 learned_addr learned my addr 172.21.15.3:0/3725231074 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141864559 0 0) 0x7f8cc40b0ce0 con 0x7f8cc415cb50 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc37fe700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 0x7f8cc41642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb0003570 con 0x7f8cc415cb50 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789161713 0 0) 0x7f8cb0003570 con 0x7f8cc415cb50 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 msgr2=0x7f8cc41642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 0x7f8cc41642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 >> v1:172.21.15.153:6789/0 conn(0x7f8cc40cd950 legacy=0x7f8cc40b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40b0760 con 0x7f8cc415cb50 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f8cbc001970 con 0x7f8cc415cb50 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 >> v1:172.21.15.3:6789/0 conn(0x7f8cc415cb50 legacy=0x7f8cc40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 0x7f8cb0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb0004da0 0x7f8cb00051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40cccc0 con 0x7f8cb00040c0 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb000bc70 con 0x7f8cb0004da0 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.011+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb000be30 con 0x7f8cc40b2910 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc2ffd700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 0x7f8cb0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc37fe700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb0004da0 0x7f8cb00051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc27fc700 1 -- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 msgr2=0x7f8cb0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b2910 0x7f8cb0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc27fc700 1 -- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb0004da0 msgr2=0x7f8cb00051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb0004da0 0x7f8cb00051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc27fc700 1 -- 172.21.15.3:0/3725231074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb0003570 con 0x7f8cb00040c0 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc37fe700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb0004da0 0x7f8cb00051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cb0004810 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001880 tx=0x7f8cac007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5171b222b940b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac012420 con 0x7f8cb00040c0 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac012580 con 0x7f8cb00040c0 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.012+0000 7f8cc1ffb700 1 -- 172.21.15.3:0/3725231074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac012920 con 0x7f8cb00040c0 2023-10-06T04:54:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.013+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3725231074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 msgr2=0x7f8cb0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.013+0000 7f8cc92c7700 1 --2- 172.21.15.3:0/3725231074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cb0004810 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001880 tx=0x7f8cac007d30 comp rx=0 tx=0).stop 2023-10-06T04:54:46.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.013+0000 7f8cc37fe700 1 -- 172.21.15.3:0/3725231074 reap_dead start 2023-10-06T04:54:46.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.013+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3725231074 shutdown_connections 2023-10-06T04:54:46.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.013+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3725231074 >> 172.21.15.3:0/3725231074 conn(0x7f8cc4070310 msgr2=0x7f8cc4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:46.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.013+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3725231074 shutdown_connections 2023-10-06T04:54:46.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.013+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3725231074 wait complete. 2023-10-06T04:54:46.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.014+0000 7f8cc92c7700 1 Processor -- start 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.014+0000 7f8cc92c7700 1 -- start start 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc92c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cc40c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc92c7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc40cfe10 0x7f8cc40af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc92c7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b41d0 0x7f8cc40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc92c7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4169c00 con 0x7f8cb00040c0 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc92c7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40b4bb0 con 0x7f8cc40cfe10 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc92c7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40c4950 con 0x7f8cc40b41d0 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cc40c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cc40c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44978/0 (socket says 172.21.15.3:44978) 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 -- 172.21.15.3:0/3932116686 learned_addr learned my addr 172.21.15.3:0/3932116686 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc37fe700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b41d0 0x7f8cc40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc40cfe10 0x7f8cc40af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 -- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b41d0 msgr2=0x7f8cc40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b41d0 0x7f8cc40b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 -- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc40cfe10 msgr2=0x7f8cc40af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc40cfe10 0x7f8cc40af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 -- 172.21.15.3:0/3932116686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40c5700 con 0x7f8cb00040c0 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc37fe700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b41d0 0x7f8cc40b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc40cfe10 0x7f8cc40af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.015+0000 7f8cc2ffd700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cc40c4410 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc001460 tx=0x7f8cbc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.016+0000 7f8cbaffd700 1 -- 172.21.15.3:0/3932116686 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc005d80 con 0x7f8cb00040c0 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.016+0000 7f8cbaffd700 1 -- 172.21.15.3:0/3932116686 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc00d9d0 con 0x7f8cb00040c0 2023-10-06T04:54:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.016+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc40afb70 con 0x7f8cb00040c0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.016+0000 7f8cbaffd700 1 -- 172.21.15.3:0/3932116686 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc00dd90 con 0x7f8cb00040c0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.016+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc40c4e60 con 0x7f8cb00040c0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.017+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc4169d50 con 0x7f8cb00040c0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.017+0000 7f8cbaffd700 1 -- 172.21.15.3:0/3932116686 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8cbc0072e0 con 0x7f8cb00040c0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.017+0000 7f8cbaffd700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ca4066f30 0x7f8ca40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.018+0000 7f8cbaffd700 1 -- 172.21.15.3:0/3932116686 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8cbc084c00 con 0x7f8cb00040c0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.018+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ca4066f30 0x7f8ca40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.018+0000 7f8cc27fc700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ca4066f30 0x7f8ca40693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40b5530 tx=0x7f8cac015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.022+0000 7f8cbaffd700 1 -- 172.21.15.3:0/3932116686 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cc4169d50 con 0x7f8cb00040c0 2023-10-06T04:54:46.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.211+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cc4169d50 con 0x7f8ca4066f30 2023-10-06T04:54:46.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.220+0000 7f8cbaffd700 1 -- 172.21.15.3:0/3932116686 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308672 (secure 0 0 0) 0x7f8cc4169d50 con 0x7f8ca4066f30 2023-10-06T04:54:46.225 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ca4066f30 msgr2=0x7f8ca40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ca4066f30 0x7f8ca40693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40b5530 tx=0x7f8cac015000 comp rx=0 tx=0).stop 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 msgr2=0x7f8cc40c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cc40c4410 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc001460 tx=0x7f8cbc009040 comp rx=0 tx=0).stop 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 shutdown_connections 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8ca4066f30 0x7f8ca40693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8cc40b41d0 0x7f8cc40b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb00040c0 0x7f8cc40c4410 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 --2- 172.21.15.3:0/3932116686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc40cfe10 0x7f8cc40af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 >> 172.21.15.3:0/3932116686 conn(0x7f8cc4070310 msgr2=0x7f8cc40cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 shutdown_connections 2023-10-06T04:54:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.223+0000 7f8cc92c7700 1 -- 172.21.15.3:0/3932116686 wait complete. 2023-10-06T04:54:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:54:46.237 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-10-06T04:54:45.458470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425588,"kb_used_data":6324,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752549132,"statfs":{"total":773094113280,"available":770610311168,"internally_reserved":0,"allocated":6475776,"data_stored":3465669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001559"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333786+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331685+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.335105+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329702+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:55.591082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330610+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327617+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:01.768927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330876+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329686+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328812+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:33.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332622+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:54.642506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331759+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333352+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328323+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328695+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:29.292684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327879+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:14.188545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331666+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328892+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330778+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:47.801935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325842+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332133+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326112+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326304+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328233+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331026+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331474+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329585+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332529+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331609+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332943+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:05.163312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331203+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.065580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332535+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:26.889286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339794+0000","last_change":"2023-10-06T04:54:40.339794+0000","last_active":"2023-10-06T04:54:40.339794+0000","last_peered":"2023-10-06T04:54:40.339794+0000","last_clean":"2023-10-06T04:54:40.339794+0000","last_became_active":"2023-10-06T04:54:40.339482+0000","last_became_peered":"2023-10-06T04:54:40.339482+0000","last_unstale":"2023-10-06T04:54:40.339794+0000","last_undegraded":"2023-10-06T04:54:40.339794+0000","last_fullsized":"2023-10-06T04:54:40.339794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328444+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340091+0000","last_change":"2023-10-06T04:54:40.340091+0000","last_active":"2023-10-06T04:54:40.340091+0000","last_peered":"2023-10-06T04:54:40.340091+0000","last_clean":"2023-10-06T04:54:40.340091+0000","last_became_active":"2023-10-06T04:54:40.339729+0000","last_became_peered":"2023-10-06T04:54:40.339729+0000","last_unstale":"2023-10-06T04:54:40.340091+0000","last_undegraded":"2023-10-06T04:54:40.340091+0000","last_fullsized":"2023-10-06T04:54:40.340091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330525+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:05.603150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344126+0000","last_change":"2023-10-06T04:54:40.344126+0000","last_active":"2023-10-06T04:54:40.344126+0000","last_peered":"2023-10-06T04:54:40.344126+0000","last_clean":"2023-10-06T04:54:40.344126+0000","last_became_active":"2023-10-06T04:54:40.343824+0000","last_became_peered":"2023-10-06T04:54:40.343824+0000","last_unstale":"2023-10-06T04:54:40.344126+0000","last_undegraded":"2023-10-06T04:54:40.344126+0000","last_fullsized":"2023-10-06T04:54:40.344126+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:32.239178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329189+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:33.176469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338831+0000","last_change":"2023-10-06T04:54:40.338831+0000","last_active":"2023-10-06T04:54:40.338831+0000","last_peered":"2023-10-06T04:54:40.338831+0000","last_clean":"2023-10-06T04:54:40.338831+0000","last_became_active":"2023-10-06T04:54:40.338570+0000","last_became_peered":"2023-10-06T04:54:40.338570+0000","last_unstale":"2023-10-06T04:54:40.338831+0000","last_undegraded":"2023-10-06T04:54:40.338831+0000","last_fullsized":"2023-10-06T04:54:40.338831+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:32.170845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327374+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:43.786031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.332965+0000","last_change":"2023-10-06T04:54:40.332965+0000","last_active":"2023-10-06T04:54:40.332965+0000","last_peered":"2023-10-06T04:54:40.332965+0000","last_clean":"2023-10-06T04:54:40.332965+0000","last_became_active":"2023-10-06T04:54:40.332661+0000","last_became_peered":"2023-10-06T04:54:40.332661+0000","last_unstale":"2023-10-06T04:54:40.332965+0000","last_undegraded":"2023-10-06T04:54:40.332965+0000","last_fullsized":"2023-10-06T04:54:40.332965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.328723+0000","last_change":"2023-10-06T04:54:40.328723+0000","last_active":"2023-10-06T04:54:40.328723+0000","last_peered":"2023-10-06T04:54:40.328723+0000","last_clean":"2023-10-06T04:54:40.328723+0000","last_became_active":"2023-10-06T04:54:40.328424+0000","last_became_peered":"2023-10-06T04:54:40.328424+0000","last_unstale":"2023-10-06T04:54:40.328723+0000","last_undegraded":"2023-10-06T04:54:40.328723+0000","last_fullsized":"2023-10-06T04:54:40.328723+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334363+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:37.872096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344255+0000","last_change":"2023-10-06T04:54:40.344255+0000","last_active":"2023-10-06T04:54:40.344255+0000","last_peered":"2023-10-06T04:54:40.344255+0000","last_clean":"2023-10-06T04:54:40.344255+0000","last_became_active":"2023-10-06T04:54:40.343918+0000","last_became_peered":"2023-10-06T04:54:40.343918+0000","last_unstale":"2023-10-06T04:54:40.344255+0000","last_undegraded":"2023-10-06T04:54:40.344255+0000","last_fullsized":"2023-10-06T04:54:40.344255+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326195+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338640+0000","last_change":"2023-10-06T04:54:40.338640+0000","last_active":"2023-10-06T04:54:40.338640+0000","last_peered":"2023-10-06T04:54:40.338640+0000","last_clean":"2023-10-06T04:54:40.338640+0000","last_became_active":"2023-10-06T04:54:40.338385+0000","last_became_peered":"2023-10-06T04:54:40.338385+0000","last_unstale":"2023-10-06T04:54:40.338640+0000","last_undegraded":"2023-10-06T04:54:40.338640+0000","last_fullsized":"2023-10-06T04:54:40.338640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:40.835360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332028+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340720+0000","last_change":"2023-10-06T04:54:40.340720+0000","last_active":"2023-10-06T04:54:40.340720+0000","last_peered":"2023-10-06T04:54:40.340720+0000","last_clean":"2023-10-06T04:54:40.340720+0000","last_became_active":"2023-10-06T04:54:40.340447+0000","last_became_peered":"2023-10-06T04:54:40.340447+0000","last_unstale":"2023-10-06T04:54:40.340720+0000","last_undegraded":"2023-10-06T04:54:40.340720+0000","last_fullsized":"2023-10-06T04:54:40.340720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:38.363715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325883+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339697+0000","last_change":"2023-10-06T04:54:40.339697+0000","last_active":"2023-10-06T04:54:40.339697+0000","last_peered":"2023-10-06T04:54:40.339697+0000","last_clean":"2023-10-06T04:54:40.339697+0000","last_became_active":"2023-10-06T04:54:40.339299+0000","last_became_peered":"2023-10-06T04:54:40.339299+0000","last_unstale":"2023-10-06T04:54:40.339697+0000","last_undegraded":"2023-10-06T04:54:40.339697+0000","last_fullsized":"2023-10-06T04:54:40.339697+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:12.235281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334708+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:36.233559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339769+0000","last_change":"2023-10-06T04:54:40.339769+0000","last_active":"2023-10-06T04:54:40.339769+0000","last_peered":"2023-10-06T04:54:40.339769+0000","last_clean":"2023-10-06T04:54:40.339769+0000","last_became_active":"2023-10-06T04:54:40.339511+0000","last_became_peered":"2023-10-06T04:54:40.339511+0000","last_unstale":"2023-10-06T04:54:40.339769+0000","last_undegraded":"2023-10-06T04:54:40.339769+0000","last_fullsized":"2023-10-06T04:54:40.339769+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:25.589985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328269+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341385+0000","last_change":"2023-10-06T04:54:40.341385+0000","last_active":"2023-10-06T04:54:40.341385+0000","last_peered":"2023-10-06T04:54:40.341385+0000","last_clean":"2023-10-06T04:54:40.341385+0000","last_became_active":"2023-10-06T04:54:40.341078+0000","last_became_peered":"2023-10-06T04:54:40.341078+0000","last_unstale":"2023-10-06T04:54:40.341385+0000","last_undegraded":"2023-10-06T04:54:40.341385+0000","last_fullsized":"2023-10-06T04:54:40.341385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328102+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344621+0000","last_change":"2023-10-06T04:54:40.344621+0000","last_active":"2023-10-06T04:54:40.344621+0000","last_peered":"2023-10-06T04:54:40.344621+0000","last_clean":"2023-10-06T04:54:40.344621+0000","last_became_active":"2023-10-06T04:54:40.344347+0000","last_became_peered":"2023-10-06T04:54:40.344347+0000","last_unstale":"2023-10-06T04:54:40.344621+0000","last_undegraded":"2023-10-06T04:54:40.344621+0000","last_fullsized":"2023-10-06T04:54:40.344621+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:41.740059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330179+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342909+0000","last_change":"2023-10-06T04:54:40.342909+0000","last_active":"2023-10-06T04:54:40.342909+0000","last_peered":"2023-10-06T04:54:40.342909+0000","last_clean":"2023-10-06T04:54:40.342909+0000","last_became_active":"2023-10-06T04:54:40.342316+0000","last_became_peered":"2023-10-06T04:54:40.342316+0000","last_unstale":"2023-10-06T04:54:40.342909+0000","last_undegraded":"2023-10-06T04:54:40.342909+0000","last_fullsized":"2023-10-06T04:54:40.342909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333878+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:04.598076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341225+0000","last_change":"2023-10-06T04:54:40.341225+0000","last_active":"2023-10-06T04:54:40.341225+0000","last_peered":"2023-10-06T04:54:40.341225+0000","last_clean":"2023-10-06T04:54:40.341225+0000","last_became_active":"2023-10-06T04:54:40.340018+0000","last_became_peered":"2023-10-06T04:54:40.340018+0000","last_unstale":"2023-10-06T04:54:40.341225+0000","last_undegraded":"2023-10-06T04:54:40.341225+0000","last_fullsized":"2023-10-06T04:54:40.341225+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331646+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.345606+0000","last_change":"2023-10-06T04:54:40.345606+0000","last_active":"2023-10-06T04:54:40.345606+0000","last_peered":"2023-10-06T04:54:40.345606+0000","last_clean":"2023-10-06T04:54:40.345606+0000","last_became_active":"2023-10-06T04:54:40.345364+0000","last_became_peered":"2023-10-06T04:54:40.345364+0000","last_unstale":"2023-10-06T04:54:40.345606+0000","last_undegraded":"2023-10-06T04:54:40.345606+0000","last_fullsized":"2023-10-06T04:54:40.345606+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:32.478317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330958+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:35.749980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340969+0000","last_change":"2023-10-06T04:54:40.340969+0000","last_active":"2023-10-06T04:54:40.340969+0000","last_peered":"2023-10-06T04:54:40.340969+0000","last_clean":"2023-10-06T04:54:40.340969+0000","last_became_active":"2023-10-06T04:54:40.340716+0000","last_became_peered":"2023-10-06T04:54:40.340716+0000","last_unstale":"2023-10-06T04:54:40.340969+0000","last_undegraded":"2023-10-06T04:54:40.340969+0000","last_fullsized":"2023-10-06T04:54:40.340969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327245+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.375936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342698+0000","last_change":"2023-10-06T04:54:40.342698+0000","last_active":"2023-10-06T04:54:40.342698+0000","last_peered":"2023-10-06T04:54:40.342698+0000","last_clean":"2023-10-06T04:54:40.342698+0000","last_became_active":"2023-10-06T04:54:40.342336+0000","last_became_peered":"2023-10-06T04:54:40.342336+0000","last_unstale":"2023-10-06T04:54:40.342698+0000","last_undegraded":"2023-10-06T04:54:40.342698+0000","last_fullsized":"2023-10-06T04:54:40.342698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:35.638003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330725+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343535+0000","last_change":"2023-10-06T04:54:40.343535+0000","last_active":"2023-10-06T04:54:40.343535+0000","last_peered":"2023-10-06T04:54:40.343535+0000","last_clean":"2023-10-06T04:54:40.343535+0000","last_became_active":"2023-10-06T04:54:40.343213+0000","last_became_peered":"2023-10-06T04:54:40.343213+0000","last_unstale":"2023-10-06T04:54:40.343535+0000","last_undegraded":"2023-10-06T04:54:40.343535+0000","last_fullsized":"2023-10-06T04:54:40.343535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:24.202745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333947+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:28.396681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344767+0000","last_change":"2023-10-06T04:54:40.344767+0000","last_active":"2023-10-06T04:54:40.344767+0000","last_peered":"2023-10-06T04:54:40.344767+0000","last_clean":"2023-10-06T04:54:40.344767+0000","last_became_active":"2023-10-06T04:54:40.344507+0000","last_became_peered":"2023-10-06T04:54:40.344507+0000","last_unstale":"2023-10-06T04:54:40.344767+0000","last_undegraded":"2023-10-06T04:54:40.344767+0000","last_fullsized":"2023-10-06T04:54:40.344767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329365+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:18.138617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341127+0000","last_change":"2023-10-06T04:54:40.341127+0000","last_active":"2023-10-06T04:54:40.341127+0000","last_peered":"2023-10-06T04:54:40.341127+0000","last_clean":"2023-10-06T04:54:40.341127+0000","last_became_active":"2023-10-06T04:54:40.339641+0000","last_became_peered":"2023-10-06T04:54:40.339641+0000","last_unstale":"2023-10-06T04:54:40.341127+0000","last_undegraded":"2023-10-06T04:54:40.341127+0000","last_fullsized":"2023-10-06T04:54:40.341127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.636882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.333730+0000","last_change":"2023-10-06T04:54:40.333730+0000","last_active":"2023-10-06T04:54:40.333730+0000","last_peered":"2023-10-06T04:54:40.333730+0000","last_clean":"2023-10-06T04:54:40.333730+0000","last_became_active":"2023-10-06T04:54:40.333468+0000","last_became_peered":"2023-10-06T04:54:40.333468+0000","last_unstale":"2023-10-06T04:54:40.333730+0000","last_undegraded":"2023-10-06T04:54:40.333730+0000","last_fullsized":"2023-10-06T04:54:40.333730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:34.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332948+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344570+0000","last_change":"2023-10-06T04:54:40.344570+0000","last_active":"2023-10-06T04:54:40.344570+0000","last_peered":"2023-10-06T04:54:40.344570+0000","last_clean":"2023-10-06T04:54:40.344570+0000","last_became_active":"2023-10-06T04:54:40.344295+0000","last_became_peered":"2023-10-06T04:54:40.344295+0000","last_unstale":"2023-10-06T04:54:40.344570+0000","last_undegraded":"2023-10-06T04:54:40.344570+0000","last_fullsized":"2023-10-06T04:54:40.344570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:40.208274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327281+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:47.664286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344877+0000","last_change":"2023-10-06T04:54:40.344877+0000","last_active":"2023-10-06T04:54:40.344877+0000","last_peered":"2023-10-06T04:54:40.344877+0000","last_clean":"2023-10-06T04:54:40.344877+0000","last_became_active":"2023-10-06T04:54:40.344580+0000","last_became_peered":"2023-10-06T04:54:40.344580+0000","last_unstale":"2023-10-06T04:54:40.344877+0000","last_undegraded":"2023-10-06T04:54:40.344877+0000","last_fullsized":"2023-10-06T04:54:40.344877+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328360+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340640+0000","last_change":"2023-10-06T04:54:40.340640+0000","last_active":"2023-10-06T04:54:40.340640+0000","last_peered":"2023-10-06T04:54:40.340640+0000","last_clean":"2023-10-06T04:54:40.340640+0000","last_became_active":"2023-10-06T04:54:40.340384+0000","last_became_peered":"2023-10-06T04:54:40.340384+0000","last_unstale":"2023-10-06T04:54:40.340640+0000","last_undegraded":"2023-10-06T04:54:40.340640+0000","last_fullsized":"2023-10-06T04:54:40.340640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.769263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323560+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:54:39.323560+0000","last_peered":"2023-10-06T04:54:39.323560+0000","last_clean":"2023-10-06T04:54:39.323560+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:54:39.323560+0000","last_undegraded":"2023-10-06T04:54:39.323560+0000","last_fullsized":"2023-10-06T04:54:39.323560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334410+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:15.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343623+0000","last_change":"2023-10-06T04:54:40.343623+0000","last_active":"2023-10-06T04:54:40.343623+0000","last_peered":"2023-10-06T04:54:40.343623+0000","last_clean":"2023-10-06T04:54:40.343623+0000","last_became_active":"2023-10-06T04:54:40.343366+0000","last_became_peered":"2023-10-06T04:54:40.343366+0000","last_unstale":"2023-10-06T04:54:40.343623+0000","last_undegraded":"2023-10-06T04:54:40.343623+0000","last_fullsized":"2023-10-06T04:54:40.343623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:33.074026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.325164+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:54:39.325164+0000","last_peered":"2023-10-06T04:54:39.325164+0000","last_clean":"2023-10-06T04:54:39.325164+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:54:39.325164+0000","last_undegraded":"2023-10-06T04:54:39.325164+0000","last_fullsized":"2023-10-06T04:54:39.325164+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330526+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:04.376901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344112+0000","last_change":"2023-10-06T04:54:40.344112+0000","last_active":"2023-10-06T04:54:40.344112+0000","last_peered":"2023-10-06T04:54:40.344112+0000","last_clean":"2023-10-06T04:54:40.344112+0000","last_became_active":"2023-10-06T04:54:40.343905+0000","last_became_peered":"2023-10-06T04:54:40.343905+0000","last_unstale":"2023-10-06T04:54:40.344112+0000","last_undegraded":"2023-10-06T04:54:40.344112+0000","last_fullsized":"2023-10-06T04:54:40.344112+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.324421+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:54:39.324421+0000","last_peered":"2023-10-06T04:54:39.324421+0000","last_clean":"2023-10-06T04:54:39.324421+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:54:39.324421+0000","last_undegraded":"2023-10-06T04:54:39.324421+0000","last_fullsized":"2023-10-06T04:54:39.324421+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":190,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321139+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:54:39.321139+0000","last_peered":"2023-10-06T04:54:39.321139+0000","last_clean":"2023-10-06T04:54:39.321139+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:54:39.321139+0000","last_undegraded":"2023-10-06T04:54:39.321139+0000","last_fullsized":"2023-10-06T04:54:39.321139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329601+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343032+0000","last_change":"2023-10-06T04:54:40.343032+0000","last_active":"2023-10-06T04:54:40.343032+0000","last_peered":"2023-10-06T04:54:40.343032+0000","last_clean":"2023-10-06T04:54:40.343032+0000","last_became_active":"2023-10-06T04:54:40.342591+0000","last_became_peered":"2023-10-06T04:54:40.342591+0000","last_unstale":"2023-10-06T04:54:40.343032+0000","last_undegraded":"2023-10-06T04:54:40.343032+0000","last_fullsized":"2023-10-06T04:54:40.343032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321070+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:54:39.321070+0000","last_peered":"2023-10-06T04:54:39.321070+0000","last_clean":"2023-10-06T04:54:39.321070+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:54:39.321070+0000","last_undegraded":"2023-10-06T04:54:39.321070+0000","last_fullsized":"2023-10-06T04:54:39.321070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329145+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:46.287413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.334344+0000","last_change":"2023-10-06T04:54:40.334344+0000","last_active":"2023-10-06T04:54:40.334344+0000","last_peered":"2023-10-06T04:54:40.334344+0000","last_clean":"2023-10-06T04:54:40.334344+0000","last_became_active":"2023-10-06T04:54:40.334079+0000","last_became_peered":"2023-10-06T04:54:40.334079+0000","last_unstale":"2023-10-06T04:54:40.334344+0000","last_undegraded":"2023-10-06T04:54:40.334344+0000","last_fullsized":"2023-10-06T04:54:40.334344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:28.287060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321036+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:54:39.321036+0000","last_peered":"2023-10-06T04:54:39.321036+0000","last_clean":"2023-10-06T04:54:39.321036+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:54:39.321036+0000","last_undegraded":"2023-10-06T04:54:39.321036+0000","last_fullsized":"2023-10-06T04:54:39.321036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333407+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:10.632704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340001+0000","last_change":"2023-10-06T04:54:40.340001+0000","last_active":"2023-10-06T04:54:40.340001+0000","last_peered":"2023-10-06T04:54:40.340001+0000","last_clean":"2023-10-06T04:54:40.340001+0000","last_became_active":"2023-10-06T04:54:40.339665+0000","last_became_peered":"2023-10-06T04:54:40.339665+0000","last_unstale":"2023-10-06T04:54:40.340001+0000","last_undegraded":"2023-10-06T04:54:40.340001+0000","last_fullsized":"2023-10-06T04:54:40.340001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:07.045545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.328478+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:54:39.328478+0000","last_peered":"2023-10-06T04:54:39.328478+0000","last_clean":"2023-10-06T04:54:39.328478+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:54:39.328478+0000","last_undegraded":"2023-10-06T04:54:39.328478+0000","last_fullsized":"2023-10-06T04:54:39.328478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330328+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:56.766026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343461+0000","last_change":"2023-10-06T04:54:40.343461+0000","last_active":"2023-10-06T04:54:40.343461+0000","last_peered":"2023-10-06T04:54:40.343461+0000","last_clean":"2023-10-06T04:54:40.343461+0000","last_became_active":"2023-10-06T04:54:40.343184+0000","last_became_peered":"2023-10-06T04:54:40.343184+0000","last_unstale":"2023-10-06T04:54:40.343461+0000","last_undegraded":"2023-10-06T04:54:40.343461+0000","last_fullsized":"2023-10-06T04:54:40.343461+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:47.709557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323296+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:54:39.323296+0000","last_peered":"2023-10-06T04:54:39.323296+0000","last_clean":"2023-10-06T04:54:39.323296+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:54:39.323296+0000","last_undegraded":"2023-10-06T04:54:39.323296+0000","last_fullsized":"2023-10-06T04:54:39.323296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331007+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342683+0000","last_change":"2023-10-06T04:54:40.342683+0000","last_active":"2023-10-06T04:54:40.342683+0000","last_peered":"2023-10-06T04:54:40.342683+0000","last_clean":"2023-10-06T04:54:40.342683+0000","last_became_active":"2023-10-06T04:54:40.342261+0000","last_became_peered":"2023-10-06T04:54:40.342261+0000","last_unstale":"2023-10-06T04:54:40.342683+0000","last_undegraded":"2023-10-06T04:54:40.342683+0000","last_fullsized":"2023-10-06T04:54:40.342683+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323303+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:54:39.323303+0000","last_peered":"2023-10-06T04:54:39.323303+0000","last_clean":"2023-10-06T04:54:39.323303+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:54:39.323303+0000","last_undegraded":"2023-10-06T04:54:39.323303+0000","last_fullsized":"2023-10-06T04:54:39.323303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306700,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065140,"statfs":{"total":96636764160,"available":96322703360,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.751,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.66400000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.80200000000000005,"15min":0.84499999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70099999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":1.111,"15min":1.111},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.874,"15min":0.874},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77500000000000002,"5min":1.214,"15min":1.214},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72599999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.751,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63300000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302604,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069236,"statfs":{"total":96636764160,"available":96326897664,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.48399999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51000000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.379,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.42399999999999999,"15min":0.42899999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54100000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.376,"5min":0.42799999999999999,"15min":0.437},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57499999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.63400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73699999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":1.026,"15min":1.026},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.46200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.38},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36499999999999999,"15min":0.36599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64400000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.747,"5min":1.048,"15min":1.048},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.22700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.57999999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.24399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68700000000000006,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85399999999999998,"5min":1.246,"15min":1.246},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.40699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.877,"15min":0.877},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306136,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065704,"statfs":{"total":96636764160,"available":96323280896,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44500000000000001,"5min":1.415,"15min":1.415},"last":0.25},{"interface":"front","average":{"1min":0.37,"5min":0.58599999999999997,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51400000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.44600000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.45700000000000002,"15min":0.47099999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53600000000000003,"5min":1.482,"15min":1.482},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":5.069,"15min":5.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57799999999999996,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.71299999999999997,"15min":0.73799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94799999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80200000000000005,"15min":0.83999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.504}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.77300000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":1.452,"15min":1.452},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.81399999999999995,"15min":0.85399999999999998},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":6.367,"15min":6.367},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":5,"15min":5},"last":0.84199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.81999999999999995,"15min":0.876},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63200000000000001,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.85299999999999998,"15min":0.90300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67900000000000005,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.503},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.85799999999999998,"15min":0.91400000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.68400000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64300000000000002,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.75},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.70299999999999996,"15min":0.73899999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61599999999999999,"5min":5.415,"15min":5.415},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67200000000000004,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66200000000000003,"5min":4.202,"15min":4.202},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74099999999999999,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":0.872},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":655360,"data_stored":283119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70699999999999996,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63700000000000001,"15min":0.65500000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72399999999999998,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.69299999999999995,"15min":0.71599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.746,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.44}]},{"osd":3,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.0049999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.499},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.008,"15min":1.0760000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.751},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76500000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:54:46.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-10-06T04:54:46.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.333786+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332500+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331685+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.335105+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-10-06T04:54:46.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329702+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:43:55.591082+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330610+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.327617+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:00:01.768927+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330876+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329686+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328812+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:10:33.032323+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332622+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:15:54.642506+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331759+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.333352+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-10-06T04:54:46.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328323+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328695+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:06:29.292684+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.327879+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:06:14.188545+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331666+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328892+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330778+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:27:47.801935+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.325842+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332133+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.326112+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.326304+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:54:46.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328233+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331026+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331474+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329585+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-10-06T04:54:46.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332529+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329500+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331609+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332943+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:41:05.163312+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2023-10-06T04:54:46.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331203+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:57:57.065580+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332535+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:28:26.889286+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328444+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330525+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:01:05.603150+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329189+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:49:33.176469+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-10-06T04:54:46.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.327374+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:31:43.786031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331810+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.334363+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:24:37.872096+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.326195+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332028+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.325883+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.334708+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:19:36.233559+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328269+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328102+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330179+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.333878+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:28:04.598076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331646+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330958+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:47:35.749980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-10-06T04:54:46.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.327245+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:36:40.375936+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330725+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.333947+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:11:28.396681+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329365+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:08:18.138617+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328810+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.332948+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.327281+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:38:47.664286+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.328360+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.334410+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:32:15.134553+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-10-06T04:54:46.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330526+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:34:04.376901+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329601+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.329145+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:36:46.287413+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.333407+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:54:10.632704+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-10-06T04:54:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.330328+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:56.766026+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:54:46.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-10-06T04:54:46.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:54:40.317895+0000', 'last_change': '2023-10-06T04:54:40.331007+0000', 'last_active': '2023-10-06T04:54:40.317895+0000', 'last_peered': '2023-10-06T04:54:40.317895+0000', 'last_clean': '2023-10-06T04:54:40.317895+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:54:40.317895+0000', 'last_undegraded': '2023-10-06T04:54:40.317895+0000', 'last_fullsized': '2023-10-06T04:54:40.317895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'last_clean_scrub_stamp': '2023-10-06T04:54:40.317895+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:54:46.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T04:54:46.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.394+0000 7f62c1701700 1 Processor -- start 2023-10-06T04:54:46.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.394+0000 7f62c1701700 1 -- start start 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.394+0000 7f62c1701700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f62bc0b39b0 0x7f62bc15d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62c1701700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc15aee0 con 0x7f62bc15aa30 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62c1701700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc15dc10 con 0x7f62bc0b3d80 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62c1701700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62bc15def0 con 0x7f62bc0b39b0 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62ba7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62bc15aa30 0x7f62bc159110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48222/0 (socket says 172.21.15.3:48222) 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62ba7fc700 1 -- 172.21.15.3:0/3413897458 learned_addr learned my addr 172.21.15.3:0/3413897458 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62baffd700 1 --2- 172.21.15.3:0/3413897458 >> v2:172.21.15.153:3300/0 conn(0x7f62bc0b39b0 0x7f62bc15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62b9ffb700 1 -- 172.21.15.3:0/3413897458 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063242419 0 0) 0x7f62bc15aee0 con 0x7f62bc15aa30 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62b9ffb700 1 -- 172.21.15.3:0/3413897458 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a8003570 con 0x7f62bc15aa30 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62baffd700 1 -- 172.21.15.3:0/3413897458 >> v1:172.21.15.153:6789/0 conn(0x7f62bc0b3d80 legacy=0x7f62bc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62baffd700 1 -- 172.21.15.3:0/3413897458 >> v1:172.21.15.3:6789/0 conn(0x7f62bc15aa30 legacy=0x7f62bc159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62b9ffb700 1 -- 172.21.15.3:0/3413897458 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361378774 0 0) 0x7f62a8003570 con 0x7f62bc15aa30 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62baffd700 1 -- 172.21.15.3:0/3413897458 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc15e1d0 con 0x7f62bc0b39b0 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.395+0000 7f62baffd700 1 --2- 172.21.15.3:0/3413897458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b39b0 0x7f62bc15d310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0098e0 tx=0x7f62ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=273dc3d0730ef752 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62b9ffb700 1 -- 172.21.15.3:0/3413897458 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4001470 con 0x7f62bc0b39b0 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62b9ffb700 1 -- 172.21.15.3:0/3413897458 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac011420 con 0x7f62bc0b39b0 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62b9ffb700 1 -- 172.21.15.3:0/3413897458 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0117c0 con 0x7f62bc0b39b0 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62c1701700 1 -- 172.21.15.3:0/3413897458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b39b0 msgr2=0x7f62bc15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62c1701700 1 --2- 172.21.15.3:0/3413897458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b39b0 0x7f62bc15d310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0098e0 tx=0x7f62ac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62c1701700 1 -- 172.21.15.3:0/3413897458 shutdown_connections 2023-10-06T04:54:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62c1701700 1 --2- 172.21.15.3:0/3413897458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b39b0 0x7f62bc15d310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62c1701700 1 -- 172.21.15.3:0/3413897458 >> 172.21.15.3:0/3413897458 conn(0x7f62bc067ac0 msgr2=0x7f62bc067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62c1701700 1 -- 172.21.15.3:0/3413897458 shutdown_connections 2023-10-06T04:54:46.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.396+0000 7f62c1701700 1 -- 172.21.15.3:0/3413897458 wait complete. 2023-10-06T04:54:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.397+0000 7f62c1701700 1 Processor -- start 2023-10-06T04:54:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.397+0000 7f62c1701700 1 -- start start 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62c1701700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62bc0b39b0 0x7f62bc0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62c1701700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b3d80 0x7f62bc16aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62c1701700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc15aa30 0x7f62bc0c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62c1701700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0c9550 con 0x7f62bc15aa30 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62c1701700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0ca0d0 con 0x7f62bc0b3d80 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62c1701700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0c79d0 con 0x7f62bc0b39b0 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62bb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc15aa30 0x7f62bc0c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62bb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc15aa30 0x7f62bc0c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44994/0 (socket says 172.21.15.3:44994) 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.398+0000 7f62bb7fe700 1 -- 172.21.15.3:0/3014331873 learned_addr learned my addr 172.21.15.3:0/3014331873 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62ba7fc700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b3d80 0x7f62bc16aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62baffd700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62bc0b39b0 0x7f62bc0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62bb7fe700 1 -- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62bc0b39b0 msgr2=0x7f62bc0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62bb7fe700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62bc0b39b0 0x7f62bc0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62bb7fe700 1 -- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b3d80 msgr2=0x7f62bc16aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62bb7fe700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b3d80 0x7f62bc16aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62bb7fe700 1 -- 172.21.15.3:0/3014331873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac009420 con 0x7f62bc15aa30 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62bb7fe700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc15aa30 0x7f62bc0c7490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f62b400c8e0 tx=0x7f62b4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62b2ffd700 1 -- 172.21.15.3:0/3014331873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400ce80 con 0x7f62bc15aa30 2023-10-06T04:54:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62b2ffd700 1 -- 172.21.15.3:0/3014331873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b400a7e0 con 0x7f62bc15aa30 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc0c97a0 con 0x7f62bc15aa30 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62b2ffd700 1 -- 172.21.15.3:0/3014331873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400ab80 con 0x7f62bc15aa30 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.399+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc0c9d60 con 0x7f62bc15aa30 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.400+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc0ca280 con 0x7f62bc15aa30 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.400+0000 7f62b2ffd700 1 -- 172.21.15.3:0/3014331873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62b4012420 con 0x7f62bc15aa30 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.401+0000 7f62b2ffd700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f629c066f30 0x7f629c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.401+0000 7f62b2ffd700 1 -- 172.21.15.3:0/3014331873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f62b4084980 con 0x7f62bc15aa30 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.401+0000 7f62baffd700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f629c066f30 0x7f629c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.401+0000 7f62baffd700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f629c066f30 0x7f629c0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0091e0 tx=0x7f62ac016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.405+0000 7f62b2ffd700 1 -- 172.21.15.3:0/3014331873 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62bc0ca280 con 0x7f62bc15aa30 2023-10-06T04:54:46.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.675+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f62bc0644c0 con 0x7f62bc15aa30 2023-10-06T04:54:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.675+0000 7f62b2ffd700 1 -- 172.21.15.3:0/3014331873 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1456 (secure 0 0 0) 0x7f62b4055520 con 0x7f62bc15aa30 2023-10-06T04:54:46.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:46.677 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":166,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1696567930,"num_in_osds":8,"osd_in_since":1696567922,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2483802112,"bytes_avail":770610311168,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":31,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5721}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-06T04:54:29.454591+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T04:54:46.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f629c066f30 msgr2=0x7f629c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f629c066f30 0x7f629c0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0091e0 tx=0x7f62ac016000 comp rx=0 tx=0).stop 2023-10-06T04:54:46.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc15aa30 msgr2=0x7f62bc0c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc15aa30 0x7f62bc0c7490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f62b400c8e0 tx=0x7f62b4001710 comp rx=0 tx=0).stop 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 shutdown_connections 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f629c066f30 0x7f629c0693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc15aa30 0x7f62bc0c7490 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62bc0b39b0 0x7f62bc0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 --2- 172.21.15.3:0/3014331873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62bc0b3d80 0x7f62bc16aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 >> 172.21.15.3:0/3014331873 conn(0x7f62bc067ac0 msgr2=0x7f62bc15b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 shutdown_connections 2023-10-06T04:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.677+0000 7f62c1701700 1 -- 172.21.15.3:0/3014331873 wait complete. 2023-10-06T04:54:46.691 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T04:54:46.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:54:46.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.809+0000 7f1554b12700 1 Processor -- start 2023-10-06T04:54:46.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.809+0000 7f1554b12700 1 -- start start 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.809+0000 7f1554b12700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f1554b12700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1550159080 0x7f1550159500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f1554b12700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f155015e0d0 con 0x7f15500b2900 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f1554b12700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15500b0e00 con 0x7f1550159080 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f1554b12700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15500cbd70 con 0x7f15500b2530 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1550159080 0x7f1550159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1550159080 0x7f1550159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45006/0 (socket says 172.21.15.3:45006) 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 -- 172.21.15.3:0/581865355 learned_addr learned my addr 172.21.15.3:0/581865355 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154e59c700 1 --2- 172.21.15.3:0/581865355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 -- 172.21.15.3:0/581865355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 msgr2=0x7f15500b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 --2- 172.21.15.3:0/581865355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 -- 172.21.15.3:0/581865355 >> v1:172.21.15.153:6789/0 conn(0x7f15500b2900 legacy=0x7f155015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 -- 172.21.15.3:0/581865355 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500cb160 con 0x7f1550159080 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154ed9d700 1 --2- 172.21.15.3:0/581865355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550159080 0x7f1550159500 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1540009f20 tx=0x7f1540009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbfeb58e4ebf53d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154d59a700 1 -- 172.21.15.3:0/581865355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154000e070 con 0x7f1550159080 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154d59a700 1 -- 172.21.15.3:0/581865355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154000ba80 con 0x7f1550159080 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.810+0000 7f154d59a700 1 -- 172.21.15.3:0/581865355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400092e0 con 0x7f1550159080 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 -- 172.21.15.3:0/581865355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550159080 msgr2=0x7f1550159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 --2- 172.21.15.3:0/581865355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550159080 0x7f1550159500 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1540009f20 tx=0x7f1540009840 comp rx=0 tx=0).stop 2023-10-06T04:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 -- 172.21.15.3:0/581865355 shutdown_connections 2023-10-06T04:54:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 --2- 172.21.15.3:0/581865355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550159080 0x7f1550159500 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 --2- 172.21.15.3:0/581865355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 -- 172.21.15.3:0/581865355 >> 172.21.15.3:0/581865355 conn(0x7f1550070310 msgr2=0x7f1550070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 -- 172.21.15.3:0/581865355 shutdown_connections 2023-10-06T04:54:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.811+0000 7f1554b12700 1 -- 172.21.15.3:0/581865355 wait complete. 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.812+0000 7f1554b12700 1 Processor -- start 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.812+0000 7f1554b12700 1 -- start start 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f1554b12700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500d0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f1554b12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15500b2900 0x7f155016a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f1554b12700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550159080 0x7f15500c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f1554b12700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155015af30 con 0x7f15500b2900 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f1554b12700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155015b180 con 0x7f1550159080 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f1554b12700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155015b460 con 0x7f15500b2530 2023-10-06T04:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15500b2900 0x7f155016a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15500b2900 0x7f155016a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45016/0 (socket says 172.21.15.3:45016) 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 -- 172.21.15.3:0/463927772 learned_addr learned my addr 172.21.15.3:0/463927772 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154ed9d700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550159080 0x7f15500c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154e59c700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 -- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 msgr2=0x7f15500d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 -- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550159080 msgr2=0x7f15500c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550159080 0x7f15500c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 -- 172.21.15.3:0/463927772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540009ae0 con 0x7f15500b2900 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f154dd9b700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15500b2900 0x7f155016a2c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f153800c590 tx=0x7f153800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.813+0000 7f15467fc700 1 -- 172.21.15.3:0/463927772 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538011590 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.814+0000 7f15467fc700 1 -- 172.21.15.3:0/463927772 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15380116f0 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.814+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155015b740 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.814+0000 7f15467fc700 1 -- 172.21.15.3:0/463927772 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538011ab0 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.814+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1550154f30 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.815+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1550071680 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.815+0000 7f15467fc700 1 -- 172.21.15.3:0/463927772 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f153801b3b0 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.815+0000 7f15467fc700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1530066f30 0x7f15300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.816+0000 7f15467fc700 1 -- 172.21.15.3:0/463927772 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1538083a00 con 0x7f15500b2900 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.816+0000 7f154e59c700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1530066f30 0x7f15300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.816+0000 7f154e59c700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1530066f30 0x7f15300693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1548006fd0 tx=0x7f1548008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:46.820+0000 7f15467fc700 1 -- 172.21.15.3:0/463927772 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1538050b10 con 0x7f15500b2900 2023-10-06T04:54:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.011+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15500d0df0 con 0x7f1530066f30 2023-10-06T04:54:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.020+0000 7f15467fc700 1 -- 172.21.15.3:0/463927772 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308672 (secure 0 0 0) 0x7f154800f710 con 0x7f1530066f30 2023-10-06T04:54:47.025 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1530066f30 msgr2=0x7f15300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1530066f30 0x7f15300693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1548006fd0 tx=0x7f1548008040 comp rx=0 tx=0).stop 2023-10-06T04:54:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15500b2900 msgr2=0x7f155016a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15500b2900 0x7f155016a2c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f153800c590 tx=0x7f153800c950 comp rx=0 tx=0).stop 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 shutdown_connections 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1530066f30 0x7f15300693e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550159080 0x7f15500c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15500b2530 0x7f15500d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 --2- 172.21.15.3:0/463927772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15500b2900 0x7f155016a2c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 >> 172.21.15.3:0/463927772 conn(0x7f1550070310 msgr2=0x7f15500cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 shutdown_connections 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:47.023+0000 7f1554b12700 1 -- 172.21.15.3:0/463927772 wait complete. 2023-10-06T04:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:54:47.037 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-10-06T04:54:45.458470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425588,"kb_used_data":6324,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752549132,"statfs":{"total":773094113280,"available":770610311168,"internally_reserved":0,"allocated":6475776,"data_stored":3465669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001559"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333786+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331685+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.335105+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329702+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:55.591082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330610+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327617+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:01.768927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330876+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329686+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328812+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:33.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332622+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:54.642506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331759+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333352+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328323+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328695+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:29.292684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327879+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:14.188545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331666+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328892+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330778+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:47.801935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325842+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332133+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326112+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326304+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328233+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331026+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331474+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329585+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332529+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329500+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331609+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332943+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:05.163312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331203+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.065580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332535+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:26.889286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339794+0000","last_change":"2023-10-06T04:54:40.339794+0000","last_active":"2023-10-06T04:54:40.339794+0000","last_peered":"2023-10-06T04:54:40.339794+0000","last_clean":"2023-10-06T04:54:40.339794+0000","last_became_active":"2023-10-06T04:54:40.339482+0000","last_became_peered":"2023-10-06T04:54:40.339482+0000","last_unstale":"2023-10-06T04:54:40.339794+0000","last_undegraded":"2023-10-06T04:54:40.339794+0000","last_fullsized":"2023-10-06T04:54:40.339794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328444+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:42.745683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340091+0000","last_change":"2023-10-06T04:54:40.340091+0000","last_active":"2023-10-06T04:54:40.340091+0000","last_peered":"2023-10-06T04:54:40.340091+0000","last_clean":"2023-10-06T04:54:40.340091+0000","last_became_active":"2023-10-06T04:54:40.339729+0000","last_became_peered":"2023-10-06T04:54:40.339729+0000","last_unstale":"2023-10-06T04:54:40.340091+0000","last_undegraded":"2023-10-06T04:54:40.340091+0000","last_fullsized":"2023-10-06T04:54:40.340091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330525+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:05.603150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344126+0000","last_change":"2023-10-06T04:54:40.344126+0000","last_active":"2023-10-06T04:54:40.344126+0000","last_peered":"2023-10-06T04:54:40.344126+0000","last_clean":"2023-10-06T04:54:40.344126+0000","last_became_active":"2023-10-06T04:54:40.343824+0000","last_became_peered":"2023-10-06T04:54:40.343824+0000","last_unstale":"2023-10-06T04:54:40.344126+0000","last_undegraded":"2023-10-06T04:54:40.344126+0000","last_fullsized":"2023-10-06T04:54:40.344126+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:32.239178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329189+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:33.176469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338831+0000","last_change":"2023-10-06T04:54:40.338831+0000","last_active":"2023-10-06T04:54:40.338831+0000","last_peered":"2023-10-06T04:54:40.338831+0000","last_clean":"2023-10-06T04:54:40.338831+0000","last_became_active":"2023-10-06T04:54:40.338570+0000","last_became_peered":"2023-10-06T04:54:40.338570+0000","last_unstale":"2023-10-06T04:54:40.338831+0000","last_undegraded":"2023-10-06T04:54:40.338831+0000","last_fullsized":"2023-10-06T04:54:40.338831+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:32.170845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327374+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:43.786031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.332965+0000","last_change":"2023-10-06T04:54:40.332965+0000","last_active":"2023-10-06T04:54:40.332965+0000","last_peered":"2023-10-06T04:54:40.332965+0000","last_clean":"2023-10-06T04:54:40.332965+0000","last_became_active":"2023-10-06T04:54:40.332661+0000","last_became_peered":"2023-10-06T04:54:40.332661+0000","last_unstale":"2023-10-06T04:54:40.332965+0000","last_undegraded":"2023-10-06T04:54:40.332965+0000","last_fullsized":"2023-10-06T04:54:40.332965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.328723+0000","last_change":"2023-10-06T04:54:40.328723+0000","last_active":"2023-10-06T04:54:40.328723+0000","last_peered":"2023-10-06T04:54:40.328723+0000","last_clean":"2023-10-06T04:54:40.328723+0000","last_became_active":"2023-10-06T04:54:40.328424+0000","last_became_peered":"2023-10-06T04:54:40.328424+0000","last_unstale":"2023-10-06T04:54:40.328723+0000","last_undegraded":"2023-10-06T04:54:40.328723+0000","last_fullsized":"2023-10-06T04:54:40.328723+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334363+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:37.872096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344255+0000","last_change":"2023-10-06T04:54:40.344255+0000","last_active":"2023-10-06T04:54:40.344255+0000","last_peered":"2023-10-06T04:54:40.344255+0000","last_clean":"2023-10-06T04:54:40.344255+0000","last_became_active":"2023-10-06T04:54:40.343918+0000","last_became_peered":"2023-10-06T04:54:40.343918+0000","last_unstale":"2023-10-06T04:54:40.344255+0000","last_undegraded":"2023-10-06T04:54:40.344255+0000","last_fullsized":"2023-10-06T04:54:40.344255+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.326195+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.338640+0000","last_change":"2023-10-06T04:54:40.338640+0000","last_active":"2023-10-06T04:54:40.338640+0000","last_peered":"2023-10-06T04:54:40.338640+0000","last_clean":"2023-10-06T04:54:40.338640+0000","last_became_active":"2023-10-06T04:54:40.338385+0000","last_became_peered":"2023-10-06T04:54:40.338385+0000","last_unstale":"2023-10-06T04:54:40.338640+0000","last_undegraded":"2023-10-06T04:54:40.338640+0000","last_fullsized":"2023-10-06T04:54:40.338640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:40.835360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332028+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340720+0000","last_change":"2023-10-06T04:54:40.340720+0000","last_active":"2023-10-06T04:54:40.340720+0000","last_peered":"2023-10-06T04:54:40.340720+0000","last_clean":"2023-10-06T04:54:40.340720+0000","last_became_active":"2023-10-06T04:54:40.340447+0000","last_became_peered":"2023-10-06T04:54:40.340447+0000","last_unstale":"2023-10-06T04:54:40.340720+0000","last_undegraded":"2023-10-06T04:54:40.340720+0000","last_fullsized":"2023-10-06T04:54:40.340720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:38.363715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.325883+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339697+0000","last_change":"2023-10-06T04:54:40.339697+0000","last_active":"2023-10-06T04:54:40.339697+0000","last_peered":"2023-10-06T04:54:40.339697+0000","last_clean":"2023-10-06T04:54:40.339697+0000","last_became_active":"2023-10-06T04:54:40.339299+0000","last_became_peered":"2023-10-06T04:54:40.339299+0000","last_unstale":"2023-10-06T04:54:40.339697+0000","last_undegraded":"2023-10-06T04:54:40.339697+0000","last_fullsized":"2023-10-06T04:54:40.339697+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:12.235281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334708+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:36.233559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.339769+0000","last_change":"2023-10-06T04:54:40.339769+0000","last_active":"2023-10-06T04:54:40.339769+0000","last_peered":"2023-10-06T04:54:40.339769+0000","last_clean":"2023-10-06T04:54:40.339769+0000","last_became_active":"2023-10-06T04:54:40.339511+0000","last_became_peered":"2023-10-06T04:54:40.339511+0000","last_unstale":"2023-10-06T04:54:40.339769+0000","last_undegraded":"2023-10-06T04:54:40.339769+0000","last_fullsized":"2023-10-06T04:54:40.339769+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:25.589985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328269+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:06.528366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341385+0000","last_change":"2023-10-06T04:54:40.341385+0000","last_active":"2023-10-06T04:54:40.341385+0000","last_peered":"2023-10-06T04:54:40.341385+0000","last_clean":"2023-10-06T04:54:40.341385+0000","last_became_active":"2023-10-06T04:54:40.341078+0000","last_became_peered":"2023-10-06T04:54:40.341078+0000","last_unstale":"2023-10-06T04:54:40.341385+0000","last_undegraded":"2023-10-06T04:54:40.341385+0000","last_fullsized":"2023-10-06T04:54:40.341385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328102+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:47.251817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344621+0000","last_change":"2023-10-06T04:54:40.344621+0000","last_active":"2023-10-06T04:54:40.344621+0000","last_peered":"2023-10-06T04:54:40.344621+0000","last_clean":"2023-10-06T04:54:40.344621+0000","last_became_active":"2023-10-06T04:54:40.344347+0000","last_became_peered":"2023-10-06T04:54:40.344347+0000","last_unstale":"2023-10-06T04:54:40.344621+0000","last_undegraded":"2023-10-06T04:54:40.344621+0000","last_fullsized":"2023-10-06T04:54:40.344621+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:41.740059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330179+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:33.483941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342909+0000","last_change":"2023-10-06T04:54:40.342909+0000","last_active":"2023-10-06T04:54:40.342909+0000","last_peered":"2023-10-06T04:54:40.342909+0000","last_clean":"2023-10-06T04:54:40.342909+0000","last_became_active":"2023-10-06T04:54:40.342316+0000","last_became_peered":"2023-10-06T04:54:40.342316+0000","last_unstale":"2023-10-06T04:54:40.342909+0000","last_undegraded":"2023-10-06T04:54:40.342909+0000","last_fullsized":"2023-10-06T04:54:40.342909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333878+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:04.598076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341225+0000","last_change":"2023-10-06T04:54:40.341225+0000","last_active":"2023-10-06T04:54:40.341225+0000","last_peered":"2023-10-06T04:54:40.341225+0000","last_clean":"2023-10-06T04:54:40.341225+0000","last_became_active":"2023-10-06T04:54:40.340018+0000","last_became_peered":"2023-10-06T04:54:40.340018+0000","last_unstale":"2023-10-06T04:54:40.341225+0000","last_undegraded":"2023-10-06T04:54:40.341225+0000","last_fullsized":"2023-10-06T04:54:40.341225+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331646+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.345606+0000","last_change":"2023-10-06T04:54:40.345606+0000","last_active":"2023-10-06T04:54:40.345606+0000","last_peered":"2023-10-06T04:54:40.345606+0000","last_clean":"2023-10-06T04:54:40.345606+0000","last_became_active":"2023-10-06T04:54:40.345364+0000","last_became_peered":"2023-10-06T04:54:40.345364+0000","last_unstale":"2023-10-06T04:54:40.345606+0000","last_undegraded":"2023-10-06T04:54:40.345606+0000","last_fullsized":"2023-10-06T04:54:40.345606+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:32.478317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330958+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:35.749980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340969+0000","last_change":"2023-10-06T04:54:40.340969+0000","last_active":"2023-10-06T04:54:40.340969+0000","last_peered":"2023-10-06T04:54:40.340969+0000","last_clean":"2023-10-06T04:54:40.340969+0000","last_became_active":"2023-10-06T04:54:40.340716+0000","last_became_peered":"2023-10-06T04:54:40.340716+0000","last_unstale":"2023-10-06T04:54:40.340969+0000","last_undegraded":"2023-10-06T04:54:40.340969+0000","last_fullsized":"2023-10-06T04:54:40.340969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327245+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:40.375936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342698+0000","last_change":"2023-10-06T04:54:40.342698+0000","last_active":"2023-10-06T04:54:40.342698+0000","last_peered":"2023-10-06T04:54:40.342698+0000","last_clean":"2023-10-06T04:54:40.342698+0000","last_became_active":"2023-10-06T04:54:40.342336+0000","last_became_peered":"2023-10-06T04:54:40.342336+0000","last_unstale":"2023-10-06T04:54:40.342698+0000","last_undegraded":"2023-10-06T04:54:40.342698+0000","last_fullsized":"2023-10-06T04:54:40.342698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:35.638003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330725+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:53.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343535+0000","last_change":"2023-10-06T04:54:40.343535+0000","last_active":"2023-10-06T04:54:40.343535+0000","last_peered":"2023-10-06T04:54:40.343535+0000","last_clean":"2023-10-06T04:54:40.343535+0000","last_became_active":"2023-10-06T04:54:40.343213+0000","last_became_peered":"2023-10-06T04:54:40.343213+0000","last_unstale":"2023-10-06T04:54:40.343535+0000","last_undegraded":"2023-10-06T04:54:40.343535+0000","last_fullsized":"2023-10-06T04:54:40.343535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:24.202745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333947+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:28.396681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344767+0000","last_change":"2023-10-06T04:54:40.344767+0000","last_active":"2023-10-06T04:54:40.344767+0000","last_peered":"2023-10-06T04:54:40.344767+0000","last_clean":"2023-10-06T04:54:40.344767+0000","last_became_active":"2023-10-06T04:54:40.344507+0000","last_became_peered":"2023-10-06T04:54:40.344507+0000","last_unstale":"2023-10-06T04:54:40.344767+0000","last_undegraded":"2023-10-06T04:54:40.344767+0000","last_fullsized":"2023-10-06T04:54:40.344767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329365+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:18.138617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.341127+0000","last_change":"2023-10-06T04:54:40.341127+0000","last_active":"2023-10-06T04:54:40.341127+0000","last_peered":"2023-10-06T04:54:40.341127+0000","last_clean":"2023-10-06T04:54:40.341127+0000","last_became_active":"2023-10-06T04:54:40.339641+0000","last_became_peered":"2023-10-06T04:54:40.339641+0000","last_unstale":"2023-10-06T04:54:40.341127+0000","last_undegraded":"2023-10-06T04:54:40.341127+0000","last_fullsized":"2023-10-06T04:54:40.341127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.636882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328810+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.369339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.333730+0000","last_change":"2023-10-06T04:54:40.333730+0000","last_active":"2023-10-06T04:54:40.333730+0000","last_peered":"2023-10-06T04:54:40.333730+0000","last_clean":"2023-10-06T04:54:40.333730+0000","last_became_active":"2023-10-06T04:54:40.333468+0000","last_became_peered":"2023-10-06T04:54:40.333468+0000","last_unstale":"2023-10-06T04:54:40.333730+0000","last_undegraded":"2023-10-06T04:54:40.333730+0000","last_fullsized":"2023-10-06T04:54:40.333730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:34.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.332948+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344570+0000","last_change":"2023-10-06T04:54:40.344570+0000","last_active":"2023-10-06T04:54:40.344570+0000","last_peered":"2023-10-06T04:54:40.344570+0000","last_clean":"2023-10-06T04:54:40.344570+0000","last_became_active":"2023-10-06T04:54:40.344295+0000","last_became_peered":"2023-10-06T04:54:40.344295+0000","last_unstale":"2023-10-06T04:54:40.344570+0000","last_undegraded":"2023-10-06T04:54:40.344570+0000","last_fullsized":"2023-10-06T04:54:40.344570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:40.208274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.327281+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:47.664286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344877+0000","last_change":"2023-10-06T04:54:40.344877+0000","last_active":"2023-10-06T04:54:40.344877+0000","last_peered":"2023-10-06T04:54:40.344877+0000","last_clean":"2023-10-06T04:54:40.344877+0000","last_became_active":"2023-10-06T04:54:40.344580+0000","last_became_peered":"2023-10-06T04:54:40.344580+0000","last_unstale":"2023-10-06T04:54:40.344877+0000","last_undegraded":"2023-10-06T04:54:40.344877+0000","last_fullsized":"2023-10-06T04:54:40.344877+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.328360+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340640+0000","last_change":"2023-10-06T04:54:40.340640+0000","last_active":"2023-10-06T04:54:40.340640+0000","last_peered":"2023-10-06T04:54:40.340640+0000","last_clean":"2023-10-06T04:54:40.340640+0000","last_became_active":"2023-10-06T04:54:40.340384+0000","last_became_peered":"2023-10-06T04:54:40.340384+0000","last_unstale":"2023-10-06T04:54:40.340640+0000","last_undegraded":"2023-10-06T04:54:40.340640+0000","last_fullsized":"2023-10-06T04:54:40.340640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.769263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323560+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:54:39.323560+0000","last_peered":"2023-10-06T04:54:39.323560+0000","last_clean":"2023-10-06T04:54:39.323560+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:54:39.323560+0000","last_undegraded":"2023-10-06T04:54:39.323560+0000","last_fullsized":"2023-10-06T04:54:39.323560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.334410+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:15.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343623+0000","last_change":"2023-10-06T04:54:40.343623+0000","last_active":"2023-10-06T04:54:40.343623+0000","last_peered":"2023-10-06T04:54:40.343623+0000","last_clean":"2023-10-06T04:54:40.343623+0000","last_became_active":"2023-10-06T04:54:40.343366+0000","last_became_peered":"2023-10-06T04:54:40.343366+0000","last_unstale":"2023-10-06T04:54:40.343623+0000","last_undegraded":"2023-10-06T04:54:40.343623+0000","last_fullsized":"2023-10-06T04:54:40.343623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:33.074026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.325164+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:54:39.325164+0000","last_peered":"2023-10-06T04:54:39.325164+0000","last_clean":"2023-10-06T04:54:39.325164+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:54:39.325164+0000","last_undegraded":"2023-10-06T04:54:39.325164+0000","last_fullsized":"2023-10-06T04:54:39.325164+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330526+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:04.376901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.344112+0000","last_change":"2023-10-06T04:54:40.344112+0000","last_active":"2023-10-06T04:54:40.344112+0000","last_peered":"2023-10-06T04:54:40.344112+0000","last_clean":"2023-10-06T04:54:40.344112+0000","last_became_active":"2023-10-06T04:54:40.343905+0000","last_became_peered":"2023-10-06T04:54:40.343905+0000","last_unstale":"2023-10-06T04:54:40.344112+0000","last_undegraded":"2023-10-06T04:54:40.344112+0000","last_fullsized":"2023-10-06T04:54:40.344112+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.324421+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:54:39.324421+0000","last_peered":"2023-10-06T04:54:39.324421+0000","last_clean":"2023-10-06T04:54:39.324421+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:54:39.324421+0000","last_undegraded":"2023-10-06T04:54:39.324421+0000","last_fullsized":"2023-10-06T04:54:39.324421+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":190,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321139+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:54:39.321139+0000","last_peered":"2023-10-06T04:54:39.321139+0000","last_clean":"2023-10-06T04:54:39.321139+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:54:39.321139+0000","last_undegraded":"2023-10-06T04:54:39.321139+0000","last_fullsized":"2023-10-06T04:54:39.321139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329601+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343032+0000","last_change":"2023-10-06T04:54:40.343032+0000","last_active":"2023-10-06T04:54:40.343032+0000","last_peered":"2023-10-06T04:54:40.343032+0000","last_clean":"2023-10-06T04:54:40.343032+0000","last_became_active":"2023-10-06T04:54:40.342591+0000","last_became_peered":"2023-10-06T04:54:40.342591+0000","last_unstale":"2023-10-06T04:54:40.343032+0000","last_undegraded":"2023-10-06T04:54:40.343032+0000","last_fullsized":"2023-10-06T04:54:40.343032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321070+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:54:39.321070+0000","last_peered":"2023-10-06T04:54:39.321070+0000","last_clean":"2023-10-06T04:54:39.321070+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:54:39.321070+0000","last_undegraded":"2023-10-06T04:54:39.321070+0000","last_fullsized":"2023-10-06T04:54:39.321070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.329145+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:46.287413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.334344+0000","last_change":"2023-10-06T04:54:40.334344+0000","last_active":"2023-10-06T04:54:40.334344+0000","last_peered":"2023-10-06T04:54:40.334344+0000","last_clean":"2023-10-06T04:54:40.334344+0000","last_became_active":"2023-10-06T04:54:40.334079+0000","last_became_peered":"2023-10-06T04:54:40.334079+0000","last_unstale":"2023-10-06T04:54:40.334344+0000","last_undegraded":"2023-10-06T04:54:40.334344+0000","last_fullsized":"2023-10-06T04:54:40.334344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:28.287060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.321036+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:54:39.321036+0000","last_peered":"2023-10-06T04:54:39.321036+0000","last_clean":"2023-10-06T04:54:39.321036+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:54:39.321036+0000","last_undegraded":"2023-10-06T04:54:39.321036+0000","last_fullsized":"2023-10-06T04:54:39.321036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.333407+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:10.632704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.340001+0000","last_change":"2023-10-06T04:54:40.340001+0000","last_active":"2023-10-06T04:54:40.340001+0000","last_peered":"2023-10-06T04:54:40.340001+0000","last_clean":"2023-10-06T04:54:40.340001+0000","last_became_active":"2023-10-06T04:54:40.339665+0000","last_became_peered":"2023-10-06T04:54:40.339665+0000","last_unstale":"2023-10-06T04:54:40.340001+0000","last_undegraded":"2023-10-06T04:54:40.340001+0000","last_fullsized":"2023-10-06T04:54:40.340001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:07.045545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.328478+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:54:39.328478+0000","last_peered":"2023-10-06T04:54:39.328478+0000","last_clean":"2023-10-06T04:54:39.328478+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:54:39.328478+0000","last_undegraded":"2023-10-06T04:54:39.328478+0000","last_fullsized":"2023-10-06T04:54:39.328478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.330328+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:56.766026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.343461+0000","last_change":"2023-10-06T04:54:40.343461+0000","last_active":"2023-10-06T04:54:40.343461+0000","last_peered":"2023-10-06T04:54:40.343461+0000","last_clean":"2023-10-06T04:54:40.343461+0000","last_became_active":"2023-10-06T04:54:40.343184+0000","last_became_peered":"2023-10-06T04:54:40.343184+0000","last_unstale":"2023-10-06T04:54:40.343461+0000","last_undegraded":"2023-10-06T04:54:40.343461+0000","last_fullsized":"2023-10-06T04:54:40.343461+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:47.709557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323296+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:54:39.323296+0000","last_peered":"2023-10-06T04:54:39.323296+0000","last_clean":"2023-10-06T04:54:39.323296+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:54:39.323296+0000","last_undegraded":"2023-10-06T04:54:39.323296+0000","last_fullsized":"2023-10-06T04:54:39.323296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:54:40.317895+0000","last_change":"2023-10-06T04:54:40.331007+0000","last_active":"2023-10-06T04:54:40.317895+0000","last_peered":"2023-10-06T04:54:40.317895+0000","last_clean":"2023-10-06T04:54:40.317895+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:54:40.317895+0000","last_undegraded":"2023-10-06T04:54:40.317895+0000","last_fullsized":"2023-10-06T04:54:40.317895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T04:54:40.342683+0000","last_change":"2023-10-06T04:54:40.342683+0000","last_active":"2023-10-06T04:54:40.342683+0000","last_peered":"2023-10-06T04:54:40.342683+0000","last_clean":"2023-10-06T04:54:40.342683+0000","last_became_active":"2023-10-06T04:54:40.342261+0000","last_became_peered":"2023-10-06T04:54:40.342261+0000","last_unstale":"2023-10-06T04:54:40.342683+0000","last_undegraded":"2023-10-06T04:54:40.342683+0000","last_fullsized":"2023-10-06T04:54:40.342683+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T04:54:39.323303+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:54:39.323303+0000","last_peered":"2023-10-06T04:54:39.323303+0000","last_clean":"2023-10-06T04:54:39.323303+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:54:39.323303+0000","last_undegraded":"2023-10-06T04:54:39.323303+0000","last_fullsized":"2023-10-06T04:54:39.323303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306700,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065140,"statfs":{"total":96636764160,"available":96322703360,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.751,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.66400000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.80200000000000005,"15min":0.84499999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70099999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":1.111,"15min":1.111},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.874,"15min":0.874},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77500000000000002,"5min":1.214,"15min":1.214},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72599999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.751,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63300000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302604,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069236,"statfs":{"total":96636764160,"available":96326897664,"internally_reserved":0,"allocated":1249280,"data_stored":873487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.48399999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51000000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.379,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.42399999999999999,"15min":0.42899999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54100000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.376,"5min":0.42799999999999999,"15min":0.437},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57499999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.63400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73699999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":1.026,"15min":1.026},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.46200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.38},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36499999999999999,"15min":0.36599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64400000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.747,"5min":1.048,"15min":1.048},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.22700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.57999999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.24399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68700000000000006,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85399999999999998,"5min":1.246,"15min":1.246},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.40699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.877,"15min":0.877},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306136,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065704,"statfs":{"total":96636764160,"available":96323280896,"internally_reserved":0,"allocated":671744,"data_stored":291080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44500000000000001,"5min":1.415,"15min":1.415},"last":0.25},{"interface":"front","average":{"1min":0.37,"5min":0.58599999999999997,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51400000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.44600000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.45700000000000002,"15min":0.47099999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53600000000000003,"5min":1.482,"15min":1.482},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":5.069,"15min":5.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57799999999999996,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.71299999999999997,"15min":0.73799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94799999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80200000000000005,"15min":0.83999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.504}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.77300000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":1.452,"15min":1.452},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.81399999999999995,"15min":0.85399999999999998},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":6.367,"15min":6.367},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":5,"15min":5},"last":0.84199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069812,"statfs":{"total":96636764160,"available":96327487488,"internally_reserved":0,"allocated":659456,"data_stored":284472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.81999999999999995,"15min":0.876},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63200000000000001,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.85299999999999998,"15min":0.90300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67900000000000005,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.503},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.85799999999999998,"15min":0.91400000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.68400000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64300000000000002,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.75},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.70299999999999996,"15min":0.73899999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61599999999999999,"5min":5.415,"15min":5.415},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67200000000000004,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66200000000000003,"5min":4.202,"15min":4.202},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74099999999999999,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":0.872},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069816,"statfs":{"total":96636764160,"available":96327491584,"internally_reserved":0,"allocated":655360,"data_stored":283119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70699999999999996,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63700000000000001,"15min":0.65500000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72399999999999998,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.69299999999999995,"15min":0.71599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.746,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.44}]},{"osd":3,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.0049999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.499},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.008,"15min":1.0760000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.751},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76500000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:54:50.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:54:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.175+0000 7f6872f7b700 1 Processor -- start 2023-10-06T04:54:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.176+0000 7f6872f7b700 1 -- start start 2023-10-06T04:54:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.176+0000 7f6872f7b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f686c155980 0x7f686c06aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.176+0000 7f6872f7b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f686c06ec40 0x7f686c157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.176+0000 7f6872f7b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c069790 con 0x7f686c156a30 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.176+0000 7f6872f7b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c0ae9e0 con 0x7f686c06ec40 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.176+0000 7f6872f7b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c0699e0 con 0x7f686c155980 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f686c06ec40 0x7f686c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f686c06ec40 0x7f686c157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45022/0 (socket says 172.21.15.3:45022) 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 -- 172.21.15.3:0/991823795 learned_addr learned my addr 172.21.15.3:0/991823795 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6870d17700 1 --2- 172.21.15.3:0/991823795 >> v2:172.21.15.153:3300/0 conn(0x7f686c155980 0x7f686c06aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 -- 172.21.15.3:0/991823795 >> v2:172.21.15.153:3300/0 conn(0x7f686c155980 msgr2=0x7f686c06aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 --2- 172.21.15.3:0/991823795 >> v2:172.21.15.153:3300/0 conn(0x7f686c155980 0x7f686c06aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 -- 172.21.15.3:0/991823795 >> v1:172.21.15.153:6789/0 conn(0x7f686c156a30 legacy=0x7f686c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 -- 172.21.15.3:0/991823795 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c069cc0 con 0x7f686c06ec40 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f6871518700 1 --2- 172.21.15.3:0/991823795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c157500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f686c06fba0 tx=0x7f6864009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33bfff263548d2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f686b7fe700 1 -- 172.21.15.3:0/991823795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864009e70 con 0x7f686c06ec40 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f686b7fe700 1 -- 172.21.15.3:0/991823795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686400b860 con 0x7f686c06ec40 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.177+0000 7f686b7fe700 1 -- 172.21.15.3:0/991823795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400bc00 con 0x7f686c06ec40 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 -- 172.21.15.3:0/991823795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 msgr2=0x7f686c157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 --2- 172.21.15.3:0/991823795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c157500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f686c06fba0 tx=0x7f6864009250 comp rx=0 tx=0).stop 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 -- 172.21.15.3:0/991823795 shutdown_connections 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 --2- 172.21.15.3:0/991823795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c157500 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 --2- 172.21.15.3:0/991823795 >> v2:172.21.15.153:3300/0 conn(0x7f686c155980 0x7f686c06aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 -- 172.21.15.3:0/991823795 >> 172.21.15.3:0/991823795 conn(0x7f686c06c370 msgr2=0x7f686c06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 -- 172.21.15.3:0/991823795 shutdown_connections 2023-10-06T04:54:50.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.178+0000 7f6872f7b700 1 -- 172.21.15.3:0/991823795 wait complete. 2023-10-06T04:54:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.179+0000 7f6872f7b700 1 Processor -- start 2023-10-06T04:54:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.179+0000 7f6872f7b700 1 -- start start 2023-10-06T04:54:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6872f7b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c0c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6872f7b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f686c155980 0x7f686c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6872f7b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f686c156a30 0x7f686c1472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6872f7b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0cb7f0 con 0x7f686c06ec40 2023-10-06T04:54:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6872f7b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0cba90 con 0x7f686c155980 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6872f7b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0cbd70 con 0x7f686c156a30 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c0c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c0c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45038/0 (socket says 172.21.15.3:45038) 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 -- 172.21.15.3:0/3490997454 learned_addr learned my addr 172.21.15.3:0/3490997454 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f686bfff700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f686c155980 0x7f686c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6871518700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f686c156a30 0x7f686c1472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 -- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f686c156a30 msgr2=0x7f686c1472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f686c156a30 0x7f686c1472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 -- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f686c155980 msgr2=0x7f686c0c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f686c155980 0x7f686c0c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.180+0000 7f6870d17700 1 -- 172.21.15.3:0/3490997454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640094f0 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.181+0000 7f6870d17700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c0c5490 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f685400ec80 tx=0x7f68540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.181+0000 7f6868ff9700 1 -- 172.21.15.3:0/3490997454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400c850 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.181+0000 7f6868ff9700 1 -- 172.21.15.3:0/3490997454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685400c9b0 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.181+0000 7f6868ff9700 1 -- 172.21.15.3:0/3490997454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400cd50 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.181+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c0cc050 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.181+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c0cc610 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.182+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c0cc980 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.182+0000 7f6868ff9700 1 -- 172.21.15.3:0/3490997454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6854013680 con 0x7f686c06ec40 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.183+0000 7f6868ff9700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.183+0000 7f6868ff9700 1 -- 172.21.15.3:0/3490997454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6854084960 con 0x7f686c06ec40 2023-10-06T04:54:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.183+0000 7f686bfff700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.184+0000 7f686bfff700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f685c007fd0 tx=0x7f685c001650 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.187+0000 7f6868ff9700 1 -- 172.21.15.3:0/3490997454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68540519d0 con 0x7f686c06ec40 2023-10-06T04:54:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.378+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f686c071680 con 0x7f684c066f30 2023-10-06T04:54:50.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.388+0000 7f6868ff9700 1 -- 172.21.15.3:0/3490997454 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311553 (secure 0 0 0) 0x7f686c071680 con 0x7f684c066f30 2023-10-06T04:54:50.393 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 msgr2=0x7f684c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f685c007fd0 tx=0x7f685c001650 comp rx=0 tx=0).stop 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 msgr2=0x7f686c0c5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c0c5490 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f685400ec80 tx=0x7f68540091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 shutdown_connections 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f686c156a30 0x7f686c1472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c06ec40 0x7f686c0c5490 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 --2- 172.21.15.3:0/3490997454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f686c155980 0x7f686c0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 >> 172.21.15.3:0/3490997454 conn(0x7f686c06c370 msgr2=0x7f686c0c8200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 shutdown_connections 2023-10-06T04:54:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.391+0000 7f6872f7b700 1 -- 172.21.15.3:0/3490997454 wait complete. 2023-10-06T04:54:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:54:50.405 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2023-10-06T04:54:49.459525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425916,"kb_used_data":6652,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752548804,"statfs":{"total":773094113280,"available":770609975296,"internally_reserved":0,"allocated":6811648,"data_stored":3619985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.134255"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.342557+0000","last_change":"2023-10-06T04:54:41.338342+0000","last_active":"2023-10-06T04:54:41.342557+0000","last_peered":"2023-10-06T04:54:41.342557+0000","last_clean":"2023-10-06T04:54:41.342557+0000","last_became_active":"2023-10-06T04:54:41.338073+0000","last_became_peered":"2023-10-06T04:54:41.338073+0000","last_unstale":"2023-10-06T04:54:41.342557+0000","last_undegraded":"2023-10-06T04:54:41.342557+0000","last_fullsized":"2023-10-06T04:54:41.342557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:54.642506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.341411+0000","last_change":"2023-10-06T04:54:41.338943+0000","last_active":"2023-10-06T04:54:41.341411+0000","last_peered":"2023-10-06T04:54:41.341411+0000","last_clean":"2023-10-06T04:54:41.341411+0000","last_became_active":"2023-10-06T04:54:41.338626+0000","last_became_peered":"2023-10-06T04:54:41.338626+0000","last_unstale":"2023-10-06T04:54:41.341411+0000","last_undegraded":"2023-10-06T04:54:41.341411+0000","last_fullsized":"2023-10-06T04:54:41.341411+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:48.715181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.353380+0000","last_change":"2023-10-06T04:54:41.351720+0000","last_active":"2023-10-06T04:54:41.353380+0000","last_peered":"2023-10-06T04:54:41.353380+0000","last_clean":"2023-10-06T04:54:41.353380+0000","last_became_active":"2023-10-06T04:54:41.351384+0000","last_became_peered":"2023-10-06T04:54:41.351384+0000","last_unstale":"2023-10-06T04:54:41.353380+0000","last_undegraded":"2023-10-06T04:54:41.353380+0000","last_fullsized":"2023-10-06T04:54:41.353380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:35.749980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352079+0000","last_change":"2023-10-06T04:54:41.347438+0000","last_active":"2023-10-06T04:54:41.352079+0000","last_peered":"2023-10-06T04:54:41.352079+0000","last_clean":"2023-10-06T04:54:41.352079+0000","last_became_active":"2023-10-06T04:54:41.347185+0000","last_became_peered":"2023-10-06T04:54:41.347185+0000","last_unstale":"2023-10-06T04:54:41.352079+0000","last_undegraded":"2023-10-06T04:54:41.352079+0000","last_fullsized":"2023-10-06T04:54:41.352079+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:41.213898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356280+0000","last_change":"2023-10-06T04:54:41.353814+0000","last_active":"2023-10-06T04:54:41.356280+0000","last_peered":"2023-10-06T04:54:41.356280+0000","last_clean":"2023-10-06T04:54:41.356280+0000","last_became_active":"2023-10-06T04:54:41.353524+0000","last_became_peered":"2023-10-06T04:54:41.353524+0000","last_unstale":"2023-10-06T04:54:41.356280+0000","last_undegraded":"2023-10-06T04:54:41.356280+0000","last_fullsized":"2023-10-06T04:54:41.356280+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:42.441108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350811+0000","last_change":"2023-10-06T04:54:41.346163+0000","last_active":"2023-10-06T04:54:41.350811+0000","last_peered":"2023-10-06T04:54:41.350811+0000","last_clean":"2023-10-06T04:54:41.350811+0000","last_became_active":"2023-10-06T04:54:41.345797+0000","last_became_peered":"2023-10-06T04:54:41.345797+0000","last_unstale":"2023-10-06T04:54:41.350811+0000","last_undegraded":"2023-10-06T04:54:41.350811+0000","last_fullsized":"2023-10-06T04:54:41.350811+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:12.026057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345518+0000","last_change":"2023-10-06T04:54:41.339441+0000","last_active":"2023-10-06T04:54:41.345518+0000","last_peered":"2023-10-06T04:54:41.345518+0000","last_clean":"2023-10-06T04:54:41.345518+0000","last_became_active":"2023-10-06T04:54:41.339134+0000","last_became_peered":"2023-10-06T04:54:41.339134+0000","last_unstale":"2023-10-06T04:54:41.345518+0000","last_undegraded":"2023-10-06T04:54:41.345518+0000","last_fullsized":"2023-10-06T04:54:41.345518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:35.461823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.346996+0000","last_change":"2023-10-06T04:54:41.342915+0000","last_active":"2023-10-06T04:54:41.346996+0000","last_peered":"2023-10-06T04:54:41.346996+0000","last_clean":"2023-10-06T04:54:41.346996+0000","last_became_active":"2023-10-06T04:54:41.342042+0000","last_became_peered":"2023-10-06T04:54:41.342042+0000","last_unstale":"2023-10-06T04:54:41.346996+0000","last_undegraded":"2023-10-06T04:54:41.346996+0000","last_fullsized":"2023-10-06T04:54:41.346996+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:54.642506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.351408+0000","last_change":"2023-10-06T04:54:41.345692+0000","last_active":"2023-10-06T04:54:41.351408+0000","last_peered":"2023-10-06T04:54:41.351408+0000","last_clean":"2023-10-06T04:54:41.351408+0000","last_became_active":"2023-10-06T04:54:41.344475+0000","last_became_peered":"2023-10-06T04:54:41.344475+0000","last_unstale":"2023-10-06T04:54:41.351408+0000","last_undegraded":"2023-10-06T04:54:41.351408+0000","last_fullsized":"2023-10-06T04:54:41.351408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:59.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.344707+0000","last_change":"2023-10-06T04:54:41.339345+0000","last_active":"2023-10-06T04:54:41.344707+0000","last_peered":"2023-10-06T04:54:41.344707+0000","last_clean":"2023-10-06T04:54:41.344707+0000","last_became_active":"2023-10-06T04:54:41.338937+0000","last_became_peered":"2023-10-06T04:54:41.338937+0000","last_unstale":"2023-10-06T04:54:41.344707+0000","last_undegraded":"2023-10-06T04:54:41.344707+0000","last_fullsized":"2023-10-06T04:54:41.344707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:05.603150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352333+0000","last_change":"2023-10-06T04:54:41.348170+0000","last_active":"2023-10-06T04:54:41.352333+0000","last_peered":"2023-10-06T04:54:41.352333+0000","last_clean":"2023-10-06T04:54:41.352333+0000","last_became_active":"2023-10-06T04:54:41.346499+0000","last_became_peered":"2023-10-06T04:54:41.346499+0000","last_unstale":"2023-10-06T04:54:41.352333+0000","last_undegraded":"2023-10-06T04:54:41.352333+0000","last_fullsized":"2023-10-06T04:54:41.352333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:33.176469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.343158+0000","last_change":"2023-10-06T04:54:41.338591+0000","last_active":"2023-10-06T04:54:41.343158+0000","last_peered":"2023-10-06T04:54:41.343158+0000","last_clean":"2023-10-06T04:54:41.343158+0000","last_became_active":"2023-10-06T04:54:41.337895+0000","last_became_peered":"2023-10-06T04:54:41.337895+0000","last_unstale":"2023-10-06T04:54:41.343158+0000","last_undegraded":"2023-10-06T04:54:41.343158+0000","last_fullsized":"2023-10-06T04:54:41.343158+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:33.176469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.347474+0000","last_change":"2023-10-06T04:54:41.344101+0000","last_active":"2023-10-06T04:54:41.347474+0000","last_peered":"2023-10-06T04:54:41.347474+0000","last_clean":"2023-10-06T04:54:41.347474+0000","last_became_active":"2023-10-06T04:54:41.343410+0000","last_became_peered":"2023-10-06T04:54:41.343410+0000","last_unstale":"2023-10-06T04:54:41.347474+0000","last_undegraded":"2023-10-06T04:54:41.347474+0000","last_fullsized":"2023-10-06T04:54:41.347474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350875+0000","last_change":"2023-10-06T04:54:41.345781+0000","last_active":"2023-10-06T04:54:41.350875+0000","last_peered":"2023-10-06T04:54:41.350875+0000","last_clean":"2023-10-06T04:54:41.350875+0000","last_became_active":"2023-10-06T04:54:41.345487+0000","last_became_peered":"2023-10-06T04:54:41.345487+0000","last_unstale":"2023-10-06T04:54:41.350875+0000","last_undegraded":"2023-10-06T04:54:41.350875+0000","last_fullsized":"2023-10-06T04:54:41.350875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:04.376901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350861+0000","last_change":"2023-10-06T04:54:41.344849+0000","last_active":"2023-10-06T04:54:41.350861+0000","last_peered":"2023-10-06T04:54:41.350861+0000","last_clean":"2023-10-06T04:54:41.350861+0000","last_became_active":"2023-10-06T04:54:41.344192+0000","last_became_peered":"2023-10-06T04:54:41.344192+0000","last_unstale":"2023-10-06T04:54:41.350861+0000","last_undegraded":"2023-10-06T04:54:41.350861+0000","last_fullsized":"2023-10-06T04:54:41.350861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:36.925437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356218+0000","last_change":"2023-10-06T04:54:41.347784+0000","last_active":"2023-10-06T04:54:41.356218+0000","last_peered":"2023-10-06T04:54:41.356218+0000","last_clean":"2023-10-06T04:54:41.356218+0000","last_became_active":"2023-10-06T04:54:41.347533+0000","last_became_peered":"2023-10-06T04:54:41.347533+0000","last_unstale":"2023-10-06T04:54:41.356218+0000","last_undegraded":"2023-10-06T04:54:41.356218+0000","last_fullsized":"2023-10-06T04:54:41.356218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:36.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.349723+0000","last_change":"2023-10-06T04:54:41.343136+0000","last_active":"2023-10-06T04:54:41.349723+0000","last_peered":"2023-10-06T04:54:41.349723+0000","last_clean":"2023-10-06T04:54:41.349723+0000","last_became_active":"2023-10-06T04:54:41.342779+0000","last_became_peered":"2023-10-06T04:54:41.342779+0000","last_unstale":"2023-10-06T04:54:41.349723+0000","last_undegraded":"2023-10-06T04:54:41.349723+0000","last_fullsized":"2023-10-06T04:54:41.349723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:34.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.351729+0000","last_change":"2023-10-06T04:54:41.347099+0000","last_active":"2023-10-06T04:54:41.351729+0000","last_peered":"2023-10-06T04:54:41.351729+0000","last_clean":"2023-10-06T04:54:41.351729+0000","last_became_active":"2023-10-06T04:54:41.346832+0000","last_became_peered":"2023-10-06T04:54:41.346832+0000","last_unstale":"2023-10-06T04:54:41.351729+0000","last_undegraded":"2023-10-06T04:54:41.351729+0000","last_fullsized":"2023-10-06T04:54:41.351729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356101+0000","last_change":"2023-10-06T04:54:41.346235+0000","last_active":"2023-10-06T04:54:41.356101+0000","last_peered":"2023-10-06T04:54:41.356101+0000","last_clean":"2023-10-06T04:54:41.356101+0000","last_became_active":"2023-10-06T04:54:41.345950+0000","last_became_peered":"2023-10-06T04:54:41.345950+0000","last_unstale":"2023-10-06T04:54:41.356101+0000","last_undegraded":"2023-10-06T04:54:41.356101+0000","last_fullsized":"2023-10-06T04:54:41.356101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:40.530394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345917+0000","last_change":"2023-10-06T04:54:41.340801+0000","last_active":"2023-10-06T04:54:41.345917+0000","last_peered":"2023-10-06T04:54:41.345917+0000","last_clean":"2023-10-06T04:54:41.345917+0000","last_became_active":"2023-10-06T04:54:41.340501+0000","last_became_peered":"2023-10-06T04:54:41.340501+0000","last_unstale":"2023-10-06T04:54:41.345917+0000","last_undegraded":"2023-10-06T04:54:41.345917+0000","last_fullsized":"2023-10-06T04:54:41.345917+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:35.749980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.343102+0000","last_change":"2023-10-06T04:54:41.339121+0000","last_active":"2023-10-06T04:54:41.343102+0000","last_peered":"2023-10-06T04:54:41.343102+0000","last_clean":"2023-10-06T04:54:41.343102+0000","last_became_active":"2023-10-06T04:54:41.338864+0000","last_became_peered":"2023-10-06T04:54:41.338864+0000","last_unstale":"2023-10-06T04:54:41.343102+0000","last_undegraded":"2023-10-06T04:54:41.343102+0000","last_fullsized":"2023-10-06T04:54:41.343102+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.347361+0000","last_change":"2023-10-06T04:54:41.345098+0000","last_active":"2023-10-06T04:54:41.347361+0000","last_peered":"2023-10-06T04:54:41.347361+0000","last_clean":"2023-10-06T04:54:41.347361+0000","last_became_active":"2023-10-06T04:54:41.344824+0000","last_became_peered":"2023-10-06T04:54:41.344824+0000","last_unstale":"2023-10-06T04:54:41.347361+0000","last_undegraded":"2023-10-06T04:54:41.347361+0000","last_fullsized":"2023-10-06T04:54:41.347361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.065580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.346970+0000","last_change":"2023-10-06T04:54:41.345010+0000","last_active":"2023-10-06T04:54:41.346970+0000","last_peered":"2023-10-06T04:54:41.346970+0000","last_clean":"2023-10-06T04:54:41.346970+0000","last_became_active":"2023-10-06T04:54:41.344728+0000","last_became_peered":"2023-10-06T04:54:41.344728+0000","last_unstale":"2023-10-06T04:54:41.346970+0000","last_undegraded":"2023-10-06T04:54:41.346970+0000","last_fullsized":"2023-10-06T04:54:41.346970+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:05.603150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.353386+0000","last_change":"2023-10-06T04:54:41.351712+0000","last_active":"2023-10-06T04:54:41.353386+0000","last_peered":"2023-10-06T04:54:41.353386+0000","last_clean":"2023-10-06T04:54:41.353386+0000","last_became_active":"2023-10-06T04:54:41.351135+0000","last_became_peered":"2023-10-06T04:54:41.351135+0000","last_unstale":"2023-10-06T04:54:41.353386+0000","last_undegraded":"2023-10-06T04:54:41.353386+0000","last_fullsized":"2023-10-06T04:54:41.353386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.065580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350774+0000","last_change":"2023-10-06T04:54:41.346630+0000","last_active":"2023-10-06T04:54:41.350774+0000","last_peered":"2023-10-06T04:54:41.350774+0000","last_clean":"2023-10-06T04:54:41.350774+0000","last_became_active":"2023-10-06T04:54:41.346377+0000","last_became_peered":"2023-10-06T04:54:41.346377+0000","last_unstale":"2023-10-06T04:54:41.350774+0000","last_undegraded":"2023-10-06T04:54:41.350774+0000","last_fullsized":"2023-10-06T04:54:41.350774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:41.213898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350216+0000","last_change":"2023-10-06T04:54:41.345629+0000","last_active":"2023-10-06T04:54:41.350216+0000","last_peered":"2023-10-06T04:54:41.350216+0000","last_clean":"2023-10-06T04:54:41.350216+0000","last_became_active":"2023-10-06T04:54:41.345394+0000","last_became_peered":"2023-10-06T04:54:41.345394+0000","last_unstale":"2023-10-06T04:54:41.350216+0000","last_undegraded":"2023-10-06T04:54:41.350216+0000","last_fullsized":"2023-10-06T04:54:41.350216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352598+0000","last_change":"2023-10-06T04:54:41.349513+0000","last_active":"2023-10-06T04:54:41.352598+0000","last_peered":"2023-10-06T04:54:41.352598+0000","last_clean":"2023-10-06T04:54:41.352598+0000","last_became_active":"2023-10-06T04:54:41.349242+0000","last_became_peered":"2023-10-06T04:54:41.349242+0000","last_unstale":"2023-10-06T04:54:41.352598+0000","last_undegraded":"2023-10-06T04:54:41.352598+0000","last_fullsized":"2023-10-06T04:54:41.352598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:33.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.349650+0000","last_change":"2023-10-06T04:54:41.342981+0000","last_active":"2023-10-06T04:54:41.349650+0000","last_peered":"2023-10-06T04:54:41.349650+0000","last_clean":"2023-10-06T04:54:41.349650+0000","last_became_active":"2023-10-06T04:54:41.342702+0000","last_became_peered":"2023-10-06T04:54:41.342702+0000","last_unstale":"2023-10-06T04:54:41.349650+0000","last_undegraded":"2023-10-06T04:54:41.349650+0000","last_fullsized":"2023-10-06T04:54:41.349650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350723+0000","last_change":"2023-10-06T04:54:41.344212+0000","last_active":"2023-10-06T04:54:41.350723+0000","last_peered":"2023-10-06T04:54:41.350723+0000","last_clean":"2023-10-06T04:54:41.350723+0000","last_became_active":"2023-10-06T04:54:41.343918+0000","last_became_peered":"2023-10-06T04:54:41.343918+0000","last_unstale":"2023-10-06T04:54:41.350723+0000","last_undegraded":"2023-10-06T04:54:41.350723+0000","last_fullsized":"2023-10-06T04:54:41.350723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:14.072652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.344344+0000","last_change":"2023-10-06T04:54:41.342651+0000","last_active":"2023-10-06T04:54:41.344344+0000","last_peered":"2023-10-06T04:54:41.344344+0000","last_clean":"2023-10-06T04:54:41.344344+0000","last_became_active":"2023-10-06T04:54:41.342373+0000","last_became_peered":"2023-10-06T04:54:41.342373+0000","last_unstale":"2023-10-06T04:54:41.344344+0000","last_undegraded":"2023-10-06T04:54:41.344344+0000","last_fullsized":"2023-10-06T04:54:41.344344+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:33.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.355233+0000","last_change":"2023-10-06T04:54:41.349746+0000","last_active":"2023-10-06T04:54:41.355233+0000","last_peered":"2023-10-06T04:54:41.355233+0000","last_clean":"2023-10-06T04:54:41.355233+0000","last_became_active":"2023-10-06T04:54:41.349565+0000","last_became_peered":"2023-10-06T04:54:41.349565+0000","last_unstale":"2023-10-06T04:54:41.355233+0000","last_undegraded":"2023-10-06T04:54:41.355233+0000","last_fullsized":"2023-10-06T04:54:41.355233+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:58.359811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.351916+0000","last_change":"2023-10-06T04:54:41.349319+0000","last_active":"2023-10-06T04:54:41.351916+0000","last_peered":"2023-10-06T04:54:41.351916+0000","last_clean":"2023-10-06T04:54:41.351916+0000","last_became_active":"2023-10-06T04:54:41.349045+0000","last_became_peered":"2023-10-06T04:54:41.349045+0000","last_unstale":"2023-10-06T04:54:41.351916+0000","last_undegraded":"2023-10-06T04:54:41.351916+0000","last_fullsized":"2023-10-06T04:54:41.351916+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:14.072652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.347520+0000","last_change":"2023-10-06T04:54:41.343420+0000","last_active":"2023-10-06T04:54:41.347520+0000","last_peered":"2023-10-06T04:54:41.347520+0000","last_clean":"2023-10-06T04:54:41.347520+0000","last_became_active":"2023-10-06T04:54:41.343091+0000","last_became_peered":"2023-10-06T04:54:41.343091+0000","last_unstale":"2023-10-06T04:54:41.347520+0000","last_undegraded":"2023-10-06T04:54:41.347520+0000","last_fullsized":"2023-10-06T04:54:41.347520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:23.427926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"57'4","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352995+0000","last_change":"2023-10-06T04:54:40.339794+0000","last_active":"2023-10-06T04:54:41.352995+0000","last_peered":"2023-10-06T04:54:41.352995+0000","last_clean":"2023-10-06T04:54:41.352995+0000","last_became_active":"2023-10-06T04:54:40.339482+0000","last_became_peered":"2023-10-06T04:54:40.339482+0000","last_unstale":"2023-10-06T04:54:41.352995+0000","last_undegraded":"2023-10-06T04:54:41.352995+0000","last_fullsized":"2023-10-06T04:54:41.352995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.346735+0000","last_change":"2023-10-06T04:54:41.341604+0000","last_active":"2023-10-06T04:54:41.346735+0000","last_peered":"2023-10-06T04:54:41.346735+0000","last_clean":"2023-10-06T04:54:41.346735+0000","last_became_active":"2023-10-06T04:54:41.341293+0000","last_became_peered":"2023-10-06T04:54:41.341293+0000","last_unstale":"2023-10-06T04:54:41.346735+0000","last_undegraded":"2023-10-06T04:54:41.346735+0000","last_fullsized":"2023-10-06T04:54:41.346735+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:33.244802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"57'1","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345367+0000","last_change":"2023-10-06T04:54:40.340091+0000","last_active":"2023-10-06T04:54:41.345367+0000","last_peered":"2023-10-06T04:54:41.345367+0000","last_clean":"2023-10-06T04:54:41.345367+0000","last_became_active":"2023-10-06T04:54:40.339729+0000","last_became_peered":"2023-10-06T04:54:40.339729+0000","last_unstale":"2023-10-06T04:54:41.345367+0000","last_undegraded":"2023-10-06T04:54:41.345367+0000","last_fullsized":"2023-10-06T04:54:41.345367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352056+0000","last_change":"2023-10-06T04:54:41.348332+0000","last_active":"2023-10-06T04:54:41.352056+0000","last_peered":"2023-10-06T04:54:41.352056+0000","last_clean":"2023-10-06T04:54:41.352056+0000","last_became_active":"2023-10-06T04:54:41.348037+0000","last_became_peered":"2023-10-06T04:54:41.348037+0000","last_unstale":"2023-10-06T04:54:41.352056+0000","last_undegraded":"2023-10-06T04:54:41.352056+0000","last_fullsized":"2023-10-06T04:54:41.352056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:59.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"57'2","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345689+0000","last_change":"2023-10-06T04:54:40.344126+0000","last_active":"2023-10-06T04:54:41.345689+0000","last_peered":"2023-10-06T04:54:41.345689+0000","last_clean":"2023-10-06T04:54:41.345689+0000","last_became_active":"2023-10-06T04:54:40.343824+0000","last_became_peered":"2023-10-06T04:54:40.343824+0000","last_unstale":"2023-10-06T04:54:41.345689+0000","last_undegraded":"2023-10-06T04:54:41.345689+0000","last_fullsized":"2023-10-06T04:54:41.345689+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:32.239178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345850+0000","last_change":"2023-10-06T04:54:41.339741+0000","last_active":"2023-10-06T04:54:41.345850+0000","last_peered":"2023-10-06T04:54:41.345850+0000","last_clean":"2023-10-06T04:54:41.345850+0000","last_became_active":"2023-10-06T04:54:41.339489+0000","last_became_peered":"2023-10-06T04:54:41.339489+0000","last_unstale":"2023-10-06T04:54:41.345850+0000","last_undegraded":"2023-10-06T04:54:41.345850+0000","last_fullsized":"2023-10-06T04:54:41.345850+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:41.149662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"57'2","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350292+0000","last_change":"2023-10-06T04:54:40.338831+0000","last_active":"2023-10-06T04:54:41.350292+0000","last_peered":"2023-10-06T04:54:41.350292+0000","last_clean":"2023-10-06T04:54:41.350292+0000","last_became_active":"2023-10-06T04:54:40.338570+0000","last_became_peered":"2023-10-06T04:54:40.338570+0000","last_unstale":"2023-10-06T04:54:41.350292+0000","last_undegraded":"2023-10-06T04:54:41.350292+0000","last_fullsized":"2023-10-06T04:54:41.350292+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:32.170845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356821+0000","last_change":"2023-10-06T04:54:41.349235+0000","last_active":"2023-10-06T04:54:41.356821+0000","last_peered":"2023-10-06T04:54:41.356821+0000","last_clean":"2023-10-06T04:54:41.356821+0000","last_became_active":"2023-10-06T04:54:41.348962+0000","last_became_peered":"2023-10-06T04:54:41.348962+0000","last_unstale":"2023-10-06T04:54:41.356821+0000","last_undegraded":"2023-10-06T04:54:41.356821+0000","last_fullsized":"2023-10-06T04:54:41.356821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:24.603362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"57'1","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345320+0000","last_change":"2023-10-06T04:54:40.332965+0000","last_active":"2023-10-06T04:54:41.345320+0000","last_peered":"2023-10-06T04:54:41.345320+0000","last_clean":"2023-10-06T04:54:41.345320+0000","last_became_active":"2023-10-06T04:54:40.332661+0000","last_became_peered":"2023-10-06T04:54:40.332661+0000","last_unstale":"2023-10-06T04:54:41.345320+0000","last_undegraded":"2023-10-06T04:54:41.345320+0000","last_fullsized":"2023-10-06T04:54:41.345320+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.349028+0000","last_change":"2023-10-06T04:54:41.342853+0000","last_active":"2023-10-06T04:54:41.349028+0000","last_peered":"2023-10-06T04:54:41.349028+0000","last_clean":"2023-10-06T04:54:41.349028+0000","last_became_active":"2023-10-06T04:54:41.342505+0000","last_became_peered":"2023-10-06T04:54:41.342505+0000","last_unstale":"2023-10-06T04:54:41.349028+0000","last_undegraded":"2023-10-06T04:54:41.349028+0000","last_fullsized":"2023-10-06T04:54:41.349028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:05.163312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"57'1","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345571+0000","last_change":"2023-10-06T04:54:40.328723+0000","last_active":"2023-10-06T04:54:41.345571+0000","last_peered":"2023-10-06T04:54:41.345571+0000","last_clean":"2023-10-06T04:54:41.345571+0000","last_became_active":"2023-10-06T04:54:40.328424+0000","last_became_peered":"2023-10-06T04:54:40.328424+0000","last_unstale":"2023-10-06T04:54:41.345571+0000","last_undegraded":"2023-10-06T04:54:41.345571+0000","last_fullsized":"2023-10-06T04:54:41.345571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356015+0000","last_change":"2023-10-06T04:54:41.345325+0000","last_active":"2023-10-06T04:54:41.356015+0000","last_peered":"2023-10-06T04:54:41.356015+0000","last_clean":"2023-10-06T04:54:41.356015+0000","last_became_active":"2023-10-06T04:54:41.344412+0000","last_became_peered":"2023-10-06T04:54:41.344412+0000","last_unstale":"2023-10-06T04:54:41.356015+0000","last_undegraded":"2023-10-06T04:54:41.356015+0000","last_fullsized":"2023-10-06T04:54:41.356015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:36.643627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.327718+0000","last_change":"2023-10-06T04:54:40.344255+0000","last_active":"2023-10-06T04:54:41.327718+0000","last_peered":"2023-10-06T04:54:41.327718+0000","last_clean":"2023-10-06T04:54:41.327718+0000","last_became_active":"2023-10-06T04:54:40.343918+0000","last_became_peered":"2023-10-06T04:54:40.343918+0000","last_unstale":"2023-10-06T04:54:41.327718+0000","last_undegraded":"2023-10-06T04:54:41.327718+0000","last_fullsized":"2023-10-06T04:54:41.327718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.354186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.347054+0000","last_change":"2023-10-06T04:54:41.344292+0000","last_active":"2023-10-06T04:54:41.347054+0000","last_peered":"2023-10-06T04:54:41.347054+0000","last_clean":"2023-10-06T04:54:41.347054+0000","last_became_active":"2023-10-06T04:54:41.343799+0000","last_became_peered":"2023-10-06T04:54:41.343799+0000","last_unstale":"2023-10-06T04:54:41.347054+0000","last_undegraded":"2023-10-06T04:54:41.347054+0000","last_fullsized":"2023-10-06T04:54:41.347054+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:41.149662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.331617+0000","last_change":"2023-10-06T04:54:40.338640+0000","last_active":"2023-10-06T04:54:41.331617+0000","last_peered":"2023-10-06T04:54:41.331617+0000","last_clean":"2023-10-06T04:54:41.331617+0000","last_became_active":"2023-10-06T04:54:40.338385+0000","last_became_peered":"2023-10-06T04:54:40.338385+0000","last_unstale":"2023-10-06T04:54:41.331617+0000","last_undegraded":"2023-10-06T04:54:41.331617+0000","last_fullsized":"2023-10-06T04:54:41.331617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:40.835360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.349679+0000","last_change":"2023-10-06T04:54:41.346724+0000","last_active":"2023-10-06T04:54:41.349679+0000","last_peered":"2023-10-06T04:54:41.349679+0000","last_clean":"2023-10-06T04:54:41.349679+0000","last_became_active":"2023-10-06T04:54:41.346502+0000","last_became_peered":"2023-10-06T04:54:41.346502+0000","last_unstale":"2023-10-06T04:54:41.349679+0000","last_undegraded":"2023-10-06T04:54:41.349679+0000","last_fullsized":"2023-10-06T04:54:41.349679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:26.595609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.331174+0000","last_change":"2023-10-06T04:54:40.340720+0000","last_active":"2023-10-06T04:54:41.331174+0000","last_peered":"2023-10-06T04:54:41.331174+0000","last_clean":"2023-10-06T04:54:41.331174+0000","last_became_active":"2023-10-06T04:54:40.340447+0000","last_became_peered":"2023-10-06T04:54:40.340447+0000","last_unstale":"2023-10-06T04:54:41.331174+0000","last_undegraded":"2023-10-06T04:54:41.331174+0000","last_fullsized":"2023-10-06T04:54:41.331174+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:38.363715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.341017+0000","last_change":"2023-10-06T04:54:41.337131+0000","last_active":"2023-10-06T04:54:41.341017+0000","last_peered":"2023-10-06T04:54:41.341017+0000","last_clean":"2023-10-06T04:54:41.341017+0000","last_became_active":"2023-10-06T04:54:41.336818+0000","last_became_peered":"2023-10-06T04:54:41.336818+0000","last_unstale":"2023-10-06T04:54:41.341017+0000","last_undegraded":"2023-10-06T04:54:41.341017+0000","last_fullsized":"2023-10-06T04:54:41.341017+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:04.376901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.333533+0000","last_change":"2023-10-06T04:54:40.339697+0000","last_active":"2023-10-06T04:54:41.333533+0000","last_peered":"2023-10-06T04:54:41.333533+0000","last_clean":"2023-10-06T04:54:41.333533+0000","last_became_active":"2023-10-06T04:54:40.339299+0000","last_became_peered":"2023-10-06T04:54:40.339299+0000","last_unstale":"2023-10-06T04:54:41.333533+0000","last_undegraded":"2023-10-06T04:54:41.333533+0000","last_fullsized":"2023-10-06T04:54:41.333533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:12.235281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356451+0000","last_change":"2023-10-06T04:54:41.346440+0000","last_active":"2023-10-06T04:54:41.356451+0000","last_peered":"2023-10-06T04:54:41.356451+0000","last_clean":"2023-10-06T04:54:41.356451+0000","last_became_active":"2023-10-06T04:54:41.346125+0000","last_became_peered":"2023-10-06T04:54:41.346125+0000","last_unstale":"2023-10-06T04:54:41.356451+0000","last_undegraded":"2023-10-06T04:54:41.356451+0000","last_fullsized":"2023-10-06T04:54:41.356451+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:33.006574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"57'3","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356784+0000","last_change":"2023-10-06T04:54:40.339769+0000","last_active":"2023-10-06T04:54:41.356784+0000","last_peered":"2023-10-06T04:54:41.356784+0000","last_clean":"2023-10-06T04:54:41.356784+0000","last_became_active":"2023-10-06T04:54:40.339511+0000","last_became_peered":"2023-10-06T04:54:40.339511+0000","last_unstale":"2023-10-06T04:54:41.356784+0000","last_undegraded":"2023-10-06T04:54:41.356784+0000","last_fullsized":"2023-10-06T04:54:41.356784+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:25.589985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.346623+0000","last_change":"2023-10-06T04:54:41.340186+0000","last_active":"2023-10-06T04:54:41.346623+0000","last_peered":"2023-10-06T04:54:41.346623+0000","last_clean":"2023-10-06T04:54:41.346623+0000","last_became_active":"2023-10-06T04:54:41.339368+0000","last_became_peered":"2023-10-06T04:54:41.339368+0000","last_unstale":"2023-10-06T04:54:41.346623+0000","last_undegraded":"2023-10-06T04:54:41.346623+0000","last_fullsized":"2023-10-06T04:54:41.346623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:29.292684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"57'6","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.358754+0000","last_change":"2023-10-06T04:54:40.341385+0000","last_active":"2023-10-06T04:54:41.358754+0000","last_peered":"2023-10-06T04:54:41.358754+0000","last_clean":"2023-10-06T04:54:41.358754+0000","last_became_active":"2023-10-06T04:54:40.341078+0000","last_became_peered":"2023-10-06T04:54:40.341078+0000","last_unstale":"2023-10-06T04:54:41.358754+0000","last_undegraded":"2023-10-06T04:54:41.358754+0000","last_fullsized":"2023-10-06T04:54:41.358754+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350368+0000","last_change":"2023-10-06T04:54:41.342462+0000","last_active":"2023-10-06T04:54:41.350368+0000","last_peered":"2023-10-06T04:54:41.350368+0000","last_clean":"2023-10-06T04:54:41.350368+0000","last_became_active":"2023-10-06T04:54:41.341899+0000","last_became_peered":"2023-10-06T04:54:41.341899+0000","last_unstale":"2023-10-06T04:54:41.350368+0000","last_undegraded":"2023-10-06T04:54:41.350368+0000","last_fullsized":"2023-10-06T04:54:41.350368+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:08.051169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"57'2","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.358923+0000","last_change":"2023-10-06T04:54:40.344621+0000","last_active":"2023-10-06T04:54:41.358923+0000","last_peered":"2023-10-06T04:54:41.358923+0000","last_clean":"2023-10-06T04:54:41.358923+0000","last_became_active":"2023-10-06T04:54:40.344347+0000","last_became_peered":"2023-10-06T04:54:40.344347+0000","last_unstale":"2023-10-06T04:54:41.358923+0000","last_undegraded":"2023-10-06T04:54:41.358923+0000","last_fullsized":"2023-10-06T04:54:41.358923+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:41.740059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.344931+0000","last_change":"2023-10-06T04:54:41.342851+0000","last_active":"2023-10-06T04:54:41.344931+0000","last_peered":"2023-10-06T04:54:41.344931+0000","last_clean":"2023-10-06T04:54:41.344931+0000","last_became_active":"2023-10-06T04:54:41.342345+0000","last_became_peered":"2023-10-06T04:54:41.342345+0000","last_unstale":"2023-10-06T04:54:41.344931+0000","last_undegraded":"2023-10-06T04:54:41.344931+0000","last_fullsized":"2023-10-06T04:54:41.344931+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:05.163312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.330624+0000","last_change":"2023-10-06T04:54:40.342909+0000","last_active":"2023-10-06T04:54:41.330624+0000","last_peered":"2023-10-06T04:54:41.330624+0000","last_clean":"2023-10-06T04:54:41.330624+0000","last_became_active":"2023-10-06T04:54:40.342316+0000","last_became_peered":"2023-10-06T04:54:40.342316+0000","last_unstale":"2023-10-06T04:54:41.330624+0000","last_undegraded":"2023-10-06T04:54:41.330624+0000","last_fullsized":"2023-10-06T04:54:41.330624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356210+0000","last_change":"2023-10-06T04:54:41.346944+0000","last_active":"2023-10-06T04:54:41.356210+0000","last_peered":"2023-10-06T04:54:41.356210+0000","last_clean":"2023-10-06T04:54:41.356210+0000","last_became_active":"2023-10-06T04:54:41.346703+0000","last_became_peered":"2023-10-06T04:54:41.346703+0000","last_unstale":"2023-10-06T04:54:41.356210+0000","last_undegraded":"2023-10-06T04:54:41.356210+0000","last_fullsized":"2023-10-06T04:54:41.356210+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:39.142438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.334323+0000","last_change":"2023-10-06T04:54:40.341225+0000","last_active":"2023-10-06T04:54:41.334323+0000","last_peered":"2023-10-06T04:54:41.334323+0000","last_clean":"2023-10-06T04:54:41.334323+0000","last_became_active":"2023-10-06T04:54:40.340018+0000","last_became_peered":"2023-10-06T04:54:40.340018+0000","last_unstale":"2023-10-06T04:54:41.334323+0000","last_undegraded":"2023-10-06T04:54:41.334323+0000","last_fullsized":"2023-10-06T04:54:41.334323+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:04.157688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.340446+0000","last_change":"2023-10-06T04:54:41.336277+0000","last_active":"2023-10-06T04:54:41.340446+0000","last_peered":"2023-10-06T04:54:41.340446+0000","last_clean":"2023-10-06T04:54:41.340446+0000","last_became_active":"2023-10-06T04:54:41.335987+0000","last_became_peered":"2023-10-06T04:54:41.335987+0000","last_unstale":"2023-10-06T04:54:41.340446+0000","last_undegraded":"2023-10-06T04:54:41.340446+0000","last_fullsized":"2023-10-06T04:54:41.340446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:35.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"57'8","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.353551+0000","last_change":"2023-10-06T04:54:40.345606+0000","last_active":"2023-10-06T04:54:41.353551+0000","last_peered":"2023-10-06T04:54:41.353551+0000","last_clean":"2023-10-06T04:54:41.353551+0000","last_became_active":"2023-10-06T04:54:40.345364+0000","last_became_peered":"2023-10-06T04:54:40.345364+0000","last_unstale":"2023-10-06T04:54:41.353551+0000","last_undegraded":"2023-10-06T04:54:41.353551+0000","last_fullsized":"2023-10-06T04:54:41.353551+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:32.478317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.344804+0000","last_change":"2023-10-06T04:54:41.341467+0000","last_active":"2023-10-06T04:54:41.344804+0000","last_peered":"2023-10-06T04:54:41.344804+0000","last_clean":"2023-10-06T04:54:41.344804+0000","last_became_active":"2023-10-06T04:54:41.341177+0000","last_became_peered":"2023-10-06T04:54:41.341177+0000","last_unstale":"2023-10-06T04:54:41.344804+0000","last_undegraded":"2023-10-06T04:54:41.344804+0000","last_fullsized":"2023-10-06T04:54:41.344804+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:12.401192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"57'2","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352260+0000","last_change":"2023-10-06T04:54:40.340969+0000","last_active":"2023-10-06T04:54:41.352260+0000","last_peered":"2023-10-06T04:54:41.352260+0000","last_clean":"2023-10-06T04:54:41.352260+0000","last_became_active":"2023-10-06T04:54:40.340716+0000","last_became_peered":"2023-10-06T04:54:40.340716+0000","last_unstale":"2023-10-06T04:54:41.352260+0000","last_undegraded":"2023-10-06T04:54:41.352260+0000","last_fullsized":"2023-10-06T04:54:41.352260+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:16.444996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356036+0000","last_change":"2023-10-06T04:54:41.346566+0000","last_active":"2023-10-06T04:54:41.356036+0000","last_peered":"2023-10-06T04:54:41.356036+0000","last_clean":"2023-10-06T04:54:41.356036+0000","last_became_active":"2023-10-06T04:54:41.346266+0000","last_became_peered":"2023-10-06T04:54:41.346266+0000","last_unstale":"2023-10-06T04:54:41.356036+0000","last_undegraded":"2023-10-06T04:54:41.356036+0000","last_fullsized":"2023-10-06T04:54:41.356036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:38.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-06T04:54:42.563921+0000","last_change":"2023-10-06T04:54:40.342698+0000","last_active":"2023-10-06T04:54:42.563921+0000","last_peered":"2023-10-06T04:54:42.563921+0000","last_clean":"2023-10-06T04:54:42.563921+0000","last_became_active":"2023-10-06T04:54:40.342336+0000","last_became_peered":"2023-10-06T04:54:40.342336+0000","last_unstale":"2023-10-06T04:54:42.563921+0000","last_undegraded":"2023-10-06T04:54:42.563921+0000","last_fullsized":"2023-10-06T04:54:42.563921+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:35.638003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350743+0000","last_change":"2023-10-06T04:54:41.346932+0000","last_active":"2023-10-06T04:54:41.350743+0000","last_peered":"2023-10-06T04:54:41.350743+0000","last_clean":"2023-10-06T04:54:41.350743+0000","last_became_active":"2023-10-06T04:54:41.346674+0000","last_became_peered":"2023-10-06T04:54:41.346674+0000","last_unstale":"2023-10-06T04:54:41.350743+0000","last_undegraded":"2023-10-06T04:54:41.350743+0000","last_fullsized":"2023-10-06T04:54:41.350743+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:29.292684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.330332+0000","last_change":"2023-10-06T04:54:40.343535+0000","last_active":"2023-10-06T04:54:41.330332+0000","last_peered":"2023-10-06T04:54:41.330332+0000","last_clean":"2023-10-06T04:54:41.330332+0000","last_became_active":"2023-10-06T04:54:40.343213+0000","last_became_peered":"2023-10-06T04:54:40.343213+0000","last_unstale":"2023-10-06T04:54:41.330332+0000","last_undegraded":"2023-10-06T04:54:41.330332+0000","last_fullsized":"2023-10-06T04:54:41.330332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:24.202745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.341004+0000","last_change":"2023-10-06T04:54:41.339700+0000","last_active":"2023-10-06T04:54:41.341004+0000","last_peered":"2023-10-06T04:54:41.341004+0000","last_clean":"2023-10-06T04:54:41.341004+0000","last_became_active":"2023-10-06T04:54:41.339409+0000","last_became_peered":"2023-10-06T04:54:41.339409+0000","last_unstale":"2023-10-06T04:54:41.341004+0000","last_undegraded":"2023-10-06T04:54:41.341004+0000","last_fullsized":"2023-10-06T04:54:41.341004+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:15.192595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.327599+0000","last_change":"2023-10-06T04:54:40.344767+0000","last_active":"2023-10-06T04:54:41.327599+0000","last_peered":"2023-10-06T04:54:41.327599+0000","last_clean":"2023-10-06T04:54:41.327599+0000","last_became_active":"2023-10-06T04:54:40.344507+0000","last_became_peered":"2023-10-06T04:54:40.344507+0000","last_unstale":"2023-10-06T04:54:41.327599+0000","last_undegraded":"2023-10-06T04:54:41.327599+0000","last_fullsized":"2023-10-06T04:54:41.327599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345417+0000","last_change":"2023-10-06T04:54:41.342571+0000","last_active":"2023-10-06T04:54:41.345417+0000","last_peered":"2023-10-06T04:54:41.345417+0000","last_clean":"2023-10-06T04:54:41.345417+0000","last_became_active":"2023-10-06T04:54:41.342301+0000","last_became_peered":"2023-10-06T04:54:41.342301+0000","last_unstale":"2023-10-06T04:54:41.345417+0000","last_undegraded":"2023-10-06T04:54:41.345417+0000","last_fullsized":"2023-10-06T04:54:41.345417+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:17.450620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.334824+0000","last_change":"2023-10-06T04:54:40.341127+0000","last_active":"2023-10-06T04:54:41.334824+0000","last_peered":"2023-10-06T04:54:41.334824+0000","last_clean":"2023-10-06T04:54:41.334824+0000","last_became_active":"2023-10-06T04:54:40.339641+0000","last_became_peered":"2023-10-06T04:54:40.339641+0000","last_unstale":"2023-10-06T04:54:41.334824+0000","last_undegraded":"2023-10-06T04:54:41.334824+0000","last_fullsized":"2023-10-06T04:54:41.334824+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.636882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.351771+0000","last_change":"2023-10-06T04:54:41.347953+0000","last_active":"2023-10-06T04:54:41.351771+0000","last_peered":"2023-10-06T04:54:41.351771+0000","last_clean":"2023-10-06T04:54:41.351771+0000","last_became_active":"2023-10-06T04:54:41.347767+0000","last_became_peered":"2023-10-06T04:54:41.347767+0000","last_unstale":"2023-10-06T04:54:41.351771+0000","last_undegraded":"2023-10-06T04:54:41.351771+0000","last_fullsized":"2023-10-06T04:54:41.351771+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:28.470444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.326927+0000","last_change":"2023-10-06T04:54:40.333730+0000","last_active":"2023-10-06T04:54:41.326927+0000","last_peered":"2023-10-06T04:54:41.326927+0000","last_clean":"2023-10-06T04:54:41.326927+0000","last_became_active":"2023-10-06T04:54:40.333468+0000","last_became_peered":"2023-10-06T04:54:40.333468+0000","last_unstale":"2023-10-06T04:54:41.326927+0000","last_undegraded":"2023-10-06T04:54:41.326927+0000","last_fullsized":"2023-10-06T04:54:41.326927+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:34.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.349215+0000","last_change":"2023-10-06T04:54:41.342909+0000","last_active":"2023-10-06T04:54:41.349215+0000","last_peered":"2023-10-06T04:54:41.349215+0000","last_clean":"2023-10-06T04:54:41.349215+0000","last_became_active":"2023-10-06T04:54:41.342595+0000","last_became_peered":"2023-10-06T04:54:41.342595+0000","last_unstale":"2023-10-06T04:54:41.349215+0000","last_undegraded":"2023-10-06T04:54:41.349215+0000","last_fullsized":"2023-10-06T04:54:41.349215+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:41.213898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.327814+0000","last_change":"2023-10-06T04:54:40.344570+0000","last_active":"2023-10-06T04:54:41.327814+0000","last_peered":"2023-10-06T04:54:41.327814+0000","last_clean":"2023-10-06T04:54:41.327814+0000","last_became_active":"2023-10-06T04:54:40.344295+0000","last_became_peered":"2023-10-06T04:54:40.344295+0000","last_unstale":"2023-10-06T04:54:41.327814+0000","last_undegraded":"2023-10-06T04:54:41.327814+0000","last_fullsized":"2023-10-06T04:54:41.327814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:40.208274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352232+0000","last_change":"2023-10-06T04:54:41.345147+0000","last_active":"2023-10-06T04:54:41.352232+0000","last_peered":"2023-10-06T04:54:41.352232+0000","last_clean":"2023-10-06T04:54:41.352232+0000","last_became_active":"2023-10-06T04:54:41.344843+0000","last_became_peered":"2023-10-06T04:54:41.344843+0000","last_unstale":"2023-10-06T04:54:41.352232+0000","last_undegraded":"2023-10-06T04:54:41.352232+0000","last_fullsized":"2023-10-06T04:54:41.352232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:49.752409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.326237+0000","last_change":"2023-10-06T04:54:40.344877+0000","last_active":"2023-10-06T04:54:41.326237+0000","last_peered":"2023-10-06T04:54:41.326237+0000","last_clean":"2023-10-06T04:54:41.326237+0000","last_became_active":"2023-10-06T04:54:40.344580+0000","last_became_peered":"2023-10-06T04:54:40.344580+0000","last_unstale":"2023-10-06T04:54:41.326237+0000","last_undegraded":"2023-10-06T04:54:41.326237+0000","last_fullsized":"2023-10-06T04:54:41.326237+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:32.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.347030+0000","last_change":"2023-10-06T04:54:41.341515+0000","last_active":"2023-10-06T04:54:41.347030+0000","last_peered":"2023-10-06T04:54:41.347030+0000","last_clean":"2023-10-06T04:54:41.347030+0000","last_became_active":"2023-10-06T04:54:41.341263+0000","last_became_peered":"2023-10-06T04:54:41.341263+0000","last_unstale":"2023-10-06T04:54:41.347030+0000","last_undegraded":"2023-10-06T04:54:41.347030+0000","last_fullsized":"2023-10-06T04:54:41.347030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.208369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"57'1","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356854+0000","last_change":"2023-10-06T04:54:40.340640+0000","last_active":"2023-10-06T04:54:41.356854+0000","last_peered":"2023-10-06T04:54:41.356854+0000","last_clean":"2023-10-06T04:54:41.356854+0000","last_became_active":"2023-10-06T04:54:40.340384+0000","last_became_peered":"2023-10-06T04:54:40.340384+0000","last_unstale":"2023-10-06T04:54:41.356854+0000","last_undegraded":"2023-10-06T04:54:41.356854+0000","last_fullsized":"2023-10-06T04:54:41.356854+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.769263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.333986+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:54:41.333986+0000","last_peered":"2023-10-06T04:54:41.333986+0000","last_clean":"2023-10-06T04:54:41.333986+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:54:41.333986+0000","last_undegraded":"2023-10-06T04:54:41.333986+0000","last_fullsized":"2023-10-06T04:54:41.333986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345668+0000","last_change":"2023-10-06T04:54:41.343292+0000","last_active":"2023-10-06T04:54:41.345668+0000","last_peered":"2023-10-06T04:54:41.345668+0000","last_clean":"2023-10-06T04:54:41.345668+0000","last_became_active":"2023-10-06T04:54:41.343013+0000","last_became_peered":"2023-10-06T04:54:41.343013+0000","last_unstale":"2023-10-06T04:54:41.345668+0000","last_undegraded":"2023-10-06T04:54:41.345668+0000","last_fullsized":"2023-10-06T04:54:41.345668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:43.579970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.328770+0000","last_change":"2023-10-06T04:54:40.343623+0000","last_active":"2023-10-06T04:54:41.328770+0000","last_peered":"2023-10-06T04:54:41.328770+0000","last_clean":"2023-10-06T04:54:41.328770+0000","last_became_active":"2023-10-06T04:54:40.343366+0000","last_became_peered":"2023-10-06T04:54:40.343366+0000","last_unstale":"2023-10-06T04:54:41.328770+0000","last_undegraded":"2023-10-06T04:54:41.328770+0000","last_fullsized":"2023-10-06T04:54:41.328770+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:33.074026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.328853+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:54:41.328853+0000","last_peered":"2023-10-06T04:54:41.328853+0000","last_clean":"2023-10-06T04:54:41.328853+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:54:41.328853+0000","last_undegraded":"2023-10-06T04:54:41.328853+0000","last_fullsized":"2023-10-06T04:54:41.328853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.351168+0000","last_change":"2023-10-06T04:54:41.348152+0000","last_active":"2023-10-06T04:54:41.351168+0000","last_peered":"2023-10-06T04:54:41.351168+0000","last_clean":"2023-10-06T04:54:41.351168+0000","last_became_active":"2023-10-06T04:54:41.347886+0000","last_became_peered":"2023-10-06T04:54:41.347886+0000","last_unstale":"2023-10-06T04:54:41.351168+0000","last_undegraded":"2023-10-06T04:54:41.351168+0000","last_fullsized":"2023-10-06T04:54:41.351168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:18.860011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"57'2","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.344290+0000","last_change":"2023-10-06T04:54:40.344112+0000","last_active":"2023-10-06T04:54:41.344290+0000","last_peered":"2023-10-06T04:54:41.344290+0000","last_clean":"2023-10-06T04:54:41.344290+0000","last_became_active":"2023-10-06T04:54:40.343905+0000","last_became_peered":"2023-10-06T04:54:40.343905+0000","last_unstale":"2023-10-06T04:54:41.344290+0000","last_undegraded":"2023-10-06T04:54:41.344290+0000","last_fullsized":"2023-10-06T04:54:41.344290+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:32.026699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.326860+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:54:41.326860+0000","last_peered":"2023-10-06T04:54:41.326860+0000","last_clean":"2023-10-06T04:54:41.326860+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:54:41.326860+0000","last_undegraded":"2023-10-06T04:54:41.326860+0000","last_fullsized":"2023-10-06T04:54:41.326860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":192,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.329477+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:54:41.329477+0000","last_peered":"2023-10-06T04:54:41.329477+0000","last_clean":"2023-10-06T04:54:41.329477+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:54:41.329477+0000","last_undegraded":"2023-10-06T04:54:41.329477+0000","last_fullsized":"2023-10-06T04:54:41.329477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350730+0000","last_change":"2023-10-06T04:54:41.347479+0000","last_active":"2023-10-06T04:54:41.350730+0000","last_peered":"2023-10-06T04:54:41.350730+0000","last_clean":"2023-10-06T04:54:41.350730+0000","last_became_active":"2023-10-06T04:54:41.347195+0000","last_became_peered":"2023-10-06T04:54:41.347195+0000","last_unstale":"2023-10-06T04:54:41.350730+0000","last_undegraded":"2023-10-06T04:54:41.350730+0000","last_fullsized":"2023-10-06T04:54:41.350730+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.774887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.330731+0000","last_change":"2023-10-06T04:54:40.343032+0000","last_active":"2023-10-06T04:54:41.330731+0000","last_peered":"2023-10-06T04:54:41.330731+0000","last_clean":"2023-10-06T04:54:41.330731+0000","last_became_active":"2023-10-06T04:54:40.342591+0000","last_became_peered":"2023-10-06T04:54:40.342591+0000","last_unstale":"2023-10-06T04:54:41.330731+0000","last_undegraded":"2023-10-06T04:54:41.330731+0000","last_fullsized":"2023-10-06T04:54:41.330731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:38.136814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":42,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.330174+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:54:41.330174+0000","last_peered":"2023-10-06T04:54:41.330174+0000","last_clean":"2023-10-06T04:54:41.330174+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:54:41.330174+0000","last_undegraded":"2023-10-06T04:54:41.330174+0000","last_fullsized":"2023-10-06T04:54:41.330174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.345444+0000","last_change":"2023-10-06T04:54:41.342689+0000","last_active":"2023-10-06T04:54:41.345444+0000","last_peered":"2023-10-06T04:54:41.345444+0000","last_clean":"2023-10-06T04:54:41.345444+0000","last_became_active":"2023-10-06T04:54:41.342425+0000","last_became_peered":"2023-10-06T04:54:41.342425+0000","last_unstale":"2023-10-06T04:54:41.345444+0000","last_undegraded":"2023-10-06T04:54:41.345444+0000","last_fullsized":"2023-10-06T04:54:41.345444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:15.864427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.326643+0000","last_change":"2023-10-06T04:54:40.334344+0000","last_active":"2023-10-06T04:54:41.326643+0000","last_peered":"2023-10-06T04:54:41.326643+0000","last_clean":"2023-10-06T04:54:41.326643+0000","last_became_active":"2023-10-06T04:54:40.334079+0000","last_became_peered":"2023-10-06T04:54:40.334079+0000","last_unstale":"2023-10-06T04:54:41.326643+0000","last_undegraded":"2023-10-06T04:54:41.326643+0000","last_fullsized":"2023-10-06T04:54:41.326643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:28.287060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":42,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.329577+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:54:41.329577+0000","last_peered":"2023-10-06T04:54:41.329577+0000","last_clean":"2023-10-06T04:54:41.329577+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:54:41.329577+0000","last_undegraded":"2023-10-06T04:54:41.329577+0000","last_fullsized":"2023-10-06T04:54:41.329577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356042+0000","last_change":"2023-10-06T04:54:41.345534+0000","last_active":"2023-10-06T04:54:41.356042+0000","last_peered":"2023-10-06T04:54:41.356042+0000","last_clean":"2023-10-06T04:54:41.356042+0000","last_became_active":"2023-10-06T04:54:41.344323+0000","last_became_peered":"2023-10-06T04:54:41.344323+0000","last_unstale":"2023-10-06T04:54:41.356042+0000","last_undegraded":"2023-10-06T04:54:41.356042+0000","last_fullsized":"2023-10-06T04:54:41.356042+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:43.998510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.329992+0000","last_change":"2023-10-06T04:54:40.340001+0000","last_active":"2023-10-06T04:54:41.329992+0000","last_peered":"2023-10-06T04:54:41.329992+0000","last_clean":"2023-10-06T04:54:41.329992+0000","last_became_active":"2023-10-06T04:54:40.339665+0000","last_became_peered":"2023-10-06T04:54:40.339665+0000","last_unstale":"2023-10-06T04:54:41.329992+0000","last_undegraded":"2023-10-06T04:54:41.329992+0000","last_fullsized":"2023-10-06T04:54:41.329992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:07.045545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.330391+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:54:41.330391+0000","last_peered":"2023-10-06T04:54:41.330391+0000","last_clean":"2023-10-06T04:54:41.330391+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:54:41.330391+0000","last_undegraded":"2023-10-06T04:54:41.330391+0000","last_fullsized":"2023-10-06T04:54:41.330391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.356288+0000","last_change":"2023-10-06T04:54:41.354041+0000","last_active":"2023-10-06T04:54:41.356288+0000","last_peered":"2023-10-06T04:54:41.356288+0000","last_clean":"2023-10-06T04:54:41.356288+0000","last_became_active":"2023-10-06T04:54:41.353779+0000","last_became_peered":"2023-10-06T04:54:41.353779+0000","last_unstale":"2023-10-06T04:54:41.356288+0000","last_undegraded":"2023-10-06T04:54:41.356288+0000","last_fullsized":"2023-10-06T04:54:41.356288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:13.240905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"57'2","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.352887+0000","last_change":"2023-10-06T04:54:40.343461+0000","last_active":"2023-10-06T04:54:41.352887+0000","last_peered":"2023-10-06T04:54:41.352887+0000","last_clean":"2023-10-06T04:54:41.352887+0000","last_became_active":"2023-10-06T04:54:40.343184+0000","last_became_peered":"2023-10-06T04:54:40.343184+0000","last_unstale":"2023-10-06T04:54:41.352887+0000","last_undegraded":"2023-10-06T04:54:41.352887+0000","last_fullsized":"2023-10-06T04:54:41.352887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:47.709557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.335835+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:54:41.335835+0000","last_peered":"2023-10-06T04:54:41.335835+0000","last_clean":"2023-10-06T04:54:41.335835+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:54:41.335835+0000","last_undegraded":"2023-10-06T04:54:41.335835+0000","last_fullsized":"2023-10-06T04:54:41.335835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.350266+0000","last_change":"2023-10-06T04:54:41.344335+0000","last_active":"2023-10-06T04:54:41.350266+0000","last_peered":"2023-10-06T04:54:41.350266+0000","last_clean":"2023-10-06T04:54:41.350266+0000","last_became_active":"2023-10-06T04:54:41.343966+0000","last_became_peered":"2023-10-06T04:54:41.343966+0000","last_unstale":"2023-10-06T04:54:41.350266+0000","last_undegraded":"2023-10-06T04:54:41.350266+0000","last_fullsized":"2023-10-06T04:54:41.350266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:40.317895+0000","last_clean_scrub_stamp":"2023-10-06T04:54:40.317895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:18.860011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"57'1","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.349810+0000","last_change":"2023-10-06T04:54:40.342683+0000","last_active":"2023-10-06T04:54:41.349810+0000","last_peered":"2023-10-06T04:54:41.349810+0000","last_clean":"2023-10-06T04:54:41.349810+0000","last_became_active":"2023-10-06T04:54:40.342261+0000","last_became_peered":"2023-10-06T04:54:40.342261+0000","last_unstale":"2023-10-06T04:54:41.349810+0000","last_undegraded":"2023-10-06T04:54:41.349810+0000","last_fullsized":"2023-10-06T04:54:41.349810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:54:39.312271+0000","last_clean_scrub_stamp":"2023-10-06T04:54:39.312271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.858803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T04:54:41.336659+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:54:41.336659+0000","last_peered":"2023-10-06T04:54:41.336659+0000","last_clean":"2023-10-06T04:54:41.336659+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:54:41.336659+0000","last_undegraded":"2023-10-06T04:54:41.336659+0000","last_fullsized":"2023-10-06T04:54:41.336659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306752,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065088,"statfs":{"total":96636764160,"available":96322650112,"internally_reserved":0,"allocated":1302528,"data_stored":894600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.751,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.66400000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.80200000000000005,"15min":0.84499999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70099999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":1.111,"15min":1.111},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.502},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.434}]},{"osd":5,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.874,"15min":0.874},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77500000000000002,"5min":1.214,"15min":1.214},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:54:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72599999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302068,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069772,"statfs":{"total":96636764160,"available":96327446528,"internally_reserved":0,"allocated":700416,"data_stored":304871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.504},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.751,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63300000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:54:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.621},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302668,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069172,"statfs":{"total":96636764160,"available":96326832128,"internally_reserved":0,"allocated":1314816,"data_stored":896124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.48399999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.373}]},{"osd":2,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51000000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.42399999999999999,"15min":0.42899999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54100000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.376,"5min":0.42799999999999999,"15min":0.437},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57499999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.63400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73699999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.625}]},{"osd":7,"last update":"Fri Oct 6 04:54:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":1.026,"15min":1.026},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302064,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069776,"statfs":{"total":96636764160,"available":96327450624,"internally_reserved":0,"allocated":696320,"data_stored":304837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.46200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.311},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36499999999999999,"15min":0.36599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.28199999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64400000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.38500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302064,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069776,"statfs":{"total":96636764160,"available":96327450624,"internally_reserved":0,"allocated":696320,"data_stored":304837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.747,"5min":1.048,"15min":1.048},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.22700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.57999999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.31900000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68700000000000006,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85399999999999998,"5min":1.246,"15min":1.246},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.40699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.877,"15min":0.877},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.747},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306160,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065680,"statfs":{"total":96636764160,"available":96323256320,"internally_reserved":0,"allocated":696320,"data_stored":304837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44500000000000001,"5min":1.415,"15min":1.415},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.58599999999999997,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51400000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.44600000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.45700000000000002,"15min":0.47099999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53600000000000003,"5min":1.482,"15min":1.482},"last":0.377}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":5.069,"15min":5.069},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57799999999999996,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.71299999999999997,"15min":0.73799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94799999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80200000000000005,"15min":0.83999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.48699999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.77300000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":1.452,"15min":1.452},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.81399999999999995,"15min":0.85399999999999998},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":6.367,"15min":6.367},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":5,"15min":5},"last":0.53300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302076,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069764,"statfs":{"total":96636764160,"available":96327438336,"internally_reserved":0,"allocated":708608,"data_stored":306373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.81999999999999995,"15min":0.876},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63200000000000001,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.85299999999999998,"15min":0.90300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67900000000000005,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.85799999999999998,"15min":0.91400000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.68400000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64300000000000002,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.621},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.70299999999999996,"15min":0.73899999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61599999999999999,"5min":5.415,"15min":5.415},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67200000000000004,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66200000000000003,"5min":4.202,"15min":4.202},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74099999999999999,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":0.872},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.72499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.48099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302064,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069776,"statfs":{"total":96636764160,"available":96327450624,"internally_reserved":0,"allocated":696320,"data_stored":303506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70699999999999996,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63700000000000001,"15min":0.65500000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72399999999999998,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.69299999999999995,"15min":0.71599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.81799999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.746,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.0049999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.008,"15min":1.0760000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.78800000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.751},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.746},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76500000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:54:50.411 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:54:50.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:54:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.571+0000 7ff7ad539700 1 Processor -- start 2023-10-06T04:54:50.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.571+0000 7ff7ad539700 1 -- start start 2023-10-06T04:54:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7ad539700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7a813ff80 0x7ff7a8068360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7ad539700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7a806db90 0x7ff7a8156e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7ad539700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a8157350 con 0x7ff7a81569a0 2023-10-06T04:54:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7ad539700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a8067ed0 con 0x7ff7a813ff80 2023-10-06T04:54:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7ad539700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a8157580 con 0x7ff7a806db90 2023-10-06T04:54:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7a813ff80 0x7ff7a8068360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff7a81569a0 0x7ff7a806d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48226/0 (socket says 172.21.15.3:48226) 2023-10-06T04:54:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a67fc700 1 -- 172.21.15.3:0/2829757301 learned_addr learned my addr 172.21.15.3:0/2829757301 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a77fe700 1 --2- 172.21.15.3:0/2829757301 >> v2:172.21.15.153:3300/0 conn(0x7ff7a806db90 0x7ff7a8156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a5ffb700 1 -- 172.21.15.3:0/2829757301 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629205411 0 0) 0x7ff7a8157350 con 0x7ff7a81569a0 2023-10-06T04:54:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a5ffb700 1 -- 172.21.15.3:0/2829757301 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff790003570 con 0x7ff7a81569a0 2023-10-06T04:54:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a6ffd700 1 -- 172.21.15.3:0/2829757301 >> v2:172.21.15.153:3300/0 conn(0x7ff7a806db90 msgr2=0x7ff7a8156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a6ffd700 1 --2- 172.21.15.3:0/2829757301 >> v2:172.21.15.153:3300/0 conn(0x7ff7a806db90 0x7ff7a8156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a6ffd700 1 -- 172.21.15.3:0/2829757301 >> v1:172.21.15.3:6789/0 conn(0x7ff7a81569a0 legacy=0x7ff7a806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a6ffd700 1 -- 172.21.15.3:0/2829757301 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a8067880 con 0x7ff7a813ff80 2023-10-06T04:54:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a77fe700 1 --2- 172.21.15.3:0/2829757301 >> v2:172.21.15.153:3300/0 conn(0x7ff7a806db90 0x7ff7a8156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a6ffd700 1 --2- 172.21.15.3:0/2829757301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a813ff80 0x7ff7a8068360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80cb8c0 tx=0x7ff7980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb91d03535b51217 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a5ffb700 1 -- 172.21.15.3:0/2829757301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798011630 con 0x7ff7a813ff80 2023-10-06T04:54:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a5ffb700 1 -- 172.21.15.3:0/2829757301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff798011790 con 0x7ff7a813ff80 2023-10-06T04:54:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.572+0000 7ff7a5ffb700 1 -- 172.21.15.3:0/2829757301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798011b30 con 0x7ff7a813ff80 2023-10-06T04:54:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 -- 172.21.15.3:0/2829757301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a813ff80 msgr2=0x7ff7a8068360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 --2- 172.21.15.3:0/2829757301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a813ff80 0x7ff7a8068360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80cb8c0 tx=0x7ff7980091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 -- 172.21.15.3:0/2829757301 shutdown_connections 2023-10-06T04:54:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 --2- 172.21.15.3:0/2829757301 >> v2:172.21.15.153:3300/0 conn(0x7ff7a806db90 0x7ff7a8156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 --2- 172.21.15.3:0/2829757301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a813ff80 0x7ff7a8068360 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 -- 172.21.15.3:0/2829757301 >> 172.21.15.3:0/2829757301 conn(0x7ff7a80ccff0 msgr2=0x7ff7a80cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:50.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 -- 172.21.15.3:0/2829757301 shutdown_connections 2023-10-06T04:54:50.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.573+0000 7ff7ad539700 1 -- 172.21.15.3:0/2829757301 wait complete. 2023-10-06T04:54:50.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.574+0000 7ff7ad539700 1 Processor -- start 2023-10-06T04:54:50.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.574+0000 7ff7ad539700 1 -- start start 2023-10-06T04:54:50.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7ad539700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7a806db90 0x7ff7a816a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7ad539700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7a813ff80 0x7ff7a80ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7ad539700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a81569a0 0x7ff7a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7ad539700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a816a140 con 0x7ff7a81569a0 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7ad539700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80aba80 con 0x7ff7a813ff80 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7ad539700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80abd60 con 0x7ff7a806db90 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a81569a0 0x7ff7a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a81569a0 0x7ff7a806a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45050/0 (socket says 172.21.15.3:45050) 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 -- 172.21.15.3:0/3531841765 learned_addr learned my addr 172.21.15.3:0/3531841765 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:50.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a67fc700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7a813ff80 0x7ff7a80ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a6ffd700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7a806db90 0x7ff7a816a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 -- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7a806db90 msgr2=0x7ff7a816a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7a806db90 0x7ff7a816a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 -- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7a813ff80 msgr2=0x7ff7a80ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7a813ff80 0x7ff7a80ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a77fe700 1 -- 172.21.15.3:0/3531841765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798009420 con 0x7ff7a81569a0 2023-10-06T04:54:50.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.575+0000 7ff7a6ffd700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7a806db90 0x7ff7a816a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.576+0000 7ff7a77fe700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a81569a0 0x7ff7a806a840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009f90 tx=0x7ff7a00099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.576+0000 7ff78effd700 1 -- 172.21.15.3:0/3531841765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000dc90 con 0x7ff7a81569a0 2023-10-06T04:54:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.576+0000 7ff78effd700 1 -- 172.21.15.3:0/3531841765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a000f040 con 0x7ff7a81569a0 2023-10-06T04:54:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.576+0000 7ff78effd700 1 -- 172.21.15.3:0/3531841765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0015690 con 0x7ff7a81569a0 2023-10-06T04:54:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.576+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a8169dd0 con 0x7ff7a81569a0 2023-10-06T04:54:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.576+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a815ab10 con 0x7ff7a81569a0 2023-10-06T04:54:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.577+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a816a2b0 con 0x7ff7a81569a0 2023-10-06T04:54:50.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.577+0000 7ff78effd700 1 -- 172.21.15.3:0/3531841765 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7a001e3b0 con 0x7ff7a81569a0 2023-10-06T04:54:50.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.577+0000 7ff78effd700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.578+0000 7ff78effd700 1 -- 172.21.15.3:0/3531841765 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff7a0084af0 con 0x7ff7a81569a0 2023-10-06T04:54:50.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.578+0000 7ff7a6ffd700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.578+0000 7ff7a6ffd700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7980093f0 tx=0x7ff798017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:50.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.582+0000 7ff78effd700 1 -- 172.21.15.3:0/3531841765 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7a0051c00 con 0x7ff7a81569a0 2023-10-06T04:54:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.856+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff7a804c380 con 0x7ff7a81569a0 2023-10-06T04:54:50.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.860+0000 7ff78effd700 1 -- 172.21.15.3:0/3531841765 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff7a001a0e0 con 0x7ff7a81569a0 2023-10-06T04:54:50.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 msgr2=0x7ff7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7980093f0 tx=0x7ff798017000 comp rx=0 tx=0).stop 2023-10-06T04:54:50.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a81569a0 msgr2=0x7ff7a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a81569a0 0x7ff7a806a840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009f90 tx=0x7ff7a00099b0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 shutdown_connections 2023-10-06T04:54:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7a81569a0 0x7ff7a806a840 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7a806db90 0x7ff7a816a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 --2- 172.21.15.3:0/3531841765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7a813ff80 0x7ff7a80ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 >> 172.21.15.3:0/3531841765 conn(0x7ff7a80ccff0 msgr2=0x7ff7a806d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 shutdown_connections 2023-10-06T04:54:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.862+0000 7ff7ad539700 1 -- 172.21.15.3:0/3531841765 wait complete. 2023-10-06T04:54:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T04:54:50.876 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:50.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.992+0000 7f63d6520700 1 Processor -- start 2023-10-06T04:54:50.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.993+0000 7f63d6520700 1 -- start start 2023-10-06T04:54:50.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.993+0000 7f63d6520700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:50.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.993+0000 7f63d6520700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d00cf6c0 con 0x7f63d015c8f0 2023-10-06T04:54:50.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.993+0000 7f63d6520700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d00cfb50 con 0x7f63d00d06a0 2023-10-06T04:54:50.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.993+0000 7f63d6520700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00d0b20 con 0x7f63d015ccc0 2023-10-06T04:54:50.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63cffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f63d015c8f0 0x7f63d015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48234/0 (socket says 172.21.15.3:48234) 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63cffff700 1 -- 172.21.15.3:0/4254585492 learned_addr learned my addr 172.21.15.3:0/4254585492 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63cf7fe700 1 --2- 172.21.15.3:0/4254585492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63ceffd700 1 -- 172.21.15.3:0/4254585492 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168181128 0 0) 0x7f63d00cf6c0 con 0x7f63d015c8f0 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63ceffd700 1 -- 172.21.15.3:0/4254585492 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b8003570 con 0x7f63d015c8f0 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63ceffd700 1 -- 172.21.15.3:0/4254585492 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571506271 0 0) 0x7f63d00cfb50 con 0x7f63d00d06a0 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63ceffd700 1 -- 172.21.15.3:0/4254585492 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d00cf6c0 con 0x7f63d00d06a0 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63cf7fe700 1 -- 172.21.15.3:0/4254585492 >> v1:172.21.15.153:6789/0 conn(0x7f63d00d06a0 legacy=0x7f63d00b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63cf7fe700 1 -- 172.21.15.3:0/4254585492 >> v1:172.21.15.3:6789/0 conn(0x7f63d015c8f0 legacy=0x7f63d015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63cf7fe700 1 -- 172.21.15.3:0/4254585492 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00ceab0 con 0x7f63d015ccc0 2023-10-06T04:54:50.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63cf7fe700 1 --2- 172.21.15.3:0/4254585492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d00d0160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009910 tx=0x7f63c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9820d313c582a75f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63ceffd700 1 -- 172.21.15.3:0/4254585492 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011630 con 0x7f63d015ccc0 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63ceffd700 1 -- 172.21.15.3:0/4254585492 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0011790 con 0x7f63d015ccc0 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.994+0000 7f63ceffd700 1 -- 172.21.15.3:0/4254585492 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011b10 con 0x7f63d015ccc0 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.995+0000 7f63d6520700 1 -- 172.21.15.3:0/4254585492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 msgr2=0x7f63d00d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.995+0000 7f63d6520700 1 --2- 172.21.15.3:0/4254585492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d00d0160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009910 tx=0x7f63c00091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.995+0000 7f63d6520700 1 -- 172.21.15.3:0/4254585492 shutdown_connections 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.995+0000 7f63d6520700 1 --2- 172.21.15.3:0/4254585492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d00d0160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.995+0000 7f63d6520700 1 -- 172.21.15.3:0/4254585492 >> 172.21.15.3:0/4254585492 conn(0x7f63d0070310 msgr2=0x7f63d0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.995+0000 7f63d6520700 1 -- 172.21.15.3:0/4254585492 shutdown_connections 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.995+0000 7f63d6520700 1 -- 172.21.15.3:0/4254585492 wait complete. 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.996+0000 7f63d6520700 1 Processor -- start 2023-10-06T04:54:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.996+0000 7f63d6520700 1 -- start start 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63d6520700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d00d06a0 0x7f63d0159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63d6520700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d015c8f0 0x7f63d0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63d6520700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d0069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63d6520700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00697e0 con 0x7f63d00d06a0 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63d6520700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0069ac0 con 0x7f63d015c8f0 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63d6520700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0069da0 con 0x7f63d015ccc0 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d00d06a0 0x7f63d0159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d00d06a0 0x7f63d0159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45052/0 (socket says 172.21.15.3:45052) 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 -- 172.21.15.3:0/3149397896 learned_addr learned my addr 172.21.15.3:0/3149397896 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cf7fe700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d015c8f0 0x7f63d0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63d4abd700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d0069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 -- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 msgr2=0x7f63d0069160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d0069160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 -- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d015c8f0 msgr2=0x7f63d0068c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d015c8f0 0x7f63d0068c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.997+0000 7f63cffff700 1 -- 172.21.15.3:0/3149397896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c0009420 con 0x7f63d00d06a0 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.998+0000 7f63cffff700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d00d06a0 0x7f63d0159510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c790 tx=0x7f63c4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.998+0000 7f63b7fff700 1 -- 172.21.15.3:0/3149397896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4005b30 con 0x7f63d00d06a0 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.998+0000 7f63b7fff700 1 -- 172.21.15.3:0/3149397896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4005c90 con 0x7f63d00d06a0 2023-10-06T04:54:51.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.998+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d006a080 con 0x7f63d00d06a0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.998+0000 7f63b7fff700 1 -- 172.21.15.3:0/3149397896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4007300 con 0x7f63d00d06a0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.998+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d00671b0 con 0x7f63d00d06a0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.999+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d0071680 con 0x7f63d00d06a0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:50.999+0000 7f63b7fff700 1 -- 172.21.15.3:0/3149397896 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63c401b3b0 con 0x7f63d00d06a0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.000+0000 7f63b7fff700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63bc066fb0 0x7f63bc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.000+0000 7f63b7fff700 1 -- 172.21.15.3:0/3149397896 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f63c4083c10 con 0x7f63d00d06a0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.000+0000 7f63cf7fe700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63bc066fb0 0x7f63bc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.001+0000 7f63cf7fe700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63bc066fb0 0x7f63bc069460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63d015d0b0 tx=0x7f63c0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.004+0000 7f63b7fff700 1 -- 172.21.15.3:0/3149397896 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63c4050c80 con 0x7f63d00d06a0 2023-10-06T04:54:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.241+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63d00b35a0 con 0x7f63d00d06a0 2023-10-06T04:54:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.241+0000 7f63b7fff700 1 -- 172.21.15.3:0/3149397896 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7f63c4017020 con 0x7f63d00d06a0 2023-10-06T04:54:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:51.243 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:54:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63bc066fb0 msgr2=0x7f63bc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63bc066fb0 0x7f63bc069460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63d015d0b0 tx=0x7f63c0017000 comp rx=0 tx=0).stop 2023-10-06T04:54:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d00d06a0 msgr2=0x7f63d0159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d00d06a0 0x7f63d0159510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c790 tx=0x7f63c4007ea0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 shutdown_connections 2023-10-06T04:54:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63bc066fb0 0x7f63bc069460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63d015ccc0 0x7f63d0069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d00d06a0 0x7f63d0159510 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 --2- 172.21.15.3:0/3149397896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63d015c8f0 0x7f63d0068c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 >> 172.21.15.3:0/3149397896 conn(0x7f63d0070310 msgr2=0x7f63d0156e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 shutdown_connections 2023-10-06T04:54:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.243+0000 7f63d6520700 1 -- 172.21.15.3:0/3149397896 wait complete. 2023-10-06T04:54:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:54:51.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:54:40.735227+0000', 'modified': '2023-10-06T04:54:44.356805+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5721}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5721': {'gid': 5721, 'name': 'a', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/3909985257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6833', 'nonce': 3909985257}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 3909985257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:54:51.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:54:51.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T04:54:51.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.783+0000 7f11a23e2700 1 Processor -- start 2023-10-06T04:54:51.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.783+0000 7f11a23e2700 1 -- start start 2023-10-06T04:54:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f11a23e2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f119c0b3a40 0x7f119c0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f11a23e2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f119c0b3e10 0x7f119c0680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f11a23e2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c0685f0 con 0x7f119c1406b0 2023-10-06T04:54:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f11a23e2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c0cd380 con 0x7f119c0b3a40 2023-10-06T04:54:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f11a23e2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c0cd660 con 0x7f119c0b3e10 2023-10-06T04:54:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f119bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f119c0b3a40 0x7f119c0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f11a097f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f119c1406b0 0x7f119c140b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48244/0 (socket says 172.21.15.3:48244) 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.784+0000 7f11a097f700 1 -- 172.21.15.3:0/2317555570 learned_addr learned my addr 172.21.15.3:0/2317555570 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119b7fe700 1 --2- 172.21.15.3:0/2317555570 >> v2:172.21.15.153:3300/0 conn(0x7f119c0b3e10 0x7f119c0680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119bfff700 1 -- 172.21.15.3:0/2317555570 >> v2:172.21.15.153:3300/0 conn(0x7f119c0b3e10 msgr2=0x7f119c0680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119bfff700 1 --2- 172.21.15.3:0/2317555570 >> v2:172.21.15.153:3300/0 conn(0x7f119c0b3e10 0x7f119c0680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119bfff700 1 -- 172.21.15.3:0/2317555570 >> v1:172.21.15.3:6789/0 conn(0x7f119c1406b0 legacy=0x7f119c140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119affd700 1 -- 172.21.15.3:0/2317555570 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635811322 0 0) 0x7f119c0685f0 con 0x7f119c1406b0 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119bfff700 1 -- 172.21.15.3:0/2317555570 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c0cd940 con 0x7f119c0b3a40 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119bfff700 1 --2- 172.21.15.3:0/2317555570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c0b3a40 0x7f119c0ccd30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f119c0cd270 tx=0x7f118c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34deaf3d3c0c9ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119affd700 1 -- 172.21.15.3:0/2317555570 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c00b6b0 con 0x7f119c0b3a40 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119affd700 1 -- 172.21.15.3:0/2317555570 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c00b810 con 0x7f119c0b3a40 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f119affd700 1 -- 172.21.15.3:0/2317555570 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c00bbb0 con 0x7f119c0b3a40 2023-10-06T04:54:51.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 -- 172.21.15.3:0/2317555570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c0b3a40 msgr2=0x7f119c0ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2317555570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c0b3a40 0x7f119c0ccd30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f119c0cd270 tx=0x7f118c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 -- 172.21.15.3:0/2317555570 shutdown_connections 2023-10-06T04:54:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2317555570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c0b3a40 0x7f119c0ccd30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2317555570 >> v2:172.21.15.153:3300/0 conn(0x7f119c0b3e10 0x7f119c0680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 -- 172.21.15.3:0/2317555570 >> 172.21.15.3:0/2317555570 conn(0x7f119c0ce270 msgr2=0x7f119c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 -- 172.21.15.3:0/2317555570 shutdown_connections 2023-10-06T04:54:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.785+0000 7f11a23e2700 1 -- 172.21.15.3:0/2317555570 wait complete. 2023-10-06T04:54:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.786+0000 7f11a23e2700 1 Processor -- start 2023-10-06T04:54:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a23e2700 1 -- start start 2023-10-06T04:54:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a23e2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f119c0b3a40 0x7f119c0a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a23e2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f119c0b3e10 0x7f119c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a23e2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c1406b0 0x7f119c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a23e2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c06de70 con 0x7f119c1406b0 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a23e2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c169910 con 0x7f119c0b3a40 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a23e2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c16a660 con 0x7f119c0b3e10 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a097f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c1406b0 0x7f119c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.787+0000 7f11a097f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c1406b0 0x7f119c0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45074/0 (socket says 172.21.15.3:45074) 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a097f700 1 -- 172.21.15.3:0/2403772600 learned_addr learned my addr 172.21.15.3:0/2403772600 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f119b7fe700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f119c0b3e10 0x7f119c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f119bfff700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f119c0b3a40 0x7f119c0a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a097f700 1 -- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f119c0b3e10 msgr2=0x7f119c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a097f700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f119c0b3e10 0x7f119c0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a097f700 1 -- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f119c0b3a40 msgr2=0x7f119c0a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a097f700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f119c0b3a40 0x7f119c0a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a097f700 1 -- 172.21.15.3:0/2403772600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c009420 con 0x7f119c1406b0 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f119bfff700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f119c0b3a40 0x7f119c0a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a097f700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c1406b0 0x7f119c0c9d20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f119400cbf0 tx=0x7f1194001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f1193fff700 1 -- 172.21.15.3:0/2403772600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400a8d0 con 0x7f119c1406b0 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f1193fff700 1 -- 172.21.15.3:0/2403772600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119400aa30 con 0x7f119c1406b0 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f1193fff700 1 -- 172.21.15.3:0/2403772600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400adf0 con 0x7f119c1406b0 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c169b80 con 0x7f119c1406b0 2023-10-06T04:54:51.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.788+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c16a140 con 0x7f119c1406b0 2023-10-06T04:54:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.790+0000 7f1193fff700 1 -- 172.21.15.3:0/2403772600 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1194007300 con 0x7f119c1406b0 2023-10-06T04:54:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.790+0000 7f1193fff700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f117c066f30 0x7f117c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.791+0000 7f1193fff700 1 -- 172.21.15.3:0/2403772600 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1194084a80 con 0x7f119c1406b0 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.791+0000 7f119bfff700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f117c066f30 0x7f117c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.791+0000 7f119bfff700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f117c066f30 0x7f117c0693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f118c0093f0 tx=0x7f118c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.792+0000 7f11ab6dab80 1 Processor -- start 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.792+0000 7f11ab6dab80 1 -- start start 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692ca5a90 0x561692ca5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c619a0 0x561692c5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692c5eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692bdec80 con 0x561692c5fbd0 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692bb5a70 con 0x561692c619a0 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x561692bb3270 con 0x561692ca5a90 2023-10-06T04:54:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692c5eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692c5eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45090/0 (socket says 172.21.15.3:45090) 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 -- 172.21.15.3:0/3742363767 learned_addr learned my addr 172.21.15.3:0/3742363767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f119bfff700 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692ca5a90 0x561692ca5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f119b7fe700 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c619a0 0x561692c5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 -- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692ca5a90 msgr2=0x561692ca5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692ca5a90 0x561692ca5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 -- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c619a0 msgr2=0x561692c5e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c619a0 0x561692c5e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.793+0000 7f11a097f700 1 -- 172.21.15.3:0/3742363767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561692a49270 con 0x561692c5fbd0 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.794+0000 7f11a097f700 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692c5eac0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f119400afd0 tx=0x7f1194082d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f8d3370c05a1f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.794+0000 7f11917fa700 1 -- 172.21.15.3:0/3742363767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194055830 con 0x561692c5fbd0 2023-10-06T04:54:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.794+0000 7f11917fa700 1 -- 172.21.15.3:0/3742363767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194058a80 con 0x561692c5fbd0 2023-10-06T04:54:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.794+0000 7f11917fa700 1 -- 172.21.15.3:0/3742363767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119405c7d0 con 0x561692c5fbd0 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.794+0000 7f11ab6dab80 1 -- 172.21.15.3:0/3742363767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 msgr2=0x561692c5eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.794+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692c5eac0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f119400afd0 tx=0x7f1194082d00 comp rx=0 tx=0).stop 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 -- 172.21.15.3:0/3742363767 shutdown_connections 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692c5eac0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692ca5a90 0x561692ca5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/3742363767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c619a0 0x561692c5e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 -- 172.21.15.3:0/3742363767 >> 172.21.15.3:0/3742363767 conn(0x561692c60070 msgr2=0x561692c60440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 -- 172.21.15.3:0/3742363767 shutdown_connections 2023-10-06T04:54:51.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 -- 172.21.15.3:0/3742363767 wait complete. 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 -- start start 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.795+0000 7f11ab6dab80 1 Processor -- start 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692cb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692c8ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c8b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692baf060 con 0x561692c5fbd0 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692b5a890 con 0x561692ca5a90 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x561692b75ea0 con 0x561692c619a0 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f119bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692cb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11ab6dab80 10 client.0 _ll_register_callbacks cb 0x561692a4bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:54:51.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f119bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692cb5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45102/0 (socket says 172.21.15.3:45102) 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f119bfff700 1 -- 172.21.15.3:0/1815414188 learned_addr learned my addr 172.21.15.3:0/1815414188 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f119b7fe700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692c8ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.796+0000 7f11a097f700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c8b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f119bfff700 1 -- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 msgr2=0x561692c8ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f119bfff700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692c8ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f119bfff700 1 -- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 msgr2=0x561692c8b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f119bfff700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c8b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f119bfff700 1 -- 172.21.15.3:0/1815414188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c16a140 con 0x561692c5fbd0 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f119b7fe700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692c8ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f119bfff700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692cb5eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f119c0676c0 tx=0x7f118c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb61ead5257ca426 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f117a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f117a7fc700 1 -- 172.21.15.3:0/1815414188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c02f070 con 0x561692c5fbd0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f117a7fc700 1 -- 172.21.15.3:0/1815414188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f118c026640 con 0x561692c5fbd0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561692c53690 con 0x561692c5fbd0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f117a7fc700 1 -- 172.21.15.3:0/1815414188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f118c020070 con 0x561692c5fbd0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 10 client.5883 fetch_fsmap learned FSMap version 31 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x5616929ebfe0 con 0x561692c5fbd0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f117a7fc700 1 -- 172.21.15.3:0/1815414188 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561692c53690 con 0x561692c5fbd0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f117a7fc700 1 -- 172.21.15.3:0/1815414188 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f118c026a50 con 0x561692c5fbd0 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 10 client.5883 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 10 client.5883 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 20 client.5883 populate_metadata read hostname 'smithi003' 2023-10-06T04:54:51.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c5e550 0x561692c5e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 4 client.5883 mds_command: new command op to 5721 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.797+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 --> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561692c305c0 con 0x561692c5e550 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.798+0000 7f119b7fe700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c5e550 0x561692c5e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.798+0000 7f117a7fc700 10 client.5883 ms_handle_connect on v2:172.21.15.3:6833/3909985257 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.798+0000 7f119b7fe700 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c5e550 0x561692c5e920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca933421e27ae07 server_cookie=e9ae843b5a206d89 in_seq=0 out_seq=0 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.798+0000 7f117a7fc700 1 -- 172.21.15.3:0/1815414188 <== mds.0 v2:172.21.15.3:6833/3909985257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561692c305c0 con 0x561692c5e550 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.798+0000 7f117a7fc700 10 client.5883 handle_command_reply: tid=0 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.798+0000 7f11ab6dab80 1 client.5883 shutdown 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 msgr2=0x561692cb5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692cb5eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f119c0676c0 tx=0x7f118c020000 comp rx=0 tx=0).stop 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 shutdown_connections 2023-10-06T04:54:51.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c5e550 0x561692c5e920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f117a7fc700 0 client.5883 ms_handle_reset on v2:172.21.15.3:6833/3909985257 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c8b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c5fbd0 0x561692cb5eb0 secure :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f119c0676c0 tx=0x7f118c020000 comp rx=0 tx=0).stop 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/1815414188 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692c8ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 >> 172.21.15.3:0/1815414188 conn(0x561692c60070 msgr2=0x561692c5aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 shutdown_connections 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 1 -- 172.21.15.3:0/1815414188 wait complete. 2023-10-06T04:54:51.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.799+0000 7f11ab6dab80 20 client.5883 trim_cache size 0 max 16384 2023-10-06T04:54:51.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.816+0000 7f11ab6dab80 1 Processor -- start 2023-10-06T04:54:51.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.816+0000 7f11ab6dab80 1 -- start start 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.816+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692ca2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.816+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692ca5a90 0x561692ca2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.816+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c8b570 0x561692c88a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.816+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692a31e30 con 0x561692ca5a90 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.816+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692c893c0 con 0x561692c8b570 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x561692ca27a0 con 0x561692c619a0 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692ca5a90 0x561692ca2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692ca5a90 0x561692ca2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45106/0 (socket says 172.21.15.3:45106) 2023-10-06T04:54:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 -- 172.21.15.3:0/517275317 learned_addr learned my addr 172.21.15.3:0/517275317 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f11a097f700 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c8b570 0x561692c88a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119bfff700 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692ca2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 -- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 msgr2=0x561692ca2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692ca2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 -- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c8b570 msgr2=0x561692c88a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c8b570 0x561692c88a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 -- 172.21.15.3:0/517275317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561692b7d780 con 0x561692ca5a90 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f119b7fe700 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692ca5a90 0x561692ca2260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f118400a4d0 tx=0x7f1184008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9c3253fce7e84e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f1191ffb700 1 -- 172.21.15.3:0/517275317 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118401a040 con 0x561692ca5a90 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f1191ffb700 1 -- 172.21.15.3:0/517275317 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118400c040 con 0x561692ca5a90 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.817+0000 7f1191ffb700 1 -- 172.21.15.3:0/517275317 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184014660 con 0x561692ca5a90 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 -- 172.21.15.3:0/517275317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692ca5a90 msgr2=0x561692ca2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692ca5a90 0x561692ca2260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f118400a4d0 tx=0x7f1184008040 comp rx=0 tx=0).stop 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 -- 172.21.15.3:0/517275317 shutdown_connections 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692c8b570 0x561692c88a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c619a0 0x561692ca2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/517275317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692ca5a90 0x561692ca2260 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 -- 172.21.15.3:0/517275317 >> 172.21.15.3:0/517275317 conn(0x561692c60070 msgr2=0x561692ca3770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 -- 172.21.15.3:0/517275317 shutdown_connections 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 -- 172.21.15.3:0/517275317 wait complete. 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.818+0000 7f11ab6dab80 1 -- start start 2023-10-06T04:54:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 1 Processor -- start 2023-10-06T04:54:51.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c619a0 0x561692d0aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c8b570 0x561692c5b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c5bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692c83a40 con 0x561692c619a0 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561692d0b050 con 0x561692ca5a90 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x561692c86da0 con 0x561692c8b570 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f11ab6dab80 10 client.0 _ll_register_callbacks cb 0x561692a4bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f119bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c619a0 0x561692d0aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f119bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c619a0 0x561692d0aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45114/0 (socket says 172.21.15.3:45114) 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.819+0000 7f119bfff700 1 -- 172.21.15.3:0/496590820 learned_addr learned my addr 172.21.15.3:0/496590820 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f119b7fe700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c8b570 0x561692c5b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11a097f700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c5bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f119bfff700 1 -- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c8b570 msgr2=0x561692c5b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f119bfff700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c8b570 0x561692c5b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f119bfff700 1 -- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 msgr2=0x561692c5bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f119bfff700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c5bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f119bfff700 1 -- 172.21.15.3:0/496590820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840135e0 con 0x561692c619a0 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11a097f700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c5bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f119bfff700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c619a0 0x561692d0aa30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x561692c59c90 tx=0x7f118c00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212691fcd1db0dca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f117a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f117a7fc700 1 -- 172.21.15.3:0/496590820 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c02f070 con 0x561692c619a0 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f117a7fc700 1 -- 172.21.15.3:0/496590820 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f118c026640 con 0x561692c619a0 2023-10-06T04:54:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1194058a80 con 0x561692c619a0 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f117a7fc700 1 -- 172.21.15.3:0/496590820 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f118c020040 con 0x561692c619a0 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 10 client.5889 fetch_fsmap learned FSMap version 31 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x561692b64470 con 0x561692c619a0 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f117a7fc700 1 -- 172.21.15.3:0/496590820 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194058a80 con 0x561692c619a0 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f117a7fc700 1 -- 172.21.15.3:0/496590820 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f118c03c630 con 0x561692c619a0 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 10 client.5889 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 10 client.5889 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 20 client.5889 populate_metadata read hostname 'smithi003' 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c9b820 0x561692c9bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 4 client.5889 mds_command: new command op to 5721 tid=0[{"prefix": "status"}] 2023-10-06T04:54:51.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.820+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 --> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561692b51e00 con 0x561692c9b820 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f119b7fe700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c9b820 0x561692c9bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f119b7fe700 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c9b820 0x561692c9bbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bfe92a7bca4760f server_cookie=30d2fd5994a2bdc0 in_seq=0 out_seq=0 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f117a7fc700 10 client.5889 ms_handle_connect on v2:172.21.15.3:6833/3909985257 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f117a7fc700 1 -- 172.21.15.3:0/496590820 <== mds.0 v2:172.21.15.3:6833/3909985257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561692b51e00 con 0x561692c9b820 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f117a7fc700 10 client.5889 handle_command_reply: tid=0 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 client.5889 shutdown 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c619a0 msgr2=0x561692d0aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c619a0 0x561692d0aa30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x561692c59c90 tx=0x7f118c00b050 comp rx=0 tx=0).stop 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 shutdown_connections 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:6833/3909985257,v1:172.21.15.3:6835/3909985257] conn(0x561692c9b820 0x561692c9bbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561692ca5a90 0x561692c5bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561692c619a0 0x561692d0aa30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 --2- 172.21.15.3:0/496590820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x561692c8b570 0x561692c5b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f117a7fc700 0 client.5889 ms_handle_reset on v2:172.21.15.3:6833/3909985257 2023-10-06T04:54:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 >> 172.21.15.3:0/496590820 conn(0x561692c60070 msgr2=0x561692c5a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 shutdown_connections 2023-10-06T04:54:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 1 -- 172.21.15.3:0/496590820 wait complete. 2023-10-06T04:54:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.821+0000 7f11ab6dab80 20 client.5889 trim_cache size 0 max 16384 2023-10-06T04:54:51.831 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T04:54:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T04:54:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5721, 2023-10-06T04:54:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T04:54:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T04:54:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T04:54:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.497642749000001, 2023-10-06T04:54:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 31, 2023-10-06T04:54:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 58, 2023-10-06T04:54:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 58, 2023-10-06T04:54:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.574438249 2023-10-06T04:54:51.832 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:51.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f117c066f30 msgr2=0x7f117c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f117c066f30 0x7f117c0693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f118c0093f0 tx=0x7f118c017000 comp rx=0 tx=0).stop 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c1406b0 msgr2=0x7f119c0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c1406b0 0x7f119c0c9d20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f119400cbf0 tx=0x7f1194001980 comp rx=0 tx=0).stop 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 shutdown_connections 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f117c066f30 0x7f117c0693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f119c0b3a40 0x7f119c0a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f119c1406b0 0x7f119c0c9d20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 --2- 172.21.15.3:0/2403772600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f119c0b3e10 0x7f119c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 >> 172.21.15.3:0/2403772600 conn(0x7f119c0ce270 msgr2=0x7f119c13fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 shutdown_connections 2023-10-06T04:54:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.822+0000 7f11a23e2700 1 -- 172.21.15.3:0/2403772600 wait complete. 2023-10-06T04:54:51.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:54:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.953+0000 7ff26359e700 1 Processor -- start 2023-10-06T04:54:51.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.953+0000 7ff26359e700 1 -- start start 2023-10-06T04:54:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff26359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2640cb780 0x7ff264158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff26359e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff264159500 0x7ff2640cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff26359e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2640cf130 con 0x7ff2640cb3b0 2023-10-06T04:54:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff26359e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2640ce520 con 0x7ff2640cb780 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff26359e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2640ce800 con 0x7ff264159500 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2640cb780 0x7ff264158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2640cb780 0x7ff264158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45128/0 (socket says 172.21.15.3:45128) 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 -- 172.21.15.3:0/638040203 learned_addr learned my addr 172.21.15.3:0/638040203 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff262d9d700 1 --2- 172.21.15.3:0/638040203 >> v2:172.21.15.153:3300/0 conn(0x7ff264159500 0x7ff2640cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 -- 172.21.15.3:0/638040203 >> v2:172.21.15.153:3300/0 conn(0x7ff264159500 msgr2=0x7ff2640cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 --2- 172.21.15.3:0/638040203 >> v2:172.21.15.153:3300/0 conn(0x7ff264159500 0x7ff2640cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 -- 172.21.15.3:0/638040203 >> v1:172.21.15.153:6789/0 conn(0x7ff2640cb3b0 legacy=0x7ff2640cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 -- 172.21.15.3:0/638040203 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2640ceae0 con 0x7ff2640cb780 2023-10-06T04:54:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.954+0000 7ff261d9b700 1 --2- 172.21.15.3:0/638040203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2640cb780 0x7ff264158fc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff2640cbfd0 tx=0x7ff24c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb03c6c19dd6c5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26159a700 1 -- 172.21.15.3:0/638040203 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00b860 con 0x7ff2640cb780 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26159a700 1 -- 172.21.15.3:0/638040203 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c00b9c0 con 0x7ff2640cb780 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26159a700 1 -- 172.21.15.3:0/638040203 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00bd60 con 0x7ff2640cb780 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 -- 172.21.15.3:0/638040203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2640cb780 msgr2=0x7ff264158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 --2- 172.21.15.3:0/638040203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2640cb780 0x7ff264158fc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff2640cbfd0 tx=0x7ff24c009250 comp rx=0 tx=0).stop 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 -- 172.21.15.3:0/638040203 shutdown_connections 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 --2- 172.21.15.3:0/638040203 >> v2:172.21.15.153:3300/0 conn(0x7ff264159500 0x7ff2640cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 --2- 172.21.15.3:0/638040203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2640cb780 0x7ff264158fc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 -- 172.21.15.3:0/638040203 >> 172.21.15.3:0/638040203 conn(0x7ff264070310 msgr2=0x7ff264070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 -- 172.21.15.3:0/638040203 shutdown_connections 2023-10-06T04:54:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.955+0000 7ff26359e700 1 -- 172.21.15.3:0/638040203 wait complete. 2023-10-06T04:54:51.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.956+0000 7ff26359e700 1 Processor -- start 2023-10-06T04:54:51.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.956+0000 7ff26359e700 1 -- start start 2023-10-06T04:54:51.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff26359e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2640cb3b0 0x7ff2640c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff26359e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2640cb780 0x7ff264150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff26359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff264159500 0x7ff2641511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff26359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640b3770 con 0x7ff264159500 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff26359e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640b31f0 con 0x7ff2640cb3b0 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff26359e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640b3490 con 0x7ff2640cb780 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff264159500 0x7ff2641511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff264159500 0x7ff2641511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45132/0 (socket says 172.21.15.3:45132) 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 -- 172.21.15.3:0/1395840242 learned_addr learned my addr 172.21.15.3:0/1395840242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff26259c700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2640cb3b0 0x7ff2640c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff261d9b700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2640cb780 0x7ff264150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 -- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2640cb780 msgr2=0x7ff264150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2640cb780 0x7ff264150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 -- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2640cb3b0 msgr2=0x7ff2640c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2640cb3b0 0x7ff2640c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.957+0000 7ff262d9d700 1 -- 172.21.15.3:0/1395840242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0094f0 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.958+0000 7ff262d9d700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff264159500 0x7ff2641511d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff254007f20 tx=0x7ff254009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.958+0000 7ff25a7fc700 1 -- 172.21.15.3:0/1395840242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25400ed30 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.958+0000 7ff25a7fc700 1 -- 172.21.15.3:0/1395840242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254014420 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.958+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2640c7400 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.958+0000 7ff25a7fc700 1 -- 172.21.15.3:0/1395840242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2540147e0 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.958+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2640c98d0 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.959+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26404c380 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.959+0000 7ff25a7fc700 1 -- 172.21.15.3:0/1395840242 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff254014b50 con 0x7ff264159500 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.959+0000 7ff25a7fc700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff244066f30 0x7ff2440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.960+0000 7ff25a7fc700 1 -- 172.21.15.3:0/1395840242 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff254084e40 con 0x7ff264159500 2023-10-06T04:54:51.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.960+0000 7ff26259c700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff244066f30 0x7ff2440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:51.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.960+0000 7ff26259c700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff244066f30 0x7ff2440693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff25c003860 tx=0x7ff25c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:51.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:51.964+0000 7ff25a7fc700 1 -- 172.21.15.3:0/1395840242 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff254051f50 con 0x7ff264159500 2023-10-06T04:54:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.201+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2640cedc0 con 0x7ff264159500 2023-10-06T04:54:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.202+0000 7ff25a7fc700 1 -- 172.21.15.3:0/1395840242 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff254055570 con 0x7ff264159500 2023-10-06T04:54:52.203 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff244066f30 msgr2=0x7ff2440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff244066f30 0x7ff2440693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff25c003860 tx=0x7ff25c009040 comp rx=0 tx=0).stop 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff264159500 msgr2=0x7ff2641511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff264159500 0x7ff2641511d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff254007f20 tx=0x7ff254009860 comp rx=0 tx=0).stop 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 shutdown_connections 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff244066f30 0x7ff2440693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2640cb3b0 0x7ff2640c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff264159500 0x7ff2641511d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 --2- 172.21.15.3:0/1395840242 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2640cb780 0x7ff264150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 >> 172.21.15.3:0/1395840242 conn(0x7ff264070310 msgr2=0x7ff2640cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 shutdown_connections 2023-10-06T04:54:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:52.204+0000 7ff26359e700 1 -- 172.21.15.3:0/1395840242 wait complete. 2023-10-06T04:54:52.217 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:54:52.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:54:52.218 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:54:52.218 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:54:52.218 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:52.218 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:54:52.218 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:54:52.218 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:54:52.218 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:52.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:52.218 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:54:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:54:52.234 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42338sec preferred_lft 42338sec 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:54:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:54:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:52.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:52.237 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:54:52.298 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:54:52.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:52.300 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:54:52.356 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:54:52.356 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:54:52.356 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:54:52.356 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:54:52.356 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:54:52.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:52.357 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:54:52.357 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:54:52.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:54:52.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:54:52.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:54:52.357 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:54:52.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:52.459 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:54:52.473 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:54:52.473 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:54:52.473 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:52.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:52.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:54:52.539 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:54:52.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:52.539 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:52.595 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:54:52.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:52.595 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:52.650 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:54:52.716 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:52.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:52.781 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:52.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:52.782 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:54:52.837 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:54:52.837 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:54:52.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:54:52.905 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:52.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:52.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:54:52.961+0000 7fe13c29a540 -1 init, newargv = 0x55618ff938c0 newargc=15 2023-10-06T04:54:52.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[110064]: starting ceph client 2023-10-06T04:54:52.968 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[110064]: starting fuse 2023-10-06T04:54:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:52.974 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:52.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:52.975 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:54:53.030 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:54:53.031 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:53.031 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:53.032 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:53.033 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:53.033 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:53.033 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:53.033 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:53.153 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.110064.asok 2023-10-06T04:54:53.335 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.110064.asok 2023-10-06T04:54:53.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:53.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.110064.asok status 2023-10-06T04:54:53.490 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:53.490 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:54:53.490 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "110064", 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5904, 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:53.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5904 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2120956760 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:53.492 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2120956760 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5904 192.168.0.1:0/2120956760", 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2120956760", 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 31, 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 58, 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:54:53.493 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:53.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:53.500 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:54:53.500 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:53.516 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:54:53.516 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:53.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:53.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:54:53.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:54:53.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.739+0000 7f782ee0a700 1 Processor -- start 2023-10-06T04:54:53.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.739+0000 7f782ee0a700 1 -- start start 2023-10-06T04:54:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.740+0000 7f782ee0a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f782806ec40 0x7f7828157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.740+0000 7f782ee0a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782806aed0 con 0x7f782806aa80 2023-10-06T04:54:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.740+0000 7f782ee0a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7828069790 con 0x7f7828155980 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.740+0000 7f782ee0a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280ae9e0 con 0x7f782806ec40 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.740+0000 7f782d3a7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f782806ec40 0x7f7828157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.740+0000 7f7827fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f782806aa80 0x7f782806e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47408/0 (socket says 172.21.15.3:47408) 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.740+0000 7f7827fff700 1 -- 172.21.15.3:0/3519920764 learned_addr learned my addr 172.21.15.3:0/3519920764 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f78277fe700 1 -- 172.21.15.3:0/3519920764 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783927041 0 0) 0x7f782806aed0 con 0x7f782806aa80 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f78277fe700 1 -- 172.21.15.3:0/3519920764 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7814004680 con 0x7f782806aa80 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782d3a7700 1 -- 172.21.15.3:0/3519920764 >> v1:172.21.15.153:6789/0 conn(0x7f7828155980 legacy=0x7f7828156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782d3a7700 1 -- 172.21.15.3:0/3519920764 >> v1:172.21.15.3:6789/0 conn(0x7f782806aa80 legacy=0x7f782806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782d3a7700 1 -- 172.21.15.3:0/3519920764 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280699e0 con 0x7f782806ec40 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782d3a7700 1 --2- 172.21.15.3:0/3519920764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f7828157500 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f78200098e0 tx=0x7f78200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240b2d4da36a692c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f78277fe700 1 -- 172.21.15.3:0/3519920764 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000b640 con 0x7f782806ec40 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f78277fe700 1 -- 172.21.15.3:0/3519920764 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782000b7a0 con 0x7f782806ec40 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f78277fe700 1 -- 172.21.15.3:0/3519920764 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000bb40 con 0x7f782806ec40 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782ee0a700 1 -- 172.21.15.3:0/3519920764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 msgr2=0x7f7828157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3519920764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f7828157500 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f78200098e0 tx=0x7f78200091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782ee0a700 1 -- 172.21.15.3:0/3519920764 shutdown_connections 2023-10-06T04:54:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.741+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3519920764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f7828157500 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.742+0000 7f782ee0a700 1 -- 172.21.15.3:0/3519920764 >> 172.21.15.3:0/3519920764 conn(0x7f782806c370 msgr2=0x7f782806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.742+0000 7f782ee0a700 1 -- 172.21.15.3:0/3519920764 shutdown_connections 2023-10-06T04:54:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.742+0000 7f782ee0a700 1 -- 172.21.15.3:0/3519920764 wait complete. 2023-10-06T04:54:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.743+0000 7f782ee0a700 1 Processor -- start 2023-10-06T04:54:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.743+0000 7f782ee0a700 1 -- start start 2023-10-06T04:54:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.743+0000 7f782ee0a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f782806aa80 0x7f782814f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.743+0000 7f782ee0a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f78280cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782ee0a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7828155980 0x7f78280cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782ee0a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280c36e0 con 0x7f782806ec40 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782ee0a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782814fbd0 con 0x7f7828155980 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782ee0a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78280c30d0 con 0x7f782806aa80 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f78280cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f78280cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55956/0 (socket says 172.21.15.3:55956) 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 -- 172.21.15.3:0/3161089209 learned_addr learned my addr 172.21.15.3:0/3161089209 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782d3a7700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7828155980 0x7f78280cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782cba6700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f782806aa80 0x7f782814f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 -- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f782806aa80 msgr2=0x7f782814f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f782806aa80 0x7f782814f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 -- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7828155980 msgr2=0x7f78280cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7828155980 0x7f78280cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 -- 172.21.15.3:0/3161089209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820009420 con 0x7f782806ec40 2023-10-06T04:54:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782d3a7700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7828155980 0x7f78280cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782cba6700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f782806aa80 0x7f782814f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7827fff700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f78280cbd60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f781800cc40 tx=0x7f7818001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7824ff9700 1 -- 172.21.15.3:0/3161089209 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781800a9c0 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7824ff9700 1 -- 172.21.15.3:0/3161089209 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781800ab40 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78280b0ad0 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f7824ff9700 1 -- 172.21.15.3:0/3161089209 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78180070f0 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.744+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78280b0db0 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.745+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7828071680 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.746+0000 7f7824ff9700 1 -- 172.21.15.3:0/3161089209 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7818007520 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.746+0000 7f7824ff9700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7808066f30 0x7f78080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.746+0000 7f7824ff9700 1 -- 172.21.15.3:0/3161089209 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7818084b00 con 0x7f782806ec40 2023-10-06T04:54:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.746+0000 7f782cba6700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7808066f30 0x7f78080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:53.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.747+0000 7f782cba6700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7808066f30 0x7f78080693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7810001840 tx=0x7f7810009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:53.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.750+0000 7f7824ff9700 1 -- 172.21.15.3:0/3161089209 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7818051c90 con 0x7f782806ec40 2023-10-06T04:54:53.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.987+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78280c3820 con 0x7f782806ec40 2023-10-06T04:54:53.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.987+0000 7f7824ff9700 1 -- 172.21.15.3:0/3161089209 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f78180552b0 con 0x7f782806ec40 2023-10-06T04:54:53.989 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7808066f30 msgr2=0x7f78080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7808066f30 0x7f78080693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7810001840 tx=0x7f7810009040 comp rx=0 tx=0).stop 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 msgr2=0x7f78280cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f78280cbd60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f781800cc40 tx=0x7f7818001a70 comp rx=0 tx=0).stop 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 shutdown_connections 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7808066f30 0x7f78080693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7828155980 0x7f78280cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f782806aa80 0x7f782814f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 --2- 172.21.15.3:0/3161089209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f782806ec40 0x7f78280cbd60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 >> 172.21.15.3:0/3161089209 conn(0x7f782806c370 msgr2=0x7f78280c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:53.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 shutdown_connections 2023-10-06T04:54:53.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:53.989+0000 7f782ee0a700 1 -- 172.21.15.3:0/3161089209 wait complete. 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:54:54.003 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:54:54.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:54.004 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:54:54.022 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:54:54.022 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:54:54.023 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42339sec preferred_lft 42339sec 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:54.024 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:54:54.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:54.025 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:54:54.090 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:54:54.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:54.091 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:54:54.148 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:54:54.149 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:54:54.149 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:54:54.149 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:54:54.149 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:54:54.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:54.149 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:54:54.149 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:54:54.150 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:54:54.150 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:54:54.150 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:54:54.150 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:54:54.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:54.254 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:54:54.269 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T04:54:54.269 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:54:54.270 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:54:54.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:54:54.270 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T04:54:54.334 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:54:54.334 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:54.334 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:54.388 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:54:54.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:54:54.389 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:54.443 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:54:54.507 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:54.507 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:54.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:54.539 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:54.572 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:54.572 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:54.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:54.573 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:54:54.627 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:54:54.628 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:54:54.676 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:54:54.700 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:54:54.700 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:54:54.757 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T04:54:54.756+0000 7f8c6f54f540 -1 init, newargv = 0x56300bdfc190 newargc=15 2023-10-06T04:54:54.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[99376]: starting ceph client 2023-10-06T04:54:54.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[99376]: starting fuse 2023-10-06T04:54:54.765 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:54:54.766 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:54:54.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:54.767 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:54:54.821 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:54:54.822 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:54:54.822 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:54.823 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:54:54.824 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:54:54.824 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:54:54.824 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:54.824 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:54:54.824 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T04:54:54.824 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:54:54.942 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.99376.asok 2023-10-06T04:54:55.125 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.99376.asok 2023-10-06T04:54:55.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:55.126 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.99376.asok status 2023-10-06T04:54:55.276 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:54:55.276 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:54:55.276 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:54:55.276 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "99376", 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 5716, 2023-10-06T04:54:55.277 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 5716 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 3779663251 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:54:55.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 3779663251 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.5716 192.168.0.1:0/3779663251", 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/3779663251", 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 31, 2023-10-06T04:54:55.279 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 58, 2023-10-06T04:54:55.280 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:54:55.280 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:54:55.280 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:54:55.280 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:54:55.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:55.286 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:54:55.286 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:55.301 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:54:55.302 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:55.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:55.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:54:55.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-10-06T04:54:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.499+0000 7f9f7ef6c700 1 Processor -- start 2023-10-06T04:54:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.499+0000 7f9f7ef6c700 1 -- start start 2023-10-06T04:54:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7ef6c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f78150690 0x7f9f78150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7ef6c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f7815d540 0x7f9f780af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7ef6c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7815a290 con 0x7f9f780cba50 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7ef6c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f780d0020 con 0x7f9f7815d540 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7ef6c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78159f40 con 0x7f9f78150690 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7d509700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f7815d540 0x7f9f780af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7d509700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f7815d540 0x7f9f780af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55966/0 (socket says 172.21.15.3:55966) 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7d509700 1 -- 172.21.15.3:0/349070031 learned_addr learned my addr 172.21.15.3:0/349070031 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.500+0000 7f9f7cd08700 1 --2- 172.21.15.3:0/349070031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f78150690 0x7f9f78150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7d509700 1 -- 172.21.15.3:0/349070031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f78150690 msgr2=0x7f9f78150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7d509700 1 --2- 172.21.15.3:0/349070031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f78150690 0x7f9f78150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7d509700 1 -- 172.21.15.3:0/349070031 >> v1:172.21.15.153:6789/0 conn(0x7f9f780cba50 legacy=0x7f9f7815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7d509700 1 -- 172.21.15.3:0/349070031 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7815dcb0 con 0x7f9f7815d540 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7cd08700 1 --2- 172.21.15.3:0/349070031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f78150690 0x7f9f78150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7d509700 1 --2- 172.21.15.3:0/349070031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f7815d540 0x7f9f780af190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009f20 tx=0x7f9f70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b962cc9d26e119 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f777fe700 1 -- 172.21.15.3:0/349070031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000e070 con 0x7f9f7815d540 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f777fe700 1 -- 172.21.15.3:0/349070031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7000ba80 con 0x7f9f7815d540 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f777fe700 1 -- 172.21.15.3:0/349070031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f700092e0 con 0x7f9f7815d540 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/349070031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f7815d540 msgr2=0x7f9f780af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.501+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/349070031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f7815d540 0x7f9f780af190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009f20 tx=0x7f9f70009840 comp rx=0 tx=0).stop 2023-10-06T04:54:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.502+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/349070031 shutdown_connections 2023-10-06T04:54:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.502+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/349070031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f7815d540 0x7f9f780af190 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.502+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/349070031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f78150690 0x7f9f78150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.502+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/349070031 >> 172.21.15.3:0/349070031 conn(0x7f9f78070310 msgr2=0x7f9f78070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.502+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/349070031 shutdown_connections 2023-10-06T04:54:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.502+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/349070031 wait complete. 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.503+0000 7f9f7ef6c700 1 Processor -- start 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.503+0000 7f9f7ef6c700 1 -- start start 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7ef6c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f780cba50 0x7f9f78152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7ef6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f78150690 0x7f9f780c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7ef6c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f7815d540 0x7f9f780c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7ef6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78169a00 con 0x7f9f78150690 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7ef6c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78153b10 con 0x7f9f7815d540 2023-10-06T04:54:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7ef6c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78153040 con 0x7f9f780cba50 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f78150690 0x7f9f780c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f78150690 0x7f9f780c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55980/0 (socket says 172.21.15.3:55980) 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 -- 172.21.15.3:0/2758110176 learned_addr learned my addr 172.21.15.3:0/2758110176 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7d509700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f7815d540 0x7f9f780c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f7cd08700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f780cba50 0x7f9f78152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 -- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f780cba50 msgr2=0x7f9f78152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f780cba50 0x7f9f78152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 -- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f7815d540 msgr2=0x7f9f780c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f7815d540 0x7f9f780c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 -- 172.21.15.3:0/2758110176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f70009ae0 con 0x7f9f78150690 2023-10-06T04:54:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f77fff700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f78150690 0x7f9f780c5550 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c590 tx=0x7f9f60007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.504+0000 7f9f74ff9700 1 -- 172.21.15.3:0/2758110176 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60011590 con 0x7f9f78150690 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.505+0000 7f9f74ff9700 1 -- 172.21.15.3:0/2758110176 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f60011730 con 0x7f9f78150690 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.505+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f78169c70 con 0x7f9f78150690 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.505+0000 7f9f74ff9700 1 -- 172.21.15.3:0/2758110176 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60011af0 con 0x7f9f78150690 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.505+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f78153550 con 0x7f9f78150690 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.506+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f78169e00 con 0x7f9f78150690 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.506+0000 7f9f74ff9700 1 -- 172.21.15.3:0/2758110176 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f60012420 con 0x7f9f78150690 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.506+0000 7f9f74ff9700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f58066f30 0x7f9f580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.507+0000 7f9f74ff9700 1 -- 172.21.15.3:0/2758110176 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f60083900 con 0x7f9f78150690 2023-10-06T04:54:55.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.507+0000 7f9f7cd08700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f58066f30 0x7f9f580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.507+0000 7f9f7cd08700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f58066f30 0x7f9f580693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9f780b3700 tx=0x7f9f68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:55.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.511+0000 7f9f74ff9700 1 -- 172.21.15.3:0/2758110176 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f6004b5a0 con 0x7f9f78150690 2023-10-06T04:54:55.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.704+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f9f7815df90 con 0x7f9f78150690 2023-10-06T04:54:55.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.704+0000 7f9f74ff9700 1 -- 172.21.15.3:0/2758110176 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v18) v1 ==== 106+0+4 (secure 0 0 0) 0x7f9f60017070 con 0x7f9f78150690 2023-10-06T04:54:55.706 INFO:teuthology.orchestra.run.smithi003.stdout:100 2023-10-06T04:54:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f58066f30 msgr2=0x7f9f580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f58066f30 0x7f9f580693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9f780b3700 tx=0x7f9f68008040 comp rx=0 tx=0).stop 2023-10-06T04:54:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f78150690 msgr2=0x7f9f780c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f78150690 0x7f9f780c5550 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c590 tx=0x7f9f60007f30 comp rx=0 tx=0).stop 2023-10-06T04:54:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 shutdown_connections 2023-10-06T04:54:55.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f58066f30 0x7f9f580693e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f7815d540 0x7f9f780c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f780cba50 0x7f9f78152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 --2- 172.21.15.3:0/2758110176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f78150690 0x7f9f780c5550 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 >> 172.21.15.3:0/2758110176 conn(0x7f9f78070310 msgr2=0x7f9f780706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 shutdown_connections 2023-10-06T04:54:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.706+0000 7f9f7ef6c700 1 -- 172.21.15.3:0/2758110176 wait complete. 2023-10-06T04:54:55.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2023-10-06T04:54:55.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.840+0000 7f6b80851700 1 Processor -- start 2023-10-06T04:54:55.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.840+0000 7f6b80851700 1 -- start start 2023-10-06T04:54:55.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b80851700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6b7c0cc8b0 0x7f6b7c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b80851700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b7c15dd90 0x7f6b7c0aa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b80851700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7c0abdb0 con 0x7f6b7c15c720 2023-10-06T04:54:55.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b80851700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c070930 con 0x7f6b7c15dd90 2023-10-06T04:54:55.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b80851700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c0ac090 con 0x7f6b7c0cc8b0 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b79d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b7c15dd90 0x7f6b7c0aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b79d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b7c15dd90 0x7f6b7c0aa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55992/0 (socket says 172.21.15.3:55992) 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.841+0000 7f6b79d9b700 1 -- 172.21.15.3:0/2495918310 learned_addr learned my addr 172.21.15.3:0/2495918310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b7a59c700 1 --2- 172.21.15.3:0/2495918310 >> v2:172.21.15.153:3300/0 conn(0x7f6b7c0cc8b0 0x7f6b7c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b79d9b700 1 -- 172.21.15.3:0/2495918310 >> v2:172.21.15.153:3300/0 conn(0x7f6b7c0cc8b0 msgr2=0x7f6b7c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b79d9b700 1 --2- 172.21.15.3:0/2495918310 >> v2:172.21.15.153:3300/0 conn(0x7f6b7c0cc8b0 0x7f6b7c0ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b79d9b700 1 -- 172.21.15.3:0/2495918310 >> v1:172.21.15.153:6789/0 conn(0x7f6b7c15c720 legacy=0x7f6b7c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b79d9b700 1 -- 172.21.15.3:0/2495918310 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c06c510 con 0x7f6b7c15dd90 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b79d9b700 1 --2- 172.21.15.3:0/2495918310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c0aa200 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c069cf0 tx=0x7f6b64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5494400c4fca060 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b7959a700 1 -- 172.21.15.3:0/2495918310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400b860 con 0x7f6b7c15dd90 2023-10-06T04:54:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b7959a700 1 -- 172.21.15.3:0/2495918310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6400b9c0 con 0x7f6b7c15dd90 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b7959a700 1 -- 172.21.15.3:0/2495918310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400bd60 con 0x7f6b7c15dd90 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b80851700 1 -- 172.21.15.3:0/2495918310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15dd90 msgr2=0x7f6b7c0aa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.842+0000 7f6b80851700 1 --2- 172.21.15.3:0/2495918310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c0aa200 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c069cf0 tx=0x7f6b64009250 comp rx=0 tx=0).stop 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.843+0000 7f6b80851700 1 -- 172.21.15.3:0/2495918310 shutdown_connections 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.843+0000 7f6b80851700 1 --2- 172.21.15.3:0/2495918310 >> v2:172.21.15.153:3300/0 conn(0x7f6b7c0cc8b0 0x7f6b7c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.843+0000 7f6b80851700 1 --2- 172.21.15.3:0/2495918310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c0aa200 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.843+0000 7f6b80851700 1 -- 172.21.15.3:0/2495918310 >> 172.21.15.3:0/2495918310 conn(0x7f6b7c066fb0 msgr2=0x7f6b7c0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.843+0000 7f6b80851700 1 -- 172.21.15.3:0/2495918310 shutdown_connections 2023-10-06T04:54:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.843+0000 7f6b80851700 1 -- 172.21.15.3:0/2495918310 wait complete. 2023-10-06T04:54:55.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.844+0000 7f6b80851700 1 Processor -- start 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.844+0000 7f6b80851700 1 -- start start 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b7c0cc8b0 0x7f6b7c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15c720 0x7f6b7c16d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c16def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c155d50 con 0x7f6b7c15c720 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c158160 con 0x7f6b7c15dd90 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c158400 con 0x7f6b7c0cc8b0 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15c720 0x7f6b7c16d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15c720 0x7f6b7c16d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56008/0 (socket says 172.21.15.3:56008) 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 -- 172.21.15.3:0/3196259427 learned_addr learned my addr 172.21.15.3:0/3196259427 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b7ad9d700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c16def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b7a59c700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b7c0cc8b0 0x7f6b7c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 -- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b7c0cc8b0 msgr2=0x7f6b7c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b7c0cc8b0 0x7f6b7c157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 -- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b7c15dd90 msgr2=0x7f6b7c16def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c16def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 -- 172.21.15.3:0/3196259427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b640094f0 con 0x7f6b7c15c720 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b7ad9d700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c16def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b7a59c700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b7c0cc8b0 0x7f6b7c157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b79d9b700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15c720 0x7f6b7c16d9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c06e890 tx=0x7f6b64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b727fc700 1 -- 172.21.15.3:0/3196259427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64011e60 con 0x7f6b7c15c720 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b727fc700 1 -- 172.21.15.3:0/3196259427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6401a630 con 0x7f6b7c15c720 2023-10-06T04:54:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b727fc700 1 -- 172.21.15.3:0/3196259427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6401a9f0 con 0x7f6b7c15c720 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c1586e0 con 0x7f6b7c15c720 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.845+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c158ca0 con 0x7f6b7c15c720 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.846+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c0af7f0 con 0x7f6b7c15c720 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.847+0000 7f6b727fc700 1 -- 172.21.15.3:0/3196259427 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b6402d440 con 0x7f6b7c15c720 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.847+0000 7f6b727fc700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b5c066f30 0x7f6b5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.847+0000 7f6b727fc700 1 -- 172.21.15.3:0/3196259427 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6b64086130 con 0x7f6b7c15c720 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.847+0000 7f6b7a59c700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b5c066f30 0x7f6b5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:55.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.848+0000 7f6b7a59c700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b5c066f30 0x7f6b5c0693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b74006fd0 tx=0x7f6b74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:55.852+0000 7f6b727fc700 1 -- 172.21.15.3:0/3196259427 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b7c0af7f0 con 0x7f6b7c15c720 2023-10-06T04:54:56.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.046+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7f6b7c0af7f0 con 0x7f6b7c15c720 2023-10-06T04:54:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.049+0000 7f6b727fc700 1 -- 172.21.15.3:0/3196259427 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v19)=0 v19) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6b7c0af7f0 con 0x7f6b7c15c720 2023-10-06T04:54:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b5c066f30 msgr2=0x7f6b5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b5c066f30 0x7f6b5c0693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b74006fd0 tx=0x7f6b74008040 comp rx=0 tx=0).stop 2023-10-06T04:54:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15c720 msgr2=0x7f6b7c16d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15c720 0x7f6b7c16d9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c06e890 tx=0x7f6b64017000 comp rx=0 tx=0).stop 2023-10-06T04:54:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 shutdown_connections 2023-10-06T04:54:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6b5c066f30 0x7f6b5c0693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b7c15dd90 0x7f6b7c16def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6b7c0cc8b0 0x7f6b7c157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 --2- 172.21.15.3:0/3196259427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15c720 0x7f6b7c16d9b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 >> 172.21.15.3:0/3196259427 conn(0x7f6b7c066fb0 msgr2=0x7f6b7c14cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 shutdown_connections 2023-10-06T04:54:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:56.052+0000 7f6b80851700 1 -- 172.21.15.3:0/3196259427 wait complete. 2023-10-06T04:54:56.065 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> n = 600 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:54:56.066 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:54:56.067 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:54:56.068 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:54:56.068 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:54:56.068 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:54:56.068 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:54:56.068 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:54:56.068 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:54:56.069 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:54:56.069 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:54:56.069 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:54:56.069 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:54:56.069 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:54:56.069 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:59.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:54:59.127 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:54:59.127 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:54:59.127 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:54:59.127 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:54:59.127 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:54:59.128 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:54:59.129 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:54:59.252 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.110064.asok 2023-10-06T04:54:59.434 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.110064.asok 2023-10-06T04:54:59.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:54:59.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.110064.asok mds_sessions 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5904, 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5904 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:54:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2120956760 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5904 v1:192.168.0.1:0/2120956760", 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/2120956760", 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2023-10-06T04:54:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6833", 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3909985257 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6835", 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3909985257 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:54:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 1201, 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2023-10-06T04:55:53.964070+0000", 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2023-10-06T04:54:53.964070+0000", 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 216, 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:54:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-06T04:54:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 31 2023-10-06T04:54:59.594 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:54:59.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:54:59.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.716+0000 7f351248e700 1 Processor -- start 2023-10-06T04:54:59.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.716+0000 7f351248e700 1 -- start start 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351248e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f350c0cb3b0 0x7f350c0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351248e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0cf130 con 0x7f350c0cb780 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351248e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0ce520 con 0x7f350c1596d0 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351248e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c0ce800 con 0x7f350c0cb3b0 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351148c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f350c0cb3b0 0x7f350c0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f3510c8b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f350c0cb780 0x7f350c158fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47418/0 (socket says 172.21.15.3:47418) 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f3510c8b700 1 -- 172.21.15.3:0/597721841 learned_addr learned my addr 172.21.15.3:0/597721841 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351148c700 1 -- 172.21.15.3:0/597721841 >> v1:172.21.15.153:6789/0 conn(0x7f350c1596d0 legacy=0x7f350c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351148c700 1 -- 172.21.15.3:0/597721841 >> v1:172.21.15.3:6789/0 conn(0x7f350c0cb780 legacy=0x7f350c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351148c700 1 -- 172.21.15.3:0/597721841 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0ceae0 con 0x7f350c0cb3b0 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f3503fff700 1 -- 172.21.15.3:0/597721841 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115887207 0 0) 0x7f350c0cf130 con 0x7f350c0cb780 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.717+0000 7f351148c700 1 --2- 172.21.15.3:0/597721841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb3b0 0x7f350c0cf5c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f35080098e0 tx=0x7f35080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64e8c586488a346 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f3503fff700 1 -- 172.21.15.3:0/597721841 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508009e80 con 0x7f350c0cb3b0 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f3503fff700 1 -- 172.21.15.3:0/597721841 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350800b860 con 0x7f350c0cb3b0 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f3503fff700 1 -- 172.21.15.3:0/597721841 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800bbe0 con 0x7f350c0cb3b0 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f351248e700 1 -- 172.21.15.3:0/597721841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb3b0 msgr2=0x7f350c0cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f351248e700 1 --2- 172.21.15.3:0/597721841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb3b0 0x7f350c0cf5c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f35080098e0 tx=0x7f35080091a0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f351248e700 1 -- 172.21.15.3:0/597721841 shutdown_connections 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f351248e700 1 --2- 172.21.15.3:0/597721841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb3b0 0x7f350c0cf5c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f351248e700 1 -- 172.21.15.3:0/597721841 >> 172.21.15.3:0/597721841 conn(0x7f350c070310 msgr2=0x7f350c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f351248e700 1 -- 172.21.15.3:0/597721841 shutdown_connections 2023-10-06T04:54:59.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.718+0000 7f351248e700 1 -- 172.21.15.3:0/597721841 wait complete. 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.719+0000 7f351248e700 1 Processor -- start 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.719+0000 7f351248e700 1 -- start start 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f351248e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f350c0cb3b0 0x7f350c14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f351248e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb780 0x7f350c16cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f351248e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c1596d0 0x7f350c16d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f351248e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c155990 con 0x7f350c0cb780 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f351248e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c16b260 con 0x7f350c1596d0 2023-10-06T04:54:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f351248e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c16db90 con 0x7f350c0cb3b0 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f3510c8b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb780 0x7f350c16cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f3510c8b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb780 0x7f350c16cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56028/0 (socket says 172.21.15.3:56028) 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f3510c8b700 1 -- 172.21.15.3:0/969300090 learned_addr learned my addr 172.21.15.3:0/969300090 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f3511c8d700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c1596d0 0x7f350c16d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.720+0000 7f351148c700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f350c0cb3b0 0x7f350c14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f3510c8b700 1 -- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f350c0cb3b0 msgr2=0x7f350c14cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f3510c8b700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f350c0cb3b0 0x7f350c14cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f3510c8b700 1 -- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c1596d0 msgr2=0x7f350c16d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f3510c8b700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c1596d0 0x7f350c16d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f3510c8b700 1 -- 172.21.15.3:0/969300090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3508009420 con 0x7f350c0cb780 2023-10-06T04:54:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f3510c8b700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb780 0x7f350c16cfb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c810 tx=0x7f34fc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f35017fa700 1 -- 172.21.15.3:0/969300090 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc005c00 con 0x7f350c0cb780 2023-10-06T04:54:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f35017fa700 1 -- 172.21.15.3:0/969300090 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc005d60 con 0x7f350c0cb780 2023-10-06T04:54:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c16ac10 con 0x7f350c0cb780 2023-10-06T04:54:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f35017fa700 1 -- 172.21.15.3:0/969300090 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc007320 con 0x7f350c0cb780 2023-10-06T04:54:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.721+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c14d020 con 0x7f350c0cb780 2023-10-06T04:54:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.722+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c04c380 con 0x7f350c0cb780 2023-10-06T04:54:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.722+0000 7f35017fa700 1 -- 172.21.15.3:0/969300090 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34fc01b3b0 con 0x7f350c0cb780 2023-10-06T04:54:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.723+0000 7f35017fa700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34f0066f30 0x7f34f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:54:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.723+0000 7f35017fa700 1 -- 172.21.15.3:0/969300090 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f34fc083ae0 con 0x7f350c0cb780 2023-10-06T04:54:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.723+0000 7f351148c700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34f0066f30 0x7f34f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:54:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.724+0000 7f351148c700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34f0066f30 0x7f34f00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35080093f0 tx=0x7f3508017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:54:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.727+0000 7f35017fa700 1 -- 172.21.15.3:0/969300090 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34fc050bf0 con 0x7f350c0cb780 2023-10-06T04:54:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.966+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f350c16b3a0 con 0x7f350c0cb780 2023-10-06T04:54:59.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.967+0000 7f35017fa700 1 -- 172.21.15.3:0/969300090 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7f34fc017020 con 0x7f350c0cb780 2023-10-06T04:54:59.968 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:54:59.968 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:54:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.968+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34f0066f30 msgr2=0x7f34f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.968+0000 7f351248e700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34f0066f30 0x7f34f00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35080093f0 tx=0x7f3508017000 comp rx=0 tx=0).stop 2023-10-06T04:54:59.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.968+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb780 msgr2=0x7f350c16cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.968+0000 7f351248e700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb780 0x7f350c16cfb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c810 tx=0x7f34fc007d90 comp rx=0 tx=0).stop 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 shutdown_connections 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34f0066f30 0x7f34f00693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c1596d0 0x7f350c16d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f350c0cb3b0 0x7f350c14cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 --2- 172.21.15.3:0/969300090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c0cb780 0x7f350c16cfb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 >> 172.21.15.3:0/969300090 conn(0x7f350c070310 msgr2=0x7f350c0c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 shutdown_connections 2023-10-06T04:54:59.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:54:59.969+0000 7f351248e700 1 -- 172.21.15.3:0/969300090 wait complete. 2023-10-06T04:54:59.972 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:54:59.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T04:55:00.120 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":5716,"entity":{"name":{"type":"client","num":5716},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3779663251}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.3499319139999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.969520310290708,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5716 192.168.0.1:0/3779663251","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"99376","root":"/"}},{"id":5904,"entity":{"name":{"type":"client","num":5904},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2120956760}},"state":"open","num_leases":1,"num_caps":216,"request_load_avg":1228,"uptime":7.1449090709999998,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":490.91261801572836,"halflife":60},"release_caps":{"value":380.90404142372245,"halflife":60},"recall_caps_throttle":{"value":448.85417777896686,"halflife":1.5},"recall_caps_throttle2o":{"value":150.08627294446273,"halflife":0.5},"session_cache_liveness":{"value":1198.7724074315454,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1260,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5904 192.168.0.1:0/2120956760","completed_requests":[{"tid":1261,"created_ino":"0x1000000044c"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"110064","root":"/"}}] 2023-10-06T04:55:00.121 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "99376", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3779663251, "type": "any" }, "name": { "num": 5716, "type": "client" } }, "id": 5716, "inst": "client.5716 192.168.0.1:0/3779663251", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.969520310290708 }, "state": "open", "uptime": 5.349931914 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "110064", "root": "/" }, "completed_requests": [ { "created_ino": "0x1000000044c", "tid": 1261 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2120956760, "type": "any" }, "name": { "num": 5904, "type": "client" } }, "id": 5904, "inst": "client.5904 192.168.0.1:0/2120956760", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 1260, "num_caps": 216, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 490.91261801572836 }, "recall_caps_throttle": { "halflife": 1.5, "value": 448.85417777896686 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 150.08627294446273 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 380.90404142372245 }, "request_load_avg": 1228, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1198.7724074315454 }, "state": "open", "uptime": 7.144909071 } ] 2023-10-06T04:55:00.121 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T04:55:05.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.254+0000 7fc82c774700 1 Processor -- start 2023-10-06T04:55:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.254+0000 7fc82c774700 1 -- start start 2023-10-06T04:55:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc82c774700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc8240b1f10 0x7fc82415d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc82c774700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240c93f0 con 0x7fc82415cd00 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc82c774700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82415e0d0 con 0x7fc82415c930 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc82c774700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240caf50 con 0x7fc8240b1f10 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc829d0f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc82415cd00 0x7fc8240b1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60610/0 (socket says 172.21.15.3:60610) 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc829d0f700 1 -- 172.21.15.3:0/3776047790 learned_addr learned my addr 172.21.15.3:0/3776047790 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc82ad11700 1 --2- 172.21.15.3:0/3776047790 >> v2:172.21.15.153:3300/0 conn(0x7fc8240b1f10 0x7fc82415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc82950e700 1 -- 172.21.15.3:0/3776047790 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524627173 0 0) 0x7fc8240c93f0 con 0x7fc82415cd00 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.255+0000 7fc82950e700 1 -- 172.21.15.3:0/3776047790 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc810003570 con 0x7fc82415cd00 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82ad11700 1 -- 172.21.15.3:0/3776047790 >> v1:172.21.15.153:6789/0 conn(0x7fc82415c930 legacy=0x7fc82406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82950e700 1 -- 172.21.15.3:0/3776047790 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233013065 0 0) 0x7fc810003570 con 0x7fc82415cd00 2023-10-06T04:55:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82ad11700 1 -- 172.21.15.3:0/3776047790 >> v1:172.21.15.3:6789/0 conn(0x7fc82415cd00 legacy=0x7fc8240b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82ad11700 1 -- 172.21.15.3:0/3776047790 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82415d830 con 0x7fc8240b1f10 2023-10-06T04:55:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82ad11700 1 --2- 172.21.15.3:0/3776047790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8240b1f10 0x7fc82415d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009e60 tx=0x7fc81c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f03c393242ccdab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82950e700 1 -- 172.21.15.3:0/3776047790 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00e070 con 0x7fc8240b1f10 2023-10-06T04:55:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82950e700 1 -- 172.21.15.3:0/3776047790 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c00b850 con 0x7fc8240b1f10 2023-10-06T04:55:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.256+0000 7fc82950e700 1 -- 172.21.15.3:0/3776047790 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814001470 con 0x7fc8240b1f10 2023-10-06T04:55:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.257+0000 7fc82c774700 1 -- 172.21.15.3:0/3776047790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8240b1f10 msgr2=0x7fc82415d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.257+0000 7fc82c774700 1 --2- 172.21.15.3:0/3776047790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8240b1f10 0x7fc82415d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009e60 tx=0x7fc81c009780 comp rx=0 tx=0).stop 2023-10-06T04:55:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.257+0000 7fc82c774700 1 -- 172.21.15.3:0/3776047790 shutdown_connections 2023-10-06T04:55:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.257+0000 7fc82c774700 1 --2- 172.21.15.3:0/3776047790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8240b1f10 0x7fc82415d150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.257+0000 7fc82c774700 1 -- 172.21.15.3:0/3776047790 >> 172.21.15.3:0/3776047790 conn(0x7fc824066fb0 msgr2=0x7fc8240673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.257+0000 7fc82c774700 1 -- 172.21.15.3:0/3776047790 shutdown_connections 2023-10-06T04:55:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.257+0000 7fc82c774700 1 -- 172.21.15.3:0/3776047790 wait complete. 2023-10-06T04:55:05.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.259+0000 7fc82c774700 1 Processor -- start 2023-10-06T04:55:05.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.259+0000 7fc82c774700 1 -- start start 2023-10-06T04:55:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.260+0000 7fc82c774700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8240b1f10 0x7fc824147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.260+0000 7fc82c774700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc82415c930 0x7fc824157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.260+0000 7fc82c774700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc824157b80 0x7fc824158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.260+0000 7fc82c774700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824147950 con 0x7fc82415c930 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.260+0000 7fc82c774700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824158000 con 0x7fc824157b80 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.260+0000 7fc82c774700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc824159d30 con 0x7fc8240b1f10 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc82415c930 0x7fc824157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc82415c930 0x7fc824157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37568/0 (socket says 172.21.15.3:37568) 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 -- 172.21.15.3:0/1401691220 learned_addr learned my addr 172.21.15.3:0/1401691220 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc82ad11700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc824157b80 0x7fc824158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc82a510700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8240b1f10 0x7fc824147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 -- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8240b1f10 msgr2=0x7fc824147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8240b1f10 0x7fc824147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 -- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc824157b80 msgr2=0x7fc824158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc824157b80 0x7fc824158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 -- 172.21.15.3:0/1401691220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c009a20 con 0x7fc82415c930 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc82a510700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8240b1f10 0x7fc824147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc829d0f700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc82415c930 0x7fc824157640 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cb90 tx=0x7fc814001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc81a7fc700 1 -- 172.21.15.3:0/1401691220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814001d60 con 0x7fc82415c930 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc81a7fc700 1 -- 172.21.15.3:0/1401691220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc814007360 con 0x7fc82415c930 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82415a010 con 0x7fc82415c930 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.261+0000 7fc81a7fc700 1 -- 172.21.15.3:0/1401691220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814007700 con 0x7fc82415c930 2023-10-06T04:55:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.262+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82415a5d0 con 0x7fc82415c930 2023-10-06T04:55:05.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.263+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc824071680 con 0x7fc82415c930 2023-10-06T04:55:05.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.264+0000 7fc81a7fc700 1 -- 172.21.15.3:0/1401691220 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc814004a40 con 0x7fc82415c930 2023-10-06T04:55:05.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.264+0000 7fc81a7fc700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc808066f30 0x7fc8080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.264+0000 7fc81a7fc700 1 -- 172.21.15.3:0/1401691220 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc814084160 con 0x7fc82415c930 2023-10-06T04:55:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.264+0000 7fc82a510700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc808066f30 0x7fc8080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.265+0000 7fc82a510700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc808066f30 0x7fc8080693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc820001990 tx=0x7fc820009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:05.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.272+0000 7fc81a7fc700 1 -- 172.21.15.3:0/1401691220 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8140513d0 con 0x7fc82415c930 2023-10-06T04:55:05.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.520+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc82415db10 con 0x7fc82415c930 2023-10-06T04:55:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.520+0000 7fc81a7fc700 1 -- 172.21.15.3:0/1401691220 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7fc814018020 con 0x7fc82415c930 2023-10-06T04:55:05.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:05.522 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:55:05.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.522+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc808066f30 msgr2=0x7fc8080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:05.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc808066f30 0x7fc8080693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc820001990 tx=0x7fc820009040 comp rx=0 tx=0).stop 2023-10-06T04:55:05.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc82415c930 msgr2=0x7fc824157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:05.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc82415c930 0x7fc824157640 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cb90 tx=0x7fc814001ad0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 shutdown_connections 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc808066f30 0x7fc8080693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc824157b80 0x7fc824158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8240b1f10 0x7fc824147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 --2- 172.21.15.3:0/1401691220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc82415c930 0x7fc824157640 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 >> 172.21.15.3:0/1401691220 conn(0x7fc824066fb0 msgr2=0x7fc82406c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 shutdown_connections 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:05.523+0000 7fc82c774700 1 -- 172.21.15.3:0/1401691220 wait complete. 2023-10-06T04:55:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:55:05.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T04:55:05.674 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":5716,"entity":{"name":{"type":"client","num":5716},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3779663251}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":10.903861232000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9316579803548923,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5716 192.168.0.1:0/3779663251","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"99376","root":"/"}},{"id":5904,"entity":{"name":{"type":"client","num":5904},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2120956760}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":1152,"uptime":12.698838389000001,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":460.35748890329694,"halflife":60},"release_caps":{"value":466.36306509286538,"halflife":60},"recall_caps_throttle":{"value":44.397540576215235,"halflife":1.5},"recall_caps_throttle2o":{"value":0.1406053880008582,"halflife":0.5},"session_cache_liveness":{"value":1281.113042400704,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1260,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5904 192.168.0.1:0/2120956760","completed_requests":[{"tid":1261,"created_ino":"0x1000000044c"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"110064","root":"/"}}] 2023-10-06T04:55:05.675 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "99376", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3779663251, "type": "any" }, "name": { "num": 5716, "type": "client" } }, "id": 5716, "inst": "client.5716 192.168.0.1:0/3779663251", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9316579803548923 }, "state": "open", "uptime": 10.903861232 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "110064", "root": "/" }, "completed_requests": [ { "created_ino": "0x1000000044c", "tid": 1261 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2120956760, "type": "any" }, "name": { "num": 5904, "type": "client" } }, "id": 5904, "inst": "client.5904 192.168.0.1:0/2120956760", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 1260, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 460.35748890329694 }, "recall_caps_throttle": { "halflife": 1.5, "value": 44.397540576215235 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0.1406053880008582 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 466.3630650928654 }, "request_load_avg": 1152, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1281.113042400704 }, "state": "open", "uptime": 12.698838389 } ] 2023-10-06T04:55:05.676 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T04:55:05.676 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:55:05.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:55:05.721 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:55:05.762 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:55:05.763 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:55:05.763 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:55:05.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:55:05.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[110064]: fuse finished with error 0 and tester_r 0 2023-10-06T04:55:05.800 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:55:05.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:05.801 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:55:05.967 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:55:05.968 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:55:05.968 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:55:06.010 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[99376]: fuse finished with error 0 and tester_r 0 2023-10-06T04:55:06.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:55:06.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:06.012 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:55:06.163 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:55:06.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:06.163 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:55:06.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:55:06.179 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T04:55:06.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:55:06.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:06.179 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:55:06.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:55:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T04:55:06.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:06.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.337+0000 7fd8147af700 1 Processor -- start 2023-10-06T04:55:06.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.338+0000 7fd8147af700 1 -- start start 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.338+0000 7fd8147af700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c06aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.338+0000 7fd8147af700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd80c06ec40 0x7fd80c157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.338+0000 7fd8147af700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c069790 con 0x7fd80c156a30 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.338+0000 7fd8147af700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c0ae9e0 con 0x7fd80c06ec40 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd8147af700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0699e0 con 0x7fd80c155980 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd811d4a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd80c156a30 0x7fd80c06e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60614/0 (socket says 172.21.15.3:60614) 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd811d4a700 1 -- 172.21.15.3:0/1667558168 learned_addr learned my addr 172.21.15.3:0/1667558168 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd812d4c700 1 --2- 172.21.15.3:0/1667558168 >> v2:172.21.15.153:3300/0 conn(0x7fd80c06ec40 0x7fd80c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd81254b700 1 --2- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c06aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd811549700 1 -- 172.21.15.3:0/1667558168 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964962352 0 0) 0x7fd80c069790 con 0x7fd80c156a30 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd811549700 1 -- 172.21.15.3:0/1667558168 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f4003570 con 0x7fd80c156a30 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd812d4c700 1 -- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 msgr2=0x7fd80c06aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd811549700 1 -- 172.21.15.3:0/1667558168 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 985659924 0 0) 0x7fd7f4003570 con 0x7fd80c156a30 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd812d4c700 1 --2- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c06aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd812d4c700 1 -- 172.21.15.3:0/1667558168 >> v1:172.21.15.3:6789/0 conn(0x7fd80c156a30 legacy=0x7fd80c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd812d4c700 1 -- 172.21.15.3:0/1667558168 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c069cc0 con 0x7fd80c06ec40 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd81254b700 1 --2- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c06aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:06.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.339+0000 7fd812d4c700 1 --2- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c06ec40 0x7fd80c157500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd804009e60 tx=0x7fd804009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78be25373403eca4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd811549700 1 -- 172.21.15.3:0/1667558168 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800001470 con 0x7fd80c06ec40 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd811549700 1 -- 172.21.15.3:0/1667558168 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80400b7a0 con 0x7fd80c06ec40 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd811549700 1 -- 172.21.15.3:0/1667558168 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80400bb60 con 0x7fd80c06ec40 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 -- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c06ec40 msgr2=0x7fd80c157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 --2- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c06ec40 0x7fd80c157500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd804009e60 tx=0x7fd804009780 comp rx=0 tx=0).stop 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 -- 172.21.15.3:0/1667558168 shutdown_connections 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 --2- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c06ec40 0x7fd80c157500 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 --2- 172.21.15.3:0/1667558168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c06aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 -- 172.21.15.3:0/1667558168 >> 172.21.15.3:0/1667558168 conn(0x7fd80c06c370 msgr2=0x7fd80c06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:06.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 -- 172.21.15.3:0/1667558168 shutdown_connections 2023-10-06T04:55:06.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.340+0000 7fd8147af700 1 -- 172.21.15.3:0/1667558168 wait complete. 2023-10-06T04:55:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.341+0000 7fd8147af700 1 Processor -- start 2023-10-06T04:55:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.341+0000 7fd8147af700 1 -- start start 2023-10-06T04:55:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd8147af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c06ec40 0x7fd80c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd8147af700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c0c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd8147af700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c156a30 0x7fd80c0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd8147af700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0cc7e0 con 0x7fd80c06ec40 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd8147af700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0cbbd0 con 0x7fd80c156a30 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd8147af700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0cbeb0 con 0x7fd80c155980 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd81254b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c06ec40 0x7fd80c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd81254b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c06ec40 0x7fd80c0cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37576/0 (socket says 172.21.15.3:37576) 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd81254b700 1 -- 172.21.15.3:0/498484304 learned_addr learned my addr 172.21.15.3:0/498484304 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd811d4a700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c0c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.342+0000 7fd812d4c700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c156a30 0x7fd80c0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd81254b700 1 -- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 msgr2=0x7fd80c0c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd81254b700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c0c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd81254b700 1 -- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c156a30 msgr2=0x7fd80c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd81254b700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c156a30 0x7fd80c0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd81254b700 1 -- 172.21.15.3:0/498484304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc00a5d0 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd81254b700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c06ec40 0x7fd80c0cb4d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00eaf0 tx=0x7fd7fc00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd80a7fc700 1 -- 172.21.15.3:0/498484304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00c950 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd80a7fc700 1 -- 172.21.15.3:0/498484304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc00cab0 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd804009a20 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd80a7fc700 1 -- 172.21.15.3:0/498484304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00ce50 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.343+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c0cc470 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.344+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c071680 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.344+0000 7fd80a7fc700 1 -- 172.21.15.3:0/498484304 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7fc0113f0 con 0x7fd80c06ec40 2023-10-06T04:55:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.345+0000 7fd80a7fc700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7f8066f30 0x7fd7f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.345+0000 7fd80a7fc700 1 -- 172.21.15.3:0/498484304 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd7fc015030 con 0x7fd80c06ec40 2023-10-06T04:55:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.345+0000 7fd811d4a700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7f8066f30 0x7fd7f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.345+0000 7fd811d4a700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7f8066f30 0x7fd7f80693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd80c151dd0 tx=0x7fd800001aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.349+0000 7fd80a7fc700 1 -- 172.21.15.3:0/498484304 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7fc051580 con 0x7fd80c06ec40 2023-10-06T04:55:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.590+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd80c0cb2b0 con 0x7fd80c06ec40 2023-10-06T04:55:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.591+0000 7fd80a7fc700 1 -- 172.21.15.3:0/498484304 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7fd7fc01f090 con 0x7fd80c06ec40 2023-10-06T04:55:06.593 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:06.593 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:55:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7f8066f30 msgr2=0x7fd7f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7f8066f30 0x7fd7f80693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd80c151dd0 tx=0x7fd800001aa0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c06ec40 msgr2=0x7fd80c0cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c06ec40 0x7fd80c0cb4d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00eaf0 tx=0x7fd7fc00a330 comp rx=0 tx=0).stop 2023-10-06T04:55:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 shutdown_connections 2023-10-06T04:55:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7f8066f30 0x7fd7f80693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd80c156a30 0x7fd80c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c06ec40 0x7fd80c0cb4d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 --2- 172.21.15.3:0/498484304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd80c155980 0x7fd80c0c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 >> 172.21.15.3:0/498484304 conn(0x7fd80c06c370 msgr2=0x7fd80c0ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 shutdown_connections 2023-10-06T04:55:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.593+0000 7fd8147af700 1 -- 172.21.15.3:0/498484304 wait complete. 2023-10-06T04:55:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:55:06.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:55:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.724+0000 7f14ea5d0700 1 Processor -- start 2023-10-06T04:55:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.724+0000 7f14ea5d0700 1 -- start start 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.725+0000 7f14ea5d0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e415ccc0 0x7f14e40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.725+0000 7f14ea5d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14e40d06a0 0x7f14e40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.725+0000 7f14ea5d0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e40cf6c0 con 0x7f14e415c8f0 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.725+0000 7f14ea5d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e40cfb50 con 0x7f14e40d06a0 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.725+0000 7f14ea5d0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40d0b20 con 0x7f14e415ccc0 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.725+0000 7f14e8b6d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14e40d06a0 0x7f14e40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e8b6d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14e40d06a0 0x7f14e40b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37590/0 (socket says 172.21.15.3:37590) 2023-10-06T04:55:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e8b6d700 1 -- 172.21.15.3:0/855101060 learned_addr learned my addr 172.21.15.3:0/855101060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e37fe700 1 --2- 172.21.15.3:0/855101060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e415ccc0 0x7f14e40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e8b6d700 1 -- 172.21.15.3:0/855101060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e415ccc0 msgr2=0x7f14e40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e8b6d700 1 --2- 172.21.15.3:0/855101060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e415ccc0 0x7f14e40d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e8b6d700 1 -- 172.21.15.3:0/855101060 >> v1:172.21.15.153:6789/0 conn(0x7f14e415c8f0 legacy=0x7f14e415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e8b6d700 1 -- 172.21.15.3:0/855101060 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40ceab0 con 0x7f14e40d06a0 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e37fe700 1 --2- 172.21.15.3:0/855101060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e415ccc0 0x7f14e40d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e8b6d700 1 --2- 172.21.15.3:0/855101060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e40d06a0 0x7f14e40b3ba0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009f20 tx=0x7f14dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1d0f089d19be3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e2ffd700 1 -- 172.21.15.3:0/855101060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00e070 con 0x7f14e40d06a0 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e2ffd700 1 -- 172.21.15.3:0/855101060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc00ba80 con 0x7f14e40d06a0 2023-10-06T04:55:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.726+0000 7f14e2ffd700 1 -- 172.21.15.3:0/855101060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0092e0 con 0x7f14e40d06a0 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 -- 172.21.15.3:0/855101060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e40d06a0 msgr2=0x7f14e40b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/855101060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e40d06a0 0x7f14e40b3ba0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009f20 tx=0x7f14dc009840 comp rx=0 tx=0).stop 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 -- 172.21.15.3:0/855101060 shutdown_connections 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/855101060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e40d06a0 0x7f14e40b3ba0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/855101060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e415ccc0 0x7f14e40d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 -- 172.21.15.3:0/855101060 >> 172.21.15.3:0/855101060 conn(0x7f14e4070310 msgr2=0x7f14e4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 -- 172.21.15.3:0/855101060 shutdown_connections 2023-10-06T04:55:06.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.727+0000 7f14ea5d0700 1 -- 172.21.15.3:0/855101060 wait complete. 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.728+0000 7f14ea5d0700 1 Processor -- start 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.728+0000 7f14ea5d0700 1 -- start start 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14ea5d0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e40d06a0 0x7f14e4140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14ea5d0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e415c8f0 0x7f14e4169a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14ea5d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e415ccc0 0x7f14e415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14ea5d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4153900 con 0x7f14e415ccc0 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14ea5d0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4152cf0 con 0x7f14e415c8f0 2023-10-06T04:55:06.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14ea5d0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4152fd0 con 0x7f14e40d06a0 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e415ccc0 0x7f14e415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e415ccc0 0x7f14e415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37592/0 (socket says 172.21.15.3:37592) 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 -- 172.21.15.3:0/2994797423 learned_addr learned my addr 172.21.15.3:0/2994797423 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e37fe700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e415c8f0 0x7f14e4169a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e3fff700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e40d06a0 0x7f14e4140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 -- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e40d06a0 msgr2=0x7f14e4140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e40d06a0 0x7f14e4140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 -- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e415c8f0 msgr2=0x7f14e4169a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e415c8f0 0x7f14e4169a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 -- 172.21.15.3:0/2994797423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009ae0 con 0x7f14e415ccc0 2023-10-06T04:55:06.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.729+0000 7f14e8b6d700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e415ccc0 0x7f14e415d630 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f14e406eaa0 tx=0x7f14dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.730+0000 7f14dbfff700 1 -- 172.21.15.3:0/2994797423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc008d20 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.730+0000 7f14dbfff700 1 -- 172.21.15.3:0/2994797423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc008e80 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.730+0000 7f14dbfff700 1 -- 172.21.15.3:0/2994797423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc01c660 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.730+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e41532b0 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.730+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e4141360 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.731+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e40b2a80 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.731+0000 7f14dbfff700 1 -- 172.21.15.3:0/2994797423 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14dc003040 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.731+0000 7f14dbfff700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.732+0000 7f14dbfff700 1 -- 172.21.15.3:0/2994797423 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f14dc086060 con 0x7f14e415ccc0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.732+0000 7f14e3fff700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:06.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.732+0000 7f14e3fff700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14c4066f30 0x7f14c40693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14cc003a80 tx=0x7f14cc007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:06.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.736+0000 7f14dbfff700 1 -- 172.21.15.3:0/2994797423 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14dc0530e0 con 0x7f14e415ccc0 2023-10-06T04:55:06.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.933+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14e4071680 con 0x7f14e415ccc0 2023-10-06T04:55:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.933+0000 7f14dbfff700 1 -- 172.21.15.3:0/2994797423 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17058 (secure 0 0 0) 0x7f14dc056700 con 0x7f14e415ccc0 2023-10-06T04:55:06.936 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:06.936 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:06.541012+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:54:38.486656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T04:54:39.707870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.1","mappings":[{"from":4,"to":3}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:55:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14c4066f30 msgr2=0x7f14c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14c4066f30 0x7f14c40693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14cc003a80 tx=0x7f14cc007ec0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e415ccc0 msgr2=0x7f14e415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e415ccc0 0x7f14e415d630 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f14e406eaa0 tx=0x7f14dc018000 comp rx=0 tx=0).stop 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 shutdown_connections 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e415ccc0 0x7f14e415d630 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e40d06a0 0x7f14e4140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 --2- 172.21.15.3:0/2994797423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e415c8f0 0x7f14e4169a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 >> 172.21.15.3:0/2994797423 conn(0x7f14e4070310 msgr2=0x7f14e40b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:06.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 shutdown_connections 2023-10-06T04:55:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:06.936+0000 7f14ea5d0700 1 -- 172.21.15.3:0/2994797423 wait complete. 2023-10-06T04:55:06.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:54:38.486656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:54:39.707870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:55:06.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:55:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.072+0000 7f4169173700 1 Processor -- start 2023-10-06T04:55:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.072+0000 7f4169173700 1 -- start start 2023-10-06T04:55:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4169173700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41640cba50 0x7f4164150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4169173700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f416415d540 0x7f41640af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4169173700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416415a290 con 0x7f4164150690 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4169173700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41640d0020 con 0x7f41640cba50 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4169173700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4164159f40 con 0x7f416415d540 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41640cba50 0x7f4164150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41640cba50 0x7f4164150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37608/0 (socket says 172.21.15.3:37608) 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 -- 172.21.15.3:0/1302215172 learned_addr learned my addr 172.21.15.3:0/1302215172 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416359e700 1 --2- 172.21.15.3:0/1302215172 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f416415d540 0x7f41640af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 -- 172.21.15.3:0/1302215172 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f416415d540 msgr2=0x7f41640af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 --2- 172.21.15.3:0/1302215172 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f416415d540 0x7f41640af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 -- 172.21.15.3:0/1302215172 >> v1:172.21.15.153:6789/0 conn(0x7f4164150690 legacy=0x7f416415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 -- 172.21.15.3:0/1302215172 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416415dcb0 con 0x7f41640cba50 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f416259c700 1 --2- 172.21.15.3:0/1302215172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f4164150a60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f41580099b0 tx=0x7f4158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ca4fecff64271e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4161d9b700 1 -- 172.21.15.3:0/1302215172 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415800b820 con 0x7f41640cba50 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4161d9b700 1 -- 172.21.15.3:0/1302215172 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415800b980 con 0x7f41640cba50 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.073+0000 7f4161d9b700 1 -- 172.21.15.3:0/1302215172 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415800bd20 con 0x7f41640cba50 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 -- 172.21.15.3:0/1302215172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 msgr2=0x7f4164150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 --2- 172.21.15.3:0/1302215172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f4164150a60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f41580099b0 tx=0x7f4158009250 comp rx=0 tx=0).stop 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 -- 172.21.15.3:0/1302215172 shutdown_connections 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 --2- 172.21.15.3:0/1302215172 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f416415d540 0x7f41640af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 --2- 172.21.15.3:0/1302215172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f4164150a60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 -- 172.21.15.3:0/1302215172 >> 172.21.15.3:0/1302215172 conn(0x7f4164070310 msgr2=0x7f4164070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 -- 172.21.15.3:0/1302215172 shutdown_connections 2023-10-06T04:55:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.074+0000 7f4169173700 1 -- 172.21.15.3:0/1302215172 wait complete. 2023-10-06T04:55:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.075+0000 7f4169173700 1 Processor -- start 2023-10-06T04:55:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.075+0000 7f4169173700 1 -- start start 2023-10-06T04:55:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4169173700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f416414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4169173700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4164150690 0x7f416415b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4169173700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416415d540 0x7f416415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4169173700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164150510 con 0x7f41640cba50 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4169173700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640c8ab0 con 0x7f416415d540 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4169173700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41640b4230 con 0x7f4164150690 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f416414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f416414ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37610/0 (socket says 172.21.15.3:37610) 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 -- 172.21.15.3:0/622237258 learned_addr learned my addr 172.21.15.3:0/622237258 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f416259c700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4164150690 0x7f416415b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f416359e700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416415d540 0x7f416415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 -- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4164150690 msgr2=0x7f416415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4164150690 0x7f416415b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 -- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416415d540 msgr2=0x7f416415bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416415d540 0x7f416415bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 -- 172.21.15.3:0/622237258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580094f0 con 0x7f41640cba50 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.076+0000 7f4162d9d700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f416414ffd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f415400c600 tx=0x7f4154007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.077+0000 7f414affd700 1 -- 172.21.15.3:0/622237258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154011590 con 0x7f41640cba50 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.077+0000 7f414affd700 1 -- 172.21.15.3:0/622237258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154011710 con 0x7f41640cba50 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.077+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41640b4510 con 0x7f41640cba50 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.077+0000 7f414affd700 1 -- 172.21.15.3:0/622237258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154011ad0 con 0x7f41640cba50 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.077+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41640b4ad0 con 0x7f41640cba50 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.078+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416404c380 con 0x7f41640cba50 2023-10-06T04:55:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.078+0000 7f414affd700 1 -- 172.21.15.3:0/622237258 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f415401c460 con 0x7f41640cba50 2023-10-06T04:55:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.078+0000 7f414affd700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4150066f30 0x7f41500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.078+0000 7f414affd700 1 -- 172.21.15.3:0/622237258 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f41540843c0 con 0x7f41640cba50 2023-10-06T04:55:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.079+0000 7f416259c700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4150066f30 0x7f41500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.079+0000 7f416259c700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4150066f30 0x7f41500693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f41580099b0 tx=0x7f4158016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.083+0000 7f414affd700 1 -- 172.21.15.3:0/622237258 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4154051440 con 0x7f41640cba50 2023-10-06T04:55:07.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.275+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f416415df90 con 0x7f41640cba50 2023-10-06T04:55:07.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.275+0000 7f414affd700 1 -- 172.21.15.3:0/622237258 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4154054a60 con 0x7f41640cba50 2023-10-06T04:55:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:55:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4150066f30 msgr2=0x7f41500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4150066f30 0x7f41500693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f41580099b0 tx=0x7f4158016000 comp rx=0 tx=0).stop 2023-10-06T04:55:07.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 msgr2=0x7f416414ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f416414ffd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f415400c600 tx=0x7f4154007f60 comp rx=0 tx=0).stop 2023-10-06T04:55:07.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 shutdown_connections 2023-10-06T04:55:07.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4150066f30 0x7f41500693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416415d540 0x7f416415bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41640cba50 0x7f416414ffd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 --2- 172.21.15.3:0/622237258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4164150690 0x7f416415b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 >> 172.21.15.3:0/622237258 conn(0x7f4164070310 msgr2=0x7f41640b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:07.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.277+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 shutdown_connections 2023-10-06T04:55:07.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.278+0000 7f4169173700 1 -- 172.21.15.3:0/622237258 wait complete. 2023-10-06T04:55:07.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:55:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.414+0000 7f627be8c700 1 Processor -- start 2023-10-06T04:55:07.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.414+0000 7f627be8c700 1 -- start start 2023-10-06T04:55:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f627be8c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 0x7f6274150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f627be8c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627415a290 con 0x7f62740cba50 2023-10-06T04:55:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f627be8c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62740d0020 con 0x7f6274150690 2023-10-06T04:55:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f627be8c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6274159f40 con 0x7f62740af8a0 2023-10-06T04:55:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6279427700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62740cba50 0x7f62740af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60628/0 (socket says 172.21.15.3:60628) 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6279427700 1 -- 172.21.15.3:0/918091741 learned_addr learned my addr 172.21.15.3:0/918091741 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f627a429700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 0x7f6274150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433978852 0 0) 0x7f627415a290 con 0x7f62740cba50 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f625c003570 con 0x7f62740cba50 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2385646084 0 0) 0x7f625c003570 con 0x7f62740cba50 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 msgr2=0x7f6274150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 0x7f6274150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 >> v1:172.21.15.153:6789/0 conn(0x7f6274150690 legacy=0x7f627415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627415dcb0 con 0x7f62740cba50 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6268001470 con 0x7f62740cba50 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 >> v1:172.21.15.3:6789/0 conn(0x7f62740cba50 legacy=0x7f62740af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 0x7f625c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f625c003fd0 0x7f625c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c004d50 0x7f625c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6274159f40 con 0x7f625c004d50 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c00bb40 con 0x7f625c003fd0 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.415+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c00bd00 con 0x7f62740af8a0 2023-10-06T04:55:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f627a429700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c004d50 0x7f625c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f6279c28700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 0x7f625c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f6279427700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f625c003fd0 0x7f625c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f627a429700 1 -- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 msgr2=0x7f625c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f627a429700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62740af8a0 0x7f625c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f627a429700 1 -- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f625c003fd0 msgr2=0x7f625c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f627a429700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f625c003fd0 0x7f625c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f627a429700 1 -- 172.21.15.3:0/918091741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c003570 con 0x7f625c004d50 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f6279427700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f625c003fd0 0x7f625c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f627a429700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c004d50 0x7f625c0051c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f62740afca0 tx=0x7f626c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1554265db9f588b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00e710 con 0x7f625c004d50 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c00e870 con 0x7f625c004d50 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.416+0000 7f6278c26700 1 -- 172.21.15.3:0/918091741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00ec30 con 0x7f625c004d50 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.417+0000 7f627be8c700 1 -- 172.21.15.3:0/918091741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c004d50 msgr2=0x7f625c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.417+0000 7f627be8c700 1 --2- 172.21.15.3:0/918091741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c004d50 0x7f625c0051c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f62740afca0 tx=0x7f626c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.417+0000 7f627a429700 1 -- 172.21.15.3:0/918091741 reap_dead start 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.417+0000 7f627be8c700 1 -- 172.21.15.3:0/918091741 shutdown_connections 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.417+0000 7f627be8c700 1 -- 172.21.15.3:0/918091741 >> 172.21.15.3:0/918091741 conn(0x7f6274070310 msgr2=0x7f6274070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.417+0000 7f627be8c700 1 -- 172.21.15.3:0/918091741 shutdown_connections 2023-10-06T04:55:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.417+0000 7f627be8c700 1 -- 172.21.15.3:0/918091741 wait complete. 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.418+0000 7f627be8c700 1 Processor -- start 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.418+0000 7f627be8c700 1 -- start start 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627be8c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627414e6d0 0x7f62740b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627be8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627415c1a0 0x7f627416d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627be8c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627416d9c0 0x7f6274148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627be8c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627415d6c0 con 0x7f627415c1a0 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627be8c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627415aab0 con 0x7f627416d9c0 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627be8c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627416de30 con 0x7f627414e6d0 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627415c1a0 0x7f627416d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627415c1a0 0x7f627416d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37638/0 (socket says 172.21.15.3:37638) 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 -- 172.21.15.3:0/2329498892 learned_addr learned my addr 172.21.15.3:0/2329498892 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279c28700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627414e6d0 0x7f62740b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627a429700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627416d9c0 0x7f6274148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 -- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627414e6d0 msgr2=0x7f62740b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627414e6d0 0x7f62740b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 -- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627416d9c0 msgr2=0x7f6274148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627416d9c0 0x7f6274148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 -- 172.21.15.3:0/2329498892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627415c620 con 0x7f627415c1a0 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279c28700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627414e6d0 0x7f62740b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627a429700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627416d9c0 0x7f6274148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6279427700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627415c1a0 0x7f627416d480 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f626800e8f0 tx=0x7f6268000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6271ffb700 1 -- 172.21.15.3:0/2329498892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626800ee90 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6271ffb700 1 -- 172.21.15.3:0/2329498892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268010040 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627415c900 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.419+0000 7f6271ffb700 1 -- 172.21.15.3:0/2329498892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626800a990 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.420+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627415cec0 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.420+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627404c380 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.421+0000 7f6271ffb700 1 -- 172.21.15.3:0/2329498892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62680143f0 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.421+0000 7f6271ffb700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6260066f30 0x7f62600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.421+0000 7f6271ffb700 1 -- 172.21.15.3:0/2329498892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f6268083d30 con 0x7f627415c1a0 2023-10-06T04:55:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.421+0000 7f6279c28700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6260066f30 0x7f62600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.422+0000 7f6279c28700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6260066f30 0x7f62600693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6264001460 tx=0x7f62640016d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.425+0000 7f6271ffb700 1 -- 172.21.15.3:0/2329498892 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f626804d5a0 con 0x7f627415c1a0 2023-10-06T04:55:07.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.626+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f627415df90 con 0x7f627415c1a0 2023-10-06T04:55:07.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.626+0000 7f6271ffb700 1 -- 172.21.15.3:0/2329498892 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6268019070 con 0x7f627415c1a0 2023-10-06T04:55:07.630 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:55:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6260066f30 msgr2=0x7f62600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6260066f30 0x7f62600693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6264001460 tx=0x7f62640016d0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627415c1a0 msgr2=0x7f627416d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627415c1a0 0x7f627416d480 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f626800e8f0 tx=0x7f6268000fb0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 shutdown_connections 2023-10-06T04:55:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6260066f30 0x7f62600693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627416d9c0 0x7f6274148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627414e6d0 0x7f62740b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 --2- 172.21.15.3:0/2329498892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627415c1a0 0x7f627416d480 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 >> 172.21.15.3:0/2329498892 conn(0x7f6274070310 msgr2=0x7f62740cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 shutdown_connections 2023-10-06T04:55:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.629+0000 7f627be8c700 1 -- 172.21.15.3:0/2329498892 wait complete. 2023-10-06T04:55:07.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:55:07.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.764+0000 7f505759e700 1 Processor -- start 2023-10-06T04:55:07.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.764+0000 7f505759e700 1 -- start start 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.764+0000 7f505759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50581539b0 0x7f505815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505759e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5058153d80 0x7f50580aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505759e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50580ce390 con 0x7f505815d180 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505759e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505815aaf0 con 0x7f5058153d80 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505815d680 con 0x7f50581539b0 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f5056d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5058153d80 0x7f50580aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50581539b0 0x7f505815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f5055d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f505815d180 0x7f50580cb100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:35312/0 (socket says 172.21.15.3:35312) 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f5055d9b700 1 -- 172.21.15.3:0/2126496810 learned_addr learned my addr 172.21.15.3:0/2126496810 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505659c700 1 -- 172.21.15.3:0/2126496810 >> v2:172.21.15.153:3300/0 conn(0x7f5058153d80 msgr2=0x7f50580aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505659c700 1 --2- 172.21.15.3:0/2126496810 >> v2:172.21.15.153:3300/0 conn(0x7f5058153d80 0x7f50580aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505659c700 1 -- 172.21.15.3:0/2126496810 >> v1:172.21.15.153:6789/0 conn(0x7f505815d180 legacy=0x7f50580cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505659c700 1 -- 172.21.15.3:0/2126496810 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505815d960 con 0x7f50581539b0 2023-10-06T04:55:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f505559a700 1 -- 172.21.15.3:0/2126496810 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965293288 0 0) 0x7f50580ce390 con 0x7f505815d180 2023-10-06T04:55:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.765+0000 7f5056d9d700 1 --2- 172.21.15.3:0/2126496810 >> v2:172.21.15.153:3300/0 conn(0x7f5058153d80 0x7f50580aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505659c700 1 --2- 172.21.15.3:0/2126496810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50581539b0 0x7f505815a5b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5050009e60 tx=0x7f5050009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4574ef106a239877 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505559a700 1 -- 172.21.15.3:0/2126496810 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000e070 con 0x7f50581539b0 2023-10-06T04:55:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505559a700 1 -- 172.21.15.3:0/2126496810 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505000bb60 con 0x7f50581539b0 2023-10-06T04:55:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505559a700 1 -- 172.21.15.3:0/2126496810 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050012430 con 0x7f50581539b0 2023-10-06T04:55:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 -- 172.21.15.3:0/2126496810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50581539b0 msgr2=0x7f505815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 --2- 172.21.15.3:0/2126496810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50581539b0 0x7f505815a5b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5050009e60 tx=0x7f5050009780 comp rx=0 tx=0).stop 2023-10-06T04:55:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 -- 172.21.15.3:0/2126496810 shutdown_connections 2023-10-06T04:55:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 --2- 172.21.15.3:0/2126496810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50581539b0 0x7f505815a5b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 --2- 172.21.15.3:0/2126496810 >> v2:172.21.15.153:3300/0 conn(0x7f5058153d80 0x7f50580aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 -- 172.21.15.3:0/2126496810 >> 172.21.15.3:0/2126496810 conn(0x7f5058070310 msgr2=0x7f5058070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 -- 172.21.15.3:0/2126496810 shutdown_connections 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.766+0000 7f505759e700 1 -- 172.21.15.3:0/2126496810 wait complete. 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.767+0000 7f505759e700 1 Processor -- start 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f505759e700 1 -- start start 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f505759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50581539b0 0x7f50580b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f505759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5058153d80 0x7f505816a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f505759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505815d180 0x7f5058144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f505759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50580b36e0 con 0x7f5058153d80 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f505759e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50580c7410 con 0x7f50581539b0 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f505759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50580c7da0 con 0x7f505815d180 2023-10-06T04:55:07.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f5055d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5058153d80 0x7f505816a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f5055d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5058153d80 0x7f505816a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37646/0 (socket says 172.21.15.3:37646) 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.768+0000 7f5055d9b700 1 -- 172.21.15.3:0/3625566250 learned_addr learned my addr 172.21.15.3:0/3625566250 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f5056d9d700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505815d180 0x7f5058144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f505659c700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50581539b0 0x7f50580b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f5055d9b700 1 -- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505815d180 msgr2=0x7f5058144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f5055d9b700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505815d180 0x7f5058144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f5055d9b700 1 -- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50581539b0 msgr2=0x7f50580b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f5055d9b700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50581539b0 0x7f50580b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f5055d9b700 1 -- 172.21.15.3:0/3625566250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5048009420 con 0x7f5058153d80 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f5055d9b700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5058153d80 0x7f505816a140 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f504000c810 tx=0x7f5040007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f504e7fc700 1 -- 172.21.15.3:0/3625566250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040005c00 con 0x7f5058153d80 2023-10-06T04:55:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f504e7fc700 1 -- 172.21.15.3:0/3625566250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5040005d60 con 0x7f5058153d80 2023-10-06T04:55:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5050009a20 con 0x7f5058153d80 2023-10-06T04:55:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f504e7fc700 1 -- 172.21.15.3:0/3625566250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040007320 con 0x7f5058153d80 2023-10-06T04:55:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.769+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5058145240 con 0x7f5058153d80 2023-10-06T04:55:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.770+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50580c81f0 con 0x7f5058153d80 2023-10-06T04:55:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.771+0000 7f504e7fc700 1 -- 172.21.15.3:0/3625566250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f504001b400 con 0x7f5058153d80 2023-10-06T04:55:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.771+0000 7f504e7fc700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5038066f30 0x7f50380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.771+0000 7f504e7fc700 1 -- 172.21.15.3:0/3625566250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f50400842a0 con 0x7f5058153d80 2023-10-06T04:55:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.771+0000 7f505659c700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5038066f30 0x7f50380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.772+0000 7f505659c700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5038066f30 0x7f50380693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50500097c0 tx=0x7f5050019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.775+0000 7f504e7fc700 1 -- 172.21.15.3:0/3625566250 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50580c81f0 con 0x7f5058153d80 2023-10-06T04:55:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.967+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50580c81f0 con 0x7f5058153d80 2023-10-06T04:55:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.967+0000 7f504e7fc700 1 -- 172.21.15.3:0/3625566250 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f50580c81f0 con 0x7f5058153d80 2023-10-06T04:55:07.969 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:55:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5038066f30 msgr2=0x7f50380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5038066f30 0x7f50380693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50500097c0 tx=0x7f5050019000 comp rx=0 tx=0).stop 2023-10-06T04:55:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5058153d80 msgr2=0x7f505816a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5058153d80 0x7f505816a140 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f504000c810 tx=0x7f5040007db0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 shutdown_connections 2023-10-06T04:55:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5038066f30 0x7f50380693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50581539b0 0x7f50580b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505815d180 0x7f5058144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 --2- 172.21.15.3:0/3625566250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5058153d80 0x7f505816a140 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 >> 172.21.15.3:0/3625566250 conn(0x7f5058070310 msgr2=0x7f5058152d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 shutdown_connections 2023-10-06T04:55:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:07.970+0000 7f505759e700 1 -- 172.21.15.3:0/3625566250 wait complete. 2023-10-06T04:55:07.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:55:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.103+0000 7f6866653700 1 Processor -- start 2023-10-06T04:55:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.103+0000 7f6866653700 1 -- start start 2023-10-06T04:55:08.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.104+0000 7f6866653700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f6860154c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:08.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.104+0000 7f6866653700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6860068190 0x7f6860155150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:08.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.104+0000 7f6866653700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f68600b3290 0x7f68600b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:08.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.104+0000 7f6866653700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68600b4000 con 0x7f68600b3290 2023-10-06T04:55:08.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.104+0000 7f6866653700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686015b180 con 0x7f6860068190 2023-10-06T04:55:08.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.104+0000 7f6866653700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686015bcc0 con 0x7f6860067dc0 2023-10-06T04:55:08.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.104+0000 7f6864bf0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f68600b3290 0x7f68600b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f68600b3290 0x7f68600b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37660/0 (socket says 172.21.15.3:37660) 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 -- 172.21.15.3:0/336299414 learned_addr learned my addr 172.21.15.3:0/336299414 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f685ffff700 1 --2- 172.21.15.3:0/336299414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f6860154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f685f7fe700 1 --2- 172.21.15.3:0/336299414 >> v2:172.21.15.153:3300/0 conn(0x7f6860068190 0x7f6860155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 -- 172.21.15.3:0/336299414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 msgr2=0x7f6860154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 --2- 172.21.15.3:0/336299414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f6860154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 -- 172.21.15.3:0/336299414 >> v2:172.21.15.153:3300/0 conn(0x7f6860068190 msgr2=0x7f6860155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 --2- 172.21.15.3:0/336299414 >> v2:172.21.15.153:3300/0 conn(0x7f6860068190 0x7f6860155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 -- 172.21.15.3:0/336299414 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686015b390 con 0x7f68600b3290 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f685f7fe700 1 --2- 172.21.15.3:0/336299414 >> v2:172.21.15.153:3300/0 conn(0x7f6860068190 0x7f6860155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f6864bf0700 1 --2- 172.21.15.3:0/336299414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b3710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6858009f20 tx=0x7f6858009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcbeb70e270539c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f685effd700 1 -- 172.21.15.3:0/336299414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800e070 con 0x7f68600b3290 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f685effd700 1 -- 172.21.15.3:0/336299414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685800b990 con 0x7f68600b3290 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.105+0000 7f685effd700 1 -- 172.21.15.3:0/336299414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68580092e0 con 0x7f68600b3290 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 -- 172.21.15.3:0/336299414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 msgr2=0x7f68600b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 --2- 172.21.15.3:0/336299414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b3710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6858009f20 tx=0x7f6858009840 comp rx=0 tx=0).stop 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 -- 172.21.15.3:0/336299414 shutdown_connections 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 --2- 172.21.15.3:0/336299414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b3710 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 --2- 172.21.15.3:0/336299414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f6860154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 --2- 172.21.15.3:0/336299414 >> v2:172.21.15.153:3300/0 conn(0x7f6860068190 0x7f6860155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 -- 172.21.15.3:0/336299414 >> 172.21.15.3:0/336299414 conn(0x7f68600ce270 msgr2=0x7f68600ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 -- 172.21.15.3:0/336299414 shutdown_connections 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.106+0000 7f6866653700 1 -- 172.21.15.3:0/336299414 wait complete. 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.107+0000 7f6866653700 1 Processor -- start 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.107+0000 7f6866653700 1 -- start start 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6866653700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f68600af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6866653700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860068190 0x7f68600afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6866653700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6866653700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6860167940 con 0x7f68600b3290 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6866653700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68600af400 con 0x7f6860068190 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6866653700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68600ae7f0 con 0x7f6860067dc0 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37662/0 (socket says 172.21.15.3:37662) 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 -- 172.21.15.3:0/3250039648 learned_addr learned my addr 172.21.15.3:0/3250039648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f685f7fe700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860068190 0x7f68600afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f685ffff700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f68600af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 -- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 msgr2=0x7f68600af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:08.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f68600af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 -- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860068190 msgr2=0x7f68600afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860068190 0x7f68600afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 -- 172.21.15.3:0/3250039648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6858009ae0 con 0x7f68600b3290 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f685ffff700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f68600af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.108+0000 7f6864bf0700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b0310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f686006b700 tx=0x7f6858018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.109+0000 7f6847fff700 1 -- 172.21.15.3:0/3250039648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858008c30 con 0x7f68600b3290 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.109+0000 7f6847fff700 1 -- 172.21.15.3:0/3250039648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6858008dd0 con 0x7f68600b3290 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.109+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68600aead0 con 0x7f68600b3290 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.109+0000 7f6847fff700 1 -- 172.21.15.3:0/3250039648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68580256b0 con 0x7f68600b3290 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.109+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68600af090 con 0x7f68600b3290 2023-10-06T04:55:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.110+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68600af5b0 con 0x7f68600b3290 2023-10-06T04:55:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.110+0000 7f6847fff700 1 -- 172.21.15.3:0/3250039648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6858003040 con 0x7f68600b3290 2023-10-06T04:55:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.111+0000 7f6847fff700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.111+0000 7f6847fff700 1 -- 172.21.15.3:0/3250039648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f6858085fc0 con 0x7f68600b3290 2023-10-06T04:55:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.111+0000 7f685ffff700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.111+0000 7f685ffff700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6854006ea0 tx=0x7f6854008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:08.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.115+0000 7f6847fff700 1 -- 172.21.15.3:0/3250039648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6858053040 con 0x7f68600b3290 2023-10-06T04:55:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.313+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f686004c380 con 0x7f68600b3290 2023-10-06T04:55:08.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.313+0000 7f6847fff700 1 -- 172.21.15.3:0/3250039648 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6858056660 con 0x7f68600b3290 2023-10-06T04:55:08.315 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:55:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 msgr2=0x7f684c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6854006ea0 tx=0x7f6854008040 comp rx=0 tx=0).stop 2023-10-06T04:55:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 msgr2=0x7f68600b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b0310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f686006b700 tx=0x7f6858018000 comp rx=0 tx=0).stop 2023-10-06T04:55:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 shutdown_connections 2023-10-06T04:55:08.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68600b3290 0x7f68600b0310 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6860067dc0 0x7f68600af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 --2- 172.21.15.3:0/3250039648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860068190 0x7f68600afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:08.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 >> 172.21.15.3:0/3250039648 conn(0x7f68600ce270 msgr2=0x7f6860155a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:08.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 shutdown_connections 2023-10-06T04:55:08.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.315+0000 7f6866653700 1 -- 172.21.15.3:0/3250039648 wait complete. 2023-10-06T04:55:08.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:08.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.447+0000 7f2926494700 1 Processor -- start 2023-10-06T04:55:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.447+0000 7f2926494700 1 -- start start 2023-10-06T04:55:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.448+0000 7f2926494700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f292015c8f0 0x7f292015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.448+0000 7f2926494700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29200d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.448+0000 7f2926494700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200cf6c0 con 0x7f29200d06a0 2023-10-06T04:55:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.448+0000 7f2926494700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200cfb50 con 0x7f292015c8f0 2023-10-06T04:55:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.448+0000 7f2926494700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292015e050 con 0x7f292015ccc0 2023-10-06T04:55:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.448+0000 7f2924a31700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f29200d06a0 0x7f292006d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60640/0 (socket says 172.21.15.3:60640) 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.448+0000 7f2924a31700 1 -- 172.21.15.3:0/2437025974 learned_addr learned my addr 172.21.15.3:0/2437025974 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291f7fe700 1 --2- 172.21.15.3:0/2437025974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29200d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291ffff700 1 --2- 172.21.15.3:0/2437025974 >> v2:172.21.15.153:3300/0 conn(0x7f292015c8f0 0x7f292015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291effd700 1 -- 172.21.15.3:0/2437025974 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591921062 0 0) 0x7f29200cf6c0 con 0x7f29200d06a0 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291effd700 1 -- 172.21.15.3:0/2437025974 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2908003570 con 0x7f29200d06a0 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291f7fe700 1 -- 172.21.15.3:0/2437025974 >> v2:172.21.15.153:3300/0 conn(0x7f292015c8f0 msgr2=0x7f292015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291f7fe700 1 --2- 172.21.15.3:0/2437025974 >> v2:172.21.15.153:3300/0 conn(0x7f292015c8f0 0x7f292015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291f7fe700 1 -- 172.21.15.3:0/2437025974 >> v1:172.21.15.3:6789/0 conn(0x7f29200d06a0 legacy=0x7f292006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291effd700 1 -- 172.21.15.3:0/2437025974 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2503764982 0 0) 0x7f2908003570 con 0x7f29200d06a0 2023-10-06T04:55:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291f7fe700 1 -- 172.21.15.3:0/2437025974 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200d0af0 con 0x7f292015ccc0 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291f7fe700 1 --2- 172.21.15.3:0/2437025974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29200d0160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29140098e0 tx=0x7f29140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50699146c3dcac9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291effd700 1 -- 172.21.15.3:0/2437025974 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180017f0 con 0x7f292015ccc0 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291effd700 1 -- 172.21.15.3:0/2437025974 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2914011420 con 0x7f292015ccc0 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.449+0000 7f291effd700 1 -- 172.21.15.3:0/2437025974 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29140117c0 con 0x7f292015ccc0 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 -- 172.21.15.3:0/2437025974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 msgr2=0x7f29200d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 --2- 172.21.15.3:0/2437025974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29200d0160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29140098e0 tx=0x7f29140091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 -- 172.21.15.3:0/2437025974 shutdown_connections 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 --2- 172.21.15.3:0/2437025974 >> v2:172.21.15.153:3300/0 conn(0x7f292015c8f0 0x7f292015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 --2- 172.21.15.3:0/2437025974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29200d0160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 -- 172.21.15.3:0/2437025974 >> 172.21.15.3:0/2437025974 conn(0x7f2920070310 msgr2=0x7f2920070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 -- 172.21.15.3:0/2437025974 shutdown_connections 2023-10-06T04:55:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.450+0000 7f2926494700 1 -- 172.21.15.3:0/2437025974 wait complete. 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.451+0000 7f2926494700 1 Processor -- start 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.451+0000 7f2926494700 1 -- start start 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f2926494700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29200d06a0 0x7f29201699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f2926494700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f292015c8f0 0x7f29201590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f2926494700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29201595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f2926494700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292016af00 con 0x7f292015c8f0 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f2926494700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920156070 con 0x7f29200d06a0 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f2926494700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292016a2f0 con 0x7f292015ccc0 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f292015c8f0 0x7f29201590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f292015c8f0 0x7f29201590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37670/0 (socket says 172.21.15.3:37670) 2023-10-06T04:55:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 -- 172.21.15.3:0/1606967890 learned_addr learned my addr 172.21.15.3:0/1606967890 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291ffff700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29200d06a0 0x7f29201699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f2924a31700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29201595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 -- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 msgr2=0x7f29201595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29201595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 -- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29200d06a0 msgr2=0x7f29201699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29200d06a0 0x7f29201699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.452+0000 7f291f7fe700 1 -- 172.21.15.3:0/1606967890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914009420 con 0x7f292015c8f0 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.453+0000 7f291f7fe700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f292015c8f0 0x7f29201590a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2914002000 tx=0x7f2914016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.453+0000 7f2907fff700 1 -- 172.21.15.3:0/1606967890 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291401caf0 con 0x7f292015c8f0 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.453+0000 7f2907fff700 1 -- 172.21.15.3:0/1606967890 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291401cc50 con 0x7f292015c8f0 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.453+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292016a5d0 con 0x7f292015c8f0 2023-10-06T04:55:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.453+0000 7f2907fff700 1 -- 172.21.15.3:0/1606967890 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291401a450 con 0x7f292015c8f0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.453+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292016ab90 con 0x7f292015c8f0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.454+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292016b0b0 con 0x7f292015c8f0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.454+0000 7f2907fff700 1 -- 172.21.15.3:0/1606967890 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f291401a840 con 0x7f292015c8f0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.455+0000 7f2907fff700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f290c066f30 0x7f290c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.455+0000 7f2907fff700 1 -- 172.21.15.3:0/1606967890 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f2914085ef0 con 0x7f292015c8f0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.455+0000 7f291ffff700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f290c066f30 0x7f290c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.455+0000 7f291ffff700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f290c066f30 0x7f290c0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2910009db0 tx=0x7f29100091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.459+0000 7f2907fff700 1 -- 172.21.15.3:0/1606967890 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f292016b0b0 con 0x7f292015c8f0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.698+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f292016b0b0 con 0x7f292015c8f0 2023-10-06T04:55:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.698+0000 7f2907fff700 1 -- 172.21.15.3:0/1606967890 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7f292016b0b0 con 0x7f292015c8f0 2023-10-06T04:55:09.485 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:09.485 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:55:09.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f290c066f30 msgr2=0x7f290c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f290c066f30 0x7f290c0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2910009db0 tx=0x7f29100091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f292015c8f0 msgr2=0x7f29201590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f292015c8f0 0x7f29201590a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2914002000 tx=0x7f2914016000 comp rx=0 tx=0).stop 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 shutdown_connections 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f290c066f30 0x7f290c0693e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29200d06a0 0x7f29201699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f292015ccc0 0x7f29201595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.700+0000 7f2926494700 1 --2- 172.21.15.3:0/1606967890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f292015c8f0 0x7f29201590a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.701+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 >> 172.21.15.3:0/1606967890 conn(0x7f2920070310 msgr2=0x7f29200b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.701+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 shutdown_connections 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:08.701+0000 7f2926494700 1 -- 172.21.15.3:0/1606967890 wait complete. 2023-10-06T04:55:09.487 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:55:09.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:55:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.620+0000 7fe2c5a04700 1 Processor -- start 2023-10-06T04:55:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.620+0000 7fe2c5a04700 1 -- start start 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.620+0000 7fe2c5a04700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.620+0000 7fe2c5a04700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe2c00cd300 0x7fe2c00cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.620+0000 7fe2c5a04700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c00aeaa0 con 0x7fe2c0067b30 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.620+0000 7fe2c5a04700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c00aa6d0 con 0x7fe2c00cd300 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.620+0000 7fe2c5a04700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00af920 con 0x7fe2c0156ee0 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bf7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe2c00cd300 0x7fe2c00cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2be7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe2c0067b30 0x7fe2c00af210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60654/0 (socket says 172.21.15.3:60654) 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2be7fc700 1 -- 172.21.15.3:0/1273453639 learned_addr learned my addr 172.21.15.3:0/1273453639 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2beffd700 1 --2- 172.21.15.3:0/1273453639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bf7fe700 1 -- 172.21.15.3:0/1273453639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 msgr2=0x7fe2c00a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bf7fe700 1 --2- 172.21.15.3:0/1273453639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bf7fe700 1 -- 172.21.15.3:0/1273453639 >> v1:172.21.15.3:6789/0 conn(0x7fe2c0067b30 legacy=0x7fe2c00af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bf7fe700 1 -- 172.21.15.3:0/1273453639 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00cdc60 con 0x7fe2c00cd300 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bdffb700 1 -- 172.21.15.3:0/1273453639 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317305353 0 0) 0x7fe2c00aeaa0 con 0x7fe2c0067b30 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bf7fe700 1 --2- 172.21.15.3:0/1273453639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c00cd720 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00098e0 tx=0x7fe2b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cba51522582123 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bdffb700 1 -- 172.21.15.3:0/1273453639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0009e80 con 0x7fe2c00cd300 2023-10-06T04:55:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bdffb700 1 -- 172.21.15.3:0/1273453639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000b860 con 0x7fe2c00cd300 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.621+0000 7fe2bdffb700 1 -- 172.21.15.3:0/1273453639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000bbe0 con 0x7fe2c00cd300 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 -- 172.21.15.3:0/1273453639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 msgr2=0x7fe2c00cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/1273453639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c00cd720 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00098e0 tx=0x7fe2b00091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 -- 172.21.15.3:0/1273453639 shutdown_connections 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/1273453639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c00cd720 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/1273453639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 -- 172.21.15.3:0/1273453639 >> 172.21.15.3:0/1273453639 conn(0x7fe2c00687d0 msgr2=0x7fe2c0068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 -- 172.21.15.3:0/1273453639 shutdown_connections 2023-10-06T04:55:09.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.622+0000 7fe2c5a04700 1 -- 172.21.15.3:0/1273453639 wait complete. 2023-10-06T04:55:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.623+0000 7fe2c5a04700 1 Processor -- start 2023-10-06T04:55:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.623+0000 7fe2c5a04700 1 -- start start 2023-10-06T04:55:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2c5a04700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2c0067b30 0x7fe2c00ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2c5a04700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c0168b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2c5a04700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2c5a04700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0169050 con 0x7fe2c00cd300 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2c5a04700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c01493a0 con 0x7fe2c0067b30 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2c5a04700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c016cb00 con 0x7fe2c0156ee0 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c0168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c0168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37690/0 (socket says 172.21.15.3:37690) 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 -- 172.21.15.3:0/4096959167 learned_addr learned my addr 172.21.15.3:0/4096959167 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2beffd700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2c0067b30 0x7fe2c00ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2bf7fe700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 -- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 msgr2=0x7fe2c00b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 -- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2c0067b30 msgr2=0x7fe2c00ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2c0067b30 0x7fe2c00ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 -- 172.21.15.3:0/4096959167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0009420 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2bf7fe700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.624+0000 7fe2be7fc700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c0168b10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c5b0 tx=0x7fe2a8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.625+0000 7fe2b6ffd700 1 -- 172.21.15.3:0/4096959167 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8005b30 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.625+0000 7fe2b6ffd700 1 -- 172.21.15.3:0/4096959167 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8005c90 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.625+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c016cde0 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.625+0000 7fe2b6ffd700 1 -- 172.21.15.3:0/4096959167 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8007300 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.625+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c016d3a0 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.626+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c004c380 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.626+0000 7fe2b6ffd700 1 -- 172.21.15.3:0/4096959167 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2a801b3b0 con 0x7fe2c00cd300 2023-10-06T04:55:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.626+0000 7fe2b6ffd700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2a0066f30 0x7fe2a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.627+0000 7fe2b6ffd700 1 -- 172.21.15.3:0/4096959167 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fe2a8083c40 con 0x7fe2c00cd300 2023-10-06T04:55:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.627+0000 7fe2beffd700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2a0066f30 0x7fe2a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.627+0000 7fe2beffd700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2a0066f30 0x7fe2a00693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009fd0 tx=0x7fe2b8009880 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.631+0000 7fe2b6ffd700 1 -- 172.21.15.3:0/4096959167 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2a8050c20 con 0x7fe2c00cd300 2023-10-06T04:55:09.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.826+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2c0067160 con 0x7fe2c00cd300 2023-10-06T04:55:09.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.827+0000 7fe2b6ffd700 1 -- 172.21.15.3:0/4096959167 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17058 (secure 0 0 0) 0x7fe2a8017090 con 0x7fe2c00cd300 2023-10-06T04:55:09.829 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:09.829 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":60,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:07.547436+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:54:38.486656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T04:54:39.707870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.1","mappings":[{"from":4,"to":3}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:55:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2a0066f30 msgr2=0x7fe2a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2a0066f30 0x7fe2a00693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009fd0 tx=0x7fe2b8009880 comp rx=0 tx=0).stop 2023-10-06T04:55:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 msgr2=0x7fe2c0168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c0168b10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c5b0 tx=0x7fe2a8001070 comp rx=0 tx=0).stop 2023-10-06T04:55:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 shutdown_connections 2023-10-06T04:55:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2a0066f30 0x7fe2a00693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2c0067b30 0x7fe2c00ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2c0156ee0 0x7fe2c00b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 --2- 172.21.15.3:0/4096959167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2c00cd300 0x7fe2c0168b10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 >> 172.21.15.3:0/4096959167 conn(0x7fe2c00687d0 msgr2=0x7fe2c0063570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:09.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 shutdown_connections 2023-10-06T04:55:09.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.829+0000 7fe2c5a04700 1 -- 172.21.15.3:0/4096959167 wait complete. 2023-10-06T04:55:09.843 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:55:09.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:09.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.964+0000 7f1f3b28b700 1 Processor -- start 2023-10-06T04:55:09.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.964+0000 7f1f3b28b700 1 -- start start 2023-10-06T04:55:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3b28b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f3406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3b28b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f340aeca0 0x7f1f341566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3b28b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1f340adc30 0x7f1f340ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3b28b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f340ae5f0 con 0x7f1f340aeca0 2023-10-06T04:55:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3b28b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f340b0c70 con 0x7f1f340adc30 2023-10-06T04:55:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3b28b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34159cd0 con 0x7f1f340b08a0 2023-10-06T04:55:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f340aeca0 0x7f1f341566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f340aeca0 0x7f1f341566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37692/0 (socket says 172.21.15.3:37692) 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 -- 172.21.15.3:0/2914228637 learned_addr learned my addr 172.21.15.3:0/2914228637 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3a289700 1 --2- 172.21.15.3:0/2914228637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f3406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f3aa8a700 1 --2- 172.21.15.3:0/2914228637 >> v2:172.21.15.153:3300/0 conn(0x7f1f340adc30 0x7f1f340ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 -- 172.21.15.3:0/2914228637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 msgr2=0x7f1f3406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 --2- 172.21.15.3:0/2914228637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f3406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 -- 172.21.15.3:0/2914228637 >> v2:172.21.15.153:3300/0 conn(0x7f1f340adc30 msgr2=0x7f1f340ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 --2- 172.21.15.3:0/2914228637 >> v2:172.21.15.153:3300/0 conn(0x7f1f340adc30 0x7f1f340ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.965+0000 7f1f39a88700 1 -- 172.21.15.3:0/2914228637 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340af210 con 0x7f1f340aeca0 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f39a88700 1 --2- 172.21.15.3:0/2914228637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340aeca0 0x7f1f341566d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1f340af110 tx=0x7f1f28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee90c7b90c432ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f39287700 1 -- 172.21.15.3:0/2914228637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800b820 con 0x7f1f340aeca0 2023-10-06T04:55:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f39287700 1 -- 172.21.15.3:0/2914228637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800b980 con 0x7f1f340aeca0 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f39287700 1 -- 172.21.15.3:0/2914228637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800bd20 con 0x7f1f340aeca0 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 -- 172.21.15.3:0/2914228637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340aeca0 msgr2=0x7f1f341566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/2914228637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340aeca0 0x7f1f341566d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1f340af110 tx=0x7f1f28009250 comp rx=0 tx=0).stop 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 -- 172.21.15.3:0/2914228637 shutdown_connections 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/2914228637 >> v2:172.21.15.153:3300/0 conn(0x7f1f340adc30 0x7f1f340ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/2914228637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f3406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/2914228637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340aeca0 0x7f1f341566d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 -- 172.21.15.3:0/2914228637 >> 172.21.15.3:0/2914228637 conn(0x7f1f3406c370 msgr2=0x7f1f3406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 -- 172.21.15.3:0/2914228637 shutdown_connections 2023-10-06T04:55:09.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.966+0000 7f1f3b28b700 1 -- 172.21.15.3:0/2914228637 wait complete. 2023-10-06T04:55:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.968+0000 7f1f3b28b700 1 Processor -- start 2023-10-06T04:55:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.968+0000 7f1f3b28b700 1 -- start start 2023-10-06T04:55:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3b28b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340adc30 0x7f1f3406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3b28b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340aeca0 0x7f1f34155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3b28b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f34150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3b28b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34152e30 con 0x7f1f340adc30 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3b28b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3406a9a0 con 0x7f1f340aeca0 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3b28b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3406ac80 con 0x7f1f340b08a0 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340adc30 0x7f1f3406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340adc30 0x7f1f3406a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37708/0 (socket says 172.21.15.3:37708) 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 -- 172.21.15.3:0/1405150765 learned_addr learned my addr 172.21.15.3:0/1405150765 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3aa8a700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f34150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f39a88700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340aeca0 0x7f1f34155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 -- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 msgr2=0x7f1f34150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f34150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 -- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340aeca0 msgr2=0x7f1f34155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340aeca0 0x7f1f34155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 -- 172.21.15.3:0/1405150765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f280094f0 con 0x7f1f340adc30 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.969+0000 7f1f3a289700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340adc30 0x7f1f3406a170 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d860 tx=0x7f1f2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.970+0000 7f1f327fc700 1 -- 172.21.15.3:0/1405150765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c009610 con 0x7f1f340adc30 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.970+0000 7f1f327fc700 1 -- 172.21.15.3:0/1405150765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c009770 con 0x7f1f340adc30 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.970+0000 7f1f327fc700 1 -- 172.21.15.3:0/1405150765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c009b30 con 0x7f1f340adc30 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.970+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3406af60 con 0x7f1f340adc30 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.970+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3406a6b0 con 0x7f1f340adc30 2023-10-06T04:55:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.970+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3404c380 con 0x7f1f340adc30 2023-10-06T04:55:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.971+0000 7f1f327fc700 1 -- 172.21.15.3:0/1405150765 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f2c01e460 con 0x7f1f340adc30 2023-10-06T04:55:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.971+0000 7f1f327fc700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f20066f30 0x7f1f200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.971+0000 7f1f327fc700 1 -- 172.21.15.3:0/1405150765 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f1f2c084d90 con 0x7f1f340adc30 2023-10-06T04:55:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.972+0000 7f1f39a88700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f20066f30 0x7f1f200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.972+0000 7f1f39a88700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f20066f30 0x7f1f200693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f340af110 tx=0x7f1f28017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:09.976+0000 7f1f327fc700 1 -- 172.21.15.3:0/1405150765 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f2c051e10 con 0x7f1f340adc30 2023-10-06T04:55:10.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.212+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f340af4f0 con 0x7f1f340adc30 2023-10-06T04:55:10.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.212+0000 7f1f327fc700 1 -- 172.21.15.3:0/1405150765 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7f1f2c055430 con 0x7f1f340adc30 2023-10-06T04:55:10.214 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:10.214 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:55:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.214+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f20066f30 msgr2=0x7f1f200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f20066f30 0x7f1f200693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f340af110 tx=0x7f1f28017000 comp rx=0 tx=0).stop 2023-10-06T04:55:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340adc30 msgr2=0x7f1f3406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340adc30 0x7f1f3406a170 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d860 tx=0x7f1f2c008040 comp rx=0 tx=0).stop 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 shutdown_connections 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1f20066f30 0x7f1f200693e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b08a0 0x7f1f34150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f340adc30 0x7f1f3406a170 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 --2- 172.21.15.3:0/1405150765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340aeca0 0x7f1f34155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 >> 172.21.15.3:0/1405150765 conn(0x7f1f3406c370 msgr2=0x7f1f34152390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 shutdown_connections 2023-10-06T04:55:10.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.215+0000 7f1f3b28b700 1 -- 172.21.15.3:0/1405150765 wait complete. 2023-10-06T04:55:10.218 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:55:10.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:10.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.348+0000 7faf070e0700 1 Processor -- start 2023-10-06T04:55:10.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.348+0000 7faf070e0700 1 -- start start 2023-10-06T04:55:10.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faf070e0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf00156110 0x7faf0006aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faf070e0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faf001564e0 0x7faf0006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faf070e0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000c7410 con 0x7faf000c68d0 2023-10-06T04:55:10.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faf070e0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf000acbb0 con 0x7faf001564e0 2023-10-06T04:55:10.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faf070e0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000c7640 con 0x7faf00156110 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faefffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faf001564e0 0x7faf0006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faf0567d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faf000c68d0 0x7faf000c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60668/0 (socket says 172.21.15.3:60668) 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.349+0000 7faf0567d700 1 -- 172.21.15.3:0/1375157289 learned_addr learned my addr 172.21.15.3:0/1375157289 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faefffff700 1 -- 172.21.15.3:0/1375157289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf00156110 msgr2=0x7faf0006aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf04e7c700 1 --2- 172.21.15.3:0/1375157289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf00156110 0x7faf0006aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faefffff700 1 --2- 172.21.15.3:0/1375157289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf00156110 0x7faf0006aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faefffff700 1 -- 172.21.15.3:0/1375157289 >> v1:172.21.15.3:6789/0 conn(0x7faf000c68d0 legacy=0x7faf000c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faefffff700 1 -- 172.21.15.3:0/1375157289 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000c7920 con 0x7faf001564e0 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf04e7c700 1 --2- 172.21.15.3:0/1375157289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf00156110 0x7faf0006aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faeff7fe700 1 -- 172.21.15.3:0/1375157289 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653096799 0 0) 0x7faf000c7410 con 0x7faf000c68d0 2023-10-06T04:55:10.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faefffff700 1 --2- 172.21.15.3:0/1375157289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf001564e0 0x7faf0006afd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7faef40098e0 tx=0x7faef40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a837a98b1bdbdceb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faeff7fe700 1 -- 172.21.15.3:0/1375157289 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4009e80 con 0x7faf001564e0 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faeff7fe700 1 -- 172.21.15.3:0/1375157289 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef400b860 con 0x7faf001564e0 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faeff7fe700 1 -- 172.21.15.3:0/1375157289 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400bbe0 con 0x7faf001564e0 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf070e0700 1 -- 172.21.15.3:0/1375157289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf001564e0 msgr2=0x7faf0006afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf070e0700 1 --2- 172.21.15.3:0/1375157289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf001564e0 0x7faf0006afd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7faef40098e0 tx=0x7faef40091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf070e0700 1 -- 172.21.15.3:0/1375157289 shutdown_connections 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf070e0700 1 --2- 172.21.15.3:0/1375157289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf00156110 0x7faf0006aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf070e0700 1 --2- 172.21.15.3:0/1375157289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf001564e0 0x7faf0006afd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.350+0000 7faf070e0700 1 -- 172.21.15.3:0/1375157289 >> 172.21.15.3:0/1375157289 conn(0x7faf0006c370 msgr2=0x7faf0006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.351+0000 7faf070e0700 1 -- 172.21.15.3:0/1375157289 shutdown_connections 2023-10-06T04:55:10.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.351+0000 7faf070e0700 1 -- 172.21.15.3:0/1375157289 wait complete. 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.352+0000 7faf070e0700 1 Processor -- start 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.352+0000 7faf070e0700 1 -- start start 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.352+0000 7faf070e0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf000c68d0 0x7faf001523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.352+0000 7faf070e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf00156110 0x7faf0006e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.352+0000 7faf070e0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf001564e0 0x7faf0006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.352+0000 7faf070e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0015cd50 con 0x7faf00156110 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faf070e0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0015a9c0 con 0x7faf000c68d0 2023-10-06T04:55:10.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faf070e0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf00159db0 con 0x7faf001564e0 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf00156110 0x7faf0006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf00156110 0x7faf0006e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37726/0 (socket says 172.21.15.3:37726) 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 -- 172.21.15.3:0/4010726803 learned_addr learned my addr 172.21.15.3:0/4010726803 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faf0567d700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf001564e0 0x7faf0006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faf04e7c700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf000c68d0 0x7faf001523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 -- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf001564e0 msgr2=0x7faf0006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf001564e0 0x7faf0006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 -- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf000c68d0 msgr2=0x7faf001523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf000c68d0 0x7faf001523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 -- 172.21.15.3:0/4010726803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4009420 con 0x7faf00156110 2023-10-06T04:55:10.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefffff700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf00156110 0x7faf0006e590 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faf0006fcc0 tx=0x7faef4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefcff9700 1 -- 172.21.15.3:0/4010726803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400bbe0 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefcff9700 1 -- 172.21.15.3:0/4010726803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4011420 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0015a090 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faefcff9700 1 -- 172.21.15.3:0/4010726803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40117c0 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.353+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0015a650 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.354+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0015ab70 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.355+0000 7faefcff9700 1 -- 172.21.15.3:0/4010726803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faef4011bb0 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.355+0000 7faefcff9700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faeec066f30 0x7faeec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.355+0000 7faefcff9700 1 -- 172.21.15.3:0/4010726803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7faef4085e30 con 0x7faf00156110 2023-10-06T04:55:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.355+0000 7faf04e7c700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faeec066f30 0x7faeec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.356+0000 7faf04e7c700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faeec066f30 0x7faeec0693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faef000ac00 tx=0x7faef0006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:10.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.359+0000 7faefcff9700 1 -- 172.21.15.3:0/4010726803 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf0015ab70 con 0x7faf00156110 2023-10-06T04:55:10.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.600+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf0015ab30 con 0x7faf00156110 2023-10-06T04:55:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.600+0000 7faefcff9700 1 -- 172.21.15.3:0/4010726803 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4592 (secure 0 0 0) 0x7faef4011e60 con 0x7faf00156110 2023-10-06T04:55:10.602 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:10.602 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:54:40.735227+0000","modified":"2023-10-06T04:54:44.356805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/3909985257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3909985257},{"type":"v1","addr":"172.21.15.3:6835","nonce":3909985257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T04:55:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faeec066f30 msgr2=0x7faeec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faeec066f30 0x7faeec0693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faef000ac00 tx=0x7faef0006f90 comp rx=0 tx=0).stop 2023-10-06T04:55:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf00156110 msgr2=0x7faf0006e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf00156110 0x7faf0006e590 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faf0006fcc0 tx=0x7faef4017000 comp rx=0 tx=0).stop 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 shutdown_connections 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faeec066f30 0x7faeec0693e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf000c68d0 0x7faf001523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf001564e0 0x7faf0006ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 --2- 172.21.15.3:0/4010726803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf00156110 0x7faf0006e590 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 >> 172.21.15.3:0/4010726803 conn(0x7faf0006c370 msgr2=0x7faf000ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 shutdown_connections 2023-10-06T04:55:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.603+0000 7faf070e0700 1 -- 172.21.15.3:0/4010726803 wait complete. 2023-10-06T04:55:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-10-06T04:55:10.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:55:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.735+0000 7f26f65c6700 1 Processor -- start 2023-10-06T04:55:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.735+0000 7f26f65c6700 1 -- start start 2023-10-06T04:55:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f65c6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26f0157030 0x7f26f0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f65c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26f0157400 0x7f26f006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f65c6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f015aeb0 con 0x7f26f015a220 2023-10-06T04:55:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f65c6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f015a7a0 con 0x7f26f0157400 2023-10-06T04:55:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f65c6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f015aa80 con 0x7f26f0157030 2023-10-06T04:55:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f4b63700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26f0157400 0x7f26f006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26ef7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f26f015a220 0x7f26f00b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60678/0 (socket says 172.21.15.3:60678) 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26ef7fe700 1 -- 172.21.15.3:0/359699704 learned_addr learned my addr 172.21.15.3:0/359699704 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26effff700 1 --2- 172.21.15.3:0/359699704 >> v2:172.21.15.153:3300/0 conn(0x7f26f0157030 0x7f26f0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f4b63700 1 -- 172.21.15.3:0/359699704 >> v2:172.21.15.153:3300/0 conn(0x7f26f0157030 msgr2=0x7f26f0159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f4b63700 1 --2- 172.21.15.3:0/359699704 >> v2:172.21.15.153:3300/0 conn(0x7f26f0157030 0x7f26f0159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f4b63700 1 -- 172.21.15.3:0/359699704 >> v1:172.21.15.3:6789/0 conn(0x7f26f015a220 legacy=0x7f26f00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f4b63700 1 -- 172.21.15.3:0/359699704 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f006dba0 con 0x7f26f0157400 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26effff700 1 --2- 172.21.15.3:0/359699704 >> v2:172.21.15.153:3300/0 conn(0x7f26f0157030 0x7f26f0159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26f4b63700 1 --2- 172.21.15.3:0/359699704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f006e3a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f26f006b660 tx=0x7f26e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3885e380da4991f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26eeffd700 1 -- 172.21.15.3:0/359699704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800e070 con 0x7f26f0157400 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26eeffd700 1 -- 172.21.15.3:0/359699704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800bc30 con 0x7f26f0157400 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.736+0000 7f26eeffd700 1 -- 172.21.15.3:0/359699704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8012430 con 0x7f26f0157400 2023-10-06T04:55:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 -- 172.21.15.3:0/359699704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 msgr2=0x7f26f006e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 --2- 172.21.15.3:0/359699704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f006e3a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f26f006b660 tx=0x7f26e8009780 comp rx=0 tx=0).stop 2023-10-06T04:55:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 -- 172.21.15.3:0/359699704 shutdown_connections 2023-10-06T04:55:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 --2- 172.21.15.3:0/359699704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f006e3a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 --2- 172.21.15.3:0/359699704 >> v2:172.21.15.153:3300/0 conn(0x7f26f0157030 0x7f26f0159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 -- 172.21.15.3:0/359699704 >> 172.21.15.3:0/359699704 conn(0x7f26f00ce270 msgr2=0x7f26f00ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 -- 172.21.15.3:0/359699704 shutdown_connections 2023-10-06T04:55:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.737+0000 7f26f65c6700 1 -- 172.21.15.3:0/359699704 wait complete. 2023-10-06T04:55:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.738+0000 7f26f65c6700 1 Processor -- start 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.738+0000 7f26f65c6700 1 -- start start 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26f65c6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f0157030 0x7f26f016a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26f65c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f0159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26f65c6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f015a220 0x7f26f00cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26f65c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00cb950 con 0x7f26f0157400 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26f65c6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0159db0 con 0x7f26f0157030 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26f65c6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00cb2c0 con 0x7f26f015a220 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f0159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f0159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37732/0 (socket says 172.21.15.3:37732) 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 -- 172.21.15.3:0/2005599804 learned_addr learned my addr 172.21.15.3:0/2005599804 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26effff700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f0157030 0x7f26f016a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26f4b63700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f015a220 0x7f26f00cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 -- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f015a220 msgr2=0x7f26f00cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f015a220 0x7f26f00cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 -- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f0157030 msgr2=0x7f26f016a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f0157030 0x7f26f016a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.739+0000 7f26ef7fe700 1 -- 172.21.15.3:0/2005599804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8009a20 con 0x7f26f0157400 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.740+0000 7f26ef7fe700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f0159870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c590 tx=0x7f26d8001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.740+0000 7f26e7fff700 1 -- 172.21.15.3:0/2005599804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8005b30 con 0x7f26f0157400 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.740+0000 7f26e7fff700 1 -- 172.21.15.3:0/2005599804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d8005cb0 con 0x7f26f0157400 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.740+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f00aef70 con 0x7f26f0157400 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.740+0000 7f26e7fff700 1 -- 172.21.15.3:0/2005599804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8007320 con 0x7f26f0157400 2023-10-06T04:55:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.740+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f00aec90 con 0x7f26f0157400 2023-10-06T04:55:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.741+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f00644c0 con 0x7f26f0157400 2023-10-06T04:55:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.741+0000 7f26e7fff700 1 -- 172.21.15.3:0/2005599804 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26d801b3b0 con 0x7f26f0157400 2023-10-06T04:55:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.741+0000 7f26e7fff700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26d0066f30 0x7f26d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.742+0000 7f26e7fff700 1 -- 172.21.15.3:0/2005599804 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f26d8083c20 con 0x7f26f0157400 2023-10-06T04:55:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.742+0000 7f26effff700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26d0066f30 0x7f26d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.742+0000 7f26effff700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26d0066f30 0x7f26d00693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26e000ac00 tx=0x7f26e0006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.746+0000 7f26e7fff700 1 -- 172.21.15.3:0/2005599804 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26d8050ca0 con 0x7f26f0157400 2023-10-06T04:55:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.944+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f0071680 con 0x7f26f0157400 2023-10-06T04:55:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.945+0000 7f26e7fff700 1 -- 172.21.15.3:0/2005599804 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17058 (secure 0 0 0) 0x7f26d8017020 con 0x7f26f0157400 2023-10-06T04:55:10.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:10.947 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":60,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:07.547436+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:54:38.486656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T04:54:39.707870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.1","mappings":[{"from":4,"to":3}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26d0066f30 msgr2=0x7f26d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26d0066f30 0x7f26d00693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26e000ac00 tx=0x7f26e0006f90 comp rx=0 tx=0).stop 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 msgr2=0x7f26f0159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f0159870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c590 tx=0x7f26d8001020 comp rx=0 tx=0).stop 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 shutdown_connections 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26d0066f30 0x7f26d00693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f0157030 0x7f26f016a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f015a220 0x7f26f00cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 --2- 172.21.15.3:0/2005599804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f0157400 0x7f26f0159870 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 >> 172.21.15.3:0/2005599804 conn(0x7f26f00ce270 msgr2=0x7f26f00b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 shutdown_connections 2023-10-06T04:55:10.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:10.947+0000 7f26f65c6700 1 -- 172.21.15.3:0/2005599804 wait complete. 2023-10-06T04:55:10.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:55:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.079+0000 7ff87a343700 1 Processor -- start 2023-10-06T04:55:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.079+0000 7ff87a343700 1 -- start start 2023-10-06T04:55:11.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff87a343700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415ccc0 0x7ff8740d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff87a343700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8740d06a0 0x7ff8740b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff87a343700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740cf6c0 con 0x7ff87415c8f0 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff87a343700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8740cfb50 con 0x7ff8740d06a0 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff87a343700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740d0b20 con 0x7ff87415ccc0 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff8788e0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8740d06a0 0x7ff8740b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff8788e0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8740d06a0 0x7ff8740b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37738/0 (socket says 172.21.15.3:37738) 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff873fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff87415c8f0 0x7ff87415db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60686/0 (socket says 172.21.15.3:60686) 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.080+0000 7ff8788e0700 1 -- 172.21.15.3:0/639479854 learned_addr learned my addr 172.21.15.3:0/639479854 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff8737fe700 1 --2- 172.21.15.3:0/639479854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415ccc0 0x7ff8740d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff8788e0700 1 -- 172.21.15.3:0/639479854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415ccc0 msgr2=0x7ff8740d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff8788e0700 1 --2- 172.21.15.3:0/639479854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415ccc0 0x7ff8740d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff8788e0700 1 -- 172.21.15.3:0/639479854 >> v1:172.21.15.3:6789/0 conn(0x7ff87415c8f0 legacy=0x7ff87415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff872ffd700 1 -- 172.21.15.3:0/639479854 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987268688 0 0) 0x7ff8740cf6c0 con 0x7ff87415c8f0 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff8788e0700 1 -- 172.21.15.3:0/639479854 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740ceab0 con 0x7ff8740d06a0 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff8737fe700 1 --2- 172.21.15.3:0/639479854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415ccc0 0x7ff8740d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff8788e0700 1 --2- 172.21.15.3:0/639479854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff8740b3ba0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009f20 tx=0x7ff86c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6c28fb6e3ced15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff872ffd700 1 -- 172.21.15.3:0/639479854 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00e070 con 0x7ff8740d06a0 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff872ffd700 1 -- 172.21.15.3:0/639479854 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c00ba80 con 0x7ff8740d06a0 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.081+0000 7ff872ffd700 1 -- 172.21.15.3:0/639479854 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0092e0 con 0x7ff8740d06a0 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 -- 172.21.15.3:0/639479854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 msgr2=0x7ff8740b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 --2- 172.21.15.3:0/639479854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff8740b3ba0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009f20 tx=0x7ff86c009840 comp rx=0 tx=0).stop 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 -- 172.21.15.3:0/639479854 shutdown_connections 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 --2- 172.21.15.3:0/639479854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff8740b3ba0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 --2- 172.21.15.3:0/639479854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415ccc0 0x7ff8740d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 -- 172.21.15.3:0/639479854 >> 172.21.15.3:0/639479854 conn(0x7ff874070310 msgr2=0x7ff874070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 -- 172.21.15.3:0/639479854 shutdown_connections 2023-10-06T04:55:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.082+0000 7ff87a343700 1 -- 172.21.15.3:0/639479854 wait complete. 2023-10-06T04:55:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.083+0000 7ff87a343700 1 Processor -- start 2023-10-06T04:55:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.083+0000 7ff87a343700 1 -- start start 2023-10-06T04:55:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff87414ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415c8f0 0x7ff8740c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff87415ccc0 0x7ff8740ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740c6590 con 0x7ff8740d06a0 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740c6870 con 0x7ff87415ccc0 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740c6b50 con 0x7ff87415c8f0 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff87414ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff87414ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37742/0 (socket says 172.21.15.3:37742) 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 -- 172.21.15.3:0/2831360364 learned_addr learned my addr 172.21.15.3:0/2831360364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 -- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415c8f0 msgr2=0x7ff8740c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415c8f0 0x7ff8740c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 -- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff87415ccc0 msgr2=0x7ff8740ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff87415ccc0 0x7ff8740ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 -- 172.21.15.3:0/2831360364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c009ae0 con 0x7ff8740d06a0 2023-10-06T04:55:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff873fff700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff87414ce40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff8640011e0 tx=0x7ff864007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff85bfff700 1 -- 172.21.15.3:0/2831360364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff864007300 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff85bfff700 1 -- 172.21.15.3:0/2831360364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff864007460 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8740c6e30 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff85bfff700 1 -- 172.21.15.3:0/2831360364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff864004830 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.084+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8740c61f0 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.085+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff874071680 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.086+0000 7ff85bfff700 1 -- 172.21.15.3:0/2831360364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff864004be0 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.086+0000 7ff85bfff700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.086+0000 7ff85bfff700 1 -- 172.21.15.3:0/2831360364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6203+0+0 (secure 0 0 0) 0x7ff864083c30 con 0x7ff8740d06a0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.086+0000 7ff8737fe700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.087+0000 7ff8737fe700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff860066f30 0x7ff8600693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff868006fd0 tx=0x7ff868008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.090+0000 7ff85bfff700 1 -- 172.21.15.3:0/2831360364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff864050d30 con 0x7ff8740d06a0 2023-10-06T04:55:11.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.328+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff8740d0480 con 0x7ff8740d06a0 2023-10-06T04:55:11.330 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T04:55:11.329+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:55:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.333+0000 7ff85bfff700 1 -- 172.21.15.3:0/2831360364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff864018020 con 0x7ff8740d06a0 2023-10-06T04:55:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff860066f30 msgr2=0x7ff8600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff860066f30 0x7ff8600693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff868006fd0 tx=0x7ff868008040 comp rx=0 tx=0).stop 2023-10-06T04:55:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 msgr2=0x7ff87414ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff87414ce40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff8640011e0 tx=0x7ff864007e10 comp rx=0 tx=0).stop 2023-10-06T04:55:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 shutdown_connections 2023-10-06T04:55:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff87415ccc0 0x7ff8740ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8740d06a0 0x7ff87414ce40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 --2- 172.21.15.3:0/2831360364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff87415c8f0 0x7ff8740c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.335+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 >> 172.21.15.3:0/2831360364 conn(0x7ff874070310 msgr2=0x7ff8740cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.336+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 shutdown_connections 2023-10-06T04:55:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.336+0000 7ff87a343700 1 -- 172.21.15.3:0/2831360364 wait complete. 2023-10-06T04:55:11.340 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:55:11.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:55:11.369 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:55:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.470+0000 7f026dcc2700 1 Processor -- start 2023-10-06T04:55:11.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.470+0000 7f026dcc2700 1 -- start start 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f026dcc2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0268143140 0x7f0268067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f026dcc2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 0x7f026806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f026dcc2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0268068450 con 0x7f0268142d70 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f026dcc2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680cc880 con 0x7f0268143140 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f026dcc2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0268069f10 con 0x7f0268067ff0 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f0266ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0268143140 0x7f0268067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f02677fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0268142d70 0x7f02680cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60690/0 (socket says 172.21.15.3:60690) 2023-10-06T04:55:11.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f02677fe700 1 -- 172.21.15.3:0/1894309973 learned_addr learned my addr 172.21.15.3:0/1894309973 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f02667fc700 1 -- 172.21.15.3:0/1894309973 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583932991 0 0) 0x7f0260001110 con 0x7f0268142d70 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f0266ffd700 1 -- 172.21.15.3:0/1894309973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 msgr2=0x7f026806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f0266ffd700 1 --2- 172.21.15.3:0/1894309973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 0x7f026806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f0266ffd700 1 -- 172.21.15.3:0/1894309973 >> v1:172.21.15.3:6789/0 conn(0x7f0268142d70 legacy=0x7f02680cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f0266ffd700 1 -- 172.21.15.3:0/1894309973 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268069a80 con 0x7f0268143140 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.471+0000 7f0266ffd700 1 --2- 172.21.15.3:0/1894309973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268143140 0x7f0268067ab0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0250009910 tx=0x7f02500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3257e2938659b013 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f02667fc700 1 -- 172.21.15.3:0/1894309973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000b820 con 0x7f0268143140 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f02667fc700 1 -- 172.21.15.3:0/1894309973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025000b980 con 0x7f0268143140 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f02667fc700 1 -- 172.21.15.3:0/1894309973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000bd20 con 0x7f0268143140 2023-10-06T04:55:11.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 -- 172.21.15.3:0/1894309973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268143140 msgr2=0x7f0268067ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 --2- 172.21.15.3:0/1894309973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268143140 0x7f0268067ab0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0250009910 tx=0x7f02500091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 -- 172.21.15.3:0/1894309973 shutdown_connections 2023-10-06T04:55:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 --2- 172.21.15.3:0/1894309973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 0x7f026806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 --2- 172.21.15.3:0/1894309973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268143140 0x7f0268067ab0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 -- 172.21.15.3:0/1894309973 >> 172.21.15.3:0/1894309973 conn(0x7f02680ce210 msgr2=0x7f02680ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 -- 172.21.15.3:0/1894309973 shutdown_connections 2023-10-06T04:55:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.472+0000 7f026dcc2700 1 -- 172.21.15.3:0/1894309973 wait complete. 2023-10-06T04:55:11.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.473+0000 7f026dcc2700 1 Processor -- start 2023-10-06T04:55:11.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f026dcc2700 1 -- start start 2023-10-06T04:55:11.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f026dcc2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 0x7f026813e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f026dcc2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268142d70 0x7f02680c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f026dcc2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0268143140 0x7f02680b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f026dcc2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c7080 con 0x7f0268142d70 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f026dcc2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c7360 con 0x7f0268143140 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f026dcc2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c7640 con 0x7f0268067ff0 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f0266ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268142d70 0x7f02680c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f0266ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268142d70 0x7f02680c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37776/0 (socket says 172.21.15.3:37776) 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.474+0000 7f0266ffd700 1 -- 172.21.15.3:0/4192527529 learned_addr learned my addr 172.21.15.3:0/4192527529 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f02677fe700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 0x7f026813e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0267fff700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0268143140 0x7f02680b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0266ffd700 1 -- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 msgr2=0x7f026813e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0266ffd700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 0x7f026813e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0266ffd700 1 -- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0268143140 msgr2=0x7f02680b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0266ffd700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0268143140 0x7f02680b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0266ffd700 1 -- 172.21.15.3:0/4192527529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250009420 con 0x7f0268142d70 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0267fff700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0268143140 0x7f02680b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f0266ffd700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268142d70 0x7f02680c6730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f026806b690 tx=0x7f0250017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f025f7fe700 1 -- 172.21.15.3:0/4192527529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250011e20 con 0x7f0268142d70 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f025f7fe700 1 -- 172.21.15.3:0/4192527529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0250024440 con 0x7f0268142d70 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02680c7920 con 0x7f0268142d70 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f025f7fe700 1 -- 172.21.15.3:0/4192527529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250024800 con 0x7f0268142d70 2023-10-06T04:55:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.475+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02680c8120 con 0x7f0268142d70 2023-10-06T04:55:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.476+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02680c7e40 con 0x7f0268142d70 2023-10-06T04:55:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.477+0000 7f025f7fe700 1 -- 172.21.15.3:0/4192527529 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f025002d440 con 0x7f0268142d70 2023-10-06T04:55:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.477+0000 7f025f7fe700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0248066f30 0x7f02480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.477+0000 7f025f7fe700 1 -- 172.21.15.3:0/4192527529 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6289+0+0 (secure 0 0 0) 0x7f0250086240 con 0x7f0268142d70 2023-10-06T04:55:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.477+0000 7f02677fe700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0248066f30 0x7f02480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.478+0000 7f02677fe700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0248066f30 0x7f02480693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02680b3d20 tx=0x7f0260007f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.481+0000 7f025f7fe700 1 -- 172.21.15.3:0/4192527529 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02680c7e40 con 0x7f0268142d70 2023-10-06T04:55:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:11.718+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f02680c7e40 con 0x7f0268142d70 2023-10-06T04:55:12.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.335+0000 7f025f7fe700 1 -- 172.21.15.3:0/4192527529 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02680c7e40 con 0x7f0268142d70 2023-10-06T04:55:12.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.338+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0248066f30 msgr2=0x7f02480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.338+0000 7f026dcc2700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0248066f30 0x7f02480693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02680b3d20 tx=0x7f0260007f50 comp rx=0 tx=0).stop 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.338+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268142d70 msgr2=0x7f02680c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.338+0000 7f026dcc2700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268142d70 0x7f02680c6730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f026806b690 tx=0x7f0250017000 comp rx=0 tx=0).stop 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 shutdown_connections 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0248066f30 0x7f02480693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0268143140 0x7f02680b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0268067ff0 0x7f026813e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 --2- 172.21.15.3:0/4192527529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0268142d70 0x7f02680c6730 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 >> 172.21.15.3:0/4192527529 conn(0x7f02680ce210 msgr2=0x7f026806e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:12.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 shutdown_connections 2023-10-06T04:55:12.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.339+0000 7f026dcc2700 1 -- 172.21.15.3:0/4192527529 wait complete. 2023-10-06T04:55:12.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:55:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.525+0000 7fdca981b700 1 Processor -- start 2023-10-06T04:55:12.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.525+0000 7fdca981b700 1 -- start start 2023-10-06T04:55:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca981b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 0x7fdca40cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca981b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdca40d1170 0x7fdca4159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca981b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca40ccda0 con 0x7fdca41598f0 2023-10-06T04:55:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca981b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca415e400 con 0x7fdca40d1170 2023-10-06T04:55:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca981b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40d1610 con 0x7fdca415dca0 2023-10-06T04:55:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdca41598f0 0x7fdca415d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44530/0 (socket says 172.21.15.3:44530) 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca2ffd700 1 -- 172.21.15.3:0/2616364081 learned_addr learned my addr 172.21.15.3:0/2616364081 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca37fe700 1 --2- 172.21.15.3:0/2616364081 >> v2:172.21.15.153:3300/0 conn(0x7fdca40d1170 0x7fdca4159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca27fc700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 0x7fdca40cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646169076 0 0) 0x7fdca40ccda0 con 0x7fdca41598f0 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc8c003570 con 0x7fdca41598f0 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.526+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171922362 0 0) 0x7fdc8c003570 con 0x7fdca41598f0 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 msgr2=0x7fdca40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 0x7fdca40cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 >> v2:172.21.15.153:3300/0 conn(0x7fdca40d1170 msgr2=0x7fdca4159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 --2- 172.21.15.3:0/2616364081 >> v2:172.21.15.153:3300/0 conn(0x7fdca40d1170 0x7fdca4159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca40cf7d0 con 0x7fdca41598f0 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fdc94001470 con 0x7fdca41598f0 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 --2- 172.21.15.3:0/2616364081 >> v2:172.21.15.153:3300/0 conn(0x7fdca40d1170 0x7fdca4159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 >> v1:172.21.15.3:6789/0 conn(0x7fdca41598f0 legacy=0x7fdca415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 0x7fdc8c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c003fe0 0x7fdc8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc8c004d50 0x7fdc8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca415e400 con 0x7fdc8c004d50 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40d1610 con 0x7fdc8c003fe0 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c00ba90 con 0x7fdca415dca0 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc8c004d50 0x7fdc8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca2ffd700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 0x7fdc8c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca27fc700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c003fe0 0x7fdc8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 -- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 msgr2=0x7fdc8c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca415dca0 0x7fdc8c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 -- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c003fe0 msgr2=0x7fdc8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c003fe0 0x7fdc8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 -- 172.21.15.3:0/2616364081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c003570 con 0x7fdc8c004d50 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca37fe700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc8c004d50 0x7fdc8c0051c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e3c0 tx=0x7fdc9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53791761182d94e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00ec10 con 0x7fdc8c004d50 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c00f040 con 0x7fdc8c004d50 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.527+0000 7fdca1ffb700 1 -- 172.21.15.3:0/2616364081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c014480 con 0x7fdc8c004d50 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.528+0000 7fdca981b700 1 -- 172.21.15.3:0/2616364081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc8c004d50 msgr2=0x7fdc8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.528+0000 7fdca981b700 1 --2- 172.21.15.3:0/2616364081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc8c004d50 0x7fdc8c0051c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e3c0 tx=0x7fdc9c00b040 comp rx=0 tx=0).stop 2023-10-06T04:55:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.528+0000 7fdca37fe700 1 -- 172.21.15.3:0/2616364081 reap_dead start 2023-10-06T04:55:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.528+0000 7fdca981b700 1 -- 172.21.15.3:0/2616364081 shutdown_connections 2023-10-06T04:55:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.528+0000 7fdca981b700 1 -- 172.21.15.3:0/2616364081 >> 172.21.15.3:0/2616364081 conn(0x7fdca4070af0 msgr2=0x7fdca40d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.528+0000 7fdca981b700 1 -- 172.21.15.3:0/2616364081 shutdown_connections 2023-10-06T04:55:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.528+0000 7fdca981b700 1 -- 172.21.15.3:0/2616364081 wait complete. 2023-10-06T04:55:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.529+0000 7fdca981b700 1 Processor -- start 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.529+0000 7fdca981b700 1 -- start start 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca981b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c004d50 0x7fdca416adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca981b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca416a920 0x7fdca4152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca981b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca415c5c0 0x7fdca415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca981b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40b40f0 con 0x7fdca415c5c0 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca981b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40b3520 con 0x7fdc8c004d50 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca981b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4153040 con 0x7fdca416a920 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca415c5c0 0x7fdca415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca415c5c0 0x7fdca415ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44060/0 (socket says 172.21.15.3:44060) 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 -- 172.21.15.3:0/1916213344 learned_addr learned my addr 172.21.15.3:0/1916213344 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca2ffd700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c004d50 0x7fdca416adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca27fc700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca416a920 0x7fdca4152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 -- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca416a920 msgr2=0x7fdca4152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca416a920 0x7fdca4152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 -- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c004d50 msgr2=0x7fdca416adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c004d50 0x7fdca416adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.530+0000 7fdca37fe700 1 -- 172.21.15.3:0/1916213344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca40b37c0 con 0x7fdca415c5c0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.531+0000 7fdca37fe700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca415c5c0 0x7fdca415ca40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c016000 tx=0x7fdc9c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.531+0000 7fdc8affd700 1 -- 172.21.15.3:0/1916213344 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c014480 con 0x7fdca415c5c0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.531+0000 7fdc8affd700 1 -- 172.21.15.3:0/1916213344 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c01f650 con 0x7fdca415c5c0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.531+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca40b3aa0 con 0x7fdca415c5c0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.531+0000 7fdc8affd700 1 -- 172.21.15.3:0/1916213344 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c01fa10 con 0x7fdca415c5c0 2023-10-06T04:55:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.531+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca41545e0 con 0x7fdca415c5c0 2023-10-06T04:55:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.532+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca4071680 con 0x7fdca415c5c0 2023-10-06T04:55:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.532+0000 7fdc8affd700 1 -- 172.21.15.3:0/1916213344 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc9c003040 con 0x7fdca415c5c0 2023-10-06T04:55:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.533+0000 7fdc8affd700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc90066f30 0x7fdc900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.533+0000 7fdc8affd700 1 -- 172.21.15.3:0/1916213344 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6289+0+0 (secure 0 0 0) 0x7fdc9c0863d0 con 0x7fdca415c5c0 2023-10-06T04:55:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.533+0000 7fdca2ffd700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc90066f30 0x7fdc900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.533+0000 7fdca2ffd700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc90066f30 0x7fdc900693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc940010e0 tx=0x7fdc94001e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.537+0000 7fdc8affd700 1 -- 172.21.15.3:0/1916213344 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc9c0533f0 con 0x7fdca415c5c0 2023-10-06T04:55:12.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:12.730+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdca40b4230 con 0x7fdca415c5c0 2023-10-06T04:55:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.566+0000 7fdc8affd700 1 -- 172.21.15.3:0/1916213344 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdc9c056a10 con 0x7fdca415c5c0 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc90066f30 msgr2=0x7fdc900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc90066f30 0x7fdc900693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc940010e0 tx=0x7fdc94001e90 comp rx=0 tx=0).stop 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca415c5c0 msgr2=0x7fdca415ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca415c5c0 0x7fdca415ca40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c016000 tx=0x7fdc9c01a000 comp rx=0 tx=0).stop 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 shutdown_connections 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc90066f30 0x7fdc900693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc8c004d50 0x7fdca416adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca415c5c0 0x7fdca415ca40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 --2- 172.21.15.3:0/1916213344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdca416a920 0x7fdca4152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 >> 172.21.15.3:0/1916213344 conn(0x7fdca4070af0 msgr2=0x7fdca415abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 shutdown_connections 2023-10-06T04:55:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.569+0000 7fdca981b700 1 -- 172.21.15.3:0/1916213344 wait complete. 2023-10-06T04:55:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:55:13.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:55:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.716+0000 7f1830948700 1 Processor -- start 2023-10-06T04:55:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.716+0000 7f1830948700 1 -- start start 2023-10-06T04:55:13.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f1830948700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f182c0cd570 0x7f182c15d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:13.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f1830948700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f182c15a560 0x7f182c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:13.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f1830948700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c15aa00 con 0x7f182c1536e0 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f1830948700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c153ff0 con 0x7f182c15a560 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f1830948700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0ce6b0 con 0x7f182c0cd570 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f182a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f182c1536e0 0x7f182c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44540/0 (socket says 172.21.15.3:44540) 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f182a59c700 1 -- 172.21.15.3:0/3753104866 learned_addr learned my addr 172.21.15.3:0/3753104866 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.717+0000 7f182ad9d700 1 --2- 172.21.15.3:0/3753104866 >> v2:172.21.15.3:3300/0 conn(0x7f182c15a560 0x7f182c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f1829d9b700 1 --2- 172.21.15.3:0/3753104866 >> v2:172.21.15.153:3300/0 conn(0x7f182c0cd570 0x7f182c15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182ad9d700 1 -- 172.21.15.3:0/3753104866 >> v2:172.21.15.153:3300/0 conn(0x7f182c0cd570 msgr2=0x7f182c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182ad9d700 1 --2- 172.21.15.3:0/3753104866 >> v2:172.21.15.153:3300/0 conn(0x7f182c0cd570 0x7f182c15d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182ad9d700 1 -- 172.21.15.3:0/3753104866 >> v1:172.21.15.3:6789/0 conn(0x7f182c1536e0 legacy=0x7f182c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182959a700 1 -- 172.21.15.3:0/3753104866 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870486440 0 0) 0x7f182c15aa00 con 0x7f182c1536e0 2023-10-06T04:55:13.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182ad9d700 1 -- 172.21.15.3:0/3753104866 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c15d830 con 0x7f182c15a560 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182ad9d700 1 --2- 172.21.15.3:0/3753104866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c153ab0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f182c0896f0 tx=0x7f181c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d8e026f76578c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182959a700 1 -- 172.21.15.3:0/3753104866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00e070 con 0x7f182c15a560 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182959a700 1 -- 172.21.15.3:0/3753104866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00bc30 con 0x7f182c15a560 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.718+0000 7f182959a700 1 -- 172.21.15.3:0/3753104866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c012430 con 0x7f182c15a560 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 -- 172.21.15.3:0/3753104866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 msgr2=0x7f182c153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 --2- 172.21.15.3:0/3753104866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c153ab0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f182c0896f0 tx=0x7f181c009780 comp rx=0 tx=0).stop 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 -- 172.21.15.3:0/3753104866 shutdown_connections 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 --2- 172.21.15.3:0/3753104866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c153ab0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 --2- 172.21.15.3:0/3753104866 >> v2:172.21.15.153:3300/0 conn(0x7f182c0cd570 0x7f182c15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 -- 172.21.15.3:0/3753104866 >> 172.21.15.3:0/3753104866 conn(0x7f182c06f950 msgr2=0x7f182c070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 -- 172.21.15.3:0/3753104866 shutdown_connections 2023-10-06T04:55:13.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.719+0000 7f1830948700 1 -- 172.21.15.3:0/3753104866 wait complete. 2023-10-06T04:55:13.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 Processor -- start 2023-10-06T04:55:13.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 -- start start 2023-10-06T04:55:13.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f182c0cd570 0x7f182c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:13.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f182c1536e0 0x7f182c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:13.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b5440 con 0x7f182c15a560 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b4a70 con 0x7f182c0cd570 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.720+0000 7f1830948700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b3bc0 con 0x7f182c1536e0 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c15f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44072/0 (socket says 172.21.15.3:44072) 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 -- 172.21.15.3:0/2564135129 learned_addr learned my addr 172.21.15.3:0/2564135129 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182a59c700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f182c0cd570 0x7f182c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f1829d9b700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f182c1536e0 0x7f182c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 -- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f182c1536e0 msgr2=0x7f182c151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f182c1536e0 0x7f182c151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 -- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f182c0cd570 msgr2=0x7f182c0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f182c0cd570 0x7f182c0b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 -- 172.21.15.3:0/2564135129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c009a20 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f182ad9d700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c15f1c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f182c0c7210 tx=0x7f181c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f18227fc700 1 -- 172.21.15.3:0/2564135129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c01d630 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f18227fc700 1 -- 172.21.15.3:0/2564135129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c01d790 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f18227fc700 1 -- 172.21.15.3:0/2564135129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c01db50 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c152320 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.721+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c0b4140 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.722+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c0b4be0 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.723+0000 7f18227fc700 1 -- 172.21.15.3:0/2564135129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f181c024070 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.723+0000 7f18227fc700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f180c066f30 0x7f180c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.723+0000 7f18227fc700 1 -- 172.21.15.3:0/2564135129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5822+0+0 (secure 0 0 0) 0x7f181c086f30 con 0x7f182c15a560 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.723+0000 7f182a59c700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f180c066f30 0x7f180c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:13.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.724+0000 7f182a59c700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f180c066f30 0x7f180c0693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f18240011e0 tx=0x7f1824009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:13.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.728+0000 7f18227fc700 1 -- 172.21.15.3:0/2564135129 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f181c054130 con 0x7f182c15a560 2023-10-06T04:55:13.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:13.922+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f182c071680 con 0x7f182c15a560 2023-10-06T04:55:14.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.570+0000 7f18227fc700 1 -- 172.21.15.3:0/2564135129 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) v1 ==== 169+0+0 (secure 0 0 0) 0x7f181c009880 con 0x7f182c15a560 2023-10-06T04:55:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.574+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f180c066f30 msgr2=0x7f180c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.574+0000 7f1830948700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f180c066f30 0x7f180c0693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f18240011e0 tx=0x7f1824009040 comp rx=0 tx=0).stop 2023-10-06T04:55:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 msgr2=0x7f182c15f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c15f1c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f182c0c7210 tx=0x7f181c019000 comp rx=0 tx=0).stop 2023-10-06T04:55:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 shutdown_connections 2023-10-06T04:55:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f180c066f30 0x7f180c0693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f182c0cd570 0x7f182c0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f182c15a560 0x7f182c15f1c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 --2- 172.21.15.3:0/2564135129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f182c1536e0 0x7f182c151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.575+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 >> 172.21.15.3:0/2564135129 conn(0x7f182c06f950 msgr2=0x7f182c0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:14.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.576+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 shutdown_connections 2023-10-06T04:55:14.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.576+0000 7f1830948700 1 -- 172.21.15.3:0/2564135129 wait complete. 2023-10-06T04:55:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T04:55:14.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-10-06T04:55:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.718+0000 7f28bf939700 1 Processor -- start 2023-10-06T04:55:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.718+0000 7f28bf939700 1 -- start start 2023-10-06T04:55:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.719+0000 7f28bf939700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b8150690 0x7f28b8150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.719+0000 7f28bf939700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f28b80cba50 0x7f28b815d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.719+0000 7f28bf939700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b815da80 con 0x7f28b815a720 2023-10-06T04:55:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.719+0000 7f28bf939700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b815a290 con 0x7f28b80cba50 2023-10-06T04:55:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.719+0000 7f28bf939700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80d0020 con 0x7f28b8150690 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bd6d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b8150690 0x7f28b8150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bced4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f28b80cba50 0x7f28b815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bded6700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f28b815a720 0x7f28b80af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:48338/0 (socket says 172.21.15.3:48338) 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bd6d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b8150690 0x7f28b8150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:51636/0 (socket says 172.21.15.3:51636) 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bded6700 1 -- 172.21.15.3:0/217274489 learned_addr learned my addr 172.21.15.3:0/217274489 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bced4700 1 -- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b8150690 msgr2=0x7f28b8150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bced4700 1 --2- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b8150690 0x7f28b8150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bced4700 1 -- 172.21.15.3:0/217274489 >> v1:172.21.15.153:6789/0 conn(0x7f28b815a720 legacy=0x7f28b80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bced4700 1 -- 172.21.15.3:0/217274489 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8159f40 con 0x7f28b80cba50 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28b7fff700 1 -- 172.21.15.3:0/217274489 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754260887 0 0) 0x7f28b815da80 con 0x7f28b815a720 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bd6d5700 1 --2- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b8150690 0x7f28b8150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.720+0000 7f28bced4700 1 --2- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b80cba50 0x7f28b815d540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0098e0 tx=0x7f28ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8967df5b5f54bb70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28b7fff700 1 -- 172.21.15.3:0/217274489 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00b640 con 0x7f28b80cba50 2023-10-06T04:55:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28b7fff700 1 -- 172.21.15.3:0/217274489 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac00b7a0 con 0x7f28b80cba50 2023-10-06T04:55:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28b7fff700 1 -- 172.21.15.3:0/217274489 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00bb40 con 0x7f28b80cba50 2023-10-06T04:55:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 -- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b80cba50 msgr2=0x7f28b815d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 --2- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b80cba50 0x7f28b815d540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0098e0 tx=0x7f28ac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 -- 172.21.15.3:0/217274489 shutdown_connections 2023-10-06T04:55:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 --2- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b8150690 0x7f28b8150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 --2- 172.21.15.3:0/217274489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b80cba50 0x7f28b815d540 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 -- 172.21.15.3:0/217274489 >> 172.21.15.3:0/217274489 conn(0x7f28b8070310 msgr2=0x7f28b8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 -- 172.21.15.3:0/217274489 shutdown_connections 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.721+0000 7f28bf939700 1 -- 172.21.15.3:0/217274489 wait complete. 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.722+0000 7f28bf939700 1 Processor -- start 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.722+0000 7f28bf939700 1 -- start start 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bf939700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b80cba50 0x7f28b81517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bf939700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8150690 0x7f28b815c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bf939700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b815a720 0x7f28b814fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bf939700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8151d00 con 0x7f28b8150690 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bf939700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8152b20 con 0x7f28b815a720 2023-10-06T04:55:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bf939700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b815cb50 con 0x7f28b80cba50 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bced4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8150690 0x7f28b815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bd6d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b80cba50 0x7f28b81517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bced4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8150690 0x7f28b815c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44082/0 (socket says 172.21.15.3:44082) 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bced4700 1 -- 172.21.15.3:0/4159672347 learned_addr learned my addr 172.21.15.3:0/4159672347 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bd6d5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b80cba50 0x7f28b81517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:51644/0 (socket says 172.21.15.3:51644) 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.723+0000 7f28bded6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b815a720 0x7f28b814fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bced4700 1 -- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b80cba50 msgr2=0x7f28b81517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bced4700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b80cba50 0x7f28b81517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bced4700 1 -- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b815a720 msgr2=0x7f28b814fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bced4700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b815a720 0x7f28b814fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bced4700 1 -- 172.21.15.3:0/4159672347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b0009420 con 0x7f28b8150690 2023-10-06T04:55:14.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bded6700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b815a720 0x7f28b814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bd6d5700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b80cba50 0x7f28b81517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bced4700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8150690 0x7f28b815c610 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009f80 tx=0x7f28ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28b57fa700 1 -- 172.21.15.3:0/4159672347 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00bb20 con 0x7f28b8150690 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28b57fa700 1 -- 172.21.15.3:0/4159672347 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac000c90 con 0x7f28b8150690 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac009420 con 0x7f28b8150690 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28b57fa700 1 -- 172.21.15.3:0/4159672347 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac011630 con 0x7f28b8150690 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.724+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b81521f0 con 0x7f28b8150690 2023-10-06T04:55:14.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.725+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b804c380 con 0x7f28b8150690 2023-10-06T04:55:14.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.725+0000 7f28b57fa700 1 -- 172.21.15.3:0/4159672347 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28ac0119a0 con 0x7f28b8150690 2023-10-06T04:55:14.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.726+0000 7f28b57fa700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f28a4066f30 0x7f28a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:14.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.726+0000 7f28b57fa700 1 -- 172.21.15.3:0/4159672347 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f28ac085650 con 0x7f28b8150690 2023-10-06T04:55:14.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.726+0000 7f28bd6d5700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f28a4066f30 0x7f28a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:14.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.727+0000 7f28bd6d5700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f28a4066f30 0x7f28a40693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f28b0011600 tx=0x7f28b00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:14.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.730+0000 7f28b57fa700 1 -- 172.21.15.3:0/4159672347 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28ac052a60 con 0x7f28b8150690 2023-10-06T04:55:14.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.925+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7f28b80ce4b0 con 0x7f28b8150690 2023-10-06T04:55:14.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.932+0000 7f28b57fa700 1 -- 172.21.15.3:0/4159672347 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v20) v1 ==== 106+0+0 (secure 0 0 0) 0x7f28ac056080 con 0x7f28b8150690 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f28a4066f30 msgr2=0x7f28a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f28a4066f30 0x7f28a40693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f28b0011600 tx=0x7f28b00091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8150690 msgr2=0x7f28b815c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8150690 0x7f28b815c610 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009f80 tx=0x7f28ac017000 comp rx=0 tx=0).stop 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 shutdown_connections 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f28a4066f30 0x7f28a40693e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28b815a720 0x7f28b814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28b80cba50 0x7f28b81517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 --2- 172.21.15.3:0/4159672347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8150690 0x7f28b815c610 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:14.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 >> 172.21.15.3:0/4159672347 conn(0x7f28b8070310 msgr2=0x7f28b80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:14.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 shutdown_connections 2023-10-06T04:55:14.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:14.934+0000 7f28bf939700 1 -- 172.21.15.3:0/4159672347 wait complete. 2023-10-06T04:55:14.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:55:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.067+0000 7f153b1f1700 1 Processor -- start 2023-10-06T04:55:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.067+0000 7f153b1f1700 1 -- start start 2023-10-06T04:55:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f153b1f1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15340cc140 0x7f1534159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f153b1f1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f153415a330 0x7f15340ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f153b1f1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15340ce420 con 0x7f15340cc510 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f153b1f1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153415a880 con 0x7f15340cc140 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f153b1f1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153415ab60 con 0x7f153415a330 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f1538f8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15340cc140 0x7f1534159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f1533fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f15340cc510 0x7f15340aef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44556/0 (socket says 172.21.15.3:44556) 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.068+0000 7f1533fff700 1 -- 172.21.15.3:0/947053703 learned_addr learned my addr 172.21.15.3:0/947053703 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f153978e700 1 --2- 172.21.15.3:0/947053703 >> v2:172.21.15.153:3300/0 conn(0x7f153415a330 0x7f15340ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f1538f8d700 1 -- 172.21.15.3:0/947053703 >> v2:172.21.15.153:3300/0 conn(0x7f153415a330 msgr2=0x7f15340ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f1538f8d700 1 --2- 172.21.15.3:0/947053703 >> v2:172.21.15.153:3300/0 conn(0x7f153415a330 0x7f15340ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f1538f8d700 1 -- 172.21.15.3:0/947053703 >> v1:172.21.15.3:6789/0 conn(0x7f15340cc510 legacy=0x7f15340aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f1538f8d700 1 -- 172.21.15.3:0/947053703 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15340ce0b0 con 0x7f15340cc140 2023-10-06T04:55:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f15337fe700 1 -- 172.21.15.3:0/947053703 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054805497 0 0) 0x7f15340ce420 con 0x7f15340cc510 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f1538f8d700 1 --2- 172.21.15.3:0/947053703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc140 0x7f1534159df0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1534063ea0 tx=0x7f152c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3364eb7d8070a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f15337fe700 1 -- 172.21.15.3:0/947053703 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c00e070 con 0x7f15340cc140 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f15337fe700 1 -- 172.21.15.3:0/947053703 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c00bc30 con 0x7f15340cc140 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.069+0000 7f15337fe700 1 -- 172.21.15.3:0/947053703 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c012430 con 0x7f15340cc140 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 -- 172.21.15.3:0/947053703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc140 msgr2=0x7f1534159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 --2- 172.21.15.3:0/947053703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc140 0x7f1534159df0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1534063ea0 tx=0x7f152c009780 comp rx=0 tx=0).stop 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 -- 172.21.15.3:0/947053703 shutdown_connections 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 --2- 172.21.15.3:0/947053703 >> v2:172.21.15.153:3300/0 conn(0x7f153415a330 0x7f15340ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 --2- 172.21.15.3:0/947053703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc140 0x7f1534159df0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 -- 172.21.15.3:0/947053703 >> 172.21.15.3:0/947053703 conn(0x7f15340d03e0 msgr2=0x7f15340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 -- 172.21.15.3:0/947053703 shutdown_connections 2023-10-06T04:55:15.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.070+0000 7f153b1f1700 1 -- 172.21.15.3:0/947053703 wait complete. 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.071+0000 7f153b1f1700 1 Processor -- start 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.071+0000 7f153b1f1700 1 -- start start 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15340cc140 0x7f15340cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc510 0x7f1534145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f153415a330 0x7f15340b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15341531f0 con 0x7f15340cc510 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340c44c0 con 0x7f15340cc140 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15340c3ee0 con 0x7f153415a330 2023-10-06T04:55:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc510 0x7f1534145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc510 0x7f1534145820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44088/0 (socket says 172.21.15.3:44088) 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 -- 172.21.15.3:0/501157973 learned_addr learned my addr 172.21.15.3:0/501157973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153978e700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f153415a330 0x7f15340b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1538f8d700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15340cc140 0x7f15340cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 -- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f153415a330 msgr2=0x7f15340b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f153415a330 0x7f15340b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 -- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15340cc140 msgr2=0x7f15340cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15340cc140 0x7f15340cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 -- 172.21.15.3:0/501157973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c009a20 con 0x7f15340cc510 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1533fff700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc510 0x7f1534145820 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f152800c810 tx=0x7f1528007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:15.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1530ff9700 1 -- 172.21.15.3:0/501157973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528005b30 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1530ff9700 1 -- 172.21.15.3:0/501157973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1528005c90 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f1530ff9700 1 -- 172.21.15.3:0/501157973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528007300 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1534153cd0 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.072+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1534153fb0 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.073+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153404c380 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.074+0000 7f1530ff9700 1 -- 172.21.15.3:0/501157973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f152801b3b0 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.074+0000 7f1530ff9700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.074+0000 7f1530ff9700 1 -- 172.21.15.3:0/501157973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f15280837d0 con 0x7f15340cc510 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.074+0000 7f1538f8d700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.075+0000 7f1538f8d700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1534063ea0 tx=0x7f152c019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:15.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.078+0000 7f1530ff9700 1 -- 172.21.15.3:0/501157973 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1528050be0 con 0x7f15340cc510 2023-10-06T04:55:15.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.269+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f153409db00 con 0x7f15340cc510 2023-10-06T04:55:15.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.273+0000 7f1530ff9700 1 -- 172.21.15.3:0/501157973 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f15280170a0 con 0x7f15340cc510 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 msgr2=0x7f15200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1534063ea0 tx=0x7f152c019000 comp rx=0 tx=0).stop 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc510 msgr2=0x7f1534145820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc510 0x7f1534145820 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f152800c810 tx=0x7f1528007df0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 shutdown_connections 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15340cc140 0x7f15340cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f153415a330 0x7f15340b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 --2- 172.21.15.3:0/501157973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15340cc510 0x7f1534145820 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 >> 172.21.15.3:0/501157973 conn(0x7f15340d03e0 msgr2=0x7f15340ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:15.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 shutdown_connections 2023-10-06T04:55:15.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.275+0000 7f153b1f1700 1 -- 172.21.15.3:0/501157973 wait complete. 2023-10-06T04:55:15.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2023-10-06T04:55:15.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.409+0000 7ffb2bbac700 1 Processor -- start 2023-10-06T04:55:15.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.410+0000 7ffb2bbac700 1 -- start start 2023-10-06T04:55:15.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.410+0000 7ffb2bbac700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb2415cb50 0x7ffb240b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.410+0000 7ffb2bbac700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb240b0ce0 con 0x7ffb240cd950 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.410+0000 7ffb2bbac700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb240cb820 con 0x7ffb240b0590 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.410+0000 7ffb2bbac700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb240b0a00 con 0x7ffb2415cb50 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb2a149700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffb240cd950 0x7ffb240cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44564/0 (socket says 172.21.15.3:44564) 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb2a149700 1 -- 172.21.15.3:0/708588581 learned_addr learned my addr 172.21.15.3:0/708588581 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb29948700 1 --2- 172.21.15.3:0/708588581 >> v2:172.21.15.153:3300/0 conn(0x7ffb2415cb50 0x7ffb240b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565788158 0 0) 0x7ffb240b0ce0 con 0x7ffb240cd950 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0c003570 con 0x7ffb240cd950 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 592002966 0 0) 0x7ffb0c003570 con 0x7ffb240cd950 2023-10-06T04:55:15.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 >> v2:172.21.15.153:3300/0 conn(0x7ffb2415cb50 msgr2=0x7ffb240b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 --2- 172.21.15.3:0/708588581 >> v2:172.21.15.153:3300/0 conn(0x7ffb2415cb50 0x7ffb240b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 >> v1:172.21.15.153:6789/0 conn(0x7ffb240b0590 legacy=0x7ffb240b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240cb1d0 con 0x7ffb240cd950 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7ffb140017f0 con 0x7ffb240cd950 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 >> v1:172.21.15.3:6789/0 conn(0x7ffb240cd950 legacy=0x7ffb240cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c003a60 0x7ffb0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb0c004810 0x7ffb0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb0c0051c0 0x7ffb0c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240b0a00 con 0x7ffb0c0051c0 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c00c160 con 0x7ffb0c003a60 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c00c320 con 0x7ffb0c004810 2023-10-06T04:55:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb2a149700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb0c0051c0 0x7ffb0c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb29948700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c003a60 0x7ffb0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.411+0000 7ffb29147700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb0c004810 0x7ffb0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2a149700 1 -- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb0c004810 msgr2=0x7ffb0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2a149700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb0c004810 0x7ffb0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2a149700 1 -- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c003a60 msgr2=0x7ffb0c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2a149700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c003a60 0x7ffb0c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2a149700 1 -- 172.21.15.3:0/708588581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c003570 con 0x7ffb0c0051c0 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2a149700 1 -- 172.21.15.3:0/708588581 reap_dead start 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2a149700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb0c0051c0 0x7ffb0c00ba60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400a900 tx=0x7ffb140021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd8f734673a229a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb140070d0 con 0x7ffb0c0051c0 2023-10-06T04:55:15.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14007230 con 0x7ffb0c0051c0 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb28946700 1 -- 172.21.15.3:0/708588581 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb140075d0 con 0x7ffb0c0051c0 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2bbac700 1 -- 172.21.15.3:0/708588581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb0c0051c0 msgr2=0x7ffb0c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb0c0051c0 0x7ffb0c00ba60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400a900 tx=0x7ffb140021d0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2bbac700 1 -- 172.21.15.3:0/708588581 shutdown_connections 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/708588581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb0c0051c0 0x7ffb0c00ba60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.412+0000 7ffb2bbac700 1 -- 172.21.15.3:0/708588581 >> 172.21.15.3:0/708588581 conn(0x7ffb24070310 msgr2=0x7ffb24070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.413+0000 7ffb2bbac700 1 -- 172.21.15.3:0/708588581 shutdown_connections 2023-10-06T04:55:15.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.413+0000 7ffb2bbac700 1 -- 172.21.15.3:0/708588581 wait complete. 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 Processor -- start 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 -- start start 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c0051c0 0x7ffb24150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb240cdf80 0x7ffb240c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb240b2200 0x7ffb240b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240b2bc0 con 0x7ffb240b2200 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240ce410 con 0x7ffb0c0051c0 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.414+0000 7ffb2bbac700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240ce6f0 con 0x7ffb240cdf80 2023-10-06T04:55:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb240b2200 0x7ffb240b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb240b2200 0x7ffb240b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44102/0 (socket says 172.21.15.3:44102) 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 -- 172.21.15.3:0/1109378180 learned_addr learned my addr 172.21.15.3:0/1109378180 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb29948700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c0051c0 0x7ffb24150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb29147700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb240cdf80 0x7ffb240c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 -- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb240cdf80 msgr2=0x7ffb240c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb240cdf80 0x7ffb240c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 -- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c0051c0 msgr2=0x7ffb24150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c0051c0 0x7ffb24150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 -- 172.21.15.3:0/1109378180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240ce9d0 con 0x7ffb240b2200 2023-10-06T04:55:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2a149700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb240b2200 0x7ffb240b2680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb240cac40 tx=0x7ffb14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb21ffb700 1 -- 172.21.15.3:0/1109378180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb140073a0 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb21ffb700 1 -- 172.21.15.3:0/1109378180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14007520 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb240cecb0 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb21ffb700 1 -- 172.21.15.3:0/1109378180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1401d670 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.415+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb24150ea0 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.416+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb24169850 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.417+0000 7ffb21ffb700 1 -- 172.21.15.3:0/1109378180 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb1401d9e0 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.417+0000 7ffb21ffb700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb10066f30 0x7ffb100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.417+0000 7ffb21ffb700 1 -- 172.21.15.3:0/1109378180 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ffb14086370 con 0x7ffb240b2200 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.417+0000 7ffb29948700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb10066f30 0x7ffb100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.418+0000 7ffb29948700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb10066f30 0x7ffb100693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0090f0 tx=0x7ffb1c009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.422+0000 7ffb21ffb700 1 -- 172.21.15.3:0/1109378180 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb24169850 con 0x7ffb240b2200 2023-10-06T04:55:15.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:15.703+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) v1 -- 0x7ffb24169850 con 0x7ffb240b2200 2023-10-06T04:55:16.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.578+0000 7ffb21ffb700 1 -- 172.21.15.3:0/1109378180 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v170) v1 ==== 148+0+0 (secure 0 0 0) 0x7ffb24169850 con 0x7ffb240b2200 2023-10-06T04:55:16.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb10066f30 msgr2=0x7ffb100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:16.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb10066f30 0x7ffb100693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0090f0 tx=0x7ffb1c009120 comp rx=0 tx=0).stop 2023-10-06T04:55:16.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb240b2200 msgr2=0x7ffb240b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:16.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb240b2200 0x7ffb240b2680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb240cac40 tx=0x7ffb14016000 comp rx=0 tx=0).stop 2023-10-06T04:55:16.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 shutdown_connections 2023-10-06T04:55:16.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb10066f30 0x7ffb100693e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c0051c0 0x7ffb24150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb240b2200 0x7ffb240b2680 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 --2- 172.21.15.3:0/1109378180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb240cdf80 0x7ffb240c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 >> 172.21.15.3:0/1109378180 conn(0x7ffb24070310 msgr2=0x7ffb240708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 shutdown_connections 2023-10-06T04:55:16.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.580+0000 7ffb2bbac700 1 -- 172.21.15.3:0/1109378180 wait complete. 2023-10-06T04:55:16.598 INFO:tasks.cephfs_test_runner:That the MDS will not let a client sit above mds_max_caps_per_client caps. ... ok 2023-10-06T04:55:16.598 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) When a client has inodes pinned in its cache (open files), that the MDS 2023-10-06T04:55:16.598 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T04:55:16.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2023-10-06T04:55:16.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.723+0000 7f9c7ab6c700 1 Processor -- start 2023-10-06T04:55:16.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.723+0000 7f9c7ab6c700 1 -- start start 2023-10-06T04:55:16.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.723+0000 7f9c7ab6c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c74156110 0x7f9c7406aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:16.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.723+0000 7f9c7ab6c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c741564e0 0x7f9c7406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:16.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c7ab6c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c740acbb0 con 0x7f9c740c68d0 2023-10-06T04:55:16.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c7ab6c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c740c7430 con 0x7f9c741564e0 2023-10-06T04:55:16.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c7ab6c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c740c7640 con 0x7f9c74156110 2023-10-06T04:55:16.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c73fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c741564e0 0x7f9c7406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:16.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c79109700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c740c68d0 0x7f9c740c6d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44576/0 (socket says 172.21.15.3:44576) 2023-10-06T04:55:16.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c79109700 1 -- 172.21.15.3:0/3185270575 learned_addr learned my addr 172.21.15.3:0/3185270575 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:16.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c73fff700 1 -- 172.21.15.3:0/3185270575 >> v2:172.21.15.153:3300/0 conn(0x7f9c74156110 msgr2=0x7f9c7406aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:16.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c78908700 1 --2- 172.21.15.3:0/3185270575 >> v2:172.21.15.153:3300/0 conn(0x7f9c74156110 0x7f9c7406aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c73fff700 1 --2- 172.21.15.3:0/3185270575 >> v2:172.21.15.153:3300/0 conn(0x7f9c74156110 0x7f9c7406aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c737fe700 1 -- 172.21.15.3:0/3185270575 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281102039 0 0) 0x7f9c740acbb0 con 0x7f9c740c68d0 2023-10-06T04:55:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c73fff700 1 -- 172.21.15.3:0/3185270575 >> v1:172.21.15.3:6789/0 conn(0x7f9c740c68d0 legacy=0x7f9c740c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c73fff700 1 -- 172.21.15.3:0/3185270575 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740c7920 con 0x7f9c741564e0 2023-10-06T04:55:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c73fff700 1 --2- 172.21.15.3:0/3185270575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c741564e0 0x7f9c7406afd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9c7406f6f0 tx=0x7f9c5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc16ffcea38facf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c737fe700 1 -- 172.21.15.3:0/3185270575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c00b6b0 con 0x7f9c741564e0 2023-10-06T04:55:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c737fe700 1 -- 172.21.15.3:0/3185270575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c00b810 con 0x7f9c741564e0 2023-10-06T04:55:16.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.724+0000 7f9c737fe700 1 -- 172.21.15.3:0/3185270575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c00bbb0 con 0x7f9c741564e0 2023-10-06T04:55:16.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/3185270575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c741564e0 msgr2=0x7f9c7406afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:16.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/3185270575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c741564e0 0x7f9c7406afd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9c7406f6f0 tx=0x7f9c5c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/3185270575 shutdown_connections 2023-10-06T04:55:16.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/3185270575 >> v2:172.21.15.153:3300/0 conn(0x7f9c74156110 0x7f9c7406aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/3185270575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c741564e0 0x7f9c7406afd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/3185270575 >> 172.21.15.3:0/3185270575 conn(0x7f9c7406c370 msgr2=0x7f9c7406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/3185270575 shutdown_connections 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.725+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/3185270575 wait complete. 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.726+0000 7f9c7ab6c700 1 Processor -- start 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.726+0000 7f9c7ab6c700 1 -- start start 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c7ab6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c740c68d0 0x7f9c740aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c7ab6c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c74156110 0x7f9c740c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c7ab6c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c741564e0 0x7f9c7414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c7ab6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c47a0 con 0x7f9c740c68d0 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c7ab6c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c3b60 con 0x7f9c741564e0 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c7ab6c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c4cd0 con 0x7f9c74156110 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c740c68d0 0x7f9c740aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c740c68d0 0x7f9c740aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44110/0 (socket says 172.21.15.3:44110) 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 -- 172.21.15.3:0/169248476 learned_addr learned my addr 172.21.15.3:0/169248476 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c73fff700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c74156110 0x7f9c740c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c79109700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c741564e0 0x7f9c7414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 -- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c74156110 msgr2=0x7f9c740c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:16.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c74156110 0x7f9c740c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 -- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c741564e0 msgr2=0x7f9c7414db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:16.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c741564e0 0x7f9c7414db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:16.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c78908700 1 -- 172.21.15.3:0/169248476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c009420 con 0x7f9c740c68d0 2023-10-06T04:55:16.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c73fff700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c74156110 0x7f9c740c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:16.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.727+0000 7f9c79109700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c741564e0 0x7f9c7414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:16.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.728+0000 7f9c78908700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c740c68d0 0x7f9c740aee10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c007fc0 tx=0x7f9c6c0078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.728+0000 7f9c70ff9700 1 -- 172.21.15.3:0/169248476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c016430 con 0x7f9c740c68d0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.728+0000 7f9c70ff9700 1 -- 172.21.15.3:0/169248476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c016590 con 0x7f9c740c68d0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.728+0000 7f9c70ff9700 1 -- 172.21.15.3:0/169248476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c016930 con 0x7f9c740c68d0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.728+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c740c4fb0 con 0x7f9c740c68d0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.728+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c740c5570 con 0x7f9c740c68d0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.729+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c74071680 con 0x7f9c740c68d0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.729+0000 7f9c70ff9700 1 -- 172.21.15.3:0/169248476 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c6c003020 con 0x7f9c740c68d0 2023-10-06T04:55:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.730+0000 7f9c70ff9700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c54066f60 0x7f9c54069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.730+0000 7f9c70ff9700 1 -- 172.21.15.3:0/169248476 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9c6c012070 con 0x7f9c740c68d0 2023-10-06T04:55:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.730+0000 7f9c73fff700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c54066f60 0x7f9c54069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.730+0000 7f9c73fff700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c54066f60 0x7f9c54069410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0093f0 tx=0x7f9c5c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:16.734+0000 7f9c70ff9700 1 -- 172.21.15.3:0/169248476 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c6c052650 con 0x7f9c740c68d0 2023-10-06T04:55:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.011+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) v1 -- 0x7f9c740c7c00 con 0x7f9c740c68d0 2023-10-06T04:55:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.582+0000 7f9c70ff9700 1 -- 172.21.15.3:0/169248476 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v171) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9c6c055c70 con 0x7f9c740c68d0 2023-10-06T04:55:17.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.584+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c54066f60 msgr2=0x7f9c54069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.584+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c54066f60 0x7f9c54069410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0093f0 tx=0x7f9c5c017000 comp rx=0 tx=0).stop 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.584+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c740c68d0 msgr2=0x7f9c740aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.584+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c740c68d0 0x7f9c740aee10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c007fc0 tx=0x7f9c6c0078b0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.584+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 shutdown_connections 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.584+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c54066f60 0x7f9c54069410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.584+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c741564e0 0x7f9c7414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.585+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c740c68d0 0x7f9c740aee10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.585+0000 7f9c7ab6c700 1 --2- 172.21.15.3:0/169248476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c74156110 0x7f9c740c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.585+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 >> 172.21.15.3:0/169248476 conn(0x7f9c7406c370 msgr2=0x7f9c7406e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.585+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 shutdown_connections 2023-10-06T04:55:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.585+0000 7f9c7ab6c700 1 -- 172.21.15.3:0/169248476 wait complete. 2023-10-06T04:55:17.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:55:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.725+0000 7f6227816700 1 Processor -- start 2023-10-06T04:55:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.725+0000 7f6227816700 1 -- start start 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.726+0000 7f6227816700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62200cba50 0x7f6220150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.726+0000 7f6227816700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f622015d540 0x7f62200af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.726+0000 7f6227816700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622015a290 con 0x7f6220150690 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.726+0000 7f6227816700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62200d0020 con 0x7f622015d540 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.726+0000 7f6227816700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6220159f40 con 0x7f62200cba50 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.726+0000 7f62255b2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6220150690 0x7f622015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44580/0 (socket says 172.21.15.3:44580) 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.726+0000 7f62255b2700 1 -- 172.21.15.3:0/420188605 learned_addr learned my addr 172.21.15.3:0/420188605 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f6225db3700 1 --2- 172.21.15.3:0/420188605 >> v2:172.21.15.153:3300/0 conn(0x7f622015d540 0x7f62200af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f6224db1700 1 --2- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62200cba50 0x7f6220150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f621ffff700 1 -- 172.21.15.3:0/420188605 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170124721 0 0) 0x7f622015a290 con 0x7f6220150690 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f621ffff700 1 -- 172.21.15.3:0/420188605 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6208003570 con 0x7f6220150690 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f6225db3700 1 -- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62200cba50 msgr2=0x7f6220150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f6225db3700 1 --2- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62200cba50 0x7f6220150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f6225db3700 1 -- 172.21.15.3:0/420188605 >> v1:172.21.15.3:6789/0 conn(0x7f6220150690 legacy=0x7f622015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f621ffff700 1 -- 172.21.15.3:0/420188605 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803979222 0 0) 0x7f6208003570 con 0x7f6220150690 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f6225db3700 1 -- 172.21.15.3:0/420188605 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622015dcb0 con 0x7f622015d540 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f6225db3700 1 --2- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 0x7f62200af190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62100098e0 tx=0x7f62100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7fffbde8de46355 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f621ffff700 1 -- 172.21.15.3:0/420188605 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62180017f0 con 0x7f622015d540 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f621ffff700 1 -- 172.21.15.3:0/420188605 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6210011420 con 0x7f622015d540 2023-10-06T04:55:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.727+0000 7f621ffff700 1 -- 172.21.15.3:0/420188605 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100117c0 con 0x7f622015d540 2023-10-06T04:55:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 -- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 msgr2=0x7f62200af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 --2- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 0x7f62200af190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62100098e0 tx=0x7f62100091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 -- 172.21.15.3:0/420188605 shutdown_connections 2023-10-06T04:55:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 --2- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 0x7f62200af190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 --2- 172.21.15.3:0/420188605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62200cba50 0x7f6220150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 -- 172.21.15.3:0/420188605 >> 172.21.15.3:0/420188605 conn(0x7f6220070310 msgr2=0x7f6220070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 -- 172.21.15.3:0/420188605 shutdown_connections 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.728+0000 7f6227816700 1 -- 172.21.15.3:0/420188605 wait complete. 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.729+0000 7f6227816700 1 Processor -- start 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.729+0000 7f6227816700 1 -- start start 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6227816700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62200cba50 0x7f62201532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6227816700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6220150690 0x7f62200b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6227816700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 0x7f62200b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6227816700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6220151010 con 0x7f62200cba50 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6227816700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62201512b0 con 0x7f622015d540 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6227816700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62200b59e0 con 0x7f6220150690 2023-10-06T04:55:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f62255b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62200cba50 0x7f62201532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f62255b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62200cba50 0x7f62201532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44120/0 (socket says 172.21.15.3:44120) 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f62255b2700 1 -- 172.21.15.3:0/3190473376 learned_addr learned my addr 172.21.15.3:0/3190473376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6225db3700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 0x7f62200b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.730+0000 7f6224db1700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6220150690 0x7f62200b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f62255b2700 1 -- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6220150690 msgr2=0x7f62200b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f62255b2700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6220150690 0x7f62200b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f62255b2700 1 -- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 msgr2=0x7f62200b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f62255b2700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 0x7f62200b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f62255b2700 1 -- 172.21.15.3:0/3190473376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210009420 con 0x7f62200cba50 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f62255b2700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62200cba50 0x7f62201532c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f621800ccd0 tx=0x7f62180016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f621d7fa700 1 -- 172.21.15.3:0/3190473376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218004dc0 con 0x7f62200cba50 2023-10-06T04:55:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f621d7fa700 1 -- 172.21.15.3:0/3190473376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621800a6c0 con 0x7f62200cba50 2023-10-06T04:55:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62200b5cc0 con 0x7f62200cba50 2023-10-06T04:55:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f621d7fa700 1 -- 172.21.15.3:0/3190473376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800aa80 con 0x7f62200cba50 2023-10-06T04:55:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.731+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6220153800 con 0x7f62200cba50 2023-10-06T04:55:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.732+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62200afea0 con 0x7f62200cba50 2023-10-06T04:55:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.733+0000 7f621d7fa700 1 -- 172.21.15.3:0/3190473376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6218007150 con 0x7f62200cba50 2023-10-06T04:55:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.733+0000 7f621d7fa700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f620c066f30 0x7f620c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.733+0000 7f621d7fa700 1 -- 172.21.15.3:0/3190473376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f6218083e70 con 0x7f62200cba50 2023-10-06T04:55:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.733+0000 7f6224db1700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f620c066f30 0x7f620c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.734+0000 7f6224db1700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f620c066f30 0x7f620c0693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62200cb000 tx=0x7f62140091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.737+0000 7f621d7fa700 1 -- 172.21.15.3:0/3190473376 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6218051300 con 0x7f62200cba50 2023-10-06T04:55:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.929+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f622004c380 con 0x7f62200cba50 2023-10-06T04:55:17.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.932+0000 7f621d7fa700 1 -- 172.21.15.3:0/3190473376 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6218018020 con 0x7f62200cba50 2023-10-06T04:55:17.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f620c066f30 msgr2=0x7f620c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f620c066f30 0x7f620c0693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62200cb000 tx=0x7f62140091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62200cba50 msgr2=0x7f62201532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62200cba50 0x7f62201532c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f621800ccd0 tx=0x7f62180016f0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 shutdown_connections 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f620c066f30 0x7f620c0693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f622015d540 0x7f62200b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62200cba50 0x7f62201532c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 --2- 172.21.15.3:0/3190473376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6220150690 0x7f62200b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 >> 172.21.15.3:0/3190473376 conn(0x7f6220070310 msgr2=0x7f62200c6370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 shutdown_connections 2023-10-06T04:55:17.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:17.935+0000 7f6227816700 1 -- 172.21.15.3:0/3190473376 wait complete. 2023-10-06T04:55:17.949 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:55:17.950 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:55:17.990 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:55:18.033 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:55:18.033 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:55:18.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:18.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.149+0000 7fec238d0700 1 Processor -- start 2023-10-06T04:55:18.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.149+0000 7fec238d0700 1 -- start start 2023-10-06T04:55:18.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec238d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec1c15c8f0 0x7fec1c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:18.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec238d0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15ccc0 0x7fec1c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:18.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec238d0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1c0cf6c0 con 0x7fec1c0d06a0 2023-10-06T04:55:18.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec238d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c0cfb50 con 0x7fec1c15c8f0 2023-10-06T04:55:18.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec238d0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c15e050 con 0x7fec1c15ccc0 2023-10-06T04:55:18.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec2166c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec1c15c8f0 0x7fec1c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec2166c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec1c15c8f0 0x7fec1c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44126/0 (socket says 172.21.15.3:44126) 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec2166c700 1 -- 172.21.15.3:0/4171440157 learned_addr learned my addr 172.21.15.3:0/4171440157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.150+0000 7fec20e6b700 1 --2- 172.21.15.3:0/4171440157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15ccc0 0x7fec1c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec2166c700 1 -- 172.21.15.3:0/4171440157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15ccc0 msgr2=0x7fec1c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec2166c700 1 --2- 172.21.15.3:0/4171440157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15ccc0 0x7fec1c0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec2166c700 1 -- 172.21.15.3:0/4171440157 >> v1:172.21.15.153:6789/0 conn(0x7fec1c0d06a0 legacy=0x7fec1c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec2166c700 1 -- 172.21.15.3:0/4171440157 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c0d0af0 con 0x7fec1c15c8f0 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec2166c700 1 --2- 172.21.15.3:0/4171440157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c15c8f0 0x7fec1c15db10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec140099b0 tx=0x7fec14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02dcf4b9e54b2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec1bfff700 1 -- 172.21.15.3:0/4171440157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400b860 con 0x7fec1c15c8f0 2023-10-06T04:55:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec1bfff700 1 -- 172.21.15.3:0/4171440157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1400b9c0 con 0x7fec1c15c8f0 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec1bfff700 1 -- 172.21.15.3:0/4171440157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400bd60 con 0x7fec1c15c8f0 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec238d0700 1 -- 172.21.15.3:0/4171440157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c15c8f0 msgr2=0x7fec1c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec238d0700 1 --2- 172.21.15.3:0/4171440157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c15c8f0 0x7fec1c15db10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec140099b0 tx=0x7fec14009250 comp rx=0 tx=0).stop 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec238d0700 1 -- 172.21.15.3:0/4171440157 shutdown_connections 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec238d0700 1 --2- 172.21.15.3:0/4171440157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c15c8f0 0x7fec1c15db10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec238d0700 1 --2- 172.21.15.3:0/4171440157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15ccc0 0x7fec1c0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.151+0000 7fec238d0700 1 -- 172.21.15.3:0/4171440157 >> 172.21.15.3:0/4171440157 conn(0x7fec1c070310 msgr2=0x7fec1c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.152+0000 7fec238d0700 1 -- 172.21.15.3:0/4171440157 shutdown_connections 2023-10-06T04:55:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.152+0000 7fec238d0700 1 -- 172.21.15.3:0/4171440157 wait complete. 2023-10-06T04:55:18.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 Processor -- start 2023-10-06T04:55:18.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 -- start start 2023-10-06T04:55:18.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c0d06a0 0x7fec1c153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:18.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15c8f0 0x7fec1c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:18.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1c15ccc0 0x7fec1c16a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c152740 con 0x7fec1c0d06a0 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c150590 con 0x7fec1c15ccc0 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.153+0000 7fec238d0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0b4230 con 0x7fec1c15c8f0 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c0d06a0 0x7fec1c153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c0d06a0 0x7fec1c153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44136/0 (socket says 172.21.15.3:44136) 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 -- 172.21.15.3:0/1150249042 learned_addr learned my addr 172.21.15.3:0/1150249042 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec20e6b700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15c8f0 0x7fec1c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec21e6d700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1c15ccc0 0x7fec1c16a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 -- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15c8f0 msgr2=0x7fec1c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15c8f0 0x7fec1c0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 -- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1c15ccc0 msgr2=0x7fec1c16a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1c15ccc0 0x7fec1c16a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 -- 172.21.15.3:0/1150249042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec140094f0 con 0x7fec1c0d06a0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec20e6b700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15c8f0 0x7fec1c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec21e6d700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1c15ccc0 0x7fec1c16a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec2166c700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c0d06a0 0x7fec1c153260 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec1c06eaa0 tx=0x7fec14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec197fa700 1 -- 172.21.15.3:0/1150249042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14024440 con 0x7fec1c0d06a0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec197fa700 1 -- 172.21.15.3:0/1150249042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec140245c0 con 0x7fec1c0d06a0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c0b4510 con 0x7fec1c0d06a0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec197fa700 1 -- 172.21.15.3:0/1150249042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14024980 con 0x7fec1c0d06a0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.154+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c0b4ad0 con 0x7fec1c0d06a0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.155+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c071680 con 0x7fec1c0d06a0 2023-10-06T04:55:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.156+0000 7fec197fa700 1 -- 172.21.15.3:0/1150249042 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec1401a420 con 0x7fec1c0d06a0 2023-10-06T04:55:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.156+0000 7fec197fa700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec08066f30 0x7fec080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.156+0000 7fec197fa700 1 -- 172.21.15.3:0/1150249042 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fec14086250 con 0x7fec1c0d06a0 2023-10-06T04:55:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.156+0000 7fec20e6b700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec08066f30 0x7fec080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.157+0000 7fec20e6b700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec08066f30 0x7fec080693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fec1c15d0b0 tx=0x7fec10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.160+0000 7fec197fa700 1 -- 172.21.15.3:0/1150249042 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec14053660 con 0x7fec1c0d06a0 2023-10-06T04:55:18.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.400+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec1c0b35a0 con 0x7fec1c0d06a0 2023-10-06T04:55:18.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.400+0000 7fec197fa700 1 -- 172.21.15.3:0/1150249042 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fec14015de0 con 0x7fec1c0d06a0 2023-10-06T04:55:18.403 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:18.403 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1632094644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1632094644},{"type":"v1","addr":"172.21.15.153:6836","nonce":1632094644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/315751707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":315751707},{"type":"v1","addr":"172.21.15.153:6837","nonce":315751707}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/823537926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":823537926},{"type":"v1","addr":"172.21.15.3:6834","nonce":823537926}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1522383076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1522383076},{"type":"v1","addr":"172.21.15.3:6835","nonce":1522383076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[]} 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec08066f30 msgr2=0x7fec080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec08066f30 0x7fec080693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fec1c15d0b0 tx=0x7fec10008040 comp rx=0 tx=0).stop 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c0d06a0 msgr2=0x7fec1c153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c0d06a0 0x7fec1c153260 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec1c06eaa0 tx=0x7fec14017000 comp rx=0 tx=0).stop 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 shutdown_connections 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fec08066f30 0x7fec080693e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1c15ccc0 0x7fec1c16a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1c0d06a0 0x7fec1c153260 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 --2- 172.21.15.3:0/1150249042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec1c15c8f0 0x7fec1c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 >> 172.21.15.3:0/1150249042 conn(0x7fec1c070310 msgr2=0x7fec1c158400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 shutdown_connections 2023-10-06T04:55:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:18.402+0000 7fec238d0700 1 -- 172.21.15.3:0/1150249042 wait complete. 2023-10-06T04:55:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 33 2023-10-06T04:55:18.417 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:55:18.417 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:55:18.417 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:55:18.418 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:55:18.418 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:55:18.418 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:55:18.418 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:55:18.418 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:55:18.418 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:55:18.418 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:55:18.419 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:55:18.419 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:55:18.419 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:55:18.419 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:55:18.419 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:55:18.419 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:55:18.427 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:55:18.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:55:18.430 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:55:18.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:55:18.431 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:55:18.431 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:55:18.469 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:55:18.470 DEBUG:teuthology.parallel:result is None 2023-10-06T04:55:18.472 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:55:18.472 DEBUG:teuthology.parallel:result is None 2023-10-06T04:55:18.473 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:55:18.473 DEBUG:teuthology.parallel:result is None 2023-10-06T04:55:18.543 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:55:18.555 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:55:18.555 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:55:18.556 INFO:tasks.ceph.mds.b.smithi153.stderr:2023-10-06T04:55:18.554+0000 7f5bfc12f700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T04:55:19.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T04:55:24.427 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:55:24.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:55:24.430 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:55:24.431 DEBUG:teuthology.parallel:result is None 2023-10-06T04:55:24.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:55:24.535 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:55:24.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.565+0000 7f3757006700 1 Processor -- start 2023-10-06T04:55:24.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.565+0000 7f3757006700 1 -- start start 2023-10-06T04:55:24.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.565+0000 7f3757006700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3750153bd0 0x7f375015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.565+0000 7f3757006700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.565+0000 7f3757006700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375015d9f0 con 0x7f375015a2e0 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.565+0000 7f3757006700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375015a710 con 0x7f3750153bd0 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.565+0000 7f3757006700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37500b0aa0 con 0x7f3750153fa0 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f3754da2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3750153bd0 0x7f375015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f37555a3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f375015a2e0 0x7f375006b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39026/0 (socket says 172.21.15.3:39026) 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f37555a3700 1 -- 172.21.15.3:0/2868286662 learned_addr learned my addr 172.21.15.3:0/2868286662 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f374ffff700 1 --2- 172.21.15.3:0/2868286662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f3754da2700 1 -- 172.21.15.3:0/2868286662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 msgr2=0x7f3750159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f3754da2700 1 --2- 172.21.15.3:0/2868286662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:24.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f3754da2700 1 -- 172.21.15.3:0/2868286662 >> v1:172.21.15.3:6789/0 conn(0x7f375015a2e0 legacy=0x7f375006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f3754da2700 1 -- 172.21.15.3:0/2868286662 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375015a980 con 0x7f3750153bd0 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f374f7fe700 1 -- 172.21.15.3:0/2868286662 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669392193 0 0) 0x7f375015d9f0 con 0x7f375015a2e0 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f374ffff700 1 --2- 172.21.15.3:0/2868286662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f3754da2700 1 --2- 172.21.15.3:0/2868286662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375015de80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f37440098e0 tx=0x7f37440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273f5054be69217b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f374f7fe700 1 -- 172.21.15.3:0/2868286662 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744009e80 con 0x7f3750153bd0 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f374f7fe700 1 -- 172.21.15.3:0/2868286662 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400b860 con 0x7f3750153bd0 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.566+0000 7f374f7fe700 1 -- 172.21.15.3:0/2868286662 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400bbe0 con 0x7f3750153bd0 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 -- 172.21.15.3:0/2868286662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 msgr2=0x7f375015de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 --2- 172.21.15.3:0/2868286662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375015de80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f37440098e0 tx=0x7f37440091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 -- 172.21.15.3:0/2868286662 shutdown_connections 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 --2- 172.21.15.3:0/2868286662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375015de80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 --2- 172.21.15.3:0/2868286662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 -- 172.21.15.3:0/2868286662 >> 172.21.15.3:0/2868286662 conn(0x7f37500cdc30 msgr2=0x7f375006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 -- 172.21.15.3:0/2868286662 shutdown_connections 2023-10-06T04:55:24.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.567+0000 7f3757006700 1 -- 172.21.15.3:0/2868286662 wait complete. 2023-10-06T04:55:24.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.568+0000 7f3757006700 1 Processor -- start 2023-10-06T04:55:24.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.568+0000 7f3757006700 1 -- start start 2023-10-06T04:55:24.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3757006700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375016db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:24.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3757006700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:24.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3757006700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f375015a2e0 0x7f3750150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3757006700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37501554e0 con 0x7f3750153bd0 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3757006700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3750154f20 con 0x7f375015a2e0 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3757006700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3750155200 con 0x7f3750153fa0 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375016db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375016db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42762/0 (socket says 172.21.15.3:42762) 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 -- 172.21.15.3:0/1787078225 learned_addr learned my addr 172.21.15.3:0/1787078225 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f374ffff700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f37555a3700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f375015a2e0 0x7f3750150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 -- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 msgr2=0x7f3750152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 -- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f375015a2e0 msgr2=0x7f3750150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f375015a2e0 0x7f3750150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.569+0000 7f3754da2700 1 -- 172.21.15.3:0/1787078225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744009420 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.570+0000 7f3754da2700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375016db30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3750070d80 tx=0x7f3744017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.570+0000 7f374cff9700 1 -- 172.21.15.3:0/1787078225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400bbe0 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.570+0000 7f374cff9700 1 -- 172.21.15.3:0/1787078225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744011420 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.570+0000 7f374cff9700 1 -- 172.21.15.3:0/1787078225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440117a0 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.570+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37500aa050 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.570+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3750153200 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.571+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37500aa4f0 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.571+0000 7f374cff9700 1 -- 172.21.15.3:0/1787078225 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3744011b50 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.572+0000 7f374cff9700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3734066f30 0x7f37340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.572+0000 7f374cff9700 1 -- 172.21.15.3:0/1787078225 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3744085c90 con 0x7f3750153bd0 2023-10-06T04:55:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.572+0000 7f374ffff700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3734066f30 0x7f37340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:24.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.572+0000 7f374ffff700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3734066f30 0x7f37340693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f373c006f00 tx=0x7f373c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:24.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.576+0000 7f374cff9700 1 -- 172.21.15.3:0/1787078225 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37440530a0 con 0x7f3750153bd0 2023-10-06T04:55:24.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:24.638 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:24.791+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3750071680 con 0x7f3750153bd0 2023-10-06T04:55:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.599+0000 7f374cff9700 1 -- 172.21.15.3:0/1787078225 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f37440566c0 con 0x7f3750153bd0 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3734066f30 msgr2=0x7f37340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3734066f30 0x7f37340693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f373c006f00 tx=0x7f373c008040 comp rx=0 tx=0).stop 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 msgr2=0x7f375016db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375016db30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3750070d80 tx=0x7f3744017000 comp rx=0 tx=0).stop 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 shutdown_connections 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3734066f30 0x7f37340693e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f375015a2e0 0x7f3750150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3750153bd0 0x7f375016db30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 --2- 172.21.15.3:0/1787078225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3750153fa0 0x7f3750152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 >> 172.21.15.3:0/1787078225 conn(0x7f37500cdc30 msgr2=0x7f3750152160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 shutdown_connections 2023-10-06T04:55:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.602+0000 7f3757006700 1 -- 172.21.15.3:0/1787078225 wait complete. 2023-10-06T04:55:25.606 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T04:55:25.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:55:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.744+0000 7fbbb3dac700 1 Processor -- start 2023-10-06T04:55:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.744+0000 7fbbb3dac700 1 -- start start 2023-10-06T04:55:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.745+0000 7fbbb3dac700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbbac157700 0x7fbbac0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.745+0000 7fbbb3dac700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157ad0 0x7fbbac0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.745+0000 7fbbb3dac700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbac0b2e50 con 0x7fbbac0b1940 2023-10-06T04:55:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.745+0000 7fbbb3dac700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac0b1da0 con 0x7fbbac157700 2023-10-06T04:55:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.745+0000 7fbbb3dac700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac158290 con 0x7fbbac157ad0 2023-10-06T04:55:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1b48700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbbac157700 0x7fbbac0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1347700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157ad0 0x7fbbac0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb2349700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbbac0b1940 0x7fbbac067b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:50800/0 (socket says 172.21.15.3:50800) 2023-10-06T04:55:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb2349700 1 -- 172.21.15.3:0/936236549 learned_addr learned my addr 172.21.15.3:0/936236549 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1b48700 1 -- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157ad0 msgr2=0x7fbbac0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1b48700 1 --2- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157ad0 0x7fbbac0b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1b48700 1 -- 172.21.15.3:0/936236549 >> v1:172.21.15.153:6789/0 conn(0x7fbbac0b1940 legacy=0x7fbbac067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1b48700 1 -- 172.21.15.3:0/936236549 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac0b1fc0 con 0x7fbbac157700 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb0b46700 1 -- 172.21.15.3:0/936236549 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014995544 0 0) 0x7fbbac0b2e50 con 0x7fbbac0b1940 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1347700 1 --2- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157ad0 0x7fbbac0b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb1b48700 1 --2- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157700 0x7fbbac0cc270 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098e0 tx=0x7fbba40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab62dc7c2169ecd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.746+0000 7fbbb0b46700 1 -- 172.21.15.3:0/936236549 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400b640 con 0x7fbbac157700 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb0b46700 1 -- 172.21.15.3:0/936236549 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400b7a0 con 0x7fbbac157700 2023-10-06T04:55:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb0b46700 1 -- 172.21.15.3:0/936236549 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400bb40 con 0x7fbbac157700 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 -- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157700 msgr2=0x7fbbac0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157700 0x7fbbac0cc270 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098e0 tx=0x7fbba40091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 -- 172.21.15.3:0/936236549 shutdown_connections 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157700 0x7fbbac0cc270 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/936236549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157ad0 0x7fbbac0b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 -- 172.21.15.3:0/936236549 >> 172.21.15.3:0/936236549 conn(0x7fbbac0ce2c0 msgr2=0x7fbbac0ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 -- 172.21.15.3:0/936236549 shutdown_connections 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.747+0000 7fbbb3dac700 1 -- 172.21.15.3:0/936236549 wait complete. 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.748+0000 7fbbb3dac700 1 Processor -- start 2023-10-06T04:55:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.748+0000 7fbbb3dac700 1 -- start start 2023-10-06T04:55:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb3dac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0b1940 0x7fbbac155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb3dac700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157700 0x7fbbac0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb3dac700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157ad0 0x7fbbac0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb3dac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac150920 con 0x7fbbac0b1940 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb3dac700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac14fda0 con 0x7fbbac157ad0 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb3dac700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac150080 con 0x7fbbac157700 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1b48700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0b1940 0x7fbbac155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1b48700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0b1940 0x7fbbac155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42776/0 (socket says 172.21.15.3:42776) 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1b48700 1 -- 172.21.15.3:0/1459078785 learned_addr learned my addr 172.21.15.3:0/1459078785 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb2349700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157ad0 0x7fbbac0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1347700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157700 0x7fbbac0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1b48700 1 -- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157700 msgr2=0x7fbbac0ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1b48700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157700 0x7fbbac0ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1b48700 1 -- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157ad0 msgr2=0x7fbbac0afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.749+0000 7fbbb1b48700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157ad0 0x7fbbac0afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.750+0000 7fbbb1b48700 1 -- 172.21.15.3:0/1459078785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009420 con 0x7fbbac0b1940 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.750+0000 7fbbb1b48700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0b1940 0x7fbbac155660 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009f80 tx=0x7fbba4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.750+0000 7fbba9ffb700 1 -- 172.21.15.3:0/1459078785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400b910 con 0x7fbbac0b1940 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.750+0000 7fbba9ffb700 1 -- 172.21.15.3:0/1459078785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400ba70 con 0x7fbbac0b1940 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.750+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba4009420 con 0x7fbbac0b1940 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.750+0000 7fbba9ffb700 1 -- 172.21.15.3:0/1459078785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4024670 con 0x7fbbac0b1940 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.750+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac150640 con 0x7fbbac0b1940 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.751+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac0644c0 con 0x7fbbac0b1940 2023-10-06T04:55:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.751+0000 7fbba9ffb700 1 -- 172.21.15.3:0/1459078785 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbba4024aa0 con 0x7fbbac0b1940 2023-10-06T04:55:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.752+0000 7fbba9ffb700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.752+0000 7fbba9ffb700 1 -- 172.21.15.3:0/1459078785 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbba4085c60 con 0x7fbbac0b1940 2023-10-06T04:55:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.752+0000 7fbbb1347700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.753+0000 7fbbb1347700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbac069990 tx=0x7fbba00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:25.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:25.756+0000 7fbba9ffb700 1 -- 172.21.15.3:0/1459078785 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbba40530c0 con 0x7fbbac0b1940 2023-10-06T04:55:26.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T04:55:26.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.028+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbbac071680 con 0x7fbbac0b1940 2023-10-06T04:55:26.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.028+0000 7fbba9ffb700 1 -- 172.21.15.3:0/1459078785 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbba40566e0 con 0x7fbbac0b1940 2023-10-06T04:55:26.031 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:26.031 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:55:26.032 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.034 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T04:55:26.035 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.036 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.038 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T04:55:26.039 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.040 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.042 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.042 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T04:55:26.042 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T04:55:26.042 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.042 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.042 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T04:55:26.042 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:55:26.043 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.043 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.043 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T04:55:26.043 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T04:55:26.043 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.043 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.044 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:55:26.044 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:55:26.044 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:55:26.044 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.044 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.045 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T04:55:26.045 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T04:55:26.045 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.045 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.045 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:55:26.045 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:55:26.046 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:55:26.046 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.046 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.046 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T04:55:26.047 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T04:55:26.047 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.047 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:55:26.047 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:55:26.047 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:55:26.047 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:55:26.047 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.048 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.048 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T04:55:26.048 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T04:55:26.048 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.048 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:55:26.048 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:55:26.049 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:55:26.049 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.049 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.049 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.049 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T04:55:26.050 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T04:55:26.050 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.050 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:55:26.050 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:55:26.050 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.051 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:26.051 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:26.051 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T04:55:26.051 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T04:55:26.051 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:55:26.052 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:55:26.052 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:55:26.052 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:55:26.052 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.052 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:26.052 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:55:26.053 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:55:26.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 msgr2=0x7fbb980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbac069990 tx=0x7fbba00091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0b1940 msgr2=0x7fbbac155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0b1940 0x7fbbac155660 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009f80 tx=0x7fbba4017000 comp rx=0 tx=0).stop 2023-10-06T04:55:26.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 shutdown_connections 2023-10-06T04:55:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac157ad0 0x7fbbac0afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0b1940 0x7fbbac155660 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 --2- 172.21.15.3:0/1459078785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac157700 0x7fbbac0ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 >> 172.21.15.3:0/1459078785 conn(0x7fbbac0ce2c0 msgr2=0x7fbbac0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 shutdown_connections 2023-10-06T04:55:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.031+0000 7fbbb3dac700 1 -- 172.21.15.3:0/1459078785 wait complete. 2023-10-06T04:55:26.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:55:26.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.173+0000 7f0127122700 1 Processor -- start 2023-10-06T04:55:26.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.173+0000 7f0127122700 1 -- start start 2023-10-06T04:55:26.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f0127122700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f01200b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f0127122700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01200cb440 0x7f01200b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f0127122700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01200cc5c0 con 0x7f0120159500 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f0127122700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01200cda60 con 0x7f01200cb440 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f0127122700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01200b2ca0 con 0x7f01201598d0 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f01256bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01200cb440 0x7f01200b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f01256bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01200cb440 0x7f01200b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42788/0 (socket says 172.21.15.3:42788) 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.174+0000 7f01256bf700 1 -- 172.21.15.3:0/1491998323 learned_addr learned my addr 172.21.15.3:0/1491998323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f011ffff700 1 --2- 172.21.15.3:0/1491998323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f01200b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f01256bf700 1 -- 172.21.15.3:0/1491998323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 msgr2=0x7f01200b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f01256bf700 1 --2- 172.21.15.3:0/1491998323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f01200b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f01256bf700 1 -- 172.21.15.3:0/1491998323 >> v1:172.21.15.153:6789/0 conn(0x7f0120159500 legacy=0x7f01200cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f01256bf700 1 -- 172.21.15.3:0/1491998323 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01200cb9b0 con 0x7f01200cb440 2023-10-06T04:55:26.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f01256bf700 1 --2- 172.21.15.3:0/1491998323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200b2760 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f01080099b0 tx=0x7f0108009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6c4a87d7718118 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f011f7fe700 1 -- 172.21.15.3:0/1491998323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800b860 con 0x7f01200cb440 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f011f7fe700 1 -- 172.21.15.3:0/1491998323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010800b9c0 con 0x7f01200cb440 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.175+0000 7f011f7fe700 1 -- 172.21.15.3:0/1491998323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800bd60 con 0x7f01200cb440 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 -- 172.21.15.3:0/1491998323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 msgr2=0x7f01200b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 --2- 172.21.15.3:0/1491998323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200b2760 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f01080099b0 tx=0x7f0108009250 comp rx=0 tx=0).stop 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 -- 172.21.15.3:0/1491998323 shutdown_connections 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 --2- 172.21.15.3:0/1491998323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200b2760 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 --2- 172.21.15.3:0/1491998323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f01200b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 -- 172.21.15.3:0/1491998323 >> 172.21.15.3:0/1491998323 conn(0x7f0120070310 msgr2=0x7f0120070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 -- 172.21.15.3:0/1491998323 shutdown_connections 2023-10-06T04:55:26.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.176+0000 7f0127122700 1 -- 172.21.15.3:0/1491998323 wait complete. 2023-10-06T04:55:26.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.177+0000 7f0127122700 1 Processor -- start 2023-10-06T04:55:26.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.177+0000 7f0127122700 1 -- start start 2023-10-06T04:55:26.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0120159500 0x7f012015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f0120155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01200c85e0 con 0x7f01200cb440 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012015c320 con 0x7f0120159500 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01200c4500 con 0x7f01201598d0 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42790/0 (socket says 172.21.15.3:42790) 2023-10-06T04:55:26.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 -- 172.21.15.3:0/2262703911 learned_addr learned my addr 172.21.15.3:0/2262703911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 -- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 msgr2=0x7f0120155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f011ffff700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0120159500 0x7f012015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f01256bf700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f0120155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f0120155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 -- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0120159500 msgr2=0x7f012015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0120159500 0x7f012015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 -- 172.21.15.3:0/2262703911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080094f0 con 0x7f01200cb440 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f011ffff700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0120159500 0x7f012015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f01256bf700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f0120155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0124ebe700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200c7440 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f011800c590 tx=0x7f011800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f011cff9700 1 -- 172.21.15.3:0/2262703911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118011590 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f011cff9700 1 -- 172.21.15.3:0/2262703911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180116f0 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01200c7a60 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f011cff9700 1 -- 172.21.15.3:0/2262703911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118011ab0 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.178+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01200c80a0 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.179+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0120071680 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.180+0000 7f011cff9700 1 -- 172.21.15.3:0/2262703911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f011801b3b0 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.180+0000 7f011cff9700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0100066f30 0x7f01000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.180+0000 7f011cff9700 1 -- 172.21.15.3:0/2262703911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f01180838f0 con 0x7f01200cb440 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.180+0000 7f011ffff700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0100066f30 0x7f01000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.181+0000 7f011ffff700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0100066f30 0x7f01000693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f01200b5920 tx=0x7f011000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.184+0000 7f011cff9700 1 -- 172.21.15.3:0/2262703911 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0118050d50 con 0x7f01200cb440 2023-10-06T04:55:26.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.378+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01200c83c0 con 0x7f01200cb440 2023-10-06T04:55:26.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.379+0000 7f011cff9700 1 -- 172.21.15.3:0/2262703911 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+13603 (secure 0 0 0) 0x7f0118017070 con 0x7f01200cb440 2023-10-06T04:55:26.381 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:26.382 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":68,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:25.593793+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:55:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.380+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0100066f30 msgr2=0x7f01000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.380+0000 7f0127122700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0100066f30 0x7f01000693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f01200b5920 tx=0x7f011000b040 comp rx=0 tx=0).stop 2023-10-06T04:55:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.380+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 msgr2=0x7f01200c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.380+0000 7f0127122700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200c7440 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f011800c590 tx=0x7f011800c950 comp rx=0 tx=0).stop 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 shutdown_connections 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0100066f30 0x7f01000693e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f01201598d0 0x7f0120155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01200cb440 0x7f01200c7440 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 --2- 172.21.15.3:0/2262703911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0120159500 0x7f012015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 >> 172.21.15.3:0/2262703911 conn(0x7f0120070310 msgr2=0x7f01200cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 shutdown_connections 2023-10-06T04:55:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.381+0000 7f0127122700 1 -- 172.21.15.3:0/2262703911 wait complete. 2023-10-06T04:55:26.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:55:26.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:55:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.516+0000 7f52b697a700 1 Processor -- start 2023-10-06T04:55:26.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.516+0000 7f52b697a700 1 -- start start 2023-10-06T04:55:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b697a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52b0067ca0 0x7f52b00b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b697a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0068070 0x7f52b0156d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b697a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b00b1ad0 con 0x7f52b0157290 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b697a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b00b3e80 con 0x7f52b0067ca0 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b697a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00b0f90 con 0x7f52b0068070 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b5177700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0068070 0x7f52b0156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b5978700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52b0067ca0 0x7f52b00b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b5177700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0068070 0x7f52b0156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:53432/0 (socket says 172.21.15.3:53432) 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.517+0000 7f52b5177700 1 -- 172.21.15.3:0/4057488460 learned_addr learned my addr 172.21.15.3:0/4057488460 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b5177700 1 -- 172.21.15.3:0/4057488460 >> v2:172.21.15.153:3300/0 conn(0x7f52b0067ca0 msgr2=0x7f52b00b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b5177700 1 --2- 172.21.15.3:0/4057488460 >> v2:172.21.15.153:3300/0 conn(0x7f52b0067ca0 0x7f52b00b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b5177700 1 -- 172.21.15.3:0/4057488460 >> v1:172.21.15.153:6789/0 conn(0x7f52b0157290 legacy=0x7f52b00b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b5177700 1 -- 172.21.15.3:0/4057488460 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0157710 con 0x7f52b0068070 2023-10-06T04:55:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b5978700 1 --2- 172.21.15.3:0/4057488460 >> v2:172.21.15.153:3300/0 conn(0x7f52b0067ca0 0x7f52b00b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b4976700 1 -- 172.21.15.3:0/4057488460 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662903832 0 0) 0x7f52b00b1ad0 con 0x7f52b0157290 2023-10-06T04:55:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b5177700 1 --2- 172.21.15.3:0/4057488460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0068070 0x7f52b0156d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52a40099b0 tx=0x7f52a4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4ce79e34c1ce6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b4976700 1 -- 172.21.15.3:0/4057488460 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4009ea0 con 0x7f52b0068070 2023-10-06T04:55:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b4976700 1 -- 172.21.15.3:0/4057488460 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a400b860 con 0x7f52b0068070 2023-10-06T04:55:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.518+0000 7f52b4976700 1 -- 172.21.15.3:0/4057488460 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400bc00 con 0x7f52b0068070 2023-10-06T04:55:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 -- 172.21.15.3:0/4057488460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0068070 msgr2=0x7f52b0156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 --2- 172.21.15.3:0/4057488460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0068070 0x7f52b0156d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52a40099b0 tx=0x7f52a4009250 comp rx=0 tx=0).stop 2023-10-06T04:55:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 -- 172.21.15.3:0/4057488460 shutdown_connections 2023-10-06T04:55:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 --2- 172.21.15.3:0/4057488460 >> v2:172.21.15.153:3300/0 conn(0x7f52b0067ca0 0x7f52b00b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 --2- 172.21.15.3:0/4057488460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0068070 0x7f52b0156d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 -- 172.21.15.3:0/4057488460 >> 172.21.15.3:0/4057488460 conn(0x7f52b006ce40 msgr2=0x7f52b00afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:26.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 -- 172.21.15.3:0/4057488460 shutdown_connections 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.519+0000 7f52b697a700 1 -- 172.21.15.3:0/4057488460 wait complete. 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 Processor -- start 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 -- start start 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0067ca0 0x7f52b014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b0068070 0x7f52b0153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b0157290 0x7f52b0154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0153a70 con 0x7f52b0068070 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01557c0 con 0x7f52b0157290 2023-10-06T04:55:26.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.520+0000 7f52b697a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0153420 con 0x7f52b0067ca0 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b0068070 0x7f52b0153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b0068070 0x7f52b0153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42794/0 (socket says 172.21.15.3:42794) 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 -- 172.21.15.3:0/304257799 learned_addr learned my addr 172.21.15.3:0/304257799 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5978700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0067ca0 0x7f52b014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b6179700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b0157290 0x7f52b0154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 -- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0067ca0 msgr2=0x7f52b014de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0067ca0 0x7f52b014de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 -- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b0157290 msgr2=0x7f52b0154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b0157290 0x7f52b0154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 -- 172.21.15.3:0/304257799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a0009a20 con 0x7f52b0068070 2023-10-06T04:55:26.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b5177700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b0068070 0x7f52b0153f00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52b00b10d0 tx=0x7f52a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52adffb700 1 -- 172.21.15.3:0/304257799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400bbe0 con 0x7f52b0068070 2023-10-06T04:55:26.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52adffb700 1 -- 172.21.15.3:0/304257799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4011420 con 0x7f52b0068070 2023-10-06T04:55:26.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a40094f0 con 0x7f52b0068070 2023-10-06T04:55:26.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.521+0000 7f52adffb700 1 -- 172.21.15.3:0/304257799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40117e0 con 0x7f52b0068070 2023-10-06T04:55:26.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.522+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b0155aa0 con 0x7f52b0068070 2023-10-06T04:55:26.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.523+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b004c380 con 0x7f52b0068070 2023-10-06T04:55:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.523+0000 7f52adffb700 1 -- 172.21.15.3:0/304257799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52a4011c10 con 0x7f52b0068070 2023-10-06T04:55:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.523+0000 7f52adffb700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f529c066f30 0x7f529c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.523+0000 7f52adffb700 1 -- 172.21.15.3:0/304257799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f52a4085d40 con 0x7f52b0068070 2023-10-06T04:55:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.524+0000 7f52b5978700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f529c066f30 0x7f529c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.524+0000 7f52b5978700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f529c066f30 0x7f529c0693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f52a00099f0 tx=0x7f52a000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.528+0000 7f52adffb700 1 -- 172.21.15.3:0/304257799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52a40531a0 con 0x7f52b0068070 2023-10-06T04:55:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.722+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52b00b11a0 con 0x7f52b0068070 2023-10-06T04:55:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.723+0000 7f52adffb700 1 -- 172.21.15.3:0/304257799 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52a4015020 con 0x7f52b0068070 2023-10-06T04:55:26.725 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f529c066f30 msgr2=0x7f529c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f529c066f30 0x7f529c0693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f52a00099f0 tx=0x7f52a000b040 comp rx=0 tx=0).stop 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b0068070 msgr2=0x7f52b0153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b0068070 0x7f52b0153f00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52b00b10d0 tx=0x7f52a4017000 comp rx=0 tx=0).stop 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 shutdown_connections 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f529c066f30 0x7f529c0693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b0157290 0x7f52b0154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52b0067ca0 0x7f52b014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 --2- 172.21.15.3:0/304257799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b0068070 0x7f52b0153f00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 >> 172.21.15.3:0/304257799 conn(0x7f52b006ce40 msgr2=0x7f52b0157930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 shutdown_connections 2023-10-06T04:55:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.725+0000 7f52b697a700 1 -- 172.21.15.3:0/304257799 wait complete. 2023-10-06T04:55:26.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:55:26.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.859+0000 7efd11831700 1 Processor -- start 2023-10-06T04:55:26.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.860+0000 7efd11831700 1 -- start start 2023-10-06T04:55:26.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.860+0000 7efd11831700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efd0c067dc0 0x7efd0c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.860+0000 7efd11831700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.860+0000 7efd11831700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c0691c0 con 0x7efd0c0b1b00 2023-10-06T04:55:26.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.860+0000 7efd11831700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c0694a0 con 0x7efd0c067dc0 2023-10-06T04:55:26.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.860+0000 7efd11831700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c069780 con 0x7efd0c15cf10 2023-10-06T04:55:26.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0affd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efd0c067dc0 0x7efd0c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c06d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:53464/0 (socket says 172.21.15.3:53464) 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 -- 172.21.15.3:0/1232556245 learned_addr learned my addr 172.21.15.3:0/1232556245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0affd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efd0c067dc0 0x7efd0c0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:33352/0 (socket says 172.21.15.3:33352) 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0a7fc700 1 --1- 172.21.15.3:0/1232556245 >> v1:172.21.15.153:6789/0 conn(0x7efd0c0b1b00 0x7efd0c15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:50830/0 (socket says 172.21.15.3:50830) 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd09ffb700 1 -- 172.21.15.3:0/1232556245 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541922723 0 0) 0x7efd0c0691c0 con 0x7efd0c0b1b00 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd09ffb700 1 -- 172.21.15.3:0/1232556245 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf8003570 con 0x7efd0c0b1b00 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 -- 172.21.15.3:0/1232556245 >> v2:172.21.15.153:3300/0 conn(0x7efd0c067dc0 msgr2=0x7efd0c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 --2- 172.21.15.3:0/1232556245 >> v2:172.21.15.153:3300/0 conn(0x7efd0c067dc0 0x7efd0c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 -- 172.21.15.3:0/1232556245 >> v1:172.21.15.153:6789/0 conn(0x7efd0c0b1b00 legacy=0x7efd0c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 -- 172.21.15.3:0/1232556245 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c069a60 con 0x7efd0c15cf10 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0affd700 1 --2- 172.21.15.3:0/1232556245 >> v2:172.21.15.153:3300/0 conn(0x7efd0c067dc0 0x7efd0c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.861+0000 7efd0b7fe700 1 --2- 172.21.15.3:0/1232556245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c06d480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd04009f20 tx=0x7efd04009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b6ee8030ef3374 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd09ffb700 1 -- 172.21.15.3:0/1232556245 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0400e070 con 0x7efd0c15cf10 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd09ffb700 1 -- 172.21.15.3:0/1232556245 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0400b970 con 0x7efd0c15cf10 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd09ffb700 1 -- 172.21.15.3:0/1232556245 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd040092e0 con 0x7efd0c15cf10 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 -- 172.21.15.3:0/1232556245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 msgr2=0x7efd0c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 --2- 172.21.15.3:0/1232556245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c06d480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd04009f20 tx=0x7efd04009840 comp rx=0 tx=0).stop 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 -- 172.21.15.3:0/1232556245 shutdown_connections 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 --2- 172.21.15.3:0/1232556245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c06d480 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 --2- 172.21.15.3:0/1232556245 >> v2:172.21.15.153:3300/0 conn(0x7efd0c067dc0 0x7efd0c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 -- 172.21.15.3:0/1232556245 >> 172.21.15.3:0/1232556245 conn(0x7efd0c0ce270 msgr2=0x7efd0c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:26.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 -- 172.21.15.3:0/1232556245 shutdown_connections 2023-10-06T04:55:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.862+0000 7efd11831700 1 -- 172.21.15.3:0/1232556245 wait complete. 2023-10-06T04:55:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.863+0000 7efd11831700 1 Processor -- start 2023-10-06T04:55:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd11831700 1 -- start start 2023-10-06T04:55:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd11831700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd0c067dc0 0x7efd0c0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd11831700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd0c0b1b00 0x7efd0c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd11831700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c16a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd11831700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0cc200 con 0x7efd0c067dc0 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd11831700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0cb680 con 0x7efd0c0b1b00 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd11831700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0cb960 con 0x7efd0c15cf10 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd0affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd0c067dc0 0x7efd0c0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd0affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd0c067dc0 0x7efd0c0ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42802/0 (socket says 172.21.15.3:42802) 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd0affd700 1 -- 172.21.15.3:0/874376729 learned_addr learned my addr 172.21.15.3:0/874376729 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd0b7fe700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c16a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.864+0000 7efd0a7fc700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd0c0b1b00 0x7efd0c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd0affd700 1 -- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 msgr2=0x7efd0c16a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd0affd700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c16a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd0affd700 1 -- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd0c0b1b00 msgr2=0x7efd0c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd0affd700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd0c0b1b00 0x7efd0c0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd0affd700 1 -- 172.21.15.3:0/874376729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0093c0 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd0affd700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd0c067dc0 0x7efd0c0ca910 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00ec80 tx=0x7efcfc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd02ffd700 1 -- 172.21.15.3:0/874376729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc005a00 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd02ffd700 1 -- 172.21.15.3:0/874376729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc005b60 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd04009ae0 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd02ffd700 1 -- 172.21.15.3:0/874376729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc00c850 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.865+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c0cbf20 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.866+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c06e5e0 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.866+0000 7efd02ffd700 1 -- 172.21.15.3:0/874376729 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcfc00cbc0 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.867+0000 7efd02ffd700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efcec066f30 0x7efcec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.867+0000 7efd02ffd700 1 -- 172.21.15.3:0/874376729 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efcfc084840 con 0x7efd0c067dc0 2023-10-06T04:55:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.867+0000 7efd0a7fc700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efcec066f30 0x7efcec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.868+0000 7efd0a7fc700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efcec066f30 0x7efcec0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0a6140 tx=0x7efcf4007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:26.871+0000 7efd02ffd700 1 -- 172.21.15.3:0/874376729 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcfc055730 con 0x7efd0c067dc0 2023-10-06T04:55:27.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.065+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd0c0adea0 con 0x7efd0c067dc0 2023-10-06T04:55:27.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.065+0000 7efd02ffd700 1 -- 172.21.15.3:0/874376729 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcfc0200a0 con 0x7efd0c067dc0 2023-10-06T04:55:27.068 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:55:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efcec066f30 msgr2=0x7efcec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efcec066f30 0x7efcec0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0a6140 tx=0x7efcf4007d40 comp rx=0 tx=0).stop 2023-10-06T04:55:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd0c067dc0 msgr2=0x7efd0c0ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd0c067dc0 0x7efd0c0ca910 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00ec80 tx=0x7efcfc009120 comp rx=0 tx=0).stop 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 shutdown_connections 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efcec066f30 0x7efcec0693e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd0c15cf10 0x7efd0c16a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd0c067dc0 0x7efd0c0ca910 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 --2- 172.21.15.3:0/874376729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd0c0b1b00 0x7efd0c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 >> 172.21.15.3:0/874376729 conn(0x7efd0c0ce270 msgr2=0x7efd0c156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 shutdown_connections 2023-10-06T04:55:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.067+0000 7efd11831700 1 -- 172.21.15.3:0/874376729 wait complete. 2023-10-06T04:55:27.082 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:55:27.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.199+0000 7fd288f33700 1 Processor -- start 2023-10-06T04:55:27.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.199+0000 7fd288f33700 1 -- start start 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd288f33700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd288f33700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2840b2740 0x7fd2840cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd288f33700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2840b0ce0 con 0x7fd28415cb50 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd288f33700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840cb820 con 0x7fd2840b2740 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd288f33700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840b0760 con 0x7fd2840cd950 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd282d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2840b2740 0x7fd2840cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd28259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd28415cb50 0x7fd2840b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39034/0 (socket says 172.21.15.3:39034) 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd282d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2840b2740 0x7fd2840cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42816/0 (socket says 172.21.15.3:42816) 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd28259c700 1 -- 172.21.15.3:0/4246697019 learned_addr learned my addr 172.21.15.3:0/4246697019 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:27.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd281d9b700 1 --2- 172.21.15.3:0/4246697019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd282d9d700 1 -- 172.21.15.3:0/4246697019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 msgr2=0x7fd2840b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd282d9d700 1 --2- 172.21.15.3:0/4246697019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd282d9d700 1 -- 172.21.15.3:0/4246697019 >> v1:172.21.15.3:6789/0 conn(0x7fd28415cb50 legacy=0x7fd2840b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd282d9d700 1 -- 172.21.15.3:0/4246697019 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840b0a00 con 0x7fd2840b2740 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd28159a700 1 -- 172.21.15.3:0/4246697019 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579376210 0 0) 0x7fd2840b0ce0 con 0x7fd28415cb50 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd282d9d700 1 --2- 172.21.15.3:0/4246697019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd2840cbcb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd2740098e0 tx=0x7fd2740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a15e4d99e71cc8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd28159a700 1 -- 172.21.15.3:0/4246697019 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274009e80 con 0x7fd2840b2740 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd28159a700 1 -- 172.21.15.3:0/4246697019 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27400b860 con 0x7fd2840b2740 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.200+0000 7fd28159a700 1 -- 172.21.15.3:0/4246697019 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400bbe0 con 0x7fd2840b2740 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 -- 172.21.15.3:0/4246697019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 msgr2=0x7fd2840cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 --2- 172.21.15.3:0/4246697019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd2840cbcb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd2740098e0 tx=0x7fd2740091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 -- 172.21.15.3:0/4246697019 shutdown_connections 2023-10-06T04:55:27.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 --2- 172.21.15.3:0/4246697019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd2840cbcb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 --2- 172.21.15.3:0/4246697019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 -- 172.21.15.3:0/4246697019 >> 172.21.15.3:0/4246697019 conn(0x7fd284070310 msgr2=0x7fd284070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:27.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 -- 172.21.15.3:0/4246697019 shutdown_connections 2023-10-06T04:55:27.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.201+0000 7fd288f33700 1 -- 172.21.15.3:0/4246697019 wait complete. 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.202+0000 7fd288f33700 1 Processor -- start 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.202+0000 7fd288f33700 1 -- start start 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd288f33700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd284149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd288f33700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd288f33700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd28415cb50 0x7fd28414a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd288f33700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840d0310 con 0x7fd2840b2740 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd288f33700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840cf700 con 0x7fd28415cb50 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd288f33700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840cf9e0 con 0x7fd2840cd950 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd284149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd284149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42826/0 (socket says 172.21.15.3:42826) 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 -- 172.21.15.3:0/2087573824 learned_addr learned my addr 172.21.15.3:0/2087573824 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd281d9b700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd282d9d700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd28415cb50 0x7fd28414a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 -- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 msgr2=0x7fd2840c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 -- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd28415cb50 msgr2=0x7fd28414a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd28415cb50 0x7fd28414a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 -- 172.21.15.3:0/2087573824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274009420 con 0x7fd2840b2740 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd282d9d700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd28415cb50 0x7fd28414a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.203+0000 7fd28259c700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd284149750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c930 tx=0x7fd27c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.204+0000 7fd27a7fc700 1 -- 172.21.15.3:0/2087573824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c005d80 con 0x7fd2840b2740 2023-10-06T04:55:27.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.204+0000 7fd27a7fc700 1 -- 172.21.15.3:0/2087573824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c0070f0 con 0x7fd2840b2740 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.204+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2840cfcc0 con 0x7fd2840b2740 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.204+0000 7fd27a7fc700 1 -- 172.21.15.3:0/2087573824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0074b0 con 0x7fd2840b2740 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.204+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd284149c90 con 0x7fd2840b2740 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.205+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd284071680 con 0x7fd2840b2740 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.205+0000 7fd27a7fc700 1 -- 172.21.15.3:0/2087573824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd27c0078e0 con 0x7fd2840b2740 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.205+0000 7fd27a7fc700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd264066f30 0x7fd2640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.206+0000 7fd27a7fc700 1 -- 172.21.15.3:0/2087573824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd27c0840c0 con 0x7fd2840b2740 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.206+0000 7fd281d9b700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd264066f30 0x7fd2640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.206+0000 7fd281d9b700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd264066f30 0x7fd2640693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd28415c410 tx=0x7fd26c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:27.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.210+0000 7fd27a7fc700 1 -- 172.21.15.3:0/2087573824 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd27c0515a0 con 0x7fd2840b2740 2023-10-06T04:55:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.407+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd2840d0160 con 0x7fd2840b2740 2023-10-06T04:55:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.407+0000 7fd27a7fc700 1 -- 172.21.15.3:0/2087573824 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd27c054bc0 con 0x7fd2840b2740 2023-10-06T04:55:27.410 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:27.410 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:55:27.410 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:27.410 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:55:27.411 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:55:27.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:27.411 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:27.411 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:55:27.411 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:55:27.411 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:27.411 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:55:27.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd264066f30 msgr2=0x7fd2640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd264066f30 0x7fd2640693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd28415c410 tx=0x7fd26c008040 comp rx=0 tx=0).stop 2023-10-06T04:55:27.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 msgr2=0x7fd284149750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd284149750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c930 tx=0x7fd27c0016f0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 shutdown_connections 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd264066f30 0x7fd2640693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd28415cb50 0x7fd28414a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2840b2740 0x7fd284149750 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 --2- 172.21.15.3:0/2087573824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2840cd950 0x7fd2840c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.409+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 >> 172.21.15.3:0/2087573824 conn(0x7fd284070310 msgr2=0x7fd2840b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.410+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 shutdown_connections 2023-10-06T04:55:27.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.410+0000 7fd288f33700 1 -- 172.21.15.3:0/2087573824 wait complete. 2023-10-06T04:55:27.424 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:55:27.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:55:27.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.545+0000 7fd1c537a700 1 Processor -- start 2023-10-06T04:55:27.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.545+0000 7fd1c537a700 1 -- start start 2023-10-06T04:55:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.545+0000 7fd1c537a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00af520 0x7fd1c00ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.545+0000 7fd1c537a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1c00cd1c0 0x7fd1c00cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.545+0000 7fd1c537a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c0159c00 con 0x7fd1c00af150 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.545+0000 7fd1c537a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c00cd640 con 0x7fd1c00cd1c0 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1c537a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c015e000 con 0x7fd1c00af520 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1c4b79700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1c00cd1c0 0x7fd1c00cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1bffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd1c00af150 0x7fd1c00c68a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39038/0 (socket says 172.21.15.3:39038) 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1bffff700 1 -- 172.21.15.3:0/1920158520 learned_addr learned my addr 172.21.15.3:0/1920158520 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1bf7fe700 1 --2- 172.21.15.3:0/1920158520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00af520 0x7fd1c00ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1c4b79700 1 -- 172.21.15.3:0/1920158520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00af520 msgr2=0x7fd1c00ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1c4b79700 1 --2- 172.21.15.3:0/1920158520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00af520 0x7fd1c00ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1c4b79700 1 -- 172.21.15.3:0/1920158520 >> v1:172.21.15.3:6789/0 conn(0x7fd1c00af150 legacy=0x7fd1c00c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1c4b79700 1 -- 172.21.15.3:0/1920158520 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00ce0f0 con 0x7fd1c00cd1c0 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1beffd700 1 -- 172.21.15.3:0/1920158520 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977537631 0 0) 0x7fd1c0159c00 con 0x7fd1c00af150 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1c4b79700 1 --2- 172.21.15.3:0/1920158520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00cd1c0 0x7fd1c00cbc70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009e60 tx=0x7fd1b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2668690707ed9bb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1beffd700 1 -- 172.21.15.3:0/1920158520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800e070 con 0x7fd1c00cd1c0 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1beffd700 1 -- 172.21.15.3:0/1920158520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b800bc30 con 0x7fd1c00cd1c0 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.546+0000 7fd1beffd700 1 -- 172.21.15.3:0/1920158520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8012430 con 0x7fd1c00cd1c0 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 -- 172.21.15.3:0/1920158520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00cd1c0 msgr2=0x7fd1c00cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 --2- 172.21.15.3:0/1920158520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00cd1c0 0x7fd1c00cbc70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009e60 tx=0x7fd1b8009780 comp rx=0 tx=0).stop 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 -- 172.21.15.3:0/1920158520 shutdown_connections 2023-10-06T04:55:27.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 --2- 172.21.15.3:0/1920158520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00cd1c0 0x7fd1c00cbc70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 --2- 172.21.15.3:0/1920158520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00af520 0x7fd1c00ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 -- 172.21.15.3:0/1920158520 >> 172.21.15.3:0/1920158520 conn(0x7fd1c0070310 msgr2=0x7fd1c0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:27.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 -- 172.21.15.3:0/1920158520 shutdown_connections 2023-10-06T04:55:27.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.547+0000 7fd1c537a700 1 -- 172.21.15.3:0/1920158520 wait complete. 2023-10-06T04:55:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.548+0000 7fd1c537a700 1 Processor -- start 2023-10-06T04:55:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.548+0000 7fd1c537a700 1 -- start start 2023-10-06T04:55:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1c537a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00af150 0x7fd1c00cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1c537a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c00af520 0x7fd1c015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1c537a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00cd1c0 0x7fd1c01579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1c537a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00ccfd0 con 0x7fd1c00af150 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1c537a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00b04b0 con 0x7fd1c00af520 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1c537a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c015caa0 con 0x7fd1c00cd1c0 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00af150 0x7fd1c00cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00af150 0x7fd1c00cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42850/0 (socket says 172.21.15.3:42850) 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 -- 172.21.15.3:0/3466220997 learned_addr learned my addr 172.21.15.3:0/3466220997 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1c4b79700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00cd1c0 0x7fd1c01579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 -- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00cd1c0 msgr2=0x7fd1c01579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bf7fe700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c00af520 0x7fd1c015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00cd1c0 0x7fd1c01579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 -- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c00af520 msgr2=0x7fd1c015c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c00af520 0x7fd1c015c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 -- 172.21.15.3:0/3466220997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8009a20 con 0x7fd1c00af150 2023-10-06T04:55:27.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.549+0000 7fd1bffff700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00af150 0x7fd1c00cc980 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c810 tx=0x7fd1b0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.550+0000 7fd1a7fff700 1 -- 172.21.15.3:0/3466220997 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0005b30 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.550+0000 7fd1a7fff700 1 -- 172.21.15.3:0/3466220997 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b0005c90 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.550+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c0157f00 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.550+0000 7fd1a7fff700 1 -- 172.21.15.3:0/3466220997 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0007300 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.550+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c00afe60 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.551+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c004c380 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.551+0000 7fd1a7fff700 1 -- 172.21.15.3:0/3466220997 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1b001b400 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.552+0000 7fd1a7fff700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.552+0000 7fd1a7fff700 1 -- 172.21.15.3:0/3466220997 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd1b0083dc0 con 0x7fd1c00af150 2023-10-06T04:55:27.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.552+0000 7fd1bf7fe700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:27.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.552+0000 7fd1bf7fe700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ac066f30 0x7fd1ac0693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00b1320 tx=0x7fd1b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:27.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.556+0000 7fd1a7fff700 1 -- 172.21.15.3:0/3466220997 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1b00512a0 con 0x7fd1c00af150 2023-10-06T04:55:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:27.750+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd1c015d3f0 con 0x7fd1c00af150 2023-10-06T04:55:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.614+0000 7fd1a7fff700 1 -- 172.21.15.3:0/3466220997 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd1b00548c0 con 0x7fd1c00af150 2023-10-06T04:55:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ac066f30 msgr2=0x7fd1ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ac066f30 0x7fd1ac0693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00b1320 tx=0x7fd1b4008040 comp rx=0 tx=0).stop 2023-10-06T04:55:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00af150 msgr2=0x7fd1c00cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00af150 0x7fd1c00cc980 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c810 tx=0x7fd1b0007df0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 shutdown_connections 2023-10-06T04:55:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1c00cd1c0 0x7fd1c01579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c00af150 0x7fd1c00cc980 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.618+0000 7fd1c537a700 1 --2- 172.21.15.3:0/3466220997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c00af520 0x7fd1c015c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.619+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 >> 172.21.15.3:0/3466220997 conn(0x7fd1c0070310 msgr2=0x7fd1c00c5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.619+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 shutdown_connections 2023-10-06T04:55:28.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.620+0000 7fd1c537a700 1 -- 172.21.15.3:0/3466220997 wait complete. 2023-10-06T04:55:28.623 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:55:28.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:55:28.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.769+0000 7f6c51153700 1 Processor -- start 2023-10-06T04:55:28.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.769+0000 7f6c51153700 1 -- start start 2023-10-06T04:55:28.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.769+0000 7f6c51153700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c4c067dc0 0x7f6c4c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.769+0000 7f6c51153700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 0x7f6c4c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.769+0000 7f6c51153700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4c0691c0 con 0x7f6c4c0b1b00 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.769+0000 7f6c51153700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4c0694a0 con 0x7f6c4c067dc0 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.769+0000 7f6c51153700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c069780 con 0x7f6c4c15cf10 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c4a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6c4c0b1b00 0x7f6c4c15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39040/0 (socket says 172.21.15.3:39040) 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c4a59c700 1 -- 172.21.15.3:0/4259035700 learned_addr learned my addr 172.21.15.3:0/4259035700 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/4259035700 >> v2:172.21.15.153:3300/0 conn(0x7f6c4c067dc0 0x7f6c4c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c4b59e700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 0x7f6c4c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518146003 0 0) 0x7f6c4c0691c0 con 0x7f6c4c0b1b00 2023-10-06T04:55:28.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c34003570 con 0x7f6c4c0b1b00 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757538827 0 0) 0x7f6c34003570 con 0x7f6c4c0b1b00 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 msgr2=0x7f6c4c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 0x7f6c4c06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 >> v2:172.21.15.153:3300/0 conn(0x7f6c4c067dc0 msgr2=0x7f6c4c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 --2- 172.21.15.3:0/4259035700 >> v2:172.21.15.153:3300/0 conn(0x7f6c4c067dc0 0x7f6c4c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c069a60 con 0x7f6c4c0b1b00 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6c40001470 con 0x7f6c4c0b1b00 2023-10-06T04:55:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 >> v1:172.21.15.3:6789/0 conn(0x7f6c4c0b1b00 legacy=0x7f6c4c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c34003a60 0x7f6c34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34004810 0x7f6c34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 0x7f6c340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0694a0 con 0x7f6c34003a60 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c069780 con 0x7f6c34004810 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3400ba60 con 0x7f6c4c15cf10 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.770+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c34003a60 0x7f6c34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4a59c700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34004810 0x7f6c34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4b59e700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 0x7f6c340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4ad9d700 1 -- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 msgr2=0x7f6c340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c15cf10 0x7f6c340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4ad9d700 1 -- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34004810 msgr2=0x7f6c34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34004810 0x7f6c34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4ad9d700 1 -- 172.21.15.3:0/4259035700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34003570 con 0x7f6c34003a60 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c34003a60 0x7f6c34003e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009c20 tx=0x7f6c3c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2a2a3183cae6ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c005a00 con 0x7f6c34003a60 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c005b60 con 0x7f6c34003a60 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c49d9b700 1 -- 172.21.15.3:0/4259035700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00e710 con 0x7f6c34003a60 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.771+0000 7f6c51153700 1 -- 172.21.15.3:0/4259035700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c34003a60 msgr2=0x7f6c34003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.772+0000 7f6c51153700 1 --2- 172.21.15.3:0/4259035700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c34003a60 0x7f6c34003e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009c20 tx=0x7f6c3c009120 comp rx=0 tx=0).stop 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.772+0000 7f6c4b59e700 1 -- 172.21.15.3:0/4259035700 reap_dead start 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.772+0000 7f6c51153700 1 -- 172.21.15.3:0/4259035700 shutdown_connections 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.772+0000 7f6c51153700 1 -- 172.21.15.3:0/4259035700 >> 172.21.15.3:0/4259035700 conn(0x7f6c4c0ce270 msgr2=0x7f6c4c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.772+0000 7f6c51153700 1 -- 172.21.15.3:0/4259035700 shutdown_connections 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.772+0000 7f6c51153700 1 -- 172.21.15.3:0/4259035700 wait complete. 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.773+0000 7f6c51153700 1 Processor -- start 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.773+0000 7f6c51153700 1 -- start start 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34003a60 0x7f6c4c12edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c0aee20 0x7f6c4c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c1567e0 0x7f6c4c156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1571a0 con 0x7f6c4c0aee20 2023-10-06T04:55:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0aff80 con 0x7f6c34003a60 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c12f2f0 con 0x7f6c4c1567e0 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c0aee20 0x7f6c4c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c0aee20 0x7f6c4c15a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42868/0 (socket says 172.21.15.3:42868) 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 -- 172.21.15.3:0/2205444956 learned_addr learned my addr 172.21.15.3:0/2205444956 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4b59e700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c1567e0 0x7f6c4c156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34003a60 0x7f6c4c12edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 -- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c1567e0 msgr2=0x7f6c4c156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c1567e0 0x7f6c4c156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 -- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34003a60 msgr2=0x7f6c4c12edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34003a60 0x7f6c4c12edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 -- 172.21.15.3:0/2205444956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c0af370 con 0x7f6c4c0aee20 2023-10-06T04:55:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4b59e700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c1567e0 0x7f6c4c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c4a59c700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c0aee20 0x7f6c4c15a1d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000fd0 tx=0x7f6c40001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c32ffd700 1 -- 172.21.15.3:0/2205444956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000f870 con 0x7f6c4c0aee20 2023-10-06T04:55:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c32ffd700 1 -- 172.21.15.3:0/2205444956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4000f9d0 con 0x7f6c4c0aee20 2023-10-06T04:55:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c0af650 con 0x7f6c4c0aee20 2023-10-06T04:55:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c32ffd700 1 -- 172.21.15.3:0/2205444956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000fd90 con 0x7f6c4c0aee20 2023-10-06T04:55:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.774+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c0afc10 con 0x7f6c4c0aee20 2023-10-06T04:55:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.775+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c0b02f0 con 0x7f6c4c0aee20 2023-10-06T04:55:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.776+0000 7f6c32ffd700 1 -- 172.21.15.3:0/2205444956 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c40013600 con 0x7f6c4c0aee20 2023-10-06T04:55:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.776+0000 7f6c32ffd700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c38066f30 0x7f6c380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.776+0000 7f6c32ffd700 1 -- 172.21.15.3:0/2205444956 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f6c40084720 con 0x7f6c4c0aee20 2023-10-06T04:55:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.777+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c38066f30 0x7f6c380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.777+0000 7f6c4ad9d700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c38066f30 0x7f6c380693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009160 tx=0x7f6c3c019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.781+0000 7f6c32ffd700 1 -- 172.21.15.3:0/2205444956 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c40051a10 con 0x7f6c4c0aee20 2023-10-06T04:55:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:28.975+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6c4c04c380 con 0x7f6c4c0aee20 2023-10-06T04:55:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.626+0000 7f6c32ffd700 1 -- 172.21.15.3:0/2205444956 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6c40055030 con 0x7f6c4c0aee20 2023-10-06T04:55:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.638+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c38066f30 msgr2=0x7f6c380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.638+0000 7f6c51153700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c38066f30 0x7f6c380693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009160 tx=0x7f6c3c019000 comp rx=0 tx=0).stop 2023-10-06T04:55:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.638+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c0aee20 msgr2=0x7f6c4c15a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.638+0000 7f6c51153700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c0aee20 0x7f6c4c15a1d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000fd0 tx=0x7f6c40001e90 comp rx=0 tx=0).stop 2023-10-06T04:55:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.639+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 shutdown_connections 2023-10-06T04:55:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.639+0000 7f6c51153700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c38066f30 0x7f6c380693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.639+0000 7f6c51153700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c34003a60 0x7f6c4c12edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.639+0000 7f6c51153700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c4c1567e0 0x7f6c4c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.639+0000 7f6c51153700 1 --2- 172.21.15.3:0/2205444956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c0aee20 0x7f6c4c15a1d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.639+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 >> 172.21.15.3:0/2205444956 conn(0x7f6c4c0ce270 msgr2=0x7f6c4c13f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.641+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 shutdown_connections 2023-10-06T04:55:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.642+0000 7f6c51153700 1 -- 172.21.15.3:0/2205444956 wait complete. 2023-10-06T04:55:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:55:29.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:55:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.784+0000 7f59d9092700 1 Processor -- start 2023-10-06T04:55:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.784+0000 7f59d9092700 1 -- start start 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.784+0000 7f59d9092700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d41596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d9092700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59d406adb0 0x7f59d4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d9092700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f59d40ae9d0 0x7f59d406c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d9092700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d406dac0 con 0x7f59d406adb0 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d9092700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d415a160 con 0x7f59d40ae9d0 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d9092700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40af1b0 con 0x7f59d406a340 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59d406adb0 0x7f59d4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59d406adb0 0x7f59d4159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42876/0 (socket says 172.21.15.3:42876) 2023-10-06T04:55:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 -- 172.21.15.3:0/530764343 learned_addr learned my addr 172.21.15.3:0/530764343 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d359e700 1 --2- 172.21.15.3:0/530764343 >> v2:172.21.15.153:3300/0 conn(0x7f59d40ae9d0 0x7f59d406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d2d9d700 1 --2- 172.21.15.3:0/530764343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d41596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 -- 172.21.15.3:0/530764343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 msgr2=0x7f59d41596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 --2- 172.21.15.3:0/530764343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d41596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 -- 172.21.15.3:0/530764343 >> v2:172.21.15.153:3300/0 conn(0x7f59d40ae9d0 msgr2=0x7f59d406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 --2- 172.21.15.3:0/530764343 >> v2:172.21.15.153:3300/0 conn(0x7f59d40ae9d0 0x7f59d406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 -- 172.21.15.3:0/530764343 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d406d4b0 con 0x7f59d406adb0 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d2d9d700 1 --2- 172.21.15.3:0/530764343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d41596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d259c700 1 --2- 172.21.15.3:0/530764343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d406adb0 0x7f59d4159c20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f59d40a9f60 tx=0x7f59c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2486a4dd77234e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d1d9b700 1 -- 172.21.15.3:0/530764343 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c800b8f0 con 0x7f59d406adb0 2023-10-06T04:55:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d1d9b700 1 -- 172.21.15.3:0/530764343 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c800ba50 con 0x7f59d406adb0 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.785+0000 7f59d1d9b700 1 -- 172.21.15.3:0/530764343 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c800bdf0 con 0x7f59d406adb0 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 -- 172.21.15.3:0/530764343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d406adb0 msgr2=0x7f59d4159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 --2- 172.21.15.3:0/530764343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d406adb0 0x7f59d4159c20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f59d40a9f60 tx=0x7f59c8009250 comp rx=0 tx=0).stop 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 -- 172.21.15.3:0/530764343 shutdown_connections 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 --2- 172.21.15.3:0/530764343 >> v2:172.21.15.153:3300/0 conn(0x7f59d40ae9d0 0x7f59d406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 --2- 172.21.15.3:0/530764343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d41596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 --2- 172.21.15.3:0/530764343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d406adb0 0x7f59d4159c20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 -- 172.21.15.3:0/530764343 >> 172.21.15.3:0/530764343 conn(0x7f59d4067a40 msgr2=0x7f59d40ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 -- 172.21.15.3:0/530764343 shutdown_connections 2023-10-06T04:55:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.786+0000 7f59d9092700 1 -- 172.21.15.3:0/530764343 wait complete. 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.787+0000 7f59d9092700 1 Processor -- start 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.787+0000 7f59d9092700 1 -- start start 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d9092700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d4156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d9092700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d40ae9d0 0x7f59d4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d9092700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d415a2d0 0x7f59d415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d9092700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d415dc70 con 0x7f59d415a2d0 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d9092700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d415df50 con 0x7f59d40ae9d0 2023-10-06T04:55:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d9092700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d415e230 con 0x7f59d406a340 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d415a2d0 0x7f59d415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d415a2d0 0x7f59d415a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42892/0 (socket says 172.21.15.3:42892) 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 -- 172.21.15.3:0/3439629351 learned_addr learned my addr 172.21.15.3:0/3439629351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d2d9d700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d4156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d259c700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d40ae9d0 0x7f59d4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 -- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 msgr2=0x7f59d4156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d4156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 -- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d40ae9d0 msgr2=0x7f59d4156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d40ae9d0 0x7f59d4156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.788+0000 7f59d359e700 1 -- 172.21.15.3:0/3439629351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80094f0 con 0x7f59d415a2d0 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.789+0000 7f59d359e700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d415a2d0 0x7f59d415a740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f59cc007f20 tx=0x7f59cc009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:29.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.789+0000 7f59baffd700 1 -- 172.21.15.3:0/3439629351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00eb90 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.789+0000 7f59baffd700 1 -- 172.21.15.3:0/3439629351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00ed10 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.789+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d415e510 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.789+0000 7f59baffd700 1 -- 172.21.15.3:0/3439629351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc014670 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.789+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d415ac80 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.790+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d406c000 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.790+0000 7f59baffd700 1 -- 172.21.15.3:0/3439629351 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59cc0149e0 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.791+0000 7f59baffd700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.791+0000 7f59baffd700 1 -- 172.21.15.3:0/3439629351 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f59cc085b00 con 0x7f59d415a2d0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.791+0000 7f59d2d9d700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.791+0000 7f59d2d9d700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59c0066f30 0x7f59c00693e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006ea0 tx=0x7f59c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:29.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:29.795+0000 7f59baffd700 1 -- 172.21.15.3:0/3439629351 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59cc052cf0 con 0x7f59d415a2d0 2023-10-06T04:55:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.035+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f59d404c380 con 0x7f59d415a2d0 2023-10-06T04:55:30.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.631+0000 7f59baffd700 1 -- 172.21.15.3:0/3439629351 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f59cc056310 con 0x7f59d415a2d0 2023-10-06T04:55:30.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59c0066f30 msgr2=0x7f59c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:30.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59c0066f30 0x7f59c00693e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006ea0 tx=0x7f59c4008040 comp rx=0 tx=0).stop 2023-10-06T04:55:30.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d415a2d0 msgr2=0x7f59d415a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d415a2d0 0x7f59d415a740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f59cc007f20 tx=0x7f59cc009860 comp rx=0 tx=0).stop 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 shutdown_connections 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d415a2d0 0x7f59d415a740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59d406a340 0x7f59d4156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 --2- 172.21.15.3:0/3439629351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d40ae9d0 0x7f59d4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 >> 172.21.15.3:0/3439629351 conn(0x7f59d4067a40 msgr2=0x7f59d40a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:30.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 shutdown_connections 2023-10-06T04:55:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.641+0000 7f59d9092700 1 -- 172.21.15.3:0/3439629351 wait complete. 2023-10-06T04:55:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 11 and data pool 12 2023-10-06T04:55:30.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:55:30.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.789+0000 7effe1f5a700 1 Processor -- start 2023-10-06T04:55:30.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.789+0000 7effe1f5a700 1 -- start start 2023-10-06T04:55:30.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe1f5a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7effdc0cb780 0x7effdc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:30.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe1f5a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc159500 0x7effdc0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:30.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe1f5a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc0cf130 con 0x7effdc0cb3b0 2023-10-06T04:55:30.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe1f5a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc0ce520 con 0x7effdc0cb780 2023-10-06T04:55:30.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe1f5a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0ce800 con 0x7effdc159500 2023-10-06T04:55:30.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effdbfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7effdc0cb780 0x7effdc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:30.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe1759700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc159500 0x7effdc0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:30.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe0f58700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7effdc0cb3b0 0x7effdc0cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:50832/0 (socket says 172.21.15.3:50832) 2023-10-06T04:55:30.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe1759700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc159500 0x7effdc0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:53566/0 (socket says 172.21.15.3:53566) 2023-10-06T04:55:30.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effdbfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7effdc0cb780 0x7effdc158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:33420/0 (socket says 172.21.15.3:33420) 2023-10-06T04:55:30.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.790+0000 7effe0f58700 1 -- 172.21.15.3:0/19003099 learned_addr learned my addr 172.21.15.3:0/19003099 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:30.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effdb7fe700 1 -- 172.21.15.3:0/19003099 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523838999 0 0) 0x7effdc0cf130 con 0x7effdc0cb3b0 2023-10-06T04:55:30.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effdb7fe700 1 -- 172.21.15.3:0/19003099 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc8003570 con 0x7effdc0cb3b0 2023-10-06T04:55:30.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effe1759700 1 -- 172.21.15.3:0/19003099 >> v2:172.21.15.153:3300/0 conn(0x7effdc0cb780 msgr2=0x7effdc158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:30.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effe1759700 1 --2- 172.21.15.3:0/19003099 >> v2:172.21.15.153:3300/0 conn(0x7effdc0cb780 0x7effdc158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effe1759700 1 -- 172.21.15.3:0/19003099 >> v1:172.21.15.153:6789/0 conn(0x7effdc0cb3b0 legacy=0x7effdc0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:30.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effe1759700 1 -- 172.21.15.3:0/19003099 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc0ceae0 con 0x7effdc159500 2023-10-06T04:55:30.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effdbfff700 1 --2- 172.21.15.3:0/19003099 >> v2:172.21.15.153:3300/0 conn(0x7effdc0cb780 0x7effdc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:30.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effe1759700 1 --2- 172.21.15.3:0/19003099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc159500 0x7effdc0cccc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effd40098e0 tx=0x7effd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4be3e92b9066825 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:30.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effdb7fe700 1 -- 172.21.15.3:0/19003099 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4011630 con 0x7effdc159500 2023-10-06T04:55:30.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effdb7fe700 1 -- 172.21.15.3:0/19003099 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd4011790 con 0x7effdc159500 2023-10-06T04:55:30.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effdb7fe700 1 -- 172.21.15.3:0/19003099 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4011b30 con 0x7effdc159500 2023-10-06T04:55:30.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effe1f5a700 1 -- 172.21.15.3:0/19003099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc159500 msgr2=0x7effdc0cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:30.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.791+0000 7effe1f5a700 1 --2- 172.21.15.3:0/19003099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc159500 0x7effdc0cccc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effd40098e0 tx=0x7effd40091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.792+0000 7effe1f5a700 1 -- 172.21.15.3:0/19003099 shutdown_connections 2023-10-06T04:55:30.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.792+0000 7effe1f5a700 1 --2- 172.21.15.3:0/19003099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc159500 0x7effdc0cccc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.792+0000 7effe1f5a700 1 --2- 172.21.15.3:0/19003099 >> v2:172.21.15.153:3300/0 conn(0x7effdc0cb780 0x7effdc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.792+0000 7effe1f5a700 1 -- 172.21.15.3:0/19003099 >> 172.21.15.3:0/19003099 conn(0x7effdc070310 msgr2=0x7effdc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:30.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.792+0000 7effe1f5a700 1 -- 172.21.15.3:0/19003099 shutdown_connections 2023-10-06T04:55:30.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.792+0000 7effe1f5a700 1 -- 172.21.15.3:0/19003099 wait complete. 2023-10-06T04:55:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.793+0000 7effe1f5a700 1 Processor -- start 2023-10-06T04:55:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.793+0000 7effe1f5a700 1 -- start start 2023-10-06T04:55:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.793+0000 7effe1f5a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effdc0cb3b0 0x7effdc14d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.793+0000 7effe1f5a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc0cb780 0x7effdc154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.793+0000 7effe1f5a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc159500 0x7effdc154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.793+0000 7effe1f5a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc14f340 con 0x7effdc159500 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.793+0000 7effe1f5a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0b3df0 con 0x7effdc0cb3b0 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1f5a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0b40d0 con 0x7effdc0cb780 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc159500 0x7effdc154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc159500 0x7effdc154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42898/0 (socket says 172.21.15.3:42898) 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 -- 172.21.15.3:0/2638887146 learned_addr learned my addr 172.21.15.3:0/2638887146 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe0f58700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effdc0cb3b0 0x7effdc14d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effdbfff700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc0cb780 0x7effdc154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 -- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc0cb780 msgr2=0x7effdc154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc0cb780 0x7effdc154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 -- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effdc0cb3b0 msgr2=0x7effdc14d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effdc0cb3b0 0x7effdc14d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 -- 172.21.15.3:0/2638887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc009a20 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1759700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc159500 0x7effdc154e80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7effdc0ce940 tx=0x7effd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effd8ff9700 1 -- 172.21.15.3:0/2638887146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4011ad0 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effd8ff9700 1 -- 172.21.15.3:0/2638887146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd40195c0 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd4009420 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effd8ff9700 1 -- 172.21.15.3:0/2638887146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4019980 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.794+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc0b4690 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.795+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc14f4b0 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.796+0000 7effd8ff9700 1 -- 172.21.15.3:0/2638887146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effd4019d30 con 0x7effdc159500 2023-10-06T04:55:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.796+0000 7effd8ff9700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effbc066f30 0x7effbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.796+0000 7effd8ff9700 1 -- 172.21.15.3:0/2638887146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7effd40861e0 con 0x7effdc159500 2023-10-06T04:55:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.796+0000 7effe0f58700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effbc066f30 0x7effbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.797+0000 7effe0f58700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effbc066f30 0x7effbc0693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7effc40058d0 tx=0x7effc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.800+0000 7effd8ff9700 1 -- 172.21.15.3:0/2638887146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effd40532f0 con 0x7effdc159500 2023-10-06T04:55:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.996+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effdc04c380 con 0x7effdc159500 2023-10-06T04:55:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.997+0000 7effd8ff9700 1 -- 172.21.15.3:0/2638887146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16834 (secure 0 0 0) 0x7effd4056910 con 0x7effdc159500 2023-10-06T04:55:31.000 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:31.000 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":71,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:30.037151+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:55:27.781764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T04:55:29.007164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effbc066f30 msgr2=0x7effbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effbc066f30 0x7effbc0693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7effc40058d0 tx=0x7effc4009040 comp rx=0 tx=0).stop 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc159500 msgr2=0x7effdc154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc159500 0x7effdc154e80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7effdc0ce940 tx=0x7effd4017000 comp rx=0 tx=0).stop 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 shutdown_connections 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7effbc066f30 0x7effbc0693e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effdc0cb3b0 0x7effdc14d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc159500 0x7effdc154e80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 --2- 172.21.15.3:0/2638887146 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effdc0cb780 0x7effdc154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 >> 172.21.15.3:0/2638887146 conn(0x7effdc070310 msgr2=0x7effdc0b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 shutdown_connections 2023-10-06T04:55:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:30.999+0000 7effe1f5a700 1 -- 172.21.15.3:0/2638887146 wait complete. 2023-10-06T04:55:31.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:55:31.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.130+0000 7f911924f700 1 Processor -- start 2023-10-06T04:55:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.130+0000 7f911924f700 1 -- start start 2023-10-06T04:55:31.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f911924f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91140cb3b0 0x7f91140cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f911924f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb780 0x7f9114158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f911924f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91140cf130 con 0x7f9114159500 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f911924f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91140ce520 con 0x7f91140cb3b0 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f911924f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91140ce800 con 0x7f91140cb780 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f91137fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb780 0x7f9114158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f9113fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91140cb3b0 0x7f91140cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f91137fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb780 0x7f9114158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:53586/0 (socket says 172.21.15.3:53586) 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f9113fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91140cb3b0 0x7f91140cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:33448/0 (socket says 172.21.15.3:33448) 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f9118a4e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9114159500 0x7f91140cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:50836/0 (socket says 172.21.15.3:50836) 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.131+0000 7f91137fe700 1 -- 172.21.15.3:0/1955036196 learned_addr learned my addr 172.21.15.3:0/1955036196 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9113fff700 1 -- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb780 msgr2=0x7f9114158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9113fff700 1 --2- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb780 0x7f9114158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9113fff700 1 -- 172.21.15.3:0/1955036196 >> v1:172.21.15.153:6789/0 conn(0x7f9114159500 legacy=0x7f91140cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9113fff700 1 -- 172.21.15.3:0/1955036196 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140ceae0 con 0x7f91140cb3b0 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9112ffd700 1 -- 172.21.15.3:0/1955036196 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408535811 0 0) 0x7f91140cf130 con 0x7f9114159500 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f91137fe700 1 --2- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb780 0x7f9114158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9113fff700 1 --2- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb3b0 0x7f91140cf5c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0098e0 tx=0x7f90fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56afc50b80a53757 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9112ffd700 1 -- 172.21.15.3:0/1955036196 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00b640 con 0x7f91140cb3b0 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9112ffd700 1 -- 172.21.15.3:0/1955036196 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc00b7a0 con 0x7f91140cb3b0 2023-10-06T04:55:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f9112ffd700 1 -- 172.21.15.3:0/1955036196 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00bb40 con 0x7f91140cb3b0 2023-10-06T04:55:31.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f911924f700 1 -- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb3b0 msgr2=0x7f91140cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.132+0000 7f911924f700 1 --2- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb3b0 0x7f91140cf5c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0098e0 tx=0x7f90fc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.133+0000 7f911924f700 1 -- 172.21.15.3:0/1955036196 shutdown_connections 2023-10-06T04:55:31.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.133+0000 7f911924f700 1 --2- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb3b0 0x7f91140cf5c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.133+0000 7f911924f700 1 --2- 172.21.15.3:0/1955036196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb780 0x7f9114158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.133+0000 7f911924f700 1 -- 172.21.15.3:0/1955036196 >> 172.21.15.3:0/1955036196 conn(0x7f9114070310 msgr2=0x7f9114070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.133+0000 7f911924f700 1 -- 172.21.15.3:0/1955036196 shutdown_connections 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.133+0000 7f911924f700 1 -- 172.21.15.3:0/1955036196 wait complete. 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.134+0000 7f911924f700 1 Processor -- start 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.134+0000 7f911924f700 1 -- start start 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f911924f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb3b0 0x7f911413f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f911924f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb780 0x7f9114159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f911924f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9114159500 0x7f9114159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f911924f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911413f760 con 0x7f9114159500 2023-10-06T04:55:31.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f911924f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91140c87c0 con 0x7f91140cb780 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f911924f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91140c8aa0 con 0x7f91140cb3b0 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9114159500 0x7f9114159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9114159500 0x7f9114159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42912/0 (socket says 172.21.15.3:42912) 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 -- 172.21.15.3:0/2583546575 learned_addr learned my addr 172.21.15.3:0/2583546575 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f91137fe700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb780 0x7f9114159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9113fff700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb3b0 0x7f911413f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 -- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb3b0 msgr2=0x7f911413f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb3b0 0x7f911413f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 -- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb780 msgr2=0x7f9114159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb780 0x7f9114159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 -- 172.21.15.3:0/2583546575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91040094f0 con 0x7f9114159500 2023-10-06T04:55:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f9118a4e700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9114159500 0x7f9114159fb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f910c00ccc0 tx=0x7f910c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:31.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f910bfff700 1 -- 172.21.15.3:0/2583546575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c005df0 con 0x7f9114159500 2023-10-06T04:55:31.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f910bfff700 1 -- 172.21.15.3:0/2583546575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c00a600 con 0x7f9114159500 2023-10-06T04:55:31.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc009420 con 0x7f9114159500 2023-10-06T04:55:31.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.135+0000 7f910bfff700 1 -- 172.21.15.3:0/2583546575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00a9c0 con 0x7f9114159500 2023-10-06T04:55:31.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.136+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91140c9060 con 0x7f9114159500 2023-10-06T04:55:31.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.136+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911404c380 con 0x7f9114159500 2023-10-06T04:55:31.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.137+0000 7f910bfff700 1 -- 172.21.15.3:0/2583546575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f910c0070f0 con 0x7f9114159500 2023-10-06T04:55:31.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.137+0000 7f910bfff700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90f4066f30 0x7f90f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.137+0000 7f910bfff700 1 -- 172.21.15.3:0/2583546575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f910c084760 con 0x7f9114159500 2023-10-06T04:55:31.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.137+0000 7f9113fff700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90f4066f30 0x7f90f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.138+0000 7f9113fff700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90f4066f30 0x7f90f40693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0093f0 tx=0x7f90fc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:31.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.142+0000 7f910bfff700 1 -- 172.21.15.3:0/2583546575 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f910c0519a0 con 0x7f9114159500 2023-10-06T04:55:31.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.335+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91140cedc0 con 0x7f9114159500 2023-10-06T04:55:31.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.336+0000 7f910bfff700 1 -- 172.21.15.3:0/2583546575 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f910c054fc0 con 0x7f9114159500 2023-10-06T04:55:31.338 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:31.339 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":71,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:30.037151+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:55:27.781764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T04:55:29.007164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:55:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90f4066f30 msgr2=0x7f90f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90f4066f30 0x7f90f40693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0093f0 tx=0x7f90fc017000 comp rx=0 tx=0).stop 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9114159500 msgr2=0x7f9114159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9114159500 0x7f9114159fb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f910c00ccc0 tx=0x7f910c001a70 comp rx=0 tx=0).stop 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 shutdown_connections 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90f4066f30 0x7f90f40693e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9114159500 0x7f9114159fb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91140cb3b0 0x7f911413f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 --2- 172.21.15.3:0/2583546575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91140cb780 0x7f9114159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 >> 172.21.15.3:0/2583546575 conn(0x7f9114070310 msgr2=0x7f9114153750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 shutdown_connections 2023-10-06T04:55:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.338+0000 7f911924f700 1 -- 172.21.15.3:0/2583546575 wait complete. 2023-10-06T04:55:31.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:55:31.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.472+0000 7fe0d5064700 1 Processor -- start 2023-10-06T04:55:31.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.472+0000 7fe0d5064700 1 -- start start 2023-10-06T04:55:31.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0d5064700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0d0150690 0x7fe0d0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0d5064700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe0d015d540 0x7fe0d00af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0d5064700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d015a290 con 0x7fe0d00cba50 2023-10-06T04:55:31.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0d5064700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00d0020 con 0x7fe0d0150690 2023-10-06T04:55:31.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0d5064700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d0159f40 con 0x7fe0d015d540 2023-10-06T04:55:31.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0ced9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0d0150690 0x7fe0d0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0ce59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe0d00cba50 0x7fe0d015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39048/0 (socket says 172.21.15.3:39048) 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0ce59c700 1 -- 172.21.15.3:0/1185491575 learned_addr learned my addr 172.21.15.3:0/1185491575 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0ced9d700 1 -- 172.21.15.3:0/1185491575 >> v2:172.21.15.153:3300/0 conn(0x7fe0d015d540 msgr2=0x7fe0d00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0cf59e700 1 --2- 172.21.15.3:0/1185491575 >> v2:172.21.15.153:3300/0 conn(0x7fe0d015d540 0x7fe0d00af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0ced9d700 1 --2- 172.21.15.3:0/1185491575 >> v2:172.21.15.153:3300/0 conn(0x7fe0d015d540 0x7fe0d00af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0cdd9b700 1 -- 172.21.15.3:0/1185491575 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72278789 0 0) 0x7fe0d015a290 con 0x7fe0d00cba50 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0ced9d700 1 -- 172.21.15.3:0/1185491575 >> v1:172.21.15.3:6789/0 conn(0x7fe0d00cba50 legacy=0x7fe0d015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.473+0000 7fe0ced9d700 1 -- 172.21.15.3:0/1185491575 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d015dcb0 con 0x7fe0d0150690 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0ced9d700 1 --2- 172.21.15.3:0/1185491575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d0150a60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe0d015a200 tx=0x7fe0c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fd41843bbd311e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0cdd9b700 1 -- 172.21.15.3:0/1185491575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000e070 con 0x7fe0d0150690 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0cdd9b700 1 -- 172.21.15.3:0/1185491575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c000bc30 con 0x7fe0d0150690 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0cdd9b700 1 -- 172.21.15.3:0/1185491575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0012430 con 0x7fe0d0150690 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 -- 172.21.15.3:0/1185491575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 msgr2=0x7fe0d0150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1185491575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d0150a60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe0d015a200 tx=0x7fe0c0009780 comp rx=0 tx=0).stop 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 -- 172.21.15.3:0/1185491575 shutdown_connections 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1185491575 >> v2:172.21.15.153:3300/0 conn(0x7fe0d015d540 0x7fe0d00af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1185491575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d0150a60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 -- 172.21.15.3:0/1185491575 >> 172.21.15.3:0/1185491575 conn(0x7fe0d0070310 msgr2=0x7fe0d0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 -- 172.21.15.3:0/1185491575 shutdown_connections 2023-10-06T04:55:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.474+0000 7fe0d5064700 1 -- 172.21.15.3:0/1185491575 wait complete. 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 Processor -- start 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 -- start start 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0d00cba50 0x7fe0d01563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d015d540 0x7fe0d00b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0155f10 con 0x7fe0d0150690 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00b5520 con 0x7fe0d015d540 2023-10-06T04:55:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0d5064700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d01504a0 con 0x7fe0d00cba50 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.476+0000 7fe0ce59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d014ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42932/0 (socket says 172.21.15.3:42932) 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 -- 172.21.15.3:0/1951843363 learned_addr learned my addr 172.21.15.3:0/1951843363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ced9d700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0d00cba50 0x7fe0d01563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0cf59e700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d015d540 0x7fe0d00b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 -- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0d00cba50 msgr2=0x7fe0d01563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0d00cba50 0x7fe0d01563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 -- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d015d540 msgr2=0x7fe0d00b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d015d540 0x7fe0d00b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 -- 172.21.15.3:0/1951843363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0009a20 con 0x7fe0d0150690 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0ce59c700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d014ff60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800c7e0 tx=0x7fe0b8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0c6ffd700 1 -- 172.21.15.3:0/1951843363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8005b30 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0c6ffd700 1 -- 172.21.15.3:0/1951843363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b8005c90 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0c6ffd700 1 -- 172.21.15.3:0/1951843363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8007300 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d0155300 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.477+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d01558c0 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.478+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d01560c0 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.479+0000 7fe0c6ffd700 1 -- 172.21.15.3:0/1951843363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0b801b3b0 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.479+0000 7fe0c6ffd700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0b0066f30 0x7fe0b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.479+0000 7fe0c6ffd700 1 -- 172.21.15.3:0/1951843363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe0b8083bb0 con 0x7fe0d0150690 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.479+0000 7fe0ced9d700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0b0066f30 0x7fe0b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.480+0000 7fe0ced9d700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0b0066f30 0x7fe0b00693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe0d015a200 tx=0x7fe0c0019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.483+0000 7fe0c6ffd700 1 -- 172.21.15.3:0/1951843363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0d01560c0 con 0x7fe0d0150690 2023-10-06T04:55:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:31.722+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe0d0156050 con 0x7fe0d0150690 2023-10-06T04:55:32.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.639+0000 7fe0c6ffd700 1 -- 172.21.15.3:0/1951843363 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe0b8017020 con 0x7fe0d0150690 2023-10-06T04:55:32.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0b0066f30 msgr2=0x7fe0b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:32.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0b0066f30 0x7fe0b00693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe0d015a200 tx=0x7fe0c0019000 comp rx=0 tx=0).stop 2023-10-06T04:55:32.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 msgr2=0x7fe0d014ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:32.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d014ff60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800c7e0 tx=0x7fe0b8007df0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 shutdown_connections 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0b0066f30 0x7fe0b00693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d015d540 0x7fe0d00b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0d00cba50 0x7fe0d01563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.641+0000 7fe0d5064700 1 --2- 172.21.15.3:0/1951843363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0d0150690 0x7fe0d014ff60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.642+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 >> 172.21.15.3:0/1951843363 conn(0x7fe0d0070310 msgr2=0x7fe0d0157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.642+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 shutdown_connections 2023-10-06T04:55:32.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.642+0000 7fe0d5064700 1 -- 172.21.15.3:0/1951843363 wait complete. 2023-10-06T04:55:32.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:55:32.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.786+0000 7f538583a700 1 Processor -- start 2023-10-06T04:55:32.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.786+0000 7f538583a700 1 -- start start 2023-10-06T04:55:32.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.787+0000 7f538583a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f538006ca80 0x7f53801643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.787+0000 7f538583a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538015a960 con 0x7f5380153740 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.787+0000 7f538583a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5380159d50 con 0x7f5380153b10 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.787+0000 7f538583a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538015a030 con 0x7f538006ca80 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.787+0000 7f537effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5380153740 0x7f53800ac2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55988/0 (socket says 172.21.15.3:55988) 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.787+0000 7f537effd700 1 -- 172.21.15.3:0/4033727588 learned_addr learned my addr 172.21.15.3:0/4033727588 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236644698 0 0) 0x7f538015a960 con 0x7f5380153740 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f536c003570 con 0x7f5380153740 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537f7fe700 1 --2- 172.21.15.3:0/4033727588 >> v2:172.21.15.153:3300/0 conn(0x7f538006ca80 0x7f53801643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 489588581 0 0) 0x7f536c003570 con 0x7f5380153740 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 >> v2:172.21.15.153:3300/0 conn(0x7f538006ca80 msgr2=0x7f53801643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 --2- 172.21.15.3:0/4033727588 >> v2:172.21.15.153:3300/0 conn(0x7f538006ca80 0x7f53801643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 >> v1:172.21.15.153:6789/0 conn(0x7f5380153b10 legacy=0x7f538006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538015a310 con 0x7f5380153740 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f5378001470 con 0x7f5380153740 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 >> v1:172.21.15.3:6789/0 conn(0x7f5380153740 legacy=0x7f53800ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f536c003a60 0x7f536c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f536c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f536c0052e0 0x7f536c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538015a030 con 0x7f536c004810 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c00c230 con 0x7f536c003a60 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c00c3f0 con 0x7f536c0052e0 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537f7fe700 1 --2- 172.21.15.3:0/4033727588 >> v2:172.21.15.153:3300/0 conn(0x7f538006ca80 0x7f53801643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537e7fc700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f536c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537effd700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f536c003a60 0x7f536c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537f7fe700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f536c0052e0 0x7f536c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537e7fc700 1 -- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f536c0052e0 msgr2=0x7f536c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537e7fc700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f536c0052e0 0x7f536c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537e7fc700 1 -- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f536c003a60 msgr2=0x7f536c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537e7fc700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f536c003a60 0x7f536c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537e7fc700 1 -- 172.21.15.3:0/4033727588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c003570 con 0x7f536c004810 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537f7fe700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f536c0052e0 0x7f536c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.788+0000 7f537f7fe700 1 -- 172.21.15.3:0/4033727588 reap_dead start 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f537e7fc700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f536c004c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f53680018b0 tx=0x7f5368007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c3b63e9ad805f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368012420 con 0x7f536c004810 2023-10-06T04:55:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53680125a0 con 0x7f536c004810 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f537dffb700 1 -- 172.21.15.3:0/4033727588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368012960 con 0x7f536c004810 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f538583a700 1 -- 172.21.15.3:0/4033727588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 msgr2=0x7f536c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f538583a700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f536c004c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f53680018b0 tx=0x7f5368007d30 comp rx=0 tx=0).stop 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f538583a700 1 -- 172.21.15.3:0/4033727588 shutdown_connections 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f538583a700 1 --2- 172.21.15.3:0/4033727588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f536c004c80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f538583a700 1 -- 172.21.15.3:0/4033727588 >> 172.21.15.3:0/4033727588 conn(0x7f53800cdab0 msgr2=0x7f53800aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f538583a700 1 -- 172.21.15.3:0/4033727588 shutdown_connections 2023-10-06T04:55:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.789+0000 7f538583a700 1 -- 172.21.15.3:0/4033727588 wait complete. 2023-10-06T04:55:32.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.790+0000 7f538583a700 1 Processor -- start 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f538583a700 1 -- start start 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f538583a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f53800b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f538583a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f538014e980 0x7f5380160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f538583a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5380161160 0x7f53801405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f538583a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53800b2ec0 con 0x7f536c004810 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f538583a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538016d5d0 con 0x7f538014e980 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f538583a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538016d8b0 con 0x7f5380161160 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f537effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f53800b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f537effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f53800b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57916/0 (socket says 172.21.15.3:57916) 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.791+0000 7f537effd700 1 -- 172.21.15.3:0/952740598 learned_addr learned my addr 172.21.15.3:0/952740598 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537e7fc700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f538014e980 0x7f5380160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537f7fe700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5380161160 0x7f53801405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537effd700 1 -- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5380161160 msgr2=0x7f53801405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537effd700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5380161160 0x7f53801405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537effd700 1 -- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f538014e980 msgr2=0x7f5380160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537effd700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f538014e980 0x7f5380160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537effd700 1 -- 172.21.15.3:0/952740598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538016db90 con 0x7f536c004810 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f537effd700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f53800b2980 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5378007390 tx=0x7f5378001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f5376ffd700 1 -- 172.21.15.3:0/952740598 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378018030 con 0x7f536c004810 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f5376ffd700 1 -- 172.21.15.3:0/952740598 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537800f9a0 con 0x7f536c004810 2023-10-06T04:55:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f5376ffd700 1 -- 172.21.15.3:0/952740598 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537800fd40 con 0x7f536c004810 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538016de70 con 0x7f536c004810 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.792+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538016d280 con 0x7f536c004810 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.793+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538004c380 con 0x7f536c004810 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.794+0000 7f5376ffd700 1 -- 172.21.15.3:0/952740598 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f537801b070 con 0x7f536c004810 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.794+0000 7f5376ffd700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5360066f30 0x7f53600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.794+0000 7f5376ffd700 1 -- 172.21.15.3:0/952740598 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5378019070 con 0x7f536c004810 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.794+0000 7f537e7fc700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5360066f30 0x7f53600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.795+0000 7f537e7fc700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5360066f30 0x7f53600693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f53800af150 tx=0x7f5368015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:32.798+0000 7f5376ffd700 1 -- 172.21.15.3:0/952740598 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5378051740 con 0x7f536c004810 2023-10-06T04:55:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.037+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f538015a5f0 con 0x7f536c004810 2023-10-06T04:55:33.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.644+0000 7f5376ffd700 1 -- 172.21.15.3:0/952740598 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f537800fea0 con 0x7f536c004810 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5360066f30 msgr2=0x7f53600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5360066f30 0x7f53600693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f53800af150 tx=0x7f5368015000 comp rx=0 tx=0).stop 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 msgr2=0x7f53800b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f53800b2980 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5378007390 tx=0x7f5378001e50 comp rx=0 tx=0).stop 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 shutdown_connections 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5360066f30 0x7f53600693e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5380161160 0x7f53801405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f536c004810 0x7f53800b2980 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 --2- 172.21.15.3:0/952740598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f538014e980 0x7f5380160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 >> 172.21.15.3:0/952740598 conn(0x7f53800cdab0 msgr2=0x7f5380068040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:33.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 shutdown_connections 2023-10-06T04:55:33.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.646+0000 7f538583a700 1 -- 172.21.15.3:0/952740598 wait complete. 2023-10-06T04:55:33.662 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:33.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.782+0000 7f41e1945700 1 Processor -- start 2023-10-06T04:55:33.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.782+0000 7f41e1945700 1 -- start start 2023-10-06T04:55:33.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.782+0000 7f41e1945700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41dc15ccc0 0x7f41dc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.782+0000 7f41e1945700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc0d06a0 0x7f41dc0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.782+0000 7f41e1945700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc0cf6c0 con 0x7f41dc15c8f0 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.782+0000 7f41e1945700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc0cfb50 con 0x7f41dc15ccc0 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.782+0000 7f41e1945700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0d0ad0 con 0x7f41dc0d06a0 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41dc15ccc0 0x7f41dc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41dc15ccc0 0x7f41dc0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57930/0 (socket says 172.21.15.3:57930) 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 -- 172.21.15.3:0/3297278031 learned_addr learned my addr 172.21.15.3:0/3297278031 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41db7fe700 1 --2- 172.21.15.3:0/3297278031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc0d06a0 0x7f41dc0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 -- 172.21.15.3:0/3297278031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc0d06a0 msgr2=0x7f41dc0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 --2- 172.21.15.3:0/3297278031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc0d06a0 0x7f41dc0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 -- 172.21.15.3:0/3297278031 >> v1:172.21.15.153:6789/0 conn(0x7f41dc15c8f0 legacy=0x7f41dc15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 -- 172.21.15.3:0/3297278031 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0ceab0 con 0x7f41dc15ccc0 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41da7fc700 1 --2- 172.21.15.3:0/3297278031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0d0160 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41d00099b0 tx=0x7f41d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4091853322d9ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41d9ffb700 1 -- 172.21.15.3:0/3297278031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000b820 con 0x7f41dc15ccc0 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41d9ffb700 1 -- 172.21.15.3:0/3297278031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000b980 con 0x7f41dc15ccc0 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.783+0000 7f41d9ffb700 1 -- 172.21.15.3:0/3297278031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000bd20 con 0x7f41dc15ccc0 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 -- 172.21.15.3:0/3297278031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 msgr2=0x7f41dc0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 --2- 172.21.15.3:0/3297278031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0d0160 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41d00099b0 tx=0x7f41d0009250 comp rx=0 tx=0).stop 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 -- 172.21.15.3:0/3297278031 shutdown_connections 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 --2- 172.21.15.3:0/3297278031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc0d06a0 0x7f41dc0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 --2- 172.21.15.3:0/3297278031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0d0160 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 -- 172.21.15.3:0/3297278031 >> 172.21.15.3:0/3297278031 conn(0x7f41dc070310 msgr2=0x7f41dc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 -- 172.21.15.3:0/3297278031 shutdown_connections 2023-10-06T04:55:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.784+0000 7f41e1945700 1 -- 172.21.15.3:0/3297278031 wait complete. 2023-10-06T04:55:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.785+0000 7f41e1945700 1 Processor -- start 2023-10-06T04:55:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.785+0000 7f41e1945700 1 -- start start 2023-10-06T04:55:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41e1945700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41dc0d06a0 0x7f41dc158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41e1945700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc15c8f0 0x7f41dc0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41e1945700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41e1945700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc15aa60 con 0x7f41dc15ccc0 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41e1945700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc15b5c0 con 0x7f41dc0d06a0 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41e1945700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc159330 con 0x7f41dc15c8f0 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57944/0 (socket says 172.21.15.3:57944) 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 -- 172.21.15.3:0/1493075067 learned_addr learned my addr 172.21.15.3:0/1493075067 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41da7fc700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc15c8f0 0x7f41dc0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41daffd700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41dc0d06a0 0x7f41dc158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 -- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc15c8f0 msgr2=0x7f41dc0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc15c8f0 0x7f41dc0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 -- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41dc0d06a0 msgr2=0x7f41dc158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41dc0d06a0 0x7f41dc158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41db7fe700 1 -- 172.21.15.3:0/1493075067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00094f0 con 0x7f41dc15ccc0 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41da7fc700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc15c8f0 0x7f41dc0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.786+0000 7f41daffd700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41dc0d06a0 0x7f41dc158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.787+0000 7f41db7fe700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0b50b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009fa0 tx=0x7f41d4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.787+0000 7f41c2ffd700 1 -- 172.21.15.3:0/1493075067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400ed20 con 0x7f41dc15ccc0 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.787+0000 7f41c2ffd700 1 -- 172.21.15.3:0/1493075067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d400ee80 con 0x7f41dc15ccc0 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.787+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc15ac90 con 0x7f41dc15ccc0 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.787+0000 7f41c2ffd700 1 -- 172.21.15.3:0/1493075067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4014670 con 0x7f41dc15ccc0 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.787+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc15b250 con 0x7f41dc15ccc0 2023-10-06T04:55:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.788+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc15b730 con 0x7f41dc15ccc0 2023-10-06T04:55:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.788+0000 7f41c2ffd700 1 -- 172.21.15.3:0/1493075067 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f41d40149e0 con 0x7f41dc15ccc0 2023-10-06T04:55:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.789+0000 7f41c2ffd700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f41c8066f30 0x7f41c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.789+0000 7f41c2ffd700 1 -- 172.21.15.3:0/1493075067 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41d4084e90 con 0x7f41dc15ccc0 2023-10-06T04:55:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.789+0000 7f41daffd700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f41c8066f30 0x7f41c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.790+0000 7f41daffd700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f41c8066f30 0x7f41c80693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f41cc002880 tx=0x7f41cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:33.793+0000 7f41c2ffd700 1 -- 172.21.15.3:0/1493075067 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41d4051fa0 con 0x7f41dc15ccc0 2023-10-06T04:55:34.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.031+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41dc071680 con 0x7f41dc15ccc0 2023-10-06T04:55:34.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.031+0000 7f41c2ffd700 1 -- 172.21.15.3:0/1493075067 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f41d40555c0 con 0x7f41dc15ccc0 2023-10-06T04:55:34.034 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:34.034 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:33.639579+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:55:34.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f41c8066f30 msgr2=0x7f41c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f41c8066f30 0x7f41c80693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f41cc002880 tx=0x7f41cc009040 comp rx=0 tx=0).stop 2023-10-06T04:55:34.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 msgr2=0x7f41dc0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0b50b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009fa0 tx=0x7f41d4009900 comp rx=0 tx=0).stop 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 shutdown_connections 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f41c8066f30 0x7f41c80693e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41dc0d06a0 0x7f41dc158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41dc15ccc0 0x7f41dc0b50b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.033+0000 7f41e1945700 1 --2- 172.21.15.3:0/1493075067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41dc15c8f0 0x7f41dc0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.034+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 >> 172.21.15.3:0/1493075067 conn(0x7f41dc070310 msgr2=0x7f41dc0cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.034+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 shutdown_connections 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.034+0000 7f41e1945700 1 -- 172.21.15.3:0/1493075067 wait complete. 2023-10-06T04:55:34.037 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 40 2023-10-06T04:55:34.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:55:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.165+0000 7f2df94e2700 1 Processor -- start 2023-10-06T04:55:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.165+0000 7f2df94e2700 1 -- start start 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.165+0000 7f2df94e2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415c8f0 0x7f2df415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.165+0000 7f2df94e2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2df415ccc0 0x7f2df40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df94e2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40cf6c0 con 0x7f2df40d06a0 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df94e2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df40cfb50 con 0x7f2df415ccc0 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df94e2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df415e050 con 0x7f2df415c8f0 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2df415ccc0 0x7f2df40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2df40d06a0 0x7f2df406d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55994/0 (socket says 172.21.15.3:55994) 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df37fe700 1 -- 172.21.15.3:0/981900894 learned_addr learned my addr 172.21.15.3:0/981900894 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:34.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df2ffd700 1 --2- 172.21.15.3:0/981900894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415c8f0 0x7f2df415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df27fc700 1 -- 172.21.15.3:0/981900894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415c8f0 msgr2=0x7f2df415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df27fc700 1 --2- 172.21.15.3:0/981900894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415c8f0 0x7f2df415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df27fc700 1 -- 172.21.15.3:0/981900894 >> v1:172.21.15.3:6789/0 conn(0x7f2df40d06a0 legacy=0x7f2df406d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df27fc700 1 -- 172.21.15.3:0/981900894 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40d0af0 con 0x7f2df415ccc0 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df1ffb700 1 -- 172.21.15.3:0/981900894 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915610805 0 0) 0x7f2df40cf6c0 con 0x7f2df40d06a0 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df27fc700 1 --2- 172.21.15.3:0/981900894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415ccc0 0x7f2df40d0160 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0098e0 tx=0x7f2ddc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5dff6880e00729 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df1ffb700 1 -- 172.21.15.3:0/981900894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc009e80 con 0x7f2df415ccc0 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df1ffb700 1 -- 172.21.15.3:0/981900894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00b860 con 0x7f2df415ccc0 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.166+0000 7f2df1ffb700 1 -- 172.21.15.3:0/981900894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00bbe0 con 0x7f2df415ccc0 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 -- 172.21.15.3:0/981900894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415ccc0 msgr2=0x7f2df40d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 --2- 172.21.15.3:0/981900894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415ccc0 0x7f2df40d0160 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0098e0 tx=0x7f2ddc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 -- 172.21.15.3:0/981900894 shutdown_connections 2023-10-06T04:55:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 --2- 172.21.15.3:0/981900894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415c8f0 0x7f2df415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 --2- 172.21.15.3:0/981900894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415ccc0 0x7f2df40d0160 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 -- 172.21.15.3:0/981900894 >> 172.21.15.3:0/981900894 conn(0x7f2df4070310 msgr2=0x7f2df4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 -- 172.21.15.3:0/981900894 shutdown_connections 2023-10-06T04:55:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.167+0000 7f2df94e2700 1 -- 172.21.15.3:0/981900894 wait complete. 2023-10-06T04:55:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.168+0000 7f2df94e2700 1 Processor -- start 2023-10-06T04:55:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.168+0000 7f2df94e2700 1 -- start start 2023-10-06T04:55:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df94e2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2df40d06a0 0x7f2df41559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df94e2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415c8f0 0x7f2df40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df94e2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415ccc0 0x7f2df40b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df94e2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40c7900 con 0x7f2df415c8f0 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df94e2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40c53a0 con 0x7f2df40d06a0 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df94e2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40c5680 con 0x7f2df415ccc0 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415c8f0 0x7f2df40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415c8f0 0x7f2df40b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57962/0 (socket says 172.21.15.3:57962) 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 -- 172.21.15.3:0/2966204992 learned_addr learned my addr 172.21.15.3:0/2966204992 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df2ffd700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2df40d06a0 0x7f2df41559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df37fe700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415ccc0 0x7f2df40b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 -- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415ccc0 msgr2=0x7f2df40b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415ccc0 0x7f2df40b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 -- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2df40d06a0 msgr2=0x7f2df41559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2df40d06a0 0x7f2df41559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.169+0000 7f2df27fc700 1 -- 172.21.15.3:0/2966204992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc009420 con 0x7f2df415c8f0 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.170+0000 7f2df27fc700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415c8f0 0x7f2df40b0090 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc013000 tx=0x7f2ddc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.170+0000 7f2deaffd700 1 -- 172.21.15.3:0/2966204992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00b9d0 con 0x7f2df415c8f0 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.170+0000 7f2deaffd700 1 -- 172.21.15.3:0/2966204992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00bb30 con 0x7f2df415c8f0 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.170+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df40c5960 con 0x7f2df415c8f0 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.170+0000 7f2deaffd700 1 -- 172.21.15.3:0/2966204992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc025670 con 0x7f2df415c8f0 2023-10-06T04:55:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.170+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df4155ef0 con 0x7f2df415c8f0 2023-10-06T04:55:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.171+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df40c5e80 con 0x7f2df415c8f0 2023-10-06T04:55:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.171+0000 7f2deaffd700 1 -- 172.21.15.3:0/2966204992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ddc023070 con 0x7f2df415c8f0 2023-10-06T04:55:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.171+0000 7f2deaffd700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.172+0000 7f2deaffd700 1 -- 172.21.15.3:0/2966204992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ddc085b70 con 0x7f2df415c8f0 2023-10-06T04:55:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.172+0000 7f2df2ffd700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.172+0000 7f2df2ffd700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dd4066f30 0x7f2dd40693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.176+0000 7f2deaffd700 1 -- 172.21.15.3:0/2966204992 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2df40c5e80 con 0x7f2df415c8f0 2023-10-06T04:55:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.372+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2df40c5e80 con 0x7f2df415c8f0 2023-10-06T04:55:34.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.373+0000 7f2deaffd700 1 -- 172.21.15.3:0/2966204992 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f2df40c5e80 con 0x7f2df415c8f0 2023-10-06T04:55:34.375 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:34.376 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:31.630582+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:55:27.781764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T04:55:29.007164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:55:34.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.374+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dd4066f30 msgr2=0x7f2dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.374+0000 7f2df94e2700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dd4066f30 0x7f2dd40693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec008040 comp rx=0 tx=0).stop 2023-10-06T04:55:34.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.374+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415c8f0 msgr2=0x7f2df40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.374+0000 7f2df94e2700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415c8f0 0x7f2df40b0090 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc013000 tx=0x7f2ddc017000 comp rx=0 tx=0).stop 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 shutdown_connections 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2df40d06a0 0x7f2df41559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2df415ccc0 0x7f2df40b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 --2- 172.21.15.3:0/2966204992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df415c8f0 0x7f2df40b0090 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 >> 172.21.15.3:0/2966204992 conn(0x7f2df4070310 msgr2=0x7f2df40b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 shutdown_connections 2023-10-06T04:55:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.375+0000 7f2df94e2700 1 -- 172.21.15.3:0/2966204992 wait complete. 2023-10-06T04:55:34.390 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:55:34.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:55:34.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.507+0000 7f55c1b14700 1 Processor -- start 2023-10-06T04:55:34.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.507+0000 7f55c1b14700 1 -- start start 2023-10-06T04:55:34.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55c1b14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc15c8f0 0x7f55bc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55c1b14700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0cf6c0 con 0x7f55bc0d0870 2023-10-06T04:55:34.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55c1b14700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0cfb50 con 0x7f55bc15ccc0 2023-10-06T04:55:34.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55c1b14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc15e050 con 0x7f55bc15c8f0 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55bbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f55bc0d0870 0x7f55bc0b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56002/0 (socket says 172.21.15.3:56002) 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55bbfff700 1 -- 172.21.15.3:0/2356349971 learned_addr learned my addr 172.21.15.3:0/2356349971 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55bb7fe700 1 --2- 172.21.15.3:0/2356349971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc15c8f0 0x7f55bc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55ba7fc700 1 -- 172.21.15.3:0/2356349971 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410634887 0 0) 0x7f55bc0cf6c0 con 0x7f55bc0d0870 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.508+0000 7f55ba7fc700 1 -- 172.21.15.3:0/2356349971 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a4003570 con 0x7f55bc0d0870 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55ba7fc700 1 -- 172.21.15.3:0/2356349971 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980872849 0 0) 0x7f55bc0cfb50 con 0x7f55bc15ccc0 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55bb7fe700 1 -- 172.21.15.3:0/2356349971 >> v1:172.21.15.153:6789/0 conn(0x7f55bc15ccc0 legacy=0x7f55bc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55bb7fe700 1 -- 172.21.15.3:0/2356349971 >> v1:172.21.15.3:6789/0 conn(0x7f55bc0d0870 legacy=0x7f55bc0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55bb7fe700 1 -- 172.21.15.3:0/2356349971 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0ceab0 con 0x7f55bc15c8f0 2023-10-06T04:55:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55bb7fe700 1 --2- 172.21.15.3:0/2356349971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc15c8f0 0x7f55bc15db10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009910 tx=0x7f55ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f511eac3047f28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55ba7fc700 1 -- 172.21.15.3:0/2356349971 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00b680 con 0x7f55bc15c8f0 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55ba7fc700 1 -- 172.21.15.3:0/2356349971 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00b7e0 con 0x7f55bc15c8f0 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55ba7fc700 1 -- 172.21.15.3:0/2356349971 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00bba0 con 0x7f55bc15c8f0 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55c1b14700 1 -- 172.21.15.3:0/2356349971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc15c8f0 msgr2=0x7f55bc15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55c1b14700 1 --2- 172.21.15.3:0/2356349971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc15c8f0 0x7f55bc15db10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009910 tx=0x7f55ac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55c1b14700 1 -- 172.21.15.3:0/2356349971 shutdown_connections 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55c1b14700 1 --2- 172.21.15.3:0/2356349971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc15c8f0 0x7f55bc15db10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.509+0000 7f55c1b14700 1 -- 172.21.15.3:0/2356349971 >> 172.21.15.3:0/2356349971 conn(0x7f55bc070310 msgr2=0x7f55bc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.510+0000 7f55c1b14700 1 -- 172.21.15.3:0/2356349971 shutdown_connections 2023-10-06T04:55:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.510+0000 7f55c1b14700 1 -- 172.21.15.3:0/2356349971 wait complete. 2023-10-06T04:55:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.511+0000 7f55c1b14700 1 Processor -- start 2023-10-06T04:55:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.511+0000 7f55c1b14700 1 -- start start 2023-10-06T04:55:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc0d0870 0x7f55bc14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55bc15c8f0 0x7f55bc154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55bc15ccc0 0x7f55bc151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc150ca0 con 0x7f55bc15ccc0 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc150090 con 0x7f55bc15c8f0 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc150370 con 0x7f55bc0d0870 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55bc15ccc0 0x7f55bc151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55bc15ccc0 0x7f55bc151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57976/0 (socket says 172.21.15.3:57976) 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 -- 172.21.15.3:0/1519628949 learned_addr learned my addr 172.21.15.3:0/1519628949 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bb7fe700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc0d0870 0x7f55bc14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55baffd700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55bc15c8f0 0x7f55bc154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 -- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc0d0870 msgr2=0x7f55bc14fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc0d0870 0x7f55bc14fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 -- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55bc15c8f0 msgr2=0x7f55bc154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55bc15c8f0 0x7f55bc154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 -- 172.21.15.3:0/1519628949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac009420 con 0x7f55bc15ccc0 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55baffd700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55bc15c8f0 0x7f55bc154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bb7fe700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc0d0870 0x7f55bc14fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55bbfff700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55bc15ccc0 0x7f55bc151130 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f55b4001460 tx=0x7f55b4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55a37fe700 1 -- 172.21.15.3:0/1519628949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b400ce80 con 0x7f55bc15ccc0 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55a37fe700 1 -- 172.21.15.3:0/1519628949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b400ab70 con 0x7f55bc15ccc0 2023-10-06T04:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc150650 con 0x7f55bc15ccc0 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55a37fe700 1 -- 172.21.15.3:0/1519628949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40070f0 con 0x7f55bc15ccc0 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.512+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc1548a0 con 0x7f55bc15ccc0 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.513+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc071680 con 0x7f55bc15ccc0 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.514+0000 7f55a37fe700 1 -- 172.21.15.3:0/1519628949 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55b4007520 con 0x7f55bc15ccc0 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.514+0000 7f55a37fe700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f55a8066f30 0x7f55a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.514+0000 7f55a37fe700 1 -- 172.21.15.3:0/1519628949 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55b4084c00 con 0x7f55bc15ccc0 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.514+0000 7f55bb7fe700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f55a8066f30 0x7f55a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.515+0000 7f55bb7fe700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f55a8066f30 0x7f55a80693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009860 tx=0x7f55ac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.518+0000 7f55a37fe700 1 -- 172.21.15.3:0/1519628949 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55b4051cf0 con 0x7f55bc15ccc0 2023-10-06T04:55:34.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.710+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55bc150de0 con 0x7f55a8066f30 2023-10-06T04:55:34.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.720+0000 7f55a37fe700 1 -- 172.21.15.3:0/1519628949 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308767 (secure 0 0 0) 0x7f55bc150de0 con 0x7f55a8066f30 2023-10-06T04:55:34.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.722+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f55a8066f30 msgr2=0x7f55a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.722+0000 7f55c1b14700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f55a8066f30 0x7f55a80693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009860 tx=0x7f55ac017000 comp rx=0 tx=0).stop 2023-10-06T04:55:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.722+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55bc15ccc0 msgr2=0x7f55bc151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.722+0000 7f55c1b14700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55bc15ccc0 0x7f55bc151130 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f55b4001460 tx=0x7f55b4001ae0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 shutdown_connections 2023-10-06T04:55:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f55a8066f30 0x7f55a80693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55bc15ccc0 0x7f55bc151130 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55bc0d0870 0x7f55bc14fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 --2- 172.21.15.3:0/1519628949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55bc15c8f0 0x7f55bc154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 >> 172.21.15.3:0/1519628949 conn(0x7f55bc070310 msgr2=0x7f55bc1566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 shutdown_connections 2023-10-06T04:55:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.723+0000 7f55c1b14700 1 -- 172.21.15.3:0/1519628949 wait complete. 2023-10-06T04:55:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:55:34.737 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2023-10-06T04:55:33.469652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2435248,"kb_used_data":7792,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539472,"statfs":{"total":773094113280,"available":770600419328,"internally_reserved":0,"allocated":7979008,"data_stored":4185933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.848173"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.635164+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634027+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630141+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:47.223457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630326+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:32.184584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.639206+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:09.194524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:54.819595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632371+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:26.055360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632057+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:14.777663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625915+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.269374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:36.692184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629995+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:47.654687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629843+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:31.939182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631919+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:03.823215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631745+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632536+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633013+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:29.108450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629698+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:45.147340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628196+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:18.527942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634491+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:29.940238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627719+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:45.969663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633388+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:10.209749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626849+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:34.163179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629948+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:48.336061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632213+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:11.480851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631083+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:28.732154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638569+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:16.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634028+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:18.240347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633198+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630555+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.772918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632025+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631841+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644855+0000","last_change":"2023-10-06T04:55:29.644855+0000","last_active":"2023-10-06T04:55:29.644855+0000","last_peered":"2023-10-06T04:55:29.644855+0000","last_clean":"2023-10-06T04:55:29.644855+0000","last_became_active":"2023-10-06T04:55:29.644553+0000","last_became_peered":"2023-10-06T04:55:29.644553+0000","last_unstale":"2023-10-06T04:55:29.644855+0000","last_undegraded":"2023-10-06T04:55:29.644855+0000","last_fullsized":"2023-10-06T04:55:29.644855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:31.518507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.640284+0000","last_change":"2023-10-06T04:55:29.640284+0000","last_active":"2023-10-06T04:55:29.640284+0000","last_peered":"2023-10-06T04:55:29.640284+0000","last_clean":"2023-10-06T04:55:29.640284+0000","last_became_active":"2023-10-06T04:55:29.638752+0000","last_became_peered":"2023-10-06T04:55:29.638752+0000","last_unstale":"2023-10-06T04:55:29.640284+0000","last_undegraded":"2023-10-06T04:55:29.640284+0000","last_fullsized":"2023-10-06T04:55:29.640284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.325782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626413+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:57.205791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.646265+0000","last_change":"2023-10-06T04:55:29.646265+0000","last_active":"2023-10-06T04:55:29.646265+0000","last_peered":"2023-10-06T04:55:29.646265+0000","last_clean":"2023-10-06T04:55:29.646265+0000","last_became_active":"2023-10-06T04:55:29.645786+0000","last_became_peered":"2023-10-06T04:55:29.645786+0000","last_unstale":"2023-10-06T04:55:29.646265+0000","last_undegraded":"2023-10-06T04:55:29.646265+0000","last_fullsized":"2023-10-06T04:55:29.646265+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:11.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627157+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:48.932063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643149+0000","last_change":"2023-10-06T04:55:29.643149+0000","last_active":"2023-10-06T04:55:29.643149+0000","last_peered":"2023-10-06T04:55:29.643149+0000","last_clean":"2023-10-06T04:55:29.643149+0000","last_became_active":"2023-10-06T04:55:29.642865+0000","last_became_peered":"2023-10-06T04:55:29.642865+0000","last_unstale":"2023-10-06T04:55:29.643149+0000","last_undegraded":"2023-10-06T04:55:29.643149+0000","last_fullsized":"2023-10-06T04:55:29.643149+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:25.603703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630411+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:12.293583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641637+0000","last_became_peered":"2023-10-06T04:55:29.641637+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:44.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630470+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:44.776671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642590+0000","last_change":"2023-10-06T04:55:29.642590+0000","last_active":"2023-10-06T04:55:29.642590+0000","last_peered":"2023-10-06T04:55:29.642590+0000","last_clean":"2023-10-06T04:55:29.642590+0000","last_became_active":"2023-10-06T04:55:29.642252+0000","last_became_peered":"2023-10-06T04:55:29.642252+0000","last_unstale":"2023-10-06T04:55:29.642590+0000","last_undegraded":"2023-10-06T04:55:29.642590+0000","last_fullsized":"2023-10-06T04:55:29.642590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:56.195511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626510+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:20.849108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643163+0000","last_change":"2023-10-06T04:55:29.643163+0000","last_active":"2023-10-06T04:55:29.643163+0000","last_peered":"2023-10-06T04:55:29.643163+0000","last_clean":"2023-10-06T04:55:29.643163+0000","last_became_active":"2023-10-06T04:55:29.642910+0000","last_became_peered":"2023-10-06T04:55:29.642910+0000","last_unstale":"2023-10-06T04:55:29.643163+0000","last_undegraded":"2023-10-06T04:55:29.643163+0000","last_fullsized":"2023-10-06T04:55:29.643163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:35.681905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631199+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:05.450563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641803+0000","last_change":"2023-10-06T04:55:29.641803+0000","last_active":"2023-10-06T04:55:29.641803+0000","last_peered":"2023-10-06T04:55:29.641803+0000","last_clean":"2023-10-06T04:55:29.641803+0000","last_became_active":"2023-10-06T04:55:29.641484+0000","last_became_peered":"2023-10-06T04:55:29.641484+0000","last_unstale":"2023-10-06T04:55:29.641803+0000","last_undegraded":"2023-10-06T04:55:29.641803+0000","last_fullsized":"2023-10-06T04:55:29.641803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.831547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633050+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:34.127918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.645173+0000","last_change":"2023-10-06T04:55:29.645173+0000","last_active":"2023-10-06T04:55:29.645173+0000","last_peered":"2023-10-06T04:55:29.645173+0000","last_clean":"2023-10-06T04:55:29.645173+0000","last_became_active":"2023-10-06T04:55:29.644899+0000","last_became_peered":"2023-10-06T04:55:29.644899+0000","last_unstale":"2023-10-06T04:55:29.645173+0000","last_undegraded":"2023-10-06T04:55:29.645173+0000","last_fullsized":"2023-10-06T04:55:29.645173+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:35.447530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628867+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:52.846620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639731+0000","last_change":"2023-10-06T04:55:29.639731+0000","last_active":"2023-10-06T04:55:29.639731+0000","last_peered":"2023-10-06T04:55:29.639731+0000","last_clean":"2023-10-06T04:55:29.639731+0000","last_became_active":"2023-10-06T04:55:29.639060+0000","last_became_peered":"2023-10-06T04:55:29.639060+0000","last_unstale":"2023-10-06T04:55:29.639731+0000","last_undegraded":"2023-10-06T04:55:29.639731+0000","last_fullsized":"2023-10-06T04:55:29.639731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:19.838829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629921+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:03.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643771+0000","last_change":"2023-10-06T04:55:29.643771+0000","last_active":"2023-10-06T04:55:29.643771+0000","last_peered":"2023-10-06T04:55:29.643771+0000","last_clean":"2023-10-06T04:55:29.643771+0000","last_became_active":"2023-10-06T04:55:29.643494+0000","last_became_peered":"2023-10-06T04:55:29.643494+0000","last_unstale":"2023-10-06T04:55:29.643771+0000","last_undegraded":"2023-10-06T04:55:29.643771+0000","last_fullsized":"2023-10-06T04:55:29.643771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:14.660406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629728+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:59.757486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636609+0000","last_change":"2023-10-06T04:55:29.636609+0000","last_active":"2023-10-06T04:55:29.636609+0000","last_peered":"2023-10-06T04:55:29.636609+0000","last_clean":"2023-10-06T04:55:29.636609+0000","last_became_active":"2023-10-06T04:55:29.636160+0000","last_became_peered":"2023-10-06T04:55:29.636160+0000","last_unstale":"2023-10-06T04:55:29.636609+0000","last_undegraded":"2023-10-06T04:55:29.636609+0000","last_fullsized":"2023-10-06T04:55:29.636609+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:32.519067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629380+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:31.703554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634753+0000","last_change":"2023-10-06T04:55:29.634753+0000","last_active":"2023-10-06T04:55:29.634753+0000","last_peered":"2023-10-06T04:55:29.634753+0000","last_clean":"2023-10-06T04:55:29.634753+0000","last_became_active":"2023-10-06T04:55:29.634471+0000","last_became_peered":"2023-10-06T04:55:29.634471+0000","last_unstale":"2023-10-06T04:55:29.634753+0000","last_undegraded":"2023-10-06T04:55:29.634753+0000","last_fullsized":"2023-10-06T04:55:29.634753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:47.921784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629562+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:38.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644259+0000","last_change":"2023-10-06T04:55:29.644259+0000","last_active":"2023-10-06T04:55:29.644259+0000","last_peered":"2023-10-06T04:55:29.644259+0000","last_clean":"2023-10-06T04:55:29.644259+0000","last_became_active":"2023-10-06T04:55:29.643992+0000","last_became_peered":"2023-10-06T04:55:29.643992+0000","last_unstale":"2023-10-06T04:55:29.644259+0000","last_undegraded":"2023-10-06T04:55:29.644259+0000","last_fullsized":"2023-10-06T04:55:29.644259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:43.348964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629557+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:34.367793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644987+0000","last_change":"2023-10-06T04:55:29.644987+0000","last_active":"2023-10-06T04:55:29.644987+0000","last_peered":"2023-10-06T04:55:29.644987+0000","last_clean":"2023-10-06T04:55:29.644987+0000","last_became_active":"2023-10-06T04:55:29.644620+0000","last_became_peered":"2023-10-06T04:55:29.644620+0000","last_unstale":"2023-10-06T04:55:29.644987+0000","last_undegraded":"2023-10-06T04:55:29.644987+0000","last_fullsized":"2023-10-06T04:55:29.644987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:57.789632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630035+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:38.834002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637724+0000","last_change":"2023-10-06T04:55:29.637724+0000","last_active":"2023-10-06T04:55:29.637724+0000","last_peered":"2023-10-06T04:55:29.637724+0000","last_clean":"2023-10-06T04:55:29.637724+0000","last_became_active":"2023-10-06T04:55:29.637235+0000","last_became_peered":"2023-10-06T04:55:29.637235+0000","last_unstale":"2023-10-06T04:55:29.637724+0000","last_undegraded":"2023-10-06T04:55:29.637724+0000","last_fullsized":"2023-10-06T04:55:29.637724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:27.721875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637907+0000","last_change":"2023-10-06T04:55:29.637907+0000","last_active":"2023-10-06T04:55:29.637907+0000","last_peered":"2023-10-06T04:55:29.637907+0000","last_clean":"2023-10-06T04:55:29.637907+0000","last_became_active":"2023-10-06T04:55:29.637626+0000","last_became_peered":"2023-10-06T04:55:29.637626+0000","last_unstale":"2023-10-06T04:55:29.637907+0000","last_undegraded":"2023-10-06T04:55:29.637907+0000","last_fullsized":"2023-10-06T04:55:29.637907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:44.959384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630578+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:39.999464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641635+0000","last_change":"2023-10-06T04:55:29.641635+0000","last_active":"2023-10-06T04:55:29.641635+0000","last_peered":"2023-10-06T04:55:29.641635+0000","last_clean":"2023-10-06T04:55:29.641635+0000","last_became_active":"2023-10-06T04:55:29.641370+0000","last_became_peered":"2023-10-06T04:55:29.641370+0000","last_unstale":"2023-10-06T04:55:29.641635+0000","last_undegraded":"2023-10-06T04:55:29.641635+0000","last_fullsized":"2023-10-06T04:55:29.641635+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625606+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:06.360553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644387+0000","last_change":"2023-10-06T04:55:29.644387+0000","last_active":"2023-10-06T04:55:29.644387+0000","last_peered":"2023-10-06T04:55:29.644387+0000","last_clean":"2023-10-06T04:55:29.644387+0000","last_became_active":"2023-10-06T04:55:29.644059+0000","last_became_peered":"2023-10-06T04:55:29.644059+0000","last_unstale":"2023-10-06T04:55:29.644387+0000","last_undegraded":"2023-10-06T04:55:29.644387+0000","last_fullsized":"2023-10-06T04:55:29.644387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631471+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644897+0000","last_change":"2023-10-06T04:55:29.644897+0000","last_active":"2023-10-06T04:55:29.644897+0000","last_peered":"2023-10-06T04:55:29.644897+0000","last_clean":"2023-10-06T04:55:29.644897+0000","last_became_active":"2023-10-06T04:55:29.644631+0000","last_became_peered":"2023-10-06T04:55:29.644631+0000","last_unstale":"2023-10-06T04:55:29.644897+0000","last_undegraded":"2023-10-06T04:55:29.644897+0000","last_fullsized":"2023-10-06T04:55:29.644897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630023+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:37.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642477+0000","last_change":"2023-10-06T04:55:29.642477+0000","last_active":"2023-10-06T04:55:29.642477+0000","last_peered":"2023-10-06T04:55:29.642477+0000","last_clean":"2023-10-06T04:55:29.642477+0000","last_became_active":"2023-10-06T04:55:29.642202+0000","last_became_peered":"2023-10-06T04:55:29.642202+0000","last_unstale":"2023-10-06T04:55:29.642477+0000","last_undegraded":"2023-10-06T04:55:29.642477+0000","last_fullsized":"2023-10-06T04:55:29.642477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:14.338401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629001+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:26.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644875+0000","last_change":"2023-10-06T04:55:29.644875+0000","last_active":"2023-10-06T04:55:29.644875+0000","last_peered":"2023-10-06T04:55:29.644875+0000","last_clean":"2023-10-06T04:55:29.644875+0000","last_became_active":"2023-10-06T04:55:29.644568+0000","last_became_peered":"2023-10-06T04:55:29.644568+0000","last_unstale":"2023-10-06T04:55:29.644875+0000","last_undegraded":"2023-10-06T04:55:29.644875+0000","last_fullsized":"2023-10-06T04:55:29.644875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:08.690122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629031+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.263825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641632+0000","last_change":"2023-10-06T04:55:29.641632+0000","last_active":"2023-10-06T04:55:29.641632+0000","last_peered":"2023-10-06T04:55:29.641632+0000","last_clean":"2023-10-06T04:55:29.641632+0000","last_became_active":"2023-10-06T04:55:29.641299+0000","last_became_peered":"2023-10-06T04:55:29.641299+0000","last_unstale":"2023-10-06T04:55:29.641632+0000","last_undegraded":"2023-10-06T04:55:29.641632+0000","last_fullsized":"2023-10-06T04:55:29.641632+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:42.695331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629260+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:06.628478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634678+0000","last_change":"2023-10-06T04:55:29.634678+0000","last_active":"2023-10-06T04:55:29.634678+0000","last_peered":"2023-10-06T04:55:29.634678+0000","last_clean":"2023-10-06T04:55:29.634678+0000","last_became_active":"2023-10-06T04:55:29.634400+0000","last_became_peered":"2023-10-06T04:55:29.634400+0000","last_unstale":"2023-10-06T04:55:29.634678+0000","last_undegraded":"2023-10-06T04:55:29.634678+0000","last_fullsized":"2023-10-06T04:55:29.634678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:04.440283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628124+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:38.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644649+0000","last_change":"2023-10-06T04:55:29.644649+0000","last_active":"2023-10-06T04:55:29.644649+0000","last_peered":"2023-10-06T04:55:29.644649+0000","last_clean":"2023-10-06T04:55:29.644649+0000","last_became_active":"2023-10-06T04:55:29.644333+0000","last_became_peered":"2023-10-06T04:55:29.644333+0000","last_unstale":"2023-10-06T04:55:29.644649+0000","last_undegraded":"2023-10-06T04:55:29.644649+0000","last_fullsized":"2023-10-06T04:55:29.644649+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:36.941087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620333+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:55:28.620333+0000","last_peered":"2023-10-06T04:55:28.620333+0000","last_clean":"2023-10-06T04:55:28.620333+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:55:28.620333+0000","last_undegraded":"2023-10-06T04:55:28.620333+0000","last_fullsized":"2023-10-06T04:55:28.620333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631507+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:37.433675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.628693+0000","last_change":"2023-10-06T04:55:29.628693+0000","last_active":"2023-10-06T04:55:29.628693+0000","last_peered":"2023-10-06T04:55:29.628693+0000","last_clean":"2023-10-06T04:55:29.628693+0000","last_became_active":"2023-10-06T04:55:29.627557+0000","last_became_peered":"2023-10-06T04:55:29.627557+0000","last_unstale":"2023-10-06T04:55:29.628693+0000","last_undegraded":"2023-10-06T04:55:29.628693+0000","last_fullsized":"2023-10-06T04:55:29.628693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:58.747206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.617715+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:55:28.617715+0000","last_peered":"2023-10-06T04:55:28.617715+0000","last_clean":"2023-10-06T04:55:28.617715+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:55:28.617715+0000","last_undegraded":"2023-10-06T04:55:28.617715+0000","last_fullsized":"2023-10-06T04:55:28.617715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632435+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.568423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636254+0000","last_change":"2023-10-06T04:55:29.636254+0000","last_active":"2023-10-06T04:55:29.636254+0000","last_peered":"2023-10-06T04:55:29.636254+0000","last_clean":"2023-10-06T04:55:29.636254+0000","last_became_active":"2023-10-06T04:55:29.635966+0000","last_became_peered":"2023-10-06T04:55:29.635966+0000","last_unstale":"2023-10-06T04:55:29.636254+0000","last_undegraded":"2023-10-06T04:55:29.636254+0000","last_fullsized":"2023-10-06T04:55:29.636254+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:09.863401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.622100+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:55:28.622100+0000","last_peered":"2023-10-06T04:55:28.622100+0000","last_clean":"2023-10-06T04:55:28.622100+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:55:28.622100+0000","last_undegraded":"2023-10-06T04:55:28.622100+0000","last_fullsized":"2023-10-06T04:55:28.622100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":202,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623115+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:55:28.623115+0000","last_peered":"2023-10-06T04:55:28.623115+0000","last_clean":"2023-10-06T04:55:28.623115+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:55:28.623115+0000","last_undegraded":"2023-10-06T04:55:28.623115+0000","last_fullsized":"2023-10-06T04:55:28.623115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630556+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:25.364931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634448+0000","last_change":"2023-10-06T04:55:29.634448+0000","last_active":"2023-10-06T04:55:29.634448+0000","last_peered":"2023-10-06T04:55:29.634448+0000","last_clean":"2023-10-06T04:55:29.634448+0000","last_became_active":"2023-10-06T04:55:29.634144+0000","last_became_peered":"2023-10-06T04:55:29.634144+0000","last_unstale":"2023-10-06T04:55:29.634448+0000","last_undegraded":"2023-10-06T04:55:29.634448+0000","last_fullsized":"2023-10-06T04:55:29.634448+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:14.628720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623209+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:55:28.623209+0000","last_peered":"2023-10-06T04:55:28.623209+0000","last_clean":"2023-10-06T04:55:28.623209+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:55:28.623209+0000","last_undegraded":"2023-10-06T04:55:28.623209+0000","last_fullsized":"2023-10-06T04:55:28.623209+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638091+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637914+0000","last_change":"2023-10-06T04:55:29.637914+0000","last_active":"2023-10-06T04:55:29.637914+0000","last_peered":"2023-10-06T04:55:29.637914+0000","last_clean":"2023-10-06T04:55:29.637914+0000","last_became_active":"2023-10-06T04:55:29.637640+0000","last_became_peered":"2023-10-06T04:55:29.637640+0000","last_unstale":"2023-10-06T04:55:29.637914+0000","last_undegraded":"2023-10-06T04:55:29.637914+0000","last_fullsized":"2023-10-06T04:55:29.637914+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:15.026328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623208+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:55:28.623208+0000","last_peered":"2023-10-06T04:55:28.623208+0000","last_clean":"2023-10-06T04:55:28.623208+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:55:28.623208+0000","last_undegraded":"2023-10-06T04:55:28.623208+0000","last_fullsized":"2023-10-06T04:55:28.623208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629855+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:36.457810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641718+0000","last_change":"2023-10-06T04:55:29.641718+0000","last_active":"2023-10-06T04:55:29.641718+0000","last_peered":"2023-10-06T04:55:29.641718+0000","last_clean":"2023-10-06T04:55:29.641718+0000","last_became_active":"2023-10-06T04:55:29.641423+0000","last_became_peered":"2023-10-06T04:55:29.641423+0000","last_unstale":"2023-10-06T04:55:29.641718+0000","last_undegraded":"2023-10-06T04:55:29.641718+0000","last_fullsized":"2023-10-06T04:55:29.641718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:26.283304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.624774+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:55:28.624774+0000","last_peered":"2023-10-06T04:55:28.624774+0000","last_clean":"2023-10-06T04:55:28.624774+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:55:28.624774+0000","last_undegraded":"2023-10-06T04:55:28.624774+0000","last_fullsized":"2023-10-06T04:55:28.624774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628576+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.902269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639321+0000","last_change":"2023-10-06T04:55:29.639321+0000","last_active":"2023-10-06T04:55:29.639321+0000","last_peered":"2023-10-06T04:55:29.639321+0000","last_clean":"2023-10-06T04:55:29.639321+0000","last_became_active":"2023-10-06T04:55:29.639057+0000","last_became_peered":"2023-10-06T04:55:29.639057+0000","last_unstale":"2023-10-06T04:55:29.639321+0000","last_undegraded":"2023-10-06T04:55:29.639321+0000","last_fullsized":"2023-10-06T04:55:29.639321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.045081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620654+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:55:28.620654+0000","last_peered":"2023-10-06T04:55:28.620654+0000","last_clean":"2023-10-06T04:55:28.620654+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:55:28.620654+0000","last_undegraded":"2023-10-06T04:55:28.620654+0000","last_fullsized":"2023-10-06T04:55:28.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632257+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:01.492662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641636+0000","last_became_peered":"2023-10-06T04:55:29.641636+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:29.769932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620664+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:55:28.620664+0000","last_peered":"2023-10-06T04:55:28.620664+0000","last_clean":"2023-10-06T04:55:28.620664+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:55:28.620664+0000","last_undegraded":"2023-10-06T04:55:28.620664+0000","last_fullsized":"2023-10-06T04:55:28.620664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.70499999999999996,"15min":0.77600000000000002},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88700000000000001,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.751,"15min":0.82799999999999996},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.90500000000000003,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":1.073,"15min":1.073},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81799999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.111,"15min":1.111},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78900000000000003,"5min":0.874,"15min":0.874},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":1.214,"15min":1.214},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.497},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.38,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60399999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.876,"15min":0.876},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63200000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.60399999999999998},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35499999999999998,"15min":0.36299999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.37,"15min":0.38},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.29899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.60699999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.64300000000000002,"15min":0.66700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63100000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85299999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.026,"15min":1.026},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.85599999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.34699999999999998,"15min":0.35499999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.373},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36699999999999999},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60699999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.624,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64400000000000002},"min":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.19500000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.54100000000000004,"5min":1.048,"15min":1.048},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.376},"min":{"1min":0.28699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.624,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.29599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69799999999999995,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82299999999999995,"5min":1.246,"15min":1.246},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.877,"15min":0.877},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89400000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.624},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306312,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065528,"statfs":{"total":96636764160,"available":96323100672,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.40999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.54800000000000004,"15min":0.60999999999999999},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.1480000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.53000000000000003,"15min":0.58599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.379,"5min":5.069,"15min":5.069},"last":0.435},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.49299999999999999},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.86099999999999999,"15min":0.83799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":14.646000000000001,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.2769999999999999,"5min":0.81799999999999995,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":14.606999999999999,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.82299999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.9219999999999999,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74199999999999999,"15min":0.79300000000000004},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4079999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76900000000000002,"15min":0.83899999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.964,"5min":6.367,"15min":6.367},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.45900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3,"5min":5,"15min":5},"last":0.71599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.313,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.752,"15min":0.85399999999999998},"min":{"1min":0.438,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82299999999999995,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.77900000000000003,"15min":0.879},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.78600000000000003,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.76200000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.70099999999999996},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80400000000000005,"5min":5.415,"15min":5.415},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65000000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":4.202,"15min":4.202},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81799999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69999999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.872,"15min":0.872},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306296,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065544,"statfs":{"total":96636764160,"available":96323117056,"internally_reserved":0,"allocated":835584,"data_stored":369015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.65400000000000003,"15min":0.71899999999999997},"min":{"1min":0.26300000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73099999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67800000000000005,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69299999999999995,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.65900000000000003,"15min":0.70499999999999996},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.81799999999999995,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63900000000000001,"15min":0.68700000000000006},"min":{"1min":0.373,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83899999999999997,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.89500000000000002,"15min":1.044},"min":{"1min":0.39500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77300000000000002,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.90500000000000003,"15min":1.0409999999999999},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72399999999999998,"5min":0.751,"15min":0.751},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55800000000000005,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:55:34.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:55:34.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.894+0000 7f459ffc1700 1 Processor -- start 2023-10-06T04:55:34.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.894+0000 7f459ffc1700 1 -- start start 2023-10-06T04:55:34.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459ffc1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f459815ccc0 0x7f45980d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459ffc1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f45980d06a0 0x7f45980b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459ffc1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45980cf6c0 con 0x7f459815c8f0 2023-10-06T04:55:34.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459ffc1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45980cfb50 con 0x7f45980d06a0 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459ffc1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45980d0b20 con 0x7f459815ccc0 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459dd5d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f459815c8f0 0x7f459815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56012/0 (socket says 172.21.15.3:56012) 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459dd5d700 1 -- 172.21.15.3:0/2658422017 learned_addr learned my addr 172.21.15.3:0/2658422017 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459e55e700 1 --2- 172.21.15.3:0/2658422017 >> v2:172.21.15.3:3300/0 conn(0x7f45980d06a0 0x7f45980b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.895+0000 7f459d55c700 1 --2- 172.21.15.3:0/2658422017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f459815ccc0 0x7f45980d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459cd5b700 1 -- 172.21.15.3:0/2658422017 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167506016 0 0) 0x7f45980cf6c0 con 0x7f459815c8f0 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459cd5b700 1 -- 172.21.15.3:0/2658422017 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4580003570 con 0x7f459815c8f0 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459e55e700 1 -- 172.21.15.3:0/2658422017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f459815ccc0 msgr2=0x7f45980d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459e55e700 1 --2- 172.21.15.3:0/2658422017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f459815ccc0 0x7f45980d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459e55e700 1 -- 172.21.15.3:0/2658422017 >> v1:172.21.15.3:6789/0 conn(0x7f459815c8f0 legacy=0x7f459815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459e55e700 1 -- 172.21.15.3:0/2658422017 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45980ceab0 con 0x7f45980d06a0 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459e55e700 1 --2- 172.21.15.3:0/2658422017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45980d06a0 0x7f45980b3ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f45900098e0 tx=0x7f45900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1de7e7cc5122a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459cd5b700 1 -- 172.21.15.3:0/2658422017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590011630 con 0x7f45980d06a0 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459cd5b700 1 -- 172.21.15.3:0/2658422017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4590011790 con 0x7f45980d06a0 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.896+0000 7f459cd5b700 1 -- 172.21.15.3:0/2658422017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590011b30 con 0x7f45980d06a0 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 -- 172.21.15.3:0/2658422017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45980d06a0 msgr2=0x7f45980b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2658422017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45980d06a0 0x7f45980b3ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f45900098e0 tx=0x7f45900091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 -- 172.21.15.3:0/2658422017 shutdown_connections 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2658422017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45980d06a0 0x7f45980b3ba0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2658422017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f459815ccc0 0x7f45980d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 -- 172.21.15.3:0/2658422017 >> 172.21.15.3:0/2658422017 conn(0x7f4598070310 msgr2=0x7f4598070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 -- 172.21.15.3:0/2658422017 shutdown_connections 2023-10-06T04:55:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.897+0000 7f459ffc1700 1 -- 172.21.15.3:0/2658422017 wait complete. 2023-10-06T04:55:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.898+0000 7f459ffc1700 1 Processor -- start 2023-10-06T04:55:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.898+0000 7f459ffc1700 1 -- start start 2023-10-06T04:55:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459ffc1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45980d06a0 0x7f4598141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459ffc1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f459815c8f0 0x7f4598155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459ffc1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459815ccc0 0x7f45981578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459ffc1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459815a6b0 con 0x7f459815ccc0 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459ffc1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598159aa0 con 0x7f459815c8f0 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459ffc1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4598159d80 con 0x7f45980d06a0 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459815ccc0 0x7f45981578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459815ccc0 0x7f45981578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58002/0 (socket says 172.21.15.3:58002) 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 -- 172.21.15.3:0/2522100272 learned_addr learned my addr 172.21.15.3:0/2522100272 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459dd5d700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45980d06a0 0x7f4598141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459d55c700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f459815c8f0 0x7f4598155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 -- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45980d06a0 msgr2=0x7f4598141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45980d06a0 0x7f4598141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 -- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f459815c8f0 msgr2=0x7f4598155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f459815c8f0 0x7f4598155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 -- 172.21.15.3:0/2522100272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590009420 con 0x7f459815ccc0 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459e55e700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459815ccc0 0x7f45981578a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f459806eaa0 tx=0x7f4590017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f4595ffb700 1 -- 172.21.15.3:0/2522100272 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590011ad0 con 0x7f459815ccc0 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f4595ffb700 1 -- 172.21.15.3:0/2522100272 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45900195c0 con 0x7f459815ccc0 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459815a060 con 0x7f459815ccc0 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.899+0000 7f4595ffb700 1 -- 172.21.15.3:0/2522100272 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590019980 con 0x7f459815ccc0 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.900+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4598142390 con 0x7f459815ccc0 2023-10-06T04:55:34.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.901+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459815a820 con 0x7f459815ccc0 2023-10-06T04:55:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.901+0000 7f4595ffb700 1 -- 172.21.15.3:0/2522100272 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4590019d30 con 0x7f459815ccc0 2023-10-06T04:55:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.901+0000 7f4595ffb700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4584066f30 0x7f45840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.901+0000 7f4595ffb700 1 -- 172.21.15.3:0/2522100272 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45900860b0 con 0x7f459815ccc0 2023-10-06T04:55:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.901+0000 7f459dd5d700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4584066f30 0x7f45840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:34.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.902+0000 7f459dd5d700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4584066f30 0x7f45840693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4588005c50 tx=0x7f4588009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:34.906+0000 7f4595ffb700 1 -- 172.21.15.3:0/2522100272 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45900531c0 con 0x7f459815ccc0 2023-10-06T04:55:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.096+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4598071680 con 0x7f4584066f30 2023-10-06T04:55:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.106+0000 7f4595ffb700 1 -- 172.21.15.3:0/2522100272 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308767 (secure 0 0 0) 0x7f4598071680 con 0x7f4584066f30 2023-10-06T04:55:35.112 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4584066f30 msgr2=0x7f45840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4584066f30 0x7f45840693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4588005c50 tx=0x7f4588009040 comp rx=0 tx=0).stop 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459815ccc0 msgr2=0x7f45981578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459815ccc0 0x7f45981578a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f459806eaa0 tx=0x7f4590017000 comp rx=0 tx=0).stop 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 shutdown_connections 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4584066f30 0x7f45840693e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459815ccc0 0x7f45981578a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45980d06a0 0x7f4598141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 --2- 172.21.15.3:0/2522100272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f459815c8f0 0x7f4598155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 >> 172.21.15.3:0/2522100272 conn(0x7f4598070310 msgr2=0x7f45980b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 shutdown_connections 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.109+0000 7f459ffc1700 1 -- 172.21.15.3:0/2522100272 wait complete. 2023-10-06T04:55:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:55:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2023-10-06T04:55:33.469652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2435248,"kb_used_data":7792,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539472,"statfs":{"total":773094113280,"available":770600419328,"internally_reserved":0,"allocated":7979008,"data_stored":4185933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.848173"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.635164+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634027+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630141+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:47.223457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630326+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:32.184584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.639206+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:09.194524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:54.819595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632371+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:26.055360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632057+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:14.777663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625915+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.269374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:36.692184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629995+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:47.654687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629843+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:31.939182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631919+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:03.823215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631745+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632536+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633013+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:29.108450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629698+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:45.147340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628196+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:18.527942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634491+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:29.940238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627719+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:45.969663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633388+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:10.209749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626849+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:34.163179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629948+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:48.336061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632213+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:11.480851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631083+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:28.732154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638569+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:16.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634028+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:18.240347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633198+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630555+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.772918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632025+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631841+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644855+0000","last_change":"2023-10-06T04:55:29.644855+0000","last_active":"2023-10-06T04:55:29.644855+0000","last_peered":"2023-10-06T04:55:29.644855+0000","last_clean":"2023-10-06T04:55:29.644855+0000","last_became_active":"2023-10-06T04:55:29.644553+0000","last_became_peered":"2023-10-06T04:55:29.644553+0000","last_unstale":"2023-10-06T04:55:29.644855+0000","last_undegraded":"2023-10-06T04:55:29.644855+0000","last_fullsized":"2023-10-06T04:55:29.644855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:31.518507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.640284+0000","last_change":"2023-10-06T04:55:29.640284+0000","last_active":"2023-10-06T04:55:29.640284+0000","last_peered":"2023-10-06T04:55:29.640284+0000","last_clean":"2023-10-06T04:55:29.640284+0000","last_became_active":"2023-10-06T04:55:29.638752+0000","last_became_peered":"2023-10-06T04:55:29.638752+0000","last_unstale":"2023-10-06T04:55:29.640284+0000","last_undegraded":"2023-10-06T04:55:29.640284+0000","last_fullsized":"2023-10-06T04:55:29.640284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.325782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626413+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:57.205791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.646265+0000","last_change":"2023-10-06T04:55:29.646265+0000","last_active":"2023-10-06T04:55:29.646265+0000","last_peered":"2023-10-06T04:55:29.646265+0000","last_clean":"2023-10-06T04:55:29.646265+0000","last_became_active":"2023-10-06T04:55:29.645786+0000","last_became_peered":"2023-10-06T04:55:29.645786+0000","last_unstale":"2023-10-06T04:55:29.646265+0000","last_undegraded":"2023-10-06T04:55:29.646265+0000","last_fullsized":"2023-10-06T04:55:29.646265+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:11.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627157+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:48.932063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643149+0000","last_change":"2023-10-06T04:55:29.643149+0000","last_active":"2023-10-06T04:55:29.643149+0000","last_peered":"2023-10-06T04:55:29.643149+0000","last_clean":"2023-10-06T04:55:29.643149+0000","last_became_active":"2023-10-06T04:55:29.642865+0000","last_became_peered":"2023-10-06T04:55:29.642865+0000","last_unstale":"2023-10-06T04:55:29.643149+0000","last_undegraded":"2023-10-06T04:55:29.643149+0000","last_fullsized":"2023-10-06T04:55:29.643149+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:25.603703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630411+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:12.293583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641637+0000","last_became_peered":"2023-10-06T04:55:29.641637+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:44.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630470+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:44.776671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642590+0000","last_change":"2023-10-06T04:55:29.642590+0000","last_active":"2023-10-06T04:55:29.642590+0000","last_peered":"2023-10-06T04:55:29.642590+0000","last_clean":"2023-10-06T04:55:29.642590+0000","last_became_active":"2023-10-06T04:55:29.642252+0000","last_became_peered":"2023-10-06T04:55:29.642252+0000","last_unstale":"2023-10-06T04:55:29.642590+0000","last_undegraded":"2023-10-06T04:55:29.642590+0000","last_fullsized":"2023-10-06T04:55:29.642590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:56.195511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626510+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:20.849108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643163+0000","last_change":"2023-10-06T04:55:29.643163+0000","last_active":"2023-10-06T04:55:29.643163+0000","last_peered":"2023-10-06T04:55:29.643163+0000","last_clean":"2023-10-06T04:55:29.643163+0000","last_became_active":"2023-10-06T04:55:29.642910+0000","last_became_peered":"2023-10-06T04:55:29.642910+0000","last_unstale":"2023-10-06T04:55:29.643163+0000","last_undegraded":"2023-10-06T04:55:29.643163+0000","last_fullsized":"2023-10-06T04:55:29.643163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:35.681905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631199+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:05.450563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641803+0000","last_change":"2023-10-06T04:55:29.641803+0000","last_active":"2023-10-06T04:55:29.641803+0000","last_peered":"2023-10-06T04:55:29.641803+0000","last_clean":"2023-10-06T04:55:29.641803+0000","last_became_active":"2023-10-06T04:55:29.641484+0000","last_became_peered":"2023-10-06T04:55:29.641484+0000","last_unstale":"2023-10-06T04:55:29.641803+0000","last_undegraded":"2023-10-06T04:55:29.641803+0000","last_fullsized":"2023-10-06T04:55:29.641803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.831547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633050+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:34.127918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.645173+0000","last_change":"2023-10-06T04:55:29.645173+0000","last_active":"2023-10-06T04:55:29.645173+0000","last_peered":"2023-10-06T04:55:29.645173+0000","last_clean":"2023-10-06T04:55:29.645173+0000","last_became_active":"2023-10-06T04:55:29.644899+0000","last_became_peered":"2023-10-06T04:55:29.644899+0000","last_unstale":"2023-10-06T04:55:29.645173+0000","last_undegraded":"2023-10-06T04:55:29.645173+0000","last_fullsized":"2023-10-06T04:55:29.645173+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:35.447530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628867+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:52.846620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639731+0000","last_change":"2023-10-06T04:55:29.639731+0000","last_active":"2023-10-06T04:55:29.639731+0000","last_peered":"2023-10-06T04:55:29.639731+0000","last_clean":"2023-10-06T04:55:29.639731+0000","last_became_active":"2023-10-06T04:55:29.639060+0000","last_became_peered":"2023-10-06T04:55:29.639060+0000","last_unstale":"2023-10-06T04:55:29.639731+0000","last_undegraded":"2023-10-06T04:55:29.639731+0000","last_fullsized":"2023-10-06T04:55:29.639731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:19.838829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629921+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:03.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643771+0000","last_change":"2023-10-06T04:55:29.643771+0000","last_active":"2023-10-06T04:55:29.643771+0000","last_peered":"2023-10-06T04:55:29.643771+0000","last_clean":"2023-10-06T04:55:29.643771+0000","last_became_active":"2023-10-06T04:55:29.643494+0000","last_became_peered":"2023-10-06T04:55:29.643494+0000","last_unstale":"2023-10-06T04:55:29.643771+0000","last_undegraded":"2023-10-06T04:55:29.643771+0000","last_fullsized":"2023-10-06T04:55:29.643771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:14.660406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629728+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:59.757486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636609+0000","last_change":"2023-10-06T04:55:29.636609+0000","last_active":"2023-10-06T04:55:29.636609+0000","last_peered":"2023-10-06T04:55:29.636609+0000","last_clean":"2023-10-06T04:55:29.636609+0000","last_became_active":"2023-10-06T04:55:29.636160+0000","last_became_peered":"2023-10-06T04:55:29.636160+0000","last_unstale":"2023-10-06T04:55:29.636609+0000","last_undegraded":"2023-10-06T04:55:29.636609+0000","last_fullsized":"2023-10-06T04:55:29.636609+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:32.519067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629380+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:31.703554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634753+0000","last_change":"2023-10-06T04:55:29.634753+0000","last_active":"2023-10-06T04:55:29.634753+0000","last_peered":"2023-10-06T04:55:29.634753+0000","last_clean":"2023-10-06T04:55:29.634753+0000","last_became_active":"2023-10-06T04:55:29.634471+0000","last_became_peered":"2023-10-06T04:55:29.634471+0000","last_unstale":"2023-10-06T04:55:29.634753+0000","last_undegraded":"2023-10-06T04:55:29.634753+0000","last_fullsized":"2023-10-06T04:55:29.634753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:47.921784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629562+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:38.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644259+0000","last_change":"2023-10-06T04:55:29.644259+0000","last_active":"2023-10-06T04:55:29.644259+0000","last_peered":"2023-10-06T04:55:29.644259+0000","last_clean":"2023-10-06T04:55:29.644259+0000","last_became_active":"2023-10-06T04:55:29.643992+0000","last_became_peered":"2023-10-06T04:55:29.643992+0000","last_unstale":"2023-10-06T04:55:29.644259+0000","last_undegraded":"2023-10-06T04:55:29.644259+0000","last_fullsized":"2023-10-06T04:55:29.644259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:43.348964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629557+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:34.367793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644987+0000","last_change":"2023-10-06T04:55:29.644987+0000","last_active":"2023-10-06T04:55:29.644987+0000","last_peered":"2023-10-06T04:55:29.644987+0000","last_clean":"2023-10-06T04:55:29.644987+0000","last_became_active":"2023-10-06T04:55:29.644620+0000","last_became_peered":"2023-10-06T04:55:29.644620+0000","last_unstale":"2023-10-06T04:55:29.644987+0000","last_undegraded":"2023-10-06T04:55:29.644987+0000","last_fullsized":"2023-10-06T04:55:29.644987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:57.789632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630035+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:38.834002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637724+0000","last_change":"2023-10-06T04:55:29.637724+0000","last_active":"2023-10-06T04:55:29.637724+0000","last_peered":"2023-10-06T04:55:29.637724+0000","last_clean":"2023-10-06T04:55:29.637724+0000","last_became_active":"2023-10-06T04:55:29.637235+0000","last_became_peered":"2023-10-06T04:55:29.637235+0000","last_unstale":"2023-10-06T04:55:29.637724+0000","last_undegraded":"2023-10-06T04:55:29.637724+0000","last_fullsized":"2023-10-06T04:55:29.637724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:27.721875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637907+0000","last_change":"2023-10-06T04:55:29.637907+0000","last_active":"2023-10-06T04:55:29.637907+0000","last_peered":"2023-10-06T04:55:29.637907+0000","last_clean":"2023-10-06T04:55:29.637907+0000","last_became_active":"2023-10-06T04:55:29.637626+0000","last_became_peered":"2023-10-06T04:55:29.637626+0000","last_unstale":"2023-10-06T04:55:29.637907+0000","last_undegraded":"2023-10-06T04:55:29.637907+0000","last_fullsized":"2023-10-06T04:55:29.637907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:44.959384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630578+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:39.999464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641635+0000","last_change":"2023-10-06T04:55:29.641635+0000","last_active":"2023-10-06T04:55:29.641635+0000","last_peered":"2023-10-06T04:55:29.641635+0000","last_clean":"2023-10-06T04:55:29.641635+0000","last_became_active":"2023-10-06T04:55:29.641370+0000","last_became_peered":"2023-10-06T04:55:29.641370+0000","last_unstale":"2023-10-06T04:55:29.641635+0000","last_undegraded":"2023-10-06T04:55:29.641635+0000","last_fullsized":"2023-10-06T04:55:29.641635+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625606+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:06.360553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644387+0000","last_change":"2023-10-06T04:55:29.644387+0000","last_active":"2023-10-06T04:55:29.644387+0000","last_peered":"2023-10-06T04:55:29.644387+0000","last_clean":"2023-10-06T04:55:29.644387+0000","last_became_active":"2023-10-06T04:55:29.644059+0000","last_became_peered":"2023-10-06T04:55:29.644059+0000","last_unstale":"2023-10-06T04:55:29.644387+0000","last_undegraded":"2023-10-06T04:55:29.644387+0000","last_fullsized":"2023-10-06T04:55:29.644387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631471+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644897+0000","last_change":"2023-10-06T04:55:29.644897+0000","last_active":"2023-10-06T04:55:29.644897+0000","last_peered":"2023-10-06T04:55:29.644897+0000","last_clean":"2023-10-06T04:55:29.644897+0000","last_became_active":"2023-10-06T04:55:29.644631+0000","last_became_peered":"2023-10-06T04:55:29.644631+0000","last_unstale":"2023-10-06T04:55:29.644897+0000","last_undegraded":"2023-10-06T04:55:29.644897+0000","last_fullsized":"2023-10-06T04:55:29.644897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630023+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:37.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642477+0000","last_change":"2023-10-06T04:55:29.642477+0000","last_active":"2023-10-06T04:55:29.642477+0000","last_peered":"2023-10-06T04:55:29.642477+0000","last_clean":"2023-10-06T04:55:29.642477+0000","last_became_active":"2023-10-06T04:55:29.642202+0000","last_became_peered":"2023-10-06T04:55:29.642202+0000","last_unstale":"2023-10-06T04:55:29.642477+0000","last_undegraded":"2023-10-06T04:55:29.642477+0000","last_fullsized":"2023-10-06T04:55:29.642477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:14.338401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629001+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:26.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644875+0000","last_change":"2023-10-06T04:55:29.644875+0000","last_active":"2023-10-06T04:55:29.644875+0000","last_peered":"2023-10-06T04:55:29.644875+0000","last_clean":"2023-10-06T04:55:29.644875+0000","last_became_active":"2023-10-06T04:55:29.644568+0000","last_became_peered":"2023-10-06T04:55:29.644568+0000","last_unstale":"2023-10-06T04:55:29.644875+0000","last_undegraded":"2023-10-06T04:55:29.644875+0000","last_fullsized":"2023-10-06T04:55:29.644875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:08.690122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629031+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.263825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641632+0000","last_change":"2023-10-06T04:55:29.641632+0000","last_active":"2023-10-06T04:55:29.641632+0000","last_peered":"2023-10-06T04:55:29.641632+0000","last_clean":"2023-10-06T04:55:29.641632+0000","last_became_active":"2023-10-06T04:55:29.641299+0000","last_became_peered":"2023-10-06T04:55:29.641299+0000","last_unstale":"2023-10-06T04:55:29.641632+0000","last_undegraded":"2023-10-06T04:55:29.641632+0000","last_fullsized":"2023-10-06T04:55:29.641632+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:42.695331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629260+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:06.628478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634678+0000","last_change":"2023-10-06T04:55:29.634678+0000","last_active":"2023-10-06T04:55:29.634678+0000","last_peered":"2023-10-06T04:55:29.634678+0000","last_clean":"2023-10-06T04:55:29.634678+0000","last_became_active":"2023-10-06T04:55:29.634400+0000","last_became_peered":"2023-10-06T04:55:29.634400+0000","last_unstale":"2023-10-06T04:55:29.634678+0000","last_undegraded":"2023-10-06T04:55:29.634678+0000","last_fullsized":"2023-10-06T04:55:29.634678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:04.440283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628124+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:38.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644649+0000","last_change":"2023-10-06T04:55:29.644649+0000","last_active":"2023-10-06T04:55:29.644649+0000","last_peered":"2023-10-06T04:55:29.644649+0000","last_clean":"2023-10-06T04:55:29.644649+0000","last_became_active":"2023-10-06T04:55:29.644333+0000","last_became_peered":"2023-10-06T04:55:29.644333+0000","last_unstale":"2023-10-06T04:55:29.644649+0000","last_undegraded":"2023-10-06T04:55:29.644649+0000","last_fullsized":"2023-10-06T04:55:29.644649+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:36.941087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620333+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:55:28.620333+0000","last_peered":"2023-10-06T04:55:28.620333+0000","last_clean":"2023-10-06T04:55:28.620333+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:55:28.620333+0000","last_undegraded":"2023-10-06T04:55:28.620333+0000","last_fullsized":"2023-10-06T04:55:28.620333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631507+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:37.433675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.628693+0000","last_change":"2023-10-06T04:55:29.628693+0000","last_active":"2023-10-06T04:55:29.628693+0000","last_peered":"2023-10-06T04:55:29.628693+0000","last_clean":"2023-10-06T04:55:29.628693+0000","last_became_active":"2023-10-06T04:55:29.627557+0000","last_became_peered":"2023-10-06T04:55:29.627557+0000","last_unstale":"2023-10-06T04:55:29.628693+0000","last_undegraded":"2023-10-06T04:55:29.628693+0000","last_fullsized":"2023-10-06T04:55:29.628693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:58.747206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.617715+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:55:28.617715+0000","last_peered":"2023-10-06T04:55:28.617715+0000","last_clean":"2023-10-06T04:55:28.617715+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:55:28.617715+0000","last_undegraded":"2023-10-06T04:55:28.617715+0000","last_fullsized":"2023-10-06T04:55:28.617715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632435+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.568423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636254+0000","last_change":"2023-10-06T04:55:29.636254+0000","last_active":"2023-10-06T04:55:29.636254+0000","last_peered":"2023-10-06T04:55:29.636254+0000","last_clean":"2023-10-06T04:55:29.636254+0000","last_became_active":"2023-10-06T04:55:29.635966+0000","last_became_peered":"2023-10-06T04:55:29.635966+0000","last_unstale":"2023-10-06T04:55:29.636254+0000","last_undegraded":"2023-10-06T04:55:29.636254+0000","last_fullsized":"2023-10-06T04:55:29.636254+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:09.863401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.622100+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:55:28.622100+0000","last_peered":"2023-10-06T04:55:28.622100+0000","last_clean":"2023-10-06T04:55:28.622100+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:55:28.622100+0000","last_undegraded":"2023-10-06T04:55:28.622100+0000","last_fullsized":"2023-10-06T04:55:28.622100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":202,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623115+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:55:28.623115+0000","last_peered":"2023-10-06T04:55:28.623115+0000","last_clean":"2023-10-06T04:55:28.623115+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:55:28.623115+0000","last_undegraded":"2023-10-06T04:55:28.623115+0000","last_fullsized":"2023-10-06T04:55:28.623115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630556+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:25.364931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634448+0000","last_change":"2023-10-06T04:55:29.634448+0000","last_active":"2023-10-06T04:55:29.634448+0000","last_peered":"2023-10-06T04:55:29.634448+0000","last_clean":"2023-10-06T04:55:29.634448+0000","last_became_active":"2023-10-06T04:55:29.634144+0000","last_became_peered":"2023-10-06T04:55:29.634144+0000","last_unstale":"2023-10-06T04:55:29.634448+0000","last_undegraded":"2023-10-06T04:55:29.634448+0000","last_fullsized":"2023-10-06T04:55:29.634448+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:14.628720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623209+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:55:28.623209+0000","last_peered":"2023-10-06T04:55:28.623209+0000","last_clean":"2023-10-06T04:55:28.623209+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:55:28.623209+0000","last_undegraded":"2023-10-06T04:55:28.623209+0000","last_fullsized":"2023-10-06T04:55:28.623209+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638091+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637914+0000","last_change":"2023-10-06T04:55:29.637914+0000","last_active":"2023-10-06T04:55:29.637914+0000","last_peered":"2023-10-06T04:55:29.637914+0000","last_clean":"2023-10-06T04:55:29.637914+0000","last_became_active":"2023-10-06T04:55:29.637640+0000","last_became_peered":"2023-10-06T04:55:29.637640+0000","last_unstale":"2023-10-06T04:55:29.637914+0000","last_undegraded":"2023-10-06T04:55:29.637914+0000","last_fullsized":"2023-10-06T04:55:29.637914+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:15.026328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623208+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:55:28.623208+0000","last_peered":"2023-10-06T04:55:28.623208+0000","last_clean":"2023-10-06T04:55:28.623208+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:55:28.623208+0000","last_undegraded":"2023-10-06T04:55:28.623208+0000","last_fullsized":"2023-10-06T04:55:28.623208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629855+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:36.457810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641718+0000","last_change":"2023-10-06T04:55:29.641718+0000","last_active":"2023-10-06T04:55:29.641718+0000","last_peered":"2023-10-06T04:55:29.641718+0000","last_clean":"2023-10-06T04:55:29.641718+0000","last_became_active":"2023-10-06T04:55:29.641423+0000","last_became_peered":"2023-10-06T04:55:29.641423+0000","last_unstale":"2023-10-06T04:55:29.641718+0000","last_undegraded":"2023-10-06T04:55:29.641718+0000","last_fullsized":"2023-10-06T04:55:29.641718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:26.283304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.624774+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:55:28.624774+0000","last_peered":"2023-10-06T04:55:28.624774+0000","last_clean":"2023-10-06T04:55:28.624774+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:55:28.624774+0000","last_undegraded":"2023-10-06T04:55:28.624774+0000","last_fullsized":"2023-10-06T04:55:28.624774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628576+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.902269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639321+0000","last_change":"2023-10-06T04:55:29.639321+0000","last_active":"2023-10-06T04:55:29.639321+0000","last_peered":"2023-10-06T04:55:29.639321+0000","last_clean":"2023-10-06T04:55:29.639321+0000","last_became_active":"2023-10-06T04:55:29.639057+0000","last_became_peered":"2023-10-06T04:55:29.639057+0000","last_unstale":"2023-10-06T04:55:29.639321+0000","last_undegraded":"2023-10-06T04:55:29.639321+0000","last_fullsized":"2023-10-06T04:55:29.639321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.045081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620654+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:55:28.620654+0000","last_peered":"2023-10-06T04:55:28.620654+0000","last_clean":"2023-10-06T04:55:28.620654+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:55:28.620654+0000","last_undegraded":"2023-10-06T04:55:28.620654+0000","last_fullsized":"2023-10-06T04:55:28.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632257+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:01.492662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641636+0000","last_became_peered":"2023-10-06T04:55:29.641636+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:29.769932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620664+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:55:28.620664+0000","last_peered":"2023-10-06T04:55:28.620664+0000","last_clean":"2023-10-06T04:55:28.620664+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:55:28.620664+0000","last_undegraded":"2023-10-06T04:55:28.620664+0000","last_fullsized":"2023-10-06T04:55:28.620664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.70499999999999996,"15min":0.77600000000000002},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88700000000000001,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.751,"15min":0.82799999999999996},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.90500000000000003,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":1.073,"15min":1.073},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81799999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.111,"15min":1.111},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78900000000000003,"5min":0.874,"15min":0.874},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":1.214,"15min":1.214},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.497},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.38,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60399999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.876,"15min":0.876},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63200000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.60399999999999998},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35499999999999998,"15min":0.36299999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.37,"15min":0.38},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.29899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.60699999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.64300000000000002,"15min":0.66700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63100000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85299999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.026,"15min":1.026},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.85599999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.34699999999999998,"15min":0.35499999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.373},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36699999999999999},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60699999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.624,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64400000000000002},"min":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.19500000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.54100000000000004,"5min":1.048,"15min":1.048},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.376},"min":{"1min":0.28699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.624,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.29599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69799999999999995,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82299999999999995,"5min":1.246,"15min":1.246},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.877,"15min":0.877},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89400000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.624},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306312,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065528,"statfs":{"total":96636764160,"available":96323100672,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.40999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.54800000000000004,"15min":0.60999999999999999},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.1480000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.53000000000000003,"15min":0.58599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.379,"5min":5.069,"15min":5.069},"last":0.435},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.49299999999999999},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.86099999999999999,"15min":0.83799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":14.646000000000001,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.2769999999999999,"5min":0.81799999999999995,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":14.606999999999999,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.82299999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.9219999999999999,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74199999999999999,"15min":0.79300000000000004},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4079999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76900000000000002,"15min":0.83899999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.964,"5min":6.367,"15min":6.367},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.45900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3,"5min":5,"15min":5},"last":0.71599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.313,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.752,"15min":0.85399999999999998},"min":{"1min":0.438,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82299999999999995,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.77900000000000003,"15min":0.879},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.78600000000000003,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.76200000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.70099999999999996},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80400000000000005,"5min":5.415,"15min":5.415},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65000000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":4.202,"15min":4.202},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81799999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69999999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.872,"15min":0.872},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306296,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065544,"statfs":{"total":96636764160,"available":96323117056,"internally_reserved":0,"allocated":835584,"data_stored":369015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.65400000000000003,"15min":0.71899999999999997},"min":{"1min":0.26300000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73099999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67800000000000005,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69299999999999995,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.65900000000000003,"15min":0.70499999999999996},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.81799999999999995,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63900000000000001,"15min":0.68700000000000006},"min":{"1min":0.373,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83899999999999997,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.89500000000000002,"15min":1.044},"min":{"1min":0.39500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77300000000000002,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.90500000000000003,"15min":1.0409999999999999},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72399999999999998,"5min":0.751,"15min":0.751},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55800000000000005,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:55:35.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:55:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.281+0000 7f94f0d5f700 1 Processor -- start 2023-10-06T04:55:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.281+0000 7f94f0d5f700 1 -- start start 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94f0d5f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94ec0b3d40 0x7f94ec164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94f0d5f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec15aeb0 con 0x7f94ec157030 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94f0d5f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec15a7a0 con 0x7f94ec157400 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94f0d5f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec15aa80 con 0x7f94ec0b3d40 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94ead9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94ec0b3d40 0x7f94ec164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94ea59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94ec157030 0x7f94ec159ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56014/0 (socket says 172.21.15.3:56014) 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94ea59c700 1 -- 172.21.15.3:0/3874615725 learned_addr learned my addr 172.21.15.3:0/3874615725 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94ead9d700 1 -- 172.21.15.3:0/3874615725 >> v1:172.21.15.153:6789/0 conn(0x7f94ec157400 legacy=0x7f94ec0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94ead9d700 1 -- 172.21.15.3:0/3874615725 >> v1:172.21.15.3:6789/0 conn(0x7f94ec157030 legacy=0x7f94ec159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94ead9d700 1 -- 172.21.15.3:0/3874615725 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec06dba0 con 0x7f94ec0b3d40 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94e959a700 1 -- 172.21.15.3:0/3874615725 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288538187 0 0) 0x7f94ec15aeb0 con 0x7f94ec157030 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.282+0000 7f94ead9d700 1 --2- 172.21.15.3:0/3874615725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec164320 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009e60 tx=0x7f94e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cd7355ad69687d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94e959a700 1 -- 172.21.15.3:0/3874615725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400e070 con 0x7f94ec0b3d40 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94e959a700 1 -- 172.21.15.3:0/3874615725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e400bbf0 con 0x7f94ec0b3d40 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94e959a700 1 -- 172.21.15.3:0/3874615725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4012430 con 0x7f94ec0b3d40 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94f0d5f700 1 -- 172.21.15.3:0/3874615725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 msgr2=0x7f94ec164320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/3874615725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec164320 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009e60 tx=0x7f94e4009780 comp rx=0 tx=0).stop 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94f0d5f700 1 -- 172.21.15.3:0/3874615725 shutdown_connections 2023-10-06T04:55:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/3874615725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec164320 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94f0d5f700 1 -- 172.21.15.3:0/3874615725 >> 172.21.15.3:0/3874615725 conn(0x7f94ec0ce270 msgr2=0x7f94ec0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94f0d5f700 1 -- 172.21.15.3:0/3874615725 shutdown_connections 2023-10-06T04:55:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.283+0000 7f94f0d5f700 1 -- 172.21.15.3:0/3874615725 wait complete. 2023-10-06T04:55:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.284+0000 7f94f0d5f700 1 Processor -- start 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.284+0000 7f94f0d5f700 1 -- start start 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94f0d5f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94f0d5f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94ec157030 0x7f94ec0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94f0d5f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94ec157400 0x7f94ec159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94f0d5f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec150c90 con 0x7f94ec0b3d40 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94f0d5f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec150f70 con 0x7f94ec157400 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94f0d5f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec151250 con 0x7f94ec157030 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec14f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58022/0 (socket says 172.21.15.3:58022) 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 -- 172.21.15.3:0/523336308 learned_addr learned my addr 172.21.15.3:0/523336308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 -- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94ec157030 msgr2=0x7f94ec0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94ec157030 0x7f94ec0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 -- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94ec157400 msgr2=0x7f94ec159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94ec157400 0x7f94ec159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.285+0000 7f94ea59c700 1 -- 172.21.15.3:0/523336308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e4009a20 con 0x7f94ec0b3d40 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.286+0000 7f94ea59c700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec14f8d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c7e0 tx=0x7f94dc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.286+0000 7f94da7fc700 1 -- 172.21.15.3:0/523336308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc005b30 con 0x7f94ec0b3d40 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.286+0000 7f94da7fc700 1 -- 172.21.15.3:0/523336308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc005c90 con 0x7f94ec0b3d40 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.286+0000 7f94da7fc700 1 -- 172.21.15.3:0/523336308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc007300 con 0x7f94ec0b3d40 2023-10-06T04:55:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.286+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec151530 con 0x7f94ec0b3d40 2023-10-06T04:55:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.286+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec14fe10 con 0x7f94ec0b3d40 2023-10-06T04:55:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.287+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec151a50 con 0x7f94ec0b3d40 2023-10-06T04:55:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.287+0000 7f94da7fc700 1 -- 172.21.15.3:0/523336308 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f94dc01b400 con 0x7f94ec0b3d40 2023-10-06T04:55:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.288+0000 7f94da7fc700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f94cc066f30 0x7f94cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.288+0000 7f94da7fc700 1 -- 172.21.15.3:0/523336308 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f94dc084240 con 0x7f94ec0b3d40 2023-10-06T04:55:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.288+0000 7f94e9d9b700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f94cc066f30 0x7f94cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.288+0000 7f94e9d9b700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f94cc066f30 0x7f94cc0693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f94ec063990 tx=0x7f94e0006c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.292+0000 7f94da7fc700 1 -- 172.21.15.3:0/523336308 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94ec151a50 con 0x7f94ec0b3d40 2023-10-06T04:55:35.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.481+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94ec151a50 con 0x7f94cc066f30 2023-10-06T04:55:35.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.490+0000 7f94da7fc700 1 -- 172.21.15.3:0/523336308 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308767 (secure 0 0 0) 0x7f94ec151a50 con 0x7f94cc066f30 2023-10-06T04:55:35.496 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:35.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f94cc066f30 msgr2=0x7f94cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f94cc066f30 0x7f94cc0693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f94ec063990 tx=0x7f94e0006c60 comp rx=0 tx=0).stop 2023-10-06T04:55:35.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 msgr2=0x7f94ec14f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec14f8d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c7e0 tx=0x7f94dc007df0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 shutdown_connections 2023-10-06T04:55:35.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f94cc066f30 0x7f94cc0693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94ec157400 0x7f94ec159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94ec0b3d40 0x7f94ec14f8d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 --2- 172.21.15.3:0/523336308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94ec157030 0x7f94ec0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 >> 172.21.15.3:0/523336308 conn(0x7f94ec0ce270 msgr2=0x7f94ec06e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 shutdown_connections 2023-10-06T04:55:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.493+0000 7f94f0d5f700 1 -- 172.21.15.3:0/523336308 wait complete. 2023-10-06T04:55:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:55:35.508 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2023-10-06T04:55:35.470082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2435248,"kb_used_data":7792,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539472,"statfs":{"total":773094113280,"available":770600419328,"internally_reserved":0,"allocated":7979008,"data_stored":4185933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001309"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.635164+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634027+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630141+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:47.223457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630326+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:32.184584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.639206+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:09.194524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:54.819595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632371+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:26.055360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632057+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:14.777663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625915+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.269374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:36.692184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629995+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:47.654687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629843+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:31.939182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631919+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:03.823215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631745+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632536+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633013+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:29.108450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629698+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:45.147340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628196+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:18.527942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634491+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:29.940238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627719+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:45.969663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633388+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:10.209749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626849+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:34.163179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629948+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:48.336061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632213+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:11.480851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631083+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:28.732154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638569+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:16.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634028+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:18.240347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633198+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630555+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.772918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632025+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631841+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644855+0000","last_change":"2023-10-06T04:55:29.644855+0000","last_active":"2023-10-06T04:55:29.644855+0000","last_peered":"2023-10-06T04:55:29.644855+0000","last_clean":"2023-10-06T04:55:29.644855+0000","last_became_active":"2023-10-06T04:55:29.644553+0000","last_became_peered":"2023-10-06T04:55:29.644553+0000","last_unstale":"2023-10-06T04:55:29.644855+0000","last_undegraded":"2023-10-06T04:55:29.644855+0000","last_fullsized":"2023-10-06T04:55:29.644855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:31.518507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.640284+0000","last_change":"2023-10-06T04:55:29.640284+0000","last_active":"2023-10-06T04:55:29.640284+0000","last_peered":"2023-10-06T04:55:29.640284+0000","last_clean":"2023-10-06T04:55:29.640284+0000","last_became_active":"2023-10-06T04:55:29.638752+0000","last_became_peered":"2023-10-06T04:55:29.638752+0000","last_unstale":"2023-10-06T04:55:29.640284+0000","last_undegraded":"2023-10-06T04:55:29.640284+0000","last_fullsized":"2023-10-06T04:55:29.640284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.325782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626413+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:57.205791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.646265+0000","last_change":"2023-10-06T04:55:29.646265+0000","last_active":"2023-10-06T04:55:29.646265+0000","last_peered":"2023-10-06T04:55:29.646265+0000","last_clean":"2023-10-06T04:55:29.646265+0000","last_became_active":"2023-10-06T04:55:29.645786+0000","last_became_peered":"2023-10-06T04:55:29.645786+0000","last_unstale":"2023-10-06T04:55:29.646265+0000","last_undegraded":"2023-10-06T04:55:29.646265+0000","last_fullsized":"2023-10-06T04:55:29.646265+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:11.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627157+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:48.932063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643149+0000","last_change":"2023-10-06T04:55:29.643149+0000","last_active":"2023-10-06T04:55:29.643149+0000","last_peered":"2023-10-06T04:55:29.643149+0000","last_clean":"2023-10-06T04:55:29.643149+0000","last_became_active":"2023-10-06T04:55:29.642865+0000","last_became_peered":"2023-10-06T04:55:29.642865+0000","last_unstale":"2023-10-06T04:55:29.643149+0000","last_undegraded":"2023-10-06T04:55:29.643149+0000","last_fullsized":"2023-10-06T04:55:29.643149+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:25.603703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630411+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:12.293583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641637+0000","last_became_peered":"2023-10-06T04:55:29.641637+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:44.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630470+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:44.776671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642590+0000","last_change":"2023-10-06T04:55:29.642590+0000","last_active":"2023-10-06T04:55:29.642590+0000","last_peered":"2023-10-06T04:55:29.642590+0000","last_clean":"2023-10-06T04:55:29.642590+0000","last_became_active":"2023-10-06T04:55:29.642252+0000","last_became_peered":"2023-10-06T04:55:29.642252+0000","last_unstale":"2023-10-06T04:55:29.642590+0000","last_undegraded":"2023-10-06T04:55:29.642590+0000","last_fullsized":"2023-10-06T04:55:29.642590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:56.195511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626510+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:20.849108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643163+0000","last_change":"2023-10-06T04:55:29.643163+0000","last_active":"2023-10-06T04:55:29.643163+0000","last_peered":"2023-10-06T04:55:29.643163+0000","last_clean":"2023-10-06T04:55:29.643163+0000","last_became_active":"2023-10-06T04:55:29.642910+0000","last_became_peered":"2023-10-06T04:55:29.642910+0000","last_unstale":"2023-10-06T04:55:29.643163+0000","last_undegraded":"2023-10-06T04:55:29.643163+0000","last_fullsized":"2023-10-06T04:55:29.643163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:35.681905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631199+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:05.450563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641803+0000","last_change":"2023-10-06T04:55:29.641803+0000","last_active":"2023-10-06T04:55:29.641803+0000","last_peered":"2023-10-06T04:55:29.641803+0000","last_clean":"2023-10-06T04:55:29.641803+0000","last_became_active":"2023-10-06T04:55:29.641484+0000","last_became_peered":"2023-10-06T04:55:29.641484+0000","last_unstale":"2023-10-06T04:55:29.641803+0000","last_undegraded":"2023-10-06T04:55:29.641803+0000","last_fullsized":"2023-10-06T04:55:29.641803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.831547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633050+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:34.127918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.645173+0000","last_change":"2023-10-06T04:55:29.645173+0000","last_active":"2023-10-06T04:55:29.645173+0000","last_peered":"2023-10-06T04:55:29.645173+0000","last_clean":"2023-10-06T04:55:29.645173+0000","last_became_active":"2023-10-06T04:55:29.644899+0000","last_became_peered":"2023-10-06T04:55:29.644899+0000","last_unstale":"2023-10-06T04:55:29.645173+0000","last_undegraded":"2023-10-06T04:55:29.645173+0000","last_fullsized":"2023-10-06T04:55:29.645173+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:35.447530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628867+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:52.846620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639731+0000","last_change":"2023-10-06T04:55:29.639731+0000","last_active":"2023-10-06T04:55:29.639731+0000","last_peered":"2023-10-06T04:55:29.639731+0000","last_clean":"2023-10-06T04:55:29.639731+0000","last_became_active":"2023-10-06T04:55:29.639060+0000","last_became_peered":"2023-10-06T04:55:29.639060+0000","last_unstale":"2023-10-06T04:55:29.639731+0000","last_undegraded":"2023-10-06T04:55:29.639731+0000","last_fullsized":"2023-10-06T04:55:29.639731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:19.838829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629921+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:03.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643771+0000","last_change":"2023-10-06T04:55:29.643771+0000","last_active":"2023-10-06T04:55:29.643771+0000","last_peered":"2023-10-06T04:55:29.643771+0000","last_clean":"2023-10-06T04:55:29.643771+0000","last_became_active":"2023-10-06T04:55:29.643494+0000","last_became_peered":"2023-10-06T04:55:29.643494+0000","last_unstale":"2023-10-06T04:55:29.643771+0000","last_undegraded":"2023-10-06T04:55:29.643771+0000","last_fullsized":"2023-10-06T04:55:29.643771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:14.660406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629728+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:59.757486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636609+0000","last_change":"2023-10-06T04:55:29.636609+0000","last_active":"2023-10-06T04:55:29.636609+0000","last_peered":"2023-10-06T04:55:29.636609+0000","last_clean":"2023-10-06T04:55:29.636609+0000","last_became_active":"2023-10-06T04:55:29.636160+0000","last_became_peered":"2023-10-06T04:55:29.636160+0000","last_unstale":"2023-10-06T04:55:29.636609+0000","last_undegraded":"2023-10-06T04:55:29.636609+0000","last_fullsized":"2023-10-06T04:55:29.636609+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:32.519067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629380+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:31.703554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634753+0000","last_change":"2023-10-06T04:55:29.634753+0000","last_active":"2023-10-06T04:55:29.634753+0000","last_peered":"2023-10-06T04:55:29.634753+0000","last_clean":"2023-10-06T04:55:29.634753+0000","last_became_active":"2023-10-06T04:55:29.634471+0000","last_became_peered":"2023-10-06T04:55:29.634471+0000","last_unstale":"2023-10-06T04:55:29.634753+0000","last_undegraded":"2023-10-06T04:55:29.634753+0000","last_fullsized":"2023-10-06T04:55:29.634753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:47.921784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629562+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:38.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644259+0000","last_change":"2023-10-06T04:55:29.644259+0000","last_active":"2023-10-06T04:55:29.644259+0000","last_peered":"2023-10-06T04:55:29.644259+0000","last_clean":"2023-10-06T04:55:29.644259+0000","last_became_active":"2023-10-06T04:55:29.643992+0000","last_became_peered":"2023-10-06T04:55:29.643992+0000","last_unstale":"2023-10-06T04:55:29.644259+0000","last_undegraded":"2023-10-06T04:55:29.644259+0000","last_fullsized":"2023-10-06T04:55:29.644259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:43.348964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629557+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:34.367793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644987+0000","last_change":"2023-10-06T04:55:29.644987+0000","last_active":"2023-10-06T04:55:29.644987+0000","last_peered":"2023-10-06T04:55:29.644987+0000","last_clean":"2023-10-06T04:55:29.644987+0000","last_became_active":"2023-10-06T04:55:29.644620+0000","last_became_peered":"2023-10-06T04:55:29.644620+0000","last_unstale":"2023-10-06T04:55:29.644987+0000","last_undegraded":"2023-10-06T04:55:29.644987+0000","last_fullsized":"2023-10-06T04:55:29.644987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:57.789632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630035+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:38.834002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637724+0000","last_change":"2023-10-06T04:55:29.637724+0000","last_active":"2023-10-06T04:55:29.637724+0000","last_peered":"2023-10-06T04:55:29.637724+0000","last_clean":"2023-10-06T04:55:29.637724+0000","last_became_active":"2023-10-06T04:55:29.637235+0000","last_became_peered":"2023-10-06T04:55:29.637235+0000","last_unstale":"2023-10-06T04:55:29.637724+0000","last_undegraded":"2023-10-06T04:55:29.637724+0000","last_fullsized":"2023-10-06T04:55:29.637724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:27.721875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637907+0000","last_change":"2023-10-06T04:55:29.637907+0000","last_active":"2023-10-06T04:55:29.637907+0000","last_peered":"2023-10-06T04:55:29.637907+0000","last_clean":"2023-10-06T04:55:29.637907+0000","last_became_active":"2023-10-06T04:55:29.637626+0000","last_became_peered":"2023-10-06T04:55:29.637626+0000","last_unstale":"2023-10-06T04:55:29.637907+0000","last_undegraded":"2023-10-06T04:55:29.637907+0000","last_fullsized":"2023-10-06T04:55:29.637907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:44.959384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630578+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:39.999464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641635+0000","last_change":"2023-10-06T04:55:29.641635+0000","last_active":"2023-10-06T04:55:29.641635+0000","last_peered":"2023-10-06T04:55:29.641635+0000","last_clean":"2023-10-06T04:55:29.641635+0000","last_became_active":"2023-10-06T04:55:29.641370+0000","last_became_peered":"2023-10-06T04:55:29.641370+0000","last_unstale":"2023-10-06T04:55:29.641635+0000","last_undegraded":"2023-10-06T04:55:29.641635+0000","last_fullsized":"2023-10-06T04:55:29.641635+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625606+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:06.360553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644387+0000","last_change":"2023-10-06T04:55:29.644387+0000","last_active":"2023-10-06T04:55:29.644387+0000","last_peered":"2023-10-06T04:55:29.644387+0000","last_clean":"2023-10-06T04:55:29.644387+0000","last_became_active":"2023-10-06T04:55:29.644059+0000","last_became_peered":"2023-10-06T04:55:29.644059+0000","last_unstale":"2023-10-06T04:55:29.644387+0000","last_undegraded":"2023-10-06T04:55:29.644387+0000","last_fullsized":"2023-10-06T04:55:29.644387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631471+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644897+0000","last_change":"2023-10-06T04:55:29.644897+0000","last_active":"2023-10-06T04:55:29.644897+0000","last_peered":"2023-10-06T04:55:29.644897+0000","last_clean":"2023-10-06T04:55:29.644897+0000","last_became_active":"2023-10-06T04:55:29.644631+0000","last_became_peered":"2023-10-06T04:55:29.644631+0000","last_unstale":"2023-10-06T04:55:29.644897+0000","last_undegraded":"2023-10-06T04:55:29.644897+0000","last_fullsized":"2023-10-06T04:55:29.644897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630023+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:37.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642477+0000","last_change":"2023-10-06T04:55:29.642477+0000","last_active":"2023-10-06T04:55:29.642477+0000","last_peered":"2023-10-06T04:55:29.642477+0000","last_clean":"2023-10-06T04:55:29.642477+0000","last_became_active":"2023-10-06T04:55:29.642202+0000","last_became_peered":"2023-10-06T04:55:29.642202+0000","last_unstale":"2023-10-06T04:55:29.642477+0000","last_undegraded":"2023-10-06T04:55:29.642477+0000","last_fullsized":"2023-10-06T04:55:29.642477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:14.338401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629001+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:26.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644875+0000","last_change":"2023-10-06T04:55:29.644875+0000","last_active":"2023-10-06T04:55:29.644875+0000","last_peered":"2023-10-06T04:55:29.644875+0000","last_clean":"2023-10-06T04:55:29.644875+0000","last_became_active":"2023-10-06T04:55:29.644568+0000","last_became_peered":"2023-10-06T04:55:29.644568+0000","last_unstale":"2023-10-06T04:55:29.644875+0000","last_undegraded":"2023-10-06T04:55:29.644875+0000","last_fullsized":"2023-10-06T04:55:29.644875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:08.690122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629031+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.263825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641632+0000","last_change":"2023-10-06T04:55:29.641632+0000","last_active":"2023-10-06T04:55:29.641632+0000","last_peered":"2023-10-06T04:55:29.641632+0000","last_clean":"2023-10-06T04:55:29.641632+0000","last_became_active":"2023-10-06T04:55:29.641299+0000","last_became_peered":"2023-10-06T04:55:29.641299+0000","last_unstale":"2023-10-06T04:55:29.641632+0000","last_undegraded":"2023-10-06T04:55:29.641632+0000","last_fullsized":"2023-10-06T04:55:29.641632+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:42.695331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629260+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:06.628478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634678+0000","last_change":"2023-10-06T04:55:29.634678+0000","last_active":"2023-10-06T04:55:29.634678+0000","last_peered":"2023-10-06T04:55:29.634678+0000","last_clean":"2023-10-06T04:55:29.634678+0000","last_became_active":"2023-10-06T04:55:29.634400+0000","last_became_peered":"2023-10-06T04:55:29.634400+0000","last_unstale":"2023-10-06T04:55:29.634678+0000","last_undegraded":"2023-10-06T04:55:29.634678+0000","last_fullsized":"2023-10-06T04:55:29.634678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:04.440283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628124+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:38.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644649+0000","last_change":"2023-10-06T04:55:29.644649+0000","last_active":"2023-10-06T04:55:29.644649+0000","last_peered":"2023-10-06T04:55:29.644649+0000","last_clean":"2023-10-06T04:55:29.644649+0000","last_became_active":"2023-10-06T04:55:29.644333+0000","last_became_peered":"2023-10-06T04:55:29.644333+0000","last_unstale":"2023-10-06T04:55:29.644649+0000","last_undegraded":"2023-10-06T04:55:29.644649+0000","last_fullsized":"2023-10-06T04:55:29.644649+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:36.941087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620333+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:55:28.620333+0000","last_peered":"2023-10-06T04:55:28.620333+0000","last_clean":"2023-10-06T04:55:28.620333+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:55:28.620333+0000","last_undegraded":"2023-10-06T04:55:28.620333+0000","last_fullsized":"2023-10-06T04:55:28.620333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631507+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:37.433675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.628693+0000","last_change":"2023-10-06T04:55:29.628693+0000","last_active":"2023-10-06T04:55:29.628693+0000","last_peered":"2023-10-06T04:55:29.628693+0000","last_clean":"2023-10-06T04:55:29.628693+0000","last_became_active":"2023-10-06T04:55:29.627557+0000","last_became_peered":"2023-10-06T04:55:29.627557+0000","last_unstale":"2023-10-06T04:55:29.628693+0000","last_undegraded":"2023-10-06T04:55:29.628693+0000","last_fullsized":"2023-10-06T04:55:29.628693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:58.747206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.617715+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:55:28.617715+0000","last_peered":"2023-10-06T04:55:28.617715+0000","last_clean":"2023-10-06T04:55:28.617715+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:55:28.617715+0000","last_undegraded":"2023-10-06T04:55:28.617715+0000","last_fullsized":"2023-10-06T04:55:28.617715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632435+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.568423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636254+0000","last_change":"2023-10-06T04:55:29.636254+0000","last_active":"2023-10-06T04:55:29.636254+0000","last_peered":"2023-10-06T04:55:29.636254+0000","last_clean":"2023-10-06T04:55:29.636254+0000","last_became_active":"2023-10-06T04:55:29.635966+0000","last_became_peered":"2023-10-06T04:55:29.635966+0000","last_unstale":"2023-10-06T04:55:29.636254+0000","last_undegraded":"2023-10-06T04:55:29.636254+0000","last_fullsized":"2023-10-06T04:55:29.636254+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:09.863401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.622100+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:55:28.622100+0000","last_peered":"2023-10-06T04:55:28.622100+0000","last_clean":"2023-10-06T04:55:28.622100+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:55:28.622100+0000","last_undegraded":"2023-10-06T04:55:28.622100+0000","last_fullsized":"2023-10-06T04:55:28.622100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":202,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623115+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:55:28.623115+0000","last_peered":"2023-10-06T04:55:28.623115+0000","last_clean":"2023-10-06T04:55:28.623115+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:55:28.623115+0000","last_undegraded":"2023-10-06T04:55:28.623115+0000","last_fullsized":"2023-10-06T04:55:28.623115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630556+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:25.364931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634448+0000","last_change":"2023-10-06T04:55:29.634448+0000","last_active":"2023-10-06T04:55:29.634448+0000","last_peered":"2023-10-06T04:55:29.634448+0000","last_clean":"2023-10-06T04:55:29.634448+0000","last_became_active":"2023-10-06T04:55:29.634144+0000","last_became_peered":"2023-10-06T04:55:29.634144+0000","last_unstale":"2023-10-06T04:55:29.634448+0000","last_undegraded":"2023-10-06T04:55:29.634448+0000","last_fullsized":"2023-10-06T04:55:29.634448+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:14.628720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623209+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:55:28.623209+0000","last_peered":"2023-10-06T04:55:28.623209+0000","last_clean":"2023-10-06T04:55:28.623209+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:55:28.623209+0000","last_undegraded":"2023-10-06T04:55:28.623209+0000","last_fullsized":"2023-10-06T04:55:28.623209+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638091+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637914+0000","last_change":"2023-10-06T04:55:29.637914+0000","last_active":"2023-10-06T04:55:29.637914+0000","last_peered":"2023-10-06T04:55:29.637914+0000","last_clean":"2023-10-06T04:55:29.637914+0000","last_became_active":"2023-10-06T04:55:29.637640+0000","last_became_peered":"2023-10-06T04:55:29.637640+0000","last_unstale":"2023-10-06T04:55:29.637914+0000","last_undegraded":"2023-10-06T04:55:29.637914+0000","last_fullsized":"2023-10-06T04:55:29.637914+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:15.026328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623208+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:55:28.623208+0000","last_peered":"2023-10-06T04:55:28.623208+0000","last_clean":"2023-10-06T04:55:28.623208+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:55:28.623208+0000","last_undegraded":"2023-10-06T04:55:28.623208+0000","last_fullsized":"2023-10-06T04:55:28.623208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629855+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:36.457810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641718+0000","last_change":"2023-10-06T04:55:29.641718+0000","last_active":"2023-10-06T04:55:29.641718+0000","last_peered":"2023-10-06T04:55:29.641718+0000","last_clean":"2023-10-06T04:55:29.641718+0000","last_became_active":"2023-10-06T04:55:29.641423+0000","last_became_peered":"2023-10-06T04:55:29.641423+0000","last_unstale":"2023-10-06T04:55:29.641718+0000","last_undegraded":"2023-10-06T04:55:29.641718+0000","last_fullsized":"2023-10-06T04:55:29.641718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:26.283304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.624774+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:55:28.624774+0000","last_peered":"2023-10-06T04:55:28.624774+0000","last_clean":"2023-10-06T04:55:28.624774+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:55:28.624774+0000","last_undegraded":"2023-10-06T04:55:28.624774+0000","last_fullsized":"2023-10-06T04:55:28.624774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628576+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.902269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639321+0000","last_change":"2023-10-06T04:55:29.639321+0000","last_active":"2023-10-06T04:55:29.639321+0000","last_peered":"2023-10-06T04:55:29.639321+0000","last_clean":"2023-10-06T04:55:29.639321+0000","last_became_active":"2023-10-06T04:55:29.639057+0000","last_became_peered":"2023-10-06T04:55:29.639057+0000","last_unstale":"2023-10-06T04:55:29.639321+0000","last_undegraded":"2023-10-06T04:55:29.639321+0000","last_fullsized":"2023-10-06T04:55:29.639321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.045081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620654+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:55:28.620654+0000","last_peered":"2023-10-06T04:55:28.620654+0000","last_clean":"2023-10-06T04:55:28.620654+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:55:28.620654+0000","last_undegraded":"2023-10-06T04:55:28.620654+0000","last_fullsized":"2023-10-06T04:55:28.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632257+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:01.492662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641636+0000","last_became_peered":"2023-10-06T04:55:29.641636+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:29.769932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620664+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:55:28.620664+0000","last_peered":"2023-10-06T04:55:28.620664+0000","last_clean":"2023-10-06T04:55:28.620664+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:55:28.620664+0000","last_undegraded":"2023-10-06T04:55:28.620664+0000","last_fullsized":"2023-10-06T04:55:28.620664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.70499999999999996,"15min":0.77600000000000002},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88700000000000001,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.751,"15min":0.82799999999999996},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.90500000000000003,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":1.073,"15min":1.073},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81799999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.111,"15min":1.111},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78900000000000003,"5min":0.874,"15min":0.874},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":1.214,"15min":1.214},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.497},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.38,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60399999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.876,"15min":0.876},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63200000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.60399999999999998},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35499999999999998,"15min":0.36299999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.37,"15min":0.38},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.29899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.60699999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.64300000000000002,"15min":0.66700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63100000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85299999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.026,"15min":1.026},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.85599999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.34699999999999998,"15min":0.35499999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.373},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36699999999999999},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60699999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.624,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64400000000000002},"min":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.19500000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.54100000000000004,"5min":1.048,"15min":1.048},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.376},"min":{"1min":0.28699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.624,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.29599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69799999999999995,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82299999999999995,"5min":1.246,"15min":1.246},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.877,"15min":0.877},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89400000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.624},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306312,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065528,"statfs":{"total":96636764160,"available":96323100672,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.40999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.54800000000000004,"15min":0.60999999999999999},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.1480000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.53000000000000003,"15min":0.58599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.379,"5min":5.069,"15min":5.069},"last":0.435},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.49299999999999999},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.86099999999999999,"15min":0.83799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":14.646000000000001,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.2769999999999999,"5min":0.81799999999999995,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":14.606999999999999,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.82299999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.9219999999999999,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74199999999999999,"15min":0.79300000000000004},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4079999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76900000000000002,"15min":0.83899999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.964,"5min":6.367,"15min":6.367},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.45900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3,"5min":5,"15min":5},"last":0.71599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.313,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.752,"15min":0.85399999999999998},"min":{"1min":0.438,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82299999999999995,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.77900000000000003,"15min":0.879},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.78600000000000003,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.76200000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.70099999999999996},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80400000000000005,"5min":5.415,"15min":5.415},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65000000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":4.202,"15min":4.202},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81799999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69999999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.872,"15min":0.872},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306296,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065544,"statfs":{"total":96636764160,"available":96323117056,"internally_reserved":0,"allocated":835584,"data_stored":369015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.65400000000000003,"15min":0.71899999999999997},"min":{"1min":0.26300000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73099999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67800000000000005,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69299999999999995,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.65900000000000003,"15min":0.70499999999999996},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.81799999999999995,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63900000000000001,"15min":0.68700000000000006},"min":{"1min":0.373,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83899999999999997,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.89500000000000002,"15min":1.044},"min":{"1min":0.39500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77300000000000002,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.90500000000000003,"15min":1.0409999999999999},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72399999999999998,"5min":0.751,"15min":0.751},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55800000000000005,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:55:35.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2023-10-06T04:55:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.635164+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:55:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2023-10-06T04:55:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.634027+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2023-10-06T04:55:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630141+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:35:47.223457+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:55:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2023-10-06T04:55:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630326+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:49:32.184584+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.639206+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:56:09.194524+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.628956+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:02:54.819595+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632371+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:35:26.055360+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2023-10-06T04:55:35.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632057+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:52:14.777663+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.625915+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:36:41.269374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631584+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:53:36.692184+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629995+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:07:47.654687+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629843+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:15:31.939182+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:55:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631919+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:26:03.823215+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629422+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:09:23.853120+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631745+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632536+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2023-10-06T04:55:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.633013+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:32:29.108450+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629698+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:03:45.147340+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.628196+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:59:18.527942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.634491+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:45:29.940238+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2023-10-06T04:55:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.627719+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:12:45.969663+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.633388+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:45:10.209749+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.626849+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:41:34.163179+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629948+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:35:48.336061+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632213+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:00:11.480851+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631083+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:53:28.732154+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.638569+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:51:16.036607+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.634028+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:23:18.240347+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632422+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2023-10-06T04:55:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.633198+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630555+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:31:33.772918+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632025+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631841+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-10-06T04:55:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630956+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.626413+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:02:57.205791+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.627157+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:16:48.932063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630411+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:08:12.293583+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630470+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:15:44.776671+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-10-06T04:55:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.626510+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:37:20.849108+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631199+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:54:05.450563+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.633050+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:32:34.127918+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.628867+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:15:52.846620+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629921+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:19:03.134120+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:55:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629728+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:42:59.757486+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629380+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:02:31.703554+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629562+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:47:38.137854+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629557+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:33:34.367793+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-10-06T04:55:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630035+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:42:38.834002+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.625584+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630578+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:50:39.999464+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.625606+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:01:06.360553+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631471+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630023+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:47:37.281809+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629001+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:46:26.643350+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629031+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:06:00.263825+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629260+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:46:06.628478+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-10-06T04:55:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.628124+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:45:38.464722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.631507+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:40:37.433675+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632435+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:13:45.568423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.630556+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:23:25.364931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-10-06T04:55:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.638091+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:55:35.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-10-06T04:55:35.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.629855+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:09:36.457810+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:55:35.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-10-06T04:55:35.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.628576+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:58:39.902269+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:55:35.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2023-10-06T04:55:35.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:55:29.614770+0000', 'last_change': '2023-10-06T04:55:29.632257+0000', 'last_active': '2023-10-06T04:55:29.614770+0000', 'last_peered': '2023-10-06T04:55:29.614770+0000', 'last_clean': '2023-10-06T04:55:29.614770+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:55:29.614770+0000', 'last_undegraded': '2023-10-06T04:55:29.614770+0000', 'last_fullsized': '2023-10-06T04:55:29.614770+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'last_clean_scrub_stamp': '2023-10-06T04:55:29.614770+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:20:01.492662+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:55:35.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T04:55:35.531 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-10-06T04:55:35.529+0000 7f88276c3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:55:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.646+0000 7f816fd55700 1 Processor -- start 2023-10-06T04:55:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.647+0000 7f816fd55700 1 -- start start 2023-10-06T04:55:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.647+0000 7f816fd55700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8168154e20 0x7f81680b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.647+0000 7f816fd55700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681551f0 0x7f8168069390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.647+0000 7f816fd55700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81680b40d0 con 0x7f81681587e0 2023-10-06T04:55:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.647+0000 7f816fd55700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81680b1180 con 0x7f8168154e20 2023-10-06T04:55:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.647+0000 7f816fd55700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81680b1460 con 0x7f81681551f0 2023-10-06T04:55:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816daf1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8168154e20 0x7f81680b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816e2f2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f81681587e0 0x7f816815b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56022/0 (socket says 172.21.15.3:56022) 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816e2f2700 1 -- 172.21.15.3:0/2589010993 learned_addr learned my addr 172.21.15.3:0/2589010993 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816d2f0700 1 --2- 172.21.15.3:0/2589010993 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681551f0 0x7f8168069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816daf1700 1 -- 172.21.15.3:0/2589010993 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681551f0 msgr2=0x7f8168069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816daf1700 1 --2- 172.21.15.3:0/2589010993 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681551f0 0x7f8168069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816daf1700 1 -- 172.21.15.3:0/2589010993 >> v1:172.21.15.3:6789/0 conn(0x7f81681587e0 legacy=0x7f816815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816caef700 1 -- 172.21.15.3:0/2589010993 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7600366 0 0) 0x7f81680b40d0 con 0x7f81681587e0 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816daf1700 1 -- 172.21.15.3:0/2589010993 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680b1740 con 0x7f8168154e20 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816daf1700 1 --2- 172.21.15.3:0/2589010993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8168154e20 0x7f81680b3780 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f81640098e0 tx=0x7f81640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0842e7229fc8534 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816caef700 1 -- 172.21.15.3:0/2589010993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816400b640 con 0x7f8168154e20 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816caef700 1 -- 172.21.15.3:0/2589010993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816400b7a0 con 0x7f8168154e20 2023-10-06T04:55:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.648+0000 7f816caef700 1 -- 172.21.15.3:0/2589010993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816400bb40 con 0x7f8168154e20 2023-10-06T04:55:35.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 -- 172.21.15.3:0/2589010993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8168154e20 msgr2=0x7f81680b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 --2- 172.21.15.3:0/2589010993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8168154e20 0x7f81680b3780 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f81640098e0 tx=0x7f81640091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 -- 172.21.15.3:0/2589010993 shutdown_connections 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 --2- 172.21.15.3:0/2589010993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8168154e20 0x7f81680b3780 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 --2- 172.21.15.3:0/2589010993 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681551f0 0x7f8168069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 -- 172.21.15.3:0/2589010993 >> 172.21.15.3:0/2589010993 conn(0x7f8168067aa0 msgr2=0x7f8168067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 -- 172.21.15.3:0/2589010993 shutdown_connections 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.649+0000 7f816fd55700 1 -- 172.21.15.3:0/2589010993 wait complete. 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.651+0000 7f816fd55700 1 Processor -- start 2023-10-06T04:55:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.651+0000 7f816fd55700 1 -- start start 2023-10-06T04:55:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816fd55700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168154e20 0x7f81680ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816fd55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81681551f0 0x7f81680cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816fd55700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681587e0 0x7f8168149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816fd55700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816814b3f0 con 0x7f81681551f0 2023-10-06T04:55:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816fd55700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816814b6d0 con 0x7f8168154e20 2023-10-06T04:55:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816fd55700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816814b9b0 con 0x7f81681587e0 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81681551f0 0x7f81680cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81681551f0 0x7f81680cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58042/0 (socket says 172.21.15.3:58042) 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 -- 172.21.15.3:0/2702614469 learned_addr learned my addr 172.21.15.3:0/2702614469 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816daf1700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168154e20 0x7f81680ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816e2f2700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681587e0 0x7f8168149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 -- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681587e0 msgr2=0x7f8168149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681587e0 0x7f8168149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 -- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168154e20 msgr2=0x7f81680ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168154e20 0x7f81680ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 -- 172.21.15.3:0/2702614469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8164009420 con 0x7f81681551f0 2023-10-06T04:55:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.652+0000 7f816d2f0700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81681551f0 0x7f81680cd530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f815800ea70 tx=0x7f81580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.653+0000 7f815dffb700 1 -- 172.21.15.3:0/2702614469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815800eeb0 con 0x7f81681551f0 2023-10-06T04:55:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.653+0000 7f815dffb700 1 -- 172.21.15.3:0/2702614469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815800c850 con 0x7f81681551f0 2023-10-06T04:55:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.653+0000 7f815dffb700 1 -- 172.21.15.3:0/2702614469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815800cc30 con 0x7f81681551f0 2023-10-06T04:55:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.653+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81680af3f0 con 0x7f81681551f0 2023-10-06T04:55:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.653+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81680aec70 con 0x7f81681551f0 2023-10-06T04:55:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.654+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81680644c0 con 0x7f81681551f0 2023-10-06T04:55:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.654+0000 7f815dffb700 1 -- 172.21.15.3:0/2702614469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8158013680 con 0x7f81681551f0 2023-10-06T04:55:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.655+0000 7f815dffb700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f814c066f30 0x7f814c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.655+0000 7f815dffb700 1 -- 172.21.15.3:0/2702614469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f81580848c0 con 0x7f81681551f0 2023-10-06T04:55:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.655+0000 7f816daf1700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f814c066f30 0x7f814c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.655+0000 7f816daf1700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f814c066f30 0x7f814c0693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f81640093f0 tx=0x7f8164017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:35.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.659+0000 7f815dffb700 1 -- 172.21.15.3:0/2702614469 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81580519d0 con 0x7f81681551f0 2023-10-06T04:55:35.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.929+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f81680af8f0 con 0x7f81681551f0 2023-10-06T04:55:35.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.930+0000 7f815dffb700 1 -- 172.21.15.3:0/2702614469 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1456 (secure 0 0 0) 0x7f8158054ff0 con 0x7f81681551f0 2023-10-06T04:55:35.933 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:35.933 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":216,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":72,"num_osds":8,"num_up_osds":8,"osd_up_since":1696567930,"num_in_osds":8,"osd_in_since":1696567922,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2493693952,"bytes_avail":770600419328,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":40,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6123}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-06T04:55:11.464758+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T04:55:35.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f814c066f30 msgr2=0x7f814c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f814c066f30 0x7f814c0693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f81640093f0 tx=0x7f8164017000 comp rx=0 tx=0).stop 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81681551f0 msgr2=0x7f81680cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81681551f0 0x7f81680cd530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f815800ea70 tx=0x7f81580091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 shutdown_connections 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f814c066f30 0x7f814c0693e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168154e20 0x7f81680ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81681587e0 0x7f8168149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 --2- 172.21.15.3:0/2702614469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81681551f0 0x7f81680cd530 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 >> 172.21.15.3:0/2702614469 conn(0x7f8168067aa0 msgr2=0x7f81680ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 shutdown_connections 2023-10-06T04:55:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:35.932+0000 7f816fd55700 1 -- 172.21.15.3:0/2702614469 wait complete. 2023-10-06T04:55:35.946 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T04:55:35.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:55:36.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.063+0000 7f2c24868700 1 Processor -- start 2023-10-06T04:55:36.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.063+0000 7f2c24868700 1 -- start start 2023-10-06T04:55:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c24868700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2c201598d0 0x7f2c200b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c24868700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c200cb440 0x7f2c200b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c24868700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c200cc5c0 con 0x7f2c20159500 2023-10-06T04:55:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c24868700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c200cda60 con 0x7f2c201598d0 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c24868700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c200b2ca0 con 0x7f2c200cb440 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2c20159500 0x7f2c200cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56038/0 (socket says 172.21.15.3:56038) 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1e59c700 1 -- 172.21.15.3:0/3235133584 learned_addr learned my addr 172.21.15.3:0/3235133584 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1ed9d700 1 --2- 172.21.15.3:0/3235133584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c200cb440 0x7f2c200b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1dd9b700 1 --2- 172.21.15.3:0/3235133584 >> v2:172.21.15.153:3300/0 conn(0x7f2c201598d0 0x7f2c200b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1d59a700 1 -- 172.21.15.3:0/3235133584 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625561005 0 0) 0x7f2c200cc5c0 con 0x7f2c20159500 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1d59a700 1 -- 172.21.15.3:0/3235133584 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c0c003570 con 0x7f2c20159500 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1d59a700 1 -- 172.21.15.3:0/3235133584 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726336843 0 0) 0x7f2c0c003570 con 0x7f2c20159500 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1ed9d700 1 -- 172.21.15.3:0/3235133584 >> v2:172.21.15.153:3300/0 conn(0x7f2c201598d0 msgr2=0x7f2c200b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1ed9d700 1 --2- 172.21.15.3:0/3235133584 >> v2:172.21.15.153:3300/0 conn(0x7f2c201598d0 0x7f2c200b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1ed9d700 1 -- 172.21.15.3:0/3235133584 >> v1:172.21.15.3:6789/0 conn(0x7f2c20159500 legacy=0x7f2c200cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:36.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.064+0000 7f2c1ed9d700 1 -- 172.21.15.3:0/3235133584 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c200cb9b0 con 0x7f2c200cb440 2023-10-06T04:55:36.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c1ed9d700 1 --2- 172.21.15.3:0/3235133584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c200cb440 0x7f2c200b2760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2c180098e0 tx=0x7f2c180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9546f3c676d482 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:36.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c1d59a700 1 -- 172.21.15.3:0/3235133584 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c100017f0 con 0x7f2c200cb440 2023-10-06T04:55:36.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c1d59a700 1 -- 172.21.15.3:0/3235133584 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c18011420 con 0x7f2c200cb440 2023-10-06T04:55:36.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c1d59a700 1 -- 172.21.15.3:0/3235133584 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c180117a0 con 0x7f2c200cb440 2023-10-06T04:55:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 -- 172.21.15.3:0/3235133584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c200cb440 msgr2=0x7f2c200b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 --2- 172.21.15.3:0/3235133584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c200cb440 0x7f2c200b2760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2c180098e0 tx=0x7f2c180091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 -- 172.21.15.3:0/3235133584 shutdown_connections 2023-10-06T04:55:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 --2- 172.21.15.3:0/3235133584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c200cb440 0x7f2c200b2760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 --2- 172.21.15.3:0/3235133584 >> v2:172.21.15.153:3300/0 conn(0x7f2c201598d0 0x7f2c200b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 -- 172.21.15.3:0/3235133584 >> 172.21.15.3:0/3235133584 conn(0x7f2c20070310 msgr2=0x7f2c20070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 -- 172.21.15.3:0/3235133584 shutdown_connections 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.065+0000 7f2c24868700 1 -- 172.21.15.3:0/3235133584 wait complete. 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.066+0000 7f2c24868700 1 Processor -- start 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.066+0000 7f2c24868700 1 -- start start 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c24868700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c200cb440 0x7f2c200c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c24868700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c20159500 0x7f2c201538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c24868700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c201598d0 0x7f2c200ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c24868700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2016d330 con 0x7f2c200cb440 2023-10-06T04:55:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c24868700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c200c6040 con 0x7f2c20159500 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c24868700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c200b44c0 con 0x7f2c201598d0 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c1e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c200cb440 0x7f2c200c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c1e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c200cb440 0x7f2c200c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58044/0 (socket says 172.21.15.3:58044) 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c1e59c700 1 -- 172.21.15.3:0/4279091892 learned_addr learned my addr 172.21.15.3:0/4279091892 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c1dd9b700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c20159500 0x7f2c201538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c1ed9d700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c201598d0 0x7f2c200ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c1e59c700 1 -- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c201598d0 msgr2=0x7f2c200ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.067+0000 7f2c1e59c700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c201598d0 0x7f2c200ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c1e59c700 1 -- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c20159500 msgr2=0x7f2c201538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c1e59c700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c20159500 0x7f2c201538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c1e59c700 1 -- 172.21.15.3:0/4279091892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18009420 con 0x7f2c200cb440 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c1e59c700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c200cb440 0x7f2c200c64d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000ccc0 tx=0x7f2c10001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c167fc700 1 -- 172.21.15.3:0/4279091892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000ce70 con 0x7f2c200cb440 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c167fc700 1 -- 172.21.15.3:0/4279091892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c10004dc0 con 0x7f2c200cb440 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c200c5470 con 0x7f2c200cb440 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c167fc700 1 -- 172.21.15.3:0/4279091892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000abd0 con 0x7f2c200cb440 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.068+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c200c5a70 con 0x7f2c200cb440 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.069+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c200c5df0 con 0x7f2c200cb440 2023-10-06T04:55:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.069+0000 7f2c167fc700 1 -- 172.21.15.3:0/4279091892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c10007150 con 0x7f2c200cb440 2023-10-06T04:55:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.070+0000 7f2c167fc700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2c00066f30 0x7f2c000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.070+0000 7f2c167fc700 1 -- 172.21.15.3:0/4279091892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c100842e0 con 0x7f2c200cb440 2023-10-06T04:55:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.070+0000 7f2c1dd9b700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2c00066f30 0x7f2c000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.070+0000 7f2c1dd9b700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2c00066f30 0x7f2c000693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c200af470 tx=0x7f2c08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.074+0000 7f2c167fc700 1 -- 172.21.15.3:0/4279091892 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c100513d0 con 0x7f2c200cb440 2023-10-06T04:55:36.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.262+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c200c6180 con 0x7f2c00066f30 2023-10-06T04:55:36.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.271+0000 7f2c167fc700 1 -- 172.21.15.3:0/4279091892 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308767 (secure 0 0 0) 0x7f2c080098e0 con 0x7f2c00066f30 2023-10-06T04:55:36.277 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:36.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2c00066f30 msgr2=0x7f2c000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2c00066f30 0x7f2c000693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c200af470 tx=0x7f2c08008040 comp rx=0 tx=0).stop 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c200cb440 msgr2=0x7f2c200c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c200cb440 0x7f2c200c64d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000ccc0 tx=0x7f2c10001670 comp rx=0 tx=0).stop 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 shutdown_connections 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2c00066f30 0x7f2c000693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c201598d0 0x7f2c200ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c200cb440 0x7f2c200c64d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 --2- 172.21.15.3:0/4279091892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c20159500 0x7f2c201538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:36.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 >> 172.21.15.3:0/4279091892 conn(0x7f2c20070310 msgr2=0x7f2c20157050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:36.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 shutdown_connections 2023-10-06T04:55:36.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:36.275+0000 7f2c24868700 1 -- 172.21.15.3:0/4279091892 wait complete. 2023-10-06T04:55:36.281 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:55:36.289 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2023-10-06T04:55:35.470082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2435248,"kb_used_data":7792,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539472,"statfs":{"total":773094113280,"available":770600419328,"internally_reserved":0,"allocated":7979008,"data_stored":4185933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001309"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.635164+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634027+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630141+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:47.223457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630326+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:32.184584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.639206+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:09.194524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:54.819595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632371+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:26.055360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632057+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:14.777663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625915+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.269374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:36.692184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629995+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:47.654687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629843+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:31.939182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631919+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:03.823215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631745+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632536+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633013+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:29.108450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629698+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:45.147340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628196+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:18.527942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634491+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:29.940238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627719+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:45.969663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633388+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:10.209749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626849+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:34.163179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629948+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:48.336061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632213+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:11.480851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631083+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:28.732154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638569+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:16.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.634028+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:18.240347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632422+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:16.519230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633198+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:15.425707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630555+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.772918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632025+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631841+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644855+0000","last_change":"2023-10-06T04:55:29.644855+0000","last_active":"2023-10-06T04:55:29.644855+0000","last_peered":"2023-10-06T04:55:29.644855+0000","last_clean":"2023-10-06T04:55:29.644855+0000","last_became_active":"2023-10-06T04:55:29.644553+0000","last_became_peered":"2023-10-06T04:55:29.644553+0000","last_unstale":"2023-10-06T04:55:29.644855+0000","last_undegraded":"2023-10-06T04:55:29.644855+0000","last_fullsized":"2023-10-06T04:55:29.644855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:31.518507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630956+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:39.505501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.640284+0000","last_change":"2023-10-06T04:55:29.640284+0000","last_active":"2023-10-06T04:55:29.640284+0000","last_peered":"2023-10-06T04:55:29.640284+0000","last_clean":"2023-10-06T04:55:29.640284+0000","last_became_active":"2023-10-06T04:55:29.638752+0000","last_became_peered":"2023-10-06T04:55:29.638752+0000","last_unstale":"2023-10-06T04:55:29.640284+0000","last_undegraded":"2023-10-06T04:55:29.640284+0000","last_fullsized":"2023-10-06T04:55:29.640284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.325782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626413+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:57.205791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.646265+0000","last_change":"2023-10-06T04:55:29.646265+0000","last_active":"2023-10-06T04:55:29.646265+0000","last_peered":"2023-10-06T04:55:29.646265+0000","last_clean":"2023-10-06T04:55:29.646265+0000","last_became_active":"2023-10-06T04:55:29.645786+0000","last_became_peered":"2023-10-06T04:55:29.645786+0000","last_unstale":"2023-10-06T04:55:29.646265+0000","last_undegraded":"2023-10-06T04:55:29.646265+0000","last_fullsized":"2023-10-06T04:55:29.646265+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:11.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.627157+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:48.932063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643149+0000","last_change":"2023-10-06T04:55:29.643149+0000","last_active":"2023-10-06T04:55:29.643149+0000","last_peered":"2023-10-06T04:55:29.643149+0000","last_clean":"2023-10-06T04:55:29.643149+0000","last_became_active":"2023-10-06T04:55:29.642865+0000","last_became_peered":"2023-10-06T04:55:29.642865+0000","last_unstale":"2023-10-06T04:55:29.643149+0000","last_undegraded":"2023-10-06T04:55:29.643149+0000","last_fullsized":"2023-10-06T04:55:29.643149+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:25.603703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630411+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:12.293583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641637+0000","last_became_peered":"2023-10-06T04:55:29.641637+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:44.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630470+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:44.776671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642590+0000","last_change":"2023-10-06T04:55:29.642590+0000","last_active":"2023-10-06T04:55:29.642590+0000","last_peered":"2023-10-06T04:55:29.642590+0000","last_clean":"2023-10-06T04:55:29.642590+0000","last_became_active":"2023-10-06T04:55:29.642252+0000","last_became_peered":"2023-10-06T04:55:29.642252+0000","last_unstale":"2023-10-06T04:55:29.642590+0000","last_undegraded":"2023-10-06T04:55:29.642590+0000","last_fullsized":"2023-10-06T04:55:29.642590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:56.195511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.626510+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:20.849108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643163+0000","last_change":"2023-10-06T04:55:29.643163+0000","last_active":"2023-10-06T04:55:29.643163+0000","last_peered":"2023-10-06T04:55:29.643163+0000","last_clean":"2023-10-06T04:55:29.643163+0000","last_became_active":"2023-10-06T04:55:29.642910+0000","last_became_peered":"2023-10-06T04:55:29.642910+0000","last_unstale":"2023-10-06T04:55:29.643163+0000","last_undegraded":"2023-10-06T04:55:29.643163+0000","last_fullsized":"2023-10-06T04:55:29.643163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:35.681905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631199+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:05.450563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641803+0000","last_change":"2023-10-06T04:55:29.641803+0000","last_active":"2023-10-06T04:55:29.641803+0000","last_peered":"2023-10-06T04:55:29.641803+0000","last_clean":"2023-10-06T04:55:29.641803+0000","last_became_active":"2023-10-06T04:55:29.641484+0000","last_became_peered":"2023-10-06T04:55:29.641484+0000","last_unstale":"2023-10-06T04:55:29.641803+0000","last_undegraded":"2023-10-06T04:55:29.641803+0000","last_fullsized":"2023-10-06T04:55:29.641803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.831547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.633050+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:34.127918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.645173+0000","last_change":"2023-10-06T04:55:29.645173+0000","last_active":"2023-10-06T04:55:29.645173+0000","last_peered":"2023-10-06T04:55:29.645173+0000","last_clean":"2023-10-06T04:55:29.645173+0000","last_became_active":"2023-10-06T04:55:29.644899+0000","last_became_peered":"2023-10-06T04:55:29.644899+0000","last_unstale":"2023-10-06T04:55:29.645173+0000","last_undegraded":"2023-10-06T04:55:29.645173+0000","last_fullsized":"2023-10-06T04:55:29.645173+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:35.447530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628867+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:52.846620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639731+0000","last_change":"2023-10-06T04:55:29.639731+0000","last_active":"2023-10-06T04:55:29.639731+0000","last_peered":"2023-10-06T04:55:29.639731+0000","last_clean":"2023-10-06T04:55:29.639731+0000","last_became_active":"2023-10-06T04:55:29.639060+0000","last_became_peered":"2023-10-06T04:55:29.639060+0000","last_unstale":"2023-10-06T04:55:29.639731+0000","last_undegraded":"2023-10-06T04:55:29.639731+0000","last_fullsized":"2023-10-06T04:55:29.639731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:19.838829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629921+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:03.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.643771+0000","last_change":"2023-10-06T04:55:29.643771+0000","last_active":"2023-10-06T04:55:29.643771+0000","last_peered":"2023-10-06T04:55:29.643771+0000","last_clean":"2023-10-06T04:55:29.643771+0000","last_became_active":"2023-10-06T04:55:29.643494+0000","last_became_peered":"2023-10-06T04:55:29.643494+0000","last_unstale":"2023-10-06T04:55:29.643771+0000","last_undegraded":"2023-10-06T04:55:29.643771+0000","last_fullsized":"2023-10-06T04:55:29.643771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:14.660406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629728+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:59.757486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636609+0000","last_change":"2023-10-06T04:55:29.636609+0000","last_active":"2023-10-06T04:55:29.636609+0000","last_peered":"2023-10-06T04:55:29.636609+0000","last_clean":"2023-10-06T04:55:29.636609+0000","last_became_active":"2023-10-06T04:55:29.636160+0000","last_became_peered":"2023-10-06T04:55:29.636160+0000","last_unstale":"2023-10-06T04:55:29.636609+0000","last_undegraded":"2023-10-06T04:55:29.636609+0000","last_fullsized":"2023-10-06T04:55:29.636609+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:32.519067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629380+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:31.703554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634753+0000","last_change":"2023-10-06T04:55:29.634753+0000","last_active":"2023-10-06T04:55:29.634753+0000","last_peered":"2023-10-06T04:55:29.634753+0000","last_clean":"2023-10-06T04:55:29.634753+0000","last_became_active":"2023-10-06T04:55:29.634471+0000","last_became_peered":"2023-10-06T04:55:29.634471+0000","last_unstale":"2023-10-06T04:55:29.634753+0000","last_undegraded":"2023-10-06T04:55:29.634753+0000","last_fullsized":"2023-10-06T04:55:29.634753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:47.921784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629562+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:38.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644259+0000","last_change":"2023-10-06T04:55:29.644259+0000","last_active":"2023-10-06T04:55:29.644259+0000","last_peered":"2023-10-06T04:55:29.644259+0000","last_clean":"2023-10-06T04:55:29.644259+0000","last_became_active":"2023-10-06T04:55:29.643992+0000","last_became_peered":"2023-10-06T04:55:29.643992+0000","last_unstale":"2023-10-06T04:55:29.644259+0000","last_undegraded":"2023-10-06T04:55:29.644259+0000","last_fullsized":"2023-10-06T04:55:29.644259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:43.348964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629557+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:34.367793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644987+0000","last_change":"2023-10-06T04:55:29.644987+0000","last_active":"2023-10-06T04:55:29.644987+0000","last_peered":"2023-10-06T04:55:29.644987+0000","last_clean":"2023-10-06T04:55:29.644987+0000","last_became_active":"2023-10-06T04:55:29.644620+0000","last_became_peered":"2023-10-06T04:55:29.644620+0000","last_unstale":"2023-10-06T04:55:29.644987+0000","last_undegraded":"2023-10-06T04:55:29.644987+0000","last_fullsized":"2023-10-06T04:55:29.644987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:57.789632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630035+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:38.834002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637724+0000","last_change":"2023-10-06T04:55:29.637724+0000","last_active":"2023-10-06T04:55:29.637724+0000","last_peered":"2023-10-06T04:55:29.637724+0000","last_clean":"2023-10-06T04:55:29.637724+0000","last_became_active":"2023-10-06T04:55:29.637235+0000","last_became_peered":"2023-10-06T04:55:29.637235+0000","last_unstale":"2023-10-06T04:55:29.637724+0000","last_undegraded":"2023-10-06T04:55:29.637724+0000","last_fullsized":"2023-10-06T04:55:29.637724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:27.721875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625584+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:00.082954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637907+0000","last_change":"2023-10-06T04:55:29.637907+0000","last_active":"2023-10-06T04:55:29.637907+0000","last_peered":"2023-10-06T04:55:29.637907+0000","last_clean":"2023-10-06T04:55:29.637907+0000","last_became_active":"2023-10-06T04:55:29.637626+0000","last_became_peered":"2023-10-06T04:55:29.637626+0000","last_unstale":"2023-10-06T04:55:29.637907+0000","last_undegraded":"2023-10-06T04:55:29.637907+0000","last_fullsized":"2023-10-06T04:55:29.637907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:44.959384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630578+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:39.999464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641635+0000","last_change":"2023-10-06T04:55:29.641635+0000","last_active":"2023-10-06T04:55:29.641635+0000","last_peered":"2023-10-06T04:55:29.641635+0000","last_clean":"2023-10-06T04:55:29.641635+0000","last_became_active":"2023-10-06T04:55:29.641370+0000","last_became_peered":"2023-10-06T04:55:29.641370+0000","last_unstale":"2023-10-06T04:55:29.641635+0000","last_undegraded":"2023-10-06T04:55:29.641635+0000","last_fullsized":"2023-10-06T04:55:29.641635+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.625606+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:06.360553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644387+0000","last_change":"2023-10-06T04:55:29.644387+0000","last_active":"2023-10-06T04:55:29.644387+0000","last_peered":"2023-10-06T04:55:29.644387+0000","last_clean":"2023-10-06T04:55:29.644387+0000","last_became_active":"2023-10-06T04:55:29.644059+0000","last_became_peered":"2023-10-06T04:55:29.644059+0000","last_unstale":"2023-10-06T04:55:29.644387+0000","last_undegraded":"2023-10-06T04:55:29.644387+0000","last_fullsized":"2023-10-06T04:55:29.644387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631471+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:55.948261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644897+0000","last_change":"2023-10-06T04:55:29.644897+0000","last_active":"2023-10-06T04:55:29.644897+0000","last_peered":"2023-10-06T04:55:29.644897+0000","last_clean":"2023-10-06T04:55:29.644897+0000","last_became_active":"2023-10-06T04:55:29.644631+0000","last_became_peered":"2023-10-06T04:55:29.644631+0000","last_unstale":"2023-10-06T04:55:29.644897+0000","last_undegraded":"2023-10-06T04:55:29.644897+0000","last_fullsized":"2023-10-06T04:55:29.644897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630023+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:37.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.642477+0000","last_change":"2023-10-06T04:55:29.642477+0000","last_active":"2023-10-06T04:55:29.642477+0000","last_peered":"2023-10-06T04:55:29.642477+0000","last_clean":"2023-10-06T04:55:29.642477+0000","last_became_active":"2023-10-06T04:55:29.642202+0000","last_became_peered":"2023-10-06T04:55:29.642202+0000","last_unstale":"2023-10-06T04:55:29.642477+0000","last_undegraded":"2023-10-06T04:55:29.642477+0000","last_fullsized":"2023-10-06T04:55:29.642477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:14.338401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629001+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:26.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644875+0000","last_change":"2023-10-06T04:55:29.644875+0000","last_active":"2023-10-06T04:55:29.644875+0000","last_peered":"2023-10-06T04:55:29.644875+0000","last_clean":"2023-10-06T04:55:29.644875+0000","last_became_active":"2023-10-06T04:55:29.644568+0000","last_became_peered":"2023-10-06T04:55:29.644568+0000","last_unstale":"2023-10-06T04:55:29.644875+0000","last_undegraded":"2023-10-06T04:55:29.644875+0000","last_fullsized":"2023-10-06T04:55:29.644875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:08.690122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629031+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.263825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641632+0000","last_change":"2023-10-06T04:55:29.641632+0000","last_active":"2023-10-06T04:55:29.641632+0000","last_peered":"2023-10-06T04:55:29.641632+0000","last_clean":"2023-10-06T04:55:29.641632+0000","last_became_active":"2023-10-06T04:55:29.641299+0000","last_became_peered":"2023-10-06T04:55:29.641299+0000","last_unstale":"2023-10-06T04:55:29.641632+0000","last_undegraded":"2023-10-06T04:55:29.641632+0000","last_fullsized":"2023-10-06T04:55:29.641632+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:42.695331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629260+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:06.628478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634678+0000","last_change":"2023-10-06T04:55:29.634678+0000","last_active":"2023-10-06T04:55:29.634678+0000","last_peered":"2023-10-06T04:55:29.634678+0000","last_clean":"2023-10-06T04:55:29.634678+0000","last_became_active":"2023-10-06T04:55:29.634400+0000","last_became_peered":"2023-10-06T04:55:29.634400+0000","last_unstale":"2023-10-06T04:55:29.634678+0000","last_undegraded":"2023-10-06T04:55:29.634678+0000","last_fullsized":"2023-10-06T04:55:29.634678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:04.440283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628124+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:38.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.644649+0000","last_change":"2023-10-06T04:55:29.644649+0000","last_active":"2023-10-06T04:55:29.644649+0000","last_peered":"2023-10-06T04:55:29.644649+0000","last_clean":"2023-10-06T04:55:29.644649+0000","last_became_active":"2023-10-06T04:55:29.644333+0000","last_became_peered":"2023-10-06T04:55:29.644333+0000","last_unstale":"2023-10-06T04:55:29.644649+0000","last_undegraded":"2023-10-06T04:55:29.644649+0000","last_fullsized":"2023-10-06T04:55:29.644649+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:36.941087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620333+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:55:28.620333+0000","last_peered":"2023-10-06T04:55:28.620333+0000","last_clean":"2023-10-06T04:55:28.620333+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:55:28.620333+0000","last_undegraded":"2023-10-06T04:55:28.620333+0000","last_fullsized":"2023-10-06T04:55:28.620333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.631507+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:37.433675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.628693+0000","last_change":"2023-10-06T04:55:29.628693+0000","last_active":"2023-10-06T04:55:29.628693+0000","last_peered":"2023-10-06T04:55:29.628693+0000","last_clean":"2023-10-06T04:55:29.628693+0000","last_became_active":"2023-10-06T04:55:29.627557+0000","last_became_peered":"2023-10-06T04:55:29.627557+0000","last_unstale":"2023-10-06T04:55:29.628693+0000","last_undegraded":"2023-10-06T04:55:29.628693+0000","last_fullsized":"2023-10-06T04:55:29.628693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:58.747206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.617715+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:55:28.617715+0000","last_peered":"2023-10-06T04:55:28.617715+0000","last_clean":"2023-10-06T04:55:28.617715+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:55:28.617715+0000","last_undegraded":"2023-10-06T04:55:28.617715+0000","last_fullsized":"2023-10-06T04:55:28.617715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632435+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.568423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.636254+0000","last_change":"2023-10-06T04:55:29.636254+0000","last_active":"2023-10-06T04:55:29.636254+0000","last_peered":"2023-10-06T04:55:29.636254+0000","last_clean":"2023-10-06T04:55:29.636254+0000","last_became_active":"2023-10-06T04:55:29.635966+0000","last_became_peered":"2023-10-06T04:55:29.635966+0000","last_unstale":"2023-10-06T04:55:29.636254+0000","last_undegraded":"2023-10-06T04:55:29.636254+0000","last_fullsized":"2023-10-06T04:55:29.636254+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:09.863401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.622100+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:55:28.622100+0000","last_peered":"2023-10-06T04:55:28.622100+0000","last_clean":"2023-10-06T04:55:28.622100+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:55:28.622100+0000","last_undegraded":"2023-10-06T04:55:28.622100+0000","last_fullsized":"2023-10-06T04:55:28.622100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":202,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623115+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:55:28.623115+0000","last_peered":"2023-10-06T04:55:28.623115+0000","last_clean":"2023-10-06T04:55:28.623115+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:55:28.623115+0000","last_undegraded":"2023-10-06T04:55:28.623115+0000","last_fullsized":"2023-10-06T04:55:28.623115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.630556+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:25.364931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.634448+0000","last_change":"2023-10-06T04:55:29.634448+0000","last_active":"2023-10-06T04:55:29.634448+0000","last_peered":"2023-10-06T04:55:29.634448+0000","last_clean":"2023-10-06T04:55:29.634448+0000","last_became_active":"2023-10-06T04:55:29.634144+0000","last_became_peered":"2023-10-06T04:55:29.634144+0000","last_unstale":"2023-10-06T04:55:29.634448+0000","last_undegraded":"2023-10-06T04:55:29.634448+0000","last_fullsized":"2023-10-06T04:55:29.634448+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:14.628720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623209+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:55:28.623209+0000","last_peered":"2023-10-06T04:55:28.623209+0000","last_clean":"2023-10-06T04:55:28.623209+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:55:28.623209+0000","last_undegraded":"2023-10-06T04:55:28.623209+0000","last_fullsized":"2023-10-06T04:55:28.623209+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.638091+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:15.638999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.637914+0000","last_change":"2023-10-06T04:55:29.637914+0000","last_active":"2023-10-06T04:55:29.637914+0000","last_peered":"2023-10-06T04:55:29.637914+0000","last_clean":"2023-10-06T04:55:29.637914+0000","last_became_active":"2023-10-06T04:55:29.637640+0000","last_became_peered":"2023-10-06T04:55:29.637640+0000","last_unstale":"2023-10-06T04:55:29.637914+0000","last_undegraded":"2023-10-06T04:55:29.637914+0000","last_fullsized":"2023-10-06T04:55:29.637914+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:15.026328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.623208+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:55:28.623208+0000","last_peered":"2023-10-06T04:55:28.623208+0000","last_clean":"2023-10-06T04:55:28.623208+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:55:28.623208+0000","last_undegraded":"2023-10-06T04:55:28.623208+0000","last_fullsized":"2023-10-06T04:55:28.623208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.629855+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:36.457810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641718+0000","last_change":"2023-10-06T04:55:29.641718+0000","last_active":"2023-10-06T04:55:29.641718+0000","last_peered":"2023-10-06T04:55:29.641718+0000","last_clean":"2023-10-06T04:55:29.641718+0000","last_became_active":"2023-10-06T04:55:29.641423+0000","last_became_peered":"2023-10-06T04:55:29.641423+0000","last_unstale":"2023-10-06T04:55:29.641718+0000","last_undegraded":"2023-10-06T04:55:29.641718+0000","last_fullsized":"2023-10-06T04:55:29.641718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:26.283304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.624774+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:55:28.624774+0000","last_peered":"2023-10-06T04:55:28.624774+0000","last_clean":"2023-10-06T04:55:28.624774+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:55:28.624774+0000","last_undegraded":"2023-10-06T04:55:28.624774+0000","last_fullsized":"2023-10-06T04:55:28.624774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.628576+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.902269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.639321+0000","last_change":"2023-10-06T04:55:29.639321+0000","last_active":"2023-10-06T04:55:29.639321+0000","last_peered":"2023-10-06T04:55:29.639321+0000","last_clean":"2023-10-06T04:55:29.639321+0000","last_became_active":"2023-10-06T04:55:29.639057+0000","last_became_peered":"2023-10-06T04:55:29.639057+0000","last_unstale":"2023-10-06T04:55:29.639321+0000","last_undegraded":"2023-10-06T04:55:29.639321+0000","last_fullsized":"2023-10-06T04:55:29.639321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.045081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620654+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:55:28.620654+0000","last_peered":"2023-10-06T04:55:28.620654+0000","last_clean":"2023-10-06T04:55:28.620654+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:55:28.620654+0000","last_undegraded":"2023-10-06T04:55:28.620654+0000","last_fullsized":"2023-10-06T04:55:28.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:55:29.614770+0000","last_change":"2023-10-06T04:55:29.632257+0000","last_active":"2023-10-06T04:55:29.614770+0000","last_peered":"2023-10-06T04:55:29.614770+0000","last_clean":"2023-10-06T04:55:29.614770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:55:29.614770+0000","last_undegraded":"2023-10-06T04:55:29.614770+0000","last_fullsized":"2023-10-06T04:55:29.614770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:01.492662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T04:55:29.641928+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:29.641928+0000","last_peered":"2023-10-06T04:55:29.641928+0000","last_clean":"2023-10-06T04:55:29.641928+0000","last_became_active":"2023-10-06T04:55:29.641636+0000","last_became_peered":"2023-10-06T04:55:29.641636+0000","last_unstale":"2023-10-06T04:55:29.641928+0000","last_undegraded":"2023-10-06T04:55:29.641928+0000","last_fullsized":"2023-10-06T04:55:29.641928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:29.769932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T04:55:28.620664+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:55:28.620664+0000","last_peered":"2023-10-06T04:55:28.620664+0000","last_clean":"2023-10-06T04:55:28.620664+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:55:28.620664+0000","last_undegraded":"2023-10-06T04:55:28.620664+0000","last_fullsized":"2023-10-06T04:55:28.620664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.70499999999999996,"15min":0.77600000000000002},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88700000000000001,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.751,"15min":0.82799999999999996},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.90500000000000003,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":1.073,"15min":1.073},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81799999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.111,"15min":1.111},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78900000000000003,"5min":0.874,"15min":0.874},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":1.214,"15min":1.214},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.497},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.38,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60399999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.876,"15min":0.876},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63200000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.60399999999999998},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":1441792,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35499999999999998,"15min":0.36299999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.37,"15min":0.38},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.29899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.60699999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.64300000000000002,"15min":0.66700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63100000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85299999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.026,"15min":1.026},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.85599999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.34699999999999998,"15min":0.35499999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.373},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36699999999999999},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60699999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.624,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64400000000000002},"min":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.19500000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.54100000000000004,"5min":1.048,"15min":1.048},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.376},"min":{"1min":0.28699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.624,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.29599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69799999999999995,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82299999999999995,"5min":1.246,"15min":1.246},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.877,"15min":0.877},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89400000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.624},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306312,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065528,"statfs":{"total":96636764160,"available":96323100672,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.40999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.54800000000000004,"15min":0.60999999999999999},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.1480000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.53000000000000003,"15min":0.58599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.379,"5min":5.069,"15min":5.069},"last":0.435},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.49299999999999999},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.86099999999999999,"15min":0.83799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":14.646000000000001,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.2769999999999999,"5min":0.81799999999999995,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":14.606999999999999,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.82299999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.9219999999999999,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74199999999999999,"15min":0.79300000000000004},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4079999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76900000000000002,"15min":0.83899999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.964,"5min":6.367,"15min":6.367},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.45900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3,"5min":5,"15min":5},"last":0.71599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302216,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069624,"statfs":{"total":96636764160,"available":96327294976,"internally_reserved":0,"allocated":851968,"data_stored":376984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.313,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.752,"15min":0.85399999999999998},"min":{"1min":0.438,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82299999999999995,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.77900000000000003,"15min":0.879},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.78600000000000003,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.76200000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.70099999999999996},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80400000000000005,"5min":5.415,"15min":5.415},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65000000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":4.202,"15min":4.202},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81799999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69999999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.872,"15min":0.872},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306296,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065544,"statfs":{"total":96636764160,"available":96323117056,"internally_reserved":0,"allocated":835584,"data_stored":369015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.65400000000000003,"15min":0.71899999999999997},"min":{"1min":0.26300000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73099999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67800000000000005,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69299999999999995,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.65900000000000003,"15min":0.70499999999999996},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.81799999999999995,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63900000000000001,"15min":0.68700000000000006},"min":{"1min":0.373,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83899999999999997,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.89500000000000002,"15min":1.044},"min":{"1min":0.39500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77300000000000002,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.90500000000000003,"15min":1.0409999999999999},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72399999999999998,"5min":0.751,"15min":0.751},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55800000000000005,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:55:39.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:55:39.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.429+0000 7ffb5f59e700 1 Processor -- start 2023-10-06T04:55:39.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.429+0000 7ffb5f59e700 1 -- start start 2023-10-06T04:55:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb600cb3b0 0x7ffb600cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb600cb780 0x7ffb60158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb600cf130 con 0x7ffb60159500 2023-10-06T04:55:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5f59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb600ce520 con 0x7ffb600cb780 2023-10-06T04:55:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb600ce800 con 0x7ffb600cb3b0 2023-10-06T04:55:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb600cb780 0x7ffb60158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffb60159500 0x7ffb600cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56042/0 (socket says 172.21.15.3:56042) 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.430+0000 7ffb5ed9d700 1 -- 172.21.15.3:0/3575989939 learned_addr learned my addr 172.21.15.3:0/3575989939 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5e59c700 1 --2- 172.21.15.3:0/3575989939 >> v2:172.21.15.153:3300/0 conn(0x7ffb600cb3b0 0x7ffb600cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5dd9b700 1 -- 172.21.15.3:0/3575989939 >> v2:172.21.15.153:3300/0 conn(0x7ffb600cb3b0 msgr2=0x7ffb600cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5dd9b700 1 --2- 172.21.15.3:0/3575989939 >> v2:172.21.15.153:3300/0 conn(0x7ffb600cb3b0 0x7ffb600cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5dd9b700 1 -- 172.21.15.3:0/3575989939 >> v1:172.21.15.3:6789/0 conn(0x7ffb60159500 legacy=0x7ffb600cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5dd9b700 1 -- 172.21.15.3:0/3575989939 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600ceae0 con 0x7ffb600cb780 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5d59a700 1 -- 172.21.15.3:0/3575989939 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245935191 0 0) 0x7ffb600cf130 con 0x7ffb60159500 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5e59c700 1 --2- 172.21.15.3:0/3575989939 >> v2:172.21.15.153:3300/0 conn(0x7ffb600cb3b0 0x7ffb600cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5dd9b700 1 --2- 172.21.15.3:0/3575989939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb600cb780 0x7ffb60158fc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffb600893a0 tx=0x7ffb480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852dfcb3879d3a8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5d59a700 1 -- 172.21.15.3:0/3575989939 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800b6b0 con 0x7ffb600cb780 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5d59a700 1 -- 172.21.15.3:0/3575989939 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4800b810 con 0x7ffb600cb780 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5d59a700 1 -- 172.21.15.3:0/3575989939 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800bbb0 con 0x7ffb600cb780 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5f59e700 1 -- 172.21.15.3:0/3575989939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb600cb780 msgr2=0x7ffb60158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/3575989939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb600cb780 0x7ffb60158fc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffb600893a0 tx=0x7ffb480091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5f59e700 1 -- 172.21.15.3:0/3575989939 shutdown_connections 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/3575989939 >> v2:172.21.15.153:3300/0 conn(0x7ffb600cb3b0 0x7ffb600cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/3575989939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb600cb780 0x7ffb60158fc0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.431+0000 7ffb5f59e700 1 -- 172.21.15.3:0/3575989939 >> 172.21.15.3:0/3575989939 conn(0x7ffb60070310 msgr2=0x7ffb60070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.432+0000 7ffb5f59e700 1 -- 172.21.15.3:0/3575989939 shutdown_connections 2023-10-06T04:55:39.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.432+0000 7ffb5f59e700 1 -- 172.21.15.3:0/3575989939 wait complete. 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 Processor -- start 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 -- start start 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb600cb3b0 0x7ffb600c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cb780 0x7ffb60153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb60159500 0x7ffb600b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60154c10 con 0x7ffb60159500 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601550a0 con 0x7ffb600cb780 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600ca190 con 0x7ffb600cb3b0 2023-10-06T04:55:39.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.433+0000 7ffb5ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb60159500 0x7ffb600b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb60159500 0x7ffb600b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58068/0 (socket says 172.21.15.3:58068) 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 -- 172.21.15.3:0/623770419 learned_addr learned my addr 172.21.15.3:0/623770419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5dd9b700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cb780 0x7ffb60153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5e59c700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb600cb3b0 0x7ffb600c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 -- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb600cb3b0 msgr2=0x7ffb600c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb600cb3b0 0x7ffb600c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 -- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cb780 msgr2=0x7ffb60153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cb780 0x7ffb60153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 -- 172.21.15.3:0/623770419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb48009420 con 0x7ffb60159500 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5dd9b700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cb780 0x7ffb60153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5ed9d700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb60159500 0x7ffb600b3ad0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c530 tx=0x7ffb58007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:39.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb567fc700 1 -- 172.21.15.3:0/623770419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58005b30 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb567fc700 1 -- 172.21.15.3:0/623770419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58005c90 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb60154000 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb567fc700 1 -- 172.21.15.3:0/623770419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58007320 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.434+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601545c0 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.435+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb60154d80 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.436+0000 7ffb567fc700 1 -- 172.21.15.3:0/623770419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb5801b3b0 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.436+0000 7ffb567fc700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb40066f30 0x7ffb400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.436+0000 7ffb567fc700 1 -- 172.21.15.3:0/623770419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffb58083bb0 con 0x7ffb60159500 2023-10-06T04:55:39.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.436+0000 7ffb5e59c700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb40066f30 0x7ffb400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.437+0000 7ffb5e59c700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb40066f30 0x7ffb400693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb600cdae0 tx=0x7ffb50007710 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:39.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.440+0000 7ffb567fc700 1 -- 172.21.15.3:0/623770419 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb58050cc0 con 0x7ffb60159500 2023-10-06T04:55:39.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.634+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb6004c380 con 0x7ffb40066f30 2023-10-06T04:55:39.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.643+0000 7ffb567fc700 1 -- 172.21.15.3:0/623770419 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311556 (secure 0 0 0) 0x7ffb6004c380 con 0x7ffb40066f30 2023-10-06T04:55:39.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb40066f30 msgr2=0x7ffb400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb40066f30 0x7ffb400693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb600cdae0 tx=0x7ffb50007710 comp rx=0 tx=0).stop 2023-10-06T04:55:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb60159500 msgr2=0x7ffb600b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb60159500 0x7ffb600b3ad0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c530 tx=0x7ffb58007f90 comp rx=0 tx=0).stop 2023-10-06T04:55:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 shutdown_connections 2023-10-06T04:55:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb40066f30 0x7ffb400693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb60159500 0x7ffb600b3ad0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb600cb3b0 0x7ffb600c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 --2- 172.21.15.3:0/623770419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cb780 0x7ffb60153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.646+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 >> 172.21.15.3:0/623770419 conn(0x7ffb60070310 msgr2=0x7ffb600b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.647+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 shutdown_connections 2023-10-06T04:55:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.647+0000 7ffb5f59e700 1 -- 172.21.15.3:0/623770419 wait complete. 2023-10-06T04:55:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:55:39.660 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-10-06T04:55:39.471201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2435516,"kb_used_data":8060,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539204,"statfs":{"total":773094113280,"available":770600144896,"internally_reserved":0,"allocated":8253440,"data_stored":4307217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.837414"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.651709+0000","last_change":"2023-10-06T04:55:30.648817+0000","last_active":"2023-10-06T04:55:30.651709+0000","last_peered":"2023-10-06T04:55:30.651709+0000","last_clean":"2023-10-06T04:55:30.651709+0000","last_became_active":"2023-10-06T04:55:30.648515+0000","last_became_peered":"2023-10-06T04:55:30.648515+0000","last_unstale":"2023-10-06T04:55:30.651709+0000","last_undegraded":"2023-10-06T04:55:30.651709+0000","last_fullsized":"2023-10-06T04:55:30.651709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:34.127918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650723+0000","last_change":"2023-10-06T04:55:30.646202+0000","last_active":"2023-10-06T04:55:30.650723+0000","last_peered":"2023-10-06T04:55:30.650723+0000","last_clean":"2023-10-06T04:55:30.650723+0000","last_became_active":"2023-10-06T04:55:30.645816+0000","last_became_peered":"2023-10-06T04:55:30.645816+0000","last_unstale":"2023-10-06T04:55:30.650723+0000","last_undegraded":"2023-10-06T04:55:30.650723+0000","last_fullsized":"2023-10-06T04:55:30.650723+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:03.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.654605+0000","last_change":"2023-10-06T04:55:30.652509+0000","last_active":"2023-10-06T04:55:30.654605+0000","last_peered":"2023-10-06T04:55:30.654605+0000","last_clean":"2023-10-06T04:55:30.654605+0000","last_became_active":"2023-10-06T04:55:30.652011+0000","last_became_peered":"2023-10-06T04:55:30.652011+0000","last_unstale":"2023-10-06T04:55:30.654605+0000","last_undegraded":"2023-10-06T04:55:30.654605+0000","last_fullsized":"2023-10-06T04:55:30.654605+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:20.849108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.648189+0000","last_change":"2023-10-06T04:55:30.643427+0000","last_active":"2023-10-06T04:55:30.648189+0000","last_peered":"2023-10-06T04:55:30.648189+0000","last_clean":"2023-10-06T04:55:30.648189+0000","last_became_active":"2023-10-06T04:55:30.643062+0000","last_became_peered":"2023-10-06T04:55:30.643062+0000","last_unstale":"2023-10-06T04:55:30.648189+0000","last_undegraded":"2023-10-06T04:55:30.648189+0000","last_fullsized":"2023-10-06T04:55:30.648189+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:40.707668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.649587+0000","last_change":"2023-10-06T04:55:30.642468+0000","last_active":"2023-10-06T04:55:30.649587+0000","last_peered":"2023-10-06T04:55:30.649587+0000","last_clean":"2023-10-06T04:55:30.649587+0000","last_became_active":"2023-10-06T04:55:30.642176+0000","last_became_peered":"2023-10-06T04:55:30.642176+0000","last_unstale":"2023-10-06T04:55:30.649587+0000","last_undegraded":"2023-10-06T04:55:30.649587+0000","last_fullsized":"2023-10-06T04:55:30.649587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:44.383654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655945+0000","last_change":"2023-10-06T04:55:30.642732+0000","last_active":"2023-10-06T04:55:30.655945+0000","last_peered":"2023-10-06T04:55:30.655945+0000","last_clean":"2023-10-06T04:55:30.655945+0000","last_became_active":"2023-10-06T04:55:30.642481+0000","last_became_peered":"2023-10-06T04:55:30.642481+0000","last_unstale":"2023-10-06T04:55:30.655945+0000","last_undegraded":"2023-10-06T04:55:30.655945+0000","last_fullsized":"2023-10-06T04:55:30.655945+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:26.177512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653251+0000","last_change":"2023-10-06T04:55:30.648514+0000","last_active":"2023-10-06T04:55:30.653251+0000","last_peered":"2023-10-06T04:55:30.653251+0000","last_clean":"2023-10-06T04:55:30.653251+0000","last_became_active":"2023-10-06T04:55:30.646634+0000","last_became_peered":"2023-10-06T04:55:30.646634+0000","last_unstale":"2023-10-06T04:55:30.653251+0000","last_undegraded":"2023-10-06T04:55:30.653251+0000","last_fullsized":"2023-10-06T04:55:30.653251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.772918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.651562+0000","last_change":"2023-10-06T04:55:30.646333+0000","last_active":"2023-10-06T04:55:30.651562+0000","last_peered":"2023-10-06T04:55:30.651562+0000","last_clean":"2023-10-06T04:55:30.651562+0000","last_became_active":"2023-10-06T04:55:30.641257+0000","last_became_peered":"2023-10-06T04:55:30.641257+0000","last_unstale":"2023-10-06T04:55:30.651562+0000","last_undegraded":"2023-10-06T04:55:30.651562+0000","last_fullsized":"2023-10-06T04:55:30.651562+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:20.460008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657004+0000","last_change":"2023-10-06T04:55:30.656533+0000","last_active":"2023-10-06T04:55:30.657004+0000","last_peered":"2023-10-06T04:55:30.657004+0000","last_clean":"2023-10-06T04:55:30.657004+0000","last_became_active":"2023-10-06T04:55:30.656238+0000","last_became_peered":"2023-10-06T04:55:30.656238+0000","last_unstale":"2023-10-06T04:55:30.657004+0000","last_undegraded":"2023-10-06T04:55:30.657004+0000","last_fullsized":"2023-10-06T04:55:30.657004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:31.703554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.643613+0000","last_change":"2023-10-06T04:55:30.641315+0000","last_active":"2023-10-06T04:55:30.643613+0000","last_peered":"2023-10-06T04:55:30.643613+0000","last_clean":"2023-10-06T04:55:30.643613+0000","last_became_active":"2023-10-06T04:55:30.641021+0000","last_became_peered":"2023-10-06T04:55:30.641021+0000","last_unstale":"2023-10-06T04:55:30.643613+0000","last_undegraded":"2023-10-06T04:55:30.643613+0000","last_fullsized":"2023-10-06T04:55:30.643613+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:36.457810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.660606+0000","last_change":"2023-10-06T04:55:30.658578+0000","last_active":"2023-10-06T04:55:30.660606+0000","last_peered":"2023-10-06T04:55:30.660606+0000","last_clean":"2023-10-06T04:55:30.660606+0000","last_became_active":"2023-10-06T04:55:30.658161+0000","last_became_peered":"2023-10-06T04:55:30.658161+0000","last_unstale":"2023-10-06T04:55:30.660606+0000","last_undegraded":"2023-10-06T04:55:30.660606+0000","last_fullsized":"2023-10-06T04:55:30.660606+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:56.908379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.648216+0000","last_change":"2023-10-06T04:55:30.644662+0000","last_active":"2023-10-06T04:55:30.648216+0000","last_peered":"2023-10-06T04:55:30.648216+0000","last_clean":"2023-10-06T04:55:30.648216+0000","last_became_active":"2023-10-06T04:55:30.644392+0000","last_became_peered":"2023-10-06T04:55:30.644392+0000","last_unstale":"2023-10-06T04:55:30.648216+0000","last_undegraded":"2023-10-06T04:55:30.648216+0000","last_fullsized":"2023-10-06T04:55:30.648216+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:48.336061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.658729+0000","last_change":"2023-10-06T04:55:30.655175+0000","last_active":"2023-10-06T04:55:30.658729+0000","last_peered":"2023-10-06T04:55:30.658729+0000","last_clean":"2023-10-06T04:55:30.658729+0000","last_became_active":"2023-10-06T04:55:30.654025+0000","last_became_peered":"2023-10-06T04:55:30.654025+0000","last_unstale":"2023-10-06T04:55:30.658729+0000","last_undegraded":"2023-10-06T04:55:30.658729+0000","last_fullsized":"2023-10-06T04:55:30.658729+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:33.529347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655925+0000","last_change":"2023-10-06T04:55:30.642622+0000","last_active":"2023-10-06T04:55:30.655925+0000","last_peered":"2023-10-06T04:55:30.655925+0000","last_clean":"2023-10-06T04:55:30.655925+0000","last_became_active":"2023-10-06T04:55:30.642355+0000","last_became_peered":"2023-10-06T04:55:30.642355+0000","last_unstale":"2023-10-06T04:55:30.655925+0000","last_undegraded":"2023-10-06T04:55:30.655925+0000","last_fullsized":"2023-10-06T04:55:30.655925+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:05.703934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653011+0000","last_change":"2023-10-06T04:55:30.649807+0000","last_active":"2023-10-06T04:55:30.653011+0000","last_peered":"2023-10-06T04:55:30.653011+0000","last_clean":"2023-10-06T04:55:30.653011+0000","last_became_active":"2023-10-06T04:55:30.649531+0000","last_became_peered":"2023-10-06T04:55:30.649531+0000","last_unstale":"2023-10-06T04:55:30.653011+0000","last_undegraded":"2023-10-06T04:55:30.653011+0000","last_fullsized":"2023-10-06T04:55:30.653011+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.651682+0000","last_change":"2023-10-06T04:55:30.648818+0000","last_active":"2023-10-06T04:55:30.651682+0000","last_peered":"2023-10-06T04:55:30.651682+0000","last_clean":"2023-10-06T04:55:30.651682+0000","last_became_active":"2023-10-06T04:55:30.648531+0000","last_became_peered":"2023-10-06T04:55:30.648531+0000","last_unstale":"2023-10-06T04:55:30.651682+0000","last_undegraded":"2023-10-06T04:55:30.651682+0000","last_fullsized":"2023-10-06T04:55:30.651682+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:10.873681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.652160+0000","last_change":"2023-10-06T04:55:30.646504+0000","last_active":"2023-10-06T04:55:30.652160+0000","last_peered":"2023-10-06T04:55:30.652160+0000","last_clean":"2023-10-06T04:55:30.652160+0000","last_became_active":"2023-10-06T04:55:30.645989+0000","last_became_peered":"2023-10-06T04:55:30.645989+0000","last_unstale":"2023-10-06T04:55:30.652160+0000","last_undegraded":"2023-10-06T04:55:30.652160+0000","last_fullsized":"2023-10-06T04:55:30.652160+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:43.705610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.647149+0000","last_change":"2023-10-06T04:55:30.641268+0000","last_active":"2023-10-06T04:55:30.647149+0000","last_peered":"2023-10-06T04:55:30.647149+0000","last_clean":"2023-10-06T04:55:30.647149+0000","last_became_active":"2023-10-06T04:55:30.641008+0000","last_became_peered":"2023-10-06T04:55:30.641008+0000","last_unstale":"2023-10-06T04:55:30.647149+0000","last_undegraded":"2023-10-06T04:55:30.647149+0000","last_fullsized":"2023-10-06T04:55:30.647149+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.269374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655737+0000","last_change":"2023-10-06T04:55:30.640585+0000","last_active":"2023-10-06T04:55:30.655737+0000","last_peered":"2023-10-06T04:55:30.655737+0000","last_clean":"2023-10-06T04:55:30.655737+0000","last_became_active":"2023-10-06T04:55:30.640270+0000","last_became_peered":"2023-10-06T04:55:30.640270+0000","last_unstale":"2023-10-06T04:55:30.655737+0000","last_undegraded":"2023-10-06T04:55:30.655737+0000","last_fullsized":"2023-10-06T04:55:30.655737+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:56.025744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655354+0000","last_change":"2023-10-06T04:55:30.643742+0000","last_active":"2023-10-06T04:55:30.655354+0000","last_peered":"2023-10-06T04:55:30.655354+0000","last_clean":"2023-10-06T04:55:30.655354+0000","last_became_active":"2023-10-06T04:55:30.642845+0000","last_became_peered":"2023-10-06T04:55:30.642845+0000","last_unstale":"2023-10-06T04:55:30.655354+0000","last_undegraded":"2023-10-06T04:55:30.655354+0000","last_fullsized":"2023-10-06T04:55:30.655354+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:37.111226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.648040+0000","last_change":"2023-10-06T04:55:30.642476+0000","last_active":"2023-10-06T04:55:30.648040+0000","last_peered":"2023-10-06T04:55:30.648040+0000","last_clean":"2023-10-06T04:55:30.648040+0000","last_became_active":"2023-10-06T04:55:30.641298+0000","last_became_peered":"2023-10-06T04:55:30.641298+0000","last_unstale":"2023-10-06T04:55:30.648040+0000","last_undegraded":"2023-10-06T04:55:30.648040+0000","last_fullsized":"2023-10-06T04:55:30.648040+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:32.184584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657833+0000","last_change":"2023-10-06T04:55:30.655071+0000","last_active":"2023-10-06T04:55:30.657833+0000","last_peered":"2023-10-06T04:55:30.657833+0000","last_clean":"2023-10-06T04:55:30.657833+0000","last_became_active":"2023-10-06T04:55:30.654003+0000","last_became_peered":"2023-10-06T04:55:30.654003+0000","last_unstale":"2023-10-06T04:55:30.657833+0000","last_undegraded":"2023-10-06T04:55:30.657833+0000","last_fullsized":"2023-10-06T04:55:30.657833+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:15.348680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.644532+0000","last_change":"2023-10-06T04:55:30.639937+0000","last_active":"2023-10-06T04:55:30.644532+0000","last_peered":"2023-10-06T04:55:30.644532+0000","last_clean":"2023-10-06T04:55:30.644532+0000","last_became_active":"2023-10-06T04:55:30.639626+0000","last_became_peered":"2023-10-06T04:55:30.639626+0000","last_unstale":"2023-10-06T04:55:30.644532+0000","last_undegraded":"2023-10-06T04:55:30.644532+0000","last_fullsized":"2023-10-06T04:55:30.644532+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:31.939182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.654465+0000","last_change":"2023-10-06T04:55:30.651468+0000","last_active":"2023-10-06T04:55:30.654465+0000","last_peered":"2023-10-06T04:55:30.654465+0000","last_clean":"2023-10-06T04:55:30.654465+0000","last_became_active":"2023-10-06T04:55:30.651206+0000","last_became_peered":"2023-10-06T04:55:30.651206+0000","last_unstale":"2023-10-06T04:55:30.654465+0000","last_undegraded":"2023-10-06T04:55:30.654465+0000","last_fullsized":"2023-10-06T04:55:30.654465+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:20.584996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.658226+0000","last_change":"2023-10-06T04:55:30.656606+0000","last_active":"2023-10-06T04:55:30.658226+0000","last_peered":"2023-10-06T04:55:30.658226+0000","last_clean":"2023-10-06T04:55:30.658226+0000","last_became_active":"2023-10-06T04:55:30.656307+0000","last_became_peered":"2023-10-06T04:55:30.656307+0000","last_unstale":"2023-10-06T04:55:30.658226+0000","last_undegraded":"2023-10-06T04:55:30.658226+0000","last_fullsized":"2023-10-06T04:55:30.658226+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:18.527942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.648011+0000","last_change":"2023-10-06T04:55:30.643424+0000","last_active":"2023-10-06T04:55:30.648011+0000","last_peered":"2023-10-06T04:55:30.648011+0000","last_clean":"2023-10-06T04:55:30.648011+0000","last_became_active":"2023-10-06T04:55:30.643143+0000","last_became_peered":"2023-10-06T04:55:30.643143+0000","last_unstale":"2023-10-06T04:55:30.648011+0000","last_undegraded":"2023-10-06T04:55:30.648011+0000","last_fullsized":"2023-10-06T04:55:30.648011+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:47.654687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.651748+0000","last_change":"2023-10-06T04:55:30.649187+0000","last_active":"2023-10-06T04:55:30.651748+0000","last_peered":"2023-10-06T04:55:30.651748+0000","last_clean":"2023-10-06T04:55:30.651748+0000","last_became_active":"2023-10-06T04:55:30.648901+0000","last_became_peered":"2023-10-06T04:55:30.648901+0000","last_unstale":"2023-10-06T04:55:30.651748+0000","last_undegraded":"2023-10-06T04:55:30.651748+0000","last_fullsized":"2023-10-06T04:55:30.651748+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:20.584996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655319+0000","last_change":"2023-10-06T04:55:30.639449+0000","last_active":"2023-10-06T04:55:30.655319+0000","last_peered":"2023-10-06T04:55:30.655319+0000","last_clean":"2023-10-06T04:55:30.655319+0000","last_became_active":"2023-10-06T04:55:30.639156+0000","last_became_peered":"2023-10-06T04:55:30.639156+0000","last_unstale":"2023-10-06T04:55:30.655319+0000","last_undegraded":"2023-10-06T04:55:30.655319+0000","last_fullsized":"2023-10-06T04:55:30.655319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:48.751865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.646391+0000","last_change":"2023-10-06T04:55:30.643737+0000","last_active":"2023-10-06T04:55:30.646391+0000","last_peered":"2023-10-06T04:55:30.646391+0000","last_clean":"2023-10-06T04:55:30.646391+0000","last_became_active":"2023-10-06T04:55:30.643302+0000","last_became_peered":"2023-10-06T04:55:30.643302+0000","last_unstale":"2023-10-06T04:55:30.646391+0000","last_undegraded":"2023-10-06T04:55:30.646391+0000","last_fullsized":"2023-10-06T04:55:30.646391+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:16.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.646567+0000","last_change":"2023-10-06T04:55:30.645539+0000","last_active":"2023-10-06T04:55:30.646567+0000","last_peered":"2023-10-06T04:55:30.646567+0000","last_clean":"2023-10-06T04:55:30.646567+0000","last_became_active":"2023-10-06T04:55:30.645086+0000","last_became_peered":"2023-10-06T04:55:30.645086+0000","last_unstale":"2023-10-06T04:55:30.646567+0000","last_undegraded":"2023-10-06T04:55:30.646567+0000","last_fullsized":"2023-10-06T04:55:30.646567+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.568423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.647573+0000","last_change":"2023-10-06T04:55:30.641862+0000","last_active":"2023-10-06T04:55:30.647573+0000","last_peered":"2023-10-06T04:55:30.647573+0000","last_clean":"2023-10-06T04:55:30.647573+0000","last_became_active":"2023-10-06T04:55:30.641576+0000","last_became_peered":"2023-10-06T04:55:30.641576+0000","last_unstale":"2023-10-06T04:55:30.647573+0000","last_undegraded":"2023-10-06T04:55:30.647573+0000","last_fullsized":"2023-10-06T04:55:30.647573+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:34.163179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.647525+0000","last_change":"2023-10-06T04:55:30.645979+0000","last_active":"2023-10-06T04:55:30.647525+0000","last_peered":"2023-10-06T04:55:30.647525+0000","last_clean":"2023-10-06T04:55:30.647525+0000","last_became_active":"2023-10-06T04:55:30.645210+0000","last_became_peered":"2023-10-06T04:55:30.645210+0000","last_unstale":"2023-10-06T04:55:30.647525+0000","last_undegraded":"2023-10-06T04:55:30.647525+0000","last_fullsized":"2023-10-06T04:55:30.647525+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:11.480851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.646967+0000","last_change":"2023-10-06T04:55:30.640484+0000","last_active":"2023-10-06T04:55:30.646967+0000","last_peered":"2023-10-06T04:55:30.646967+0000","last_clean":"2023-10-06T04:55:30.646967+0000","last_became_active":"2023-10-06T04:55:30.640198+0000","last_became_peered":"2023-10-06T04:55:30.640198+0000","last_unstale":"2023-10-06T04:55:30.646967+0000","last_undegraded":"2023-10-06T04:55:30.646967+0000","last_fullsized":"2023-10-06T04:55:30.646967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:06.360553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.662897+0000","last_change":"2023-10-06T04:55:29.644855+0000","last_active":"2023-10-06T04:55:30.662897+0000","last_peered":"2023-10-06T04:55:30.662897+0000","last_clean":"2023-10-06T04:55:30.662897+0000","last_became_active":"2023-10-06T04:55:29.644553+0000","last_became_peered":"2023-10-06T04:55:29.644553+0000","last_unstale":"2023-10-06T04:55:30.662897+0000","last_undegraded":"2023-10-06T04:55:30.662897+0000","last_fullsized":"2023-10-06T04:55:30.662897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:31.518507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650199+0000","last_change":"2023-10-06T04:55:30.645472+0000","last_active":"2023-10-06T04:55:30.650199+0000","last_peered":"2023-10-06T04:55:30.650199+0000","last_clean":"2023-10-06T04:55:30.650199+0000","last_became_active":"2023-10-06T04:55:30.645193+0000","last_became_peered":"2023-10-06T04:55:30.645193+0000","last_unstale":"2023-10-06T04:55:30.650199+0000","last_undegraded":"2023-10-06T04:55:30.650199+0000","last_fullsized":"2023-10-06T04:55:30.650199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:38.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.648082+0000","last_change":"2023-10-06T04:55:29.640284+0000","last_active":"2023-10-06T04:55:30.648082+0000","last_peered":"2023-10-06T04:55:30.648082+0000","last_clean":"2023-10-06T04:55:30.648082+0000","last_became_active":"2023-10-06T04:55:29.638752+0000","last_became_peered":"2023-10-06T04:55:29.638752+0000","last_unstale":"2023-10-06T04:55:30.648082+0000","last_undegraded":"2023-10-06T04:55:30.648082+0000","last_fullsized":"2023-10-06T04:55:30.648082+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.325782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657314+0000","last_change":"2023-10-06T04:55:30.656691+0000","last_active":"2023-10-06T04:55:30.657314+0000","last_peered":"2023-10-06T04:55:30.657314+0000","last_clean":"2023-10-06T04:55:30.657314+0000","last_became_active":"2023-10-06T04:55:30.656429+0000","last_became_peered":"2023-10-06T04:55:30.656429+0000","last_unstale":"2023-10-06T04:55:30.657314+0000","last_undegraded":"2023-10-06T04:55:30.657314+0000","last_fullsized":"2023-10-06T04:55:30.657314+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:40.707668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650942+0000","last_change":"2023-10-06T04:55:29.646265+0000","last_active":"2023-10-06T04:55:30.650942+0000","last_peered":"2023-10-06T04:55:30.650942+0000","last_clean":"2023-10-06T04:55:30.650942+0000","last_became_active":"2023-10-06T04:55:29.645786+0000","last_became_peered":"2023-10-06T04:55:29.645786+0000","last_unstale":"2023-10-06T04:55:30.650942+0000","last_undegraded":"2023-10-06T04:55:30.650942+0000","last_fullsized":"2023-10-06T04:55:30.650942+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:11.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.644618+0000","last_change":"2023-10-06T04:55:30.640437+0000","last_active":"2023-10-06T04:55:30.644618+0000","last_peered":"2023-10-06T04:55:30.644618+0000","last_clean":"2023-10-06T04:55:30.644618+0000","last_became_active":"2023-10-06T04:55:30.640168+0000","last_became_peered":"2023-10-06T04:55:30.640168+0000","last_unstale":"2023-10-06T04:55:30.644618+0000","last_undegraded":"2023-10-06T04:55:30.644618+0000","last_fullsized":"2023-10-06T04:55:30.644618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:34.939948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.652701+0000","last_change":"2023-10-06T04:55:29.643149+0000","last_active":"2023-10-06T04:55:30.652701+0000","last_peered":"2023-10-06T04:55:30.652701+0000","last_clean":"2023-10-06T04:55:30.652701+0000","last_became_active":"2023-10-06T04:55:29.642865+0000","last_became_peered":"2023-10-06T04:55:29.642865+0000","last_unstale":"2023-10-06T04:55:30.652701+0000","last_undegraded":"2023-10-06T04:55:30.652701+0000","last_fullsized":"2023-10-06T04:55:30.652701+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:25.603703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657620+0000","last_change":"2023-10-06T04:55:30.656363+0000","last_active":"2023-10-06T04:55:30.657620+0000","last_peered":"2023-10-06T04:55:30.657620+0000","last_clean":"2023-10-06T04:55:30.657620+0000","last_became_active":"2023-10-06T04:55:30.656069+0000","last_became_peered":"2023-10-06T04:55:30.656069+0000","last_unstale":"2023-10-06T04:55:30.657620+0000","last_undegraded":"2023-10-06T04:55:30.657620+0000","last_fullsized":"2023-10-06T04:55:30.657620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:54.819595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.648067+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:30.648067+0000","last_peered":"2023-10-06T04:55:30.648067+0000","last_clean":"2023-10-06T04:55:30.648067+0000","last_became_active":"2023-10-06T04:55:29.641637+0000","last_became_peered":"2023-10-06T04:55:29.641637+0000","last_unstale":"2023-10-06T04:55:30.648067+0000","last_undegraded":"2023-10-06T04:55:30.648067+0000","last_fullsized":"2023-10-06T04:55:30.648067+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:44.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.660587+0000","last_change":"2023-10-06T04:55:30.658416+0000","last_active":"2023-10-06T04:55:30.660587+0000","last_peered":"2023-10-06T04:55:30.660587+0000","last_clean":"2023-10-06T04:55:30.660587+0000","last_became_active":"2023-10-06T04:55:30.657822+0000","last_became_peered":"2023-10-06T04:55:30.657822+0000","last_unstale":"2023-10-06T04:55:30.660587+0000","last_undegraded":"2023-10-06T04:55:30.660587+0000","last_fullsized":"2023-10-06T04:55:30.660587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:26.177512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.658631+0000","last_change":"2023-10-06T04:55:29.642590+0000","last_active":"2023-10-06T04:55:30.658631+0000","last_peered":"2023-10-06T04:55:30.658631+0000","last_clean":"2023-10-06T04:55:30.658631+0000","last_became_active":"2023-10-06T04:55:29.642252+0000","last_became_peered":"2023-10-06T04:55:29.642252+0000","last_unstale":"2023-10-06T04:55:30.658631+0000","last_undegraded":"2023-10-06T04:55:30.658631+0000","last_fullsized":"2023-10-06T04:55:30.658631+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:56.195511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657313+0000","last_change":"2023-10-06T04:55:30.656683+0000","last_active":"2023-10-06T04:55:30.657313+0000","last_peered":"2023-10-06T04:55:30.657313+0000","last_clean":"2023-10-06T04:55:30.657313+0000","last_became_active":"2023-10-06T04:55:30.656339+0000","last_became_peered":"2023-10-06T04:55:30.656339+0000","last_unstale":"2023-10-06T04:55:30.657313+0000","last_undegraded":"2023-10-06T04:55:30.657313+0000","last_fullsized":"2023-10-06T04:55:30.657313+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:52.846620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.631335+0000","last_change":"2023-10-06T04:55:29.643163+0000","last_active":"2023-10-06T04:55:30.631335+0000","last_peered":"2023-10-06T04:55:30.631335+0000","last_clean":"2023-10-06T04:55:30.631335+0000","last_became_active":"2023-10-06T04:55:29.642910+0000","last_became_peered":"2023-10-06T04:55:29.642910+0000","last_unstale":"2023-10-06T04:55:30.631335+0000","last_undegraded":"2023-10-06T04:55:30.631335+0000","last_fullsized":"2023-10-06T04:55:30.631335+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:35.681905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653197+0000","last_change":"2023-10-06T04:55:30.647630+0000","last_active":"2023-10-06T04:55:30.653197+0000","last_peered":"2023-10-06T04:55:30.653197+0000","last_clean":"2023-10-06T04:55:30.653197+0000","last_became_active":"2023-10-06T04:55:30.646676+0000","last_became_peered":"2023-10-06T04:55:30.646676+0000","last_unstale":"2023-10-06T04:55:30.653197+0000","last_undegraded":"2023-10-06T04:55:30.653197+0000","last_fullsized":"2023-10-06T04:55:30.653197+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:59.757486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.633204+0000","last_change":"2023-10-06T04:55:29.641803+0000","last_active":"2023-10-06T04:55:30.633204+0000","last_peered":"2023-10-06T04:55:30.633204+0000","last_clean":"2023-10-06T04:55:30.633204+0000","last_became_active":"2023-10-06T04:55:29.641484+0000","last_became_peered":"2023-10-06T04:55:29.641484+0000","last_unstale":"2023-10-06T04:55:30.633204+0000","last_undegraded":"2023-10-06T04:55:30.633204+0000","last_fullsized":"2023-10-06T04:55:30.633204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.831547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650023+0000","last_change":"2023-10-06T04:55:30.644858+0000","last_active":"2023-10-06T04:55:30.650023+0000","last_peered":"2023-10-06T04:55:30.650023+0000","last_clean":"2023-10-06T04:55:30.650023+0000","last_became_active":"2023-10-06T04:55:30.644449+0000","last_became_peered":"2023-10-06T04:55:30.644449+0000","last_unstale":"2023-10-06T04:55:30.650023+0000","last_undegraded":"2023-10-06T04:55:30.650023+0000","last_fullsized":"2023-10-06T04:55:30.650023+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:05.703934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.634925+0000","last_change":"2023-10-06T04:55:29.645173+0000","last_active":"2023-10-06T04:55:30.634925+0000","last_peered":"2023-10-06T04:55:30.634925+0000","last_clean":"2023-10-06T04:55:30.634925+0000","last_became_active":"2023-10-06T04:55:29.644899+0000","last_became_peered":"2023-10-06T04:55:29.644899+0000","last_unstale":"2023-10-06T04:55:30.634925+0000","last_undegraded":"2023-10-06T04:55:30.634925+0000","last_fullsized":"2023-10-06T04:55:30.634925+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:35.447530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.641563+0000","last_change":"2023-10-06T04:55:30.639102+0000","last_active":"2023-10-06T04:55:30.641563+0000","last_peered":"2023-10-06T04:55:30.641563+0000","last_clean":"2023-10-06T04:55:30.641563+0000","last_became_active":"2023-10-06T04:55:30.638760+0000","last_became_peered":"2023-10-06T04:55:30.638760+0000","last_unstale":"2023-10-06T04:55:30.641563+0000","last_undegraded":"2023-10-06T04:55:30.641563+0000","last_fullsized":"2023-10-06T04:55:30.641563+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:37.951367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.632829+0000","last_change":"2023-10-06T04:55:29.639731+0000","last_active":"2023-10-06T04:55:30.632829+0000","last_peered":"2023-10-06T04:55:30.632829+0000","last_clean":"2023-10-06T04:55:30.632829+0000","last_became_active":"2023-10-06T04:55:29.639060+0000","last_became_peered":"2023-10-06T04:55:29.639060+0000","last_unstale":"2023-10-06T04:55:30.632829+0000","last_undegraded":"2023-10-06T04:55:30.632829+0000","last_fullsized":"2023-10-06T04:55:30.632829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:19.838829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650903+0000","last_change":"2023-10-06T04:55:30.633974+0000","last_active":"2023-10-06T04:55:30.650903+0000","last_peered":"2023-10-06T04:55:30.650903+0000","last_clean":"2023-10-06T04:55:30.650903+0000","last_became_active":"2023-10-06T04:55:30.633646+0000","last_became_peered":"2023-10-06T04:55:30.633646+0000","last_unstale":"2023-10-06T04:55:30.650903+0000","last_undegraded":"2023-10-06T04:55:30.650903+0000","last_fullsized":"2023-10-06T04:55:30.650903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:56.908379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"71'3","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657975+0000","last_change":"2023-10-06T04:55:29.643771+0000","last_active":"2023-10-06T04:55:30.657975+0000","last_peered":"2023-10-06T04:55:30.657975+0000","last_clean":"2023-10-06T04:55:30.657975+0000","last_became_active":"2023-10-06T04:55:29.643494+0000","last_became_peered":"2023-10-06T04:55:29.643494+0000","last_unstale":"2023-10-06T04:55:30.657975+0000","last_undegraded":"2023-10-06T04:55:30.657975+0000","last_fullsized":"2023-10-06T04:55:30.657975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:14.660406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650163+0000","last_change":"2023-10-06T04:55:30.647429+0000","last_active":"2023-10-06T04:55:30.650163+0000","last_peered":"2023-10-06T04:55:30.650163+0000","last_clean":"2023-10-06T04:55:30.650163+0000","last_became_active":"2023-10-06T04:55:30.647175+0000","last_became_peered":"2023-10-06T04:55:30.647175+0000","last_unstale":"2023-10-06T04:55:30.650163+0000","last_undegraded":"2023-10-06T04:55:30.650163+0000","last_fullsized":"2023-10-06T04:55:30.650163+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:57.205791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"71'6","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.660281+0000","last_change":"2023-10-06T04:55:29.636609+0000","last_active":"2023-10-06T04:55:30.660281+0000","last_peered":"2023-10-06T04:55:30.660281+0000","last_clean":"2023-10-06T04:55:30.660281+0000","last_became_active":"2023-10-06T04:55:29.636160+0000","last_became_peered":"2023-10-06T04:55:29.636160+0000","last_unstale":"2023-10-06T04:55:30.660281+0000","last_undegraded":"2023-10-06T04:55:30.660281+0000","last_fullsized":"2023-10-06T04:55:30.660281+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:32.519067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653576+0000","last_change":"2023-10-06T04:55:30.653129+0000","last_active":"2023-10-06T04:55:30.653576+0000","last_peered":"2023-10-06T04:55:30.653576+0000","last_clean":"2023-10-06T04:55:30.653576+0000","last_became_active":"2023-10-06T04:55:30.652884+0000","last_became_peered":"2023-10-06T04:55:30.652884+0000","last_unstale":"2023-10-06T04:55:30.653576+0000","last_undegraded":"2023-10-06T04:55:30.653576+0000","last_fullsized":"2023-10-06T04:55:30.653576+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:44.359243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650857+0000","last_change":"2023-10-06T04:55:29.634753+0000","last_active":"2023-10-06T04:55:30.650857+0000","last_peered":"2023-10-06T04:55:30.650857+0000","last_clean":"2023-10-06T04:55:30.650857+0000","last_became_active":"2023-10-06T04:55:29.634471+0000","last_became_peered":"2023-10-06T04:55:29.634471+0000","last_unstale":"2023-10-06T04:55:30.650857+0000","last_undegraded":"2023-10-06T04:55:30.650857+0000","last_fullsized":"2023-10-06T04:55:30.650857+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:47.921784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.643300+0000","last_change":"2023-10-06T04:55:30.640507+0000","last_active":"2023-10-06T04:55:30.643300+0000","last_peered":"2023-10-06T04:55:30.643300+0000","last_clean":"2023-10-06T04:55:30.643300+0000","last_became_active":"2023-10-06T04:55:30.640224+0000","last_became_peered":"2023-10-06T04:55:30.640224+0000","last_unstale":"2023-10-06T04:55:30.643300+0000","last_undegraded":"2023-10-06T04:55:30.643300+0000","last_fullsized":"2023-10-06T04:55:30.643300+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:45.147340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.632937+0000","last_change":"2023-10-06T04:55:29.644259+0000","last_active":"2023-10-06T04:55:30.632937+0000","last_peered":"2023-10-06T04:55:30.632937+0000","last_clean":"2023-10-06T04:55:30.632937+0000","last_became_active":"2023-10-06T04:55:29.643992+0000","last_became_peered":"2023-10-06T04:55:29.643992+0000","last_unstale":"2023-10-06T04:55:30.632937+0000","last_undegraded":"2023-10-06T04:55:30.632937+0000","last_fullsized":"2023-10-06T04:55:30.632937+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:43.348964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.642154+0000","last_change":"2023-10-06T04:55:30.640397+0000","last_active":"2023-10-06T04:55:30.642154+0000","last_peered":"2023-10-06T04:55:30.642154+0000","last_clean":"2023-10-06T04:55:30.642154+0000","last_became_active":"2023-10-06T04:55:30.640084+0000","last_became_peered":"2023-10-06T04:55:30.640084+0000","last_unstale":"2023-10-06T04:55:30.642154+0000","last_undegraded":"2023-10-06T04:55:30.642154+0000","last_fullsized":"2023-10-06T04:55:30.642154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:06.628478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.632907+0000","last_change":"2023-10-06T04:55:29.644987+0000","last_active":"2023-10-06T04:55:30.632907+0000","last_peered":"2023-10-06T04:55:30.632907+0000","last_clean":"2023-10-06T04:55:30.632907+0000","last_became_active":"2023-10-06T04:55:29.644620+0000","last_became_peered":"2023-10-06T04:55:29.644620+0000","last_unstale":"2023-10-06T04:55:30.632907+0000","last_undegraded":"2023-10-06T04:55:30.632907+0000","last_fullsized":"2023-10-06T04:55:30.632907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:57.789632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653730+0000","last_change":"2023-10-06T04:55:30.650792+0000","last_active":"2023-10-06T04:55:30.653730+0000","last_peered":"2023-10-06T04:55:30.653730+0000","last_clean":"2023-10-06T04:55:30.653730+0000","last_became_active":"2023-10-06T04:55:30.650505+0000","last_became_peered":"2023-10-06T04:55:30.650505+0000","last_unstale":"2023-10-06T04:55:30.653730+0000","last_undegraded":"2023-10-06T04:55:30.653730+0000","last_fullsized":"2023-10-06T04:55:30.653730+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:21.465605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"71'8","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655469+0000","last_change":"2023-10-06T04:55:29.637724+0000","last_active":"2023-10-06T04:55:30.655469+0000","last_peered":"2023-10-06T04:55:30.655469+0000","last_clean":"2023-10-06T04:55:30.655469+0000","last_became_active":"2023-10-06T04:55:29.637235+0000","last_became_peered":"2023-10-06T04:55:29.637235+0000","last_unstale":"2023-10-06T04:55:30.655469+0000","last_undegraded":"2023-10-06T04:55:30.655469+0000","last_fullsized":"2023-10-06T04:55:30.655469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:27.721875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.644820+0000","last_change":"2023-10-06T04:55:30.639713+0000","last_active":"2023-10-06T04:55:30.644820+0000","last_peered":"2023-10-06T04:55:30.644820+0000","last_clean":"2023-10-06T04:55:30.644820+0000","last_became_active":"2023-10-06T04:55:30.639477+0000","last_became_peered":"2023-10-06T04:55:30.639477+0000","last_unstale":"2023-10-06T04:55:30.644820+0000","last_undegraded":"2023-10-06T04:55:30.644820+0000","last_fullsized":"2023-10-06T04:55:30.644820+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:28.732154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655245+0000","last_change":"2023-10-06T04:55:29.637907+0000","last_active":"2023-10-06T04:55:30.655245+0000","last_peered":"2023-10-06T04:55:30.655245+0000","last_clean":"2023-10-06T04:55:30.655245+0000","last_became_active":"2023-10-06T04:55:29.637626+0000","last_became_peered":"2023-10-06T04:55:29.637626+0000","last_unstale":"2023-10-06T04:55:30.655245+0000","last_undegraded":"2023-10-06T04:55:30.655245+0000","last_fullsized":"2023-10-06T04:55:30.655245+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:44.959384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653966+0000","last_change":"2023-10-06T04:55:30.650588+0000","last_active":"2023-10-06T04:55:30.653966+0000","last_peered":"2023-10-06T04:55:30.653966+0000","last_clean":"2023-10-06T04:55:30.653966+0000","last_became_active":"2023-10-06T04:55:30.650301+0000","last_became_peered":"2023-10-06T04:55:30.650301+0000","last_unstale":"2023-10-06T04:55:30.653966+0000","last_undegraded":"2023-10-06T04:55:30.653966+0000","last_fullsized":"2023-10-06T04:55:30.653966+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:13.269448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"72'1","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T04:55:31.636275+0000","last_change":"2023-10-06T04:55:29.641635+0000","last_active":"2023-10-06T04:55:31.636275+0000","last_peered":"2023-10-06T04:55:31.636275+0000","last_clean":"2023-10-06T04:55:31.636275+0000","last_became_active":"2023-10-06T04:55:29.641370+0000","last_became_peered":"2023-10-06T04:55:29.641370+0000","last_unstale":"2023-10-06T04:55:31.636275+0000","last_undegraded":"2023-10-06T04:55:31.636275+0000","last_fullsized":"2023-10-06T04:55:31.636275+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657302+0000","last_change":"2023-10-06T04:55:30.656610+0000","last_active":"2023-10-06T04:55:30.657302+0000","last_peered":"2023-10-06T04:55:30.657302+0000","last_clean":"2023-10-06T04:55:30.657302+0000","last_became_active":"2023-10-06T04:55:30.656359+0000","last_became_peered":"2023-10-06T04:55:30.656359+0000","last_unstale":"2023-10-06T04:55:30.657302+0000","last_undegraded":"2023-10-06T04:55:30.657302+0000","last_fullsized":"2023-10-06T04:55:30.657302+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:36.692184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.636157+0000","last_change":"2023-10-06T04:55:29.644387+0000","last_active":"2023-10-06T04:55:30.636157+0000","last_peered":"2023-10-06T04:55:30.636157+0000","last_clean":"2023-10-06T04:55:30.636157+0000","last_became_active":"2023-10-06T04:55:29.644059+0000","last_became_peered":"2023-10-06T04:55:29.644059+0000","last_unstale":"2023-10-06T04:55:30.636157+0000","last_undegraded":"2023-10-06T04:55:30.636157+0000","last_fullsized":"2023-10-06T04:55:30.636157+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650123+0000","last_change":"2023-10-06T04:55:30.645155+0000","last_active":"2023-10-06T04:55:30.650123+0000","last_peered":"2023-10-06T04:55:30.650123+0000","last_clean":"2023-10-06T04:55:30.650123+0000","last_became_active":"2023-10-06T04:55:30.644875+0000","last_became_peered":"2023-10-06T04:55:30.644875+0000","last_unstale":"2023-10-06T04:55:30.650123+0000","last_undegraded":"2023-10-06T04:55:30.650123+0000","last_fullsized":"2023-10-06T04:55:30.650123+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.902269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.636695+0000","last_change":"2023-10-06T04:55:29.644897+0000","last_active":"2023-10-06T04:55:30.636695+0000","last_peered":"2023-10-06T04:55:30.636695+0000","last_clean":"2023-10-06T04:55:30.636695+0000","last_became_active":"2023-10-06T04:55:29.644631+0000","last_became_peered":"2023-10-06T04:55:29.644631+0000","last_unstale":"2023-10-06T04:55:30.636695+0000","last_undegraded":"2023-10-06T04:55:30.636695+0000","last_fullsized":"2023-10-06T04:55:30.636695+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:39.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653787+0000","last_change":"2023-10-06T04:55:30.651270+0000","last_active":"2023-10-06T04:55:30.653787+0000","last_peered":"2023-10-06T04:55:30.653787+0000","last_clean":"2023-10-06T04:55:30.653787+0000","last_became_active":"2023-10-06T04:55:30.650986+0000","last_became_peered":"2023-10-06T04:55:30.650986+0000","last_unstale":"2023-10-06T04:55:30.653787+0000","last_undegraded":"2023-10-06T04:55:30.653787+0000","last_fullsized":"2023-10-06T04:55:30.653787+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:32.910233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.630593+0000","last_change":"2023-10-06T04:55:29.642477+0000","last_active":"2023-10-06T04:55:30.630593+0000","last_peered":"2023-10-06T04:55:30.630593+0000","last_clean":"2023-10-06T04:55:30.630593+0000","last_became_active":"2023-10-06T04:55:29.642202+0000","last_became_peered":"2023-10-06T04:55:29.642202+0000","last_unstale":"2023-10-06T04:55:30.630593+0000","last_undegraded":"2023-10-06T04:55:30.630593+0000","last_fullsized":"2023-10-06T04:55:30.630593+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:14.338401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.654508+0000","last_change":"2023-10-06T04:55:30.653878+0000","last_active":"2023-10-06T04:55:30.654508+0000","last_peered":"2023-10-06T04:55:30.654508+0000","last_clean":"2023-10-06T04:55:30.654508+0000","last_became_active":"2023-10-06T04:55:30.653595+0000","last_became_peered":"2023-10-06T04:55:30.653595+0000","last_unstale":"2023-10-06T04:55:30.654508+0000","last_undegraded":"2023-10-06T04:55:30.654508+0000","last_fullsized":"2023-10-06T04:55:30.654508+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:47.223457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.638602+0000","last_change":"2023-10-06T04:55:29.644875+0000","last_active":"2023-10-06T04:55:30.638602+0000","last_peered":"2023-10-06T04:55:30.638602+0000","last_clean":"2023-10-06T04:55:30.638602+0000","last_became_active":"2023-10-06T04:55:29.644568+0000","last_became_peered":"2023-10-06T04:55:29.644568+0000","last_unstale":"2023-10-06T04:55:30.638602+0000","last_undegraded":"2023-10-06T04:55:30.638602+0000","last_fullsized":"2023-10-06T04:55:30.638602+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:08.690122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.649983+0000","last_change":"2023-10-06T04:55:30.647061+0000","last_active":"2023-10-06T04:55:30.649983+0000","last_peered":"2023-10-06T04:55:30.649983+0000","last_clean":"2023-10-06T04:55:30.649983+0000","last_became_active":"2023-10-06T04:55:30.646658+0000","last_became_peered":"2023-10-06T04:55:30.646658+0000","last_unstale":"2023-10-06T04:55:30.649983+0000","last_undegraded":"2023-10-06T04:55:30.649983+0000","last_fullsized":"2023-10-06T04:55:30.649983+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:48.932063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.632820+0000","last_change":"2023-10-06T04:55:29.641632+0000","last_active":"2023-10-06T04:55:30.632820+0000","last_peered":"2023-10-06T04:55:30.632820+0000","last_clean":"2023-10-06T04:55:30.632820+0000","last_became_active":"2023-10-06T04:55:29.641299+0000","last_became_peered":"2023-10-06T04:55:29.641299+0000","last_unstale":"2023-10-06T04:55:30.632820+0000","last_undegraded":"2023-10-06T04:55:30.632820+0000","last_fullsized":"2023-10-06T04:55:30.632820+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:42.695331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.639112+0000","last_change":"2023-10-06T04:55:30.635890+0000","last_active":"2023-10-06T04:55:30.639112+0000","last_peered":"2023-10-06T04:55:30.639112+0000","last_clean":"2023-10-06T04:55:30.639112+0000","last_became_active":"2023-10-06T04:55:30.635624+0000","last_became_peered":"2023-10-06T04:55:30.635624+0000","last_unstale":"2023-10-06T04:55:30.639112+0000","last_undegraded":"2023-10-06T04:55:30.639112+0000","last_fullsized":"2023-10-06T04:55:30.639112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:45.969663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.626424+0000","last_change":"2023-10-06T04:55:29.634678+0000","last_active":"2023-10-06T04:55:30.626424+0000","last_peered":"2023-10-06T04:55:30.626424+0000","last_clean":"2023-10-06T04:55:30.626424+0000","last_became_active":"2023-10-06T04:55:29.634400+0000","last_became_peered":"2023-10-06T04:55:29.634400+0000","last_unstale":"2023-10-06T04:55:30.626424+0000","last_undegraded":"2023-10-06T04:55:30.626424+0000","last_fullsized":"2023-10-06T04:55:30.626424+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:04.440283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.658229+0000","last_change":"2023-10-06T04:55:30.650420+0000","last_active":"2023-10-06T04:55:30.658229+0000","last_peered":"2023-10-06T04:55:30.658229+0000","last_clean":"2023-10-06T04:55:30.658229+0000","last_became_active":"2023-10-06T04:55:30.650057+0000","last_became_peered":"2023-10-06T04:55:30.650057+0000","last_unstale":"2023-10-06T04:55:30.658229+0000","last_undegraded":"2023-10-06T04:55:30.658229+0000","last_fullsized":"2023-10-06T04:55:30.658229+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:16.796832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.650673+0000","last_change":"2023-10-06T04:55:29.644649+0000","last_active":"2023-10-06T04:55:30.650673+0000","last_peered":"2023-10-06T04:55:30.650673+0000","last_clean":"2023-10-06T04:55:30.650673+0000","last_became_active":"2023-10-06T04:55:29.644333+0000","last_became_peered":"2023-10-06T04:55:29.644333+0000","last_unstale":"2023-10-06T04:55:30.650673+0000","last_undegraded":"2023-10-06T04:55:30.650673+0000","last_fullsized":"2023-10-06T04:55:30.650673+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:36.941087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.630524+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:55:30.630524+0000","last_peered":"2023-10-06T04:55:30.630524+0000","last_clean":"2023-10-06T04:55:30.630524+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:55:30.630524+0000","last_undegraded":"2023-10-06T04:55:30.630524+0000","last_fullsized":"2023-10-06T04:55:30.630524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.649557+0000","last_change":"2023-10-06T04:55:30.639741+0000","last_active":"2023-10-06T04:55:30.649557+0000","last_peered":"2023-10-06T04:55:30.649557+0000","last_clean":"2023-10-06T04:55:30.649557+0000","last_became_active":"2023-10-06T04:55:30.639428+0000","last_became_peered":"2023-10-06T04:55:30.639428+0000","last_unstale":"2023-10-06T04:55:30.649557+0000","last_undegraded":"2023-10-06T04:55:30.649557+0000","last_fullsized":"2023-10-06T04:55:30.649557+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:18.378622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.652695+0000","last_change":"2023-10-06T04:55:29.628693+0000","last_active":"2023-10-06T04:55:30.652695+0000","last_peered":"2023-10-06T04:55:30.652695+0000","last_clean":"2023-10-06T04:55:30.652695+0000","last_became_active":"2023-10-06T04:55:29.627557+0000","last_became_peered":"2023-10-06T04:55:29.627557+0000","last_unstale":"2023-10-06T04:55:30.652695+0000","last_undegraded":"2023-10-06T04:55:30.652695+0000","last_fullsized":"2023-10-06T04:55:30.652695+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:58.747206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.627750+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:55:30.627750+0000","last_peered":"2023-10-06T04:55:30.627750+0000","last_clean":"2023-10-06T04:55:30.627750+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:55:30.627750+0000","last_undegraded":"2023-10-06T04:55:30.627750+0000","last_fullsized":"2023-10-06T04:55:30.627750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.643476+0000","last_change":"2023-10-06T04:55:30.641735+0000","last_active":"2023-10-06T04:55:30.643476+0000","last_peered":"2023-10-06T04:55:30.643476+0000","last_clean":"2023-10-06T04:55:30.643476+0000","last_became_active":"2023-10-06T04:55:30.641329+0000","last_became_peered":"2023-10-06T04:55:30.641329+0000","last_unstale":"2023-10-06T04:55:30.643476+0000","last_undegraded":"2023-10-06T04:55:30.643476+0000","last_fullsized":"2023-10-06T04:55:30.643476+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:27.293583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.627877+0000","last_change":"2023-10-06T04:55:29.636254+0000","last_active":"2023-10-06T04:55:30.627877+0000","last_peered":"2023-10-06T04:55:30.627877+0000","last_clean":"2023-10-06T04:55:30.627877+0000","last_became_active":"2023-10-06T04:55:29.635966+0000","last_became_peered":"2023-10-06T04:55:29.635966+0000","last_unstale":"2023-10-06T04:55:30.627877+0000","last_undegraded":"2023-10-06T04:55:30.627877+0000","last_fullsized":"2023-10-06T04:55:30.627877+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:09.863401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":58,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.627354+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:55:30.627354+0000","last_peered":"2023-10-06T04:55:30.627354+0000","last_clean":"2023-10-06T04:55:30.627354+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:55:30.627354+0000","last_undegraded":"2023-10-06T04:55:30.627354+0000","last_fullsized":"2023-10-06T04:55:30.627354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":204,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.633374+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:55:30.633374+0000","last_peered":"2023-10-06T04:55:30.633374+0000","last_clean":"2023-10-06T04:55:30.633374+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:55:30.633374+0000","last_undegraded":"2023-10-06T04:55:30.633374+0000","last_fullsized":"2023-10-06T04:55:30.633374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.642651+0000","last_change":"2023-10-06T04:55:30.641657+0000","last_active":"2023-10-06T04:55:30.642651+0000","last_peered":"2023-10-06T04:55:30.642651+0000","last_clean":"2023-10-06T04:55:30.642651+0000","last_became_active":"2023-10-06T04:55:30.641330+0000","last_became_peered":"2023-10-06T04:55:30.641330+0000","last_unstale":"2023-10-06T04:55:30.642651+0000","last_undegraded":"2023-10-06T04:55:30.642651+0000","last_fullsized":"2023-10-06T04:55:30.642651+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.263825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.626889+0000","last_change":"2023-10-06T04:55:29.634448+0000","last_active":"2023-10-06T04:55:30.626889+0000","last_peered":"2023-10-06T04:55:30.626889+0000","last_clean":"2023-10-06T04:55:30.626889+0000","last_became_active":"2023-10-06T04:55:29.634144+0000","last_became_peered":"2023-10-06T04:55:29.634144+0000","last_unstale":"2023-10-06T04:55:30.626889+0000","last_undegraded":"2023-10-06T04:55:30.626889+0000","last_fullsized":"2023-10-06T04:55:30.626889+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:14.628720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.633006+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:55:30.633006+0000","last_peered":"2023-10-06T04:55:30.633006+0000","last_clean":"2023-10-06T04:55:30.633006+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:55:30.633006+0000","last_undegraded":"2023-10-06T04:55:30.633006+0000","last_fullsized":"2023-10-06T04:55:30.633006+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.652470+0000","last_change":"2023-10-06T04:55:30.650432+0000","last_active":"2023-10-06T04:55:30.652470+0000","last_peered":"2023-10-06T04:55:30.652470+0000","last_clean":"2023-10-06T04:55:30.652470+0000","last_became_active":"2023-10-06T04:55:30.650130+0000","last_became_peered":"2023-10-06T04:55:30.650130+0000","last_unstale":"2023-10-06T04:55:30.652470+0000","last_undegraded":"2023-10-06T04:55:30.652470+0000","last_fullsized":"2023-10-06T04:55:30.652470+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:27.293583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.628036+0000","last_change":"2023-10-06T04:55:29.637914+0000","last_active":"2023-10-06T04:55:30.628036+0000","last_peered":"2023-10-06T04:55:30.628036+0000","last_clean":"2023-10-06T04:55:30.628036+0000","last_became_active":"2023-10-06T04:55:29.637640+0000","last_became_peered":"2023-10-06T04:55:29.637640+0000","last_unstale":"2023-10-06T04:55:30.628036+0000","last_undegraded":"2023-10-06T04:55:30.628036+0000","last_fullsized":"2023-10-06T04:55:30.628036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:15.026328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.633428+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:55:30.633428+0000","last_peered":"2023-10-06T04:55:30.633428+0000","last_clean":"2023-10-06T04:55:30.633428+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:55:30.633428+0000","last_undegraded":"2023-10-06T04:55:30.633428+0000","last_fullsized":"2023-10-06T04:55:30.633428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.653590+0000","last_change":"2023-10-06T04:55:30.650662+0000","last_active":"2023-10-06T04:55:30.653590+0000","last_peered":"2023-10-06T04:55:30.653590+0000","last_clean":"2023-10-06T04:55:30.653590+0000","last_became_active":"2023-10-06T04:55:30.650361+0000","last_became_peered":"2023-10-06T04:55:30.650361+0000","last_unstale":"2023-10-06T04:55:30.653590+0000","last_undegraded":"2023-10-06T04:55:30.653590+0000","last_fullsized":"2023-10-06T04:55:30.653590+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:10.873681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.658365+0000","last_change":"2023-10-06T04:55:29.641718+0000","last_active":"2023-10-06T04:55:30.658365+0000","last_peered":"2023-10-06T04:55:30.658365+0000","last_clean":"2023-10-06T04:55:30.658365+0000","last_became_active":"2023-10-06T04:55:29.641423+0000","last_became_peered":"2023-10-06T04:55:29.641423+0000","last_unstale":"2023-10-06T04:55:30.658365+0000","last_undegraded":"2023-10-06T04:55:30.658365+0000","last_fullsized":"2023-10-06T04:55:30.658365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:26.283304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.631962+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:55:30.631962+0000","last_peered":"2023-10-06T04:55:30.631962+0000","last_clean":"2023-10-06T04:55:30.631962+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:55:30.631962+0000","last_undegraded":"2023-10-06T04:55:30.631962+0000","last_fullsized":"2023-10-06T04:55:30.631962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.655693+0000","last_change":"2023-10-06T04:55:30.638286+0000","last_active":"2023-10-06T04:55:30.655693+0000","last_peered":"2023-10-06T04:55:30.655693+0000","last_clean":"2023-10-06T04:55:30.655693+0000","last_became_active":"2023-10-06T04:55:30.638010+0000","last_became_peered":"2023-10-06T04:55:30.638010+0000","last_unstale":"2023-10-06T04:55:30.655693+0000","last_undegraded":"2023-10-06T04:55:30.655693+0000","last_fullsized":"2023-10-06T04:55:30.655693+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:33.254947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.626591+0000","last_change":"2023-10-06T04:55:29.639321+0000","last_active":"2023-10-06T04:55:30.626591+0000","last_peered":"2023-10-06T04:55:30.626591+0000","last_clean":"2023-10-06T04:55:30.626591+0000","last_became_active":"2023-10-06T04:55:29.639057+0000","last_became_peered":"2023-10-06T04:55:29.639057+0000","last_unstale":"2023-10-06T04:55:30.626591+0000","last_undegraded":"2023-10-06T04:55:30.626591+0000","last_fullsized":"2023-10-06T04:55:30.626591+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.045081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.631685+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:55:30.631685+0000","last_peered":"2023-10-06T04:55:30.631685+0000","last_clean":"2023-10-06T04:55:30.631685+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:55:30.631685+0000","last_undegraded":"2023-10-06T04:55:30.631685+0000","last_fullsized":"2023-10-06T04:55:30.631685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.649609+0000","last_change":"2023-10-06T04:55:30.646071+0000","last_active":"2023-10-06T04:55:30.649609+0000","last_peered":"2023-10-06T04:55:30.649609+0000","last_clean":"2023-10-06T04:55:30.649609+0000","last_became_active":"2023-10-06T04:55:30.645807+0000","last_became_peered":"2023-10-06T04:55:30.645807+0000","last_unstale":"2023-10-06T04:55:30.649609+0000","last_undegraded":"2023-10-06T04:55:30.649609+0000","last_fullsized":"2023-10-06T04:55:30.649609+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:29.614770+0000","last_clean_scrub_stamp":"2023-10-06T04:55:29.614770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:03.694995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.657505+0000","last_change":"2023-10-06T04:55:29.641928+0000","last_active":"2023-10-06T04:55:30.657505+0000","last_peered":"2023-10-06T04:55:30.657505+0000","last_clean":"2023-10-06T04:55:30.657505+0000","last_became_active":"2023-10-06T04:55:29.641636+0000","last_became_peered":"2023-10-06T04:55:29.641636+0000","last_unstale":"2023-10-06T04:55:30.657505+0000","last_undegraded":"2023-10-06T04:55:30.657505+0000","last_fullsized":"2023-10-06T04:55:30.657505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:55:28.604490+0000","last_clean_scrub_stamp":"2023-10-06T04:55:28.604490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:29.769932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T04:55:30.633485+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:55:30.633485+0000","last_peered":"2023-10-06T04:55:30.633485+0000","last_clean":"2023-10-06T04:55:30.633485+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:55:30.633485+0000","last_undegraded":"2023-10-06T04:55:30.633485+0000","last_fullsized":"2023-10-06T04:55:30.633485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306916,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064924,"statfs":{"total":96636764160,"available":96322482176,"internally_reserved":0,"allocated":1470464,"data_stored":979778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.70499999999999996,"15min":0.77600000000000002},"min":{"1min":0.28999999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88700000000000001,"5min":7.1779999999999999,"15min":7.1779999999999999},"last":0.498},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.751,"15min":0.82799999999999996},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.90500000000000003,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":1.073,"15min":1.073},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81799999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.111,"15min":1.111},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.621},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78900000000000003,"5min":0.874,"15min":0.874},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":1.214,"15min":1.214},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.73199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":392277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.38,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60399999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.876,"15min":0.876},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63200000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 04:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.60399999999999998},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306920,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064920,"statfs":{"total":96636764160,"available":96322478080,"internally_reserved":0,"allocated":1474560,"data_stored":979868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35499999999999998,"15min":0.36299999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.37,"15min":0.38},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.29599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.60699999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.376},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.497},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.64300000000000002,"15min":0.66700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63100000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85299999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:55:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.026,"15min":1.026},"last":0.434},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.85599999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302240,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069600,"statfs":{"total":96636764160,"available":96327270400,"internally_reserved":0,"allocated":876544,"data_stored":390741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.34699999999999998,"15min":0.35499999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.246},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.307}]},{"osd":2,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36699999999999999},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60699999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.624,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:55:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.64400000000000002},"min":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":391455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.19500000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.54100000000000004,"5min":1.048,"15min":1.048},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.376},"min":{"1min":0.28699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.31900000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.624,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.29599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69799999999999995,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.378}]},{"osd":4,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82299999999999995,"5min":1.246,"15min":1.246},"last":0.503},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.442}]},{"osd":5,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.877,"15min":0.877},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89400000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.622},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306340,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065500,"statfs":{"total":96636764160,"available":96323072000,"internally_reserved":0,"allocated":880640,"data_stored":391295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.40999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.223},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.54800000000000004,"15min":0.60999999999999999},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.1480000000000001,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":0.29299999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.34599999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.53000000000000003,"15min":0.58599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.379,"5min":5.069,"15min":5.069},"last":0.318},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.49299999999999999},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.86099999999999999,"15min":0.83799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":14.646000000000001,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.2769999999999999,"5min":0.81799999999999995,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":14.606999999999999,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.82299999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.9219999999999999,"5min":6.4009999999999998,"15min":6.4009999999999998},"last":0.629}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74199999999999999,"15min":0.79300000000000004},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4079999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76900000000000002,"15min":0.83899999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.964,"5min":6.367,"15min":6.367},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.45900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3,"5min":5,"15min":5},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302260,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069580,"statfs":{"total":96636764160,"available":96327249920,"internally_reserved":0,"allocated":897024,"data_stored":392381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.313,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.752,"15min":0.85399999999999998},"min":{"1min":0.438,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82299999999999995,"5min":5.4569999999999999,"15min":5.4569999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.77900000000000003,"15min":0.879},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":5.5510000000000002,"15min":5.5510000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.78600000000000003,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.76200000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65300000000000002,"15min":0.70099999999999996},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":4.0739999999999998,"15min":4.0739999999999998},"last":0.496},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80400000000000005,"5min":5.415,"15min":5.415},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65000000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":4.202,"15min":4.202},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":4.2130000000000001,"15min":4.2130000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81799999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69999999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.872,"15min":0.872},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306336,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065504,"statfs":{"total":96636764160,"available":96323076096,"internally_reserved":0,"allocated":876544,"data_stored":389422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.65400000000000003,"15min":0.71899999999999997},"min":{"1min":0.26300000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73099999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.67800000000000005,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69299999999999995,"5min":6.7949999999999999,"15min":6.7949999999999999},"last":0.624},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.65900000000000003,"15min":0.70499999999999996},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.81799999999999995,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63900000000000001,"15min":0.68700000000000006},"min":{"1min":0.373,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83899999999999997,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.89500000000000002,"15min":1.044},"min":{"1min":0.39500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77300000000000002,"5min":9.7210000000000001,"15min":9.7210000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.90500000000000003,"15min":1.0409999999999999},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72399999999999998,"5min":0.751,"15min":0.751},"last":0.375},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55800000000000005,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:55:39.668 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:55:39.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:55:39.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.826+0000 7f188ed50700 1 Processor -- start 2023-10-06T04:55:39.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.826+0000 7f188ed50700 1 -- start start 2023-10-06T04:55:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f188ed50700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 0x7f188815de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f188ed50700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188815e3b0 con 0x7f1888159eb0 2023-10-06T04:55:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f188ed50700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18880afca0 con 0x7f18880cfb80 2023-10-06T04:55:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f188ed50700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188815d510 con 0x7f188815da10 2023-10-06T04:55:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f1887fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1888159eb0 0x7f18880ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56048/0 (socket says 172.21.15.3:56048) 2023-10-06T04:55:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f1887fff700 1 -- 172.21.15.3:0/1375308301 learned_addr learned my addr 172.21.15.3:0/1375308301 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142537169 0 0) 0x7f188815e3b0 con 0x7f1888159eb0 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1870003570 con 0x7f1888159eb0 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.827+0000 7f188d2ed700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 0x7f188815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39679005 0 0) 0x7f1870003570 con 0x7f1888159eb0 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 msgr2=0x7f188815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 0x7f188815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 >> v1:172.21.15.153:6789/0 conn(0x7f18880cfb80 legacy=0x7f18880af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188815a480 con 0x7f1888159eb0 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f187c001470 con 0x7f1888159eb0 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 >> v1:172.21.15.3:6789/0 conn(0x7f1888159eb0 legacy=0x7f18880ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 0x7f1870003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870003fd0 0x7f1870004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870004d50 0x7f18700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188815d510 con 0x7f1870003fd0 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187000bb40 con 0x7f1870004d50 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187000bd00 con 0x7f188815da10 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f1887fff700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870003fd0 0x7f1870004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f188caec700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 0x7f1870003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f188d2ed700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870004d50 0x7f18700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f1887fff700 1 -- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 msgr2=0x7f1870003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f1887fff700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f188815da10 0x7f1870003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f1887fff700 1 -- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870004d50 msgr2=0x7f18700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f1887fff700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870004d50 0x7f18700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f1887fff700 1 -- 172.21.15.3:0/1375308301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1870003570 con 0x7f1870003fd0 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f188d2ed700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870004d50 0x7f18700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f1887fff700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870003fd0 0x7f1870004810 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c3c0 tx=0x7f187c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61de7f2fa567bedd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c00cc90 con 0x7f1870003fd0 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.828+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c00cdf0 con 0x7f1870003fd0 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f18877fe700 1 -- 172.21.15.3:0/1375308301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c005b70 con 0x7f1870003fd0 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f188ed50700 1 -- 172.21.15.3:0/1375308301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870003fd0 msgr2=0x7f1870004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f188ed50700 1 --2- 172.21.15.3:0/1375308301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870003fd0 0x7f1870004810 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c3c0 tx=0x7f187c001e70 comp rx=0 tx=0).stop 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f188d2ed700 1 -- 172.21.15.3:0/1375308301 reap_dead start 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f188ed50700 1 -- 172.21.15.3:0/1375308301 shutdown_connections 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f188ed50700 1 -- 172.21.15.3:0/1375308301 >> 172.21.15.3:0/1375308301 conn(0x7f1888070310 msgr2=0x7f1888070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:39.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f188ed50700 1 -- 172.21.15.3:0/1375308301 shutdown_connections 2023-10-06T04:55:39.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.829+0000 7f188ed50700 1 -- 172.21.15.3:0/1375308301 wait complete. 2023-10-06T04:55:39.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.830+0000 7f188ed50700 1 Processor -- start 2023-10-06T04:55:39.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.830+0000 7f188ed50700 1 -- start start 2023-10-06T04:55:39.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188ed50700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870003fd0 0x7f18880c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188ed50700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18880ca650 0x7f1888147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188ed50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1888148820 0x7f18880c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188ed50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18880c8f20 con 0x7f1888148820 2023-10-06T04:55:39.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188ed50700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18880b50e0 con 0x7f1870003fd0 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188ed50700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18880c8c40 con 0x7f18880ca650 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1888148820 0x7f18880c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1888148820 0x7f18880c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58094/0 (socket says 172.21.15.3:58094) 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 -- 172.21.15.3:0/3399359670 learned_addr learned my addr 172.21.15.3:0/3399359670 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188caec700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870003fd0 0x7f18880c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f1887fff700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18880ca650 0x7f1888147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 -- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18880ca650 msgr2=0x7f1888147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18880ca650 0x7f1888147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 -- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870003fd0 msgr2=0x7f18880c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870003fd0 0x7f18880c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:39.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.831+0000 7f188d2ed700 1 -- 172.21.15.3:0/3399359670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18880b4a90 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.832+0000 7f188d2ed700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1888148820 0x7f18880c74f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f18880c7480 tx=0x7f1878008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.832+0000 7f1884ff9700 1 -- 172.21.15.3:0/3399359670 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878011e70 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.832+0000 7f1884ff9700 1 -- 172.21.15.3:0/3399359670 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1878015670 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.832+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18880b4d70 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.832+0000 7f1884ff9700 1 -- 172.21.15.3:0/3399359670 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878015a10 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.832+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18881483b0 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.833+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188804c380 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.833+0000 7f1884ff9700 1 -- 172.21.15.3:0/3399359670 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f187801e400 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.834+0000 7f1884ff9700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1874066f30 0x7f18740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.834+0000 7f1884ff9700 1 -- 172.21.15.3:0/3399359670 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f18780849d0 con 0x7f1888148820 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.834+0000 7f188caec700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1874066f30 0x7f18740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:39.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.834+0000 7f188caec700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1874066f30 0x7f18740693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1880001460 tx=0x7f1880001700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:39.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:39.838+0000 7f1884ff9700 1 -- 172.21.15.3:0/3399359670 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1878055570 con 0x7f1888148820 2023-10-06T04:55:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.110+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f188806f7e0 con 0x7f1888148820 2023-10-06T04:55:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.114+0000 7f1884ff9700 1 -- 172.21.15.3:0/3399359670 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1878055100 con 0x7f1888148820 2023-10-06T04:55:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1874066f30 msgr2=0x7f18740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1874066f30 0x7f18740693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1880001460 tx=0x7f1880001700 comp rx=0 tx=0).stop 2023-10-06T04:55:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1888148820 msgr2=0x7f18880c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1888148820 0x7f18880c74f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f18880c7480 tx=0x7f1878008040 comp rx=0 tx=0).stop 2023-10-06T04:55:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 shutdown_connections 2023-10-06T04:55:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1874066f30 0x7f18740693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1870003fd0 0x7f18880c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1888148820 0x7f18880c74f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 --2- 172.21.15.3:0/3399359670 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18880ca650 0x7f1888147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 >> 172.21.15.3:0/3399359670 conn(0x7f1888070310 msgr2=0x7f18880c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 shutdown_connections 2023-10-06T04:55:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.116+0000 7f188ed50700 1 -- 172.21.15.3:0/3399359670 wait complete. 2023-10-06T04:55:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T04:55:40.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:40.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.247+0000 7f6e32821700 1 Processor -- start 2023-10-06T04:55:40.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.247+0000 7f6e32821700 1 -- start start 2023-10-06T04:55:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e32821700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e2c15c8f0 0x7f6e2c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e32821700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6e2c0d0870 0x7f6e2c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e32821700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0cf6c0 con 0x7f6e2c15ccc0 2023-10-06T04:55:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e32821700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2c0cfb50 con 0x7f6e2c15c8f0 2023-10-06T04:55:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e32821700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2c15e050 con 0x7f6e2c0d0870 2023-10-06T04:55:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e2c15c8f0 0x7f6e2c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e2c15c8f0 0x7f6e2c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58108/0 (socket says 172.21.15.3:58108) 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 -- 172.21.15.3:0/2295347007 learned_addr learned my addr 172.21.15.3:0/2295347007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e30dbe700 1 --2- 172.21.15.3:0/2295347007 >> v2:172.21.15.153:3300/0 conn(0x7f6e2c0d0870 0x7f6e2c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 -- 172.21.15.3:0/2295347007 >> v2:172.21.15.153:3300/0 conn(0x7f6e2c0d0870 msgr2=0x7f6e2c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 --2- 172.21.15.3:0/2295347007 >> v2:172.21.15.153:3300/0 conn(0x7f6e2c0d0870 0x7f6e2c0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 -- 172.21.15.3:0/2295347007 >> v1:172.21.15.153:6789/0 conn(0x7f6e2c15ccc0 legacy=0x7f6e2c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 -- 172.21.15.3:0/2295347007 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0ceab0 con 0x7f6e2c15c8f0 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e30dbe700 1 --2- 172.21.15.3:0/2295347007 >> v2:172.21.15.153:3300/0 conn(0x7f6e2c0d0870 0x7f6e2c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2bfff700 1 --2- 172.21.15.3:0/2295347007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c15c8f0 0x7f6e2c15db10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0099b0 tx=0x7f6e1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234dea5e137480af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2affd700 1 -- 172.21.15.3:0/2295347007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00b7e0 con 0x7f6e2c15c8f0 2023-10-06T04:55:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2affd700 1 -- 172.21.15.3:0/2295347007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c00b940 con 0x7f6e2c15c8f0 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.248+0000 7f6e2affd700 1 -- 172.21.15.3:0/2295347007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00bce0 con 0x7f6e2c15c8f0 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 -- 172.21.15.3:0/2295347007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c15c8f0 msgr2=0x7f6e2c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 --2- 172.21.15.3:0/2295347007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c15c8f0 0x7f6e2c15db10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0099b0 tx=0x7f6e1c009250 comp rx=0 tx=0).stop 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 -- 172.21.15.3:0/2295347007 shutdown_connections 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 --2- 172.21.15.3:0/2295347007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c15c8f0 0x7f6e2c15db10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 --2- 172.21.15.3:0/2295347007 >> v2:172.21.15.153:3300/0 conn(0x7f6e2c0d0870 0x7f6e2c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 -- 172.21.15.3:0/2295347007 >> 172.21.15.3:0/2295347007 conn(0x7f6e2c070310 msgr2=0x7f6e2c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 -- 172.21.15.3:0/2295347007 shutdown_connections 2023-10-06T04:55:40.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.249+0000 7f6e32821700 1 -- 172.21.15.3:0/2295347007 wait complete. 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.250+0000 7f6e32821700 1 Processor -- start 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.250+0000 7f6e32821700 1 -- start start 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e32821700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c0d0870 0x7f6e2c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e32821700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2c15c8f0 0x7f6e2c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e32821700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2c15ccc0 0x7f6e2c13fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e32821700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0b5400 con 0x7f6e2c0d0870 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e32821700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0b47f0 con 0x7f6e2c15ccc0 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e32821700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0b4ad0 con 0x7f6e2c15c8f0 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c0d0870 0x7f6e2c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c0d0870 0x7f6e2c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58118/0 (socket says 172.21.15.3:58118) 2023-10-06T04:55:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 -- 172.21.15.3:0/1516931606 learned_addr learned my addr 172.21.15.3:0/1516931606 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e30dbe700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2c15ccc0 0x7f6e2c13fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2b7fe700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2c15c8f0 0x7f6e2c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 -- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2c15c8f0 msgr2=0x7f6e2c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2c15c8f0 0x7f6e2c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 -- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2c15ccc0 msgr2=0x7f6e2c13fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2c15ccc0 0x7f6e2c13fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2bfff700 1 -- 172.21.15.3:0/1516931606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c0094f0 con 0x7f6e2c0d0870 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e2b7fe700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2c15c8f0 0x7f6e2c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.251+0000 7f6e30dbe700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2c15ccc0 0x7f6e2c13fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.252+0000 7f6e2bfff700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c0d0870 0x7f6e2c0b5890 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c06eaa0 tx=0x7f6e1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.252+0000 7f6e13fff700 1 -- 172.21.15.3:0/1516931606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c01b450 con 0x7f6e2c0d0870 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.252+0000 7f6e13fff700 1 -- 172.21.15.3:0/1516931606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c020070 con 0x7f6e2c0d0870 2023-10-06T04:55:40.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.252+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c0b4db0 con 0x7f6e2c0d0870 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.252+0000 7f6e13fff700 1 -- 172.21.15.3:0/1516931606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c01b820 con 0x7f6e2c0d0870 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.252+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c154060 con 0x7f6e2c0d0870 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.253+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c071680 con 0x7f6e2c0d0870 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.253+0000 7f6e13fff700 1 -- 172.21.15.3:0/1516931606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e1c022030 con 0x7f6e2c0d0870 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.253+0000 7f6e13fff700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.254+0000 7f6e13fff700 1 -- 172.21.15.3:0/1516931606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e1c086700 con 0x7f6e2c0d0870 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.254+0000 7f6e2b7fe700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:40.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.254+0000 7f6e2b7fe700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e18066f30 0x7f6e180693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c16ce10 tx=0x7f6e20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:40.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.258+0000 7f6e13fff700 1 -- 172.21.15.3:0/1516931606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e1c053810 con 0x7f6e2c0d0870 2023-10-06T04:55:40.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.495+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e2c071680 con 0x7f6e2c0d0870 2023-10-06T04:55:40.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.495+0000 7f6e13fff700 1 -- 172.21.15.3:0/1516931606 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f6e1c056e30 con 0x7f6e2c0d0870 2023-10-06T04:55:40.497 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:40.497 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:33.639579+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:55:40.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e18066f30 msgr2=0x7f6e180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:40.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e18066f30 0x7f6e180693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c16ce10 tx=0x7f6e20009040 comp rx=0 tx=0).stop 2023-10-06T04:55:40.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c0d0870 msgr2=0x7f6e2c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:40.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c0d0870 0x7f6e2c0b5890 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c06eaa0 tx=0x7f6e1c016000 comp rx=0 tx=0).stop 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 shutdown_connections 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2c15ccc0 0x7f6e2c13fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c0d0870 0x7f6e2c0b5890 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 --2- 172.21.15.3:0/1516931606 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2c15c8f0 0x7f6e2c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 >> 172.21.15.3:0/1516931606 conn(0x7f6e2c070310 msgr2=0x7f6e2c15f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 shutdown_connections 2023-10-06T04:55:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:40.497+0000 7f6e32821700 1 -- 172.21.15.3:0/1516931606 wait complete. 2023-10-06T04:55:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 40 2023-10-06T04:55:40.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:55:30.037469+0000', 'modified': '2023-10-06T04:55:33.639579+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6123}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6123': {'gid': 6123, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/4081212306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 4081212306}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 4081212306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:55:40.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:55:40.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T04:55:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.035+0000 7f88253a7700 1 Processor -- start 2023-10-06T04:55:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.035+0000 7f88253a7700 1 -- start start 2023-10-06T04:55:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.036+0000 7f88253a7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f88201571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.036+0000 7f88253a7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882015e4c0 con 0x7f88200b3da0 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.036+0000 7f88253a7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882015d8b0 con 0x7f88200b39d0 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.036+0000 7f88253a7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882015db90 con 0x7f882006d480 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.036+0000 7f881f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f88200b3da0 0x7f882015a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56054/0 (socket says 172.21.15.3:56054) 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.036+0000 7f881f7fe700 1 -- 172.21.15.3:0/3464706409 learned_addr learned my addr 172.21.15.3:0/3464706409 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.036+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3464706409 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f88201571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881dffb700 1 -- 172.21.15.3:0/3464706409 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541548782 0 0) 0x7f882015e4c0 con 0x7f88200b3da0 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881dffb700 1 -- 172.21.15.3:0/3464706409 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f880c003570 con 0x7f88200b3da0 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881dffb700 1 -- 172.21.15.3:0/3464706409 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806216487 0 0) 0x7f882015d8b0 con 0x7f88200b39d0 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881dffb700 1 -- 172.21.15.3:0/3464706409 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f882015e4c0 con 0x7f88200b39d0 2023-10-06T04:55:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881e7fc700 1 -- 172.21.15.3:0/3464706409 >> v1:172.21.15.153:6789/0 conn(0x7f88200b39d0 legacy=0x7f88200cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881e7fc700 1 -- 172.21.15.3:0/3464706409 >> v1:172.21.15.3:6789/0 conn(0x7f88200b3da0 legacy=0x7f882015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881e7fc700 1 -- 172.21.15.3:0/3464706409 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882015de70 con 0x7f882006d480 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3464706409 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f88201571d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8808009910 tx=0x7f88080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5fa602fb9a35a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881dffb700 1 -- 172.21.15.3:0/3464706409 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808011630 con 0x7f882006d480 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881dffb700 1 -- 172.21.15.3:0/3464706409 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8808011790 con 0x7f882006d480 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.037+0000 7f881dffb700 1 -- 172.21.15.3:0/3464706409 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808011b10 con 0x7f882006d480 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.038+0000 7f88253a7700 1 -- 172.21.15.3:0/3464706409 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 msgr2=0x7f88201571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.038+0000 7f88253a7700 1 --2- 172.21.15.3:0/3464706409 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f88201571d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8808009910 tx=0x7f88080091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.038+0000 7f88253a7700 1 -- 172.21.15.3:0/3464706409 shutdown_connections 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.038+0000 7f88253a7700 1 --2- 172.21.15.3:0/3464706409 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f88201571d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.038+0000 7f88253a7700 1 -- 172.21.15.3:0/3464706409 >> 172.21.15.3:0/3464706409 conn(0x7f88200ce2c0 msgr2=0x7f88200ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.038+0000 7f88253a7700 1 -- 172.21.15.3:0/3464706409 shutdown_connections 2023-10-06T04:55:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.038+0000 7f88253a7700 1 -- 172.21.15.3:0/3464706409 wait complete. 2023-10-06T04:55:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.039+0000 7f88253a7700 1 Processor -- start 2023-10-06T04:55:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.039+0000 7f88253a7700 1 -- start start 2023-10-06T04:55:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f88253a7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f882014c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f88253a7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88200b39d0 0x7f882016ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f88253a7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200b3da0 0x7f882016ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f88253a7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882015d340 con 0x7f88200b3da0 2023-10-06T04:55:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f88253a7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882014ed70 con 0x7f88200b39d0 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f88253a7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882014f050 con 0x7f882006d480 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200b3da0 0x7f882016ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200b3da0 0x7f882016ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58134/0 (socket says 172.21.15.3:58134) 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 -- 172.21.15.3:0/1943831469 learned_addr learned my addr 172.21.15.3:0/1943831469 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881e7fc700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88200b39d0 0x7f882016ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881effd700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f882014c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 -- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 msgr2=0x7f882014c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f882014c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 -- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88200b39d0 msgr2=0x7f882016ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88200b39d0 0x7f882016ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 -- 172.21.15.3:0/1943831469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8808009420 con 0x7f88200b3da0 2023-10-06T04:55:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f881f7fe700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200b3da0 0x7f882016ca70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f881000cc20 tx=0x7f8810001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.040+0000 7f8816ffd700 1 -- 172.21.15.3:0/1943831469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000a960 con 0x7f88200b3da0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.041+0000 7f8816ffd700 1 -- 172.21.15.3:0/1943831469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881000aac0 con 0x7f88200b3da0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.041+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882014f330 con 0x7f88200b3da0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.041+0000 7f8816ffd700 1 -- 172.21.15.3:0/1943831469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000ae80 con 0x7f88200b3da0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.041+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882015ccd0 con 0x7f88200b3da0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.042+0000 7f8816ffd700 1 -- 172.21.15.3:0/1943831469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8810007300 con 0x7f88200b3da0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.042+0000 7f8816ffd700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8800066f30 0x7f88000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.043+0000 7f8816ffd700 1 -- 172.21.15.3:0/1943831469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f88100849f0 con 0x7f88200b3da0 2023-10-06T04:55:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.043+0000 7f881effd700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8800066f30 0x7f88000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.043+0000 7f881effd700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8800066f30 0x7f88000693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f8818009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.044+0000 7f882e69fb80 1 Processor -- start 2023-10-06T04:55:41.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.044+0000 7f882e69fb80 1 -- start start 2023-10-06T04:55:41.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e8ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e8689a0 0x55673e865580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e866bd0 0x55673e865ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f882e69fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e7e5c80 con 0x55673e8689a0 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e7bca70 con 0x55673e866bd0 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673e7ba270 con 0x55673e8aca90 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e8689a0 0x55673e865580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e8689a0 0x55673e865580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58138/0 (socket says 172.21.15.3:58138) 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 -- 172.21.15.3:0/4200650049 learned_addr learned my addr 172.21.15.3:0/4200650049 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881effd700 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e8ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881f7fe700 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e866bd0 0x55673e865ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 -- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 msgr2=0x55673e8ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e8ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 -- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e866bd0 msgr2=0x55673e865ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e866bd0 0x55673e865ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.045+0000 7f881e7fc700 1 -- 172.21.15.3:0/4200650049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55673e650270 con 0x55673e8689a0 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f881e7fc700 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e8689a0 0x55673e865580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8808013000 tx=0x7f8808017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b05a0c2f0fea61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f87fffff700 1 -- 172.21.15.3:0/4200650049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808020070 con 0x55673e8689a0 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f87fffff700 1 -- 172.21.15.3:0/4200650049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8808011e80 con 0x55673e8689a0 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f87fffff700 1 -- 172.21.15.3:0/4200650049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808006b10 con 0x55673e8689a0 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f882e69fb80 1 -- 172.21.15.3:0/4200650049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e8689a0 msgr2=0x55673e865580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e8689a0 0x55673e865580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8808013000 tx=0x7f8808017000 comp rx=0 tx=0).stop 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f882e69fb80 1 -- 172.21.15.3:0/4200650049 shutdown_connections 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e866bd0 0x55673e865ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e8ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4200650049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e8689a0 0x55673e865580 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.046+0000 7f882e69fb80 1 -- 172.21.15.3:0/4200650049 >> 172.21.15.3:0/4200650049 conn(0x55673e867070 msgr2=0x55673e867440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.047+0000 7f882e69fb80 1 -- 172.21.15.3:0/4200650049 shutdown_connections 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.047+0000 7f882e69fb80 1 -- 172.21.15.3:0/4200650049 wait complete. 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.047+0000 7f882e69fb80 1 -- start start 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.047+0000 7f882e69fb80 1 Processor -- start 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e88def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e88f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f882e69fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e7b6060 con 0x55673e866bd0 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e761890 con 0x55673e8689a0 2023-10-06T04:55:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673e77cea0 con 0x55673e8aca90 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f882e69fb80 10 client.0 _ll_register_callbacks cb 0x55673e652f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e88def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e88def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58140/0 (socket says 172.21.15.3:58140) 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 -- 172.21.15.3:0/3609503026 learned_addr learned my addr 172.21.15.3:0/3609503026 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881f7fe700 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e88f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 -- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 msgr2=0x55673e88f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e88f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 -- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 msgr2=0x55673e8bfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8bfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.048+0000 7f881effd700 1 -- 172.21.15.3:0/3609503026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882015de70 con 0x55673e866bd0 2023-10-06T04:55:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f881effd700 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e88def0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f881800ef20 tx=0x7f8818014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa1e02f1a86a286 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f87fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3609503026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818013da0 con 0x55673e866bd0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3609503026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f881801b070 con 0x55673e866bd0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55673e8375c0 con 0x55673e866bd0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3609503026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8818021620 con 0x55673e866bd0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 10 client.6285 fetch_fsmap learned FSMap version 40 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55673e5f2fe0 con 0x55673e866bd0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3609503026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55673e8375c0 con 0x55673e866bd0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3609503026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8818021780 con 0x55673e866bd0 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 10 client.6285 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T04:55:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 10 client.6285 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 20 client.6285 populate_metadata read hostname 'smithi003' 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8a9b10 0x55673e88c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 4 client.6285 mds_command: new command op to 6123 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 --> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55673e758e00 con 0x55673e8a9b10 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.049+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8a9b10 0x55673e88c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.050+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8a9b10 0x55673e88c2a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a227a03b269bace server_cookie=338a91ddceb37c57 in_seq=0 out_seq=0 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.050+0000 7f87fd7fa700 10 client.6285 ms_handle_connect on v2:172.21.15.3:6834/4081212306 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.050+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3609503026 <== mds.0 v2:172.21.15.3:6834/4081212306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55673e758e00 con 0x55673e8a9b10 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.050+0000 7f87fd7fa700 10 client.6285 handle_command_reply: tid=0 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.050+0000 7f882e69fb80 1 client.6285 shutdown 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.050+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 msgr2=0x55673e88def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.050+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e88def0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f881800ef20 tx=0x7f8818014000 comp rx=0 tx=0).stop 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 shutdown_connections 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8a9b10 0x55673e88c2a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e8aca90 0x55673e88f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f87fd7fa700 0 client.6285 ms_handle_reset on v2:172.21.15.3:6834/4081212306 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e88def0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3609503026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8bfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 >> 172.21.15.3:0/3609503026 conn(0x55673e867070 msgr2=0x55673e862720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 shutdown_connections 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 1 -- 172.21.15.3:0/3609503026 wait complete. 2023-10-06T04:55:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.051+0000 7f882e69fb80 20 client.6285 trim_cache size 0 max 16384 2023-10-06T04:55:41.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 Processor -- start 2023-10-06T04:55:41.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 -- start start 2023-10-06T04:55:41.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e867610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e638e30 con 0x55673e866bd0 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e869210 con 0x55673e8689a0 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.068+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673e863ce0 con 0x55673e867070 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e867610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e867610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58148/0 (socket says 172.21.15.3:58148) 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 -- 172.21.15.3:0/4085063894 learned_addr learned my addr 172.21.15.3:0/4085063894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 -- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 msgr2=0x55673e8ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881f7fe700 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881effd700 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 -- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 msgr2=0x55673e8be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 -- 172.21.15.3:0/4085063894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55673e784780 con 0x55673e866bd0 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881f7fe700 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881effd700 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f881e7fc700 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e867610 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8808003050 tx=0x7f8808014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad7cea3f1206710 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f8814ff9700 1 -- 172.21.15.3:0/4085063894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808020040 con 0x55673e866bd0 2023-10-06T04:55:41.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f8814ff9700 1 -- 172.21.15.3:0/4085063894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8808000bc0 con 0x55673e866bd0 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.069+0000 7f8814ff9700 1 -- 172.21.15.3:0/4085063894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808039430 con 0x55673e866bd0 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 -- 172.21.15.3:0/4085063894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 msgr2=0x55673e867610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e867610 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8808003050 tx=0x7f8808014000 comp rx=0 tx=0).stop 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 -- 172.21.15.3:0/4085063894 shutdown_connections 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 --2- 172.21.15.3:0/4085063894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e867610 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 -- 172.21.15.3:0/4085063894 >> 172.21.15.3:0/4085063894 conn(0x55673e88b9e0 msgr2=0x55673e862720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 -- 172.21.15.3:0/4085063894 shutdown_connections 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 -- 172.21.15.3:0/4085063894 wait complete. 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 -- start start 2023-10-06T04:55:41.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.070+0000 7f882e69fb80 1 Processor -- start 2023-10-06T04:55:41.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e911360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f882e69fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f882e69fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e88f1f0 con 0x55673e866bd0 2023-10-06T04:55:41.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673e911b50 con 0x55673e8689a0 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f882e69fb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673e865890 con 0x55673e867070 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f882e69fb80 10 client.0 _ll_register_callbacks cb 0x55673e652f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f881effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e911360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f881effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e911360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58158/0 (socket says 172.21.15.3:58158) 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.071+0000 7f881effd700 1 -- 172.21.15.3:0/3749298575 learned_addr learned my addr 172.21.15.3:0/3749298575 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881f7fe700 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881effd700 1 -- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 msgr2=0x55673e8af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881effd700 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881effd700 1 -- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 msgr2=0x55673e8afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881effd700 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881effd700 1 -- 172.21.15.3:0/3749298575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55673e650270 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f881effd700 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e911360 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f881800e5d0 tx=0x7f8818016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3171c3c3821222a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f87fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3749298575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818039070 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3749298575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8818021ce0 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55673e85a690 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3749298575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8818006820 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f882e69fb80 10 client.6291 fetch_fsmap learned FSMap version 40 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55673e76b470 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3749298575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55673e85a690 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.072+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3749298575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f881803c020 con 0x55673e866bd0 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 10 client.6291 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T04:55:41.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 10 client.6291 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 20 client.6291 populate_metadata read hostname 'smithi003' 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8aca90 0x55673e8a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 4 client.6291 mds_command: new command op to 6123 tid=0[{"prefix": "status"}] 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 --> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55673e758b20 con 0x55673e8aca90 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8aca90 0x55673e8a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f87fd7fa700 10 client.6291 ms_handle_connect on v2:172.21.15.3:6834/4081212306 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f881e7fc700 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8aca90 0x55673e8a5e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef524ec5cc1ddedb server_cookie=57d25d220878b282 in_seq=0 out_seq=0 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f87fd7fa700 1 -- 172.21.15.3:0/3749298575 <== mds.0 v2:172.21.15.3:6834/4081212306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55673e758b20 con 0x55673e8aca90 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f87fd7fa700 10 client.6291 handle_command_reply: tid=0 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 client.6291 shutdown 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 msgr2=0x55673e911360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e911360 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f881800e5d0 tx=0x7f8818016000 comp rx=0 tx=0).stop 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 shutdown_connections 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:6834/4081212306,v1:172.21.15.3:6835/4081212306] conn(0x55673e8aca90 0x55673e8a5e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55673e8689a0 0x55673e8afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55673e866bd0 0x55673e911360 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 --2- 172.21.15.3:0/3749298575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55673e867070 0x55673e8af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f87fd7fa700 0 client.6291 ms_handle_reset on v2:172.21.15.3:6834/4081212306 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 >> 172.21.15.3:0/3749298575 conn(0x55673e88b9e0 msgr2=0x55673e862750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.073+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 shutdown_connections 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f882e69fb80 1 -- 172.21.15.3:0/3749298575 wait complete. 2023-10-06T04:55:41.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f882e69fb80 20 client.6291 trim_cache size 0 max 16384 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6123, 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.443072852, 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 40, 2023-10-06T04:55:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 72, 2023-10-06T04:55:41.085 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 72, 2023-10-06T04:55:41.085 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.537814354999998 2023-10-06T04:55:41.085 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8800066f30 msgr2=0x7f88000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8800066f30 0x7f88000693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f8818009040 comp rx=0 tx=0).stop 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200b3da0 msgr2=0x7f882016ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200b3da0 0x7f882016ca70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f881000cc20 tx=0x7f8810001a40 comp rx=0 tx=0).stop 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 shutdown_connections 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8800066f30 0x7f88000693e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88200b3da0 0x7f882016ca70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f882006d480 0x7f882014c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 --2- 172.21.15.3:0/1943831469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88200b39d0 0x7f882016ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 >> 172.21.15.3:0/1943831469 conn(0x7f88200ce2c0 msgr2=0x7f88200af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 shutdown_connections 2023-10-06T04:55:41.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.074+0000 7f88253a7700 1 -- 172.21.15.3:0/1943831469 wait complete. 2023-10-06T04:55:41.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:55:41.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.205+0000 7f1dced4d700 1 Processor -- start 2023-10-06T04:55:41.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.205+0000 7f1dced4d700 1 -- start start 2023-10-06T04:55:41.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dced4d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dc815ccc0 0x7f1dc80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dced4d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1dc80d06a0 0x7f1dc80b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dced4d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc80cf6c0 con 0x7f1dc815c8f0 2023-10-06T04:55:41.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dced4d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80cfb50 con 0x7f1dc815ccc0 2023-10-06T04:55:41.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dced4d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80d0b00 con 0x7f1dc80d06a0 2023-10-06T04:55:41.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dc815ccc0 0x7f1dc80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dccae9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1dc815c8f0 0x7f1dc815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56056/0 (socket says 172.21.15.3:56056) 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dccae9700 1 -- 172.21.15.3:0/3887630016 learned_addr learned my addr 172.21.15.3:0/3887630016 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dc815ccc0 0x7f1dc80d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58162/0 (socket says 172.21.15.3:58162) 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dcd2ea700 1 --2- 172.21.15.3:0/3887630016 >> v2:172.21.15.153:3300/0 conn(0x7f1dc80d06a0 0x7f1dc80b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc7fff700 1 -- 172.21.15.3:0/3887630016 >> v2:172.21.15.153:3300/0 conn(0x7f1dc80d06a0 msgr2=0x7f1dc80b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc7fff700 1 --2- 172.21.15.3:0/3887630016 >> v2:172.21.15.153:3300/0 conn(0x7f1dc80d06a0 0x7f1dc80b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc7fff700 1 -- 172.21.15.3:0/3887630016 >> v1:172.21.15.3:6789/0 conn(0x7f1dc815c8f0 legacy=0x7f1dc815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc7fff700 1 -- 172.21.15.3:0/3887630016 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc80ceab0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc77fe700 1 -- 172.21.15.3:0/3887630016 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605848414 0 0) 0x7f1dc80cf6c0 con 0x7f1dc815c8f0 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.206+0000 7f1dc7fff700 1 --2- 172.21.15.3:0/3887630016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80d0160 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80b3240 tx=0x7f1db40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfecca61a6057863 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dc77fe700 1 -- 172.21.15.3:0/3887630016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db400b6b0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dc77fe700 1 -- 172.21.15.3:0/3887630016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db400b810 con 0x7f1dc815ccc0 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dc77fe700 1 -- 172.21.15.3:0/3887630016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db400bbb0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 -- 172.21.15.3:0/3887630016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 msgr2=0x7f1dc80d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 --2- 172.21.15.3:0/3887630016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80d0160 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80b3240 tx=0x7f1db40091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 -- 172.21.15.3:0/3887630016 shutdown_connections 2023-10-06T04:55:41.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 --2- 172.21.15.3:0/3887630016 >> v2:172.21.15.153:3300/0 conn(0x7f1dc80d06a0 0x7f1dc80b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 --2- 172.21.15.3:0/3887630016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80d0160 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 -- 172.21.15.3:0/3887630016 >> 172.21.15.3:0/3887630016 conn(0x7f1dc8070310 msgr2=0x7f1dc8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 -- 172.21.15.3:0/3887630016 shutdown_connections 2023-10-06T04:55:41.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.207+0000 7f1dced4d700 1 -- 172.21.15.3:0/3887630016 wait complete. 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.208+0000 7f1dced4d700 1 Processor -- start 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.208+0000 7f1dced4d700 1 -- start start 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dced4d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc80d06a0 0x7f1dc8148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dced4d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc815c8f0 0x7f1dc80c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dced4d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dced4d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc81494e0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dced4d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc816aa10 con 0x7f1dc80d06a0 2023-10-06T04:55:41.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dced4d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc8169ba0 con 0x7f1dc815c8f0 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58178/0 (socket says 172.21.15.3:58178) 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 -- 172.21.15.3:0/1080406845 learned_addr learned my addr 172.21.15.3:0/1080406845 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dccae9700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc80d06a0 0x7f1dc8148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dc7fff700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc815c8f0 0x7f1dc80c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 -- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc815c8f0 msgr2=0x7f1dc80c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc815c8f0 0x7f1dc80c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 -- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc80d06a0 msgr2=0x7f1dc8148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc80d06a0 0x7f1dc8148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dcd2ea700 1 -- 172.21.15.3:0/1080406845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4009420 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.209+0000 7f1dccae9700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc80d06a0 0x7f1dc8148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.210+0000 7f1dcd2ea700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80b1ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009fd0 tx=0x7f1dc0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.210+0000 7f1dc4ff9700 1 -- 172.21.15.3:0/1080406845 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000ed50 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.210+0000 7f1dc4ff9700 1 -- 172.21.15.3:0/1080406845 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc000eeb0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.210+0000 7f1dc4ff9700 1 -- 172.21.15.3:0/1080406845 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0014650 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.210+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8169e00 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.210+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc816a3c0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.211+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc816aba0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.211+0000 7f1dc4ff9700 1 -- 172.21.15.3:0/1080406845 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1dc00149c0 con 0x7f1dc815ccc0 2023-10-06T04:55:41.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.211+0000 7f1dc4ff9700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1dac066f30 0x7f1dac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:41.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.212+0000 7f1dc4ff9700 1 -- 172.21.15.3:0/1080406845 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1dc0084e20 con 0x7f1dc815ccc0 2023-10-06T04:55:41.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.212+0000 7f1dccae9700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1dac066f30 0x7f1dac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:41.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.212+0000 7f1dccae9700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1dac066f30 0x7f1dac0693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc007fd0 tx=0x7f1dbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:41.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.216+0000 7f1dc4ff9700 1 -- 172.21.15.3:0/1080406845 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dc0051f30 con 0x7f1dc815ccc0 2023-10-06T04:55:41.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.453+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1dc8071680 con 0x7f1dc815ccc0 2023-10-06T04:55:41.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.454+0000 7f1dc4ff9700 1 -- 172.21.15.3:0/1080406845 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1dc0055550 con 0x7f1dc815ccc0 2023-10-06T04:55:41.455 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:55:41.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.455+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1dac066f30 msgr2=0x7f1dac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.455+0000 7f1dced4d700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1dac066f30 0x7f1dac0693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc007fd0 tx=0x7f1dbc009040 comp rx=0 tx=0).stop 2023-10-06T04:55:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.455+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 msgr2=0x7f1dc80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.455+0000 7f1dced4d700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80b1ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009fd0 tx=0x7f1dc0009900 comp rx=0 tx=0).stop 2023-10-06T04:55:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.455+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 shutdown_connections 2023-10-06T04:55:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.455+0000 7f1dced4d700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1dac066f30 0x7f1dac0693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.455+0000 7f1dced4d700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc80d06a0 0x7f1dc8148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.456+0000 7f1dced4d700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc815ccc0 0x7f1dc80b1ff0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.456+0000 7f1dced4d700 1 --2- 172.21.15.3:0/1080406845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc815c8f0 0x7f1dc80c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:41.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.456+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 >> 172.21.15.3:0/1080406845 conn(0x7f1dc8070310 msgr2=0x7f1dc80708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:41.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.456+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 shutdown_connections 2023-10-06T04:55:41.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:41.456+0000 7f1dced4d700 1 -- 172.21.15.3:0/1080406845 wait complete. 2023-10-06T04:55:41.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:55:41.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:55:41.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:55:41.469 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:55:41.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:55:41.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:55:41.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:55:41.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:55:41.470 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:55:41.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:41.471 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:55:41.486 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:55:41.486 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:55:41.487 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42289sec preferred_lft 42289sec 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:41.488 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:55:41.489 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:55:41.490 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:41.490 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:55:41.490 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:55:41.490 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:55:41.490 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:41.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:41.491 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:55:41.550 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:55:41.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:41.552 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:55:41.608 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:55:41.608 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:55:41.608 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:55:41.608 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:55:41.608 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:55:41.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:41.609 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:55:41.609 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:55:41.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:55:41.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:55:41.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:55:41.609 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:55:41.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:41.711 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:55:41.727 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:55:41.728 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:55:41.728 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:55:41.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:41.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:55:41.793 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:55:41.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:55:41.794 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:55:41.849 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:55:41.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:55:41.849 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:55:41.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:55:41.970 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:55:41.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:55:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:55:42.035 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:55:42.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:42.036 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:55:42.091 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:55:42.091 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:55:42.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:55:42.162 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:55:42.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:55:42.217 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:55:42.216+0000 7ff2585b8540 -1 init, newargv = 0x55ef8ab0d8e0 newargc=15 2023-10-06T04:55:42.217 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[112942]: starting ceph client 2023-10-06T04:55:42.222 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[112942]: starting fuse 2023-10-06T04:55:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:55:42.229 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:55:42.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:42.230 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:55:42.284 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:55:42.284 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:55:42.285 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:55:42.286 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:55:42.287 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:55:42.287 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:55:42.287 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:55:42.287 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:42.287 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:55:42.287 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:55:42.407 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.112942.asok 2023-10-06T04:55:42.589 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112942.asok 2023-10-06T04:55:42.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:42.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112942.asok status 2023-10-06T04:55:42.746 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:55:42.746 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:55:42.746 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:55:42.746 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "112942", 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6303, 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:55:42.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6303 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1297719750 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:55:42.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1297719750 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.6303 192.168.0.1:0/1297719750", 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/1297719750", 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 40, 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 72, 2023-10-06T04:55:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:55:42.750 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:55:42.750 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:55:42.750 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:55:42.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:42.757 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:55:42.757 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:55:42.772 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:55:42.773 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:55:42.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:42.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:55:42.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:55:42.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.996+0000 7f2ad6764700 1 Processor -- start 2023-10-06T04:55:42.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.996+0000 7f2ad6764700 1 -- start start 2023-10-06T04:55:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.996+0000 7f2ad6764700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ad01598d0 0x7f2ad00b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.996+0000 7f2ad6764700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad00cc5c0 con 0x7f2ad00cb440 2023-10-06T04:55:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.996+0000 7f2ad6764700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad00cb9b0 con 0x7f2ad0159500 2023-10-06T04:55:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.996+0000 7f2ad6764700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad00cbc90 con 0x7f2ad01598d0 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2ad4d01700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2ad00cb440 0x7f2ad00cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39798/0 (socket says 172.21.15.3:39798) 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2ad4d01700 1 -- 172.21.15.3:0/4144322646 learned_addr learned my addr 172.21.15.3:0/4144322646 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2acf7fe700 1 --2- 172.21.15.3:0/4144322646 >> v2:172.21.15.153:3300/0 conn(0x7f2ad01598d0 0x7f2ad00b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617180106 0 0) 0x7f2ad00cc5c0 con 0x7f2ad00cb440 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab8003570 con 0x7f2ad00cb440 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517710520 0 0) 0x7f2ad00cb9b0 con 0x7f2ad0159500 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad00cc5c0 con 0x7f2ad0159500 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3015401333 0 0) 0x7f2ab8003570 con 0x7f2ad00cb440 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 >> v2:172.21.15.153:3300/0 conn(0x7f2ad01598d0 msgr2=0x7f2ad00b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 --2- 172.21.15.3:0/4144322646 >> v2:172.21.15.153:3300/0 conn(0x7f2ad01598d0 0x7f2ad00b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 >> v1:172.21.15.153:6789/0 conn(0x7f2ad0159500 legacy=0x7f2ad00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00cbf70 con 0x7f2ad00cb440 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f2ac80017f0 con 0x7f2ad00cb440 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 >> v1:172.21.15.3:6789/0 conn(0x7f2ad00cb440 legacy=0x7f2ad00cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ab8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ab8004810 0x7f2ab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8005210 0x7f2ab800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00cbc90 con 0x7f2ab8003c40 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab800c130 con 0x7f2ab8005210 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab800c2c0 con 0x7f2ab8004810 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.997+0000 7f2acffff700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ab8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acf7fe700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ab8004810 0x7f2ab8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2ad4d01700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8005210 0x7f2ab800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acffff700 1 -- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ab8004810 msgr2=0x7f2ab8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acffff700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ab8004810 0x7f2ab8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acffff700 1 -- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8005210 msgr2=0x7f2ab800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acffff700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8005210 0x7f2ab800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acffff700 1 -- 172.21.15.3:0/4144322646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8003570 con 0x7f2ab8003c40 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acf7fe700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ab8004810 0x7f2ab8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2ad4d01700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8005210 0x7f2ab800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2ad4d01700 1 -- 172.21.15.3:0/4144322646 reap_dead start 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2acffff700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ab8004030 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0001a40 tx=0x7f2ac0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113a8cdef10328c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000c590 con 0x7f2ab8003c40 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac000c6f0 con 0x7f2ab8003c40 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2aceffd700 1 -- 172.21.15.3:0/4144322646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000ca90 con 0x7f2ab8003c40 2023-10-06T04:55:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2ad6764700 1 -- 172.21.15.3:0/4144322646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 msgr2=0x7f2ab8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.998+0000 7f2ad6764700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ab8004030 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0001a40 tx=0x7f2ac0007d30 comp rx=0 tx=0).stop 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.999+0000 7f2ad6764700 1 -- 172.21.15.3:0/4144322646 shutdown_connections 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.999+0000 7f2ad6764700 1 --2- 172.21.15.3:0/4144322646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ab8004030 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.999+0000 7f2ad6764700 1 -- 172.21.15.3:0/4144322646 >> 172.21.15.3:0/4144322646 conn(0x7f2ad0070310 msgr2=0x7f2ad0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.999+0000 7f2ad6764700 1 -- 172.21.15.3:0/4144322646 shutdown_connections 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:42.999+0000 7f2ad6764700 1 -- 172.21.15.3:0/4144322646 wait complete. 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.000+0000 7f2ad6764700 1 Processor -- start 2023-10-06T04:55:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.000+0000 7f2ad6764700 1 -- start start 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ad01457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ad00c5860 0x7f2ad0154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad00c5cd0 0x7f2ad0154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00c61b0 con 0x7f2ab8003c40 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0145cf0 con 0x7f2ad00c5cd0 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad015a6e0 con 0x7f2ad00c5860 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ad01457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ad01457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36938/0 (socket says 172.21.15.3:36938) 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 -- 172.21.15.3:0/260527304 learned_addr learned my addr 172.21.15.3:0/260527304 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad4d01700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad00c5cd0 0x7f2ad0154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acf7fe700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ad00c5860 0x7f2ad0154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 -- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ad00c5860 msgr2=0x7f2ad0154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ad00c5860 0x7f2ad0154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 -- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad00c5cd0 msgr2=0x7f2ad0154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad00c5cd0 0x7f2ad0154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 -- 172.21.15.3:0/260527304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad015a9c0 con 0x7f2ab8003c40 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad4d01700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad00c5cd0 0x7f2ad0154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2acffff700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ad01457b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c3c0 tx=0x7f2ac0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ab7fff700 1 -- 172.21.15.3:0/260527304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000c590 con 0x7f2ab8003c40 2023-10-06T04:55:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ab7fff700 1 -- 172.21.15.3:0/260527304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac000ce40 con 0x7f2ab8003c40 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad015aca0 con 0x7f2ab8003c40 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ab7fff700 1 -- 172.21.15.3:0/260527304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac001e630 con 0x7f2ab8003c40 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.001+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad0155c00 con 0x7f2ab8003c40 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.002+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad015b140 con 0x7f2ab8003c40 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.003+0000 7f2ab7fff700 1 -- 172.21.15.3:0/260527304 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ac0019030 con 0x7f2ab8003c40 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.003+0000 7f2ab7fff700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2abc066f30 0x7f2abc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.003+0000 7f2ab7fff700 1 -- 172.21.15.3:0/260527304 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ac0085fb0 con 0x7f2ab8003c40 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.003+0000 7f2acf7fe700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2abc066f30 0x7f2abc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.004+0000 7f2acf7fe700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2abc066f30 0x7f2abc0693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0153fb0 tx=0x7f2ac4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.007+0000 7f2ab7fff700 1 -- 172.21.15.3:0/260527304 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ac00b75a0 con 0x7f2ab8003c40 2023-10-06T04:55:43.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.246+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2ad015b2f0 con 0x7f2ab8003c40 2023-10-06T04:55:43.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.246+0000 7f2ab7fff700 1 -- 172.21.15.3:0/260527304 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2ac0022060 con 0x7f2ab8003c40 2023-10-06T04:55:43.248 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:55:43.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.248+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2abc066f30 msgr2=0x7f2abc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:43.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2abc066f30 0x7f2abc0693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0153fb0 tx=0x7f2ac4008040 comp rx=0 tx=0).stop 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 msgr2=0x7f2ad01457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ad01457b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c3c0 tx=0x7f2ac0015000 comp rx=0 tx=0).stop 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 shutdown_connections 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2abc066f30 0x7f2abc0693e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad00c5cd0 0x7f2ad0154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8003c40 0x7f2ad01457b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 --2- 172.21.15.3:0/260527304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ad00c5860 0x7f2ad0154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 >> 172.21.15.3:0/260527304 conn(0x7f2ad0070310 msgr2=0x7f2ad00b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 shutdown_connections 2023-10-06T04:55:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:43.249+0000 7f2ad6764700 1 -- 172.21.15.3:0/260527304 wait complete. 2023-10-06T04:55:43.262 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:55:43.262 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:55:43.262 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:55:43.262 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:55:43.263 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:55:43.263 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:55:43.263 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:55:43.263 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:55:43.263 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:55:43.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:43.263 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:55:43.282 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42289sec preferred_lft 42289sec 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:55:43.283 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:43.284 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:55:43.285 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:55:43.285 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T04:55:43.285 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:55:43.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:43.285 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:55:43.350 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:55:43.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:43.352 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:55:43.410 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:55:43.410 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:55:43.410 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:55:43.410 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:55:43.411 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:55:43.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:43.411 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:55:43.412 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:55:43.412 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:55:43.412 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:55:43.412 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:55:43.412 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:55:43.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:43.518 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:55:43.533 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T04:55:43.533 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:55:43.533 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:55:43.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:55:43.533 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T04:55:43.598 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:55:43.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:55:43.598 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:55:43.653 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:55:43.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:55:43.653 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:55:43.707 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:55:43.772 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:55:43.773 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:55:43.837 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:55:43.837 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:55:43.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:43.838 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:55:43.892 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:55:43.892 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:55:43.935 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:55:43.959 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:55:43.959 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:55:44.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T04:55:44.013+0000 7f36c4e6e540 -1 init, newargv = 0x55fd7bfe1fe0 newargc=15 2023-10-06T04:55:44.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[99939]: starting ceph client 2023-10-06T04:55:44.020 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[99939]: starting fuse 2023-10-06T04:55:44.024 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:55:44.026 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:55:44.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:44.027 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:55:44.081 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:55:44.081 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:55:44.081 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:55:44.082 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:55:44.083 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:55:44.084 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:55:44.084 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:55:44.084 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:55:44.084 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:55:44.084 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:55:44.084 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T04:55:44.084 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:55:44.201 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.99939.asok 2023-10-06T04:55:44.383 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.99939.asok 2023-10-06T04:55:44.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:44.383 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.99939.asok status 2023-10-06T04:55:44.533 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:55:44.533 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:55:44.533 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:55:44.533 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "99939", 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 5576, 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:55:44.534 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 5576 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2412354662 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:55:44.535 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2412354662 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.5576 192.168.0.1:0/2412354662", 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/2412354662", 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 40, 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 72, 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:55:44.536 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:55:44.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:44.543 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:55:44.543 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:55:44.558 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:55:44.558 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:55:44.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:44.559 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:55:44.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-10-06T04:55:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.758+0000 7efdd0dd0700 1 Processor -- start 2023-10-06T04:55:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.758+0000 7efdd0dd0700 1 -- start start 2023-10-06T04:55:44.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.759+0000 7efdd0dd0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efdcc143140 0x7efdcc067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:44.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.759+0000 7efdd0dd0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc067ff0 0x7efdcc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:44.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.759+0000 7efdd0dd0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdcc068450 con 0x7efdcc142d70 2023-10-06T04:55:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.759+0000 7efdd0dd0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc0cc880 con 0x7efdcc143140 2023-10-06T04:55:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.759+0000 7efdd0dd0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc069f10 con 0x7efdcc067ff0 2023-10-06T04:55:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efdcc143140 0x7efdcc067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdcad9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc067ff0 0x7efdcc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efdcc143140 0x7efdcc067ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:57532/0 (socket says 172.21.15.3:57532) 2023-10-06T04:55:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdca59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7efdcc142d70 0x7efdcc0cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:45720/0 (socket says 172.21.15.3:45720) 2023-10-06T04:55:44.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 -- 172.21.15.3:0/2468527186 learned_addr learned my addr 172.21.15.3:0/2468527186 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 -- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc067ff0 msgr2=0x7efdcc06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 --2- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc067ff0 0x7efdcc06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 -- 172.21.15.3:0/2468527186 >> v1:172.21.15.153:6789/0 conn(0x7efdcc142d70 legacy=0x7efdcc0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 -- 172.21.15.3:0/2468527186 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc069a80 con 0x7efdcc143140 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc959a700 1 -- 172.21.15.3:0/2468527186 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054599289 0 0) 0x7efdc4001490 con 0x7efdcc142d70 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdcad9d700 1 --2- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc067ff0 0x7efdcc06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.760+0000 7efdc9d9b700 1 --2- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc143140 0x7efdcc067ab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efdb40099e0 tx=0x7efdb4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61586e4a642b383 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdc959a700 1 -- 172.21.15.3:0/2468527186 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb400b820 con 0x7efdcc143140 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdc959a700 1 -- 172.21.15.3:0/2468527186 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb400b980 con 0x7efdcc143140 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdc959a700 1 -- 172.21.15.3:0/2468527186 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb400bd20 con 0x7efdcc143140 2023-10-06T04:55:44.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc143140 msgr2=0x7efdcc067ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc143140 0x7efdcc067ab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efdb40099e0 tx=0x7efdb4009250 comp rx=0 tx=0).stop 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2468527186 shutdown_connections 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc067ff0 0x7efdcc06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2468527186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc143140 0x7efdcc067ab0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2468527186 >> 172.21.15.3:0/2468527186 conn(0x7efdcc0ce210 msgr2=0x7efdcc0ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2468527186 shutdown_connections 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.761+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2468527186 wait complete. 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.762+0000 7efdd0dd0700 1 Processor -- start 2023-10-06T04:55:44.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.762+0000 7efdd0dd0700 1 -- start start 2023-10-06T04:55:44.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdd0dd0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc067ff0 0x7efdcc1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:44.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdd0dd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdcc142d70 0x7efdcc0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdd0dd0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc143140 0x7efdcc0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdd0dd0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0c9030 con 0x7efdcc142d70 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdd0dd0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc159630 con 0x7efdcc067ff0 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdd0dd0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0b2c70 con 0x7efdcc143140 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdc9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdcc142d70 0x7efdcc0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdcad9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc143140 0x7efdcc0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdca59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc067ff0 0x7efdcc1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdc9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdcc142d70 0x7efdcc0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36942/0 (socket says 172.21.15.3:36942) 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.763+0000 7efdc9d9b700 1 -- 172.21.15.3:0/2557860865 learned_addr learned my addr 172.21.15.3:0/2557860865 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdca59c700 1 -- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc143140 msgr2=0x7efdcc0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:44.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdca59c700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc143140 0x7efdcc0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdca59c700 1 -- 172.21.15.3:0/2557860865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdcc142d70 msgr2=0x7efdcc0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdca59c700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdcc142d70 0x7efdcc0c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdca59c700 1 -- 172.21.15.3:0/2557860865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009420 con 0x7efdcc067ff0 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdc9d9b700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdcc142d70 0x7efdcc0c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdcad9d700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc143140 0x7efdcc0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdca59c700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc067ff0 0x7efdcc1590f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efdc400cc70 tx=0x7efdc4001a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdc27fc700 1 -- 172.21.15.3:0/2557860865 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4005df0 con 0x7efdcc067ff0 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdc27fc700 1 -- 172.21.15.3:0/2557860865 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc400a730 con 0x7efdcc067ff0 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb40094f0 con 0x7efdcc067ff0 2023-10-06T04:55:44.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdc27fc700 1 -- 172.21.15.3:0/2557860865 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400aaf0 con 0x7efdcc067ff0 2023-10-06T04:55:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.764+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc0c89e0 con 0x7efdcc067ff0 2023-10-06T04:55:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.765+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc04c380 con 0x7efdcc067ff0 2023-10-06T04:55:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.766+0000 7efdc27fc700 1 -- 172.21.15.3:0/2557860865 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdc40070f0 con 0x7efdcc067ff0 2023-10-06T04:55:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.766+0000 7efdc27fc700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efdac066f30 0x7efdac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.766+0000 7efdc27fc700 1 -- 172.21.15.3:0/2557860865 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efdc4084520 con 0x7efdcc067ff0 2023-10-06T04:55:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.766+0000 7efdc9d9b700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efdac066f30 0x7efdac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.767+0000 7efdc9d9b700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efdac066f30 0x7efdac0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0b41c0 tx=0x7efdb4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:44.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.771+0000 7efdc27fc700 1 -- 172.21.15.3:0/2557860865 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdc40517a0 con 0x7efdcc067ff0 2023-10-06T04:55:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.965+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7efdcc068450 con 0x7efdcc067ff0 2023-10-06T04:55:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.969+0000 7efdc27fc700 1 -- 172.21.15.3:0/2557860865 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v23)=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7efdcc068450 con 0x7efdcc067ff0 2023-10-06T04:55:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.971+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efdac066f30 msgr2=0x7efdac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.971+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efdac066f30 0x7efdac0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0b41c0 tx=0x7efdb4017000 comp rx=0 tx=0).stop 2023-10-06T04:55:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.971+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc067ff0 msgr2=0x7efdcc1590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.971+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc067ff0 0x7efdcc1590f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efdc400cc70 tx=0x7efdc4001a30 comp rx=0 tx=0).stop 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.971+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 shutdown_connections 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.972+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efdac066f30 0x7efdac0693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.972+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdcc067ff0 0x7efdcc1590f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.972+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efdcc143140 0x7efdcc0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.972+0000 7efdd0dd0700 1 --2- 172.21.15.3:0/2557860865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdcc142d70 0x7efdcc0c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.972+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 >> 172.21.15.3:0/2557860865 conn(0x7efdcc0ce210 msgr2=0x7efdcc0af690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.972+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 shutdown_connections 2023-10-06T04:55:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:44.972+0000 7efdd0dd0700 1 -- 172.21.15.3:0/2557860865 wait complete. 2023-10-06T04:55:44.985 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2023-10-06T04:55:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.106+0000 7fcf8460a700 1 Processor -- start 2023-10-06T04:55:45.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.106+0000 7fcf8460a700 1 -- start start 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf8460a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c150690 0x7fcf7c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf8460a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcf7c0cba50 0x7fcf7c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf8460a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c15da80 con 0x7fcf7c15a720 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf8460a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c15a290 con 0x7fcf7c0cba50 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf8460a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0d0020 con 0x7fcf7c150690 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf82ba7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcf7c15a720 0x7fcf7c0af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39810/0 (socket says 172.21.15.3:39810) 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf82ba7700 1 -- 172.21.15.3:0/4119081521 learned_addr learned my addr 172.21.15.3:0/4119081521 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf81ba5700 1 --2- 172.21.15.3:0/4119081521 >> v2:172.21.15.153:3300/0 conn(0x7fcf7c0cba50 0x7fcf7c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf823a6700 1 --2- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c150690 0x7fcf7c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf813a4700 1 -- 172.21.15.3:0/4119081521 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150940584 0 0) 0x7fcf7c15da80 con 0x7fcf7c15a720 2023-10-06T04:55:45.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf813a4700 1 -- 172.21.15.3:0/4119081521 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf64003570 con 0x7fcf7c15a720 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf81ba5700 1 -- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c150690 msgr2=0x7fcf7c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf81ba5700 1 --2- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c150690 0x7fcf7c150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf81ba5700 1 -- 172.21.15.3:0/4119081521 >> v1:172.21.15.3:6789/0 conn(0x7fcf7c15a720 legacy=0x7fcf7c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf81ba5700 1 -- 172.21.15.3:0/4119081521 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c159f40 con 0x7fcf7c0cba50 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf823a6700 1 --2- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c150690 0x7fcf7c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.107+0000 7fcf81ba5700 1 --2- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 0x7fcf7c15d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcf700098e0 tx=0x7fcf700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e0111631836063a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf813a4700 1 -- 172.21.15.3:0/4119081521 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70011630 con 0x7fcf7c0cba50 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf813a4700 1 -- 172.21.15.3:0/4119081521 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf70011790 con 0x7fcf7c0cba50 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf813a4700 1 -- 172.21.15.3:0/4119081521 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70011b30 con 0x7fcf7c0cba50 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 -- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 msgr2=0x7fcf7c15d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 --2- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 0x7fcf7c15d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcf700098e0 tx=0x7fcf700091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 -- 172.21.15.3:0/4119081521 shutdown_connections 2023-10-06T04:55:45.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 --2- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c150690 0x7fcf7c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 --2- 172.21.15.3:0/4119081521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 0x7fcf7c15d540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 -- 172.21.15.3:0/4119081521 >> 172.21.15.3:0/4119081521 conn(0x7fcf7c070310 msgr2=0x7fcf7c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:45.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 -- 172.21.15.3:0/4119081521 shutdown_connections 2023-10-06T04:55:45.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.108+0000 7fcf8460a700 1 -- 172.21.15.3:0/4119081521 wait complete. 2023-10-06T04:55:45.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.109+0000 7fcf8460a700 1 Processor -- start 2023-10-06T04:55:45.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.109+0000 7fcf8460a700 1 -- start start 2023-10-06T04:55:45.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf8460a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 0x7fcf7c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf8460a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c150690 0x7fcf7c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf8460a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c15a720 0x7fcf7c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf8460a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c15da80 con 0x7fcf7c150690 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf8460a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0b3410 con 0x7fcf7c0cba50 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf8460a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c15d1e0 con 0x7fcf7c15a720 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c150690 0x7fcf7c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c150690 0x7fcf7c0b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36948/0 (socket says 172.21.15.3:36948) 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 -- 172.21.15.3:0/3818161389 learned_addr learned my addr 172.21.15.3:0/3818161389 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf823a6700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 0x7fcf7c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf82ba7700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c15a720 0x7fcf7c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 -- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c15a720 msgr2=0x7fcf7c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c15a720 0x7fcf7c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 -- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 msgr2=0x7fcf7c14fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 0x7fcf7c14fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.110+0000 7fcf81ba5700 1 -- 172.21.15.3:0/3818161389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009a20 con 0x7fcf7c150690 2023-10-06T04:55:45.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.111+0000 7fcf81ba5700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c150690 0x7fcf7c0b14d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0c71f0 tx=0x7fcf70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.111+0000 7fcf7a7fc700 1 -- 172.21.15.3:0/3818161389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70011900 con 0x7fcf7c150690 2023-10-06T04:55:45.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.111+0000 7fcf7a7fc700 1 -- 172.21.15.3:0/3818161389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf70011a60 con 0x7fcf7c150690 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.111+0000 7fcf7a7fc700 1 -- 172.21.15.3:0/3818161389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70025640 con 0x7fcf7c150690 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.111+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf70009420 con 0x7fcf7c150690 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.111+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c15d7a0 con 0x7fcf7c150690 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.112+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c0b35a0 con 0x7fcf7c150690 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.112+0000 7fcf7a7fc700 1 -- 172.21.15.3:0/3818161389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf70025a30 con 0x7fcf7c150690 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.112+0000 7fcf7a7fc700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf68066f30 0x7fcf680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.112+0000 7fcf7a7fc700 1 -- 172.21.15.3:0/3818161389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcf70085e20 con 0x7fcf7c150690 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.113+0000 7fcf823a6700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf68066f30 0x7fcf680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.113+0000 7fcf823a6700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf68066f30 0x7fcf680693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009fd0 tx=0x7fcf6c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.117+0000 7fcf7a7fc700 1 -- 172.21.15.3:0/3818161389 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf70052e90 con 0x7fcf7c150690 2023-10-06T04:55:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.312+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7fcf7c0b31e0 con 0x7fcf7c150690 2023-10-06T04:55:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.315+0000 7fcf7a7fc700 1 -- 172.21.15.3:0/3818161389 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v24)=0 v24) v1 ==== 119+0+0 (secure 0 0 0) 0x7fcf700564b0 con 0x7fcf7c150690 2023-10-06T04:55:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf68066f30 msgr2=0x7fcf680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf68066f30 0x7fcf680693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009fd0 tx=0x7fcf6c00b040 comp rx=0 tx=0).stop 2023-10-06T04:55:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c150690 msgr2=0x7fcf7c0b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c150690 0x7fcf7c0b14d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0c71f0 tx=0x7fcf70017000 comp rx=0 tx=0).stop 2023-10-06T04:55:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 shutdown_connections 2023-10-06T04:55:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcf68066f30 0x7fcf680693e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf7c0cba50 0x7fcf7c14fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf7c15a720 0x7fcf7c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 --2- 172.21.15.3:0/3818161389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7c150690 0x7fcf7c0b14d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.317+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 >> 172.21.15.3:0/3818161389 conn(0x7fcf7c070310 msgr2=0x7fcf7c157aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.318+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 shutdown_connections 2023-10-06T04:55:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.318+0000 7fcf8460a700 1 -- 172.21.15.3:0/3818161389 wait complete. 2023-10-06T04:55:45.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2023-10-06T04:55:45.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.451+0000 7f0ef5131700 1 Processor -- start 2023-10-06T04:55:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.451+0000 7f0ef5131700 1 -- start start 2023-10-06T04:55:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0ef5131700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0ef00cba50 0x7f0ef0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0ef5131700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ef015d540 0x7f0ef00af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0ef5131700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef015a290 con 0x7f0ef0150690 2023-10-06T04:55:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0ef5131700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef00d0020 con 0x7f0ef015d540 2023-10-06T04:55:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0ef5131700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef0159f40 con 0x7f0ef00cba50 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eef59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ef015d540 0x7f0ef00af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eeed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ef0150690 0x7f0ef015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39824/0 (socket says 172.21.15.3:39824) 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eef59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ef015d540 0x7f0ef00af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36958/0 (socket says 172.21.15.3:36958) 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eeed9d700 1 -- 172.21.15.3:0/4184558518 learned_addr learned my addr 172.21.15.3:0/4184558518 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eee59c700 1 --2- 172.21.15.3:0/4184558518 >> v2:172.21.15.153:3300/0 conn(0x7f0ef00cba50 0x7f0ef0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eef59e700 1 -- 172.21.15.3:0/4184558518 >> v2:172.21.15.153:3300/0 conn(0x7f0ef00cba50 msgr2=0x7f0ef0150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eef59e700 1 --2- 172.21.15.3:0/4184558518 >> v2:172.21.15.153:3300/0 conn(0x7f0ef00cba50 0x7f0ef0150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eef59e700 1 -- 172.21.15.3:0/4184558518 >> v1:172.21.15.3:6789/0 conn(0x7f0ef0150690 legacy=0x7f0ef015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eedd9b700 1 -- 172.21.15.3:0/4184558518 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185085945 0 0) 0x7f0ef015a290 con 0x7f0ef0150690 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.452+0000 7f0eef59e700 1 -- 172.21.15.3:0/4184558518 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef015dcb0 con 0x7f0ef015d540 2023-10-06T04:55:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0eef59e700 1 --2- 172.21.15.3:0/4184558518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef00af190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ce1e0 tx=0x7f0ee0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7cba890d84c7cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0eedd9b700 1 -- 172.21.15.3:0/4184558518 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000e070 con 0x7f0ef015d540 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0eedd9b700 1 -- 172.21.15.3:0/4184558518 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee000bb60 con 0x7f0ef015d540 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0eedd9b700 1 -- 172.21.15.3:0/4184558518 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0012430 con 0x7f0ef015d540 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 -- 172.21.15.3:0/4184558518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 msgr2=0x7f0ef00af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 --2- 172.21.15.3:0/4184558518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef00af190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ce1e0 tx=0x7f0ee0009780 comp rx=0 tx=0).stop 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 -- 172.21.15.3:0/4184558518 shutdown_connections 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 --2- 172.21.15.3:0/4184558518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef00af190 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 --2- 172.21.15.3:0/4184558518 >> v2:172.21.15.153:3300/0 conn(0x7f0ef00cba50 0x7f0ef0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 -- 172.21.15.3:0/4184558518 >> 172.21.15.3:0/4184558518 conn(0x7f0ef0070310 msgr2=0x7f0ef0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 -- 172.21.15.3:0/4184558518 shutdown_connections 2023-10-06T04:55:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.453+0000 7f0ef5131700 1 -- 172.21.15.3:0/4184558518 wait complete. 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 Processor -- start 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 -- start start 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ef00cba50 0x7f0ef014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef0150690 0x7f0ef016ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef0152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00b3bb0 con 0x7f0ef015d540 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef0152340 con 0x7f0ef0150690 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.455+0000 7f0ef5131700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00b3280 con 0x7f0ef00cba50 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef0152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef0152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36962/0 (socket says 172.21.15.3:36962) 2023-10-06T04:55:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 -- 172.21.15.3:0/67031860 learned_addr learned my addr 172.21.15.3:0/67031860 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eeed9d700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ef00cba50 0x7f0ef014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eee59c700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef0150690 0x7f0ef016ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 -- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ef00cba50 msgr2=0x7f0ef014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ef00cba50 0x7f0ef014fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 -- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef0150690 msgr2=0x7f0ef016ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef0150690 0x7f0ef016ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 -- 172.21.15.3:0/67031860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0009a20 con 0x7f0ef015d540 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0eef59e700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef0152fb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00c71f0 tx=0x7f0ee0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/67031860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0012e60 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/67031860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee001d630 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef00b3560 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/67031860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee001d9f0 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.456+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef016b2b0 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.457+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef00b3d60 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.458+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/67031860 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ee0024070 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.458+0000 7f0ee6ffd700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ed0066f30 0x7f0ed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.458+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/67031860 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0ee0086f00 con 0x7f0ef015d540 2023-10-06T04:55:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.458+0000 7f0eeed9d700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ed0066f30 0x7f0ed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.459+0000 7f0eeed9d700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ed0066f30 0x7f0ed00693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00b4ea0 tx=0x7f0ee8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.462+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/67031860 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ef00b3d60 con 0x7f0ef015d540 2023-10-06T04:55:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.658+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f0ef004c380 con 0x7f0ef015d540 2023-10-06T04:55:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.661+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/67031860 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v25)=0 v25) v1 ==== 128+0+0 (secure 0 0 0) 0x7f0ee0054010 con 0x7f0ef015d540 2023-10-06T04:55:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ed0066f30 msgr2=0x7f0ed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ed0066f30 0x7f0ed00693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00b4ea0 tx=0x7f0ee8009040 comp rx=0 tx=0).stop 2023-10-06T04:55:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 msgr2=0x7f0ef0152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef0152fb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00c71f0 tx=0x7f0ee0019000 comp rx=0 tx=0).stop 2023-10-06T04:55:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 shutdown_connections 2023-10-06T04:55:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ed0066f30 0x7f0ed00693e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef015d540 0x7f0ef0152fb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ef00cba50 0x7f0ef014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 --2- 172.21.15.3:0/67031860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef0150690 0x7f0ef016ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 >> 172.21.15.3:0/67031860 conn(0x7f0ef0070310 msgr2=0x7f0ef00af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 shutdown_connections 2023-10-06T04:55:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.663+0000 7f0ef5131700 1 -- 172.21.15.3:0/67031860 wait complete. 2023-10-06T04:55:45.677 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2023-10-06T04:55:45.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.797+0000 7f75091f5700 1 Processor -- start 2023-10-06T04:55:45.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.797+0000 7f75091f5700 1 -- start start 2023-10-06T04:55:45.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f75091f5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75040cfb80 0x7f75040af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f75091f5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7504159eb0 0x7f750415da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f75091f5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750415d510 con 0x7f75040afad0 2023-10-06T04:55:45.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f75091f5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750415a480 con 0x7f7504159eb0 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f75091f5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750415a760 con 0x7f75040cfb80 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f750259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f75040afad0 0x7f75040ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39838/0 (socket says 172.21.15.3:39838) 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f750259c700 1 -- 172.21.15.3:0/107963674 learned_addr learned my addr 172.21.15.3:0/107963674 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f7502d9d700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75040cfb80 0x7f75040af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f750359e700 1 --2- 172.21.15.3:0/107963674 >> v2:172.21.15.153:3300/0 conn(0x7f7504159eb0 0x7f750415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922230349 0 0) 0x7f750415d510 con 0x7f75040afad0 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.798+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f0003570 con 0x7f75040afad0 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 167899557 0 0) 0x7f74f0003570 con 0x7f75040afad0 2023-10-06T04:55:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75040cfb80 msgr2=0x7f75040af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75040cfb80 0x7f75040af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 >> v2:172.21.15.153:3300/0 conn(0x7f7504159eb0 msgr2=0x7f750415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 --2- 172.21.15.3:0/107963674 >> v2:172.21.15.153:3300/0 conn(0x7f7504159eb0 0x7f750415da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750415aa40 con 0x7f75040afad0 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f74ec001470 con 0x7f75040afad0 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 >> v1:172.21.15.3:6789/0 conn(0x7f75040afad0 legacy=0x7f75040ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f0003a60 0x7f74f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74f0004810 0x7f74f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f00051c0 0x7f74f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750415a480 con 0x7f74f00051c0 2023-10-06T04:55:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750415a760 con 0x7f74f0003a60 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f000bfa0 con 0x7f74f0004810 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 --2- 172.21.15.3:0/107963674 >> v2:172.21.15.153:3300/0 conn(0x7f7504159eb0 0x7f750415da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7502d9d700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75040cfb80 0x7f75040af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f00051c0 0x7f74f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750259c700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74f0004810 0x7f74f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7502d9d700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f0003a60 0x7f74f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 -- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74f0004810 msgr2=0x7f74f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74f0004810 0x7f74f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 -- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f0003a60 msgr2=0x7f74f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f0003a60 0x7f74f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 -- 172.21.15.3:0/107963674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0003570 con 0x7f74f00051c0 2023-10-06T04:55:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7502d9d700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f0003a60 0x7f74f0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f750359e700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f00051c0 0x7f74f000ba60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009db0 tx=0x7f74f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c735aedd8e03415b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4009740 con 0x7f74f00051c0 2023-10-06T04:55:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f400e920 con 0x7f74f00051c0 2023-10-06T04:55:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.799+0000 7f7501d9b700 1 -- 172.21.15.3:0/107963674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f400eca0 con 0x7f74f00051c0 2023-10-06T04:55:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.800+0000 7f75091f5700 1 -- 172.21.15.3:0/107963674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f00051c0 msgr2=0x7f74f000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.800+0000 7f75091f5700 1 --2- 172.21.15.3:0/107963674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f00051c0 0x7f74f000ba60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009db0 tx=0x7f74f40091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.800+0000 7f750359e700 1 -- 172.21.15.3:0/107963674 reap_dead start 2023-10-06T04:55:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.800+0000 7f75091f5700 1 -- 172.21.15.3:0/107963674 shutdown_connections 2023-10-06T04:55:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.800+0000 7f75091f5700 1 -- 172.21.15.3:0/107963674 >> 172.21.15.3:0/107963674 conn(0x7f7504070310 msgr2=0x7f7504070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.800+0000 7f75091f5700 1 -- 172.21.15.3:0/107963674 shutdown_connections 2023-10-06T04:55:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.800+0000 7f75091f5700 1 -- 172.21.15.3:0/107963674 wait complete. 2023-10-06T04:55:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.801+0000 7f75091f5700 1 Processor -- start 2023-10-06T04:55:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.801+0000 7f75091f5700 1 -- start start 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f75091f5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7504145f30 0x7f75040b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f75091f5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75040b3720 0x7f75040b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f75091f5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75040b2b00 0x7f7504146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f75091f5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7504156d50 con 0x7f75040b2b00 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f75091f5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75040b12d0 con 0x7f75040b3720 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f75091f5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75040b2130 con 0x7f7504145f30 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75040b2b00 0x7f7504146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75040b2b00 0x7f7504146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36976/0 (socket says 172.21.15.3:36976) 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 -- 172.21.15.3:0/2457789892 learned_addr learned my addr 172.21.15.3:0/2457789892 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750259c700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75040b3720 0x7f75040b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f7502d9d700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7504145f30 0x7f75040b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 -- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7504145f30 msgr2=0x7f75040b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7504145f30 0x7f75040b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 -- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75040b3720 msgr2=0x7f75040b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75040b3720 0x7f75040b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 -- 172.21.15.3:0/2457789892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040b1520 con 0x7f75040b2b00 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.802+0000 7f750359e700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75040b2b00 0x7f7504146800 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e790 tx=0x7f74f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.803+0000 7f74faffd700 1 -- 172.21.15.3:0/2457789892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4022070 con 0x7f75040b2b00 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.803+0000 7f74faffd700 1 -- 172.21.15.3:0/2457789892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f4009740 con 0x7f75040b2b00 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.803+0000 7f74faffd700 1 -- 172.21.15.3:0/2457789892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4013d90 con 0x7f75040b2b00 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.803+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75040b1800 con 0x7f75040b2b00 2023-10-06T04:55:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.803+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75040b1dc0 con 0x7f75040b2b00 2023-10-06T04:55:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.803+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750404c380 con 0x7f75040b2b00 2023-10-06T04:55:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.804+0000 7f74faffd700 1 -- 172.21.15.3:0/2457789892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74f4027620 con 0x7f75040b2b00 2023-10-06T04:55:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.804+0000 7f74faffd700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f74e4066f30 0x7f74e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.804+0000 7f74faffd700 1 -- 172.21.15.3:0/2457789892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f74f4085ca0 con 0x7f75040b2b00 2023-10-06T04:55:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.804+0000 7f7502d9d700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f74e4066f30 0x7f74e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.805+0000 7f7502d9d700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f74e4066f30 0x7f74e40693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f75040ca7f0 tx=0x7f74fc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:45.809+0000 7f74faffd700 1 -- 172.21.15.3:0/2457789892 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74f4052e30 con 0x7f75040b2b00 2023-10-06T04:55:46.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.003+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7f75040ce470 con 0x7f75040b2b00 2023-10-06T04:55:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.006+0000 7f74faffd700 1 -- 172.21.15.3:0/2457789892 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v26)=0 v26) v1 ==== 124+0+0 (secure 0 0 0) 0x7f74f4056450 con 0x7f75040b2b00 2023-10-06T04:55:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f74e4066f30 msgr2=0x7f74e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f74e4066f30 0x7f74e40693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f75040ca7f0 tx=0x7f74fc00b040 comp rx=0 tx=0).stop 2023-10-06T04:55:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75040b2b00 msgr2=0x7f7504146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75040b2b00 0x7f7504146800 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e790 tx=0x7f74f4019000 comp rx=0 tx=0).stop 2023-10-06T04:55:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 shutdown_connections 2023-10-06T04:55:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f74e4066f30 0x7f74e40693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75040b2b00 0x7f7504146800 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7504145f30 0x7f75040b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 --2- 172.21.15.3:0/2457789892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75040b3720 0x7f75040b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 >> 172.21.15.3:0/2457789892 conn(0x7f7504070310 msgr2=0x7f750406e220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 shutdown_connections 2023-10-06T04:55:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.008+0000 7f75091f5700 1 -- 172.21.15.3:0/2457789892 wait complete. 2023-10-06T04:55:46.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-10-06T04:55:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.141+0000 7f938a414700 1 Processor -- start 2023-10-06T04:55:46.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.141+0000 7f938a414700 1 -- start start 2023-10-06T04:55:46.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f938a414700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 0x7f9384150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f938a414700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938415a290 con 0x7f938415d540 2023-10-06T04:55:46.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f938a414700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93840d0020 con 0x7f93840cba50 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f938a414700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9384159f40 con 0x7f9384150690 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f93889b1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f938415d540 0x7f93840af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39854/0 (socket says 172.21.15.3:39854) 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f93889b1700 1 -- 172.21.15.3:0/2607771841 learned_addr learned my addr 172.21.15.3:0/2607771841 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045952113 0 0) 0x7f938415a290 con 0x7f938415d540 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f9383fff700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 0x7f9384150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.142+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9370003570 con 0x7f938415d540 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3600582962 0 0) 0x7f9370003570 con 0x7f938415d540 2023-10-06T04:55:46.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 msgr2=0x7f9384150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 0x7f9384150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 >> v1:172.21.15.153:6789/0 conn(0x7f93840cba50 legacy=0x7f938415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938415dcb0 con 0x7f938415d540 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f937c0017f0 con 0x7f938415d540 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 >> v1:172.21.15.3:6789/0 conn(0x7f938415d540 legacy=0x7f93840af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f9370003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9370004810 0x7f9370004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 0x7f93700051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384159f40 con 0x7f9370003a90 2023-10-06T04:55:46.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937000bb60 con 0x7f9370004810 2023-10-06T04:55:46.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937000bd20 con 0x7f9384150690 2023-10-06T04:55:46.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9383fff700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f9370003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f93837fe700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9370004810 0x7f9370004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f93889b1700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 0x7f93700051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9383fff700 1 -- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 msgr2=0x7f93700051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9383fff700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9384150690 0x7f93700051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9383fff700 1 -- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9370004810 msgr2=0x7f9370004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9383fff700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9370004810 0x7f9370004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9383fff700 1 -- 172.21.15.3:0/2607771841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370003570 con 0x7f9370003a90 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9383fff700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f9370003ea0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9374009c00 tx=0x7f93740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e190f6d8623dead server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.143+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400e710 con 0x7f9370003a90 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937400e870 con 0x7f9370003a90 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f9382ffd700 1 -- 172.21.15.3:0/2607771841 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400ec30 con 0x7f9370003a90 2023-10-06T04:55:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f938a414700 1 -- 172.21.15.3:0/2607771841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 msgr2=0x7f9370003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f938a414700 1 --2- 172.21.15.3:0/2607771841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f9370003ea0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9374009c00 tx=0x7f93740091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f93889b1700 1 -- 172.21.15.3:0/2607771841 reap_dead start 2023-10-06T04:55:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f938a414700 1 -- 172.21.15.3:0/2607771841 shutdown_connections 2023-10-06T04:55:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f938a414700 1 -- 172.21.15.3:0/2607771841 >> 172.21.15.3:0/2607771841 conn(0x7f9384070310 msgr2=0x7f9384070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f938a414700 1 -- 172.21.15.3:0/2607771841 shutdown_connections 2023-10-06T04:55:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.144+0000 7f938a414700 1 -- 172.21.15.3:0/2607771841 wait complete. 2023-10-06T04:55:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.145+0000 7f938a414700 1 Processor -- start 2023-10-06T04:55:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.145+0000 7f938a414700 1 -- start start 2023-10-06T04:55:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f938a414700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f93840b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f938a414700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93840b37d0 0x7f938416d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f938a414700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938416dd00 0x7f93840b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f938a414700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938416e180 con 0x7f9370003a90 2023-10-06T04:55:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f938a414700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93840b0580 con 0x7f938416dd00 2023-10-06T04:55:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f938a414700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840c9c20 con 0x7f93840b37d0 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f93840b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f93840b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36992/0 (socket says 172.21.15.3:36992) 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 -- 172.21.15.3:0/2233930193 learned_addr learned my addr 172.21.15.3:0/2233930193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f93837fe700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93840b37d0 0x7f938416d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f93889b1700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938416dd00 0x7f93840b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 -- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93840b37d0 msgr2=0x7f938416d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93840b37d0 0x7f938416d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 -- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938416dd00 msgr2=0x7f93840b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938416dd00 0x7f93840b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.146+0000 7f9383fff700 1 -- 172.21.15.3:0/2233930193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93840c9f00 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.147+0000 7f9383fff700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f93840b3290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f937000bb30 tx=0x7f937401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.147+0000 7f937bfff700 1 -- 172.21.15.3:0/2233930193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374022070 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.147+0000 7f937bfff700 1 -- 172.21.15.3:0/2233930193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937400e9e0 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.147+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93840ca1e0 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.147+0000 7f937bfff700 1 -- 172.21.15.3:0/2233930193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937401c450 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.147+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93840b51c0 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.148+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93840b0350 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.148+0000 7f937bfff700 1 -- 172.21.15.3:0/2233930193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f937401c860 con 0x7f9370003a90 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.149+0000 7f937bfff700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9364066f30 0x7f93640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.149+0000 7f937bfff700 1 -- 172.21.15.3:0/2233930193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9374085c40 con 0x7f9370003a90 2023-10-06T04:55:46.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.149+0000 7f93837fe700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9364066f30 0x7f93640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.149+0000 7f93837fe700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9364066f30 0x7f93640693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:46.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.153+0000 7f937bfff700 1 -- 172.21.15.3:0/2233930193 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93840b0350 con 0x7f9370003a90 2023-10-06T04:55:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.347+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f93840b0350 con 0x7f9370003a90 2023-10-06T04:55:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.348+0000 7f937bfff700 1 -- 172.21.15.3:0/2233930193 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v26) v1 ==== 106+0+4 (secure 0 0 0) 0x7f93840b0350 con 0x7f9370003a90 2023-10-06T04:55:46.350 INFO:teuthology.orchestra.run.smithi003.stdout:100 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9364066f30 msgr2=0x7f93640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9364066f30 0x7f93640693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c007d30 comp rx=0 tx=0).stop 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 msgr2=0x7f93840b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f93840b3290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f937000bb30 tx=0x7f937401a000 comp rx=0 tx=0).stop 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 shutdown_connections 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9364066f30 0x7f93640693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938416dd00 0x7f93840b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9370003a90 0x7f93840b3290 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 --2- 172.21.15.3:0/2233930193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f93840b37d0 0x7f938416d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 >> 172.21.15.3:0/2233930193 conn(0x7f9384070310 msgr2=0x7f9384157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:46.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 shutdown_connections 2023-10-06T04:55:46.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.350+0000 7f938a414700 1 -- 172.21.15.3:0/2233930193 wait complete. 2023-10-06T04:55:46.364 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-10-06T04:55:46.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.483+0000 7f08f36a6700 1 Processor -- start 2023-10-06T04:55:46.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.484+0000 7f08f36a6700 1 -- start start 2023-10-06T04:55:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.484+0000 7f08f36a6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f08ec068190 0x7f08ec15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.484+0000 7f08f36a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08ec15cd40 0x7f08ec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.484+0000 7f08f36a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec15d130 con 0x7f08ec067dc0 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.484+0000 7f08f36a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec0691c0 con 0x7f08ec15cd40 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.484+0000 7f08f36a6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec0694a0 con 0x7f08ec068190 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1442700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f08ec067dc0 0x7f08ec0b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39868/0 (socket says 172.21.15.3:39868) 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1442700 1 -- 172.21.15.3:0/907913234 learned_addr learned my addr 172.21.15.3:0/907913234 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1c43700 1 --2- 172.21.15.3:0/907913234 >> v2:172.21.15.3:3300/0 conn(0x7f08ec15cd40 0x7f08ec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f0c41700 1 --2- 172.21.15.3:0/907913234 >> v2:172.21.15.153:3300/0 conn(0x7f08ec068190 0x7f08ec15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08ebfff700 1 -- 172.21.15.3:0/907913234 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476158619 0 0) 0x7f08ec15d130 con 0x7f08ec067dc0 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08ebfff700 1 -- 172.21.15.3:0/907913234 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d4003570 con 0x7f08ec067dc0 2023-10-06T04:55:46.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1c43700 1 -- 172.21.15.3:0/907913234 >> v2:172.21.15.153:3300/0 conn(0x7f08ec068190 msgr2=0x7f08ec15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1c43700 1 --2- 172.21.15.3:0/907913234 >> v2:172.21.15.153:3300/0 conn(0x7f08ec068190 0x7f08ec15c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1c43700 1 -- 172.21.15.3:0/907913234 >> v1:172.21.15.3:6789/0 conn(0x7f08ec067dc0 legacy=0x7f08ec0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1c43700 1 -- 172.21.15.3:0/907913234 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec069780 con 0x7f08ec15cd40 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08f1c43700 1 --2- 172.21.15.3:0/907913234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec15cd40 0x7f08ec06d480 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0b0ce0 tx=0x7f08e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83623a183e21653f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08ebfff700 1 -- 172.21.15.3:0/907913234 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400e070 con 0x7f08ec15cd40 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08ebfff700 1 -- 172.21.15.3:0/907913234 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400bb60 con 0x7f08ec15cd40 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.485+0000 7f08ebfff700 1 -- 172.21.15.3:0/907913234 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4012430 con 0x7f08ec15cd40 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 -- 172.21.15.3:0/907913234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec15cd40 msgr2=0x7f08ec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 --2- 172.21.15.3:0/907913234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec15cd40 0x7f08ec06d480 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0b0ce0 tx=0x7f08e4009780 comp rx=0 tx=0).stop 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 -- 172.21.15.3:0/907913234 shutdown_connections 2023-10-06T04:55:46.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 --2- 172.21.15.3:0/907913234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec15cd40 0x7f08ec06d480 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 --2- 172.21.15.3:0/907913234 >> v2:172.21.15.153:3300/0 conn(0x7f08ec068190 0x7f08ec15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 -- 172.21.15.3:0/907913234 >> 172.21.15.3:0/907913234 conn(0x7f08ec0ce270 msgr2=0x7f08ec0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:46.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 -- 172.21.15.3:0/907913234 shutdown_connections 2023-10-06T04:55:46.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.486+0000 7f08f36a6700 1 -- 172.21.15.3:0/907913234 wait complete. 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.487+0000 7f08f36a6700 1 Processor -- start 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.487+0000 7f08f36a6700 1 -- start start 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f36a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec067dc0 0x7f08ec13bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f36a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08ec068190 0x7f08ec15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f36a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08ec15cd40 0x7f08ec15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f36a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0b09e0 con 0x7f08ec067dc0 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f36a6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec13c430 con 0x7f08ec068190 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f36a6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec154d60 con 0x7f08ec15cd40 2023-10-06T04:55:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec067dc0 0x7f08ec13bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec067dc0 0x7f08ec13bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37008/0 (socket says 172.21.15.3:37008) 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 -- 172.21.15.3:0/312783951 learned_addr learned my addr 172.21.15.3:0/312783951 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f0c41700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08ec068190 0x7f08ec15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1c43700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08ec15cd40 0x7f08ec15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 -- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08ec15cd40 msgr2=0x7f08ec15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08ec15cd40 0x7f08ec15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 -- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08ec068190 msgr2=0x7f08ec15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08ec068190 0x7f08ec15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 -- 172.21.15.3:0/312783951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4009a20 con 0x7f08ec067dc0 2023-10-06T04:55:46.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1c43700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08ec15cd40 0x7f08ec15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.488+0000 7f08f1442700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec067dc0 0x7f08ec13bef0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c7c0 tx=0x7f08dc007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.489+0000 7f08e97fa700 1 -- 172.21.15.3:0/312783951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0072e0 con 0x7f08ec067dc0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.489+0000 7f08e97fa700 1 -- 172.21.15.3:0/312783951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc007440 con 0x7f08ec067dc0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.489+0000 7f08e97fa700 1 -- 172.21.15.3:0/312783951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0057f0 con 0x7f08ec067dc0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.489+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec16da50 con 0x7f08ec067dc0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.489+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec16d400 con 0x7f08ec067dc0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.490+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec04c380 con 0x7f08ec067dc0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.490+0000 7f08e97fa700 1 -- 172.21.15.3:0/312783951 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08dc005c20 con 0x7f08ec067dc0 2023-10-06T04:55:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.490+0000 7f08e97fa700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.491+0000 7f08e97fa700 1 -- 172.21.15.3:0/312783951 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f08dc083b30 con 0x7f08ec067dc0 2023-10-06T04:55:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.491+0000 7f08f0c41700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.491+0000 7f08f0c41700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f08ec155290 tx=0x7f08e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.495+0000 7f08e97fa700 1 -- 172.21.15.3:0/312783951 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08dc04b5a0 con 0x7f08ec067dc0 2023-10-06T04:55:46.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.692+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f08ec069a60 con 0x7f08ec067dc0 2023-10-06T04:55:46.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.692+0000 7f08e97fa700 1 -- 172.21.15.3:0/312783951 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v26) v1 ==== 112+0+10 (secure 0 0 0) 0x7f08dc017020 con 0x7f08ec067dc0 2023-10-06T04:55:46.694 INFO:teuthology.orchestra.run.smithi003.stdout:60.000000 2023-10-06T04:55:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f08d8066f30 msgr2=0x7f08d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f08ec155290 tx=0x7f08e0008040 comp rx=0 tx=0).stop 2023-10-06T04:55:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec067dc0 msgr2=0x7f08ec13bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec067dc0 0x7f08ec13bef0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c7c0 tx=0x7f08dc007c80 comp rx=0 tx=0).stop 2023-10-06T04:55:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 shutdown_connections 2023-10-06T04:55:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08ec15cd40 0x7f08ec15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec067dc0 0x7f08ec13bef0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 --2- 172.21.15.3:0/312783951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08ec068190 0x7f08ec15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 >> 172.21.15.3:0/312783951 conn(0x7f08ec0ce270 msgr2=0x7f08ec06d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 shutdown_connections 2023-10-06T04:55:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:46.694+0000 7f08f36a6700 1 -- 172.21.15.3:0/312783951 wait complete. 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:55:46.708 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:55:46.709 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:55:46.710 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:55:46.710 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:55:46.710 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:55:46.710 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:46.710 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:55:46.710 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:55:46.793 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.112942.asok 2023-10-06T04:55:46.975 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112942.asok 2023-10-06T04:55:46.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:55:46.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112942.asok mds_sessions 2023-10-06T04:55:47.129 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6303, 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6303 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1297719750 2023-10-06T04:55:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.6303 v1:192.168.0.1:0/1297719750", 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/1297719750", 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-10-06T04:55:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6834", 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 4081212306 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6835", 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 4081212306 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 0, 2023-10-06T04:55:47.132 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2023-10-06T04:56:43.218351+0000", 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2023-10-06T04:55:43.218351+0000", 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 1, 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 40 2023-10-06T04:55:47.133 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:55:47.140 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:55:47.140 DEBUG:teuthology.orchestra.run.smithi003:> import sys 2023-10-06T04:55:47.140 DEBUG:teuthology.orchestra.run.smithi003:> import time 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> n = 400 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> if not os.path.exists(abs_path): 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(abs_path) 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> handles = [] 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(0, n): 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> fname = "file_"+str(i) 2023-10-06T04:55:47.141 DEBUG:teuthology.orchestra.run.smithi003:> path = os.path.join(abs_path, fname) 2023-10-06T04:55:47.142 DEBUG:teuthology.orchestra.run.smithi003:> handles.append(open(path, '"'"'w'"'"')) 2023-10-06T04:55:47.142 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:55:47.142 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2023-10-06T04:55:47.142 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2023-10-06T04:55:47.142 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:55:47.144 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:47.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.265+0000 7f6d1144e700 1 Processor -- start 2023-10-06T04:55:47.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.265+0000 7f6d1144e700 1 -- start start 2023-10-06T04:55:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d1144e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6d0c06b7d0 0x7f6d0c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d1144e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0c0b1c90 con 0x7f6d0c156410 2023-10-06T04:55:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d1144e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0c0b1080 con 0x7f6d0c0b0900 2023-10-06T04:55:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d1144e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0c0b1360 con 0x7f6d0c06b7d0 2023-10-06T04:55:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d0affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d0c156410 0x7f6d0c06b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39884/0 (socket says 172.21.15.3:39884) 2023-10-06T04:55:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d0affd700 1 -- 172.21.15.3:0/1364862386 learned_addr learned my addr 172.21.15.3:0/1364862386 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d0a7fc700 1 --2- 172.21.15.3:0/1364862386 >> v2:172.21.15.153:3300/0 conn(0x7f6d0c06b7d0 0x7f6d0c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245939223 0 0) 0x7f6d0c0b1c90 con 0x7f6d0c156410 2023-10-06T04:55:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf8003570 con 0x7f6d0c156410 2023-10-06T04:55:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3992908940 0 0) 0x7f6cf8003570 con 0x7f6d0c156410 2023-10-06T04:55:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 >> v2:172.21.15.153:3300/0 conn(0x7f6d0c06b7d0 msgr2=0x7f6d0c0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 --2- 172.21.15.3:0/1364862386 >> v2:172.21.15.153:3300/0 conn(0x7f6d0c06b7d0 0x7f6d0c0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 >> v1:172.21.15.153:6789/0 conn(0x7f6d0c0b0900 legacy=0x7f6d0c0c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c0b1640 con 0x7f6d0c156410 2023-10-06T04:55:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6cfc001470 con 0x7f6d0c156410 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 >> v1:172.21.15.3:6789/0 conn(0x7f6d0c156410 legacy=0x7f6d0c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cf8003a60 0x7f6cf8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf8004810 0x7f6cf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf80051c0 0x7f6cf800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0b1360 con 0x7f6cf80051c0 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf800c0d0 con 0x7f6cf8003a60 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf800c290 con 0x7f6cf8004810 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.266+0000 7f6d0a7fc700 1 --2- 172.21.15.3:0/1364862386 >> v2:172.21.15.153:3300/0 conn(0x7f6d0c06b7d0 0x7f6d0c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf80051c0 0x7f6cf800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0affd700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cf8003a60 0x7f6cf8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0a7fc700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf8004810 0x7f6cf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 -- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf8004810 msgr2=0x7f6cf8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf8004810 0x7f6cf8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 -- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cf8003a60 msgr2=0x7f6cf8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cf8003a60 0x7f6cf8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 -- 172.21.15.3:0/1364862386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf8003570 con 0x7f6cf80051c0 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 -- 172.21.15.3:0/1364862386 reap_dead start 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d0b7fe700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf80051c0 0x7f6cf800ba60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6d04001fc0 tx=0x7f6d04001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392bd70775b5bc9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04001950 con 0x7f6cf80051c0 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0400c8b0 con 0x7f6cf80051c0 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.267+0000 7f6d09ffb700 1 -- 172.21.15.3:0/1364862386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400cc90 con 0x7f6cf80051c0 2023-10-06T04:55:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.268+0000 7f6d1144e700 1 -- 172.21.15.3:0/1364862386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf80051c0 msgr2=0x7f6cf800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.268+0000 7f6d1144e700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf80051c0 0x7f6cf800ba60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6d04001fc0 tx=0x7f6d04001640 comp rx=0 tx=0).stop 2023-10-06T04:55:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.268+0000 7f6d1144e700 1 -- 172.21.15.3:0/1364862386 shutdown_connections 2023-10-06T04:55:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.268+0000 7f6d1144e700 1 --2- 172.21.15.3:0/1364862386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf80051c0 0x7f6cf800ba60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.268+0000 7f6d1144e700 1 -- 172.21.15.3:0/1364862386 >> 172.21.15.3:0/1364862386 conn(0x7f6d0c0cdc30 msgr2=0x7f6d0c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.268+0000 7f6d1144e700 1 -- 172.21.15.3:0/1364862386 shutdown_connections 2023-10-06T04:55:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.268+0000 7f6d1144e700 1 -- 172.21.15.3:0/1364862386 wait complete. 2023-10-06T04:55:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.269+0000 7f6d1144e700 1 Processor -- start 2023-10-06T04:55:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.269+0000 7f6d1144e700 1 -- start start 2023-10-06T04:55:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d1144e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf80051c0 0x7f6d0c06ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d1144e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c0c8a40 0x7f6d0c0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d1144e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d0c1509e0 0x7f6d0c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d1144e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c151380 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d1144e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0c7050 con 0x7f6d0c1509e0 2023-10-06T04:55:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d1144e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0c7330 con 0x7f6cf80051c0 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d0a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c0c8a40 0x7f6d0c0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d0a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c0c8a40 0x7f6d0c0c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37036/0 (socket says 172.21.15.3:37036) 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d0a7fc700 1 -- 172.21.15.3:0/793191529 learned_addr learned my addr 172.21.15.3:0/793191529 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d0affd700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf80051c0 0x7f6d0c06ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d0b7fe700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d0c1509e0 0x7f6d0c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d0a7fc700 1 -- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf80051c0 msgr2=0x7f6d0c06ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.270+0000 7f6d0a7fc700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf80051c0 0x7f6d0c06ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d0a7fc700 1 -- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d0c1509e0 msgr2=0x7f6d0c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d0a7fc700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d0c1509e0 0x7f6d0c150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d0a7fc700 1 -- 172.21.15.3:0/793191529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c0c7610 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d0a7fc700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c0c8a40 0x7f6d0c0c8e90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4011960 tx=0x7f6cf4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d02ffd700 1 -- 172.21.15.3:0/793191529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4015670 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d02ffd700 1 -- 172.21.15.3:0/793191529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf40157d0 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c0c78f0 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d02ffd700 1 -- 172.21.15.3:0/793191529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4015b50 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.271+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c0c6cd0 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.272+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c169860 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.272+0000 7f6d02ffd700 1 -- 172.21.15.3:0/793191529 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cf4013410 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.273+0000 7f6d02ffd700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6cec066f30 0x7f6cec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.273+0000 7f6d02ffd700 1 -- 172.21.15.3:0/793191529 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6cf4084e80 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.273+0000 7f6d0affd700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6cec066f30 0x7f6cec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.274+0000 7f6d0affd700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6cec066f30 0x7f6cec0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c158ae0 tx=0x7f6cfc001e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.278+0000 7f6d02ffd700 1 -- 172.21.15.3:0/793191529 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cf4055a20 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.537+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d0c071680 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.538+0000 7f6d02ffd700 1 -- 172.21.15.3:0/793191529 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f6cf40555b0 con 0x7f6d0c0c8a40 2023-10-06T04:55:47.540 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:47.540 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:33.639579+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:55:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6cec066f30 msgr2=0x7f6cec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6cec066f30 0x7f6cec0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c158ae0 tx=0x7f6cfc001e50 comp rx=0 tx=0).stop 2023-10-06T04:55:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c0c8a40 msgr2=0x7f6d0c0c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c0c8a40 0x7f6d0c0c8e90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4011960 tx=0x7f6cf4008040 comp rx=0 tx=0).stop 2023-10-06T04:55:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 shutdown_connections 2023-10-06T04:55:47.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6cec066f30 0x7f6cec0693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d0c1509e0 0x7f6d0c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6cf80051c0 0x7f6d0c06ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 --2- 172.21.15.3:0/793191529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c0c8a40 0x7f6d0c0c8e90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:47.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 >> 172.21.15.3:0/793191529 conn(0x7f6d0c0cdc30 msgr2=0x7f6d0c152cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:47.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 shutdown_connections 2023-10-06T04:55:47.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:47.540+0000 7f6d1144e700 1 -- 172.21.15.3:0/793191529 wait complete. 2023-10-06T04:55:47.544 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 40 2023-10-06T04:55:47.555 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T04:55:47.707 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":5576,"entity":{"name":{"type":"client","num":5576},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2412354662}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.6779532590000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9892982023559789,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5576 192.168.0.1:0/2412354662","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"99939","root":"/"}},{"id":6303,"entity":{"name":{"type":"client","num":6303},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1297719750}},"state":"open","num_leases":1,"num_caps":390,"request_load_avg":391,"uptime":5.4759303780000002,"requests_in_flight":388,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":199.87989209515464,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":195.25151924922361,"halflife":1.5},"recall_caps_throttle2o":{"value":186.09010206243565,"halflife":0.5},"session_cache_liveness":{"value":781.53653332801969,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6303 192.168.0.1:0/1297719750","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"112942","root":"/"}}] 2023-10-06T04:55:47.707 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "99939", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2412354662, "type": "any" }, "name": { "num": 5576, "type": "client" } }, "id": 5576, "inst": "client.5576 192.168.0.1:0/2412354662", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.989298202355979 }, "state": "open", "uptime": 3.677953259 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "112942", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1297719750, "type": "any" }, "name": { "num": 6303, "type": "client" } }, "id": 6303, "inst": "client.6303 192.168.0.1:0/1297719750", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 390, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 199.87989209515464 }, "recall_caps_throttle": { "halflife": 1.5, "value": 195.2515192492236 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 186.09010206243565 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 391, "requests_in_flight": 388, "session_cache_liveness": { "halflife": 300, "value": 781.5365333280197 }, "state": "open", "uptime": 5.475930378 } ] 2023-10-06T04:55:47.708 DEBUG:tasks.ceph_test_case:wait_until_equal: 390 != 402, waiting (timeout=600)... 2023-10-06T04:55:52.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:55:52.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.842+0000 7f9900735700 1 Processor -- start 2023-10-06T04:55:52.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.842+0000 7f9900735700 1 -- start start 2023-10-06T04:55:52.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f9900735700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f80b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:52.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f9900735700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f80b0ce0 con 0x7f98f80cd950 2023-10-06T04:55:52.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f9900735700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f80cb820 con 0x7f98f80b0590 2023-10-06T04:55:52.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f9900735700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f80b0a00 con 0x7f98f815cb50 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fecd2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f98f80cd950 0x7f98f80cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37816/0 (socket says 172.21.15.3:37816) 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fecd2700 1 -- 172.21.15.3:0/1154035890 learned_addr learned my addr 172.21.15.3:0/1154035890 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fe4d1700 1 --2- 172.21.15.3:0/1154035890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f80b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098409185 0 0) 0x7f98f80b0ce0 con 0x7f98f80cd950 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e4003570 con 0x7f98f80cd950 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072307087 0 0) 0x7f98f80cb820 con 0x7f98f80b0590 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f80b0ce0 con 0x7f98f80b0590 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fe4d1700 1 -- 172.21.15.3:0/1154035890 >> v1:172.21.15.153:6789/0 conn(0x7f98f80b0590 legacy=0x7f98f80b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3793347247 0 0) 0x7f98e4003570 con 0x7f98f80cd950 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fe4d1700 1 -- 172.21.15.3:0/1154035890 >> v1:172.21.15.3:6789/0 conn(0x7f98f80cd950 legacy=0x7f98f80cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.843+0000 7f98fe4d1700 1 -- 172.21.15.3:0/1154035890 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f80cb1d0 con 0x7f98f815cb50 2023-10-06T04:55:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f98fe4d1700 1 --2- 172.21.15.3:0/1154035890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f80b0050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009900 tx=0x7f98f4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552f379049165f3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:52.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00017f0 con 0x7f98f815cb50 2023-10-06T04:55:52.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4011420 con 0x7f98f815cb50 2023-10-06T04:55:52.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f98fd4cf700 1 -- 172.21.15.3:0/1154035890 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40117c0 con 0x7f98f815cb50 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f9900735700 1 -- 172.21.15.3:0/1154035890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 msgr2=0x7f98f80b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f9900735700 1 --2- 172.21.15.3:0/1154035890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f80b0050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009900 tx=0x7f98f4009120 comp rx=0 tx=0).stop 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f9900735700 1 -- 172.21.15.3:0/1154035890 shutdown_connections 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.844+0000 7f9900735700 1 --2- 172.21.15.3:0/1154035890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f80b0050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.845+0000 7f9900735700 1 -- 172.21.15.3:0/1154035890 >> 172.21.15.3:0/1154035890 conn(0x7f98f8070310 msgr2=0x7f98f8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.845+0000 7f9900735700 1 -- 172.21.15.3:0/1154035890 shutdown_connections 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.845+0000 7f9900735700 1 -- 172.21.15.3:0/1154035890 wait complete. 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 Processor -- start 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 -- start start 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f80b0590 0x7f98f8154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f80cd950 0x7f98f80cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f8158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f80cf5a0 con 0x7f98f80b0590 2023-10-06T04:55:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8154560 con 0x7f98f80cd950 2023-10-06T04:55:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f9900735700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f80ce990 con 0x7f98f815cb50 2023-10-06T04:55:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.846+0000 7f98fe4d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f80b0590 0x7f98f8154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f80b0590 0x7f98f8154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41352/0 (socket says 172.21.15.3:41352) 2023-10-06T04:55:52.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 -- 172.21.15.3:0/1572981531 learned_addr learned my addr 172.21.15.3:0/1572981531 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:52.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fecd2700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f8158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fdcd0700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f80cd950 0x7f98f80cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 -- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 msgr2=0x7f98f8158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f8158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 -- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f80cd950 msgr2=0x7f98f80cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f80cd950 0x7f98f80cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 -- 172.21.15.3:0/1572981531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40093c0 con 0x7f98f80b0590 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fecd2700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f8158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98fe4d1700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f80b0590 0x7f98f8154020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009160 tx=0x7f98f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98ee7fc700 1 -- 172.21.15.3:0/1572981531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f401cc90 con 0x7f98f80b0590 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98ee7fc700 1 -- 172.21.15.3:0/1572981531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f401cdf0 con 0x7f98f80b0590 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f80cec70 con 0x7f98f80b0590 2023-10-06T04:55:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f98ee7fc700 1 -- 172.21.15.3:0/1572981531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f401a6a0 con 0x7f98f80b0590 2023-10-06T04:55:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.847+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f80cf230 con 0x7f98f80b0590 2023-10-06T04:55:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.848+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f8071680 con 0x7f98f80b0590 2023-10-06T04:55:52.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.849+0000 7f98ee7fc700 1 -- 172.21.15.3:0/1572981531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98f401aa90 con 0x7f98f80b0590 2023-10-06T04:55:52.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.849+0000 7f98ee7fc700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98dc066f30 0x7f98dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:52.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.849+0000 7f98ee7fc700 1 -- 172.21.15.3:0/1572981531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98f4085f60 con 0x7f98f80b0590 2023-10-06T04:55:52.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.849+0000 7f98fdcd0700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98dc066f30 0x7f98dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:52.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.850+0000 7f98fdcd0700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98dc066f30 0x7f98dc0693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f98f80d1430 tx=0x7f98e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:52.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:52.853+0000 7f98ee7fc700 1 -- 172.21.15.3:0/1572981531 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98f4053070 con 0x7f98f80b0590 2023-10-06T04:55:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.093+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98f80cb4b0 con 0x7f98f80b0590 2023-10-06T04:55:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.094+0000 7f98ee7fc700 1 -- 172.21.15.3:0/1572981531 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f98f4056690 con 0x7f98f80b0590 2023-10-06T04:55:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:53.096 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:51.531356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:55:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98dc066f30 msgr2=0x7f98dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98dc066f30 0x7f98dc0693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f98f80d1430 tx=0x7f98e8009040 comp rx=0 tx=0).stop 2023-10-06T04:55:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f80b0590 msgr2=0x7f98f8154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f80b0590 0x7f98f8154020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009160 tx=0x7f98f4016000 comp rx=0 tx=0).stop 2023-10-06T04:55:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 shutdown_connections 2023-10-06T04:55:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98dc066f30 0x7f98dc0693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98f815cb50 0x7f98f8158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f80b0590 0x7f98f8154020 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 --2- 172.21.15.3:0/1572981531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f80cd950 0x7f98f80cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 >> 172.21.15.3:0/1572981531 conn(0x7f98f8070310 msgr2=0x7f98f80708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:53.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 shutdown_connections 2023-10-06T04:55:53.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.096+0000 7f9900735700 1 -- 172.21.15.3:0/1572981531 wait complete. 2023-10-06T04:55:53.099 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2023-10-06T04:55:53.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T04:55:53.249 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":5576,"entity":{"name":{"type":"client","num":5576},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2412354662}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":9.2208828159999996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9512587028234565,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5576 192.168.0.1:0/2412354662","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"99939","root":"/"}},{"id":6303,"entity":{"name":{"type":"client","num":6303},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1297719750}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":378,"uptime":11.018859935,"requests_in_flight":0,"num_completed_requests":400,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1158.0916422372288,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":385.17578135162103,"halflife":1.5},"recall_caps_throttle2o":{"value":117.16786687666476,"halflife":0.5},"session_cache_liveness":{"value":795.28628322791428,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6303 192.168.0.1:0/1297719750","completed_requests":[{"tid":5,"created_ino":"0x10000000001"},{"tid":6,"created_ino":"0x100000001f6"},{"tid":7,"created_ino":"0x100000001f7"},{"tid":8,"created_ino":"0x100000001f8"},{"tid":9,"created_ino":"0x100000001f9"},{"tid":10,"created_ino":"0x100000001fa"},{"tid":11,"created_ino":"0x100000001fb"},{"tid":12,"created_ino":"0x100000001fc"},{"tid":13,"created_ino":"0x100000001fd"},{"tid":14,"created_ino":"0x100000001fe"},{"tid":15,"created_ino":"0x100000001ff"},{"tid":16,"created_ino":"0x10000000200"},{"tid":17,"created_ino":"0x10000000201"},{"tid":18,"created_ino":"0x10000000202"},{"tid":19,"created_ino":"0x10000000203"},{"tid":20,"created_ino":"0x10000000204"},{"tid":21,"created_ino":"0x10000000205"},{"tid":22,"created_ino":"0x10000000206"},{"tid":23,"created_ino":"0x10000000207"},{"tid":24,"created_ino":"0x10000000208"},{"tid":25,"created_ino":"0x10000000209"},{"tid":26,"created_ino":"0x1000000020a"},{"tid":27,"created_ino":"0x1000000020b"},{"tid":28,"created_ino":"0x1000000020c"},{"tid":29,"created_ino":"0x1000000020d"},{"tid":30,"created_ino":"0x1000000020e"},{"tid":31,"created_ino":"0x1000000020f"},{"tid":32,"created_ino":"0x10000000210"},{"tid":33,"created_ino":"0x10000000211"},{"tid":34,"created_ino":"0x10000000212"},{"tid":35,"created_ino":"0x10000000213"},{"tid":36,"created_ino":"0x10000000214"},{"tid":37,"created_ino":"0x10000000215"},{"tid":38,"created_ino":"0x10000000216"},{"tid":39,"created_ino":"0x10000000217"},{"tid":40,"created_ino":"0x10000000218"},{"tid":41,"created_ino":"0x10000000219"},{"tid":42,"created_ino":"0x1000000021a"},{"tid":43,"created_ino":"0x1000000021b"},{"tid":44,"created_ino":"0x1000000021c"},{"tid":45,"created_ino":"0x1000000021d"},{"tid":46,"created_ino":"0x1000000021e"},{"tid":47,"created_ino":"0x1000000021f"},{"tid":48,"created_ino":"0x10000000220"},{"tid":49,"created_ino":"0x10000000221"},{"tid":50,"created_ino":"0x10000000222"},{"tid":51,"created_ino":"0x10000000223"},{"tid":52,"created_ino":"0x10000000224"},{"tid":53,"created_ino":"0x10000000225"},{"tid":54,"created_ino":"0x10000000226"},{"tid":55,"created_ino":"0x10000000227"},{"tid":56,"created_ino":"0x10000000228"},{"tid":57,"created_ino":"0x10000000229"},{"tid":58,"created_ino":"0x1000000022a"},{"tid":59,"created_ino":"0x1000000022b"},{"tid":60,"created_ino":"0x1000000022c"},{"tid":61,"created_ino":"0x1000000022d"},{"tid":62,"created_ino":"0x1000000022e"},{"tid":63,"created_ino":"0x1000000022f"},{"tid":64,"created_ino":"0x10000000230"},{"tid":65,"created_ino":"0x10000000231"},{"tid":66,"created_ino":"0x10000000232"},{"tid":67,"created_ino":"0x10000000233"},{"tid":68,"created_ino":"0x10000000234"},{"tid":69,"created_ino":"0x10000000235"},{"tid":70,"created_ino":"0x10000000236"},{"tid":71,"created_ino":"0x10000000237"},{"tid":72,"created_ino":"0x10000000238"},{"tid":73,"created_ino":"0x10000000239"},{"tid":74,"created_ino":"0x1000000023a"},{"tid":75,"created_ino":"0x1000000023b"},{"tid":76,"created_ino":"0x1000000023c"},{"tid":77,"created_ino":"0x1000000023d"},{"tid":78,"created_ino":"0x1000000023e"},{"tid":79,"created_ino":"0x1000000023f"},{"tid":80,"created_ino":"0x10000000240"},{"tid":81,"created_ino":"0x10000000241"},{"tid":82,"created_ino":"0x10000000242"},{"tid":83,"created_ino":"0x10000000243"},{"tid":84,"created_ino":"0x10000000244"},{"tid":85,"created_ino":"0x10000000245"},{"tid":86,"created_ino":"0x10000000246"},{"tid":87,"created_ino":"0x10000000247"},{"tid":88,"created_ino":"0x10000000248"},{"tid":89,"created_ino":"0x10000000249"},{"tid":90,"created_ino":"0x1000000024a"},{"tid":91,"created_ino":"0x1000000024b"},{"tid":92,"created_ino":"0x1000000024c"},{"tid":93,"created_ino":"0x1000000024d"},{"tid":94,"created_ino":"0x1000000024e"},{"tid":95,"created_ino":"0x1000000024f"},{"tid":96,"created_ino":"0x10000000250"},{"tid":97,"created_ino":"0x10000000251"},{"tid":98,"created_ino":"0x10000000252"},{"tid":99,"created_ino":"0x10000000253"},{"tid":100,"created_ino":"0x10000000254"},{"tid":101,"created_ino":"0x10000000255"},{"tid":102,"created_ino":"0x10000000256"},{"tid":103,"created_ino":"0x10000000257"},{"tid":104,"created_ino":"0x10000000258"},{"tid":105,"created_ino":"0x10000000259"},{"tid":106,"created_ino":"0x1000000025a"},{"tid":107,"created_ino":"0x1000000025b"},{"tid":108,"created_ino":"0x1000000025c"},{"tid":109,"created_ino":"0x1000000025d"},{"tid":110,"created_ino":"0x1000000025e"},{"tid":111,"created_ino":"0x1000000025f"},{"tid":112,"created_ino":"0x10000000260"},{"tid":113,"created_ino":"0x10000000261"},{"tid":114,"created_ino":"0x10000000262"},{"tid":115,"created_ino":"0x10000000263"},{"tid":116,"created_ino":"0x10000000264"},{"tid":117,"created_ino":"0x10000000265"},{"tid":118,"created_ino":"0x10000000266"},{"tid":119,"created_ino":"0x10000000267"},{"tid":120,"created_ino":"0x10000000268"},{"tid":121,"created_ino":"0x10000000269"},{"tid":122,"created_ino":"0x1000000026a"},{"tid":123,"created_ino":"0x1000000026b"},{"tid":124,"created_ino":"0x1000000026c"},{"tid":125,"created_ino":"0x1000000026d"},{"tid":126,"created_ino":"0x1000000026e"},{"tid":127,"created_ino":"0x1000000026f"},{"tid":128,"created_ino":"0x10000000270"},{"tid":129,"created_ino":"0x10000000271"},{"tid":130,"created_ino":"0x10000000272"},{"tid":131,"created_ino":"0x10000000273"},{"tid":132,"created_ino":"0x10000000274"},{"tid":133,"created_ino":"0x10000000275"},{"tid":134,"created_ino":"0x10000000276"},{"tid":135,"created_ino":"0x10000000277"},{"tid":136,"created_ino":"0x10000000278"},{"tid":137,"created_ino":"0x10000000279"},{"tid":138,"created_ino":"0x1000000027a"},{"tid":139,"created_ino":"0x1000000027b"},{"tid":140,"created_ino":"0x1000000027c"},{"tid":141,"created_ino":"0x1000000027d"},{"tid":142,"created_ino":"0x1000000027e"},{"tid":143,"created_ino":"0x1000000027f"},{"tid":144,"created_ino":"0x10000000280"},{"tid":145,"created_ino":"0x10000000281"},{"tid":146,"created_ino":"0x10000000282"},{"tid":147,"created_ino":"0x10000000283"},{"tid":148,"created_ino":"0x10000000284"},{"tid":149,"created_ino":"0x10000000285"},{"tid":150,"created_ino":"0x10000000286"},{"tid":151,"created_ino":"0x10000000287"},{"tid":152,"created_ino":"0x10000000288"},{"tid":153,"created_ino":"0x10000000289"},{"tid":154,"created_ino":"0x1000000028a"},{"tid":155,"created_ino":"0x1000000028b"},{"tid":156,"created_ino":"0x1000000028c"},{"tid":157,"created_ino":"0x1000000028d"},{"tid":158,"created_ino":"0x1000000028e"},{"tid":159,"created_ino":"0x1000000028f"},{"tid":160,"created_ino":"0x10000000290"},{"tid":161,"created_ino":"0x10000000291"},{"tid":162,"created_ino":"0x10000000292"},{"tid":163,"created_ino":"0x10000000293"},{"tid":164,"created_ino":"0x10000000294"},{"tid":165,"created_ino":"0x10000000295"},{"tid":166,"created_ino":"0x10000000296"},{"tid":167,"created_ino":"0x10000000297"},{"tid":168,"created_ino":"0x10000000298"},{"tid":169,"created_ino":"0x10000000299"},{"tid":170,"created_ino":"0x1000000029a"},{"tid":171,"created_ino":"0x1000000029b"},{"tid":172,"created_ino":"0x1000000029c"},{"tid":173,"created_ino":"0x1000000029d"},{"tid":174,"created_ino":"0x1000000029e"},{"tid":175,"created_ino":"0x1000000029f"},{"tid":176,"created_ino":"0x100000002a0"},{"tid":177,"created_ino":"0x100000002a1"},{"tid":178,"created_ino":"0x100000002a2"},{"tid":179,"created_ino":"0x100000002a3"},{"tid":180,"created_ino":"0x100000002a4"},{"tid":181,"created_ino":"0x100000002a5"},{"tid":182,"created_ino":"0x100000002a6"},{"tid":183,"created_ino":"0x100000002a7"},{"tid":184,"created_ino":"0x100000002a8"},{"tid":185,"created_ino":"0x100000002a9"},{"tid":186,"created_ino":"0x100000002aa"},{"tid":187,"created_ino":"0x100000002ab"},{"tid":188,"created_ino":"0x100000002ac"},{"tid":189,"created_ino":"0x100000002ad"},{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"},{"tid":205,"created_ino":"0x100000002bd"},{"tid":206,"created_ino":"0x100000002be"},{"tid":207,"created_ino":"0x100000002bf"},{"tid":208,"created_ino":"0x100000002c0"},{"tid":209,"created_ino":"0x100000002c1"},{"tid":210,"created_ino":"0x100000002c2"},{"tid":211,"created_ino":"0x100000002c3"},{"tid":212,"created_ino":"0x100000002c4"},{"tid":213,"created_ino":"0x100000002c5"},{"tid":214,"created_ino":"0x100000002c6"},{"tid":215,"created_ino":"0x100000002c7"},{"tid":216,"created_ino":"0x100000002c8"},{"tid":217,"created_ino":"0x100000002c9"},{"tid":218,"created_ino":"0x100000002ca"},{"tid":219,"created_ino":"0x100000002cb"},{"tid":220,"created_ino":"0x100000002cc"},{"tid":221,"created_ino":"0x100000002cd"},{"tid":222,"created_ino":"0x100000002ce"},{"tid":223,"created_ino":"0x100000002cf"},{"tid":224,"created_ino":"0x100000002d0"},{"tid":225,"created_ino":"0x100000002d1"},{"tid":226,"created_ino":"0x100000002d2"},{"tid":227,"created_ino":"0x100000002d3"},{"tid":228,"created_ino":"0x100000002d4"},{"tid":229,"created_ino":"0x100000002d5"},{"tid":230,"created_ino":"0x100000002d6"},{"tid":231,"created_ino":"0x100000002d7"},{"tid":232,"created_ino":"0x100000002d8"},{"tid":233,"created_ino":"0x100000002d9"},{"tid":234,"created_ino":"0x100000002da"},{"tid":235,"created_ino":"0x100000002db"},{"tid":236,"created_ino":"0x100000002dc"},{"tid":237,"created_ino":"0x100000002dd"},{"tid":238,"created_ino":"0x100000002de"},{"tid":239,"created_ino":"0x100000002df"},{"tid":240,"created_ino":"0x100000002e0"},{"tid":241,"created_ino":"0x100000002e1"},{"tid":242,"created_ino":"0x100000002e2"},{"tid":243,"created_ino":"0x100000002e3"},{"tid":244,"created_ino":"0x100000002e4"},{"tid":245,"created_ino":"0x100000002e5"},{"tid":246,"created_ino":"0x100000002e6"},{"tid":247,"created_ino":"0x100000002e7"},{"tid":248,"created_ino":"0x100000002e8"},{"tid":249,"created_ino":"0x100000002e9"},{"tid":250,"created_ino":"0x100000002ea"},{"tid":251,"created_ino":"0x100000002eb"},{"tid":252,"created_ino":"0x100000002ec"},{"tid":253,"created_ino":"0x100000002ed"},{"tid":254,"created_ino":"0x100000002ee"},{"tid":255,"created_ino":"0x100000002ef"},{"tid":256,"created_ino":"0x100000002f0"},{"tid":257,"created_ino":"0x100000002f1"},{"tid":258,"created_ino":"0x100000002f2"},{"tid":259,"created_ino":"0x100000002f3"},{"tid":260,"created_ino":"0x100000002f4"},{"tid":261,"created_ino":"0x100000002f5"},{"tid":262,"created_ino":"0x100000002f6"},{"tid":263,"created_ino":"0x100000002f7"},{"tid":264,"created_ino":"0x100000002f8"},{"tid":265,"created_ino":"0x100000002f9"},{"tid":266,"created_ino":"0x100000002fa"},{"tid":267,"created_ino":"0x100000002fb"},{"tid":268,"created_ino":"0x100000002fc"},{"tid":269,"created_ino":"0x100000002fd"},{"tid":270,"created_ino":"0x100000002fe"},{"tid":271,"created_ino":"0x100000002ff"},{"tid":272,"created_ino":"0x10000000300"},{"tid":273,"created_ino":"0x10000000301"},{"tid":274,"created_ino":"0x10000000302"},{"tid":275,"created_ino":"0x10000000303"},{"tid":276,"created_ino":"0x10000000304"},{"tid":277,"created_ino":"0x10000000305"},{"tid":278,"created_ino":"0x10000000306"},{"tid":279,"created_ino":"0x10000000307"},{"tid":280,"created_ino":"0x10000000308"},{"tid":281,"created_ino":"0x10000000309"},{"tid":282,"created_ino":"0x1000000030a"},{"tid":283,"created_ino":"0x1000000030b"},{"tid":284,"created_ino":"0x1000000030c"},{"tid":285,"created_ino":"0x1000000030d"},{"tid":286,"created_ino":"0x1000000030e"},{"tid":287,"created_ino":"0x1000000030f"},{"tid":288,"created_ino":"0x10000000310"},{"tid":289,"created_ino":"0x10000000311"},{"tid":290,"created_ino":"0x10000000312"},{"tid":291,"created_ino":"0x10000000313"},{"tid":292,"created_ino":"0x10000000314"},{"tid":293,"created_ino":"0x10000000315"},{"tid":294,"created_ino":"0x10000000316"},{"tid":295,"created_ino":"0x10000000317"},{"tid":296,"created_ino":"0x10000000318"},{"tid":297,"created_ino":"0x10000000319"},{"tid":298,"created_ino":"0x1000000031a"},{"tid":299,"created_ino":"0x1000000031b"},{"tid":300,"created_ino":"0x1000000031c"},{"tid":301,"created_ino":"0x1000000031d"},{"tid":302,"created_ino":"0x1000000031e"},{"tid":303,"created_ino":"0x1000000031f"},{"tid":304,"created_ino":"0x10000000320"},{"tid":305,"created_ino":"0x10000000321"},{"tid":306,"created_ino":"0x10000000322"},{"tid":307,"created_ino":"0x10000000323"},{"tid":308,"created_ino":"0x10000000324"},{"tid":309,"created_ino":"0x10000000325"},{"tid":310,"created_ino":"0x10000000326"},{"tid":311,"created_ino":"0x10000000327"},{"tid":312,"created_ino":"0x10000000328"},{"tid":313,"created_ino":"0x10000000329"},{"tid":314,"created_ino":"0x1000000032a"},{"tid":315,"created_ino":"0x1000000032b"},{"tid":316,"created_ino":"0x1000000032c"},{"tid":317,"created_ino":"0x1000000032d"},{"tid":318,"created_ino":"0x1000000032e"},{"tid":319,"created_ino":"0x1000000032f"},{"tid":320,"created_ino":"0x10000000330"},{"tid":321,"created_ino":"0x10000000331"},{"tid":322,"created_ino":"0x10000000332"},{"tid":323,"created_ino":"0x10000000333"},{"tid":324,"created_ino":"0x10000000334"},{"tid":325,"created_ino":"0x10000000335"},{"tid":326,"created_ino":"0x10000000336"},{"tid":327,"created_ino":"0x10000000337"},{"tid":328,"created_ino":"0x10000000338"},{"tid":329,"created_ino":"0x10000000339"},{"tid":330,"created_ino":"0x1000000033a"},{"tid":331,"created_ino":"0x1000000033b"},{"tid":332,"created_ino":"0x1000000033c"},{"tid":333,"created_ino":"0x1000000033d"},{"tid":334,"created_ino":"0x1000000033e"},{"tid":335,"created_ino":"0x1000000033f"},{"tid":336,"created_ino":"0x10000000340"},{"tid":337,"created_ino":"0x10000000341"},{"tid":338,"created_ino":"0x10000000342"},{"tid":339,"created_ino":"0x10000000343"},{"tid":340,"created_ino":"0x10000000344"},{"tid":341,"created_ino":"0x10000000345"},{"tid":342,"created_ino":"0x10000000346"},{"tid":343,"created_ino":"0x10000000347"},{"tid":344,"created_ino":"0x10000000348"},{"tid":345,"created_ino":"0x10000000349"},{"tid":346,"created_ino":"0x1000000034a"},{"tid":347,"created_ino":"0x1000000034b"},{"tid":348,"created_ino":"0x1000000034c"},{"tid":349,"created_ino":"0x1000000034d"},{"tid":350,"created_ino":"0x1000000034e"},{"tid":351,"created_ino":"0x1000000034f"},{"tid":352,"created_ino":"0x10000000350"},{"tid":353,"created_ino":"0x10000000351"},{"tid":354,"created_ino":"0x10000000352"},{"tid":355,"created_ino":"0x10000000353"},{"tid":356,"created_ino":"0x10000000354"},{"tid":357,"created_ino":"0x10000000355"},{"tid":358,"created_ino":"0x10000000356"},{"tid":360,"created_ino":"0x10000000357"},{"tid":361,"created_ino":"0x10000000358"},{"tid":362,"created_ino":"0x10000000359"},{"tid":363,"created_ino":"0x1000000035a"},{"tid":364,"created_ino":"0x1000000035b"},{"tid":365,"created_ino":"0x1000000035c"},{"tid":366,"created_ino":"0x1000000035d"},{"tid":367,"created_ino":"0x1000000035e"},{"tid":368,"created_ino":"0x1000000035f"},{"tid":369,"created_ino":"0x10000000360"},{"tid":370,"created_ino":"0x10000000361"},{"tid":371,"created_ino":"0x10000000362"},{"tid":372,"created_ino":"0x10000000363"},{"tid":373,"created_ino":"0x10000000364"},{"tid":374,"created_ino":"0x10000000365"},{"tid":375,"created_ino":"0x10000000366"},{"tid":376,"created_ino":"0x10000000367"},{"tid":377,"created_ino":"0x10000000368"},{"tid":378,"created_ino":"0x10000000369"},{"tid":379,"created_ino":"0x1000000036a"},{"tid":380,"created_ino":"0x1000000036b"},{"tid":381,"created_ino":"0x1000000036c"},{"tid":382,"created_ino":"0x1000000036d"},{"tid":383,"created_ino":"0x1000000036e"},{"tid":384,"created_ino":"0x1000000036f"},{"tid":385,"created_ino":"0x10000000370"},{"tid":386,"created_ino":"0x10000000371"},{"tid":387,"created_ino":"0x10000000372"},{"tid":388,"created_ino":"0x10000000373"},{"tid":389,"created_ino":"0x10000000374"},{"tid":390,"created_ino":"0x10000000375"},{"tid":391,"created_ino":"0x10000000376"},{"tid":392,"created_ino":"0x10000000377"},{"tid":393,"created_ino":"0x10000000378"},{"tid":394,"created_ino":"0x10000000379"},{"tid":395,"created_ino":"0x1000000037a"},{"tid":396,"created_ino":"0x1000000037b"},{"tid":397,"created_ino":"0x1000000037c"},{"tid":398,"created_ino":"0x1000000037d"},{"tid":399,"created_ino":"0x1000000037e"},{"tid":400,"created_ino":"0x1000000037f"},{"tid":401,"created_ino":"0x10000000380"},{"tid":402,"created_ino":"0x10000000381"},{"tid":403,"created_ino":"0x10000000382"},{"tid":404,"created_ino":"0x10000000383"},{"tid":405,"created_ino":"0x10000000384"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000385","length":602}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"112942","root":"/"}}] 2023-10-06T04:55:53.252 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "99939", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2412354662, "type": "any" }, "name": { "num": 5576, "type": "client" } }, "id": 5576, "inst": "client.5576 192.168.0.1:0/2412354662", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9512587028234565 }, "state": "open", "uptime": 9.220882816 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "112942", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 5 }, { "created_ino": "0x100000001f6", "tid": 6 }, { "created_ino": "0x100000001f7", "tid": 7 }, { "created_ino": "0x100000001f8", "tid": 8 }, { "created_ino": "0x100000001f9", "tid": 9 }, { "created_ino": "0x100000001fa", "tid": 10 }, { "created_ino": "0x100000001fb", "tid": 11 }, { "created_ino": "0x100000001fc", "tid": 12 }, { "created_ino": "0x100000001fd", "tid": 13 }, { "created_ino": "0x100000001fe", "tid": 14 }, { "created_ino": "0x100000001ff", "tid": 15 }, { "created_ino": "0x10000000200", "tid": 16 }, { "created_ino": "0x10000000201", "tid": 17 }, { "created_ino": "0x10000000202", "tid": 18 }, { "created_ino": "0x10000000203", "tid": 19 }, { "created_ino": "0x10000000204", "tid": 20 }, { "created_ino": "0x10000000205", "tid": 21 }, { "created_ino": "0x10000000206", "tid": 22 }, { "created_ino": "0x10000000207", "tid": 23 }, { "created_ino": "0x10000000208", "tid": 24 }, { "created_ino": "0x10000000209", "tid": 25 }, { "created_ino": "0x1000000020a", "tid": 26 }, { "created_ino": "0x1000000020b", "tid": 27 }, { "created_ino": "0x1000000020c", "tid": 28 }, { "created_ino": "0x1000000020d", "tid": 29 }, { "created_ino": "0x1000000020e", "tid": 30 }, { "created_ino": "0x1000000020f", "tid": 31 }, { "created_ino": "0x10000000210", "tid": 32 }, { "created_ino": "0x10000000211", "tid": 33 }, { "created_ino": "0x10000000212", "tid": 34 }, { "created_ino": "0x10000000213", "tid": 35 }, { "created_ino": "0x10000000214", "tid": 36 }, { "created_ino": "0x10000000215", "tid": 37 }, { "created_ino": "0x10000000216", "tid": 38 }, { "created_ino": "0x10000000217", "tid": 39 }, { "created_ino": "0x10000000218", "tid": 40 }, { "created_ino": "0x10000000219", "tid": 41 }, { "created_ino": "0x1000000021a", "tid": 42 }, { "created_ino": "0x1000000021b", "tid": 43 }, { "created_ino": "0x1000000021c", "tid": 44 }, { "created_ino": "0x1000000021d", "tid": 45 }, { "created_ino": "0x1000000021e", "tid": 46 }, { "created_ino": "0x1000000021f", "tid": 47 }, { "created_ino": "0x10000000220", "tid": 48 }, { "created_ino": "0x10000000221", "tid": 49 }, { "created_ino": "0x10000000222", "tid": 50 }, { "created_ino": "0x10000000223", "tid": 51 }, { "created_ino": "0x10000000224", "tid": 52 }, { "created_ino": "0x10000000225", "tid": 53 }, { "created_ino": "0x10000000226", "tid": 54 }, { "created_ino": "0x10000000227", "tid": 55 }, { "created_ino": "0x10000000228", "tid": 56 }, { "created_ino": "0x10000000229", "tid": 57 }, { "created_ino": "0x1000000022a", "tid": 58 }, { "created_ino": "0x1000000022b", "tid": 59 }, { "created_ino": "0x1000000022c", "tid": 60 }, { "created_ino": "0x1000000022d", "tid": 61 }, { "created_ino": "0x1000000022e", "tid": 62 }, { "created_ino": "0x1000000022f", "tid": 63 }, { "created_ino": "0x10000000230", "tid": 64 }, { "created_ino": "0x10000000231", "tid": 65 }, { "created_ino": "0x10000000232", "tid": 66 }, { "created_ino": "0x10000000233", "tid": 67 }, { "created_ino": "0x10000000234", "tid": 68 }, { "created_ino": "0x10000000235", "tid": 69 }, { "created_ino": "0x10000000236", "tid": 70 }, { "created_ino": "0x10000000237", "tid": 71 }, { "created_ino": "0x10000000238", "tid": 72 }, { "created_ino": "0x10000000239", "tid": 73 }, { "created_ino": "0x1000000023a", "tid": 74 }, { "created_ino": "0x1000000023b", "tid": 75 }, { "created_ino": "0x1000000023c", "tid": 76 }, { "created_ino": "0x1000000023d", "tid": 77 }, { "created_ino": "0x1000000023e", "tid": 78 }, { "created_ino": "0x1000000023f", "tid": 79 }, { "created_ino": "0x10000000240", "tid": 80 }, { "created_ino": "0x10000000241", "tid": 81 }, { "created_ino": "0x10000000242", "tid": 82 }, { "created_ino": "0x10000000243", "tid": 83 }, { "created_ino": "0x10000000244", "tid": 84 }, { "created_ino": "0x10000000245", "tid": 85 }, { "created_ino": "0x10000000246", "tid": 86 }, { "created_ino": "0x10000000247", "tid": 87 }, { "created_ino": "0x10000000248", "tid": 88 }, { "created_ino": "0x10000000249", "tid": 89 }, { "created_ino": "0x1000000024a", "tid": 90 }, { "created_ino": "0x1000000024b", "tid": 91 }, { "created_ino": "0x1000000024c", "tid": 92 }, { "created_ino": "0x1000000024d", "tid": 93 }, { "created_ino": "0x1000000024e", "tid": 94 }, { "created_ino": "0x1000000024f", "tid": 95 }, { "created_ino": "0x10000000250", "tid": 96 }, { "created_ino": "0x10000000251", "tid": 97 }, { "created_ino": "0x10000000252", "tid": 98 }, { "created_ino": "0x10000000253", "tid": 99 }, { "created_ino": "0x10000000254", "tid": 100 }, { "created_ino": "0x10000000255", "tid": 101 }, { "created_ino": "0x10000000256", "tid": 102 }, { "created_ino": "0x10000000257", "tid": 103 }, { "created_ino": "0x10000000258", "tid": 104 }, { "created_ino": "0x10000000259", "tid": 105 }, { "created_ino": "0x1000000025a", "tid": 106 }, { "created_ino": "0x1000000025b", "tid": 107 }, { "created_ino": "0x1000000025c", "tid": 108 }, { "created_ino": "0x1000000025d", "tid": 109 }, { "created_ino": "0x1000000025e", "tid": 110 }, { "created_ino": "0x1000000025f", "tid": 111 }, { "created_ino": "0x10000000260", "tid": 112 }, { "created_ino": "0x10000000261", "tid": 113 }, { "created_ino": "0x10000000262", "tid": 114 }, { "created_ino": "0x10000000263", "tid": 115 }, { "created_ino": "0x10000000264", "tid": 116 }, { "created_ino": "0x10000000265", "tid": 117 }, { "created_ino": "0x10000000266", "tid": 118 }, { "created_ino": "0x10000000267", "tid": 119 }, { "created_ino": "0x10000000268", "tid": 120 }, { "created_ino": "0x10000000269", "tid": 121 }, { "created_ino": "0x1000000026a", "tid": 122 }, { "created_ino": "0x1000000026b", "tid": 123 }, { "created_ino": "0x1000000026c", "tid": 124 }, { "created_ino": "0x1000000026d", "tid": 125 }, { "created_ino": "0x1000000026e", "tid": 126 }, { "created_ino": "0x1000000026f", "tid": 127 }, { "created_ino": "0x10000000270", "tid": 128 }, { "created_ino": "0x10000000271", "tid": 129 }, { "created_ino": "0x10000000272", "tid": 130 }, { "created_ino": "0x10000000273", "tid": 131 }, { "created_ino": "0x10000000274", "tid": 132 }, { "created_ino": "0x10000000275", "tid": 133 }, { "created_ino": "0x10000000276", "tid": 134 }, { "created_ino": "0x10000000277", "tid": 135 }, { "created_ino": "0x10000000278", "tid": 136 }, { "created_ino": "0x10000000279", "tid": 137 }, { "created_ino": "0x1000000027a", "tid": 138 }, { "created_ino": "0x1000000027b", "tid": 139 }, { "created_ino": "0x1000000027c", "tid": 140 }, { "created_ino": "0x1000000027d", "tid": 141 }, { "created_ino": "0x1000000027e", "tid": 142 }, { "created_ino": "0x1000000027f", "tid": 143 }, { "created_ino": "0x10000000280", "tid": 144 }, { "created_ino": "0x10000000281", "tid": 145 }, { "created_ino": "0x10000000282", "tid": 146 }, { "created_ino": "0x10000000283", "tid": 147 }, { "created_ino": "0x10000000284", "tid": 148 }, { "created_ino": "0x10000000285", "tid": 149 }, { "created_ino": "0x10000000286", "tid": 150 }, { "created_ino": "0x10000000287", "tid": 151 }, { "created_ino": "0x10000000288", "tid": 152 }, { "created_ino": "0x10000000289", "tid": 153 }, { "created_ino": "0x1000000028a", "tid": 154 }, { "created_ino": "0x1000000028b", "tid": 155 }, { "created_ino": "0x1000000028c", "tid": 156 }, { "created_ino": "0x1000000028d", "tid": 157 }, { "created_ino": "0x1000000028e", "tid": 158 }, { "created_ino": "0x1000000028f", "tid": 159 }, { "created_ino": "0x10000000290", "tid": 160 }, { "created_ino": "0x10000000291", "tid": 161 }, { "created_ino": "0x10000000292", "tid": 162 }, { "created_ino": "0x10000000293", "tid": 163 }, { "created_ino": "0x10000000294", "tid": 164 }, { "created_ino": "0x10000000295", "tid": 165 }, { "created_ino": "0x10000000296", "tid": 166 }, { "created_ino": "0x10000000297", "tid": 167 }, { "created_ino": "0x10000000298", "tid": 168 }, { "created_ino": "0x10000000299", "tid": 169 }, { "created_ino": "0x1000000029a", "tid": 170 }, { "created_ino": "0x1000000029b", "tid": 171 }, { "created_ino": "0x1000000029c", "tid": 172 }, { "created_ino": "0x1000000029d", "tid": 173 }, { "created_ino": "0x1000000029e", "tid": 174 }, { "created_ino": "0x1000000029f", "tid": 175 }, { "created_ino": "0x100000002a0", "tid": 176 }, { "created_ino": "0x100000002a1", "tid": 177 }, { "created_ino": "0x100000002a2", "tid": 178 }, { "created_ino": "0x100000002a3", "tid": 179 }, { "created_ino": "0x100000002a4", "tid": 180 }, { "created_ino": "0x100000002a5", "tid": 181 }, { "created_ino": "0x100000002a6", "tid": 182 }, { "created_ino": "0x100000002a7", "tid": 183 }, { "created_ino": "0x100000002a8", "tid": 184 }, { "created_ino": "0x100000002a9", "tid": 185 }, { "created_ino": "0x100000002aa", "tid": 186 }, { "created_ino": "0x100000002ab", "tid": 187 }, { "created_ino": "0x100000002ac", "tid": 188 }, { "created_ino": "0x100000002ad", "tid": 189 }, { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 }, { "created_ino": "0x100000002bd", "tid": 205 }, { "created_ino": "0x100000002be", "tid": 206 }, { "created_ino": "0x100000002bf", "tid": 207 }, { "created_ino": "0x100000002c0", "tid": 208 }, { "created_ino": "0x100000002c1", "tid": 209 }, { "created_ino": "0x100000002c2", "tid": 210 }, { "created_ino": "0x100000002c3", "tid": 211 }, { "created_ino": "0x100000002c4", "tid": 212 }, { "created_ino": "0x100000002c5", "tid": 213 }, { "created_ino": "0x100000002c6", "tid": 214 }, { "created_ino": "0x100000002c7", "tid": 215 }, { "created_ino": "0x100000002c8", "tid": 216 }, { "created_ino": "0x100000002c9", "tid": 217 }, { "created_ino": "0x100000002ca", "tid": 218 }, { "created_ino": "0x100000002cb", "tid": 219 }, { "created_ino": "0x100000002cc", "tid": 220 }, { "created_ino": "0x100000002cd", "tid": 221 }, { "created_ino": "0x100000002ce", "tid": 222 }, { "created_ino": "0x100000002cf", "tid": 223 }, { "created_ino": "0x100000002d0", "tid": 224 }, { "created_ino": "0x100000002d1", "tid": 225 }, { "created_ino": "0x100000002d2", "tid": 226 }, { "created_ino": "0x100000002d3", "tid": 227 }, { "created_ino": "0x100000002d4", "tid": 228 }, { "created_ino": "0x100000002d5", "tid": 229 }, { "created_ino": "0x100000002d6", "tid": 230 }, { "created_ino": "0x100000002d7", "tid": 231 }, { "created_ino": "0x100000002d8", "tid": 232 }, { "created_ino": "0x100000002d9", "tid": 233 }, { "created_ino": "0x100000002da", "tid": 234 }, { "created_ino": "0x100000002db", "tid": 235 }, { "created_ino": "0x100000002dc", "tid": 236 }, { "created_ino": "0x100000002dd", "tid": 237 }, { "created_ino": "0x100000002de", "tid": 238 }, { "created_ino": "0x100000002df", "tid": 239 }, { "created_ino": "0x100000002e0", "tid": 240 }, { "created_ino": "0x100000002e1", "tid": 241 }, { "created_ino": "0x100000002e2", "tid": 242 }, { "created_ino": "0x100000002e3", "tid": 243 }, { "created_ino": "0x100000002e4", "tid": 244 }, { "created_ino": "0x100000002e5", "tid": 245 }, { "created_ino": "0x100000002e6", "tid": 246 }, { "created_ino": "0x100000002e7", "tid": 247 }, { "created_ino": "0x100000002e8", "tid": 248 }, { "created_ino": "0x100000002e9", "tid": 249 }, { "created_ino": "0x100000002ea", "tid": 250 }, { "created_ino": "0x100000002eb", "tid": 251 }, { "created_ino": "0x100000002ec", "tid": 252 }, { "created_ino": "0x100000002ed", "tid": 253 }, { "created_ino": "0x100000002ee", "tid": 254 }, { "created_ino": "0x100000002ef", "tid": 255 }, { "created_ino": "0x100000002f0", "tid": 256 }, { "created_ino": "0x100000002f1", "tid": 257 }, { "created_ino": "0x100000002f2", "tid": 258 }, { "created_ino": "0x100000002f3", "tid": 259 }, { "created_ino": "0x100000002f4", "tid": 260 }, { "created_ino": "0x100000002f5", "tid": 261 }, { "created_ino": "0x100000002f6", "tid": 262 }, { "created_ino": "0x100000002f7", "tid": 263 }, { "created_ino": "0x100000002f8", "tid": 264 }, { "created_ino": "0x100000002f9", "tid": 265 }, { "created_ino": "0x100000002fa", "tid": 266 }, { "created_ino": "0x100000002fb", "tid": 267 }, { "created_ino": "0x100000002fc", "tid": 268 }, { "created_ino": "0x100000002fd", "tid": 269 }, { "created_ino": "0x100000002fe", "tid": 270 }, { "created_ino": "0x100000002ff", "tid": 271 }, { "created_ino": "0x10000000300", "tid": 272 }, { "created_ino": "0x10000000301", "tid": 273 }, { "created_ino": "0x10000000302", "tid": 274 }, { "created_ino": "0x10000000303", "tid": 275 }, { "created_ino": "0x10000000304", "tid": 276 }, { "created_ino": "0x10000000305", "tid": 277 }, { "created_ino": "0x10000000306", "tid": 278 }, { "created_ino": "0x10000000307", "tid": 279 }, { "created_ino": "0x10000000308", "tid": 280 }, { "created_ino": "0x10000000309", "tid": 281 }, { "created_ino": "0x1000000030a", "tid": 282 }, { "created_ino": "0x1000000030b", "tid": 283 }, { "created_ino": "0x1000000030c", "tid": 284 }, { "created_ino": "0x1000000030d", "tid": 285 }, { "created_ino": "0x1000000030e", "tid": 286 }, { "created_ino": "0x1000000030f", "tid": 287 }, { "created_ino": "0x10000000310", "tid": 288 }, { "created_ino": "0x10000000311", "tid": 289 }, { "created_ino": "0x10000000312", "tid": 290 }, { "created_ino": "0x10000000313", "tid": 291 }, { "created_ino": "0x10000000314", "tid": 292 }, { "created_ino": "0x10000000315", "tid": 293 }, { "created_ino": "0x10000000316", "tid": 294 }, { "created_ino": "0x10000000317", "tid": 295 }, { "created_ino": "0x10000000318", "tid": 296 }, { "created_ino": "0x10000000319", "tid": 297 }, { "created_ino": "0x1000000031a", "tid": 298 }, { "created_ino": "0x1000000031b", "tid": 299 }, { "created_ino": "0x1000000031c", "tid": 300 }, { "created_ino": "0x1000000031d", "tid": 301 }, { "created_ino": "0x1000000031e", "tid": 302 }, { "created_ino": "0x1000000031f", "tid": 303 }, { "created_ino": "0x10000000320", "tid": 304 }, { "created_ino": "0x10000000321", "tid": 305 }, { "created_ino": "0x10000000322", "tid": 306 }, { "created_ino": "0x10000000323", "tid": 307 }, { "created_ino": "0x10000000324", "tid": 308 }, { "created_ino": "0x10000000325", "tid": 309 }, { "created_ino": "0x10000000326", "tid": 310 }, { "created_ino": "0x10000000327", "tid": 311 }, { "created_ino": "0x10000000328", "tid": 312 }, { "created_ino": "0x10000000329", "tid": 313 }, { "created_ino": "0x1000000032a", "tid": 314 }, { "created_ino": "0x1000000032b", "tid": 315 }, { "created_ino": "0x1000000032c", "tid": 316 }, { "created_ino": "0x1000000032d", "tid": 317 }, { "created_ino": "0x1000000032e", "tid": 318 }, { "created_ino": "0x1000000032f", "tid": 319 }, { "created_ino": "0x10000000330", "tid": 320 }, { "created_ino": "0x10000000331", "tid": 321 }, { "created_ino": "0x10000000332", "tid": 322 }, { "created_ino": "0x10000000333", "tid": 323 }, { "created_ino": "0x10000000334", "tid": 324 }, { "created_ino": "0x10000000335", "tid": 325 }, { "created_ino": "0x10000000336", "tid": 326 }, { "created_ino": "0x10000000337", "tid": 327 }, { "created_ino": "0x10000000338", "tid": 328 }, { "created_ino": "0x10000000339", "tid": 329 }, { "created_ino": "0x1000000033a", "tid": 330 }, { "created_ino": "0x1000000033b", "tid": 331 }, { "created_ino": "0x1000000033c", "tid": 332 }, { "created_ino": "0x1000000033d", "tid": 333 }, { "created_ino": "0x1000000033e", "tid": 334 }, { "created_ino": "0x1000000033f", "tid": 335 }, { "created_ino": "0x10000000340", "tid": 336 }, { "created_ino": "0x10000000341", "tid": 337 }, { "created_ino": "0x10000000342", "tid": 338 }, { "created_ino": "0x10000000343", "tid": 339 }, { "created_ino": "0x10000000344", "tid": 340 }, { "created_ino": "0x10000000345", "tid": 341 }, { "created_ino": "0x10000000346", "tid": 342 }, { "created_ino": "0x10000000347", "tid": 343 }, { "created_ino": "0x10000000348", "tid": 344 }, { "created_ino": "0x10000000349", "tid": 345 }, { "created_ino": "0x1000000034a", "tid": 346 }, { "created_ino": "0x1000000034b", "tid": 347 }, { "created_ino": "0x1000000034c", "tid": 348 }, { "created_ino": "0x1000000034d", "tid": 349 }, { "created_ino": "0x1000000034e", "tid": 350 }, { "created_ino": "0x1000000034f", "tid": 351 }, { "created_ino": "0x10000000350", "tid": 352 }, { "created_ino": "0x10000000351", "tid": 353 }, { "created_ino": "0x10000000352", "tid": 354 }, { "created_ino": "0x10000000353", "tid": 355 }, { "created_ino": "0x10000000354", "tid": 356 }, { "created_ino": "0x10000000355", "tid": 357 }, { "created_ino": "0x10000000356", "tid": 358 }, { "created_ino": "0x10000000357", "tid": 360 }, { "created_ino": "0x10000000358", "tid": 361 }, { "created_ino": "0x10000000359", "tid": 362 }, { "created_ino": "0x1000000035a", "tid": 363 }, { "created_ino": "0x1000000035b", "tid": 364 }, { "created_ino": "0x1000000035c", "tid": 365 }, { "created_ino": "0x1000000035d", "tid": 366 }, { "created_ino": "0x1000000035e", "tid": 367 }, { "created_ino": "0x1000000035f", "tid": 368 }, { "created_ino": "0x10000000360", "tid": 369 }, { "created_ino": "0x10000000361", "tid": 370 }, { "created_ino": "0x10000000362", "tid": 371 }, { "created_ino": "0x10000000363", "tid": 372 }, { "created_ino": "0x10000000364", "tid": 373 }, { "created_ino": "0x10000000365", "tid": 374 }, { "created_ino": "0x10000000366", "tid": 375 }, { "created_ino": "0x10000000367", "tid": 376 }, { "created_ino": "0x10000000368", "tid": 377 }, { "created_ino": "0x10000000369", "tid": 378 }, { "created_ino": "0x1000000036a", "tid": 379 }, { "created_ino": "0x1000000036b", "tid": 380 }, { "created_ino": "0x1000000036c", "tid": 381 }, { "created_ino": "0x1000000036d", "tid": 382 }, { "created_ino": "0x1000000036e", "tid": 383 }, { "created_ino": "0x1000000036f", "tid": 384 }, { "created_ino": "0x10000000370", "tid": 385 }, { "created_ino": "0x10000000371", "tid": 386 }, { "created_ino": "0x10000000372", "tid": 387 }, { "created_ino": "0x10000000373", "tid": 388 }, { "created_ino": "0x10000000374", "tid": 389 }, { "created_ino": "0x10000000375", "tid": 390 }, { "created_ino": "0x10000000376", "tid": 391 }, { "created_ino": "0x10000000377", "tid": 392 }, { "created_ino": "0x10000000378", "tid": 393 }, { "created_ino": "0x10000000379", "tid": 394 }, { "created_ino": "0x1000000037a", "tid": 395 }, { "created_ino": "0x1000000037b", "tid": 396 }, { "created_ino": "0x1000000037c", "tid": 397 }, { "created_ino": "0x1000000037d", "tid": 398 }, { "created_ino": "0x1000000037e", "tid": 399 }, { "created_ino": "0x1000000037f", "tid": 400 }, { "created_ino": "0x10000000380", "tid": 401 }, { "created_ino": "0x10000000381", "tid": 402 }, { "created_ino": "0x10000000382", "tid": 403 }, { "created_ino": "0x10000000383", "tid": 404 }, { "created_ino": "0x10000000384", "tid": 405 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1297719750, "type": "any" }, "name": { "num": 6303, "type": "client" } }, "id": 6303, "inst": "client.6303 192.168.0.1:0/1297719750", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 400, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 602, "start": "0x10000000385" } ], "recall_caps": { "halflife": 60, "value": 1158.0916422372288 }, "recall_caps_throttle": { "halflife": 1.5, "value": 385.175781351621 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 117.16786687666476 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 378, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 795.2862832279143 }, "state": "open", "uptime": 11.018859935 } ] 2023-10-06T04:55:53.258 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-10-06T04:55:53.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:55:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.373+0000 7f9e6ddd3700 1 Processor -- start 2023-10-06T04:55:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.373+0000 7f9e6ddd3700 1 -- start start 2023-10-06T04:55:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e6ddd3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9e681551f0 0x7f9e68069390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e6ddd3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e681587e0 0x7f9e680698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e6ddd3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680b3e90 con 0x7f9e68154e20 2023-10-06T04:55:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e6ddd3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680b40d0 con 0x7f9e681587e0 2023-10-06T04:55:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e6ddd3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680b1180 con 0x7f9e681551f0 2023-10-06T04:55:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e677fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9e68154e20 0x7f9e680b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37830/0 (socket says 172.21.15.3:37830) 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e677fe700 1 -- 172.21.15.3:0/2590853771 learned_addr learned my addr 172.21.15.3:0/2590853771 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.374+0000 7f9e67fff700 1 --2- 172.21.15.3:0/2590853771 >> v2:172.21.15.3:3300/0 conn(0x7f9e681587e0 0x7f9e680698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e66ffd700 1 --2- 172.21.15.3:0/2590853771 >> v2:172.21.15.153:3300/0 conn(0x7f9e681551f0 0x7f9e68069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e667fc700 1 -- 172.21.15.3:0/2590853771 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072247514 0 0) 0x7f9e58001110 con 0x7f9e68154e20 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e667fc700 1 -- 172.21.15.3:0/2590853771 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e50003570 con 0x7f9e68154e20 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e67fff700 1 -- 172.21.15.3:0/2590853771 >> v2:172.21.15.153:3300/0 conn(0x7f9e681551f0 msgr2=0x7f9e68069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e67fff700 1 --2- 172.21.15.3:0/2590853771 >> v2:172.21.15.153:3300/0 conn(0x7f9e681551f0 0x7f9e68069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e67fff700 1 -- 172.21.15.3:0/2590853771 >> v1:172.21.15.3:6789/0 conn(0x7f9e68154e20 legacy=0x7f9e680b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e67fff700 1 -- 172.21.15.3:0/2590853771 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680b1460 con 0x7f9e681587e0 2023-10-06T04:55:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e66ffd700 1 --2- 172.21.15.3:0/2590853771 >> v2:172.21.15.153:3300/0 conn(0x7f9e681551f0 0x7f9e68069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e67fff700 1 --2- 172.21.15.3:0/2590853771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e680698d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9e6806b690 tx=0x7f9e60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343185056d62fced server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e667fc700 1 -- 172.21.15.3:0/2590853771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000e070 con 0x7f9e681587e0 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e667fc700 1 -- 172.21.15.3:0/2590853771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6000bb60 con 0x7f9e681587e0 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.375+0000 7f9e667fc700 1 -- 172.21.15.3:0/2590853771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60012430 con 0x7f9e681587e0 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/2590853771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 msgr2=0x7f9e680698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/2590853771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e680698d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9e6806b690 tx=0x7f9e60009780 comp rx=0 tx=0).stop 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/2590853771 shutdown_connections 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/2590853771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e680698d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/2590853771 >> v2:172.21.15.153:3300/0 conn(0x7f9e681551f0 0x7f9e68069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/2590853771 >> 172.21.15.3:0/2590853771 conn(0x7f9e68067aa0 msgr2=0x7f9e68067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/2590853771 shutdown_connections 2023-10-06T04:55:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.376+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/2590853771 wait complete. 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.377+0000 7f9e6ddd3700 1 Processor -- start 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.377+0000 7f9e6ddd3700 1 -- start start 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e6ddd3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e68154e20 0x7f9e6816ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e6ddd3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e681551f0 0x7f9e6816b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e6ddd3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e6814c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e6ddd3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680c7d90 con 0x7f9e681587e0 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e6ddd3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6816acf0 con 0x7f9e68154e20 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e6ddd3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680c7460 con 0x7f9e681551f0 2023-10-06T04:55:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e6814c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e6814c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41366/0 (socket says 172.21.15.3:41366) 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 -- 172.21.15.3:0/230714011 learned_addr learned my addr 172.21.15.3:0/230714011 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e677fe700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e68154e20 0x7f9e6816ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e66ffd700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e681551f0 0x7f9e6816b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 -- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e681551f0 msgr2=0x7f9e6816b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e681551f0 0x7f9e6816b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 -- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e68154e20 msgr2=0x7f9e6816ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e68154e20 0x7f9e6816ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 -- 172.21.15.3:0/230714011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e60009a20 con 0x7f9e681587e0 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e677fe700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e68154e20 0x7f9e6816ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.378+0000 7f9e67fff700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e6814c8b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9e600097c0 tx=0x7f9e60019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.379+0000 7f9e4f7fe700 1 -- 172.21.15.3:0/230714011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60012e00 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.379+0000 7f9e4f7fe700 1 -- 172.21.15.3:0/230714011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6001d630 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.379+0000 7f9e4f7fe700 1 -- 172.21.15.3:0/230714011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6001d9f0 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.379+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e680c7740 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.379+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6814cdf0 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.380+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e680c7f40 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.380+0000 7f9e4f7fe700 1 -- 172.21.15.3:0/230714011 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e60024070 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.380+0000 7f9e4f7fe700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9e54066f30 0x7f9e540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.380+0000 7f9e4f7fe700 1 -- 172.21.15.3:0/230714011 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9e60086dc0 con 0x7f9e681587e0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.381+0000 7f9e677fe700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9e54066f30 0x7f9e540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.381+0000 7f9e677fe700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9e54066f30 0x7f9e540693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9e6806f660 tx=0x7f9e58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.385+0000 7f9e4f7fe700 1 -- 172.21.15.3:0/230714011 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e680c7f40 con 0x7f9e681587e0 2023-10-06T04:55:53.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.625+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9e680644c0 con 0x7f9e681587e0 2023-10-06T04:55:53.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.626+0000 7f9e4f7fe700 1 -- 172.21.15.3:0/230714011 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f9e6002c0c0 con 0x7f9e681587e0 2023-10-06T04:55:53.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:53.627 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:55:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9e54066f30 msgr2=0x7f9e540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9e54066f30 0x7f9e540693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9e6806f660 tx=0x7f9e58009040 comp rx=0 tx=0).stop 2023-10-06T04:55:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 msgr2=0x7f9e6814c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e6814c8b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9e600097c0 tx=0x7f9e60019000 comp rx=0 tx=0).stop 2023-10-06T04:55:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 shutdown_connections 2023-10-06T04:55:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9e54066f30 0x7f9e540693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e68154e20 0x7f9e6816ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e681587e0 0x7f9e6814c8b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 --2- 172.21.15.3:0/230714011 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e681551f0 0x7f9e6816b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 >> 172.21.15.3:0/230714011 conn(0x7f9e68067aa0 msgr2=0x7f9e680b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:53.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 shutdown_connections 2023-10-06T04:55:53.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.628+0000 7f9e6ddd3700 1 -- 172.21.15.3:0/230714011 wait complete. 2023-10-06T04:55:53.641 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T04:55:53.641 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-10-06T04:55:53.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:55:53.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.758+0000 7fa42e6ce700 1 Processor -- start 2023-10-06T04:55:53.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.758+0000 7fa42e6ce700 1 -- start start 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.758+0000 7fa42e6ce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa42815cb80 0x7fa42815dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.758+0000 7fa42e6ce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4280afed0 0x7fa4280cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.758+0000 7fa42e6ce700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4280cf2a0 con 0x7fa42815c7b0 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.758+0000 7fa42e6ce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280cc7d0 con 0x7fa42815cb80 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.758+0000 7fa42e6ce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280ce970 con 0x7fa4280afed0 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa42815cb80 0x7fa42815dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa42815cb80 0x7fa42815dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41376/0 (socket says 172.21.15.3:41376) 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 -- 172.21.15.3:0/357684109 learned_addr learned my addr 172.21.15.3:0/357684109 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa42cc6b700 1 --2- 172.21.15.3:0/357684109 >> v2:172.21.15.153:3300/0 conn(0x7fa4280afed0 0x7fa4280cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 -- 172.21.15.3:0/357684109 >> v2:172.21.15.153:3300/0 conn(0x7fa4280afed0 msgr2=0x7fa4280cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 --2- 172.21.15.3:0/357684109 >> v2:172.21.15.153:3300/0 conn(0x7fa4280afed0 0x7fa4280cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 -- 172.21.15.3:0/357684109 >> v1:172.21.15.153:6789/0 conn(0x7fa42815c7b0 legacy=0x7fa4280cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 -- 172.21.15.3:0/357684109 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280cec50 con 0x7fa42815cb80 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa4277fe700 1 --2- 172.21.15.3:0/357684109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa42815dd00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa42806fdd0 tx=0x7fa41c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6524eccd236844 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa426ffd700 1 -- 172.21.15.3:0/357684109 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00b8f0 con 0x7fa42815cb80 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa426ffd700 1 -- 172.21.15.3:0/357684109 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c00ba50 con 0x7fa42815cb80 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.759+0000 7fa426ffd700 1 -- 172.21.15.3:0/357684109 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00bdf0 con 0x7fa42815cb80 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 -- 172.21.15.3:0/357684109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 msgr2=0x7fa42815dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/357684109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa42815dd00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa42806fdd0 tx=0x7fa41c009250 comp rx=0 tx=0).stop 2023-10-06T04:55:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 -- 172.21.15.3:0/357684109 shutdown_connections 2023-10-06T04:55:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/357684109 >> v2:172.21.15.153:3300/0 conn(0x7fa4280afed0 0x7fa4280cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/357684109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa42815dd00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 -- 172.21.15.3:0/357684109 >> 172.21.15.3:0/357684109 conn(0x7fa428070310 msgr2=0x7fa428070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 -- 172.21.15.3:0/357684109 shutdown_connections 2023-10-06T04:55:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.760+0000 7fa42e6ce700 1 -- 172.21.15.3:0/357684109 wait complete. 2023-10-06T04:55:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.761+0000 7fa42e6ce700 1 Processor -- start 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.761+0000 7fa42e6ce700 1 -- start start 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42e6ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4280afed0 0x7fa4280aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42e6ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa42815c7b0 0x7fa4280c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42e6ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa4280c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42e6ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281591b0 con 0x7fa42815cb80 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42e6ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280c9cb0 con 0x7fa42815c7b0 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42e6ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa428147ac0 con 0x7fa4280afed0 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa4280c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa4280c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41388/0 (socket says 172.21.15.3:41388) 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 -- 172.21.15.3:0/4013507713 learned_addr learned my addr 172.21.15.3:0/4013507713 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa4277fe700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa42815c7b0 0x7fa4280c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 -- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4280afed0 msgr2=0x7fa4280aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4280afed0 0x7fa4280aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 -- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa42815c7b0 msgr2=0x7fa4280c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa42815c7b0 0x7fa4280c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 -- 172.21.15.3:0/4013507713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0094f0 con 0x7fa42815cb80 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa42cc6b700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa4280c7f10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa420009fd0 tx=0x7fa420009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa40ffff700 1 -- 172.21.15.3:0/4013507713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42000ed50 con 0x7fa42815cb80 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.762+0000 7fa40ffff700 1 -- 172.21.15.3:0/4013507713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42000eeb0 con 0x7fa42815cb80 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.763+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4280c92e0 con 0x7fa42815cb80 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.763+0000 7fa40ffff700 1 -- 172.21.15.3:0/4013507713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420014650 con 0x7fa42815cb80 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.763+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4280c86d0 con 0x7fa42815cb80 2023-10-06T04:55:53.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.764+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4280c95c0 con 0x7fa42815cb80 2023-10-06T04:55:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.764+0000 7fa40ffff700 1 -- 172.21.15.3:0/4013507713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4200149c0 con 0x7fa42815cb80 2023-10-06T04:55:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.764+0000 7fa40ffff700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa414066f30 0x7fa4140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.764+0000 7fa40ffff700 1 -- 172.21.15.3:0/4013507713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa420084e90 con 0x7fa42815cb80 2023-10-06T04:55:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.764+0000 7fa427fff700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa414066f30 0x7fa4140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.765+0000 7fa427fff700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa414066f30 0x7fa4140693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa418001730 tx=0x7fa418007b40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:53.768+0000 7fa40ffff700 1 -- 172.21.15.3:0/4013507713 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa420051fa0 con 0x7fa42815cb80 2023-10-06T04:55:54.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.004+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa428071680 con 0x7fa42815cb80 2023-10-06T04:55:54.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.005+0000 7fa40ffff700 1 -- 172.21.15.3:0/4013507713 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fa4200555c0 con 0x7fa42815cb80 2023-10-06T04:55:54.006 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:55:54.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa414066f30 msgr2=0x7fa4140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa414066f30 0x7fa4140693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa418001730 tx=0x7fa418007b40 comp rx=0 tx=0).stop 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 msgr2=0x7fa4280c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa4280c7f10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa420009fd0 tx=0x7fa420009900 comp rx=0 tx=0).stop 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 shutdown_connections 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa414066f30 0x7fa4140693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42815cb80 0x7fa4280c7f10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4280afed0 0x7fa4280aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 --2- 172.21.15.3:0/4013507713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa42815c7b0 0x7fa4280c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 >> 172.21.15.3:0/4013507713 conn(0x7fa428070310 msgr2=0x7fa428159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 shutdown_connections 2023-10-06T04:55:54.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:54.007+0000 7fa42e6ce700 1 -- 172.21.15.3:0/4013507713 wait complete. 2023-10-06T04:55:54.020 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:55:54.020 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:55:54.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:54.931 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:59.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:55:59.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.152+0000 7faf43beb700 1 Processor -- start 2023-10-06T04:55:59.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.152+0000 7faf43beb700 1 -- start start 2023-10-06T04:55:59.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf43beb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faf3c140c50 0x7faf3c0b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:59.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf43beb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:59.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf43beb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0cc050 con 0x7faf3c0aed80 2023-10-06T04:55:59.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf43beb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0cb1c0 con 0x7faf3c140c50 2023-10-06T04:55:59.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf43beb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0cb440 con 0x7faf3c0b2790 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf41987700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faf3c140c50 0x7faf3c0b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf41186700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf42188700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7faf3c0aed80 0x7faf3c0cc4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34094/0 (socket says 172.21.15.3:34094) 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf41186700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c0b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:42628/0 (socket says 172.21.15.3:42628) 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf42188700 1 -- 172.21.15.3:0/1849769528 learned_addr learned my addr 172.21.15.3:0/1849769528 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.153+0000 7faf40985700 1 -- 172.21.15.3:0/1849769528 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264960395 0 0) 0x7faf3c0cc050 con 0x7faf3c0aed80 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf40985700 1 -- 172.21.15.3:0/1849769528 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003570 con 0x7faf3c0aed80 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf41987700 1 -- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 msgr2=0x7faf3c0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:55:59.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf41987700 1 --2- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c0b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf41987700 1 -- 172.21.15.3:0/1849769528 >> v1:172.21.15.153:6789/0 conn(0x7faf3c0aed80 legacy=0x7faf3c0cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf41987700 1 -- 172.21.15.3:0/1849769528 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0cb720 con 0x7faf3c140c50 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf41186700 1 --2- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c0b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf41987700 1 --2- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 0x7faf3c0b0650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf340098e0 tx=0x7faf340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5164f93ab2ba547f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf40985700 1 -- 172.21.15.3:0/1849769528 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34011630 con 0x7faf3c140c50 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf40985700 1 -- 172.21.15.3:0/1849769528 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf34011790 con 0x7faf3c140c50 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.154+0000 7faf40985700 1 -- 172.21.15.3:0/1849769528 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34011b30 con 0x7faf3c140c50 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 -- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 msgr2=0x7faf3c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 --2- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 0x7faf3c0b0650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf340098e0 tx=0x7faf340091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 -- 172.21.15.3:0/1849769528 shutdown_connections 2023-10-06T04:55:59.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 --2- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 0x7faf3c0b0650 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 --2- 172.21.15.3:0/1849769528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c0b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 -- 172.21.15.3:0/1849769528 >> 172.21.15.3:0/1849769528 conn(0x7faf3c0cd170 msgr2=0x7faf3c0cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:59.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 -- 172.21.15.3:0/1849769528 shutdown_connections 2023-10-06T04:55:59.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.155+0000 7faf43beb700 1 -- 172.21.15.3:0/1849769528 wait complete. 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.156+0000 7faf43beb700 1 Processor -- start 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.156+0000 7faf43beb700 1 -- start start 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf43beb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0aed80 0x7faf3c13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf43beb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c13be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf43beb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 0x7faf3c0add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf43beb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c13f1a0 con 0x7faf3c0aed80 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf43beb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0aafc0 con 0x7faf3c140c50 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf43beb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c13e590 con 0x7faf3c0b2790 2023-10-06T04:55:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0aed80 0x7faf3c13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0aed80 0x7faf3c13fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41400/0 (socket says 172.21.15.3:41400) 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 -- 172.21.15.3:0/3607308451 learned_addr learned my addr 172.21.15.3:0/3607308451 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf42188700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 0x7faf3c0add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41186700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c13be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 -- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 msgr2=0x7faf3c13be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c13be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 -- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 msgr2=0x7faf3c0add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:59.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 0x7faf3c0add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 -- 172.21.15.3:0/3607308451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009420 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.157+0000 7faf41987700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0aed80 0x7faf3c13fa20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0af9b0 tx=0x7faf34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.158+0000 7faf39ffb700 1 -- 172.21.15.3:0/3607308451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34000d10 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.158+0000 7faf39ffb700 1 -- 172.21.15.3:0/3607308451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf34011b10 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.158+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf34009420 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.158+0000 7faf39ffb700 1 -- 172.21.15.3:0/3607308451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34025660 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.158+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c13ee30 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.159+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c0647b0 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.159+0000 7faf39ffb700 1 -- 172.21.15.3:0/3607308451 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf34025a50 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.160+0000 7faf39ffb700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf28066f30 0x7faf280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.160+0000 7faf39ffb700 1 -- 172.21.15.3:0/3607308451 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faf34085d90 con 0x7faf3c0aed80 2023-10-06T04:55:59.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.160+0000 7faf41186700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf28066f30 0x7faf280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:55:59.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.161+0000 7faf41186700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf28066f30 0x7faf280693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0c9e30 tx=0x7faf300091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:55:59.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.164+0000 7faf39ffb700 1 -- 172.21.15.3:0/3607308451 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf34052ea0 con 0x7faf3c0aed80 2023-10-06T04:55:59.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.404+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faf3c0afa60 con 0x7faf3c0aed80 2023-10-06T04:55:59.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.404+0000 7faf39ffb700 1 -- 172.21.15.3:0/3607308451 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7faf340564c0 con 0x7faf3c0aed80 2023-10-06T04:55:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:55:59.406 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf28066f30 msgr2=0x7faf280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf28066f30 0x7faf280693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0c9e30 tx=0x7faf300091a0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0aed80 msgr2=0x7faf3c13fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0aed80 0x7faf3c13fa20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0af9b0 tx=0x7faf34017000 comp rx=0 tx=0).stop 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 shutdown_connections 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf28066f30 0x7faf280693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c140c50 0x7faf3c0add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0aed80 0x7faf3c13fa20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 --2- 172.21.15.3:0/3607308451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf3c0b2790 0x7faf3c13be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:55:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 >> 172.21.15.3:0/3607308451 conn(0x7faf3c0cd170 msgr2=0x7faf3c067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:55:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 shutdown_connections 2023-10-06T04:55:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:55:59.406+0000 7faf43beb700 1 -- 172.21.15.3:0/3607308451 wait complete. 2023-10-06T04:55:59.419 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:55:59.420 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:04.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:04.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.550+0000 7f722bbab700 1 Processor -- start 2023-10-06T04:56:04.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.551+0000 7f722bbab700 1 -- start start 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.551+0000 7f722bbab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f722414c860 0x7f72240afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.551+0000 7f722bbab700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7224159d30 0x7f72240b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f722bbab700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7224157130 con 0x7f722414cc30 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f722bbab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72241573f0 con 0x7f722414c860 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f722bbab700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72241576d0 con 0x7f7224159d30 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229947700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f722414c860 0x7f72240afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229146700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f722414cc30 0x7f7224159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34610/0 (socket says 172.21.15.3:34610) 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229146700 1 -- 172.21.15.3:0/2170918826 learned_addr learned my addr 172.21.15.3:0/2170918826 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f722a148700 1 --2- 172.21.15.3:0/2170918826 >> v2:172.21.15.153:3300/0 conn(0x7f7224159d30 0x7f72240b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229947700 1 -- 172.21.15.3:0/2170918826 >> v2:172.21.15.153:3300/0 conn(0x7f7224159d30 msgr2=0x7f72240b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229947700 1 --2- 172.21.15.3:0/2170918826 >> v2:172.21.15.153:3300/0 conn(0x7f7224159d30 0x7f72240b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229947700 1 -- 172.21.15.3:0/2170918826 >> v1:172.21.15.3:6789/0 conn(0x7f722414cc30 legacy=0x7f7224159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229947700 1 -- 172.21.15.3:0/2170918826 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72241579b0 con 0x7f722414c860 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7228945700 1 -- 172.21.15.3:0/2170918826 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065632730 0 0) 0x7f7224157130 con 0x7f722414cc30 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f722a148700 1 --2- 172.21.15.3:0/2170918826 >> v2:172.21.15.153:3300/0 conn(0x7f7224159d30 0x7f72240b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7229947700 1 --2- 172.21.15.3:0/2170918826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414c860 0x7f72240afe70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f722406bd30 tx=0x7f721c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa157fc7aa9cc1d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7228945700 1 -- 172.21.15.3:0/2170918826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c00b6b0 con 0x7f722414c860 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7228945700 1 -- 172.21.15.3:0/2170918826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c00b810 con 0x7f722414c860 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.552+0000 7f7228945700 1 -- 172.21.15.3:0/2170918826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c00bbb0 con 0x7f722414c860 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 -- 172.21.15.3:0/2170918826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414c860 msgr2=0x7f72240afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 --2- 172.21.15.3:0/2170918826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414c860 0x7f72240afe70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f722406bd30 tx=0x7f721c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 -- 172.21.15.3:0/2170918826 shutdown_connections 2023-10-06T04:56:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 --2- 172.21.15.3:0/2170918826 >> v2:172.21.15.153:3300/0 conn(0x7f7224159d30 0x7f72240b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 --2- 172.21.15.3:0/2170918826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414c860 0x7f72240afe70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 -- 172.21.15.3:0/2170918826 >> 172.21.15.3:0/2170918826 conn(0x7f722406c370 msgr2=0x7f722406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 -- 172.21.15.3:0/2170918826 shutdown_connections 2023-10-06T04:56:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.553+0000 7f722bbab700 1 -- 172.21.15.3:0/2170918826 wait complete. 2023-10-06T04:56:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.554+0000 7f722bbab700 1 Processor -- start 2023-10-06T04:56:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.554+0000 7f722bbab700 1 -- start start 2023-10-06T04:56:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f722bbab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722414c860 0x7f7224142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f722bbab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414cc30 0x7f72240c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f722bbab700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7224159d30 0x7f72240ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f722bbab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72241434c0 con 0x7f722414cc30 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f722bbab700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72240cb310 con 0x7f722414c860 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f722bbab700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72240ca700 con 0x7f7224159d30 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414cc30 0x7f72240c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414cc30 0x7f72240c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53042/0 (socket says 172.21.15.3:53042) 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 -- 172.21.15.3:0/3580574087 learned_addr learned my addr 172.21.15.3:0/3580574087 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229947700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722414c860 0x7f7224142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f722a148700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7224159d30 0x7f72240ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 -- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7224159d30 msgr2=0x7f72240ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7224159d30 0x7f72240ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 -- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722414c860 msgr2=0x7f7224142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722414c860 0x7f7224142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.555+0000 7f7229146700 1 -- 172.21.15.3:0/3580574087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009420 con 0x7f722414cc30 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.556+0000 7f7229146700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414cc30 0x7f72240c9b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f721800c7e0 tx=0x7f7218007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.556+0000 7f7221ffb700 1 -- 172.21.15.3:0/3580574087 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218005c00 con 0x7f722414cc30 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.556+0000 7f7221ffb700 1 -- 172.21.15.3:0/3580574087 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218005d60 con 0x7f722414cc30 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.556+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72240ca9e0 con 0x7f722414cc30 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.556+0000 7f7221ffb700 1 -- 172.21.15.3:0/3580574087 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180072e0 con 0x7f722414cc30 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.556+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72240cafa0 con 0x7f722414cc30 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.557+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7224071680 con 0x7f722414cc30 2023-10-06T04:56:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.557+0000 7f7221ffb700 1 -- 172.21.15.3:0/3580574087 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f721801b3b0 con 0x7f722414cc30 2023-10-06T04:56:04.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.558+0000 7f7221ffb700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7210066f30 0x7f72100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:04.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.558+0000 7f7221ffb700 1 -- 172.21.15.3:0/3580574087 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7218083a80 con 0x7f722414cc30 2023-10-06T04:56:04.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.558+0000 7f7229947700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7210066f30 0x7f72100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:04.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.558+0000 7f7229947700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7210066f30 0x7f72100693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f721c0091e0 tx=0x7f721c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:04.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.562+0000 7f7221ffb700 1 -- 172.21.15.3:0/3580574087 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f721804b5a0 con 0x7f722414cc30 2023-10-06T04:56:04.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.804+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f722406f2a0 con 0x7f722414cc30 2023-10-06T04:56:04.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.804+0000 7f7221ffb700 1 -- 172.21.15.3:0/3580574087 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f7218017080 con 0x7f722414cc30 2023-10-06T04:56:04.806 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:04.806 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.806+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7210066f30 msgr2=0x7f72100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.806+0000 7f722bbab700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7210066f30 0x7f72100693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f721c0091e0 tx=0x7f721c017000 comp rx=0 tx=0).stop 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414cc30 msgr2=0x7f72240c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414cc30 0x7f72240c9b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f721800c7e0 tx=0x7f7218007d90 comp rx=0 tx=0).stop 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 shutdown_connections 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7210066f30 0x7f72100693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722414c860 0x7f7224142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7224159d30 0x7f72240ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 --2- 172.21.15.3:0/3580574087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f722414cc30 0x7f72240c9b40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 >> 172.21.15.3:0/3580574087 conn(0x7f722406c370 msgr2=0x7f722406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 shutdown_connections 2023-10-06T04:56:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:04.807+0000 7f722bbab700 1 -- 172.21.15.3:0/3580574087 wait complete. 2023-10-06T04:56:04.820 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:04.820 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:09.821 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:09.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.950+0000 7f82b7e9e700 1 Processor -- start 2023-10-06T04:56:09.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.951+0000 7f82b7e9e700 1 -- start start 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.951+0000 7f82b7e9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00c8520 0x7f82b01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.951+0000 7f82b7e9e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82b00acb90 0x7f82b006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.951+0000 7f82b7e9e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b006ef00 con 0x7f82b006eab0 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.951+0000 7f82b7e9e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b0157800 con 0x7f82b00acb90 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.951+0000 7f82b7e9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00ad070 con 0x7f82b00c8520 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82b00acb90 0x7f82b006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82b00acb90 0x7f82b006e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53050/0 (socket says 172.21.15.3:53050) 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 -- 172.21.15.3:0/3178025088 learned_addr learned my addr 172.21.15.3:0/3178025088 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5c3a700 1 --2- 172.21.15.3:0/3178025088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00c8520 0x7f82b01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 -- 172.21.15.3:0/3178025088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00c8520 msgr2=0x7f82b01572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 --2- 172.21.15.3:0/3178025088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00c8520 0x7f82b01572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 -- 172.21.15.3:0/3178025088 >> v1:172.21.15.153:6789/0 conn(0x7f82b006eab0 legacy=0x7f82b0159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 -- 172.21.15.3:0/3178025088 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00ad350 con 0x7f82b00acb90 2023-10-06T04:56:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b5439700 1 --2- 172.21.15.3:0/3178025088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b00acb90 0x7f82b006e570 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f82a40099b0 tx=0x7f82a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d505f56fa3243f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b4c38700 1 -- 172.21.15.3:0/3178025088 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400b860 con 0x7f82b00acb90 2023-10-06T04:56:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b4c38700 1 -- 172.21.15.3:0/3178025088 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a400b9c0 con 0x7f82b00acb90 2023-10-06T04:56:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.952+0000 7f82b4c38700 1 -- 172.21.15.3:0/3178025088 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400bd60 con 0x7f82b00acb90 2023-10-06T04:56:09.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 -- 172.21.15.3:0/3178025088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b00acb90 msgr2=0x7f82b006e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/3178025088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b00acb90 0x7f82b006e570 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f82a40099b0 tx=0x7f82a4009250 comp rx=0 tx=0).stop 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 -- 172.21.15.3:0/3178025088 shutdown_connections 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/3178025088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00c8520 0x7f82b01572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/3178025088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b00acb90 0x7f82b006e570 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 -- 172.21.15.3:0/3178025088 >> 172.21.15.3:0/3178025088 conn(0x7f82b006c370 msgr2=0x7f82b006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 -- 172.21.15.3:0/3178025088 shutdown_connections 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.953+0000 7f82b7e9e700 1 -- 172.21.15.3:0/3178025088 wait complete. 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.954+0000 7f82b7e9e700 1 Processor -- start 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.954+0000 7f82b7e9e700 1 -- start start 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b7e9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b006eab0 0x7f82b0155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b7e9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00acb90 0x7f82b00a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b7e9e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b00c8520 0x7f82b01541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b7e9e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0155830 con 0x7f82b006eab0 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b7e9e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0156200 con 0x7f82b00c8520 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b7e9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0154c20 con 0x7f82b00acb90 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b006eab0 0x7f82b0155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b006eab0 0x7f82b0155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53066/0 (socket says 172.21.15.3:53066) 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 -- 172.21.15.3:0/537902316 learned_addr learned my addr 172.21.15.3:0/537902316 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5439700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00acb90 0x7f82b00a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b643b700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b00c8520 0x7f82b01541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 -- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00acb90 msgr2=0x7f82b00a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00acb90 0x7f82b00a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 -- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b00c8520 msgr2=0x7f82b01541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b00c8520 0x7f82b01541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 -- 172.21.15.3:0/537902316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a40094f0 con 0x7f82b006eab0 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.955+0000 7f82b5c3a700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b006eab0 0x7f82b0155cc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f82a800d860 tx=0x7f82a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.956+0000 7f82adffb700 1 -- 172.21.15.3:0/537902316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8009610 con 0x7f82b006eab0 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.956+0000 7f82adffb700 1 -- 172.21.15.3:0/537902316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a8009770 con 0x7f82b006eab0 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.956+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b0154f00 con 0x7f82b006eab0 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.956+0000 7f82adffb700 1 -- 172.21.15.3:0/537902316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8009b30 con 0x7f82b006eab0 2023-10-06T04:56:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.956+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01554c0 con 0x7f82b006eab0 2023-10-06T04:56:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.957+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b006d060 con 0x7f82b006eab0 2023-10-06T04:56:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.957+0000 7f82adffb700 1 -- 172.21.15.3:0/537902316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82a801e460 con 0x7f82b006eab0 2023-10-06T04:56:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.957+0000 7f82adffb700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f829c066f30 0x7f829c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.957+0000 7f82adffb700 1 -- 172.21.15.3:0/537902316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f82a8084cf0 con 0x7f82b006eab0 2023-10-06T04:56:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.958+0000 7f82b5439700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f829c066f30 0x7f829c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.958+0000 7f82b5439700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f829c066f30 0x7f829c0693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f82a40099b0 tx=0x7f82a4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:09.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:09.962+0000 7f82adffb700 1 -- 172.21.15.3:0/537902316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82a8089070 con 0x7f82b006eab0 2023-10-06T04:56:10.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.196+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f82b0071680 con 0x7f82b006eab0 2023-10-06T04:56:10.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.196+0000 7f82adffb700 1 -- 172.21.15.3:0/537902316 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f82a8051e00 con 0x7f82b006eab0 2023-10-06T04:56:10.198 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:10.198 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:10.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f829c066f30 msgr2=0x7f829c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:10.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f829c066f30 0x7f829c0693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f82a40099b0 tx=0x7f82a4017000 comp rx=0 tx=0).stop 2023-10-06T04:56:10.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b006eab0 msgr2=0x7f82b0155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:10.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b006eab0 0x7f82b0155cc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f82a800d860 tx=0x7f82a8008040 comp rx=0 tx=0).stop 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 shutdown_connections 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f829c066f30 0x7f829c0693e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b00c8520 0x7f82b01541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b006eab0 0x7f82b0155cc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 --2- 172.21.15.3:0/537902316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82b00acb90 0x7f82b00a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 >> 172.21.15.3:0/537902316 conn(0x7f82b006c370 msgr2=0x7f82b00b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.198+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 shutdown_connections 2023-10-06T04:56:10.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:10.199+0000 7f82b7e9e700 1 -- 172.21.15.3:0/537902316 wait complete. 2023-10-06T04:56:10.212 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:10.212 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:15.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.344+0000 7f3d18dec700 1 Processor -- start 2023-10-06T04:56:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.344+0000 7f3d18dec700 1 -- start start 2023-10-06T04:56:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.345+0000 7f3d18dec700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d1415a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.345+0000 7f3d18dec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d1415d240 0x7f3d140ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.345+0000 7f3d18dec700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d140cd5f0 con 0x7f3d140cd150 2023-10-06T04:56:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.345+0000 7f3d18dec700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1415aba0 con 0x7f3d1415d240 2023-10-06T04:56:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.345+0000 7f3d18dec700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14150b30 con 0x7f3d140b27b0 2023-10-06T04:56:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d11d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d1415d240 0x7f3d140ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d12d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3d140cd150 0x7f3d140aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33650/0 (socket says 172.21.15.3:33650) 2023-10-06T04:56:15.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d12d9d700 1 -- 172.21.15.3:0/530948105 learned_addr learned my addr 172.21.15.3:0/530948105 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:56:15.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d1259c700 1 --2- 172.21.15.3:0/530948105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d1415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:15.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d11d9b700 1 -- 172.21.15.3:0/530948105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 msgr2=0x7f3d1415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:15.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d11d9b700 1 --2- 172.21.15.3:0/530948105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d1415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d11d9b700 1 -- 172.21.15.3:0/530948105 >> v1:172.21.15.3:6789/0 conn(0x7f3d140cd150 legacy=0x7f3d140aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:15.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d11d9b700 1 -- 172.21.15.3:0/530948105 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1415d6d0 con 0x7f3d1415d240 2023-10-06T04:56:15.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d1259c700 1 --2- 172.21.15.3:0/530948105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d1415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:56:15.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d1159a700 1 -- 172.21.15.3:0/530948105 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738320888 0 0) 0x7f3d140cd5f0 con 0x7f3d140cd150 2023-10-06T04:56:15.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d11d9b700 1 --2- 172.21.15.3:0/530948105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d140ccc10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009910 tx=0x7f3cfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163cb22d7cb9221d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:15.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d1159a700 1 -- 172.21.15.3:0/530948105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00b820 con 0x7f3d1415d240 2023-10-06T04:56:15.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d1159a700 1 -- 172.21.15.3:0/530948105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc00b980 con 0x7f3d1415d240 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.346+0000 7f3d1159a700 1 -- 172.21.15.3:0/530948105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00bd20 con 0x7f3d1415d240 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 -- 172.21.15.3:0/530948105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 msgr2=0x7f3d140ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 --2- 172.21.15.3:0/530948105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d140ccc10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009910 tx=0x7f3cfc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 -- 172.21.15.3:0/530948105 shutdown_connections 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 --2- 172.21.15.3:0/530948105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d1415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 --2- 172.21.15.3:0/530948105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d140ccc10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 -- 172.21.15.3:0/530948105 >> 172.21.15.3:0/530948105 conn(0x7f3d14070310 msgr2=0x7f3d14070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 -- 172.21.15.3:0/530948105 shutdown_connections 2023-10-06T04:56:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.347+0000 7f3d18dec700 1 -- 172.21.15.3:0/530948105 wait complete. 2023-10-06T04:56:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.348+0000 7f3d18dec700 1 Processor -- start 2023-10-06T04:56:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.348+0000 7f3d18dec700 1 -- start start 2023-10-06T04:56:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d18dec700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d140c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d18dec700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d140cd150 0x7f3d140c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d18dec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d1416ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d18dec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140c8d90 con 0x7f3d1415d240 2023-10-06T04:56:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d18dec700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140c87d0 con 0x7f3d140cd150 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d18dec700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140c8ab0 con 0x7f3d140b27b0 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d1416ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d1416ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54322/0 (socket says 172.21.15.3:54322) 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 -- 172.21.15.3:0/363656435 learned_addr learned my addr 172.21.15.3:0/363656435 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d1259c700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d140c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d11d9b700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d140cd150 0x7f3d140c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 -- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 msgr2=0x7f3d140c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:15.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d140c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 -- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d140cd150 msgr2=0x7f3d140c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:15.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d140cd150 0x7f3d140c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 -- 172.21.15.3:0/363656435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc009420 con 0x7f3d1415d240 2023-10-06T04:56:15.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.349+0000 7f3d12d9d700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d1416ce50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0011e0 tx=0x7f3d0c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.350+0000 7f3d0a7fc700 1 -- 172.21.15.3:0/363656435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c005b30 con 0x7f3d1415d240 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.350+0000 7f3d0a7fc700 1 -- 172.21.15.3:0/363656435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c005c90 con 0x7f3d1415d240 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.350+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1415af10 con 0x7f3d1415d240 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.350+0000 7f3d0a7fc700 1 -- 172.21.15.3:0/363656435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c007320 con 0x7f3d1415d240 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.350+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1415b2a0 con 0x7f3d1415d240 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.351+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1415ad50 con 0x7f3d1415d240 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.351+0000 7f3d0a7fc700 1 -- 172.21.15.3:0/363656435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d0c007690 con 0x7f3d1415d240 2023-10-06T04:56:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.351+0000 7f3d0a7fc700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3cf4066f30 0x7f3cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.352+0000 7f3d0a7fc700 1 -- 172.21.15.3:0/363656435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d0c083b50 con 0x7f3d1415d240 2023-10-06T04:56:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.352+0000 7f3d1259c700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3cf4066f30 0x7f3cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.352+0000 7f3d1259c700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3cf4066f30 0x7f3cf40693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400d6e0 tx=0x7f3d04007850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.356+0000 7f3d0a7fc700 1 -- 172.21.15.3:0/363656435 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d0c050bd0 con 0x7f3d1415d240 2023-10-06T04:56:15.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.593+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3d14071680 con 0x7f3d1415d240 2023-10-06T04:56:15.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.593+0000 7f3d0a7fc700 1 -- 172.21.15.3:0/363656435 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f3d0c017080 con 0x7f3d1415d240 2023-10-06T04:56:15.595 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:15.595 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:15.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3cf4066f30 msgr2=0x7f3cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:15.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3cf4066f30 0x7f3cf40693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400d6e0 tx=0x7f3d04007850 comp rx=0 tx=0).stop 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 msgr2=0x7f3d1416ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d1416ce50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0011e0 tx=0x7f3d0c007d30 comp rx=0 tx=0).stop 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 shutdown_connections 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3cf4066f30 0x7f3cf40693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415d240 0x7f3d1416ce50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d140b27b0 0x7f3d140c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 --2- 172.21.15.3:0/363656435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d140cd150 0x7f3d140c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 >> 172.21.15.3:0/363656435 conn(0x7f3d14070310 msgr2=0x7f3d140b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 shutdown_connections 2023-10-06T04:56:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:15.595+0000 7f3d18dec700 1 -- 172.21.15.3:0/363656435 wait complete. 2023-10-06T04:56:15.608 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:15.608 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:20.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.741+0000 7f3cba575700 1 Processor -- start 2023-10-06T04:56:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.741+0000 7f3cba575700 1 -- start start 2023-10-06T04:56:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.741+0000 7f3cba575700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 0x7f3cb40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.741+0000 7f3cba575700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb40cf2a0 con 0x7f3cb415c7b0 2023-10-06T04:56:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.741+0000 7f3cba575700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb40b05e0 con 0x7f3cb415cb80 2023-10-06T04:56:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.741+0000 7f3cba575700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40cc7d0 con 0x7f3cb415dd00 2023-10-06T04:56:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3cb415c7b0 0x7f3cb40cb390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33666/0 (socket says 172.21.15.3:33666) 2023-10-06T04:56:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb3fff700 1 -- 172.21.15.3:0/1413440334 learned_addr learned my addr 172.21.15.3:0/1413440334 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:56:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434117058 0 0) 0x7f3cb40cf2a0 con 0x7f3cb415c7b0 2023-10-06T04:56:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c9c003570 con 0x7f3cb415c7b0 2023-10-06T04:56:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb8b12700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 0x7f3cb40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1760202038 0 0) 0x7f3c9c003570 con 0x7f3cb415c7b0 2023-10-06T04:56:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 msgr2=0x7f3cb40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 0x7f3cb40cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 >> v1:172.21.15.153:6789/0 conn(0x7f3cb415cb80 legacy=0x7f3cb40afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40ce970 con 0x7f3cb415c7b0 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f3ca4001470 con 0x7f3cb415c7b0 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 >> v1:172.21.15.3:6789/0 conn(0x7f3cb415c7b0 legacy=0x7f3cb40cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 0x7f3c9c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c9c004010 0x7f3c9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c9c004d50 0x7f3c9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40cc7d0 con 0x7f3c9c004d50 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c00bbf0 con 0x7f3c9c004010 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c00bdb0 con 0x7f3cb415dd00 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.742+0000 7f3cb8b12700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c9c004d50 0x7f3c9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb3fff700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 0x7f3c9c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb37fe700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c9c004010 0x7f3c9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb8b12700 1 -- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 msgr2=0x7f3c9c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb8b12700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb415dd00 0x7f3c9c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb8b12700 1 -- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c9c004010 msgr2=0x7f3c9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb8b12700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c9c004010 0x7f3c9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb8b12700 1 -- 172.21.15.3:0/1413440334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c003570 con 0x7f3c9c004d50 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb8b12700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c9c004d50 0x7f3c9c0051c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0094d0 tx=0x7f3cac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c34006e43431736 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00ea90 con 0x7f3c9c004d50 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac00f040 con 0x7f3c9c004d50 2023-10-06T04:56:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.743+0000 7f3cb2ffd700 1 -- 172.21.15.3:0/1413440334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00ee20 con 0x7f3c9c004d50 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.744+0000 7f3cba575700 1 -- 172.21.15.3:0/1413440334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c9c004d50 msgr2=0x7f3c9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.744+0000 7f3cba575700 1 --2- 172.21.15.3:0/1413440334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c9c004d50 0x7f3c9c0051c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0094d0 tx=0x7f3cac00b040 comp rx=0 tx=0).stop 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.744+0000 7f3cb8b12700 1 -- 172.21.15.3:0/1413440334 reap_dead start 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.744+0000 7f3cba575700 1 -- 172.21.15.3:0/1413440334 shutdown_connections 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.744+0000 7f3cba575700 1 -- 172.21.15.3:0/1413440334 >> 172.21.15.3:0/1413440334 conn(0x7f3cb4070310 msgr2=0x7f3cb4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.744+0000 7f3cba575700 1 -- 172.21.15.3:0/1413440334 shutdown_connections 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.744+0000 7f3cba575700 1 -- 172.21.15.3:0/1413440334 wait complete. 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.745+0000 7f3cba575700 1 Processor -- start 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.745+0000 7f3cba575700 1 -- start start 2023-10-06T04:56:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cba575700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb41449c0 0x7f3cb40ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cba575700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb4144d90 0x7f3cb40c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cba575700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb40c7bb0 0x7f3cb40c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cba575700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c9270 con 0x7f3cb41449c0 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cba575700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40ca090 con 0x7f3cb40c7bb0 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cba575700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c9480 con 0x7f3cb4144d90 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb41449c0 0x7f3cb40ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb41449c0 0x7f3cb40ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54328/0 (socket says 172.21.15.3:54328) 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 -- 172.21.15.3:0/3621276170 learned_addr learned my addr 172.21.15.3:0/3621276170 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb37fe700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb4144d90 0x7f3cb40c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb8b12700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb40c7bb0 0x7f3cb40c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 -- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb4144d90 msgr2=0x7f3cb40c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb4144d90 0x7f3cb40c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 -- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb40c7bb0 msgr2=0x7f3cb40c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb40c7bb0 0x7f3cb40c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 -- 172.21.15.3:0/3621276170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40c9760 con 0x7f3cb41449c0 2023-10-06T04:56:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cb3fff700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb41449c0 0x7f3cb40ca520 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40073c0 tx=0x7f3ca4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3c9bfff700 1 -- 172.21.15.3:0/3621276170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4018030 con 0x7f3cb41449c0 2023-10-06T04:56:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3c9bfff700 1 -- 172.21.15.3:0/3621276170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca400f8d0 con 0x7f3cb41449c0 2023-10-06T04:56:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3c9bfff700 1 -- 172.21.15.3:0/3621276170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400fc90 con 0x7f3cb41449c0 2023-10-06T04:56:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.746+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb40c9a40 con 0x7f3cb41449c0 2023-10-06T04:56:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.747+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb4156510 con 0x7f3cb41449c0 2023-10-06T04:56:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.747+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4071680 con 0x7f3cb41449c0 2023-10-06T04:56:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.748+0000 7f3c9bfff700 1 -- 172.21.15.3:0/3621276170 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ca401b070 con 0x7f3cb41449c0 2023-10-06T04:56:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.748+0000 7f3c9bfff700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.748+0000 7f3c9bfff700 1 -- 172.21.15.3:0/3621276170 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ca4019070 con 0x7f3cb41449c0 2023-10-06T04:56:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.748+0000 7f3cb37fe700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.749+0000 7f3cb37fe700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ca0066f30 0x7f3ca00693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8000f60 tx=0x7f3ca8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.753+0000 7f3c9bfff700 1 -- 172.21.15.3:0/3621276170 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ca4051830 con 0x7f3cb41449c0 2023-10-06T04:56:20.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.987+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3cb40ce1b0 con 0x7f3cb41449c0 2023-10-06T04:56:20.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.987+0000 7f3c9bfff700 1 -- 172.21.15.3:0/3621276170 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f3ca4054e50 con 0x7f3cb41449c0 2023-10-06T04:56:20.989 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:20.989 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ca0066f30 msgr2=0x7f3ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ca0066f30 0x7f3ca00693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8000f60 tx=0x7f3ca8009040 comp rx=0 tx=0).stop 2023-10-06T04:56:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb41449c0 msgr2=0x7f3cb40ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb41449c0 0x7f3cb40ca520 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40073c0 tx=0x7f3ca4001e50 comp rx=0 tx=0).stop 2023-10-06T04:56:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 shutdown_connections 2023-10-06T04:56:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cb40c7bb0 0x7f3cb40c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cb41449c0 0x7f3cb40ca520 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 --2- 172.21.15.3:0/3621276170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cb4144d90 0x7f3cb40c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 >> 172.21.15.3:0/3621276170 conn(0x7f3cb4070310 msgr2=0x7f3cb41573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 shutdown_connections 2023-10-06T04:56:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:20.989+0000 7f3cba575700 1 -- 172.21.15.3:0/3621276170 wait complete. 2023-10-06T04:56:21.002 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:21.002 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:24.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:24.992 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:26.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.132+0000 7f96e8fa2700 1 Processor -- start 2023-10-06T04:56:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.132+0000 7f96e8fa2700 1 -- start start 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e8fa2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96e40af530 0x7f96e40cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e8fa2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f96e415dbb0 0x7f96e40afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e8fa2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e40ce5e0 con 0x7f96e40b0230 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e8fa2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e41598b0 con 0x7f96e40af530 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e8fa2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e40ce8c0 con 0x7f96e415dbb0 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96e40af530 0x7f96e40cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f96e40b0230 0x7f96e40ccc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37430/0 (socket says 172.21.15.3:37430) 2023-10-06T04:56:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e3fff700 1 -- 172.21.15.3:0/3894893676 learned_addr learned my addr 172.21.15.3:0/3894893676 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e2ffd700 1 --2- 172.21.15.3:0/3894893676 >> v2:172.21.15.153:3300/0 conn(0x7f96e415dbb0 0x7f96e40afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e37fe700 1 -- 172.21.15.3:0/3894893676 >> v2:172.21.15.153:3300/0 conn(0x7f96e415dbb0 msgr2=0x7f96e40afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e37fe700 1 --2- 172.21.15.3:0/3894893676 >> v2:172.21.15.153:3300/0 conn(0x7f96e415dbb0 0x7f96e40afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e37fe700 1 -- 172.21.15.3:0/3894893676 >> v1:172.21.15.3:6789/0 conn(0x7f96e40b0230 legacy=0x7f96e40ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e37fe700 1 -- 172.21.15.3:0/3894893676 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e40ceba0 con 0x7f96e40af530 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.133+0000 7f96e2ffd700 1 --2- 172.21.15.3:0/3894893676 >> v2:172.21.15.153:3300/0 conn(0x7f96e415dbb0 0x7f96e40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e37fe700 1 --2- 172.21.15.3:0/3894893676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40af530 0x7f96e40cb230 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f96e406fad0 tx=0x7f96d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa00b2f77fcb1d9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e27fc700 1 -- 172.21.15.3:0/3894893676 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400b740 con 0x7f96e40af530 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e27fc700 1 -- 172.21.15.3:0/3894893676 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d400b8a0 con 0x7f96e40af530 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e27fc700 1 -- 172.21.15.3:0/3894893676 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400bc40 con 0x7f96e40af530 2023-10-06T04:56:26.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 -- 172.21.15.3:0/3894893676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40af530 msgr2=0x7f96e40cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/3894893676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40af530 0x7f96e40cb230 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f96e406fad0 tx=0x7f96d40091a0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 -- 172.21.15.3:0/3894893676 shutdown_connections 2023-10-06T04:56:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/3894893676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40af530 0x7f96e40cb230 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/3894893676 >> v2:172.21.15.153:3300/0 conn(0x7f96e415dbb0 0x7f96e40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 -- 172.21.15.3:0/3894893676 >> 172.21.15.3:0/3894893676 conn(0x7f96e4070de0 msgr2=0x7f96e406ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 -- 172.21.15.3:0/3894893676 shutdown_connections 2023-10-06T04:56:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.134+0000 7f96e8fa2700 1 -- 172.21.15.3:0/3894893676 wait complete. 2023-10-06T04:56:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.135+0000 7f96e8fa2700 1 Processor -- start 2023-10-06T04:56:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e8fa2700 1 -- start start 2023-10-06T04:56:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e8fa2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96e40af530 0x7f96e416d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e8fa2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40b0230 0x7f96e40b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e8fa2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96e415dbb0 0x7f96e40b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e8fa2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e40c5d30 con 0x7f96e40b0230 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e8fa2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e40c6010 con 0x7f96e415dbb0 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e8fa2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e40c62f0 con 0x7f96e40af530 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40b0230 0x7f96e40b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40b0230 0x7f96e40b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55664/0 (socket says 172.21.15.3:55664) 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e2ffd700 1 -- 172.21.15.3:0/4132906971 learned_addr learned my addr 172.21.15.3:0/4132906971 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e37fe700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96e40af530 0x7f96e416d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.136+0000 7f96e3fff700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96e415dbb0 0x7f96e40b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:26.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e2ffd700 1 -- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96e40af530 msgr2=0x7f96e416d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e2ffd700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96e40af530 0x7f96e416d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e2ffd700 1 -- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96e415dbb0 msgr2=0x7f96e40b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e2ffd700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96e415dbb0 0x7f96e40b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e2ffd700 1 -- 172.21.15.3:0/4132906971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009420 con 0x7f96e40b0230 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e2ffd700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40b0230 0x7f96e40b3060 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f96d800d860 tx=0x7f96d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96cb7fe700 1 -- 172.21.15.3:0/4132906971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d80125f0 con 0x7f96e40b0230 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96cb7fe700 1 -- 172.21.15.3:0/4132906971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d8012750 con 0x7f96e40b0230 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e40c65d0 con 0x7f96e40b0230 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96cb7fe700 1 -- 172.21.15.3:0/4132906971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8012b10 con 0x7f96e40b0230 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.137+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e40c5a70 con 0x7f96e40b0230 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.138+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e404c380 con 0x7f96e40b0230 2023-10-06T04:56:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.139+0000 7f96cb7fe700 1 -- 172.21.15.3:0/4132906971 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96d8013480 con 0x7f96e40b0230 2023-10-06T04:56:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.139+0000 7f96cb7fe700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96d0066f30 0x7f96d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.139+0000 7f96cb7fe700 1 -- 172.21.15.3:0/4132906971 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f96d8084940 con 0x7f96e40b0230 2023-10-06T04:56:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.139+0000 7f96e37fe700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96d0066f30 0x7f96d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.140+0000 7f96e37fe700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96d0066f30 0x7f96d00693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96d40093f0 tx=0x7f96d4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.143+0000 7f96cb7fe700 1 -- 172.21.15.3:0/4132906971 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96d8051a50 con 0x7f96e40b0230 2023-10-06T04:56:26.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.380+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f96e40cdfd0 con 0x7f96e40b0230 2023-10-06T04:56:26.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.381+0000 7f96cb7fe700 1 -- 172.21.15.3:0/4132906971 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f96d8055070 con 0x7f96e40b0230 2023-10-06T04:56:26.382 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:26.382 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96d0066f30 msgr2=0x7f96d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96d0066f30 0x7f96d00693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96d40093f0 tx=0x7f96d4017000 comp rx=0 tx=0).stop 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40b0230 msgr2=0x7f96e40b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40b0230 0x7f96e40b3060 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f96d800d860 tx=0x7f96d8008040 comp rx=0 tx=0).stop 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 shutdown_connections 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96d0066f30 0x7f96d00693e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96e415dbb0 0x7f96e40b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96e40af530 0x7f96e416d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 --2- 172.21.15.3:0/4132906971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e40b0230 0x7f96e40b3060 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 >> 172.21.15.3:0/4132906971 conn(0x7f96e4070de0 msgr2=0x7f96e40cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:26.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 shutdown_connections 2023-10-06T04:56:26.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:26.383+0000 7f96e8fa2700 1 -- 172.21.15.3:0/4132906971 wait complete. 2023-10-06T04:56:26.397 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:26.397 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:31.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.526+0000 7ff364298700 1 Processor -- start 2023-10-06T04:56:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.526+0000 7ff364298700 1 -- start start 2023-10-06T04:56:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.527+0000 7ff364298700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c0b27b0 0x7ff35c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.527+0000 7ff364298700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff35c15d240 0x7ff35c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.527+0000 7ff364298700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35c0cd5f0 con 0x7ff35c0cd150 2023-10-06T04:56:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.527+0000 7ff364298700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35c15aba0 con 0x7ff35c15d240 2023-10-06T04:56:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.527+0000 7ff364298700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c150b30 con 0x7ff35c0b27b0 2023-10-06T04:56:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362034700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c0b27b0 0x7ff35c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff361833700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff35c15d240 0x7ff35c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362835700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff35c0cd150 0x7ff35c0aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:36512/0 (socket says 172.21.15.3:36512) 2023-10-06T04:56:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362835700 1 -- 172.21.15.3:0/1845669512 learned_addr learned my addr 172.21.15.3:0/1845669512 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff361032700 1 -- 172.21.15.3:0/1845669512 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881440299 0 0) 0x7ff35c0cd5f0 con 0x7ff35c0cd150 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff361032700 1 -- 172.21.15.3:0/1845669512 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff344003570 con 0x7ff35c0cd150 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362034700 1 -- 172.21.15.3:0/1845669512 >> v2:172.21.15.153:3300/0 conn(0x7ff35c15d240 msgr2=0x7ff35c0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362034700 1 --2- 172.21.15.3:0/1845669512 >> v2:172.21.15.153:3300/0 conn(0x7ff35c15d240 0x7ff35c0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362034700 1 -- 172.21.15.3:0/1845669512 >> v1:172.21.15.153:6789/0 conn(0x7ff35c0cd150 legacy=0x7ff35c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362034700 1 -- 172.21.15.3:0/1845669512 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c15d6d0 con 0x7ff35c0b27b0 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff361833700 1 --2- 172.21.15.3:0/1845669512 >> v2:172.21.15.153:3300/0 conn(0x7ff35c15d240 0x7ff35c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.528+0000 7ff362034700 1 --2- 172.21.15.3:0/1845669512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c0b27b0 0x7ff35c15a660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009e90 tx=0x7ff34c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564eb881bb2d3ab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff361032700 1 -- 172.21.15.3:0/1845669512 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c00e070 con 0x7ff35c0b27b0 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff361032700 1 -- 172.21.15.3:0/1845669512 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c00bb40 con 0x7ff35c0b27b0 2023-10-06T04:56:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff361032700 1 -- 172.21.15.3:0/1845669512 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c012430 con 0x7ff35c0b27b0 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 -- 172.21.15.3:0/1845669512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c0b27b0 msgr2=0x7ff35c15a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 --2- 172.21.15.3:0/1845669512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c0b27b0 0x7ff35c15a660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009e90 tx=0x7ff34c009780 comp rx=0 tx=0).stop 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 -- 172.21.15.3:0/1845669512 shutdown_connections 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 --2- 172.21.15.3:0/1845669512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c0b27b0 0x7ff35c15a660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 --2- 172.21.15.3:0/1845669512 >> v2:172.21.15.153:3300/0 conn(0x7ff35c15d240 0x7ff35c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 -- 172.21.15.3:0/1845669512 >> 172.21.15.3:0/1845669512 conn(0x7ff35c070310 msgr2=0x7ff35c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 -- 172.21.15.3:0/1845669512 shutdown_connections 2023-10-06T04:56:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.529+0000 7ff364298700 1 -- 172.21.15.3:0/1845669512 wait complete. 2023-10-06T04:56:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.530+0000 7ff364298700 1 Processor -- start 2023-10-06T04:56:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.530+0000 7ff364298700 1 -- start start 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff364298700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff35c0b27b0 0x7ff35c14ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff364298700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff35c0cd150 0x7ff35c153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff364298700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c15d240 0x7ff35c0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff364298700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c1520a0 con 0x7ff35c0b27b0 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff364298700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c1522d0 con 0x7ff35c0cd150 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff364298700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c1525b0 con 0x7ff35c15d240 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff362034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff35c0b27b0 0x7ff35c14ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff362034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff35c0b27b0 0x7ff35c14ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55674/0 (socket says 172.21.15.3:55674) 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff362034700 1 -- 172.21.15.3:0/1624429938 learned_addr learned my addr 172.21.15.3:0/1624429938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff361833700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff35c0cd150 0x7ff35c153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.531+0000 7ff362835700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c15d240 0x7ff35c0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff362034700 1 -- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c15d240 msgr2=0x7ff35c0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff362034700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c15d240 0x7ff35c0c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff362034700 1 -- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff35c0cd150 msgr2=0x7ff35c153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff362034700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff35c0cd150 0x7ff35c153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff362034700 1 -- 172.21.15.3:0/1624429938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff350009420 con 0x7ff35c0b27b0 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff362034700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff35c0b27b0 0x7ff35c14ff20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00b610 tx=0x7ff34c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff35a7fc700 1 -- 172.21.15.3:0/1624429938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c012be0 con 0x7ff35c0b27b0 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff35a7fc700 1 -- 172.21.15.3:0/1624429938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c012d60 con 0x7ff35c0b27b0 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c009a20 con 0x7ff35c0b27b0 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff35a7fc700 1 -- 172.21.15.3:0/1624429938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c0276c0 con 0x7ff35c0b27b0 2023-10-06T04:56:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.532+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c152bf0 con 0x7ff35c0b27b0 2023-10-06T04:56:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.533+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35c070f70 con 0x7ff35c0b27b0 2023-10-06T04:56:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.533+0000 7ff35a7fc700 1 -- 172.21.15.3:0/1624429938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff34c024070 con 0x7ff35c0b27b0 2023-10-06T04:56:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.534+0000 7ff35a7fc700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff348066f30 0x7ff3480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.534+0000 7ff35a7fc700 1 -- 172.21.15.3:0/1624429938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff34c086c90 con 0x7ff35c0b27b0 2023-10-06T04:56:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.534+0000 7ff361833700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff348066f30 0x7ff3480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.535+0000 7ff361833700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff348066f30 0x7ff3480693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0d0df0 tx=0x7ff3500091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.538+0000 7ff35a7fc700 1 -- 172.21.15.3:0/1624429938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff34c053d00 con 0x7ff35c0b27b0 2023-10-06T04:56:31.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.776+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff35c071680 con 0x7ff35c0b27b0 2023-10-06T04:56:31.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.776+0000 7ff35a7fc700 1 -- 172.21.15.3:0/1624429938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7ff34c057320 con 0x7ff35c0b27b0 2023-10-06T04:56:31.779 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:31.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff348066f30 msgr2=0x7ff3480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:31.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff348066f30 0x7ff3480693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0d0df0 tx=0x7ff3500091a0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff35c0b27b0 msgr2=0x7ff35c14ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:31.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff35c0b27b0 0x7ff35c14ff20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00b610 tx=0x7ff34c019000 comp rx=0 tx=0).stop 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 shutdown_connections 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff348066f30 0x7ff3480693e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff35c15d240 0x7ff35c0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff35c0b27b0 0x7ff35c14ff20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 --2- 172.21.15.3:0/1624429938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff35c0cd150 0x7ff35c153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 >> 172.21.15.3:0/1624429938 conn(0x7ff35c070310 msgr2=0x7ff35c0c5220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 shutdown_connections 2023-10-06T04:56:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:31.778+0000 7ff364298700 1 -- 172.21.15.3:0/1624429938 wait complete. 2023-10-06T04:56:31.792 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:31.793 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:36.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:36.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.922+0000 7fe04514b700 1 Processor -- start 2023-10-06T04:56:36.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe04514b700 1 -- start start 2023-10-06T04:56:36.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe04514b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0400b39b0 0x7fe04015d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe04514b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe04015aa30 0x7fe040159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe04514b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0400b3d80 0x7fe040159650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe04514b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04015aee0 con 0x7fe0400b3d80 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe04514b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe040159b90 con 0x7fe04015aa30 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe04514b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04015dc10 con 0x7fe0400b39b0 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.923+0000 7fe03f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0400b3d80 0x7fe040159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0400b3d80 0x7fe040159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33538/0 (socket says 172.21.15.3:33538) 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 -- 172.21.15.3:0/4241082245 learned_addr learned my addr 172.21.15.3:0/4241082245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03ed9d700 1 --2- 172.21.15.3:0/4241082245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0400b39b0 0x7fe04015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03e59c700 1 --2- 172.21.15.3:0/4241082245 >> v2:172.21.15.153:3300/0 conn(0x7fe04015aa30 0x7fe040159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 -- 172.21.15.3:0/4241082245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0400b39b0 msgr2=0x7fe04015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 --2- 172.21.15.3:0/4241082245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0400b39b0 0x7fe04015d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 -- 172.21.15.3:0/4241082245 >> v2:172.21.15.153:3300/0 conn(0x7fe04015aa30 msgr2=0x7fe040159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 --2- 172.21.15.3:0/4241082245 >> v2:172.21.15.153:3300/0 conn(0x7fe04015aa30 0x7fe040159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 -- 172.21.15.3:0/4241082245 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04015def0 con 0x7fe0400b3d80 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03ed9d700 1 --2- 172.21.15.3:0/4241082245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0400b39b0 0x7fe04015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03e59c700 1 --2- 172.21.15.3:0/4241082245 >> v2:172.21.15.153:3300/0 conn(0x7fe04015aa30 0x7fe040159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:56:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03f59e700 1 --2- 172.21.15.3:0/4241082245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b3d80 0x7fe040159650 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe04006b6f0 tx=0x7fe030009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09276bf7d1d142a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03dd9b700 1 -- 172.21.15.3:0/4241082245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03000e070 con 0x7fe0400b3d80 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03dd9b700 1 -- 172.21.15.3:0/4241082245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03000ba50 con 0x7fe0400b3d80 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.924+0000 7fe03dd9b700 1 -- 172.21.15.3:0/4241082245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300092e0 con 0x7fe0400b3d80 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 -- 172.21.15.3:0/4241082245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b3d80 msgr2=0x7fe040159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 --2- 172.21.15.3:0/4241082245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b3d80 0x7fe040159650 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe04006b6f0 tx=0x7fe030009840 comp rx=0 tx=0).stop 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 -- 172.21.15.3:0/4241082245 shutdown_connections 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 --2- 172.21.15.3:0/4241082245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b3d80 0x7fe040159650 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 --2- 172.21.15.3:0/4241082245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0400b39b0 0x7fe04015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 --2- 172.21.15.3:0/4241082245 >> v2:172.21.15.153:3300/0 conn(0x7fe04015aa30 0x7fe040159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:36.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 -- 172.21.15.3:0/4241082245 >> 172.21.15.3:0/4241082245 conn(0x7fe040067ac0 msgr2=0x7fe040067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:36.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 -- 172.21.15.3:0/4241082245 shutdown_connections 2023-10-06T04:56:36.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.925+0000 7fe04514b700 1 -- 172.21.15.3:0/4241082245 wait complete. 2023-10-06T04:56:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.926+0000 7fe04514b700 1 Processor -- start 2023-10-06T04:56:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.926+0000 7fe04514b700 1 -- start start 2023-10-06T04:56:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe04514b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b39b0 0x7fe040169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe04514b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0400b3d80 0x7fe040168b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe04514b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe04015aa30 0x7fe0400af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe04514b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04014e6b0 con 0x7fe0400b39b0 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe04514b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe040153ab0 con 0x7fe0400b3d80 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe04514b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe040151e40 con 0x7fe04015aa30 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b39b0 0x7fe040169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b39b0 0x7fe040169c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33544/0 (socket says 172.21.15.3:33544) 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 -- 172.21.15.3:0/2456758381 learned_addr learned my addr 172.21.15.3:0/2456758381 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03e59c700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0400b3d80 0x7fe040168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03f59e700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe04015aa30 0x7fe0400af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 -- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe04015aa30 msgr2=0x7fe0400af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe04015aa30 0x7fe0400af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 -- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0400b3d80 msgr2=0x7fe040168b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0400b3d80 0x7fe040168b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03ed9d700 1 -- 172.21.15.3:0/2456758381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe030009ae0 con 0x7fe0400b39b0 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.927+0000 7fe03e59c700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0400b3d80 0x7fe040168b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.928+0000 7fe03ed9d700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b39b0 0x7fe040169c20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c8e0 tx=0x7fe038008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.928+0000 7fe036ffd700 1 -- 172.21.15.3:0/2456758381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038007590 con 0x7fe0400b39b0 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.928+0000 7fe036ffd700 1 -- 172.21.15.3:0/2456758381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380076f0 con 0x7fe0400b39b0 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.928+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04016a160 con 0x7fe0400b39b0 2023-10-06T04:56:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.928+0000 7fe036ffd700 1 -- 172.21.15.3:0/2456758381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038007a70 con 0x7fe0400b39b0 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.928+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04016cda0 con 0x7fe0400b39b0 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.929+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0400644c0 con 0x7fe0400b39b0 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.929+0000 7fe036ffd700 1 -- 172.21.15.3:0/2456758381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0380113f0 con 0x7fe0400b39b0 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.930+0000 7fe036ffd700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe020066f30 0x7fe0200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.930+0000 7fe036ffd700 1 -- 172.21.15.3:0/2456758381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe038084820 con 0x7fe0400b39b0 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.930+0000 7fe03e59c700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe020066f30 0x7fe0200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.930+0000 7fe03e59c700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe020066f30 0x7fe0200693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe04016d8c0 tx=0x7fe028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:36.934+0000 7fe036ffd700 1 -- 172.21.15.3:0/2456758381 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0380553c0 con 0x7fe0400b39b0 2023-10-06T04:56:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.172+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe04015e1d0 con 0x7fe0400b39b0 2023-10-06T04:56:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.173+0000 7fe036ffd700 1 -- 172.21.15.3:0/2456758381 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fe038054f50 con 0x7fe0400b39b0 2023-10-06T04:56:37.176 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:37.176 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe020066f30 msgr2=0x7fe0200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe020066f30 0x7fe0200693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe04016d8c0 tx=0x7fe028008040 comp rx=0 tx=0).stop 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b39b0 msgr2=0x7fe040169c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b39b0 0x7fe040169c20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c8e0 tx=0x7fe038008040 comp rx=0 tx=0).stop 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 shutdown_connections 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe020066f30 0x7fe0200693e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe04015aa30 0x7fe0400af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0400b39b0 0x7fe040169c20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 --2- 172.21.15.3:0/2456758381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0400b3d80 0x7fe040168b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 >> 172.21.15.3:0/2456758381 conn(0x7fe040067ac0 msgr2=0x7fe04015c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 shutdown_connections 2023-10-06T04:56:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:37.175+0000 7fe04514b700 1 -- 172.21.15.3:0/2456758381 wait complete. 2023-10-06T04:56:37.189 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:37.189 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:42.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:42.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 Processor -- start 2023-10-06T04:56:42.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 -- start start 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4b4159560 0x7fd4b40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd4b40b0b30 0x7fd4b406c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b406c8b0 0x7fd4b415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b40aec90 con 0x7fd4b4159560 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b40b0fa0 con 0x7fd4b40b0b30 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.317+0000 7fd4b998b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b406cd30 con 0x7fd4b406c8b0 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4b4159560 0x7fd4b40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4b4159560 0x7fd4b40b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44084/0 (socket says 172.21.15.3:44084) 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 -- 172.21.15.3:0/1429298618 learned_addr learned my addr 172.21.15.3:0/1429298618 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b37fe700 1 --2- 172.21.15.3:0/1429298618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b406c8b0 0x7fd4b415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b27fc700 1 --2- 172.21.15.3:0/1429298618 >> v2:172.21.15.153:3300/0 conn(0x7fd4b40b0b30 0x7fd4b406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 -- 172.21.15.3:0/1429298618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b406c8b0 msgr2=0x7fd4b415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 --2- 172.21.15.3:0/1429298618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b406c8b0 0x7fd4b415d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 -- 172.21.15.3:0/1429298618 >> v2:172.21.15.153:3300/0 conn(0x7fd4b40b0b30 msgr2=0x7fd4b406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 --2- 172.21.15.3:0/1429298618 >> v2:172.21.15.153:3300/0 conn(0x7fd4b40b0b30 0x7fd4b406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 -- 172.21.15.3:0/1429298618 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b40ae010 con 0x7fd4b4159560 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b37fe700 1 --2- 172.21.15.3:0/1429298618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b406c8b0 0x7fd4b415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b27fc700 1 --2- 172.21.15.3:0/1429298618 >> v2:172.21.15.153:3300/0 conn(0x7fd4b40b0b30 0x7fd4b406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:56:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b2ffd700 1 --2- 172.21.15.3:0/1429298618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b40b05f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40adc50 tx=0x7fd4ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e279e6e3d92a4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b1ffb700 1 -- 172.21.15.3:0/1429298618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00b820 con 0x7fd4b4159560 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b1ffb700 1 -- 172.21.15.3:0/1429298618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac00b980 con 0x7fd4b4159560 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.318+0000 7fd4b1ffb700 1 -- 172.21.15.3:0/1429298618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00bd20 con 0x7fd4b4159560 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 -- 172.21.15.3:0/1429298618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 msgr2=0x7fd4b40b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 --2- 172.21.15.3:0/1429298618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b40b05f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40adc50 tx=0x7fd4ac009250 comp rx=0 tx=0).stop 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 -- 172.21.15.3:0/1429298618 shutdown_connections 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 --2- 172.21.15.3:0/1429298618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b406c8b0 0x7fd4b415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 --2- 172.21.15.3:0/1429298618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b40b05f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 --2- 172.21.15.3:0/1429298618 >> v2:172.21.15.153:3300/0 conn(0x7fd4b40b0b30 0x7fd4b406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 -- 172.21.15.3:0/1429298618 >> 172.21.15.3:0/1429298618 conn(0x7fd4b40cd170 msgr2=0x7fd4b40cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:42.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 -- 172.21.15.3:0/1429298618 shutdown_connections 2023-10-06T04:56:42.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.319+0000 7fd4b998b700 1 -- 172.21.15.3:0/1429298618 wait complete. 2023-10-06T04:56:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.320+0000 7fd4b998b700 1 Processor -- start 2023-10-06T04:56:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.320+0000 7fd4b998b700 1 -- start start 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b998b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b406c8b0 0x7fd4b40ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b998b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b40b0b30 0x7fd4b406ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b998b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b998b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40ac780 con 0x7fd4b4159560 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b998b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40aa900 con 0x7fd4b406c8b0 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b998b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40a9cb0 con 0x7fd4b40b0b30 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b415d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44098/0 (socket says 172.21.15.3:44098) 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 -- 172.21.15.3:0/2600071189 learned_addr learned my addr 172.21.15.3:0/2600071189 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b2ffd700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b406c8b0 0x7fd4b40ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b27fc700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b40b0b30 0x7fd4b406ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 -- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b40b0b30 msgr2=0x7fd4b406ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b40b0b30 0x7fd4b406ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 -- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b406c8b0 msgr2=0x7fd4b40ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b406c8b0 0x7fd4b40ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 -- 172.21.15.3:0/2600071189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0094f0 con 0x7fd4b4159560 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b27fc700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b40b0b30 0x7fd4b406ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.321+0000 7fd4b37fe700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b415d770 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400ce40 tx=0x7fd4a40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.322+0000 7fd4aaffd700 1 -- 172.21.15.3:0/2600071189 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400ad20 con 0x7fd4b4159560 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.322+0000 7fd4aaffd700 1 -- 172.21.15.3:0/2600071189 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4016420 con 0x7fd4b4159560 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.322+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b40aa2b0 con 0x7fd4b4159560 2023-10-06T04:56:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.322+0000 7fd4aaffd700 1 -- 172.21.15.3:0/2600071189 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400aea0 con 0x7fd4b4159560 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.322+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b40a9940 con 0x7fd4b4159560 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.323+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b40aa110 con 0x7fd4b4159560 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.323+0000 7fd4aaffd700 1 -- 172.21.15.3:0/2600071189 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4a40169f0 con 0x7fd4b4159560 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.323+0000 7fd4aaffd700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd494066f30 0x7fd4940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.324+0000 7fd4aaffd700 1 -- 172.21.15.3:0/2600071189 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4a4012070 con 0x7fd4b4159560 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.324+0000 7fd4b2ffd700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd494066f30 0x7fd4940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.324+0000 7fd4b2ffd700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd494066f30 0x7fd4940693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40adc50 tx=0x7fd4ac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.328+0000 7fd4aaffd700 1 -- 172.21.15.3:0/2600071189 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4a404e5a0 con 0x7fd4b4159560 2023-10-06T04:56:42.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.565+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd4b4071680 con 0x7fd4b4159560 2023-10-06T04:56:42.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.565+0000 7fd4aaffd700 1 -- 172.21.15.3:0/2600071189 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fd4b40aa110 con 0x7fd4b4159560 2023-10-06T04:56:42.568 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:42.568 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:42.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd494066f30 msgr2=0x7fd4940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:42.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd494066f30 0x7fd4940693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40adc50 tx=0x7fd4ac017000 comp rx=0 tx=0).stop 2023-10-06T04:56:42.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 msgr2=0x7fd4b415d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:42.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b415d770 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400ce40 tx=0x7fd4a40077d0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 shutdown_connections 2023-10-06T04:56:42.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd494066f30 0x7fd4940693e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b406c8b0 0x7fd4b40ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4b4159560 0x7fd4b415d770 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 --2- 172.21.15.3:0/2600071189 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4b40b0b30 0x7fd4b406ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:42.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 >> 172.21.15.3:0/2600071189 conn(0x7fd4b40cd170 msgr2=0x7fd4b4156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:42.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 shutdown_connections 2023-10-06T04:56:42.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:42.567+0000 7fd4b998b700 1 -- 172.21.15.3:0/2600071189 wait complete. 2023-10-06T04:56:42.582 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:42.582 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:47.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.713+0000 7fa85f59e700 1 Processor -- start 2023-10-06T04:56:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.713+0000 7fa85f59e700 1 -- start start 2023-10-06T04:56:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.714+0000 7fa85f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa86015a6a0 0x7fa86015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.714+0000 7fa85f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa86015d660 0x7fa8600ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.714+0000 7fa85f59e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600cb240 con 0x7fa8601592c0 2023-10-06T04:56:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.714+0000 7fa85f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86015e0c0 con 0x7fa86015d660 2023-10-06T04:56:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.714+0000 7fa85f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600cb7e0 con 0x7fa86015a6a0 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85ed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa86015d660 0x7fa8600ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85e59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa8601592c0 0x7fa8600ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:54552/0 (socket says 172.21.15.3:54552) 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85ed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa86015d660 0x7fa8600ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:47802/0 (socket says 172.21.15.3:47802) 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85e59c700 1 -- 172.21.15.3:0/2783595032 learned_addr learned my addr 172.21.15.3:0/2783595032 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85dd9b700 1 --2- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa86015a6a0 0x7fa86015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85ed9d700 1 -- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa86015a6a0 msgr2=0x7fa86015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85ed9d700 1 --2- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa86015a6a0 0x7fa86015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85ed9d700 1 -- 172.21.15.3:0/2783595032 >> v1:172.21.15.153:6789/0 conn(0x7fa8601592c0 legacy=0x7fa8600ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85d59a700 1 -- 172.21.15.3:0/2783595032 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487203763 0 0) 0x7fa8600cb240 con 0x7fa8601592c0 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85ed9d700 1 -- 172.21.15.3:0/2783595032 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600ce900 con 0x7fa86015d660 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85ed9d700 1 --2- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 0x7fa8600ccc50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e60 tx=0x7fa850009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f63fce03d74504 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85d59a700 1 -- 172.21.15.3:0/2783595032 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000e070 con 0x7fa86015d660 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85d59a700 1 -- 172.21.15.3:0/2783595032 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85000bb60 con 0x7fa86015d660 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.715+0000 7fa85d59a700 1 -- 172.21.15.3:0/2783595032 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850012430 con 0x7fa86015d660 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 -- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 msgr2=0x7fa8600ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 --2- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 0x7fa8600ccc50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e60 tx=0x7fa850009780 comp rx=0 tx=0).stop 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 -- 172.21.15.3:0/2783595032 shutdown_connections 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 --2- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 0x7fa8600ccc50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 --2- 172.21.15.3:0/2783595032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa86015a6a0 0x7fa86015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 -- 172.21.15.3:0/2783595032 >> 172.21.15.3:0/2783595032 conn(0x7fa860089d90 msgr2=0x7fa8600996b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 -- 172.21.15.3:0/2783595032 shutdown_connections 2023-10-06T04:56:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.716+0000 7fa85f59e700 1 -- 172.21.15.3:0/2783595032 wait complete. 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.717+0000 7fa85f59e700 1 Processor -- start 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.717+0000 7fa85f59e700 1 -- start start 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8601592c0 0x7fa860151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa86015a6a0 0x7fa86006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 0x7fa8600b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860169060 con 0x7fa86015a6a0 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860169320 con 0x7fa86015d660 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa860169600 con 0x7fa8601592c0 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa86015a6a0 0x7fa86006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa86015a6a0 0x7fa86006eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44110/0 (socket says 172.21.15.3:44110) 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85dd9b700 1 -- 172.21.15.3:0/1461951023 learned_addr learned my addr 172.21.15.3:0/1461951023 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85e59c700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8601592c0 0x7fa860151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85ed9d700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 0x7fa8600b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.718+0000 7fa85dd9b700 1 -- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8601592c0 msgr2=0x7fa860151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa85dd9b700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8601592c0 0x7fa860151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa85dd9b700 1 -- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 msgr2=0x7fa8600b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa85dd9b700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 0x7fa8600b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa85dd9b700 1 -- 172.21.15.3:0/1461951023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa850009a20 con 0x7fa86015a6a0 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa85dd9b700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa86015a6a0 0x7fa86006eb60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e930 tx=0x7fa8480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa8567fc700 1 -- 172.21.15.3:0/1461951023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848013680 con 0x7fa86015a6a0 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa8567fc700 1 -- 172.21.15.3:0/1461951023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8480137e0 con 0x7fa86015a6a0 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa8567fc700 1 -- 172.21.15.3:0/1461951023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848013b80 con 0x7fa86015a6a0 2023-10-06T04:56:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8601698e0 con 0x7fa86015a6a0 2023-10-06T04:56:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.719+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa860169ea0 con 0x7fa86015a6a0 2023-10-06T04:56:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.720+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86004c380 con 0x7fa86015a6a0 2023-10-06T04:56:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.720+0000 7fa8567fc700 1 -- 172.21.15.3:0/1461951023 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa84800c3e0 con 0x7fa86015a6a0 2023-10-06T04:56:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.721+0000 7fa8567fc700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa840066f30 0x7fa8400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.721+0000 7fa8567fc700 1 -- 172.21.15.3:0/1461951023 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa8480847e0 con 0x7fa86015a6a0 2023-10-06T04:56:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.721+0000 7fa85e59c700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa840066f30 0x7fa8400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.722+0000 7fa85e59c700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa840066f30 0x7fa8400693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa858007160 tx=0x7fa858007f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.725+0000 7fa8567fc700 1 -- 172.21.15.3:0/1461951023 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa848051850 con 0x7fa86015a6a0 2023-10-06T04:56:47.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.963+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa8600cdc20 con 0x7fa86015a6a0 2023-10-06T04:56:47.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.963+0000 7fa8567fc700 1 -- 172.21.15.3:0/1461951023 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fa848054e70 con 0x7fa86015a6a0 2023-10-06T04:56:47.966 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:47.966 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:47.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa840066f30 msgr2=0x7fa8400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa840066f30 0x7fa8400693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa858007160 tx=0x7fa858007f50 comp rx=0 tx=0).stop 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa86015a6a0 msgr2=0x7fa86006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa86015a6a0 0x7fa86006eb60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e930 tx=0x7fa8480091a0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 shutdown_connections 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa840066f30 0x7fa8400693e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa86015d660 0x7fa8600b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8601592c0 0x7fa860151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 --2- 172.21.15.3:0/1461951023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa86015a6a0 0x7fa86006eb60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 >> 172.21.15.3:0/1461951023 conn(0x7fa860089d90 msgr2=0x7fa8600c5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:47.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 shutdown_connections 2023-10-06T04:56:47.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:47.965+0000 7fa85f59e700 1 -- 172.21.15.3:0/1461951023 wait complete. 2023-10-06T04:56:47.979 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:47.979 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:52.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:53.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.112+0000 7fcce0314700 1 Processor -- start 2023-10-06T04:56:53.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.112+0000 7fcce0314700 1 -- start start 2023-10-06T04:56:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.113+0000 7fcce0314700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd815a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.113+0000 7fcce0314700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fccd81598d0 0x7fccd815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.113+0000 7fcce0314700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd80b3670 0x7fccd80b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.113+0000 7fcce0314700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd815d860 con 0x7fccd80b3670 2023-10-06T04:56:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.113+0000 7fcce0314700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd806de20 con 0x7fccd81598d0 2023-10-06T04:56:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.113+0000 7fcce0314700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd815da90 con 0x7fccd8159500 2023-10-06T04:56:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd80b3670 0x7fccd80b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd80b3670 0x7fccd80b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56098/0 (socket says 172.21.15.3:56098) 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 -- 172.21.15.3:0/1421554324 learned_addr learned my addr 172.21.15.3:0/1421554324 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccdd8af700 1 --2- 172.21.15.3:0/1421554324 >> v2:172.21.15.153:3300/0 conn(0x7fccd81598d0 0x7fccd815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde0b0700 1 --2- 172.21.15.3:0/1421554324 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd815a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 -- 172.21.15.3:0/1421554324 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 msgr2=0x7fccd815a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 --2- 172.21.15.3:0/1421554324 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd815a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 -- 172.21.15.3:0/1421554324 >> v2:172.21.15.153:3300/0 conn(0x7fccd81598d0 msgr2=0x7fccd815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 --2- 172.21.15.3:0/1421554324 >> v2:172.21.15.153:3300/0 conn(0x7fccd81598d0 0x7fccd815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 -- 172.21.15.3:0/1421554324 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd815dd70 con 0x7fccd80b3670 2023-10-06T04:56:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccde8b1700 1 --2- 172.21.15.3:0/1421554324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd80b3670 0x7fccd80b3ab0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fccd806b720 tx=0x7fccc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1b41739c631d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccdd0ae700 1 -- 172.21.15.3:0/1421554324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800e070 con 0x7fccd80b3670 2023-10-06T04:56:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccdd0ae700 1 -- 172.21.15.3:0/1421554324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc800b970 con 0x7fccd80b3670 2023-10-06T04:56:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.114+0000 7fccdd0ae700 1 -- 172.21.15.3:0/1421554324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc80092e0 con 0x7fccd80b3670 2023-10-06T04:56:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 -- 172.21.15.3:0/1421554324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd80b3670 msgr2=0x7fccd80b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 --2- 172.21.15.3:0/1421554324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd80b3670 0x7fccd80b3ab0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fccd806b720 tx=0x7fccc8009840 comp rx=0 tx=0).stop 2023-10-06T04:56:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 -- 172.21.15.3:0/1421554324 shutdown_connections 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 --2- 172.21.15.3:0/1421554324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd80b3670 0x7fccd80b3ab0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 --2- 172.21.15.3:0/1421554324 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd815a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 --2- 172.21.15.3:0/1421554324 >> v2:172.21.15.153:3300/0 conn(0x7fccd81598d0 0x7fccd815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 -- 172.21.15.3:0/1421554324 >> 172.21.15.3:0/1421554324 conn(0x7fccd8067aa0 msgr2=0x7fccd8067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 -- 172.21.15.3:0/1421554324 shutdown_connections 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.115+0000 7fcce0314700 1 -- 172.21.15.3:0/1421554324 wait complete. 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.116+0000 7fcce0314700 1 Processor -- start 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.116+0000 7fcce0314700 1 -- start start 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fcce0314700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd80b3670 0x7fccd8158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fcce0314700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd81528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fcce0314700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81598d0 0x7fccd813a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fcce0314700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd814e860 con 0x7fccd81598d0 2023-10-06T04:56:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fcce0314700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80adc70 con 0x7fccd80b3670 2023-10-06T04:56:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fcce0314700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd814f680 con 0x7fccd8159500 2023-10-06T04:56:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81598d0 0x7fccd813a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81598d0 0x7fccd813a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56112/0 (socket says 172.21.15.3:56112) 2023-10-06T04:56:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 -- 172.21.15.3:0/1224199699 learned_addr learned my addr 172.21.15.3:0/1224199699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccdd8af700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd81528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde0b0700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd80b3670 0x7fccd8158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 -- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 msgr2=0x7fccd81528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd81528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 -- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd80b3670 msgr2=0x7fccd8158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd80b3670 0x7fccd8158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 -- 172.21.15.3:0/1224199699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8009ae0 con 0x7fccd81598d0 2023-10-06T04:56:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.117+0000 7fccde8b1700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81598d0 0x7fccd813a3d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fccd80676a0 tx=0x7fccc8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.118+0000 7fccd67fc700 1 -- 172.21.15.3:0/1224199699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8008b80 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.118+0000 7fccd67fc700 1 -- 172.21.15.3:0/1224199699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc8008ce0 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.118+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd814ea70 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.118+0000 7fccd67fc700 1 -- 172.21.15.3:0/1224199699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc80266b0 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.118+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd814f030 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.119+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd80644c0 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.119+0000 7fccd67fc700 1 -- 172.21.15.3:0/1224199699 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fccc8003040 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.119+0000 7fccd67fc700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccc4066f30 0x7fccc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.120+0000 7fccd67fc700 1 -- 172.21.15.3:0/1224199699 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fccc8086560 con 0x7fccd81598d0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.120+0000 7fccde0b0700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccc4066f30 0x7fccc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.120+0000 7fccde0b0700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccc4066f30 0x7fccc40693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fccd80b0dc0 tx=0x7fccd0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.124+0000 7fccd67fc700 1 -- 172.21.15.3:0/1224199699 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccc8053720 con 0x7fccd81598d0 2023-10-06T04:56:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.362+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fccd814f7c0 con 0x7fccd81598d0 2023-10-06T04:56:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.363+0000 7fccd67fc700 1 -- 172.21.15.3:0/1224199699 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fccc8056d40 con 0x7fccd81598d0 2023-10-06T04:56:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:53.367 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:53.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.364+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccc4066f30 msgr2=0x7fccc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:53.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.364+0000 7fcce0314700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccc4066f30 0x7fccc40693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fccd80b0dc0 tx=0x7fccd0008040 comp rx=0 tx=0).stop 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81598d0 msgr2=0x7fccd813a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81598d0 0x7fccd813a3d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fccd80676a0 tx=0x7fccc8018000 comp rx=0 tx=0).stop 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 shutdown_connections 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fccc4066f30 0x7fccc40693e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd80b3670 0x7fccd8158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81598d0 0x7fccd813a3d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 --2- 172.21.15.3:0/1224199699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccd8159500 0x7fccd81528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 >> 172.21.15.3:0/1224199699 conn(0x7fccd8067aa0 msgr2=0x7fccd814d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 shutdown_connections 2023-10-06T04:56:53.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:53.365+0000 7fcce0314700 1 -- 172.21.15.3:0/1224199699 wait complete. 2023-10-06T04:56:53.379 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:53.380 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:56:55.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:55.077 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:58.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:56:58.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.512+0000 7f19eea20700 1 Processor -- start 2023-10-06T04:56:58.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.512+0000 7f19eea20700 1 -- start start 2023-10-06T04:56:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.512+0000 7f19eea20700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19e80b27b0 0x7f19e815a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.512+0000 7f19eea20700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e80cd320 0x7f19e80aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.512+0000 7f19eea20700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e815d6d0 con 0x7f19e815d240 2023-10-06T04:56:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.512+0000 7f19eea20700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e815aba0 con 0x7f19e80b27b0 2023-10-06T04:56:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.512+0000 7f19eea20700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e8150b30 con 0x7f19e80cd320 2023-10-06T04:56:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19e80b27b0 0x7f19e815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19e80b27b0 0x7f19e815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56124/0 (socket says 172.21.15.3:56124) 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 -- 172.21.15.3:0/1975210332 learned_addr learned my addr 172.21.15.3:0/1975210332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19ecfbd700 1 --2- 172.21.15.3:0/1975210332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e80cd320 0x7f19e80aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 -- 172.21.15.3:0/1975210332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e80cd320 msgr2=0x7f19e80aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 --2- 172.21.15.3:0/1975210332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e80cd320 0x7f19e80aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 -- 172.21.15.3:0/1975210332 >> v1:172.21.15.153:6789/0 conn(0x7f19e815d240 legacy=0x7f19e80ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 -- 172.21.15.3:0/1975210332 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e815d9b0 con 0x7f19e80b27b0 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e7fff700 1 --2- 172.21.15.3:0/1975210332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80b27b0 0x7f19e815a660 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5dd9b48388ab73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e6ffd700 1 -- 172.21.15.3:0/1975210332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800b860 con 0x7f19e80b27b0 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e6ffd700 1 -- 172.21.15.3:0/1975210332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d800b9c0 con 0x7f19e80b27b0 2023-10-06T04:56:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.513+0000 7f19e6ffd700 1 -- 172.21.15.3:0/1975210332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800bd60 con 0x7f19e80b27b0 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 -- 172.21.15.3:0/1975210332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80b27b0 msgr2=0x7f19e815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 --2- 172.21.15.3:0/1975210332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80b27b0 0x7f19e815a660 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8009250 comp rx=0 tx=0).stop 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 -- 172.21.15.3:0/1975210332 shutdown_connections 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 --2- 172.21.15.3:0/1975210332 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e80cd320 0x7f19e80aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 --2- 172.21.15.3:0/1975210332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80b27b0 0x7f19e815a660 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 -- 172.21.15.3:0/1975210332 >> 172.21.15.3:0/1975210332 conn(0x7f19e8070310 msgr2=0x7f19e8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 -- 172.21.15.3:0/1975210332 shutdown_connections 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.514+0000 7f19eea20700 1 -- 172.21.15.3:0/1975210332 wait complete. 2023-10-06T04:56:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.515+0000 7f19eea20700 1 Processor -- start 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.515+0000 7f19eea20700 1 -- start start 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19eea20700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19e80b27b0 0x7f19e81535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19eea20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80cd320 0x7f19e80ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19eea20700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e815d240 0x7f19e80ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19eea20700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80ca420 con 0x7f19e80cd320 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19eea20700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e8153120 con 0x7f19e80b27b0 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19eea20700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e8153af0 con 0x7f19e815d240 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80cd320 0x7f19e80ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80cd320 0x7f19e80ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56138/0 (socket says 172.21.15.3:56138) 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19e77fe700 1 -- 172.21.15.3:0/3726622956 learned_addr learned my addr 172.21.15.3:0/3726622956 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19e7fff700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19e80b27b0 0x7f19e81535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.516+0000 7f19ecfbd700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e815d240 0x7f19e80ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19e77fe700 1 -- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e815d240 msgr2=0x7f19e80ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19e77fe700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e815d240 0x7f19e80ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19e77fe700 1 -- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19e80b27b0 msgr2=0x7f19e81535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19e77fe700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19e80b27b0 0x7f19e81535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19e77fe700 1 -- 172.21.15.3:0/3726622956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80094f0 con 0x7f19e80cd320 2023-10-06T04:56:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19e77fe700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80cd320 0x7f19e80ce850 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00c4e0 tx=0x7f19dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19cffff700 1 -- 172.21.15.3:0/3726622956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011590 con 0x7f19e80cd320 2023-10-06T04:56:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19cffff700 1 -- 172.21.15.3:0/3726622956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc011710 con 0x7f19e80cd320 2023-10-06T04:56:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19cffff700 1 -- 172.21.15.3:0/3726622956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011ad0 con 0x7f19e80cd320 2023-10-06T04:56:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e80c9e60 con 0x7f19e80cd320 2023-10-06T04:56:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.517+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e814f950 con 0x7f19e80cd320 2023-10-06T04:56:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.518+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e8152ef0 con 0x7f19e80cd320 2023-10-06T04:56:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.518+0000 7f19cffff700 1 -- 172.21.15.3:0/3726622956 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19dc01b3b0 con 0x7f19e80cd320 2023-10-06T04:56:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.519+0000 7f19cffff700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f19d4066f30 0x7f19d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:56:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.519+0000 7f19cffff700 1 -- 172.21.15.3:0/3726622956 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f19dc083a50 con 0x7f19e80cd320 2023-10-06T04:56:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.519+0000 7f19e7fff700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f19d4066f30 0x7f19d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:56:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.520+0000 7f19e7fff700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f19d4066f30 0x7f19d40693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:56:58.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.524+0000 7f19cffff700 1 -- 172.21.15.3:0/3726622956 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19dc050b60 con 0x7f19e80cd320 2023-10-06T04:56:58.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.768+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f19e8153290 con 0x7f19e80cd320 2023-10-06T04:56:58.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.768+0000 7f19cffff700 1 -- 172.21.15.3:0/3726622956 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f19dc017020 con 0x7f19e80cd320 2023-10-06T04:56:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:56:58.770 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:56:58.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f19d4066f30 msgr2=0x7f19d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:58.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f19d4066f30 0x7f19d40693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8017000 comp rx=0 tx=0).stop 2023-10-06T04:56:58.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80cd320 msgr2=0x7f19e80ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:56:58.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80cd320 0x7f19e80ce850 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00c4e0 tx=0x7f19dc00c7f0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 shutdown_connections 2023-10-06T04:56:58.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f19d4066f30 0x7f19d40693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19e80b27b0 0x7f19e81535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19e815d240 0x7f19e80ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 --2- 172.21.15.3:0/3726622956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e80cd320 0x7f19e80ce850 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:56:58.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 >> 172.21.15.3:0/3726622956 conn(0x7f19e8070310 msgr2=0x7f19e80c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:56:58.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 shutdown_connections 2023-10-06T04:56:58.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:56:58.770+0000 7f19eea20700 1 -- 172.21.15.3:0/3726622956 wait complete. 2023-10-06T04:56:58.784 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:56:58.784 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:03.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:03.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.915+0000 7f5dcadec700 1 Processor -- start 2023-10-06T04:57:03.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.915+0000 7f5dcadec700 1 -- start start 2023-10-06T04:57:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.916+0000 7f5dcadec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5dc4142a50 0x7f5dc40b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.916+0000 7f5dcadec700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc415ad00 con 0x7f5dc406ca80 2023-10-06T04:57:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.916+0000 7f5dcadec700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc415a0f0 con 0x7f5dc4159b50 2023-10-06T04:57:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.916+0000 7f5dcadec700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc415a3d0 con 0x7f5dc4142a50 2023-10-06T04:57:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.916+0000 7f5dc9389700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5dc406ca80 0x7f5dc415b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45662/0 (socket says 172.21.15.3:45662) 2023-10-06T04:57:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.916+0000 7f5dc8b88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5dc4142a50 0x7f5dc40b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.916+0000 7f5dc9389700 1 -- 172.21.15.3:0/1442443529 learned_addr learned my addr 172.21.15.3:0/1442443529 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:57:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc8b88700 1 -- 172.21.15.3:0/1442443529 >> v1:172.21.15.153:6789/0 conn(0x7f5dc4159b50 legacy=0x7f5dc406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc8b88700 1 -- 172.21.15.3:0/1442443529 >> v1:172.21.15.3:6789/0 conn(0x7f5dc406ca80 legacy=0x7f5dc415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc8b88700 1 -- 172.21.15.3:0/1442443529 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc415a6b0 con 0x7f5dc4142a50 2023-10-06T04:57:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc37fe700 1 -- 172.21.15.3:0/1442443529 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091787324 0 0) 0x7f5dc415ad00 con 0x7f5dc406ca80 2023-10-06T04:57:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc8b88700 1 --2- 172.21.15.3:0/1442443529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4142a50 0x7f5dc40b0820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e60 tx=0x7f5dbc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76134146fbf9f4e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc37fe700 1 -- 172.21.15.3:0/1442443529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00e070 con 0x7f5dc4142a50 2023-10-06T04:57:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc37fe700 1 -- 172.21.15.3:0/1442443529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00bc30 con 0x7f5dc4142a50 2023-10-06T04:57:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.917+0000 7f5dc37fe700 1 -- 172.21.15.3:0/1442443529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc012430 con 0x7f5dc4142a50 2023-10-06T04:57:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.918+0000 7f5dcadec700 1 -- 172.21.15.3:0/1442443529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4142a50 msgr2=0x7f5dc40b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.918+0000 7f5dcadec700 1 --2- 172.21.15.3:0/1442443529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4142a50 0x7f5dc40b0820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e60 tx=0x7f5dbc009780 comp rx=0 tx=0).stop 2023-10-06T04:57:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.918+0000 7f5dcadec700 1 -- 172.21.15.3:0/1442443529 shutdown_connections 2023-10-06T04:57:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.918+0000 7f5dcadec700 1 --2- 172.21.15.3:0/1442443529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4142a50 0x7f5dc40b0820 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.918+0000 7f5dcadec700 1 -- 172.21.15.3:0/1442443529 >> 172.21.15.3:0/1442443529 conn(0x7f5dc40cd190 msgr2=0x7f5dc40cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.918+0000 7f5dcadec700 1 -- 172.21.15.3:0/1442443529 shutdown_connections 2023-10-06T04:57:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.918+0000 7f5dcadec700 1 -- 172.21.15.3:0/1442443529 wait complete. 2023-10-06T04:57:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.919+0000 7f5dcadec700 1 Processor -- start 2023-10-06T04:57:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.919+0000 7f5dcadec700 1 -- start start 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dcadec700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5dc406ca80 0x7f5dc4140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dcadec700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc4142a50 0x7f5dc40ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dcadec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4159b50 0x7f5dc40ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dcadec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41410a0 con 0x7f5dc4159b50 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dcadec700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40ad4b0 con 0x7f5dc4142a50 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dcadec700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40ab8d0 con 0x7f5dc406ca80 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4159b50 0x7f5dc40ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4159b50 0x7f5dc40ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53616/0 (socket says 172.21.15.3:53616) 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 -- 172.21.15.3:0/2456948760 learned_addr learned my addr 172.21.15.3:0/2456948760 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc8b88700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5dc406ca80 0x7f5dc4140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc3fff700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc4142a50 0x7f5dc40ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 -- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5dc406ca80 msgr2=0x7f5dc4140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5dc406ca80 0x7f5dc4140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 -- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc4142a50 msgr2=0x7f5dc40ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc4142a50 0x7f5dc40ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc9389700 1 -- 172.21.15.3:0/2456948760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009a20 con 0x7f5dc4159b50 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc8b88700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5dc406ca80 0x7f5dc4140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.920+0000 7f5dc3fff700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc4142a50 0x7f5dc40ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.921+0000 7f5dc9389700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4159b50 0x7f5dc40ae9e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c590 tx=0x7f5dac007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.921+0000 7f5dc0ff9700 1 -- 172.21.15.3:0/2456948760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac005b30 con 0x7f5dc4159b50 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.921+0000 7f5dc0ff9700 1 -- 172.21.15.3:0/2456948760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac005cb0 con 0x7f5dc4159b50 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.921+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc40ac8a0 con 0x7f5dc4159b50 2023-10-06T04:57:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.921+0000 7f5dc0ff9700 1 -- 172.21.15.3:0/2456948760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac007320 con 0x7f5dc4159b50 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.921+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc40ace60 con 0x7f5dc4159b50 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.922+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc40cab60 con 0x7f5dc4159b50 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.922+0000 7f5dc0ff9700 1 -- 172.21.15.3:0/2456948760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dac0076d0 con 0x7f5dc4159b50 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.923+0000 7f5dc0ff9700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5da4066f30 0x7f5da40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.923+0000 7f5dc0ff9700 1 -- 172.21.15.3:0/2456948760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5dac083bb0 con 0x7f5dc4159b50 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.923+0000 7f5dc8b88700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5da4066f30 0x7f5da40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.923+0000 7f5dc8b88700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5da4066f30 0x7f5da40693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e60 tx=0x7f5dbc019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:03.927+0000 7f5dc0ff9700 1 -- 172.21.15.3:0/2456948760 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dc40cab60 con 0x7f5dc4159b50 2023-10-06T04:57:04.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.163+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5dc40cab60 con 0x7f5dc4159b50 2023-10-06T04:57:04.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.163+0000 7f5dc0ff9700 1 -- 172.21.15.3:0/2456948760 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f5dc40cab60 con 0x7f5dc4159b50 2023-10-06T04:57:04.165 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:04.165 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.165+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5da4066f30 msgr2=0x7f5da40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.165+0000 7f5dcadec700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5da4066f30 0x7f5da40693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e60 tx=0x7f5dbc019000 comp rx=0 tx=0).stop 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.165+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4159b50 msgr2=0x7f5dc40ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.165+0000 7f5dcadec700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4159b50 0x7f5dc40ae9e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c590 tx=0x7f5dac007cd0 comp rx=0 tx=0).stop 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.165+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 shutdown_connections 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.165+0000 7f5dcadec700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5da4066f30 0x7f5da40693e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.165+0000 7f5dcadec700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc4159b50 0x7f5dc40ae9e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.166+0000 7f5dcadec700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5dc406ca80 0x7f5dc4140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.166+0000 7f5dcadec700 1 --2- 172.21.15.3:0/2456948760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc4142a50 0x7f5dc40ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.166+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 >> 172.21.15.3:0/2456948760 conn(0x7f5dc40cd190 msgr2=0x7f5dc415b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.166+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 shutdown_connections 2023-10-06T04:57:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:04.166+0000 7f5dcadec700 1 -- 172.21.15.3:0/2456948760 wait complete. 2023-10-06T04:57:04.179 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:04.179 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:09.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.310+0000 7f3b7a0d1700 1 Processor -- start 2023-10-06T04:57:09.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.310+0000 7f3b7a0d1700 1 -- start start 2023-10-06T04:57:09.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b7a0d1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b74154e20 0x7f3b740b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b7a0d1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3b741551f0 0x7f3b74069390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b7a0d1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b740b40d0 con 0x7f3b741587e0 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b7a0d1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b740b1180 con 0x7f3b741551f0 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b7a0d1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b740b1460 con 0x7f3b74154e20 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b6bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3b741551f0 0x7f3b74069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b737fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b74154e20 0x7f3b740b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b6bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3b741551f0 0x7f3b74069390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:34998/0 (socket says 172.21.15.3:34998) 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b73fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3b741587e0 0x7f3b7415b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:37510/0 (socket says 172.21.15.3:37510) 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b737fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b74154e20 0x7f3b740b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:52212/0 (socket says 172.21.15.3:52212) 2023-10-06T04:57:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b6bfff700 1 -- 172.21.15.3:0/2145276949 learned_addr learned my addr 172.21.15.3:0/2145276949 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.311+0000 7f3b72ffd700 1 -- 172.21.15.3:0/2145276949 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516758339 0 0) 0x7f3b740b40d0 con 0x7f3b741587e0 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b737fe700 1 -- 172.21.15.3:0/2145276949 >> v2:172.21.15.153:3300/0 conn(0x7f3b741551f0 msgr2=0x7f3b74069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b737fe700 1 --2- 172.21.15.3:0/2145276949 >> v2:172.21.15.153:3300/0 conn(0x7f3b741551f0 0x7f3b74069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b737fe700 1 -- 172.21.15.3:0/2145276949 >> v1:172.21.15.153:6789/0 conn(0x7f3b741587e0 legacy=0x7f3b7415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b737fe700 1 -- 172.21.15.3:0/2145276949 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b740b1740 con 0x7f3b74154e20 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/2145276949 >> v2:172.21.15.153:3300/0 conn(0x7f3b741551f0 0x7f3b74069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b737fe700 1 --2- 172.21.15.3:0/2145276949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b74154e20 0x7f3b740b3780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098e0 tx=0x7f3b640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5564e781557976 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b72ffd700 1 -- 172.21.15.3:0/2145276949 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400b640 con 0x7f3b74154e20 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b72ffd700 1 -- 172.21.15.3:0/2145276949 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6400b7a0 con 0x7f3b74154e20 2023-10-06T04:57:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.312+0000 7f3b72ffd700 1 -- 172.21.15.3:0/2145276949 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400bb40 con 0x7f3b74154e20 2023-10-06T04:57:09.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/2145276949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b74154e20 msgr2=0x7f3b740b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:09.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/2145276949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b74154e20 0x7f3b740b3780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098e0 tx=0x7f3b640091a0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/2145276949 shutdown_connections 2023-10-06T04:57:09.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/2145276949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b74154e20 0x7f3b740b3780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/2145276949 >> v2:172.21.15.153:3300/0 conn(0x7f3b741551f0 0x7f3b74069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/2145276949 >> 172.21.15.3:0/2145276949 conn(0x7f3b74067aa0 msgr2=0x7f3b74067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/2145276949 shutdown_connections 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.313+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/2145276949 wait complete. 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.314+0000 7f3b7a0d1700 1 Processor -- start 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.314+0000 7f3b7a0d1700 1 -- start start 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.314+0000 7f3b7a0d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b74154e20 0x7f3b740ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.314+0000 7f3b7a0d1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b741551f0 0x7f3b74140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.314+0000 7f3b7a0d1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b741587e0 0x7f3b7414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b7a0d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7414faf0 con 0x7f3b74154e20 2023-10-06T04:57:09.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b7a0d1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74141240 con 0x7f3b741587e0 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b7a0d1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b740ce880 con 0x7f3b741551f0 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b74154e20 0x7f3b740ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b74154e20 0x7f3b740ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53632/0 (socket says 172.21.15.3:53632) 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 -- 172.21.15.3:0/3582536074 learned_addr learned my addr 172.21.15.3:0/3582536074 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b741551f0 0x7f3b74140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b73fff700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b741587e0 0x7f3b7414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 -- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b741551f0 msgr2=0x7f3b74140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b741551f0 0x7f3b74140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 -- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b741587e0 msgr2=0x7f3b7414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b741587e0 0x7f3b7414ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 -- 172.21.15.3:0/3582536074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0094f0 con 0x7f3b74154e20 2023-10-06T04:57:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b737fe700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b74154e20 0x7f3b740ce1c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b15a0 tx=0x7f3b64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/3582536074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400bb20 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/3582536074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64011420 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/3582536074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b640117a0 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b64009420 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.315+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7414f4a0 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.316+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7414fdd0 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.317+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/3582536074 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b64011b10 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.317+0000 7f3b6b7fe700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b54066f30 0x7f3b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.317+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/3582536074 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b64085a50 con 0x7f3b74154e20 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.317+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b54066f30 0x7f3b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.318+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b54066f30 0x7f3b540693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b7406e7e0 tx=0x7f3b5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:09.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.322+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/3582536074 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b64052be0 con 0x7f3b74154e20 2023-10-06T04:57:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.559+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3b740644c0 con 0x7f3b74154e20 2023-10-06T04:57:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.559+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/3582536074 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f3b64056200 con 0x7f3b74154e20 2023-10-06T04:57:09.561 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b54066f30 msgr2=0x7f3b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b54066f30 0x7f3b540693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b7406e7e0 tx=0x7f3b5c008040 comp rx=0 tx=0).stop 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b74154e20 msgr2=0x7f3b740ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b74154e20 0x7f3b740ce1c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b15a0 tx=0x7f3b64017000 comp rx=0 tx=0).stop 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 shutdown_connections 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b54066f30 0x7f3b540693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b741587e0 0x7f3b7414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b74154e20 0x7f3b740ce1c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 --2- 172.21.15.3:0/3582536074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b741551f0 0x7f3b74140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 >> 172.21.15.3:0/3582536074 conn(0x7f3b74067aa0 msgr2=0x7f3b741527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 shutdown_connections 2023-10-06T04:57:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:09.561+0000 7f3b7a0d1700 1 -- 172.21.15.3:0/3582536074 wait complete. 2023-10-06T04:57:09.574 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:09.574 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:14.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.710+0000 7faff3bcd700 1 Processor -- start 2023-10-06T04:57:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.710+0000 7faff3bcd700 1 -- start start 2023-10-06T04:57:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff3bcd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec15d240 0x7fafec15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff3bcd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fafec0cd320 0x7fafec0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff3bcd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec15d6d0 con 0x7fafec0b27b0 2023-10-06T04:57:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff3bcd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafec15aba0 con 0x7fafec0cd320 2023-10-06T04:57:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff3bcd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec150b30 con 0x7fafec15d240 2023-10-06T04:57:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff216a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fafec0cd320 0x7fafec0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff216a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fafec0cd320 0x7fafec0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38762/0 (socket says 172.21.15.3:38762) 2023-10-06T04:57:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff216a700 1 -- 172.21.15.3:0/3831072568 learned_addr learned my addr 172.21.15.3:0/3831072568 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff1168700 1 --2- 172.21.15.3:0/3831072568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec15d240 0x7fafec15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff216a700 1 -- 172.21.15.3:0/3831072568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec15d240 msgr2=0x7fafec15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff216a700 1 --2- 172.21.15.3:0/3831072568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec15d240 0x7fafec15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff216a700 1 -- 172.21.15.3:0/3831072568 >> v1:172.21.15.153:6789/0 conn(0x7fafec0b27b0 legacy=0x7fafec0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.711+0000 7faff216a700 1 -- 172.21.15.3:0/3831072568 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec15d9b0 con 0x7fafec0cd320 2023-10-06T04:57:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff216a700 1 --2- 172.21.15.3:0/3831072568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0aeef0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009f20 tx=0x7fafe4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f788801de70832b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff0967700 1 -- 172.21.15.3:0/3831072568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400e070 con 0x7fafec0cd320 2023-10-06T04:57:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff0967700 1 -- 172.21.15.3:0/3831072568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe400b9d0 con 0x7fafec0cd320 2023-10-06T04:57:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff0967700 1 -- 172.21.15.3:0/3831072568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe40092e0 con 0x7fafec0cd320 2023-10-06T04:57:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff3bcd700 1 -- 172.21.15.3:0/3831072568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 msgr2=0x7fafec0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff3bcd700 1 --2- 172.21.15.3:0/3831072568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0aeef0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009f20 tx=0x7fafe4009840 comp rx=0 tx=0).stop 2023-10-06T04:57:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff3bcd700 1 -- 172.21.15.3:0/3831072568 shutdown_connections 2023-10-06T04:57:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff3bcd700 1 --2- 172.21.15.3:0/3831072568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0aeef0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff3bcd700 1 --2- 172.21.15.3:0/3831072568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec15d240 0x7fafec15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.712+0000 7faff3bcd700 1 -- 172.21.15.3:0/3831072568 >> 172.21.15.3:0/3831072568 conn(0x7fafec070310 msgr2=0x7fafec070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.713+0000 7faff3bcd700 1 -- 172.21.15.3:0/3831072568 shutdown_connections 2023-10-06T04:57:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.713+0000 7faff3bcd700 1 -- 172.21.15.3:0/3831072568 wait complete. 2023-10-06T04:57:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 Processor -- start 2023-10-06T04:57:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 -- start start 2023-10-06T04:57:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec0b27b0 0x7fafec14ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec15d240 0x7fafec0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec0c8820 con 0x7fafec0cd320 2023-10-06T04:57:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec0c9440 con 0x7fafec15d240 2023-10-06T04:57:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff3bcd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec0c8260 con 0x7fafec0b27b0 2023-10-06T04:57:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff1168700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff1168700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38766/0 (socket says 172.21.15.3:38766) 2023-10-06T04:57:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.714+0000 7faff1168700 1 -- 172.21.15.3:0/2116770260 learned_addr learned my addr 172.21.15.3:0/2116770260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff1969700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec0b27b0 0x7fafec14ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff216a700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec15d240 0x7fafec0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff1168700 1 -- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec0b27b0 msgr2=0x7fafec14ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff1168700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec0b27b0 0x7fafec14ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff1168700 1 -- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec15d240 msgr2=0x7fafec0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff1168700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec15d240 0x7fafec0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff1168700 1 -- 172.21.15.3:0/2116770260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe4009ae0 con 0x7fafec0cd320 2023-10-06T04:57:14.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff216a700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec15d240 0x7fafec0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:57:14.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff1168700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0b4660 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e930 tx=0x7fafe00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:14.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7fafddffb700 1 -- 172.21.15.3:0/2116770260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0013680 con 0x7fafec0cd320 2023-10-06T04:57:14.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7fafddffb700 1 -- 172.21.15.3:0/2116770260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe00137e0 con 0x7fafec0cd320 2023-10-06T04:57:14.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7fafddffb700 1 -- 172.21.15.3:0/2116770260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0013b80 con 0x7fafec0cd320 2023-10-06T04:57:14.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec0c8540 con 0x7fafec0cd320 2023-10-06T04:57:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.715+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec0b4ba0 con 0x7fafec0cd320 2023-10-06T04:57:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.716+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec0c95b0 con 0x7fafec0cd320 2023-10-06T04:57:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.717+0000 7fafddffb700 1 -- 172.21.15.3:0/2116770260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafe000c460 con 0x7fafec0cd320 2023-10-06T04:57:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.717+0000 7fafddffb700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fafd0066f30 0x7fafd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.717+0000 7fafddffb700 1 -- 172.21.15.3:0/2116770260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fafe0084800 con 0x7fafec0cd320 2023-10-06T04:57:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.717+0000 7faff1969700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fafd0066f30 0x7fafd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:14.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.718+0000 7faff1969700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fafd0066f30 0x7fafd00693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fafec15f900 tx=0x7fafe8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.722+0000 7fafddffb700 1 -- 172.21.15.3:0/2116770260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafe0051910 con 0x7fafec0cd320 2023-10-06T04:57:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.959+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fafec071680 con 0x7fafec0cd320 2023-10-06T04:57:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.959+0000 7fafddffb700 1 -- 172.21.15.3:0/2116770260 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fafe0054f30 con 0x7fafec0cd320 2023-10-06T04:57:14.962 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:14.962 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fafd0066f30 msgr2=0x7fafd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fafd0066f30 0x7fafd00693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fafec15f900 tx=0x7fafe8009040 comp rx=0 tx=0).stop 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 msgr2=0x7fafec0b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0b4660 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e930 tx=0x7fafe00091a0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 shutdown_connections 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fafd0066f30 0x7fafd00693e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec15d240 0x7fafec0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fafec0b27b0 0x7fafec14ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 --2- 172.21.15.3:0/2116770260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafec0cd320 0x7fafec0b4660 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.961+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 >> 172.21.15.3:0/2116770260 conn(0x7fafec070310 msgr2=0x7fafec0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:14.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.962+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 shutdown_connections 2023-10-06T04:57:14.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:14.962+0000 7faff3bcd700 1 -- 172.21.15.3:0/2116770260 wait complete. 2023-10-06T04:57:14.975 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:14.976 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:19.977 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.106+0000 7f5b3f766700 1 Processor -- start 2023-10-06T04:57:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.106+0000 7f5b3f766700 1 -- start start 2023-10-06T04:57:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.107+0000 7f5b3f766700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5b380b39b0 0x7f5b3815d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.107+0000 7f5b3f766700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b3815aa30 0x7f5b38159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.107+0000 7f5b3f766700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3815aee0 con 0x7f5b380b3d80 2023-10-06T04:57:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.107+0000 7f5b3f766700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3815dc10 con 0x7f5b380b39b0 2023-10-06T04:57:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.107+0000 7f5b3f766700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3815def0 con 0x7f5b3815aa30 2023-10-06T04:57:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3d502700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5b380b39b0 0x7f5b3815d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3cd01700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b3815aa30 0x7f5b38159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3d502700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5b380b39b0 0x7f5b3815d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:45816/0 (socket says 172.21.15.3:45816) 2023-10-06T04:57:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3dd03700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5b380b3d80 0x7f5b38159650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:49666/0 (socket says 172.21.15.3:49666) 2023-10-06T04:57:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3d502700 1 -- 172.21.15.3:0/3152766383 learned_addr learned my addr 172.21.15.3:0/3152766383 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3cd01700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b3815aa30 0x7f5b38159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:60208/0 (socket says 172.21.15.3:60208) 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3cd01700 1 -- 172.21.15.3:0/3152766383 >> v2:172.21.15.153:3300/0 conn(0x7f5b380b39b0 msgr2=0x7f5b3815d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3cd01700 1 --2- 172.21.15.3:0/3152766383 >> v2:172.21.15.153:3300/0 conn(0x7f5b380b39b0 0x7f5b3815d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3cd01700 1 -- 172.21.15.3:0/3152766383 >> v1:172.21.15.153:6789/0 conn(0x7f5b380b3d80 legacy=0x7f5b38159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3cd01700 1 -- 172.21.15.3:0/3152766383 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3815e1d0 con 0x7f5b3815aa30 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3d502700 1 --2- 172.21.15.3:0/3152766383 >> v2:172.21.15.153:3300/0 conn(0x7f5b380b39b0 0x7f5b3815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b3cd01700 1 --2- 172.21.15.3:0/3152766383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b3815aa30 0x7f5b38159110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0098e0 tx=0x7f5b2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5cf766877931f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.108+0000 7f5b37fff700 1 -- 172.21.15.3:0/3152766383 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00b640 con 0x7f5b3815aa30 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b37fff700 1 -- 172.21.15.3:0/3152766383 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c00b7a0 con 0x7f5b3815aa30 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b37fff700 1 -- 172.21.15.3:0/3152766383 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00bb40 con 0x7f5b3815aa30 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 -- 172.21.15.3:0/3152766383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b3815aa30 msgr2=0x7f5b38159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 --2- 172.21.15.3:0/3152766383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b3815aa30 0x7f5b38159110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0098e0 tx=0x7f5b2c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 -- 172.21.15.3:0/3152766383 shutdown_connections 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 --2- 172.21.15.3:0/3152766383 >> v2:172.21.15.153:3300/0 conn(0x7f5b380b39b0 0x7f5b3815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 --2- 172.21.15.3:0/3152766383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b3815aa30 0x7f5b38159110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 -- 172.21.15.3:0/3152766383 >> 172.21.15.3:0/3152766383 conn(0x7f5b38067ac0 msgr2=0x7f5b38067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 -- 172.21.15.3:0/3152766383 shutdown_connections 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.109+0000 7f5b3f766700 1 -- 172.21.15.3:0/3152766383 wait complete. 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.110+0000 7f5b3f766700 1 Processor -- start 2023-10-06T04:57:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.110+0000 7f5b3f766700 1 -- start start 2023-10-06T04:57:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3f766700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b380b39b0 0x7f5b380afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3f766700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b380b3d80 0x7f5b3816a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3f766700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b3815aa30 0x7f5b3806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3f766700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3816aaf0 con 0x7f5b380b39b0 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3f766700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b380b00b0 con 0x7f5b3815aa30 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3f766700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b380b0390 con 0x7f5b380b3d80 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3d502700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b380b39b0 0x7f5b380afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3d502700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b380b39b0 0x7f5b380afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38778/0 (socket says 172.21.15.3:38778) 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3d502700 1 -- 172.21.15.3:0/212069588 learned_addr learned my addr 172.21.15.3:0/212069588 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3dd03700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b3815aa30 0x7f5b3806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.111+0000 7f5b3cd01700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b380b3d80 0x7f5b3816a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3d502700 1 -- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b380b3d80 msgr2=0x7f5b3816a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3d502700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b380b3d80 0x7f5b3816a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3d502700 1 -- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b3815aa30 msgr2=0x7f5b3806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3d502700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b3815aa30 0x7f5b3806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3d502700 1 -- 172.21.15.3:0/212069588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300094f0 con 0x7f5b380b39b0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3d502700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b380b39b0 0x7f5b380afad0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000ec50 tx=0x7f5b30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b357fa700 1 -- 172.21.15.3:0/212069588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000c850 con 0x7f5b380b39b0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b357fa700 1 -- 172.21.15.3:0/212069588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3000c9b0 con 0x7f5b380b39b0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c009420 con 0x7f5b380b39b0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b357fa700 1 -- 172.21.15.3:0/212069588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000cd50 con 0x7f5b380b39b0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.112+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b380b0950 con 0x7f5b380b39b0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.113+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3814fb90 con 0x7f5b380b39b0 2023-10-06T04:57:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.113+0000 7f5b357fa700 1 -- 172.21.15.3:0/212069588 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b30013680 con 0x7f5b380b39b0 2023-10-06T04:57:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.114+0000 7f5b357fa700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5b24066f30 0x7f5b240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.114+0000 7f5b357fa700 1 -- 172.21.15.3:0/212069588 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5b30084790 con 0x7f5b380b39b0 2023-10-06T04:57:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.114+0000 7f5b3cd01700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5b24066f30 0x7f5b240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.115+0000 7f5b3cd01700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5b24066f30 0x7f5b240693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0093f0 tx=0x7f5b2c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:20.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.118+0000 7f5b357fa700 1 -- 172.21.15.3:0/212069588 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b300518a0 con 0x7f5b380b39b0 2023-10-06T04:57:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.352+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5b380644c0 con 0x7f5b380b39b0 2023-10-06T04:57:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.353+0000 7f5b357fa700 1 -- 172.21.15.3:0/212069588 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f5b30054ec0 con 0x7f5b380b39b0 2023-10-06T04:57:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:20.355 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5b24066f30 msgr2=0x7f5b240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5b24066f30 0x7f5b240693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0093f0 tx=0x7f5b2c017000 comp rx=0 tx=0).stop 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b380b39b0 msgr2=0x7f5b380afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b380b39b0 0x7f5b380afad0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000ec50 tx=0x7f5b30009250 comp rx=0 tx=0).stop 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 shutdown_connections 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5b24066f30 0x7f5b240693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b3815aa30 0x7f5b3806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b380b39b0 0x7f5b380afad0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 --2- 172.21.15.3:0/212069588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b380b3d80 0x7f5b3816a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 >> 172.21.15.3:0/212069588 conn(0x7f5b38067ac0 msgr2=0x7f5b38159910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 shutdown_connections 2023-10-06T04:57:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:20.355+0000 7f5b3f766700 1 -- 172.21.15.3:0/212069588 wait complete. 2023-10-06T04:57:20.368 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:20.368 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:25.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:25.154 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:25.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:25.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.499+0000 7f2fb8014700 1 Processor -- start 2023-10-06T04:57:25.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.499+0000 7f2fb8014700 1 -- start start 2023-10-06T04:57:25.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb8014700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2fb015c7b0 0x7f2fb015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb8014700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb00afed0 0x7f2fb00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb8014700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb00cf2a0 con 0x7f2fb015dd00 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb8014700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb00cc7d0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb8014700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00ce970 con 0x7f2fb00afed0 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb65b1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb00afed0 0x7f2fb00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb5db0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2fb015c7b0 0x7f2fb015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb65b1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb00afed0 0x7f2fb00cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:44002/0 (socket says 172.21.15.3:44002) 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb65b1700 1 -- 172.21.15.3:0/2018045832 learned_addr learned my addr 172.21.15.3:0/2018045832 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb5db0700 1 --2- 172.21.15.3:0/2018045832 >> v2:172.21.15.153:3300/0 conn(0x7f2fb015c7b0 0x7f2fb015cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:34908/0 (socket says 172.21.15.3:34908) 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.500+0000 7f2fb55af700 1 --1- 172.21.15.3:0/2018045832 >> v1:172.21.15.153:6789/0 conn(0x7f2fb015dd00 0x7f2fb00cb390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:40296/0 (socket says 172.21.15.3:40296) 2023-10-06T04:57:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb65b1700 1 -- 172.21.15.3:0/2018045832 >> v2:172.21.15.153:3300/0 conn(0x7f2fb015c7b0 msgr2=0x7f2fb015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb65b1700 1 --2- 172.21.15.3:0/2018045832 >> v2:172.21.15.153:3300/0 conn(0x7f2fb015c7b0 0x7f2fb015cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb65b1700 1 -- 172.21.15.3:0/2018045832 >> v1:172.21.15.153:6789/0 conn(0x7f2fb015dd00 legacy=0x7f2fb00cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb65b1700 1 -- 172.21.15.3:0/2018045832 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00cec50 con 0x7f2fb00afed0 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb5db0700 1 --2- 172.21.15.3:0/2018045832 >> v2:172.21.15.153:3300/0 conn(0x7f2fb015c7b0 0x7f2fb015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb65b1700 1 --2- 172.21.15.3:0/2018045832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb00afed0 0x7f2fb00cce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009f20 tx=0x7f2fa8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb0c4029d69a7cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb4dae700 1 -- 172.21.15.3:0/2018045832 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800e070 con 0x7f2fb00afed0 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb4dae700 1 -- 172.21.15.3:0/2018045832 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa800ba60 con 0x7f2fb00afed0 2023-10-06T04:57:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.501+0000 7f2fb4dae700 1 -- 172.21.15.3:0/2018045832 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80092e0 con 0x7f2fb00afed0 2023-10-06T04:57:25.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 -- 172.21.15.3:0/2018045832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb00afed0 msgr2=0x7f2fb00cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:25.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 --2- 172.21.15.3:0/2018045832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb00afed0 0x7f2fb00cce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009f20 tx=0x7f2fa8009840 comp rx=0 tx=0).stop 2023-10-06T04:57:25.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 -- 172.21.15.3:0/2018045832 shutdown_connections 2023-10-06T04:57:25.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 --2- 172.21.15.3:0/2018045832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb00afed0 0x7f2fb00cce60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 --2- 172.21.15.3:0/2018045832 >> v2:172.21.15.153:3300/0 conn(0x7f2fb015c7b0 0x7f2fb015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 -- 172.21.15.3:0/2018045832 >> 172.21.15.3:0/2018045832 conn(0x7f2fb0070310 msgr2=0x7f2fb0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:25.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 -- 172.21.15.3:0/2018045832 shutdown_connections 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.502+0000 7f2fb8014700 1 -- 172.21.15.3:0/2018045832 wait complete. 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.503+0000 7f2fb8014700 1 Processor -- start 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.503+0000 7f2fb8014700 1 -- start start 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb8014700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fb00afed0 0x7f2fb00ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb8014700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb015c7b0 0x7f2fb00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb8014700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb015dd00 0x7f2fb00b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb8014700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00aecd0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb8014700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00b4830 con 0x7f2fb00afed0 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb8014700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00b3c20 con 0x7f2fb015dd00 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb015c7b0 0x7f2fb00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb015c7b0 0x7f2fb00b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58246/0 (socket says 172.21.15.3:58246) 2023-10-06T04:57:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 -- 172.21.15.3:0/1200597898 learned_addr learned my addr 172.21.15.3:0/1200597898 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb65b1700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb015dd00 0x7f2fb00b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb5db0700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fb00afed0 0x7f2fb00ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 -- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb015dd00 msgr2=0x7f2fb00b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb015dd00 0x7f2fb00b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 -- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fb00afed0 msgr2=0x7f2fb00ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fb00afed0 0x7f2fb00ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 -- 172.21.15.3:0/1200597898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009420 con 0x7f2fb015c7b0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fb55af700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb015c7b0 0x7f2fb00b3240 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c7c0 tx=0x7f2fa0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.504+0000 7f2fa5ffb700 1 -- 172.21.15.3:0/1200597898 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0005a70 con 0x7f2fb015c7b0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.505+0000 7f2fa5ffb700 1 -- 172.21.15.3:0/1200597898 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa0005bf0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.505+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa8009ae0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.505+0000 7f2fa5ffb700 1 -- 172.21.15.3:0/1200597898 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00070f0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.505+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb00b41e0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.506+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb00b49c0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.506+0000 7f2fa5ffb700 1 -- 172.21.15.3:0/1200597898 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fa0007460 con 0x7f2fb015c7b0 2023-10-06T04:57:25.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.507+0000 7f2fa5ffb700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f94066f30 0x7f2f940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.507+0000 7f2fa5ffb700 1 -- 172.21.15.3:0/1200597898 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2fa00840c0 con 0x7f2fb015c7b0 2023-10-06T04:57:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.507+0000 7f2fb5db0700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f94066f30 0x7f2f940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.507+0000 7f2fb5db0700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f94066f30 0x7f2f940693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0093f0 tx=0x7f2fac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.511+0000 7f2fa5ffb700 1 -- 172.21.15.3:0/1200597898 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fa0051250 con 0x7f2fb015c7b0 2023-10-06T04:57:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.747+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2fb00cef30 con 0x7f2fb015c7b0 2023-10-06T04:57:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.747+0000 7f2fa5ffb700 1 -- 172.21.15.3:0/1200597898 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f2fa0054870 con 0x7f2fb015c7b0 2023-10-06T04:57:25.749 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:25.749 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f94066f30 msgr2=0x7f2f940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f94066f30 0x7f2f940693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0093f0 tx=0x7f2fac008040 comp rx=0 tx=0).stop 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb015c7b0 msgr2=0x7f2fb00b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb015c7b0 0x7f2fb00b3240 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c7c0 tx=0x7f2fa0007f70 comp rx=0 tx=0).stop 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 shutdown_connections 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f94066f30 0x7f2f940693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fb00afed0 0x7f2fb00ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2fb015dd00 0x7f2fb00b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 --2- 172.21.15.3:0/1200597898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb015c7b0 0x7f2fb00b3240 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 >> 172.21.15.3:0/1200597898 conn(0x7f2fb0070310 msgr2=0x7f2fb0159930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 shutdown_connections 2023-10-06T04:57:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:25.749+0000 7f2fb8014700 1 -- 172.21.15.3:0/1200597898 wait complete. 2023-10-06T04:57:25.762 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:25.762 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:30.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.893+0000 7f6f8ca5d700 1 Processor -- start 2023-10-06T04:57:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.893+0000 7f6f8ca5d700 1 -- start start 2023-10-06T04:57:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.894+0000 7f6f8ca5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f880cd010 0x7f6f880cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.894+0000 7f6f8ca5d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6f8815d2e0 0x7f6f88159190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.894+0000 7f6f8ca5d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f880ce520 con 0x7f6f880cd3e0 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.894+0000 7f6f8ca5d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8815d850 con 0x7f6f880cd010 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.894+0000 7f6f8ca5d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8815db30 con 0x7f6f8815d2e0 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f880cd010 0x7f6f880cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f86d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6f880cd3e0 0x7f6f8806d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41932/0 (socket says 172.21.15.3:41932) 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f86d9d700 1 -- 172.21.15.3:0/1195266868 learned_addr learned my addr 172.21.15.3:0/1195266868 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f85d9b700 1 --2- 172.21.15.3:0/1195266868 >> v2:172.21.15.153:3300/0 conn(0x7f6f8815d2e0 0x7f6f88159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8659c700 1 -- 172.21.15.3:0/1195266868 >> v2:172.21.15.153:3300/0 conn(0x7f6f8815d2e0 msgr2=0x7f6f88159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8659c700 1 --2- 172.21.15.3:0/1195266868 >> v2:172.21.15.153:3300/0 conn(0x7f6f8815d2e0 0x7f6f88159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8659c700 1 -- 172.21.15.3:0/1195266868 >> v1:172.21.15.3:6789/0 conn(0x7f6f880cd3e0 legacy=0x7f6f8806d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8659c700 1 -- 172.21.15.3:0/1195266868 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8815de10 con 0x7f6f880cd010 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8659c700 1 --2- 172.21.15.3:0/1195266868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f880cd010 0x7f6f880cb5e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6f880d14c0 tx=0x7f6f80009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc901402bbe72712 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8559a700 1 -- 172.21.15.3:0/1195266868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000e070 con 0x7f6f880cd010 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8559a700 1 -- 172.21.15.3:0/1195266868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8000bc30 con 0x7f6f880cd010 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.895+0000 7f6f8559a700 1 -- 172.21.15.3:0/1195266868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80012430 con 0x7f6f880cd010 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/1195266868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f880cd010 msgr2=0x7f6f880cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/1195266868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f880cd010 0x7f6f880cb5e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6f880d14c0 tx=0x7f6f80009780 comp rx=0 tx=0).stop 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/1195266868 shutdown_connections 2023-10-06T04:57:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/1195266868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f880cd010 0x7f6f880cb5e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:30.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/1195266868 >> v2:172.21.15.153:3300/0 conn(0x7f6f8815d2e0 0x7f6f88159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:30.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/1195266868 >> 172.21.15.3:0/1195266868 conn(0x7f6f88070310 msgr2=0x7f6f88070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:30.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/1195266868 shutdown_connections 2023-10-06T04:57:30.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.896+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/1195266868 wait complete. 2023-10-06T04:57:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.897+0000 7f6f8ca5d700 1 Processor -- start 2023-10-06T04:57:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.897+0000 7f6f8ca5d700 1 -- start start 2023-10-06T04:57:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f8ca5d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f880cd010 0x7f6f8816cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f8ca5d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f880cd3e0 0x7f6f88152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f8ca5d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8815d2e0 0x7f6f88156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f8ca5d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f88153260 con 0x7f6f8815d2e0 2023-10-06T04:57:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f8ca5d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8816d5a0 con 0x7f6f880cd010 2023-10-06T04:57:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f8ca5d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8816d880 con 0x7f6f880cd3e0 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8815d2e0 0x7f6f88156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8815d2e0 0x7f6f88156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58254/0 (socket says 172.21.15.3:58254) 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 -- 172.21.15.3:0/81717141 learned_addr learned my addr 172.21.15.3:0/81717141 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f8659c700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f880cd010 0x7f6f8816cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f85d9b700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f880cd3e0 0x7f6f88152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 -- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f880cd3e0 msgr2=0x7f6f88152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f880cd3e0 0x7f6f88152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 -- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f880cd010 msgr2=0x7f6f8816cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f880cd010 0x7f6f8816cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 -- 172.21.15.3:0/81717141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f80009a20 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.898+0000 7f6f86d9d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8815d2e0 0x7f6f88156f60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800c590 tx=0x7f6f78007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.899+0000 7f6f7e7fc700 1 -- 172.21.15.3:0/81717141 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78005b30 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.899+0000 7f6f7e7fc700 1 -- 172.21.15.3:0/81717141 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f78005cb0 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.899+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8816db60 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.899+0000 7f6f7e7fc700 1 -- 172.21.15.3:0/81717141 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78007320 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.899+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f881574a0 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.900+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f881576c0 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.900+0000 7f6f7e7fc700 1 -- 172.21.15.3:0/81717141 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f7801b3b0 con 0x7f6f8815d2e0 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.900+0000 7f6f7e7fc700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f68066f30 0x7f6f680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:30.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.901+0000 7f6f7e7fc700 1 -- 172.21.15.3:0/81717141 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6f78083bd0 con 0x7f6f8815d2e0 2023-10-06T04:57:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.901+0000 7f6f8659c700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f68066f30 0x7f6f680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.901+0000 7f6f8659c700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f68066f30 0x7f6f680693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f880d14c0 tx=0x7f6f80019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:30.905+0000 7f6f7e7fc700 1 -- 172.21.15.3:0/81717141 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f78050ce0 con 0x7f6f8815d2e0 2023-10-06T04:57:31.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.143+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6f88169d30 con 0x7f6f8815d2e0 2023-10-06T04:57:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.143+0000 7f6f7e7fc700 1 -- 172.21.15.3:0/81717141 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f6f88169d30 con 0x7f6f8815d2e0 2023-10-06T04:57:31.146 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:31.146 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.145+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f68066f30 msgr2=0x7f6f680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.145+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f68066f30 0x7f6f680693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f880d14c0 tx=0x7f6f80019000 comp rx=0 tx=0).stop 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.145+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8815d2e0 msgr2=0x7f6f88156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.145+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8815d2e0 0x7f6f88156f60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800c590 tx=0x7f6f78007ef0 comp rx=0 tx=0).stop 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 shutdown_connections 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f68066f30 0x7f6f680693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f880cd010 0x7f6f8816cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8815d2e0 0x7f6f88156f60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 --2- 172.21.15.3:0/81717141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f880cd3e0 0x7f6f88152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 >> 172.21.15.3:0/81717141 conn(0x7f6f88070310 msgr2=0x7f6f881586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 shutdown_connections 2023-10-06T04:57:31.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:31.146+0000 7f6f8ca5d700 1 -- 172.21.15.3:0/81717141 wait complete. 2023-10-06T04:57:31.159 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:31.159 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:36.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.290+0000 7f92a759e700 1 Processor -- start 2023-10-06T04:57:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.290+0000 7f92a759e700 1 -- start start 2023-10-06T04:57:36.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.291+0000 7f92a759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 0x7f92a80c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.291+0000 7f92a759e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a80b36e0 con 0x7f92a80c52c0 2023-10-06T04:57:36.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.291+0000 7f92a759e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a80c6d90 con 0x7f92a80c5bd0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.291+0000 7f92a759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80c7070 con 0x7f92a80b4ae0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.291+0000 7f92a659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f92a80c52c0 0x7f92a80b3b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39634/0 (socket says 172.21.15.3:39634) 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.291+0000 7f92a659c700 1 -- 172.21.15.3:0/3170771903 learned_addr learned my addr 172.21.15.3:0/3170771903 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.291+0000 7f92a5d9b700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 0x7f92a80c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997626125 0 0) 0x7f92a80b36e0 con 0x7f92a80c52c0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9294003570 con 0x7f92a80c52c0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957504422 0 0) 0x7f92a80c6d90 con 0x7f92a80c5bd0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a80b36e0 con 0x7f92a80c5bd0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3559097506 0 0) 0x7f9294003570 con 0x7f92a80c52c0 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 msgr2=0x7f92a80c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 0x7f92a80c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 >> v1:172.21.15.153:6789/0 conn(0x7f92a80c5bd0 legacy=0x7f92a80ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a80b3090 con 0x7f92a80c52c0 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f92a0001470 con 0x7f92a80c52c0 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 >> v1:172.21.15.3:6789/0 conn(0x7f92a80c52c0 legacy=0x7f92a80b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 0x7f9294003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9294004180 0x7f9294004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80c7070 con 0x7f9294004d50 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929400bba0 con 0x7f9294004180 2023-10-06T04:57:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929400bd60 con 0x7f92a80b4ae0 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a6d9d700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a5d9b700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9294004180 0x7f9294004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a659c700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 0x7f9294003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a6d9d700 1 -- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 msgr2=0x7f9294003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a6d9d700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b4ae0 0x7f9294003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a6d9d700 1 -- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9294004180 msgr2=0x7f9294004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a6d9d700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9294004180 0x7f9294004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.292+0000 7f92a6d9d700 1 -- 172.21.15.3:0/3170771903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294003570 con 0x7f9294004d50 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a6d9d700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92940051c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9298002120 tx=0x7f92980017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57af01db668b3c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298001ab0 con 0x7f9294004d50 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929800c980 con 0x7f9294004d50 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a559a700 1 -- 172.21.15.3:0/3170771903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800cd20 con 0x7f9294004d50 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a759e700 1 -- 172.21.15.3:0/3170771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 msgr2=0x7f92940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a759e700 1 --2- 172.21.15.3:0/3170771903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92940051c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9298002120 tx=0x7f92980017a0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a6d9d700 1 -- 172.21.15.3:0/3170771903 reap_dead start 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a759e700 1 -- 172.21.15.3:0/3170771903 shutdown_connections 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a759e700 1 -- 172.21.15.3:0/3170771903 >> 172.21.15.3:0/3170771903 conn(0x7f92a8070310 msgr2=0x7f92a8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a759e700 1 -- 172.21.15.3:0/3170771903 shutdown_connections 2023-10-06T04:57:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.293+0000 7f92a759e700 1 -- 172.21.15.3:0/3170771903 wait complete. 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 Processor -- start 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 -- start start 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92a81584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a8153ea0 0x7f92a8154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a814e940 0x7f92a814edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8150330 con 0x7f9294004d50 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8154850 con 0x7f92a814e940 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a814f6b0 con 0x7f92a8153ea0 2023-10-06T04:57:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.295+0000 7f92a659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92a81584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92a81584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58366/0 (socket says 172.21.15.3:58366) 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 -- 172.21.15.3:0/1706818145 learned_addr learned my addr 172.21.15.3:0/1706818145 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a5d9b700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a8153ea0 0x7f92a8154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a6d9d700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a814e940 0x7f92a814edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 -- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a8153ea0 msgr2=0x7f92a8154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a8153ea0 0x7f92a8154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 -- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a814e940 msgr2=0x7f92a814edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a814e940 0x7f92a814edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 -- 172.21.15.3:0/1706818145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a814f990 con 0x7f9294004d50 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a659c700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92a81584e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f92a0007390 tx=0x7f92a0001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f929e7fc700 1 -- 172.21.15.3:0/1706818145 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0011590 con 0x7f9294004d50 2023-10-06T04:57:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f929e7fc700 1 -- 172.21.15.3:0/1706818145 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a00116f0 con 0x7f9294004d50 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a814fc70 con 0x7f9294004d50 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f929e7fc700 1 -- 172.21.15.3:0/1706818145 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0011ab0 con 0x7f9294004d50 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.296+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a814f2f0 con 0x7f9294004d50 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.297+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a804c380 con 0x7f9294004d50 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.298+0000 7f929e7fc700 1 -- 172.21.15.3:0/1706818145 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92a001d460 con 0x7f9294004d50 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.298+0000 7f929e7fc700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.298+0000 7f929e7fc700 1 -- 172.21.15.3:0/1706818145 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f92a0084760 con 0x7f9294004d50 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.298+0000 7f92a5d9b700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.299+0000 7f92a5d9b700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f92a81550f0 tx=0x7f92900091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:36.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.302+0000 7f929e7fc700 1 -- 172.21.15.3:0/1706818145 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92a0051870 con 0x7f9294004d50 2023-10-06T04:57:36.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.540+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f92a80b4550 con 0x7f9294004d50 2023-10-06T04:57:36.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.540+0000 7f929e7fc700 1 -- 172.21.15.3:0/1706818145 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f92a0054e90 con 0x7f9294004d50 2023-10-06T04:57:36.542 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:36.542 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:36.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.542+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 msgr2=0x7f92880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:36.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.542+0000 7f92a759e700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f92a81550f0 tx=0x7f92900091a0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.542+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 msgr2=0x7f92a81584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.542+0000 7f92a759e700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92a81584e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f92a0007390 tx=0x7f92a0001df0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 shutdown_connections 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a814e940 0x7f92a814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9294004d50 0x7f92a81584e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 --2- 172.21.15.3:0/1706818145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a8153ea0 0x7f92a8154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 >> 172.21.15.3:0/1706818145 conn(0x7f92a8070310 msgr2=0x7f92a809d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 shutdown_connections 2023-10-06T04:57:36.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:36.543+0000 7f92a759e700 1 -- 172.21.15.3:0/1706818145 wait complete. 2023-10-06T04:57:36.556 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:36.556 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:41.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:41.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.687+0000 7f7e63039700 1 Processor -- start 2023-10-06T04:57:41.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.687+0000 7f7e63039700 1 -- start start 2023-10-06T04:57:41.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e63039700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7e5c0ca710 0x7f7e5c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:41.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e63039700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e5c0caae0 0x7f7e5c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:41.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e63039700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c15b300 0x7f7e5c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:41.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e63039700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c15d2b0 con 0x7f7e5c0caae0 2023-10-06T04:57:41.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e63039700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c15a1a0 con 0x7f7e5c0ca710 2023-10-06T04:57:41.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e63039700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c15a480 con 0x7f7e5c15b300 2023-10-06T04:57:41.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e5c0caae0 0x7f7e5c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e5c0caae0 0x7f7e5c15adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58370/0 (socket says 172.21.15.3:58370) 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 -- 172.21.15.3:0/3680985414 learned_addr learned my addr 172.21.15.3:0/3680985414 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e60dd5700 1 --2- 172.21.15.3:0/3680985414 >> v2:172.21.15.153:3300/0 conn(0x7f7e5c0ca710 0x7f7e5c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e615d6700 1 --2- 172.21.15.3:0/3680985414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c15b300 0x7f7e5c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 -- 172.21.15.3:0/3680985414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c15b300 msgr2=0x7f7e5c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 --2- 172.21.15.3:0/3680985414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c15b300 0x7f7e5c06c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 -- 172.21.15.3:0/3680985414 >> v2:172.21.15.153:3300/0 conn(0x7f7e5c0ca710 msgr2=0x7f7e5c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 --2- 172.21.15.3:0/3680985414 >> v2:172.21.15.153:3300/0 conn(0x7f7e5c0ca710 0x7f7e5c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e5bfff700 1 -- 172.21.15.3:0/3680985414 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c15a760 con 0x7f7e5c0caae0 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.688+0000 7f7e615d6700 1 --2- 172.21.15.3:0/3680985414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c15b300 0x7f7e5c06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e5bfff700 1 --2- 172.21.15.3:0/3680985414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0caae0 0x7f7e5c15adc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0cd9b0 tx=0x7f7e48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c538dc201b175c0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:41.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e5b7fe700 1 -- 172.21.15.3:0/3680985414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48009e70 con 0x7f7e5c0caae0 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e5b7fe700 1 -- 172.21.15.3:0/3680985414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4800b860 con 0x7f7e5c0caae0 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e5b7fe700 1 -- 172.21.15.3:0/3680985414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800bc00 con 0x7f7e5c0caae0 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 -- 172.21.15.3:0/3680985414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0caae0 msgr2=0x7f7e5c15adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 --2- 172.21.15.3:0/3680985414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0caae0 0x7f7e5c15adc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0cd9b0 tx=0x7f7e48009250 comp rx=0 tx=0).stop 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 -- 172.21.15.3:0/3680985414 shutdown_connections 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 --2- 172.21.15.3:0/3680985414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c15b300 0x7f7e5c06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 --2- 172.21.15.3:0/3680985414 >> v2:172.21.15.153:3300/0 conn(0x7f7e5c0ca710 0x7f7e5c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 --2- 172.21.15.3:0/3680985414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0caae0 0x7f7e5c15adc0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 -- 172.21.15.3:0/3680985414 >> 172.21.15.3:0/3680985414 conn(0x7f7e5c06ea50 msgr2=0x7f7e5c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.689+0000 7f7e63039700 1 -- 172.21.15.3:0/3680985414 shutdown_connections 2023-10-06T04:57:41.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.690+0000 7f7e63039700 1 -- 172.21.15.3:0/3680985414 wait complete. 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.691+0000 7f7e63039700 1 Processor -- start 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.691+0000 7f7e63039700 1 -- start start 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e63039700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0ca710 0x7f7e5c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e63039700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c0caae0 0x7f7e5c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e63039700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e5c0b1910 0x7f7e5c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e63039700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0b2670 con 0x7f7e5c0ca710 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e63039700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0c7140 con 0x7f7e5c0b1910 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e63039700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0b1dd0 con 0x7f7e5c0caae0 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0ca710 0x7f7e5c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0ca710 0x7f7e5c0c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58372/0 (socket says 172.21.15.3:58372) 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 -- 172.21.15.3:0/444649034 learned_addr learned my addr 172.21.15.3:0/444649034 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:41.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e5bfff700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c0caae0 0x7f7e5c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e615d6700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e5c0b1910 0x7f7e5c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 -- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c0caae0 msgr2=0x7f7e5c15b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c0caae0 0x7f7e5c15b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 -- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e5c0b1910 msgr2=0x7f7e5c1533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e5c0b1910 0x7f7e5c1533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 -- 172.21.15.3:0/444649034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e480094f0 con 0x7f7e5c0ca710 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e5bfff700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c0caae0 0x7f7e5c15b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e60dd5700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0ca710 0x7f7e5c0c97a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e930 tx=0x7f7e500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.692+0000 7f7e58ff9700 1 -- 172.21.15.3:0/444649034 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000eed0 con 0x7f7e5c0ca710 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.693+0000 7f7e58ff9700 1 -- 172.21.15.3:0/444649034 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000c850 con 0x7f7e5c0ca710 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.693+0000 7f7e58ff9700 1 -- 172.21.15.3:0/444649034 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000cbf0 con 0x7f7e5c0ca710 2023-10-06T04:57:41.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.693+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c0b20b0 con 0x7f7e5c0ca710 2023-10-06T04:57:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.693+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c0c9ce0 con 0x7f7e5c0ca710 2023-10-06T04:57:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.693+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c0c6f10 con 0x7f7e5c0ca710 2023-10-06T04:57:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.694+0000 7f7e58ff9700 1 -- 172.21.15.3:0/444649034 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e50013470 con 0x7f7e5c0ca710 2023-10-06T04:57:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.695+0000 7f7e58ff9700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7e40066f30 0x7f7e400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.695+0000 7f7e58ff9700 1 -- 172.21.15.3:0/444649034 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e50084eb0 con 0x7f7e5c0ca710 2023-10-06T04:57:41.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.695+0000 7f7e5bfff700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7e40066f30 0x7f7e400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:41.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.695+0000 7f7e5bfff700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7e40066f30 0x7f7e400693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0cd9b0 tx=0x7f7e48017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.700+0000 7f7e58ff9700 1 -- 172.21.15.3:0/444649034 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e50052040 con 0x7f7e5c0ca710 2023-10-06T04:57:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.933+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7e5c0c6f10 con 0x7f7e5c0ca710 2023-10-06T04:57:41.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.934+0000 7f7e58ff9700 1 -- 172.21.15.3:0/444649034 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f7e5c0c6f10 con 0x7f7e5c0ca710 2023-10-06T04:57:41.935 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:41.936 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.935+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7e40066f30 msgr2=0x7f7e400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.935+0000 7f7e63039700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7e40066f30 0x7f7e400693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0cd9b0 tx=0x7f7e48017000 comp rx=0 tx=0).stop 2023-10-06T04:57:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.935+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0ca710 msgr2=0x7f7e5c0c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:41.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.935+0000 7f7e63039700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0ca710 0x7f7e5c0c97a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e930 tx=0x7f7e500091a0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 shutdown_connections 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7e40066f30 0x7f7e400693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e5c0b1910 0x7f7e5c1533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e5c0ca710 0x7f7e5c0c97a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 --2- 172.21.15.3:0/444649034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e5c0caae0 0x7f7e5c15b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 >> 172.21.15.3:0/444649034 conn(0x7f7e5c06ea50 msgr2=0x7f7e5c1572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 shutdown_connections 2023-10-06T04:57:41.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:41.936+0000 7f7e63039700 1 -- 172.21.15.3:0/444649034 wait complete. 2023-10-06T04:57:41.949 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:41.949 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:46.950 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:47.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.081+0000 7f4d0dde3700 1 Processor -- start 2023-10-06T04:57:47.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.081+0000 7f4d0dde3700 1 -- start start 2023-10-06T04:57:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d0dde3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d0815d240 0x7f4d0815a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d0dde3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d0dde3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0815d6d0 con 0x7f4d080b27b0 2023-10-06T04:57:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d0dde3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0815aba0 con 0x7f4d0815d240 2023-10-06T04:57:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d0dde3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08150b30 con 0x7f4d080cd320 2023-10-06T04:57:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d06ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d0815d240 0x7f4d0815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d077fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4d080b27b0 0x7f4d080ccc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53132/0 (socket says 172.21.15.3:53132) 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d06ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d0815d240 0x7f4d0815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:54816/0 (socket says 172.21.15.3:54816) 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d077fe700 1 -- 172.21.15.3:0/201351909 learned_addr learned my addr 172.21.15.3:0/201351909 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:44212/0 (socket says 172.21.15.3:44212) 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d06ffd700 1 -- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 msgr2=0x7f4d080aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d06ffd700 1 --2- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d06ffd700 1 -- 172.21.15.3:0/201351909 >> v1:172.21.15.153:6789/0 conn(0x7f4d080b27b0 legacy=0x7f4d080ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d06ffd700 1 -- 172.21.15.3:0/201351909 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0815d9b0 con 0x7f4d0815d240 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.082+0000 7f4d067fc700 1 -- 172.21.15.3:0/201351909 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049757911 0 0) 0x7f4d0815d6d0 con 0x7f4d080b27b0 2023-10-06T04:57:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d07fff700 1 --2- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d06ffd700 1 --2- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d0815d240 0x7f4d0815a660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0098e0 tx=0x7f4cfc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702d37d310a4d74d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d067fc700 1 -- 172.21.15.3:0/201351909 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00b640 con 0x7f4d0815d240 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d067fc700 1 -- 172.21.15.3:0/201351909 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00b7a0 con 0x7f4d0815d240 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d067fc700 1 -- 172.21.15.3:0/201351909 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00bb40 con 0x7f4d0815d240 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d0dde3700 1 -- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d0815d240 msgr2=0x7f4d0815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d0815d240 0x7f4d0815a660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0098e0 tx=0x7f4cfc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d0dde3700 1 -- 172.21.15.3:0/201351909 shutdown_connections 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/201351909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d0815d240 0x7f4d0815a660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.083+0000 7f4d0dde3700 1 -- 172.21.15.3:0/201351909 >> 172.21.15.3:0/201351909 conn(0x7f4d08070310 msgr2=0x7f4d08070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.084+0000 7f4d0dde3700 1 -- 172.21.15.3:0/201351909 shutdown_connections 2023-10-06T04:57:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.084+0000 7f4d0dde3700 1 -- 172.21.15.3:0/201351909 wait complete. 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 Processor -- start 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 -- start start 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d080b27b0 0x7f4d0814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d0815d240 0x7f4d0815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08150460 con 0x7f4d0815d240 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08152f20 con 0x7f4d080b27b0 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.085+0000 7f4d0dde3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081525f0 con 0x7f4d080cd320 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d0815d240 0x7f4d0815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d0815d240 0x7f4d0815ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43264/0 (socket says 172.21.15.3:43264) 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 -- 172.21.15.3:0/3357202790 learned_addr learned my addr 172.21.15.3:0/3357202790 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d077fe700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d080b27b0 0x7f4d0814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d06ffd700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 -- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 msgr2=0x7f4d080c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 -- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d080b27b0 msgr2=0x7f4d0814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d080b27b0 0x7f4d0814ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 -- 172.21.15.3:0/3357202790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009a20 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d07fff700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d0815d240 0x7f4d0815ad70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000c0f0 tx=0x7f4d0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4cef7fe700 1 -- 172.21.15.3:0/3357202790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000cd90 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4cef7fe700 1 -- 172.21.15.3:0/3357202790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0000f040 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc009420 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4cef7fe700 1 -- 172.21.15.3:0/3357202790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000146d0 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.086+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d08152bb0 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.087+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d081530b0 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.088+0000 7f4cef7fe700 1 -- 172.21.15.3:0/3357202790 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d00003020 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.088+0000 7f4cef7fe700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4cf4066f30 0x7f4cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.088+0000 7f4cef7fe700 1 -- 172.21.15.3:0/3357202790 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4d00085170 con 0x7f4d0815d240 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.089+0000 7f4d077fe700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4cf4066f30 0x7f4cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.089+0000 7f4d077fe700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4cf4066f30 0x7f4cf40693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8001230 tx=0x7f4cf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:47.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.093+0000 7f4cef7fe700 1 -- 172.21.15.3:0/3357202790 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d000521e0 con 0x7f4d0815d240 2023-10-06T04:57:47.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.330+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4d08071680 con 0x7f4d0815d240 2023-10-06T04:57:47.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.331+0000 7f4cef7fe700 1 -- 172.21.15.3:0/3357202790 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f4d00055800 con 0x7f4d0815d240 2023-10-06T04:57:47.333 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:47.333 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:47.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4cf4066f30 msgr2=0x7f4cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4cf4066f30 0x7f4cf40693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8001230 tx=0x7f4cf8009040 comp rx=0 tx=0).stop 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d0815d240 msgr2=0x7f4d0815ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d0815d240 0x7f4d0815ad70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000c0f0 tx=0x7f4d0000b040 comp rx=0 tx=0).stop 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 shutdown_connections 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4cf4066f30 0x7f4cf40693e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d080b27b0 0x7f4d0814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d0815d240 0x7f4d0815ad70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 --2- 172.21.15.3:0/3357202790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d080cd320 0x7f4d080c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 >> 172.21.15.3:0/3357202790 conn(0x7f4d08070310 msgr2=0x7f4d080706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:47.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 shutdown_connections 2023-10-06T04:57:47.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:47.333+0000 7f4d0dde3700 1 -- 172.21.15.3:0/3357202790 wait complete. 2023-10-06T04:57:47.346 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:47.346 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:52.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:52.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.476+0000 7fd75f593700 1 Processor -- start 2023-10-06T04:57:52.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.476+0000 7fd75f593700 1 -- start start 2023-10-06T04:57:52.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75f593700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd75815d240 0x7fd75815a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75f593700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd7580cd320 0x7fd7580aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75f593700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75815d6d0 con 0x7fd7580b27b0 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75f593700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75815aba0 con 0x7fd75815d240 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75f593700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd758150b30 con 0x7fd7580cd320 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd75815d240 0x7fd75815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd75815d240 0x7fd75815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34468/0 (socket says 172.21.15.3:34468) 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 -- 172.21.15.3:0/4035724464 learned_addr learned my addr 172.21.15.3:0/4035724464 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75db30700 1 --2- 172.21.15.3:0/4035724464 >> v2:172.21.15.153:3300/0 conn(0x7fd7580cd320 0x7fd7580aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 -- 172.21.15.3:0/4035724464 >> v2:172.21.15.153:3300/0 conn(0x7fd7580cd320 msgr2=0x7fd7580aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:52.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 --2- 172.21.15.3:0/4035724464 >> v2:172.21.15.153:3300/0 conn(0x7fd7580cd320 0x7fd7580aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 -- 172.21.15.3:0/4035724464 >> v1:172.21.15.153:6789/0 conn(0x7fd7580b27b0 legacy=0x7fd7580ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 -- 172.21.15.3:0/4035724464 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75815d9b0 con 0x7fd75815d240 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd75cb2e700 1 --2- 172.21.15.3:0/4035724464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75815d240 0x7fd75815a660 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd75815e2b0 tx=0x7fd748009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba850fc001bbe3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.477+0000 7fd74ffff700 1 -- 172.21.15.3:0/4035724464 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74800b820 con 0x7fd75815d240 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd74ffff700 1 -- 172.21.15.3:0/4035724464 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74800b980 con 0x7fd75815d240 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd74ffff700 1 -- 172.21.15.3:0/4035724464 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74800bd20 con 0x7fd75815d240 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 -- 172.21.15.3:0/4035724464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75815d240 msgr2=0x7fd75815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 --2- 172.21.15.3:0/4035724464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75815d240 0x7fd75815a660 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd75815e2b0 tx=0x7fd748009250 comp rx=0 tx=0).stop 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 -- 172.21.15.3:0/4035724464 shutdown_connections 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 --2- 172.21.15.3:0/4035724464 >> v2:172.21.15.153:3300/0 conn(0x7fd7580cd320 0x7fd7580aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 --2- 172.21.15.3:0/4035724464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75815d240 0x7fd75815a660 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 -- 172.21.15.3:0/4035724464 >> 172.21.15.3:0/4035724464 conn(0x7fd758070310 msgr2=0x7fd758070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:52.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 -- 172.21.15.3:0/4035724464 shutdown_connections 2023-10-06T04:57:52.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.478+0000 7fd75f593700 1 -- 172.21.15.3:0/4035724464 wait complete. 2023-10-06T04:57:52.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.479+0000 7fd75f593700 1 Processor -- start 2023-10-06T04:57:52.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.479+0000 7fd75f593700 1 -- start start 2023-10-06T04:57:52.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75f593700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7580b27b0 0x7fd7580c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:52.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75f593700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7580cd320 0x7fd75814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75f593700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd75815d240 0x7fd758169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75f593700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580ca420 con 0x7fd7580cd320 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75f593700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7581534c0 con 0x7fd75815d240 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75f593700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580c9e60 con 0x7fd7580b27b0 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75cb2e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7580cd320 0x7fd75814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75cb2e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7580cd320 0x7fd75814fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34478/0 (socket says 172.21.15.3:34478) 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75cb2e700 1 -- 172.21.15.3:0/1860268880 learned_addr learned my addr 172.21.15.3:0/1860268880 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75d32f700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7580b27b0 0x7fd7580c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.480+0000 7fd75db30700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd75815d240 0x7fd758169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:52.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75cb2e700 1 -- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7580b27b0 msgr2=0x7fd7580c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75cb2e700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7580b27b0 0x7fd7580c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75cb2e700 1 -- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd75815d240 msgr2=0x7fd758169c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75cb2e700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd75815d240 0x7fd758169c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75cb2e700 1 -- 172.21.15.3:0/1860268880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480094f0 con 0x7fd7580cd320 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75db30700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd75815d240 0x7fd758169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75cb2e700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7580cd320 0x7fd75814fdd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd748009290 tx=0x7fd748016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd74d7fa700 1 -- 172.21.15.3:0/1860268880 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748011d70 con 0x7fd7580cd320 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd74d7fa700 1 -- 172.21.15.3:0/1860268880 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748020070 con 0x7fd7580cd320 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7580ca140 con 0x7fd7580cd320 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd74d7fa700 1 -- 172.21.15.3:0/1860268880 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74801b6a0 con 0x7fd7580cd320 2023-10-06T04:57:52.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.481+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758150f20 con 0x7fd7580cd320 2023-10-06T04:57:52.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.482+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758152fb0 con 0x7fd7580cd320 2023-10-06T04:57:52.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.482+0000 7fd74d7fa700 1 -- 172.21.15.3:0/1860268880 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd748022030 con 0x7fd7580cd320 2023-10-06T04:57:52.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.483+0000 7fd74d7fa700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd73c066f30 0x7fd73c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:52.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.483+0000 7fd74d7fa700 1 -- 172.21.15.3:0/1860268880 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd748086410 con 0x7fd7580cd320 2023-10-06T04:57:52.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.483+0000 7fd75d32f700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd73c066f30 0x7fd73c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:52.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.483+0000 7fd75d32f700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd73c066f30 0x7fd73c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd754006fd0 tx=0x7fd7540090f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:52.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.487+0000 7fd74d7fa700 1 -- 172.21.15.3:0/1860268880 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd758152fb0 con 0x7fd7580cd320 2023-10-06T04:57:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.729+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd758152fb0 con 0x7fd7580cd320 2023-10-06T04:57:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.730+0000 7fd74d7fa700 1 -- 172.21.15.3:0/1860268880 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fd758152fb0 con 0x7fd7580cd320 2023-10-06T04:57:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:52.733 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd73c066f30 msgr2=0x7fd73c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd73c066f30 0x7fd73c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd754006fd0 tx=0x7fd7540090f0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7580cd320 msgr2=0x7fd75814fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7580cd320 0x7fd75814fdd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd748009290 tx=0x7fd748016000 comp rx=0 tx=0).stop 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 shutdown_connections 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd73c066f30 0x7fd73c0693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd75815d240 0x7fd758169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7580b27b0 0x7fd7580c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 --2- 172.21.15.3:0/1860268880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7580cd320 0x7fd75814fdd0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 >> 172.21.15.3:0/1860268880 conn(0x7fd758070310 msgr2=0x7fd7580b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 shutdown_connections 2023-10-06T04:57:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:52.732+0000 7fd75f593700 1 -- 172.21.15.3:0/1860268880 wait complete. 2023-10-06T04:57:52.747 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:52.747 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:57:55.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:55.224 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:57.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:57:57.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.877+0000 7fee101ad700 1 Processor -- start 2023-10-06T04:57:57.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.877+0000 7fee101ad700 1 -- start start 2023-10-06T04:57:57.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee101ad700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee080b27b0 0x7fee0815a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:57.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee101ad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee0815d240 0x7fee080ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:57.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee101ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee080cd150 0x7fee080aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:57.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee101ad700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0815aba0 con 0x7fee080cd150 2023-10-06T04:57:57.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee101ad700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee08150b30 con 0x7fee080b27b0 2023-10-06T04:57:57.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee101ad700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee080ce560 con 0x7fee0815d240 2023-10-06T04:57:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee080cd150 0x7fee080aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee080cd150 0x7fee080aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34480/0 (socket says 172.21.15.3:34480) 2023-10-06T04:57:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 -- 172.21.15.3:0/1535571120 learned_addr learned my addr 172.21.15.3:0/1535571120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0d748700 1 --2- 172.21.15.3:0/1535571120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee0815d240 0x7fee080ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0df49700 1 --2- 172.21.15.3:0/1535571120 >> v2:172.21.15.153:3300/0 conn(0x7fee080b27b0 0x7fee0815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 -- 172.21.15.3:0/1535571120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee0815d240 msgr2=0x7fee080ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 --2- 172.21.15.3:0/1535571120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee0815d240 0x7fee080ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 -- 172.21.15.3:0/1535571120 >> v2:172.21.15.153:3300/0 conn(0x7fee080b27b0 msgr2=0x7fee0815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 --2- 172.21.15.3:0/1535571120 >> v2:172.21.15.153:3300/0 conn(0x7fee080b27b0 0x7fee0815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 -- 172.21.15.3:0/1535571120 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080cd5f0 con 0x7fee080cd150 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0d748700 1 --2- 172.21.15.3:0/1535571120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee0815d240 0x7fee080ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0df49700 1 --2- 172.21.15.3:0/1535571120 >> v2:172.21.15.153:3300/0 conn(0x7fee080b27b0 0x7fee0815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.878+0000 7fee0e74a700 1 --2- 172.21.15.3:0/1535571120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080cd150 0x7fee080aeef0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8628059b43019f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee0cf47700 1 -- 172.21.15.3:0/1535571120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800b820 con 0x7fee080cd150 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee0cf47700 1 -- 172.21.15.3:0/1535571120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf800b980 con 0x7fee080cd150 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee0cf47700 1 -- 172.21.15.3:0/1535571120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800bd20 con 0x7fee080cd150 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 -- 172.21.15.3:0/1535571120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080cd150 msgr2=0x7fee080aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 --2- 172.21.15.3:0/1535571120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080cd150 0x7fee080aeef0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8009250 comp rx=0 tx=0).stop 2023-10-06T04:57:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 -- 172.21.15.3:0/1535571120 shutdown_connections 2023-10-06T04:57:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 --2- 172.21.15.3:0/1535571120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080cd150 0x7fee080aeef0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 --2- 172.21.15.3:0/1535571120 >> v2:172.21.15.153:3300/0 conn(0x7fee080b27b0 0x7fee0815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 --2- 172.21.15.3:0/1535571120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee0815d240 0x7fee080ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 -- 172.21.15.3:0/1535571120 >> 172.21.15.3:0/1535571120 conn(0x7fee08070310 msgr2=0x7fee08070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 -- 172.21.15.3:0/1535571120 shutdown_connections 2023-10-06T04:57:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.879+0000 7fee101ad700 1 -- 172.21.15.3:0/1535571120 wait complete. 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.880+0000 7fee101ad700 1 Processor -- start 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee101ad700 1 -- start start 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee101ad700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee080b27b0 0x7fee08150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee101ad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee080cd150 0x7fee080b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee101ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0815d240 0x7fee080ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee101ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08150570 con 0x7fee0815d240 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee101ad700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0816da50 con 0x7fee080b27b0 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee101ad700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee080b34f0 con 0x7fee080cd150 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee0e74a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0815d240 0x7fee080ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee0e74a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0815d240 0x7fee080ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34488/0 (socket says 172.21.15.3:34488) 2023-10-06T04:57:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.881+0000 7fee0e74a700 1 -- 172.21.15.3:0/1693235905 learned_addr learned my addr 172.21.15.3:0/1693235905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0df49700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee080b27b0 0x7fee08150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0d748700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee080cd150 0x7fee080b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0e74a700 1 -- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee080cd150 msgr2=0x7fee080b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0e74a700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee080cd150 0x7fee080b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0e74a700 1 -- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee080b27b0 msgr2=0x7fee08150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0e74a700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee080b27b0 0x7fee08150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0e74a700 1 -- 172.21.15.3:0/1693235905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80094f0 con 0x7fee0815d240 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0df49700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee080b27b0 0x7fee08150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee0e74a700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0815d240 0x7fee080ca9a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fedf80094c0 tx=0x7fedf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee05ffb700 1 -- 172.21.15.3:0/1693235905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800b820 con 0x7fee0815d240 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee05ffb700 1 -- 172.21.15.3:0/1693235905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf801a660 con 0x7fee0815d240 2023-10-06T04:57:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0816ce40 con 0x7fee0815d240 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee05ffb700 1 -- 172.21.15.3:0/1693235905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf801aa00 con 0x7fee0815d240 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.882+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0816d400 con 0x7fee0815d240 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.883+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee080c8240 con 0x7fee0815d240 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.884+0000 7fee05ffb700 1 -- 172.21.15.3:0/1693235905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedf801b430 con 0x7fee0815d240 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.884+0000 7fee05ffb700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fedf4066f30 0x7fedf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.884+0000 7fee05ffb700 1 -- 172.21.15.3:0/1693235905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fedf8085de0 con 0x7fee0815d240 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.884+0000 7fee0df49700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fedf4066f30 0x7fedf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.885+0000 7fee0df49700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fedf4066f30 0x7fedf40693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fee00009fd0 tx=0x7fee000098a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:57:57.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:57.888+0000 7fee05ffb700 1 -- 172.21.15.3:0/1693235905 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee080c8240 con 0x7fee0815d240 2023-10-06T04:57:58.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.121+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fee080c8240 con 0x7fee0815d240 2023-10-06T04:57:58.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.122+0000 7fee05ffb700 1 -- 172.21.15.3:0/1693235905 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fedf802a0c0 con 0x7fee0815d240 2023-10-06T04:57:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:57:58.125 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fedf4066f30 msgr2=0x7fedf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fedf4066f30 0x7fedf40693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fee00009fd0 tx=0x7fee000098a0 comp rx=0 tx=0).stop 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0815d240 msgr2=0x7fee080ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0815d240 0x7fee080ca9a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fedf80094c0 tx=0x7fedf8017000 comp rx=0 tx=0).stop 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 shutdown_connections 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fedf4066f30 0x7fedf40693e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee080b27b0 0x7fee08150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0815d240 0x7fee080ca9a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 --2- 172.21.15.3:0/1693235905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee080cd150 0x7fee080b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:57:58.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 >> 172.21.15.3:0/1693235905 conn(0x7fee08070310 msgr2=0x7fee080ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:57:58.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 shutdown_connections 2023-10-06T04:57:58.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:57:58.124+0000 7fee101ad700 1 -- 172.21.15.3:0/1693235905 wait complete. 2023-10-06T04:57:58.138 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:57:58.138 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:58:03.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:58:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.269+0000 7f6f4ea4e700 1 Processor -- start 2023-10-06T04:58:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.269+0000 7f6f4ea4e700 1 -- start start 2023-10-06T04:58:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f4ea4e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6f4815c7b0 0x7f6f4815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f4ea4e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 0x7f6f480cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f4ea4e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480cf2a0 con 0x7f6f480afed0 2023-10-06T04:58:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f4ea4e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f480cc7d0 con 0x7f6f4815c7b0 2023-10-06T04:58:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f4ea4e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480cb8d0 con 0x7f6f4815dd00 2023-10-06T04:58:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f4cfeb700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6f480afed0 0x7f6f480cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48576/0 (socket says 172.21.15.3:48576) 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f4cfeb700 1 -- 172.21.15.3:0/3717842196 learned_addr learned my addr 172.21.15.3:0/3717842196 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f47fff700 1 --2- 172.21.15.3:0/3717842196 >> v2:172.21.15.153:3300/0 conn(0x7f6f4815c7b0 0x7f6f4815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f477fe700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 0x7f6f480cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577233488 0 0) 0x7f6f480cf2a0 con 0x7f6f480afed0 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f34003570 con 0x7f6f480afed0 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681554626 0 0) 0x7f6f34003570 con 0x7f6f480afed0 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 msgr2=0x7f6f480cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.270+0000 7f6f46ffd700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 0x7f6f480cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 >> v2:172.21.15.153:3300/0 conn(0x7f6f4815c7b0 msgr2=0x7f6f4815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 --2- 172.21.15.3:0/3717842196 >> v2:172.21.15.153:3300/0 conn(0x7f6f4815c7b0 0x7f6f4815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480ce970 con 0x7f6f480afed0 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6f400017f0 con 0x7f6f480afed0 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 >> v1:172.21.15.3:6789/0 conn(0x7f6f480afed0 legacy=0x7f6f480cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f34003a60 0x7f6f34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f34004810 0x7f6f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 0x7f6f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480cc7d0 con 0x7f6f34004810 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480cb8d0 con 0x7f6f34003a60 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3400ba60 con 0x7f6f4815dd00 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f477fe700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f34004810 0x7f6f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f47fff700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f34003a60 0x7f6f34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f4cfeb700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 0x7f6f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f477fe700 1 -- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 msgr2=0x7f6f340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f477fe700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4815dd00 0x7f6f340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f477fe700 1 -- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f34003a60 msgr2=0x7f6f34003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f477fe700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f34003a60 0x7f6f34003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f477fe700 1 -- 172.21.15.3:0/3717842196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f34003570 con 0x7f6f34004810 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f477fe700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f34004810 0x7f6f34004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009c30 tx=0x7f6f380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257d99d51c072053 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800e710 con 0x7f6f34004810 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3800e870 con 0x7f6f34004810 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.271+0000 7f6f46ffd700 1 -- 172.21.15.3:0/3717842196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800ec30 con 0x7f6f34004810 2023-10-06T04:58:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.272+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/3717842196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f34004810 msgr2=0x7f6f34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.272+0000 7f6f4ea4e700 1 --2- 172.21.15.3:0/3717842196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f34004810 0x7f6f34004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009c30 tx=0x7f6f380091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.272+0000 7f6f4cfeb700 1 -- 172.21.15.3:0/3717842196 reap_dead start 2023-10-06T04:58:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.272+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/3717842196 shutdown_connections 2023-10-06T04:58:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.272+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/3717842196 >> 172.21.15.3:0/3717842196 conn(0x7f6f48070310 msgr2=0x7f6f48070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.272+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/3717842196 shutdown_connections 2023-10-06T04:58:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.272+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/3717842196 wait complete. 2023-10-06T04:58:03.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.273+0000 7f6f4ea4e700 1 Processor -- start 2023-10-06T04:58:03.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f4ea4e700 1 -- start start 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f4ea4e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f34004810 0x7f6f480b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f4ea4e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4816d8f0 0x7f6f481561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f4ea4e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f480b4ba0 0x7f6f480b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f4ea4e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b2da0 con 0x7f6f4816d8f0 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f4ea4e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48160390 con 0x7f6f480b4ba0 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f4ea4e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b0a10 con 0x7f6f34004810 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4816d8f0 0x7f6f481561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4816d8f0 0x7f6f481561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52790/0 (socket says 172.21.15.3:52790) 2023-10-06T04:58:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.274+0000 7f6f477fe700 1 -- 172.21.15.3:0/2912951165 learned_addr learned my addr 172.21.15.3:0/2912951165 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f47fff700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f34004810 0x7f6f480b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f4cfeb700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f480b4ba0 0x7f6f480b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f477fe700 1 -- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f34004810 msgr2=0x7f6f480b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f477fe700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f34004810 0x7f6f480b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f477fe700 1 -- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f480b4ba0 msgr2=0x7f6f480b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f477fe700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f480b4ba0 0x7f6f480b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f477fe700 1 -- 172.21.15.3:0/2912951165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4816dd80 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f477fe700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4816d8f0 0x7f6f481561a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6f34004700 tx=0x7f6f3801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f3ffff700 1 -- 172.21.15.3:0/2912951165 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38022070 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f3ffff700 1 -- 172.21.15.3:0/2912951165 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3800e9e0 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f480b27e0 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f3ffff700 1 -- 172.21.15.3:0/2912951165 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3801c450 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.275+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f481566e0 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.276+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f480b0870 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.277+0000 7f6f3ffff700 1 -- 172.21.15.3:0/2912951165 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f3801c840 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.277+0000 7f6f3ffff700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f28066f30 0x7f6f280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.277+0000 7f6f3ffff700 1 -- 172.21.15.3:0/2912951165 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6f38085f70 con 0x7f6f4816d8f0 2023-10-06T04:58:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.277+0000 7f6f47fff700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f28066f30 0x7f6f280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:03.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.278+0000 7f6f47fff700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f28066f30 0x7f6f280693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006f00 tx=0x7f6f30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:03.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.281+0000 7f6f3ffff700 1 -- 172.21.15.3:0/2912951165 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f380b75a0 con 0x7f6f4816d8f0 2023-10-06T04:58:03.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.518+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6f48157a50 con 0x7f6f4816d8f0 2023-10-06T04:58:03.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.518+0000 7f6f3ffff700 1 -- 172.21.15.3:0/2912951165 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f6f380170c0 con 0x7f6f4816d8f0 2023-10-06T04:58:03.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:03.522 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:58:03.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.520+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f28066f30 msgr2=0x7f6f280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:03.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.520+0000 7f6f4ea4e700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f28066f30 0x7f6f280693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006f00 tx=0x7f6f30008040 comp rx=0 tx=0).stop 2023-10-06T04:58:03.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.520+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4816d8f0 msgr2=0x7f6f481561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:03.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.520+0000 7f6f4ea4e700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4816d8f0 0x7f6f481561a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6f34004700 tx=0x7f6f3801a000 comp rx=0 tx=0).stop 2023-10-06T04:58:03.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 shutdown_connections 2023-10-06T04:58:03.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6f28066f30 0x7f6f280693e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f480b4ba0 0x7f6f480b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f34004810 0x7f6f480b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 --2- 172.21.15.3:0/2912951165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4816d8f0 0x7f6f481561a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:03.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 >> 172.21.15.3:0/2912951165 conn(0x7f6f48070310 msgr2=0x7f6f480c6140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:03.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 shutdown_connections 2023-10-06T04:58:03.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:03.521+0000 7f6f4ea4e700 1 -- 172.21.15.3:0/2912951165 wait complete. 2023-10-06T04:58:03.535 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:58:03.535 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:58:03.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:03.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:03.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:58:03.623 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T04:58:03.641 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:58:03.641 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:58:03.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:03.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:mount: /home/ubuntu/cephtest/mnt.0: mount point not mounted or bad option. 2023-10-06T04:58:03.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:58:03.709+0000 7ff23affd700 -1 client.6303 failed to remount (to trim kernel dentries): return code = 32 2023-10-06T04:58:03.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[112942]: fuse finished with error 0 and tester_r 0 2023-10-06T04:58:03.717 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:58:03.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:03.718 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:58:03.902 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:58:03.902 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:58:03.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:03.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[99939]: fuse finished with error 0 and tester_r 0 2023-10-06T04:58:03.939 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:58:03.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:03.939 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:58:03.997 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:58:03.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:03.997 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:58:04.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:58:04.013 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T04:58:04.013 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:58:04.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:04.014 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:58:04.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:58:04.053 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T04:58:04.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:04.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.170+0000 7fc94d16f700 1 Processor -- start 2023-10-06T04:58:04.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.170+0000 7fc94d16f700 1 -- start start 2023-10-06T04:58:04.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.170+0000 7fc94d16f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 0x7fc9480cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.170+0000 7fc94d16f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9480cc5c0 con 0x7fc9480b2930 2023-10-06T04:58:04.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.170+0000 7fc94d16f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9480cb9b0 con 0x7fc9481598d0 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.170+0000 7fc94d16f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cbc90 con 0x7fc948159500 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc94759e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc9480b2930 0x7fc9480cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48586/0 (socket says 172.21.15.3:48586) 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc94759e700 1 -- 172.21.15.3:0/3035660525 learned_addr learned my addr 172.21.15.3:0/3035660525 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308684241 0 0) 0x7fc9480cc5c0 con 0x7fc9480b2930 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc934003570 con 0x7fc9480b2930 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc946d9d700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 0x7fc9480cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573534346 0 0) 0x7fc934003570 con 0x7fc9480b2930 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 msgr2=0x7fc9480cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 0x7fc9480cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 >> v1:172.21.15.153:6789/0 conn(0x7fc9481598d0 legacy=0x7fc9480b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480cbf70 con 0x7fc9480b2930 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc940001470 con 0x7fc9480b2930 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 >> v1:172.21.15.3:6789/0 conn(0x7fc9480b2930 legacy=0x7fc9480cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc934003a90 0x7fc934003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc934004810 0x7fc934004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 0x7fc9340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cbc90 con 0x7fc934004810 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93400bb10 con 0x7fc934003a90 2023-10-06T04:58:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93400bcd0 con 0x7fc948159500 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc94659c700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc934004810 0x7fc934004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc946d9d700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc934003a90 0x7fc934003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.171+0000 7fc94759e700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 0x7fc9340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94659c700 1 -- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 msgr2=0x7fc9340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94659c700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc948159500 0x7fc9340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94659c700 1 -- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc934003a90 msgr2=0x7fc934003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94659c700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc934003a90 0x7fc934003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94659c700 1 -- 172.21.15.3:0/3035660525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc934003570 con 0x7fc934004810 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94659c700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc934004810 0x7fc934004c80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc930001960 tx=0x7fc930007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab9331b1eb2531d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93000c5d0 con 0x7fc934004810 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93000c750 con 0x7fc934004810 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc945d9b700 1 -- 172.21.15.3:0/3035660525 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93000cb10 con 0x7fc934004810 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94d16f700 1 -- 172.21.15.3:0/3035660525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc934004810 msgr2=0x7fc934004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94d16f700 1 --2- 172.21.15.3:0/3035660525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc934004810 0x7fc934004c80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc930001960 tx=0x7fc930007d10 comp rx=0 tx=0).stop 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94759e700 1 -- 172.21.15.3:0/3035660525 reap_dead start 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94d16f700 1 -- 172.21.15.3:0/3035660525 shutdown_connections 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94d16f700 1 -- 172.21.15.3:0/3035660525 >> 172.21.15.3:0/3035660525 conn(0x7fc948070310 msgr2=0x7fc948070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94d16f700 1 -- 172.21.15.3:0/3035660525 shutdown_connections 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.172+0000 7fc94d16f700 1 -- 172.21.15.3:0/3035660525 wait complete. 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 Processor -- start 2023-10-06T04:58:04.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 -- start start 2023-10-06T04:58:04.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc934004810 0x7fc9481544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9480c3e90 0x7fc948159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480c4300 0x7fc94815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480c6590 con 0x7fc9480c4300 2023-10-06T04:58:04.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815b4c0 con 0x7fc9480c3e90 2023-10-06T04:58:04.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94d16f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815ba80 con 0x7fc934004810 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.174+0000 7fc94759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480c4300 0x7fc94815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480c4300 0x7fc94815a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52806/0 (socket says 172.21.15.3:52806) 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 -- 172.21.15.3:0/2205731024 learned_addr learned my addr 172.21.15.3:0/2205731024 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc946d9d700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc934004810 0x7fc9481544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94659c700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9480c3e90 0x7fc948159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 -- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc934004810 msgr2=0x7fc9481544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc934004810 0x7fc9481544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 -- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9480c3e90 msgr2=0x7fc948159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9480c3e90 0x7fc948159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 -- 172.21.15.3:0/2205731024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9481549f0 con 0x7fc9480c4300 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94659c700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9480c3e90 0x7fc948159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:04.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94759e700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480c4300 0x7fc94815a200 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc94815d130 tx=0x7fc940001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc93effd700 1 -- 172.21.15.3:0/2205731024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94000f630 con 0x7fc9480c4300 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc93effd700 1 -- 172.21.15.3:0/2205731024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94000f790 con 0x7fc9480c4300 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc93effd700 1 -- 172.21.15.3:0/2205731024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94000fb50 con 0x7fc9480c4300 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94815a810 con 0x7fc9480c4300 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.175+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94815b710 con 0x7fc9480c4300 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.176+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94815bc10 con 0x7fc9480c4300 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.177+0000 7fc93effd700 1 -- 172.21.15.3:0/2205731024 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc94001e5c0 con 0x7fc9480c4300 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.177+0000 7fc93effd700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc928066f30 0x7fc9280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.177+0000 7fc93effd700 1 -- 172.21.15.3:0/2205731024 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc940084440 con 0x7fc9480c4300 2023-10-06T04:58:04.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.177+0000 7fc946d9d700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc928066f30 0x7fc9280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.178+0000 7fc946d9d700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc928066f30 0x7fc9280693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc9380094d0 tx=0x7fc93800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.181+0000 7fc93effd700 1 -- 172.21.15.3:0/2205731024 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc940051550 con 0x7fc9480c4300 2023-10-06T04:58:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.417+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc948071680 con 0x7fc9480c4300 2023-10-06T04:58:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.418+0000 7fc93effd700 1 -- 172.21.15.3:0/2205731024 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fc940054b70 con 0x7fc9480c4300 2023-10-06T04:58:04.421 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:04.421 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:51.531356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:58:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc928066f30 msgr2=0x7fc9280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc928066f30 0x7fc9280693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc9380094d0 tx=0x7fc93800b040 comp rx=0 tx=0).stop 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480c4300 msgr2=0x7fc94815a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480c4300 0x7fc94815a200 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc94815d130 tx=0x7fc940001e70 comp rx=0 tx=0).stop 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 shutdown_connections 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc928066f30 0x7fc9280693e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480c4300 0x7fc94815a200 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc934004810 0x7fc9481544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 --2- 172.21.15.3:0/2205731024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9480c3e90 0x7fc948159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 >> 172.21.15.3:0/2205731024 conn(0x7fc948070310 msgr2=0x7fc9480b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 shutdown_connections 2023-10-06T04:58:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.420+0000 7fc94d16f700 1 -- 172.21.15.3:0/2205731024 wait complete. 2023-10-06T04:58:04.424 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2023-10-06T04:58:04.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.553+0000 7ffac56d4700 1 Processor -- start 2023-10-06T04:58:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.553+0000 7ffac56d4700 1 -- start start 2023-10-06T04:58:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffac56d4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffac00cb3b0 0x7ffac00cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffac56d4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac00cb780 0x7ffac0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffac56d4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac00cf130 con 0x7ffac0159500 2023-10-06T04:58:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffac56d4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00ce520 con 0x7ffac00cb3b0 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffac56d4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00ce800 con 0x7ffac00cb780 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffac4ed3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffac0159500 0x7ffac00cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48598/0 (socket says 172.21.15.3:48598) 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffac4ed3700 1 -- 172.21.15.3:0/387108558 learned_addr learned my addr 172.21.15.3:0/387108558 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabf7fe700 1 --2- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac00cb780 0x7ffac0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabffff700 1 --2- 172.21.15.3:0/387108558 >> v2:172.21.15.153:3300/0 conn(0x7ffac00cb3b0 0x7ffac00cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabeffd700 1 -- 172.21.15.3:0/387108558 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685069012 0 0) 0x7ffac00cf130 con 0x7ffac0159500 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabeffd700 1 -- 172.21.15.3:0/387108558 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa8003570 con 0x7ffac0159500 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabffff700 1 -- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac00cb780 msgr2=0x7ffac0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabffff700 1 --2- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac00cb780 0x7ffac0158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabffff700 1 -- 172.21.15.3:0/387108558 >> v1:172.21.15.3:6789/0 conn(0x7ffac0159500 legacy=0x7ffac00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabffff700 1 -- 172.21.15.3:0/387108558 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00ceae0 con 0x7ffac00cb3b0 2023-10-06T04:58:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabf7fe700 1 --2- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac00cb780 0x7ffac0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.554+0000 7ffabffff700 1 --2- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 0x7ffac00cf5c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffab00098e0 tx=0x7ffab00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1b9ccff53b7d50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffabeffd700 1 -- 172.21.15.3:0/387108558 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0011630 con 0x7ffac00cb3b0 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffabeffd700 1 -- 172.21.15.3:0/387108558 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab0011790 con 0x7ffac00cb3b0 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffabeffd700 1 -- 172.21.15.3:0/387108558 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0011b30 con 0x7ffac00cb3b0 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 -- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 msgr2=0x7ffac00cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 --2- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 0x7ffac00cf5c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffab00098e0 tx=0x7ffab00091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 -- 172.21.15.3:0/387108558 shutdown_connections 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 --2- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 0x7ffac00cf5c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 --2- 172.21.15.3:0/387108558 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac00cb780 0x7ffac0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 -- 172.21.15.3:0/387108558 >> 172.21.15.3:0/387108558 conn(0x7ffac0070310 msgr2=0x7ffac0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 -- 172.21.15.3:0/387108558 shutdown_connections 2023-10-06T04:58:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.555+0000 7ffac56d4700 1 -- 172.21.15.3:0/387108558 wait complete. 2023-10-06T04:58:04.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.556+0000 7ffac56d4700 1 Processor -- start 2023-10-06T04:58:04.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.556+0000 7ffac56d4700 1 -- start start 2023-10-06T04:58:04.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffac56d4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 0x7ffac01529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffac56d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffac00cb780 0x7ffac0155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffac56d4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac0159500 0x7ffac00c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffac56d4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac016aaf0 con 0x7ffac00cb780 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffac56d4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0169ee0 con 0x7ffac00cb3b0 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffac56d4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac016a1c0 con 0x7ffac0159500 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffabf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffac00cb780 0x7ffac0155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffabf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffac00cb780 0x7ffac0155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52818/0 (socket says 172.21.15.3:52818) 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffabf7fe700 1 -- 172.21.15.3:0/2391373793 learned_addr learned my addr 172.21.15.3:0/2391373793 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffabffff700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 0x7ffac01529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.557+0000 7ffac4ed3700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac0159500 0x7ffac00c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffabf7fe700 1 -- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac0159500 msgr2=0x7ffac00c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffabf7fe700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac0159500 0x7ffac00c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffabf7fe700 1 -- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 msgr2=0x7ffac01529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffabf7fe700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 0x7ffac01529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffabf7fe700 1 -- 172.21.15.3:0/2391373793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab4009420 con 0x7ffac00cb780 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffac4ed3700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac0159500 0x7ffac00c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffabf7fe700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffac00cb780 0x7ffac0155fa0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffab400ec80 tx=0x7ffab40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffaa7fff700 1 -- 172.21.15.3:0/2391373793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab400c9d0 con 0x7ffac00cb780 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffaa7fff700 1 -- 172.21.15.3:0/2391373793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab400cb30 con 0x7ffac00cb780 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab0009420 con 0x7ffac00cb780 2023-10-06T04:58:04.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffaa7fff700 1 -- 172.21.15.3:0/2391373793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab400ceb0 con 0x7ffac00cb780 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.558+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac016a780 con 0x7ffac00cb780 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.559+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac004c380 con 0x7ffac00cb780 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.559+0000 7ffaa7fff700 1 -- 172.21.15.3:0/2391373793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffab4013680 con 0x7ffac00cb780 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.560+0000 7ffaa7fff700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffaac066f30 0x7ffaac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.560+0000 7ffaa7fff700 1 -- 172.21.15.3:0/2391373793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffab4084b10 con 0x7ffac00cb780 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.560+0000 7ffabffff700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffaac066f30 0x7ffaac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.560+0000 7ffabffff700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffaac066f30 0x7ffaac0693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffab00091e0 tx=0x7ffab0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.564+0000 7ffaa7fff700 1 -- 172.21.15.3:0/2391373793 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffab4051c20 con 0x7ffac00cb780 2023-10-06T04:58:04.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.760+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffac015cbf0 con 0x7ffac00cb780 2023-10-06T04:58:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.760+0000 7ffaa7fff700 1 -- 172.21.15.3:0/2391373793 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16821 (secure 0 0 0) 0x7ffab4017090 con 0x7ffac00cb780 2023-10-06T04:58:04.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:04.763 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:31.630582+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:55:27.781764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T04:55:29.007164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffaac066f30 msgr2=0x7ffaac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffaac066f30 0x7ffaac0693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffab00091e0 tx=0x7ffab0017000 comp rx=0 tx=0).stop 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffac00cb780 msgr2=0x7ffac0155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffac00cb780 0x7ffac0155fa0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffab400ec80 tx=0x7ffab40091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 shutdown_connections 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffaac066f30 0x7ffaac0693e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffac00cb3b0 0x7ffac01529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffac0159500 0x7ffac00c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 --2- 172.21.15.3:0/2391373793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffac00cb780 0x7ffac0155fa0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 >> 172.21.15.3:0/2391373793 conn(0x7ffac0070310 msgr2=0x7ffac0153870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 shutdown_connections 2023-10-06T04:58:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.763+0000 7ffac56d4700 1 -- 172.21.15.3:0/2391373793 wait complete. 2023-10-06T04:58:04.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:55:27.781764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:55:29.007164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:58:04.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:58:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.897+0000 7f72269b6700 1 Processor -- start 2023-10-06T04:58:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.897+0000 7f72269b6700 1 -- start start 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.898+0000 7f72269b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7220140e70 0x7f72200ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.898+0000 7f72269b6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f72200b16c0 0x7f7220067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.898+0000 7f72269b6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7220069f10 con 0x7f72200cd440 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.898+0000 7f72269b6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72200cca70 con 0x7f72200b16c0 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.898+0000 7f72269b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72200cc140 con 0x7f7220140e70 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.898+0000 7f721f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f72200cd440 0x7f72200b0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48614/0 (socket says 172.21.15.3:48614) 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.898+0000 7f721f7fe700 1 -- 172.21.15.3:0/4270745247 learned_addr learned my addr 172.21.15.3:0/4270745247 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f7224f53700 1 --2- 172.21.15.3:0/4270745247 >> v2:172.21.15.153:3300/0 conn(0x7f72200b16c0 0x7f7220067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721ffff700 1 --2- 172.21.15.3:0/4270745247 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7220140e70 0x7f72200ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721effd700 1 -- 172.21.15.3:0/4270745247 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314597774 0 0) 0x7f7220069f10 con 0x7f72200cd440 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721effd700 1 -- 172.21.15.3:0/4270745247 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f720c003570 con 0x7f72200cd440 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721ffff700 1 -- 172.21.15.3:0/4270745247 >> v2:172.21.15.153:3300/0 conn(0x7f72200b16c0 msgr2=0x7f7220067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721ffff700 1 --2- 172.21.15.3:0/4270745247 >> v2:172.21.15.153:3300/0 conn(0x7f72200b16c0 0x7f7220067ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721ffff700 1 -- 172.21.15.3:0/4270745247 >> v1:172.21.15.3:6789/0 conn(0x7f72200cd440 legacy=0x7f72200b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721ffff700 1 -- 172.21.15.3:0/4270745247 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200cc420 con 0x7f7220140e70 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721ffff700 1 --2- 172.21.15.3:0/4270745247 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7220140e70 0x7f72200ccf00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f72100098e0 tx=0x7f72100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a1bbf5d9f2b21e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721effd700 1 -- 172.21.15.3:0/4270745247 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210011630 con 0x7f7220140e70 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721effd700 1 -- 172.21.15.3:0/4270745247 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7210011790 con 0x7f7220140e70 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.899+0000 7f721effd700 1 -- 172.21.15.3:0/4270745247 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210011b30 con 0x7f7220140e70 2023-10-06T04:58:04.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 -- 172.21.15.3:0/4270745247 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7220140e70 msgr2=0x7f72200ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 --2- 172.21.15.3:0/4270745247 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7220140e70 0x7f72200ccf00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f72100098e0 tx=0x7f72100091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 -- 172.21.15.3:0/4270745247 shutdown_connections 2023-10-06T04:58:04.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 --2- 172.21.15.3:0/4270745247 >> v2:172.21.15.153:3300/0 conn(0x7f72200b16c0 0x7f7220067ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 --2- 172.21.15.3:0/4270745247 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7220140e70 0x7f72200ccf00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 -- 172.21.15.3:0/4270745247 >> 172.21.15.3:0/4270745247 conn(0x7f72200ce210 msgr2=0x7f72200ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:04.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 -- 172.21.15.3:0/4270745247 shutdown_connections 2023-10-06T04:58:04.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.900+0000 7f72269b6700 1 -- 172.21.15.3:0/4270745247 wait complete. 2023-10-06T04:58:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.901+0000 7f72269b6700 1 Processor -- start 2023-10-06T04:58:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.901+0000 7f72269b6700 1 -- start start 2023-10-06T04:58:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f72269b6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72200b16c0 0x7f72200a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f72269b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72200cd440 0x7f722013f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f72269b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7220140e70 0x7f72200c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f72269b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722015cb80 con 0x7f7220140e70 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f72269b6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722016c150 con 0x7f72200b16c0 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f72269b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722015c5c0 con 0x7f72200cd440 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7220140e70 0x7f72200c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7220140e70 0x7f72200c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52822/0 (socket says 172.21.15.3:52822) 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 -- 172.21.15.3:0/2229650520 learned_addr learned my addr 172.21.15.3:0/2229650520 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f721f7fe700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72200cd440 0x7f722013f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f721ffff700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72200b16c0 0x7f72200a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 -- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72200cd440 msgr2=0x7f722013f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72200cd440 0x7f722013f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 -- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72200b16c0 msgr2=0x7f72200a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72200b16c0 0x7f72200a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 -- 172.21.15.3:0/2229650520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210009420 con 0x7f7220140e70 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.902+0000 7f7224f53700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7220140e70 0x7f72200c7410 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7218009ec0 tx=0x7f721800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.903+0000 7f7217fff700 1 -- 172.21.15.3:0/2229650520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800cb70 con 0x7f7220140e70 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.903+0000 7f7217fff700 1 -- 172.21.15.3:0/2229650520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721800f040 con 0x7f7220140e70 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.903+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722015c8a0 con 0x7f7220140e70 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.903+0000 7f7217fff700 1 -- 172.21.15.3:0/2229650520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218014480 con 0x7f7220140e70 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.903+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722015a010 con 0x7f7220140e70 2023-10-06T04:58:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.904+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722004c380 con 0x7f7220140e70 2023-10-06T04:58:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.904+0000 7f7217fff700 1 -- 172.21.15.3:0/2229650520 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72180147f0 con 0x7f7220140e70 2023-10-06T04:58:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.904+0000 7f7217fff700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.905+0000 7f7217fff700 1 -- 172.21.15.3:0/2229650520 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7218085290 con 0x7f7220140e70 2023-10-06T04:58:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.905+0000 7f721ffff700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.905+0000 7f721ffff700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f72100091e0 tx=0x7f7210017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:04.909+0000 7f7217fff700 1 -- 172.21.15.3:0/2229650520 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72180523a0 con 0x7f7220140e70 2023-10-06T04:58:05.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.103+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f72200cc700 con 0x7f7220140e70 2023-10-06T04:58:05.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.103+0000 7f7217fff700 1 -- 172.21.15.3:0/2229650520 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7f72180559c0 con 0x7f7220140e70 2023-10-06T04:58:05.106 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:58:05.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 msgr2=0x7f72000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f72100091e0 tx=0x7f7210017000 comp rx=0 tx=0).stop 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7220140e70 msgr2=0x7f72200c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7220140e70 0x7f72200c7410 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7218009ec0 tx=0x7f721800b040 comp rx=0 tx=0).stop 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 shutdown_connections 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72200b16c0 0x7f72200a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7220140e70 0x7f72200c7410 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 --2- 172.21.15.3:0/2229650520 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72200cd440 0x7f722013f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 >> 172.21.15.3:0/2229650520 conn(0x7f72200ce210 msgr2=0x7f72200b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:05.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 shutdown_connections 2023-10-06T04:58:05.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.105+0000 7f72269b6700 1 -- 172.21.15.3:0/2229650520 wait complete. 2023-10-06T04:58:05.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:58:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.239+0000 7f502c0dc700 1 Processor -- start 2023-10-06T04:58:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.239+0000 7f502c0dc700 1 -- start start 2023-10-06T04:58:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f502c0dc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50240c9fb0 0x7f502415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f502c0dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50240c76c0 0x7f50240c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f502c0dc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502406ae70 con 0x7f502415b270 2023-10-06T04:58:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f502c0dc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50240c8020 con 0x7f50240c76c0 2023-10-06T04:58:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f502c0dc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502415a620 con 0x7f50240c9fb0 2023-10-06T04:58:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f502a679700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50240c76c0 0x7f50240c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f5029677700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f502415b270 0x7f5024159b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48620/0 (socket says 172.21.15.3:48620) 2023-10-06T04:58:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f5029677700 1 -- 172.21.15.3:0/3121277870 learned_addr learned my addr 172.21.15.3:0/3121277870 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.240+0000 7f5029e78700 1 --2- 172.21.15.3:0/3121277870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50240c9fb0 0x7f502415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f502a679700 1 -- 172.21.15.3:0/3121277870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50240c9fb0 msgr2=0x7f502415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f502a679700 1 --2- 172.21.15.3:0/3121277870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50240c9fb0 0x7f502415ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f502a679700 1 -- 172.21.15.3:0/3121277870 >> v1:172.21.15.3:6789/0 conn(0x7f502415b270 legacy=0x7f5024159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f502a679700 1 -- 172.21.15.3:0/3121277870 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502415a900 con 0x7f50240c76c0 2023-10-06T04:58:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f5028e76700 1 -- 172.21.15.3:0/3121277870 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461454501 0 0) 0x7f502406ae70 con 0x7f502415b270 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f5029e78700 1 --2- 172.21.15.3:0/3121277870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50240c9fb0 0x7f502415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f502a679700 1 --2- 172.21.15.3:0/3121277870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c7ae0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f501c009e60 tx=0x7f501c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924cc9b4956c0d49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f5028e76700 1 -- 172.21.15.3:0/3121277870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00e070 con 0x7f50240c76c0 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f5028e76700 1 -- 172.21.15.3:0/3121277870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c00bc30 con 0x7f50240c76c0 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f5028e76700 1 -- 172.21.15.3:0/3121277870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c012430 con 0x7f50240c76c0 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.241+0000 7f502c0dc700 1 -- 172.21.15.3:0/3121277870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 msgr2=0x7f50240c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.242+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3121277870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c7ae0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f501c009e60 tx=0x7f501c009780 comp rx=0 tx=0).stop 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.242+0000 7f502c0dc700 1 -- 172.21.15.3:0/3121277870 shutdown_connections 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.242+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3121277870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c7ae0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.242+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3121277870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50240c9fb0 0x7f502415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.242+0000 7f502c0dc700 1 -- 172.21.15.3:0/3121277870 >> 172.21.15.3:0/3121277870 conn(0x7f502406d480 msgr2=0x7f502406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.242+0000 7f502c0dc700 1 -- 172.21.15.3:0/3121277870 shutdown_connections 2023-10-06T04:58:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.242+0000 7f502c0dc700 1 -- 172.21.15.3:0/3121277870 wait complete. 2023-10-06T04:58:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.243+0000 7f502c0dc700 1 Processor -- start 2023-10-06T04:58:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.243+0000 7f502c0dc700 1 -- start start 2023-10-06T04:58:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502c0dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502c0dc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50240c9fb0 0x7f5024152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502c0dc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f502415b270 0x7f502414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502c0dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c25e0 con 0x7f50240c76c0 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502c0dc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c0b80 con 0x7f50240c9fb0 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502c0dc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c0e60 con 0x7f502415b270 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52840/0 (socket says 172.21.15.3:52840) 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 -- 172.21.15.3:0/3000760821 learned_addr learned my addr 172.21.15.3:0/3000760821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502a679700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f502415b270 0x7f502414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029677700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50240c9fb0 0x7f5024152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 -- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f502415b270 msgr2=0x7f502414d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f502415b270 0x7f502414d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 -- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50240c9fb0 msgr2=0x7f5024152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50240c9fb0 0x7f5024152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 -- 172.21.15.3:0/3000760821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c009a20 con 0x7f50240c76c0 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029677700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50240c9fb0 0x7f5024152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5029e78700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c20a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f502000e8d0 tx=0x7f5020008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5019ffb700 1 -- 172.21.15.3:0/3000760821 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502000ee70 con 0x7f50240c76c0 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5019ffb700 1 -- 172.21.15.3:0/3000760821 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502000c5e0 con 0x7f50240c76c0 2023-10-06T04:58:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50240c1140 con 0x7f50240c76c0 2023-10-06T04:58:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.244+0000 7f5019ffb700 1 -- 172.21.15.3:0/3000760821 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502000c980 con 0x7f50240c76c0 2023-10-06T04:58:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.245+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50240c1700 con 0x7f50240c76c0 2023-10-06T04:58:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.245+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502404c380 con 0x7f50240c76c0 2023-10-06T04:58:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.246+0000 7f5019ffb700 1 -- 172.21.15.3:0/3000760821 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5020013410 con 0x7f50240c76c0 2023-10-06T04:58:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.246+0000 7f5019ffb700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5008066f30 0x7f50080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.246+0000 7f5019ffb700 1 -- 172.21.15.3:0/3000760821 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f50200845d0 con 0x7f50240c76c0 2023-10-06T04:58:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.246+0000 7f5029677700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5008066f30 0x7f50080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.247+0000 7f5029677700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5008066f30 0x7f50080693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5014001810 tx=0x7f5014009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.250+0000 7f5019ffb700 1 -- 172.21.15.3:0/3000760821 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5020051640 con 0x7f50240c76c0 2023-10-06T04:58:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.442+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f502406ab00 con 0x7f50240c76c0 2023-10-06T04:58:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.443+0000 7f5019ffb700 1 -- 172.21.15.3:0/3000760821 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5020054c60 con 0x7f50240c76c0 2023-10-06T04:58:05.446 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5008066f30 msgr2=0x7f50080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5008066f30 0x7f50080693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5014001810 tx=0x7f5014009040 comp rx=0 tx=0).stop 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 msgr2=0x7f50240c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c20a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f502000e8d0 tx=0x7f5020008040 comp rx=0 tx=0).stop 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 shutdown_connections 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5008066f30 0x7f50080693e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f502415b270 0x7f502414d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50240c76c0 0x7f50240c20a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 --2- 172.21.15.3:0/3000760821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50240c9fb0 0x7f5024152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 >> 172.21.15.3:0/3000760821 conn(0x7f502406d480 msgr2=0x7f5024063570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 shutdown_connections 2023-10-06T04:58:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.445+0000 7f502c0dc700 1 -- 172.21.15.3:0/3000760821 wait complete. 2023-10-06T04:58:05.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:58:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.579+0000 7fbfbd32f700 1 Processor -- start 2023-10-06T04:58:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.579+0000 7fbfbd32f700 1 -- start start 2023-10-06T04:58:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfbd32f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbfb8159eb0 0x7fbfb80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfbd32f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb80afca0 con 0x7fbfb80cfb80 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfbd32f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb815d510 con 0x7fbfb80cef10 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfbd32f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb815a480 con 0x7fbfb8159eb0 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb67fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbfb8159eb0 0x7fbfb80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb6ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbfb80cfb80 0x7fbfb80af590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48624/0 (socket says 172.21.15.3:48624) 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb6ffd700 1 -- 172.21.15.3:0/3795082582 learned_addr learned my addr 172.21.15.3:0/3795082582 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb67fc700 1 -- 172.21.15.3:0/3795082582 >> v1:172.21.15.153:6789/0 conn(0x7fbfb80cef10 legacy=0x7fbfb815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb67fc700 1 -- 172.21.15.3:0/3795082582 >> v1:172.21.15.3:6789/0 conn(0x7fbfb80cfb80 legacy=0x7fbfb80af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb67fc700 1 -- 172.21.15.3:0/3795082582 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb815a760 con 0x7fbfb8159eb0 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb5ffb700 1 -- 172.21.15.3:0/3795082582 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660044940 0 0) 0x7fbfb80afca0 con 0x7fbfb80cfb80 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb67fc700 1 --2- 172.21.15.3:0/3795082582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb8159eb0 0x7fbfb80ce9d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00098e0 tx=0x7fbfa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5802161d81ecdde6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.580+0000 7fbfb5ffb700 1 -- 172.21.15.3:0/3795082582 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0009e80 con 0x7fbfb8159eb0 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfb5ffb700 1 -- 172.21.15.3:0/3795082582 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa000b820 con 0x7fbfb8159eb0 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfb5ffb700 1 -- 172.21.15.3:0/3795082582 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000bba0 con 0x7fbfb8159eb0 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfbd32f700 1 -- 172.21.15.3:0/3795082582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb8159eb0 msgr2=0x7fbfb80ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/3795082582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb8159eb0 0x7fbfb80ce9d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00098e0 tx=0x7fbfa00091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfbd32f700 1 -- 172.21.15.3:0/3795082582 shutdown_connections 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/3795082582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb8159eb0 0x7fbfb80ce9d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfbd32f700 1 -- 172.21.15.3:0/3795082582 >> 172.21.15.3:0/3795082582 conn(0x7fbfb8070310 msgr2=0x7fbfb8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfbd32f700 1 -- 172.21.15.3:0/3795082582 shutdown_connections 2023-10-06T04:58:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.581+0000 7fbfbd32f700 1 -- 172.21.15.3:0/3795082582 wait complete. 2023-10-06T04:58:05.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.582+0000 7fbfbd32f700 1 Processor -- start 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.582+0000 7fbfbd32f700 1 -- start start 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfbd32f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb80cef10 0x7fbfb8153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfbd32f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb80cfb80 0x7fbfb8152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfbd32f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb8159eb0 0x7fbfb8156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfbd32f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8154c70 con 0x7fbfb80cfb80 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfbd32f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8153dc0 con 0x7fbfb80cef10 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfbd32f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8154060 con 0x7fbfb8159eb0 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb80cfb80 0x7fbfb8152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb80cfb80 0x7fbfb8152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52854/0 (socket says 172.21.15.3:52854) 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 -- 172.21.15.3:0/228593882 learned_addr learned my addr 172.21.15.3:0/228593882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb6ffd700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb80cef10 0x7fbfb8153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb77fe700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb8159eb0 0x7fbfb8156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 -- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb8159eb0 msgr2=0x7fbfb8156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb8159eb0 0x7fbfb8156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 -- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb80cef10 msgr2=0x7fbfb8153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb80cef10 0x7fbfb8153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb67fc700 1 -- 172.21.15.3:0/228593882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0009420 con 0x7fbfb80cfb80 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.583+0000 7fbfb77fe700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb8159eb0 0x7fbfb8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.584+0000 7fbfb67fc700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb80cfb80 0x7fbfb8152ee0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00030c0 tx=0x7fbfa0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.584+0000 7fbfaeffd700 1 -- 172.21.15.3:0/228593882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000ba10 con 0x7fbfb80cfb80 2023-10-06T04:58:05.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.584+0000 7fbfaeffd700 1 -- 172.21.15.3:0/228593882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa0020070 con 0x7fbfb80cfb80 2023-10-06T04:58:05.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.584+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb8154340 con 0x7fbfb80cfb80 2023-10-06T04:58:05.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.584+0000 7fbfaeffd700 1 -- 172.21.15.3:0/228593882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa001b6a0 con 0x7fbfb80cfb80 2023-10-06T04:58:05.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.584+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb8154900 con 0x7fbfb80cfb80 2023-10-06T04:58:05.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.585+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb8155000 con 0x7fbfb80cfb80 2023-10-06T04:58:05.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.585+0000 7fbfaeffd700 1 -- 172.21.15.3:0/228593882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfa0022030 con 0x7fbfb80cfb80 2023-10-06T04:58:05.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.586+0000 7fbfaeffd700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf98066f30 0x7fbf980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.586+0000 7fbfaeffd700 1 -- 172.21.15.3:0/228593882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbfa0086750 con 0x7fbfb80cfb80 2023-10-06T04:58:05.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.586+0000 7fbfb6ffd700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf98066f30 0x7fbf980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.586+0000 7fbfb6ffd700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf98066f30 0x7fbf980693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00011e0 tx=0x7fbfb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.590+0000 7fbfaeffd700 1 -- 172.21.15.3:0/228593882 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfb8155000 con 0x7fbfb80cfb80 2023-10-06T04:58:05.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.787+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfb8155000 con 0x7fbfb80cfb80 2023-10-06T04:58:05.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.788+0000 7fbfaeffd700 1 -- 172.21.15.3:0/228593882 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbfb8155000 con 0x7fbfb80cfb80 2023-10-06T04:58:05.791 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:58:05.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf98066f30 msgr2=0x7fbf980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf98066f30 0x7fbf980693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00011e0 tx=0x7fbfb0009040 comp rx=0 tx=0).stop 2023-10-06T04:58:05.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb80cfb80 msgr2=0x7fbfb8152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb80cfb80 0x7fbfb8152ee0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00030c0 tx=0x7fbfa0016000 comp rx=0 tx=0).stop 2023-10-06T04:58:05.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 shutdown_connections 2023-10-06T04:58:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf98066f30 0x7fbf980693e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb80cef10 0x7fbfb8153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb8159eb0 0x7fbfb8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 --2- 172.21.15.3:0/228593882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfb80cfb80 0x7fbfb8152ee0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 >> 172.21.15.3:0/228593882 conn(0x7fbfb8070310 msgr2=0x7fbfb80af190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 shutdown_connections 2023-10-06T04:58:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.790+0000 7fbfbd32f700 1 -- 172.21.15.3:0/228593882 wait complete. 2023-10-06T04:58:05.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:58:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.926+0000 7f5e811f4700 1 Processor -- start 2023-10-06T04:58:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.926+0000 7f5e811f4700 1 -- start start 2023-10-06T04:58:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e811f4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5e7c0af290 0x7f5e7c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e811f4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e7c0af660 0x7f5e7c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e811f4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c0cb190 con 0x7f5e7c0cd190 2023-10-06T04:58:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e811f4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c159950 con 0x7f5e7c0af660 2023-10-06T04:58:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e811f4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0cd6a0 con 0x7f5e7c0af290 2023-10-06T04:58:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e7c0af660 0x7f5e7c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e809f3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5e7c0cd190 0x7f5e7c0c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48632/0 (socket says 172.21.15.3:48632) 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e809f3700 1 -- 172.21.15.3:0/409684173 learned_addr learned my addr 172.21.15.3:0/409684173 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7bfff700 1 --2- 172.21.15.3:0/409684173 >> v2:172.21.15.153:3300/0 conn(0x7f5e7c0af290 0x7f5e7c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7b7fe700 1 -- 172.21.15.3:0/409684173 >> v2:172.21.15.153:3300/0 conn(0x7f5e7c0af290 msgr2=0x7f5e7c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7b7fe700 1 --2- 172.21.15.3:0/409684173 >> v2:172.21.15.153:3300/0 conn(0x7f5e7c0af290 0x7f5e7c0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7b7fe700 1 -- 172.21.15.3:0/409684173 >> v1:172.21.15.3:6789/0 conn(0x7f5e7c0cd190 legacy=0x7f5e7c0c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7b7fe700 1 -- 172.21.15.3:0/409684173 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0cb3f0 con 0x7f5e7c0af660 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7b7fe700 1 --2- 172.21.15.3:0/409684173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af660 0x7f5e7c0ccc50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c089580 tx=0x7f5e6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91301e859789838f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7affd700 1 -- 172.21.15.3:0/409684173 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c011630 con 0x7f5e7c0af660 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7affd700 1 -- 172.21.15.3:0/409684173 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c011790 con 0x7f5e7c0af660 2023-10-06T04:58:05.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.927+0000 7f5e7affd700 1 -- 172.21.15.3:0/409684173 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c011b30 con 0x7f5e7c0af660 2023-10-06T04:58:05.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 -- 172.21.15.3:0/409684173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af660 msgr2=0x7f5e7c0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 --2- 172.21.15.3:0/409684173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af660 0x7f5e7c0ccc50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c089580 tx=0x7f5e6c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 -- 172.21.15.3:0/409684173 shutdown_connections 2023-10-06T04:58:05.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 --2- 172.21.15.3:0/409684173 >> v2:172.21.15.153:3300/0 conn(0x7f5e7c0af290 0x7f5e7c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 --2- 172.21.15.3:0/409684173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af660 0x7f5e7c0ccc50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 -- 172.21.15.3:0/409684173 >> 172.21.15.3:0/409684173 conn(0x7f5e7c089dd0 msgr2=0x7f5e7c0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:05.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 -- 172.21.15.3:0/409684173 shutdown_connections 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.928+0000 7f5e811f4700 1 -- 172.21.15.3:0/409684173 wait complete. 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.929+0000 7f5e811f4700 1 Processor -- start 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.929+0000 7f5e811f4700 1 -- start start 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e811f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af290 0x7f5e7c0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e811f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e7c0af660 0x7f5e7c16a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e811f4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e7c0cd190 0x7f5e7c14cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e811f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b29c0 con 0x7f5e7c0af290 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e811f4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b1ff0 con 0x7f5e7c0af660 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e811f4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b5f30 con 0x7f5e7c0cd190 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af290 0x7f5e7c0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af290 0x7f5e7c0b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52862/0 (socket says 172.21.15.3:52862) 2023-10-06T04:58:05.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 -- 172.21.15.3:0/1631752832 learned_addr learned my addr 172.21.15.3:0/1631752832 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7b7fe700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e7c0af660 0x7f5e7c16a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e809f3700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e7c0cd190 0x7f5e7c14cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 -- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e7c0cd190 msgr2=0x7f5e7c14cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e7c0cd190 0x7f5e7c14cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 -- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e7c0af660 msgr2=0x7f5e7c16a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e7c0af660 0x7f5e7c16a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.930+0000 7f5e7bfff700 1 -- 172.21.15.3:0/1631752832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0093c0 con 0x7f5e7c0af290 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.931+0000 7f5e7bfff700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af290 0x7f5e7c0b2480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000ecb0 tx=0x7f5e70009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.931+0000 7f5e63fff700 1 -- 172.21.15.3:0/1631752832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000c9d0 con 0x7f5e7c0af290 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.931+0000 7f5e63fff700 1 -- 172.21.15.3:0/1631752832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000cb30 con 0x7f5e7c0af290 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.931+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c16a5f0 con 0x7f5e7c0af290 2023-10-06T04:58:05.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.931+0000 7f5e63fff700 1 -- 172.21.15.3:0/1631752832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000ceb0 con 0x7f5e7c0af290 2023-10-06T04:58:05.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.931+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0b1740 con 0x7f5e7c0af290 2023-10-06T04:58:05.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.932+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c04c380 con 0x7f5e7c0af290 2023-10-06T04:58:05.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.932+0000 7f5e63fff700 1 -- 172.21.15.3:0/1631752832 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e70013680 con 0x7f5e7c0af290 2023-10-06T04:58:05.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.933+0000 7f5e63fff700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e68066f30 0x7f5e680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:05.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.933+0000 7f5e63fff700 1 -- 172.21.15.3:0/1631752832 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e70084d90 con 0x7f5e7c0af290 2023-10-06T04:58:05.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.933+0000 7f5e7b7fe700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e68066f30 0x7f5e680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:05.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.933+0000 7f5e7b7fe700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e68066f30 0x7f5e680693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c089580 tx=0x7f5e6c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:05.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:05.937+0000 7f5e63fff700 1 -- 172.21.15.3:0/1631752832 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e70051ea0 con 0x7f5e7c0af290 2023-10-06T04:58:06.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.129+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e7c0d0ed0 con 0x7f5e7c0af290 2023-10-06T04:58:06.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.129+0000 7f5e63fff700 1 -- 172.21.15.3:0/1631752832 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e700210a0 con 0x7f5e7c0af290 2023-10-06T04:58:06.132 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:58:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e68066f30 msgr2=0x7f5e680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e68066f30 0x7f5e680693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c089580 tx=0x7f5e6c017000 comp rx=0 tx=0).stop 2023-10-06T04:58:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af290 msgr2=0x7f5e7c0b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af290 0x7f5e7c0b2480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000ecb0 tx=0x7f5e70009120 comp rx=0 tx=0).stop 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 shutdown_connections 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e68066f30 0x7f5e680693e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e7c0cd190 0x7f5e7c14cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e7c0af290 0x7f5e7c0b2480 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 --2- 172.21.15.3:0/1631752832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e7c0af660 0x7f5e7c16a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.131+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 >> 172.21.15.3:0/1631752832 conn(0x7f5e7c089dd0 msgr2=0x7f5e7c06ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.132+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 shutdown_connections 2023-10-06T04:58:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.132+0000 7f5e811f4700 1 -- 172.21.15.3:0/1631752832 wait complete. 2023-10-06T04:58:06.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.263+0000 7f653edb3700 1 Processor -- start 2023-10-06T04:58:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.263+0000 7f653edb3700 1 -- start start 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.263+0000 7f653edb3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.263+0000 7f653edb3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65380b2740 0x7f65380cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.263+0000 7f653edb3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65380b0ce0 con 0x7f653815cb50 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.263+0000 7f653edb3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65380cb820 con 0x7f65380b2740 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.263+0000 7f653edb3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65380b0760 con 0x7f65380cd950 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653d350700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65380b2740 0x7f65380cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653cb4f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f653815cb50 0x7f65380b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48642/0 (socket says 172.21.15.3:48642) 2023-10-06T04:58:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653cb4f700 1 -- 172.21.15.3:0/3069189476 learned_addr learned my addr 172.21.15.3:0/3069189476 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f6537fff700 1 --2- 172.21.15.3:0/3069189476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653d350700 1 -- 172.21.15.3:0/3069189476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 msgr2=0x7f65380b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653d350700 1 --2- 172.21.15.3:0/3069189476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653d350700 1 -- 172.21.15.3:0/3069189476 >> v1:172.21.15.3:6789/0 conn(0x7f653815cb50 legacy=0x7f65380b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653d350700 1 -- 172.21.15.3:0/3069189476 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380b0a00 con 0x7f65380b2740 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f65377fe700 1 -- 172.21.15.3:0/3069189476 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708383238 0 0) 0x7f65380b0ce0 con 0x7f653815cb50 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f6537fff700 1 --2- 172.21.15.3:0/3069189476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f653d350700 1 --2- 172.21.15.3:0/3069189476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380cbcb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6530009e60 tx=0x7f6530009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4355a88c4c58cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f65377fe700 1 -- 172.21.15.3:0/3069189476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653000e070 con 0x7f65380b2740 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f65377fe700 1 -- 172.21.15.3:0/3069189476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653000bc30 con 0x7f65380b2740 2023-10-06T04:58:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.264+0000 7f65377fe700 1 -- 172.21.15.3:0/3069189476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530012430 con 0x7f65380b2740 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 -- 172.21.15.3:0/3069189476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 msgr2=0x7f65380cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 --2- 172.21.15.3:0/3069189476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380cbcb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6530009e60 tx=0x7f6530009780 comp rx=0 tx=0).stop 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 -- 172.21.15.3:0/3069189476 shutdown_connections 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 --2- 172.21.15.3:0/3069189476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380cbcb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 --2- 172.21.15.3:0/3069189476 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 -- 172.21.15.3:0/3069189476 >> 172.21.15.3:0/3069189476 conn(0x7f6538070310 msgr2=0x7f6538070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 -- 172.21.15.3:0/3069189476 shutdown_connections 2023-10-06T04:58:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.265+0000 7f653edb3700 1 -- 172.21.15.3:0/3069189476 wait complete. 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.266+0000 7f653edb3700 1 Processor -- start 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.266+0000 7f653edb3700 1 -- start start 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653edb3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653edb3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653edb3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f653815cb50 0x7f6538150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653edb3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6538155270 con 0x7f65380b2740 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653edb3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380c43d0 con 0x7f653815cb50 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653edb3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65380c5210 con 0x7f65380cd950 2023-10-06T04:58:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380c3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52872/0 (socket says 172.21.15.3:52872) 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 -- 172.21.15.3:0/1768069114 learned_addr learned my addr 172.21.15.3:0/1768069114 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f6537fff700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653d350700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f653815cb50 0x7f6538150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 -- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 msgr2=0x7f65380c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 -- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f653815cb50 msgr2=0x7f6538150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f653815cb50 0x7f6538150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 -- 172.21.15.3:0/1768069114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6530009a20 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.267+0000 7f653cb4f700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380c3e90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f652000c5e0 tx=0x7f6520007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.268+0000 7f6534ff9700 1 -- 172.21.15.3:0/1768069114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200072e0 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.268+0000 7f6534ff9700 1 -- 172.21.15.3:0/1768069114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520007440 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.268+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65380c4600 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.268+0000 7f6534ff9700 1 -- 172.21.15.3:0/1768069114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200057f0 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.268+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65380c4990 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.269+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65380c54f0 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.269+0000 7f6534ff9700 1 -- 172.21.15.3:0/1768069114 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6520005c40 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.269+0000 7f6534ff9700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6518066f30 0x7f65180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.269+0000 7f6534ff9700 1 -- 172.21.15.3:0/1768069114 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f65200839d0 con 0x7f65380b2740 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.270+0000 7f6537fff700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6518066f30 0x7f65180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.270+0000 7f6537fff700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6518066f30 0x7f65180693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f65380d0810 tx=0x7f6528008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.274+0000 7f6534ff9700 1 -- 172.21.15.3:0/1768069114 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6520050ae0 con 0x7f65380b2740 2023-10-06T04:58:06.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.513+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65380cb1d0 con 0x7f65380b2740 2023-10-06T04:58:06.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.513+0000 7f6534ff9700 1 -- 172.21.15.3:0/1768069114 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f6520017020 con 0x7f65380b2740 2023-10-06T04:58:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:06.516 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:51.531356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6518066f30 msgr2=0x7f65180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6518066f30 0x7f65180693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f65380d0810 tx=0x7f6528008040 comp rx=0 tx=0).stop 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 msgr2=0x7f65380c3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380c3e90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f652000c5e0 tx=0x7f6520007e80 comp rx=0 tx=0).stop 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 shutdown_connections 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6518066f30 0x7f65180693e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f653815cb50 0x7f6538150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65380b2740 0x7f65380c3e90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 --2- 172.21.15.3:0/1768069114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65380cd950 0x7f65380c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 >> 172.21.15.3:0/1768069114 conn(0x7f6538070310 msgr2=0x7f65380b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:06.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 shutdown_connections 2023-10-06T04:58:06.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.515+0000 7f653edb3700 1 -- 172.21.15.3:0/1768069114 wait complete. 2023-10-06T04:58:06.519 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2023-10-06T04:58:06.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:06.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.646+0000 7f72d08ff700 1 Processor -- start 2023-10-06T04:58:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.646+0000 7f72d08ff700 1 -- start start 2023-10-06T04:58:06.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72d08ff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72cc157420 0x7f72cc15b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72d08ff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc15be60 con 0x7f72cc157050 2023-10-06T04:58:06.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72d08ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc067f10 con 0x7f72cc0b2780 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72d08ff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc06cc80 con 0x7f72cc157420 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72cb59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72cc157420 0x7f72cc15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72cad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f72cc157050 0x7f72cc0b2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48658/0 (socket says 172.21.15.3:48658) 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72cad9d700 1 -- 172.21.15.3:0/2168490774 learned_addr learned my addr 172.21.15.3:0/2168490774 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72cb59e700 1 -- 172.21.15.3:0/2168490774 >> v1:172.21.15.153:6789/0 conn(0x7f72cc0b2780 legacy=0x7f72cc0cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72cb59e700 1 -- 172.21.15.3:0/2168490774 >> v1:172.21.15.3:6789/0 conn(0x7f72cc157050 legacy=0x7f72cc0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.647+0000 7f72cb59e700 1 -- 172.21.15.3:0/2168490774 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc067390 con 0x7f72cc157420 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72cb59e700 1 --2- 172.21.15.3:0/2168490774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc15b920 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009e60 tx=0x7f72c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0786bbf81f5dc3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72c9d9b700 1 -- 172.21.15.3:0/2168490774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400e070 con 0x7f72cc157420 2023-10-06T04:58:06.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72c9d9b700 1 -- 172.21.15.3:0/2168490774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c400bc30 con 0x7f72cc157420 2023-10-06T04:58:06.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72c9d9b700 1 -- 172.21.15.3:0/2168490774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4012430 con 0x7f72cc157420 2023-10-06T04:58:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72d08ff700 1 -- 172.21.15.3:0/2168490774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 msgr2=0x7f72cc15b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72d08ff700 1 --2- 172.21.15.3:0/2168490774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc15b920 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009e60 tx=0x7f72c4009780 comp rx=0 tx=0).stop 2023-10-06T04:58:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72d08ff700 1 -- 172.21.15.3:0/2168490774 shutdown_connections 2023-10-06T04:58:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72d08ff700 1 --2- 172.21.15.3:0/2168490774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc15b920 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72d08ff700 1 -- 172.21.15.3:0/2168490774 >> 172.21.15.3:0/2168490774 conn(0x7f72cc06d480 msgr2=0x7f72cc06d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72d08ff700 1 -- 172.21.15.3:0/2168490774 shutdown_connections 2023-10-06T04:58:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.648+0000 7f72d08ff700 1 -- 172.21.15.3:0/2168490774 wait complete. 2023-10-06T04:58:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 Processor -- start 2023-10-06T04:58:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 -- start start 2023-10-06T04:58:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc0b2780 0x7f72cc0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72cc157050 0x7f72cc158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0a58a0 con 0x7f72cc157420 2023-10-06T04:58:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0ae720 con 0x7f72cc0b2780 2023-10-06T04:58:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72d08ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0adb10 con 0x7f72cc157050 2023-10-06T04:58:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72cb59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72cb59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52878/0 (socket says 172.21.15.3:52878) 2023-10-06T04:58:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.650+0000 7f72cb59e700 1 -- 172.21.15.3:0/3551011195 learned_addr learned my addr 172.21.15.3:0/3551011195 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72cad9d700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc0b2780 0x7f72cc0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72ca59c700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72cc157050 0x7f72cc158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72cb59e700 1 -- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72cc157050 msgr2=0x7f72cc158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72cb59e700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72cc157050 0x7f72cc158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72cb59e700 1 -- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc0b2780 msgr2=0x7f72cc0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72cb59e700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc0b2780 0x7f72cc0aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72cb59e700 1 -- 172.21.15.3:0/3551011195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4009a20 con 0x7f72cc157420 2023-10-06T04:58:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72cb59e700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc159280 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f72c40097c0 tx=0x7f72c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72b2ffd700 1 -- 172.21.15.3:0/3551011195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4012e10 con 0x7f72cc157420 2023-10-06T04:58:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72b2ffd700 1 -- 172.21.15.3:0/3551011195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c401e630 con 0x7f72cc157420 2023-10-06T04:58:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc0addf0 con 0x7f72cc157420 2023-10-06T04:58:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72b2ffd700 1 -- 172.21.15.3:0/3551011195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c401e9f0 con 0x7f72cc157420 2023-10-06T04:58:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.651+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc0ae3b0 con 0x7f72cc157420 2023-10-06T04:58:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.652+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc04c380 con 0x7f72cc157420 2023-10-06T04:58:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.652+0000 7f72b2ffd700 1 -- 172.21.15.3:0/3551011195 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72c4024070 con 0x7f72cc157420 2023-10-06T04:58:06.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.653+0000 7f72b2ffd700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.653+0000 7f72b2ffd700 1 -- 172.21.15.3:0/3551011195 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f72c4086dd0 con 0x7f72cc157420 2023-10-06T04:58:06.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.653+0000 7f72cad9d700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.654+0000 7f72cad9d700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72b8066f30 0x7f72b80693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f72bc007130 tx=0x7f72bc007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:06.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.657+0000 7f72b2ffd700 1 -- 172.21.15.3:0/3551011195 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72c4053ee0 con 0x7f72cc157420 2023-10-06T04:58:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.851+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72cc070ed0 con 0x7f72cc157420 2023-10-06T04:58:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.852+0000 7f72b2ffd700 1 -- 172.21.15.3:0/3551011195 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f72c4057500 con 0x7f72cc157420 2023-10-06T04:58:06.854 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:06.855 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:31.630582+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:55:27.781764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T04:55:29.007164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72b8066f30 msgr2=0x7f72b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72b8066f30 0x7f72b80693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f72bc007130 tx=0x7f72bc007eb0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 msgr2=0x7f72cc159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc159280 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f72c40097c0 tx=0x7f72c4019000 comp rx=0 tx=0).stop 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 shutdown_connections 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc0b2780 0x7f72cc0aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72cc157420 0x7f72cc159280 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 --2- 172.21.15.3:0/3551011195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72cc157050 0x7f72cc158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.853+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 >> 172.21.15.3:0/3551011195 conn(0x7f72cc06d480 msgr2=0x7f72cc1533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.854+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 shutdown_connections 2023-10-06T04:58:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.854+0000 7f72d08ff700 1 -- 172.21.15.3:0/3551011195 wait complete. 2023-10-06T04:58:06.871 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:58:06.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:06.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.987+0000 7f0e21059700 1 Processor -- start 2023-10-06T04:58:06.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.987+0000 7f0e21059700 1 -- start start 2023-10-06T04:58:06.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.988+0000 7f0e21059700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e1c15c8f0 0x7f0e1c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.988+0000 7f0e21059700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 0x7f0e1c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.988+0000 7f0e21059700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c0cf6c0 con 0x7f0e1c0d06a0 2023-10-06T04:58:06.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.988+0000 7f0e21059700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1c0cfb50 con 0x7f0e1c15c8f0 2023-10-06T04:58:06.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.988+0000 7f0e21059700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c15e050 con 0x7f0e1c15ccc0 2023-10-06T04:58:06.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.988+0000 7f0e1b59e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0e1c0d06a0 0x7f0e1c06d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48660/0 (socket says 172.21.15.3:48660) 2023-10-06T04:58:06.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.988+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1789050666 learned_addr learned my addr 172.21.15.3:0/1789050666 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1a59c700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 0x7f0e1c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1ad9d700 1 --2- 172.21.15.3:0/1789050666 >> v2:172.21.15.153:3300/0 conn(0x7f0e1c15c8f0 0x7f0e1c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782493376 0 0) 0x7f0e1c0cf6c0 con 0x7f0e1c0d06a0 2023-10-06T04:58:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e08003570 con 0x7f0e1c0d06a0 2023-10-06T04:58:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848150132 0 0) 0x7f0e08003570 con 0x7f0e1c0d06a0 2023-10-06T04:58:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 msgr2=0x7f0e1c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 0x7f0e1c0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 >> v2:172.21.15.153:3300/0 conn(0x7f0e1c15c8f0 msgr2=0x7f0e1c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 --2- 172.21.15.3:0/1789050666 >> v2:172.21.15.153:3300/0 conn(0x7f0e1c15c8f0 0x7f0e1c15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0d0af0 con 0x7f0e1c0d06a0 2023-10-06T04:58:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f0e0c001470 con 0x7f0e1c0d06a0 2023-10-06T04:58:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 >> v1:172.21.15.3:6789/0 conn(0x7f0e1c0d06a0 legacy=0x7f0e1c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08003a60 0x7f0e08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 0x7f0e08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e08004d50 0x7f0e080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0cfb50 con 0x7f0e08004d50 2023-10-06T04:58:07.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c15e050 con 0x7f0e08003a60 2023-10-06T04:58:07.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0800ba60 con 0x7f0e1c15ccc0 2023-10-06T04:58:07.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1b59e700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e08004d50 0x7f0e080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1ad9d700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08003a60 0x7f0e08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1a59c700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 0x7f0e08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 msgr2=0x7f0e08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:07.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1b59e700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c15ccc0 0x7f0e08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08003a60 msgr2=0x7f0e08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:07.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1b59e700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08003a60 0x7f0e08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.989+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1789050666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08003570 con 0x7f0e08004d50 2023-10-06T04:58:07.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e1b59e700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e08004d50 0x7f0e080051c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c3c0 tx=0x7f0e0c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85c446d21b2ec84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00cc30 con 0x7f0e08004d50 2023-10-06T04:58:07.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00cdb0 con 0x7f0e08004d50 2023-10-06T04:58:07.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e19d9b700 1 -- 172.21.15.3:0/1789050666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c005b70 con 0x7f0e08004d50 2023-10-06T04:58:07.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e21059700 1 -- 172.21.15.3:0/1789050666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e08004d50 msgr2=0x7f0e080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e21059700 1 --2- 172.21.15.3:0/1789050666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e08004d50 0x7f0e080051c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c3c0 tx=0x7f0e0c001e90 comp rx=0 tx=0).stop 2023-10-06T04:58:07.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1789050666 reap_dead start 2023-10-06T04:58:07.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e21059700 1 -- 172.21.15.3:0/1789050666 shutdown_connections 2023-10-06T04:58:07.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e21059700 1 -- 172.21.15.3:0/1789050666 >> 172.21.15.3:0/1789050666 conn(0x7f0e1c070310 msgr2=0x7f0e1c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:07.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e21059700 1 -- 172.21.15.3:0/1789050666 shutdown_connections 2023-10-06T04:58:07.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.990+0000 7f0e21059700 1 -- 172.21.15.3:0/1789050666 wait complete. 2023-10-06T04:58:07.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.991+0000 7f0e21059700 1 Processor -- start 2023-10-06T04:58:07.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e21059700 1 -- start start 2023-10-06T04:58:07.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e21059700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e1c153e30 0x7f0e1c151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e21059700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c151f70 0x7f0e1c14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e21059700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e1c14e660 0x7f0e1c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e21059700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c14ebf0 con 0x7f0e1c14e660 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e21059700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c159770 con 0x7f0e1c153e30 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e21059700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c158dd0 con 0x7f0e1c151f70 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e1b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e1c14e660 0x7f0e1c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e1b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e1c14e660 0x7f0e1c159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52894/0 (socket says 172.21.15.3:52894) 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.992+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1324850364 learned_addr learned my addr 172.21.15.3:0/1324850364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1a59c700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c151f70 0x7f0e1c14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1ad9d700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e1c153e30 0x7f0e1c151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c151f70 msgr2=0x7f0e1c14e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1b59e700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c151f70 0x7f0e1c14e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e1c153e30 msgr2=0x7f0e1c151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1b59e700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e1c153e30 0x7f0e1c151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1b59e700 1 -- 172.21.15.3:0/1324850364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c1590b0 con 0x7f0e1c14e660 2023-10-06T04:58:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e1b59e700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e1c14e660 0x7f0e1c159b90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c004890 tx=0x7f0e0c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e12ffd700 1 -- 172.21.15.3:0/1324850364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00cc30 con 0x7f0e1c14e660 2023-10-06T04:58:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e12ffd700 1 -- 172.21.15.3:0/1324850364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00c9e0 con 0x7f0e1c14e660 2023-10-06T04:58:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c159390 con 0x7f0e1c14e660 2023-10-06T04:58:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e12ffd700 1 -- 172.21.15.3:0/1324850364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c01e680 con 0x7f0e1c14e660 2023-10-06T04:58:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.993+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c15a0d0 con 0x7f0e1c14e660 2023-10-06T04:58:07.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.994+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c1598b0 con 0x7f0e1c14e660 2023-10-06T04:58:07.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.994+0000 7f0e12ffd700 1 -- 172.21.15.3:0/1324850364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e0c01ea30 con 0x7f0e1c14e660 2023-10-06T04:58:07.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.995+0000 7f0e12ffd700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0dfc066f30 0x7f0dfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.995+0000 7f0e12ffd700 1 -- 172.21.15.3:0/1324850364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e0c085130 con 0x7f0e1c14e660 2023-10-06T04:58:07.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.995+0000 7f0e1ad9d700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0dfc066f30 0x7f0dfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.996+0000 7f0e1ad9d700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0dfc066f30 0x7f0dfc0693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e140094a0 tx=0x7f0e1400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:06.999+0000 7f0e12ffd700 1 -- 172.21.15.3:0/1324850364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e1c1598b0 con 0x7f0e1c14e660 2023-10-06T04:58:07.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.234+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e1c1598b0 con 0x7f0e1c14e660 2023-10-06T04:58:07.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.235+0000 7f0e12ffd700 1 -- 172.21.15.3:0/1324850364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f0e1c1598b0 con 0x7f0e1c14e660 2023-10-06T04:58:07.238 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:07.238 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:51.531356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:58:07.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0dfc066f30 msgr2=0x7f0dfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0dfc066f30 0x7f0dfc0693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e140094a0 tx=0x7f0e1400b040 comp rx=0 tx=0).stop 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e1c14e660 msgr2=0x7f0e1c159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e1c14e660 0x7f0e1c159b90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c004890 tx=0x7f0e0c015000 comp rx=0 tx=0).stop 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 shutdown_connections 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0dfc066f30 0x7f0dfc0693e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e1c153e30 0x7f0e1c151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e1c14e660 0x7f0e1c159b90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 --2- 172.21.15.3:0/1324850364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e1c151f70 0x7f0e1c14e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 >> 172.21.15.3:0/1324850364 conn(0x7f0e1c070310 msgr2=0x7f0e1c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 shutdown_connections 2023-10-06T04:58:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.237+0000 7f0e21059700 1 -- 172.21.15.3:0/1324850364 wait complete. 2023-10-06T04:58:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2023-10-06T04:58:07.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:07.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.370+0000 7f3da8395700 1 Processor -- start 2023-10-06T04:58:07.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.371+0000 7f3da8395700 1 -- start start 2023-10-06T04:58:07.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da8395700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3da00b1b10 0x7f3da00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da8395700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157ad0 0x7f3da0067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da8395700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da00b2e50 con 0x7f3da0157700 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da8395700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da0158290 con 0x7f3da00b1b10 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da8395700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00b1fc0 con 0x7f3da0157ad0 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3da00b1b10 0x7f3da00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3da00b1b10 0x7f3da00cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52904/0 (socket says 172.21.15.3:52904) 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 -- 172.21.15.3:0/259608318 learned_addr learned my addr 172.21.15.3:0/259608318 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da6932700 1 --2- 172.21.15.3:0/259608318 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157ad0 0x7f3da0067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 -- 172.21.15.3:0/259608318 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157ad0 msgr2=0x7f3da0067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 --2- 172.21.15.3:0/259608318 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157ad0 0x7f3da0067b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 -- 172.21.15.3:0/259608318 >> v1:172.21.15.153:6789/0 conn(0x7f3da0157700 legacy=0x7f3da00b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 -- 172.21.15.3:0/259608318 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00b22a0 con 0x7f3da00b1b10 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da5930700 1 --2- 172.21.15.3:0/259608318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da00cc270 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3d940099b0 tx=0x7f3d94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d36ba0cd323b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da512f700 1 -- 172.21.15.3:0/259608318 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400b820 con 0x7f3da00b1b10 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da512f700 1 -- 172.21.15.3:0/259608318 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9400b980 con 0x7f3da00b1b10 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.372+0000 7f3da512f700 1 -- 172.21.15.3:0/259608318 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400bd20 con 0x7f3da00b1b10 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 -- 172.21.15.3:0/259608318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 msgr2=0x7f3da00cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 --2- 172.21.15.3:0/259608318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da00cc270 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3d940099b0 tx=0x7f3d94009250 comp rx=0 tx=0).stop 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 -- 172.21.15.3:0/259608318 shutdown_connections 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 --2- 172.21.15.3:0/259608318 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157ad0 0x7f3da0067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 --2- 172.21.15.3:0/259608318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da00cc270 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 -- 172.21.15.3:0/259608318 >> 172.21.15.3:0/259608318 conn(0x7f3da00ce2c0 msgr2=0x7f3da00ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 -- 172.21.15.3:0/259608318 shutdown_connections 2023-10-06T04:58:07.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.373+0000 7f3da8395700 1 -- 172.21.15.3:0/259608318 wait complete. 2023-10-06T04:58:07.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.374+0000 7f3da8395700 1 Processor -- start 2023-10-06T04:58:07.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.374+0000 7f3da8395700 1 -- start start 2023-10-06T04:58:07.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da8395700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da01479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da8395700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157700 0x7f3da0168590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da8395700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da0157ad0 0x7f3da00ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da8395700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00aeed0 con 0x7f3da00b1b10 2023-10-06T04:58:07.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da8395700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da0168ad0 con 0x7f3da0157ad0 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da8395700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da016b4c0 con 0x7f3da0157700 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da01479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da01479b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52906/0 (socket says 172.21.15.3:52906) 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 -- 172.21.15.3:0/452791396 learned_addr learned my addr 172.21.15.3:0/452791396 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 -- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157700 msgr2=0x7f3da0168590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157700 0x7f3da0168590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 -- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da0157ad0 msgr2=0x7f3da00ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da0157ad0 0x7f3da00ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 -- 172.21.15.3:0/452791396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940094f0 con 0x7f3da00b1b10 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3da6131700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da01479b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c950 tx=0x7f3d90002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.375+0000 7f3d9e7fc700 1 -- 172.21.15.3:0/452791396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000a7e0 con 0x7f3da00b1b10 2023-10-06T04:58:07.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.376+0000 7f3d9e7fc700 1 -- 172.21.15.3:0/452791396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9000a980 con 0x7f3da00b1b10 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.376+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da016b7a0 con 0x7f3da00b1b10 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.376+0000 7f3d9e7fc700 1 -- 172.21.15.3:0/452791396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000ad40 con 0x7f3da00b1b10 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.376+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da016bd60 con 0x7f3da00b1b10 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.376+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da00644c0 con 0x7f3da00b1b10 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.377+0000 7f3d9e7fc700 1 -- 172.21.15.3:0/452791396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d90012630 con 0x7f3da00b1b10 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.377+0000 7f3d9e7fc700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d8c066f30 0x7f3d8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.377+0000 7f3d9e7fc700 1 -- 172.21.15.3:0/452791396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d900844f0 con 0x7f3da00b1b10 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.378+0000 7f3da5930700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d8c066f30 0x7f3d8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.378+0000 7f3da5930700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d8c066f30 0x7f3d8c0693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3d940099b0 tx=0x7f3d94016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.382+0000 7f3d9e7fc700 1 -- 172.21.15.3:0/452791396 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d90051600 con 0x7f3da00b1b10 2023-10-06T04:58:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.618+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3da0071680 con 0x7f3da00b1b10 2023-10-06T04:58:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.618+0000 7f3d9e7fc700 1 -- 172.21.15.3:0/452791396 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f3d90054c20 con 0x7f3da00b1b10 2023-10-06T04:58:07.623 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:07.623 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:55:30.037469+0000","modified":"2023-10-06T04:55:51.531356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4081212306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4081212306},{"type":"v1","addr":"172.21.15.3:6835","nonce":4081212306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T04:58:07.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d8c066f30 msgr2=0x7f3d8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d8c066f30 0x7f3d8c0693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3d940099b0 tx=0x7f3d94016000 comp rx=0 tx=0).stop 2023-10-06T04:58:07.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 msgr2=0x7f3da01479b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da01479b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c950 tx=0x7f3d90002760 comp rx=0 tx=0).stop 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 shutdown_connections 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d8c066f30 0x7f3d8c0693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da0157ad0 0x7f3da00ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da00b1b10 0x7f3da01479b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 --2- 172.21.15.3:0/452791396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da0157700 0x7f3da0168590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 >> 172.21.15.3:0/452791396 conn(0x7f3da00ce2c0 msgr2=0x7f3da015d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 shutdown_connections 2023-10-06T04:58:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.620+0000 7f3da8395700 1 -- 172.21.15.3:0/452791396 wait complete. 2023-10-06T04:58:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2023-10-06T04:58:07.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:07.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.752+0000 7fdef5987700 1 Processor -- start 2023-10-06T04:58:07.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.752+0000 7fdef5987700 1 -- start start 2023-10-06T04:58:07.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.753+0000 7fdef5987700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef0159500 0x7fdef00cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.753+0000 7fdef5987700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdef00b2930 0x7fdef00cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.753+0000 7fdef5987700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef00cc5c0 con 0x7fdef01598d0 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.753+0000 7fdef5987700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef00cb7b0 con 0x7fdef00b2930 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.753+0000 7fdef5987700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef00cda60 con 0x7fdef0159500 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.753+0000 7fdeef7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdef00b2930 0x7fdef00cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeee7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdef01598d0 0x7fdef00b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48676/0 (socket says 172.21.15.3:48676) 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeef7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdef00b2930 0x7fdef00cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52922/0 (socket says 172.21.15.3:52922) 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeee7fc700 1 -- 172.21.15.3:0/459593964 learned_addr learned my addr 172.21.15.3:0/459593964 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeeeffd700 1 --2- 172.21.15.3:0/459593964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef0159500 0x7fdef00cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeef7fe700 1 -- 172.21.15.3:0/459593964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef0159500 msgr2=0x7fdef00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeef7fe700 1 --2- 172.21.15.3:0/459593964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef0159500 0x7fdef00cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeef7fe700 1 -- 172.21.15.3:0/459593964 >> v1:172.21.15.3:6789/0 conn(0x7fdef01598d0 legacy=0x7fdef00b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeef7fe700 1 -- 172.21.15.3:0/459593964 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00cb9b0 con 0x7fdef00b2930 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeedffb700 1 -- 172.21.15.3:0/459593964 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747082207 0 0) 0x7fdef00cc5c0 con 0x7fdef01598d0 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeef7fe700 1 --2- 172.21.15.3:0/459593964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef00b2930 0x7fdef00cb270 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009e60 tx=0x7fdee0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de8d70feb5c0d04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeedffb700 1 -- 172.21.15.3:0/459593964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000e070 con 0x7fdef00b2930 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeedffb700 1 -- 172.21.15.3:0/459593964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee000bb60 con 0x7fdef00b2930 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.754+0000 7fdeedffb700 1 -- 172.21.15.3:0/459593964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0012430 con 0x7fdef00b2930 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 -- 172.21.15.3:0/459593964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef00b2930 msgr2=0x7fdef00cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 --2- 172.21.15.3:0/459593964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef00b2930 0x7fdef00cb270 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009e60 tx=0x7fdee0009780 comp rx=0 tx=0).stop 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 -- 172.21.15.3:0/459593964 shutdown_connections 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 --2- 172.21.15.3:0/459593964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef00b2930 0x7fdef00cb270 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 --2- 172.21.15.3:0/459593964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef0159500 0x7fdef00cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 -- 172.21.15.3:0/459593964 >> 172.21.15.3:0/459593964 conn(0x7fdef0070310 msgr2=0x7fdef0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:07.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 -- 172.21.15.3:0/459593964 shutdown_connections 2023-10-06T04:58:07.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.755+0000 7fdef5987700 1 -- 172.21.15.3:0/459593964 wait complete. 2023-10-06T04:58:07.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.756+0000 7fdef5987700 1 Processor -- start 2023-10-06T04:58:07.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.756+0000 7fdef5987700 1 -- start start 2023-10-06T04:58:07.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdef5987700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef00b2930 0x7fdef0155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdef5987700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef0159500 0x7fdef01520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdef5987700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef01598d0 0x7fdef0157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdef5987700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0156690 con 0x7fdef0159500 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdef5987700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef01560f0 con 0x7fdef00b2930 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdef5987700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef01563b0 con 0x7fdef01598d0 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef0159500 0x7fdef01520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef0159500 0x7fdef01520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52936/0 (socket says 172.21.15.3:52936) 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 -- 172.21.15.3:0/3202987293 learned_addr learned my addr 172.21.15.3:0/3202987293 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeef7fe700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef01598d0 0x7fdef0157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeeeffd700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef00b2930 0x7fdef0155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 -- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef01598d0 msgr2=0x7fdef0157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef01598d0 0x7fdef0157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 -- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef00b2930 msgr2=0x7fdef0155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef00b2930 0x7fdef0155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 -- 172.21.15.3:0/3202987293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0009a20 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.757+0000 7fdeee7fc700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef0159500 0x7fdef01520e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fded800c5b0 tx=0x7fded8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.758+0000 7fdee6ffd700 1 -- 172.21.15.3:0/3202987293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8005b30 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.758+0000 7fdee6ffd700 1 -- 172.21.15.3:0/3202987293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8005c90 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.758+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef0154570 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.758+0000 7fdee6ffd700 1 -- 172.21.15.3:0/3202987293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8007300 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.758+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef0154b30 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.759+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef0154d50 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.759+0000 7fdee6ffd700 1 -- 172.21.15.3:0/3202987293 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fded801b3b0 con 0x7fdef0159500 2023-10-06T04:58:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.759+0000 7fdee6ffd700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fded0066f30 0x7fded00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:07.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.760+0000 7fdee6ffd700 1 -- 172.21.15.3:0/3202987293 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fded8083ac0 con 0x7fdef0159500 2023-10-06T04:58:07.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.760+0000 7fdeeeffd700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fded0066f30 0x7fded00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:07.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.760+0000 7fdeeeffd700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fded0066f30 0x7fded00693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009db0 tx=0x7fdee80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:07.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.764+0000 7fdee6ffd700 1 -- 172.21.15.3:0/3202987293 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fded8050bd0 con 0x7fdef0159500 2023-10-06T04:58:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.960+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdef0071680 con 0x7fdef0159500 2023-10-06T04:58:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.961+0000 7fdee6ffd700 1 -- 172.21.15.3:0/3202987293 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16821 (secure 0 0 0) 0x7fded8017090 con 0x7fdef0159500 2023-10-06T04:58:07.966 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:07.966 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:55:31.630582+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:55:27.781764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T04:55:29.007164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fded0066f30 msgr2=0x7fded00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fded0066f30 0x7fded00693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009db0 tx=0x7fdee80091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef0159500 msgr2=0x7fdef01520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef0159500 0x7fdef01520e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fded800c5b0 tx=0x7fded8001070 comp rx=0 tx=0).stop 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 shutdown_connections 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fded0066f30 0x7fded00693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef00b2930 0x7fdef0155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdef01598d0 0x7fdef0157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 --2- 172.21.15.3:0/3202987293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef0159500 0x7fdef01520e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 >> 172.21.15.3:0/3202987293 conn(0x7fdef0070310 msgr2=0x7fdef00ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 shutdown_connections 2023-10-06T04:58:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:07.963+0000 7fdef5987700 1 -- 172.21.15.3:0/3202987293 wait complete. 2023-10-06T04:58:07.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:58:08.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.097+0000 7f20d8978700 1 Processor -- start 2023-10-06T04:58:08.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.097+0000 7f20d8978700 1 -- start start 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.097+0000 7f20d8978700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41597a0 0x7f20d40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.097+0000 7f20d8978700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20d40c62e0 0x7f20d40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.097+0000 7f20d8978700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40cc910 con 0x7f20d41593d0 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.097+0000 7f20d8978700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d40cbd40 con 0x7f20d40c62e0 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.097+0000 7f20d8978700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40cbfe0 con 0x7f20d41597a0 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20d41593d0 0x7f20d40c5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48680/0 (socket says 172.21.15.3:48680) 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20d40c62e0 0x7f20d40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d37fe700 1 -- 172.21.15.3:0/395960085 learned_addr learned my addr 172.21.15.3:0/395960085 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d2ffd700 1 --2- 172.21.15.3:0/395960085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41597a0 0x7f20d40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d27fc700 1 -- 172.21.15.3:0/395960085 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191379550 0 0) 0x7f20d40cc910 con 0x7f20d41593d0 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d27fc700 1 -- 172.21.15.3:0/395960085 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20bc003570 con 0x7f20d41593d0 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d3fff700 1 -- 172.21.15.3:0/395960085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41597a0 msgr2=0x7f20d40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d3fff700 1 --2- 172.21.15.3:0/395960085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41597a0 0x7f20d40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d3fff700 1 -- 172.21.15.3:0/395960085 >> v1:172.21.15.3:6789/0 conn(0x7f20d41593d0 legacy=0x7f20d40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d3fff700 1 -- 172.21.15.3:0/395960085 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d40cc2c0 con 0x7f20d40c62e0 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d3fff700 1 --2- 172.21.15.3:0/395960085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d40c62e0 0x7f20d40cd2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009e60 tx=0x7f20cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64dede57346f336 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d27fc700 1 -- 172.21.15.3:0/395960085 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00e070 con 0x7f20d40c62e0 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d27fc700 1 -- 172.21.15.3:0/395960085 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc00bb60 con 0x7f20d40c62e0 2023-10-06T04:58:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.098+0000 7f20d27fc700 1 -- 172.21.15.3:0/395960085 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc012430 con 0x7f20d40c62e0 2023-10-06T04:58:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 -- 172.21.15.3:0/395960085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d40c62e0 msgr2=0x7f20d40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 --2- 172.21.15.3:0/395960085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d40c62e0 0x7f20d40cd2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009e60 tx=0x7f20cc009780 comp rx=0 tx=0).stop 2023-10-06T04:58:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 -- 172.21.15.3:0/395960085 shutdown_connections 2023-10-06T04:58:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 --2- 172.21.15.3:0/395960085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d40c62e0 0x7f20d40cd2f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 --2- 172.21.15.3:0/395960085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41597a0 0x7f20d40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 -- 172.21.15.3:0/395960085 >> 172.21.15.3:0/395960085 conn(0x7f20d4089df0 msgr2=0x7f20d4099710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 -- 172.21.15.3:0/395960085 shutdown_connections 2023-10-06T04:58:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.099+0000 7f20d8978700 1 -- 172.21.15.3:0/395960085 wait complete. 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.100+0000 7f20d8978700 1 Processor -- start 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.100+0000 7f20d8978700 1 -- start start 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d8978700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d40c62e0 0x7f20d414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d8978700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41593d0 0x7f20d4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d8978700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d41597a0 0x7f20d415f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d8978700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40c8030 con 0x7f20d41597a0 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d8978700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d414dc70 con 0x7f20d40c62e0 2023-10-06T04:58:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d8978700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40c7420 con 0x7f20d41593d0 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d41597a0 0x7f20d415f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d41597a0 0x7f20d415f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52956/0 (socket says 172.21.15.3:52956) 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 -- 172.21.15.3:0/3822367479 learned_addr learned my addr 172.21.15.3:0/3822367479 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d2ffd700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41593d0 0x7f20d4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d37fe700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d40c62e0 0x7f20d414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 -- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41593d0 msgr2=0x7f20d4159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41593d0 0x7f20d4159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 -- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d40c62e0 msgr2=0x7f20d414d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d40c62e0 0x7f20d414d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 -- 172.21.15.3:0/3822367479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc009a20 con 0x7f20d41597a0 2023-10-06T04:58:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d37fe700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d40c62e0 0x7f20d414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.101+0000 7f20d3fff700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d41597a0 0x7f20d415f300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f20cc013000 tx=0x7f20cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.102+0000 7f20bb7fe700 1 -- 172.21.15.3:0/3822367479 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc027470 con 0x7f20d41597a0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.102+0000 7f20bb7fe700 1 -- 172.21.15.3:0/3822367479 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc027610 con 0x7f20d41597a0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.102+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d40c7700 con 0x7f20d41597a0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.102+0000 7f20bb7fe700 1 -- 172.21.15.3:0/3822367479 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc0279d0 con 0x7f20d41597a0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.102+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d40c7cc0 con 0x7f20d41597a0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.102+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d40c81e0 con 0x7f20d41597a0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.103+0000 7f20bb7fe700 1 -- 172.21.15.3:0/3822367479 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20cc024070 con 0x7f20d41597a0 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.103+0000 7f20bb7fe700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f20c0066f30 0x7f20c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.103+0000 7f20bb7fe700 1 -- 172.21.15.3:0/3822367479 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f20cc087130 con 0x7f20d41597a0 2023-10-06T04:58:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.104+0000 7f20d37fe700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f20c0066f30 0x7f20c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.104+0000 7f20d37fe700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f20c0066f30 0x7f20c00693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f20c40058d0 tx=0x7f20c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.108+0000 7f20bb7fe700 1 -- 172.21.15.3:0/3822367479 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20d40c81e0 con 0x7f20d41597a0 2023-10-06T04:58:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.345+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f20d404c380 con 0x7f20d41597a0 2023-10-06T04:58:08.348 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T04:58:08.345+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:58:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.352+0000 7f20bb7fe700 1 -- 172.21.15.3:0/3822367479 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f20cc057cd0 con 0x7f20d41597a0 2023-10-06T04:58:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f20c0066f30 msgr2=0x7f20c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f20c0066f30 0x7f20c00693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f20c40058d0 tx=0x7f20c4009040 comp rx=0 tx=0).stop 2023-10-06T04:58:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d41597a0 msgr2=0x7f20d415f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d41597a0 0x7f20d415f300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f20cc013000 tx=0x7f20cc019000 comp rx=0 tx=0).stop 2023-10-06T04:58:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 shutdown_connections 2023-10-06T04:58:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f20c0066f30 0x7f20c00693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d40c62e0 0x7f20d414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d41597a0 0x7f20d415f300 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 --2- 172.21.15.3:0/3822367479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f20d41593d0 0x7f20d4159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 >> 172.21.15.3:0/3822367479 conn(0x7f20d4089df0 msgr2=0x7f20d40c53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 shutdown_connections 2023-10-06T04:58:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.355+0000 7f20d8978700 1 -- 172.21.15.3:0/3822367479 wait complete. 2023-10-06T04:58:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:58:08.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:58:08.387 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:58:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.489+0000 7f734588b700 1 Processor -- start 2023-10-06T04:58:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.489+0000 7f734588b700 1 -- start start 2023-10-06T04:58:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f734588b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f73401595c0 0x7f7340067ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f734588b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73400a9850 0x7f73400ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f734588b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734015dfe0 con 0x7f73401591f0 2023-10-06T04:58:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f734588b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73400a9fc0 con 0x7f73401595c0 2023-10-06T04:58:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f734588b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73400aa2a0 con 0x7f73400a9850 2023-10-06T04:58:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f733ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f73401595c0 0x7f7340067ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f733ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f73401595c0 0x7f7340067ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52966/0 (socket says 172.21.15.3:52966) 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f733ffff700 1 -- 172.21.15.3:0/34315357 learned_addr learned my addr 172.21.15.3:0/34315357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.490+0000 7f734508a700 1 --2- 172.21.15.3:0/34315357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73400a9850 0x7f73400ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733ffff700 1 -- 172.21.15.3:0/34315357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73400a9850 msgr2=0x7f73400ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733ffff700 1 --2- 172.21.15.3:0/34315357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73400a9850 0x7f73400ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733ffff700 1 -- 172.21.15.3:0/34315357 >> v1:172.21.15.153:6789/0 conn(0x7f73401591f0 legacy=0x7f734015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733f7fe700 1 -- 172.21.15.3:0/34315357 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252650443 0 0) 0x7f734015dfe0 con 0x7f73401591f0 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733ffff700 1 -- 172.21.15.3:0/34315357 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400aa580 con 0x7f73401595c0 2023-10-06T04:58:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733ffff700 1 --2- 172.21.15.3:0/34315357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340067ad0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f734015e1d0 tx=0x7f7330009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558242a35fc24c12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733f7fe700 1 -- 172.21.15.3:0/34315357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330009e70 con 0x7f73401595c0 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733f7fe700 1 -- 172.21.15.3:0/34315357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733000b860 con 0x7f73401595c0 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f733f7fe700 1 -- 172.21.15.3:0/34315357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000bc20 con 0x7f73401595c0 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.491+0000 7f734588b700 1 -- 172.21.15.3:0/34315357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 msgr2=0x7f7340067ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.492+0000 7f734588b700 1 --2- 172.21.15.3:0/34315357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340067ad0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f734015e1d0 tx=0x7f7330009250 comp rx=0 tx=0).stop 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.492+0000 7f734588b700 1 -- 172.21.15.3:0/34315357 shutdown_connections 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.492+0000 7f734588b700 1 --2- 172.21.15.3:0/34315357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73400a9850 0x7f73400ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.492+0000 7f734588b700 1 --2- 172.21.15.3:0/34315357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340067ad0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.492+0000 7f734588b700 1 -- 172.21.15.3:0/34315357 >> 172.21.15.3:0/34315357 conn(0x7f734006d480 msgr2=0x7f734006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.492+0000 7f734588b700 1 -- 172.21.15.3:0/34315357 shutdown_connections 2023-10-06T04:58:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.492+0000 7f734588b700 1 -- 172.21.15.3:0/34315357 wait complete. 2023-10-06T04:58:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.493+0000 7f734588b700 1 Processor -- start 2023-10-06T04:58:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.493+0000 7f734588b700 1 -- start start 2023-10-06T04:58:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73400a9850 0x7f7340157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73401591f0 0x7f73401525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7340152120 con 0x7f73401595c0 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7340151510 con 0x7f73400a9850 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73401517f0 con 0x7f73401591f0 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52974/0 (socket says 172.21.15.3:52974) 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 -- 172.21.15.3:0/1822334571 learned_addr learned my addr 172.21.15.3:0/1822334571 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f733ffff700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73401591f0 0x7f73401525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f7344889700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73400a9850 0x7f7340157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 -- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73401591f0 msgr2=0x7f73401525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73401591f0 0x7f73401525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 -- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73400a9850 msgr2=0x7f7340157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73400a9850 0x7f7340157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 -- 172.21.15.3:0/1822334571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73300094f0 con 0x7f73401595c0 2023-10-06T04:58:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734508a700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340150a70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f732800ea50 tx=0x7f73280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f733cff9700 1 -- 172.21.15.3:0/1822334571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732800c920 con 0x7f73401595c0 2023-10-06T04:58:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f733cff9700 1 -- 172.21.15.3:0/1822334571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732800caa0 con 0x7f73401595c0 2023-10-06T04:58:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7340151ad0 con 0x7f73401595c0 2023-10-06T04:58:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f733cff9700 1 -- 172.21.15.3:0/1822334571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732800ce60 con 0x7f73401595c0 2023-10-06T04:58:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.494+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73400aef00 con 0x7f73401595c0 2023-10-06T04:58:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.495+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734004c380 con 0x7f73401595c0 2023-10-06T04:58:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.496+0000 7f733cff9700 1 -- 172.21.15.3:0/1822334571 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7328013680 con 0x7f73401595c0 2023-10-06T04:58:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.496+0000 7f733cff9700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7320066f30 0x7f73200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.496+0000 7f733cff9700 1 -- 172.21.15.3:0/1822334571 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7328084b70 con 0x7f73401595c0 2023-10-06T04:58:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.496+0000 7f7344889700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7320066f30 0x7f73200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.497+0000 7f7344889700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7320066f30 0x7f73200693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7338001560 tx=0x7f7338009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.500+0000 7f733cff9700 1 -- 172.21.15.3:0/1822334571 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7328051c30 con 0x7f73401595c0 2023-10-06T04:58:08.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:08.741+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7340152260 con 0x7f73401595c0 2023-10-06T04:58:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.350+0000 7f733cff9700 1 -- 172.21.15.3:0/1822334571 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7328055250 con 0x7f73401595c0 2023-10-06T04:58:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7320066f30 msgr2=0x7f73200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7320066f30 0x7f73200693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7338001560 tx=0x7f7338009040 comp rx=0 tx=0).stop 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 msgr2=0x7f7340150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340150a70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f732800ea50 tx=0x7f73280091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 shutdown_connections 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7320066f30 0x7f73200693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73400a9850 0x7f7340157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73401595c0 0x7f7340150a70 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 --2- 172.21.15.3:0/1822334571 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73401591f0 0x7f73401525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 >> 172.21.15.3:0/1822334571 conn(0x7f734006d480 msgr2=0x7f734006a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 shutdown_connections 2023-10-06T04:58:09.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.353+0000 7f734588b700 1 -- 172.21.15.3:0/1822334571 wait complete. 2023-10-06T04:58:09.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:58:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.539+0000 7f112d47a700 1 Processor -- start 2023-10-06T04:58:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.539+0000 7f112d47a700 1 -- start start 2023-10-06T04:58:09.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f112d47a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11281598f0 0x7f11280cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:09.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f112d47a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f11280d1170 0x7f112815d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:09.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f112d47a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112815df30 con 0x7f112815dad0 2023-10-06T04:58:09.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f112d47a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112815e400 con 0x7f11281598f0 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f112d47a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11280d15c0 con 0x7f11280d1170 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f1126ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11281598f0 0x7f11280cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f11277fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f112815dad0 0x7f11280cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48704/0 (socket says 172.21.15.3:48704) 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.540+0000 7f11277fe700 1 -- 172.21.15.3:0/1968956551 learned_addr learned my addr 172.21.15.3:0/1968956551 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f11267fc700 1 --2- 172.21.15.3:0/1968956551 >> v2:172.21.15.153:3300/0 conn(0x7f11280d1170 0x7f112815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1126ffd700 1 -- 172.21.15.3:0/1968956551 >> v2:172.21.15.153:3300/0 conn(0x7f11280d1170 msgr2=0x7f112815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1126ffd700 1 --2- 172.21.15.3:0/1968956551 >> v2:172.21.15.153:3300/0 conn(0x7f11280d1170 0x7f112815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1126ffd700 1 -- 172.21.15.3:0/1968956551 >> v1:172.21.15.3:6789/0 conn(0x7f112815dad0 legacy=0x7f11280cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1126ffd700 1 -- 172.21.15.3:0/1968956551 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280cf7d0 con 0x7f11281598f0 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1125ffb700 1 -- 172.21.15.3:0/1968956551 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702239592 0 0) 0x7f1118001490 con 0x7f112815dad0 2023-10-06T04:58:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1126ffd700 1 --2- 172.21.15.3:0/1968956551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11281598f0 0x7f11280cebd0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f112806db00 tx=0x7f11200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d291c4cf76ac74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1125ffb700 1 -- 172.21.15.3:0/1968956551 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000b740 con 0x7f11281598f0 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1125ffb700 1 -- 172.21.15.3:0/1968956551 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112000b8a0 con 0x7f11281598f0 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.541+0000 7f1125ffb700 1 -- 172.21.15.3:0/1968956551 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000bc40 con 0x7f11281598f0 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 -- 172.21.15.3:0/1968956551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11281598f0 msgr2=0x7f11280cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 --2- 172.21.15.3:0/1968956551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11281598f0 0x7f11280cebd0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f112806db00 tx=0x7f11200091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 -- 172.21.15.3:0/1968956551 shutdown_connections 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 --2- 172.21.15.3:0/1968956551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11281598f0 0x7f11280cebd0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 --2- 172.21.15.3:0/1968956551 >> v2:172.21.15.153:3300/0 conn(0x7f11280d1170 0x7f112815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 -- 172.21.15.3:0/1968956551 >> 172.21.15.3:0/1968956551 conn(0x7f1128070af0 msgr2=0x7f11280d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 -- 172.21.15.3:0/1968956551 shutdown_connections 2023-10-06T04:58:09.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.542+0000 7f112d47a700 1 -- 172.21.15.3:0/1968956551 wait complete. 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.543+0000 7f112d47a700 1 Processor -- start 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.543+0000 7f112d47a700 1 -- start start 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11280d1170 0x7f1128151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11281598f0 0x7f112815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f112815dad0 0x7f112815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112815b490 con 0x7f11280d1170 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112815c010 con 0x7f11281598f0 2023-10-06T04:58:09.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112815b6e0 con 0x7f112815dad0 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11280d1170 0x7f1128151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11280d1170 0x7f1128151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52986/0 (socket says 172.21.15.3:52986) 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 -- 172.21.15.3:0/1658494195 learned_addr learned my addr 172.21.15.3:0/1658494195 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f11277fe700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f112815dad0 0x7f112815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f11267fc700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11281598f0 0x7f112815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 -- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f112815dad0 msgr2=0x7f112815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f112815dad0 0x7f112815c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 -- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11281598f0 msgr2=0x7f112815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11281598f0 0x7f112815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 -- 172.21.15.3:0/1658494195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120009420 con 0x7f11280d1170 2023-10-06T04:58:09.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f1126ffd700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11280d1170 0x7f1128151610 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1128088810 tx=0x7f1120017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f111effd700 1 -- 172.21.15.3:0/1658494195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120011420 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f111effd700 1 -- 172.21.15.3:0/1658494195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1120011580 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f111effd700 1 -- 172.21.15.3:0/1658494195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120011920 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112815b9c0 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.544+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1128151b50 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.545+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1128151dd0 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.546+0000 7f111effd700 1 -- 172.21.15.3:0/1658494195 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1120011c90 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.546+0000 7f111effd700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1108066f30 0x7f11080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.546+0000 7f111effd700 1 -- 172.21.15.3:0/1658494195 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1120085c80 con 0x7f11280d1170 2023-10-06T04:58:09.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.546+0000 7f11267fc700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1108066f30 0x7f11080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:09.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.547+0000 7f11267fc700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1108066f30 0x7f11080693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1110006fd0 tx=0x7f1110008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:09.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.550+0000 7f111effd700 1 -- 172.21.15.3:0/1658494195 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1120052d40 con 0x7f11280d1170 2023-10-06T04:58:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:09.744+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1128071680 con 0x7f11280d1170 2023-10-06T04:58:10.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.664+0000 7f111effd700 1 -- 172.21.15.3:0/1658494195 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7f11200150d0 con 0x7f11280d1170 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1108066f30 msgr2=0x7f11080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1108066f30 0x7f11080693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1110006fd0 tx=0x7f1110008040 comp rx=0 tx=0).stop 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11280d1170 msgr2=0x7f1128151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11280d1170 0x7f1128151610 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1128088810 tx=0x7f1120017000 comp rx=0 tx=0).stop 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 shutdown_connections 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1108066f30 0x7f11080693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f112815dad0 0x7f112815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11280d1170 0x7f1128151610 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 --2- 172.21.15.3:0/1658494195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11281598f0 0x7f112815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 >> 172.21.15.3:0/1658494195 conn(0x7f1128070af0 msgr2=0x7f112814a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:10.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 shutdown_connections 2023-10-06T04:58:10.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.668+0000 7f112d47a700 1 -- 172.21.15.3:0/1658494195 wait complete. 2023-10-06T04:58:10.672 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:58:10.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:58:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.831+0000 7f4611e94700 1 Processor -- start 2023-10-06T04:58:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.831+0000 7f4611e94700 1 -- start start 2023-10-06T04:58:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f4611e94700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f460c1536e0 0x7f460c15d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f4611e94700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c15aa00 con 0x7f460c15a560 2023-10-06T04:58:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f4611e94700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c0ce6b0 con 0x7f460c0cd570 2023-10-06T04:58:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f4611e94700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c15d830 con 0x7f460c1536e0 2023-10-06T04:58:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f460affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f460c15a560 0x7f460c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48718/0 (socket says 172.21.15.3:48718) 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f460affd700 1 -- 172.21.15.3:0/2503094632 learned_addr learned my addr 172.21.15.3:0/2503094632 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f460b7fe700 1 --2- 172.21.15.3:0/2503094632 >> v2:172.21.15.153:3300/0 conn(0x7f460c1536e0 0x7f460c15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136942869 0 0) 0x7f460c15aa00 con 0x7f460c15a560 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.832+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f4003570 con 0x7f460c15a560 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3886211493 0 0) 0x7f45f4003570 con 0x7f460c15a560 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 >> v2:172.21.15.153:3300/0 conn(0x7f460c1536e0 msgr2=0x7f460c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 --2- 172.21.15.3:0/2503094632 >> v2:172.21.15.153:3300/0 conn(0x7f460c1536e0 0x7f460c15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 >> v1:172.21.15.153:6789/0 conn(0x7f460c0cd570 legacy=0x7f460c0aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c15db10 con 0x7f460c15a560 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f4600001470 con 0x7f460c15a560 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 >> v1:172.21.15.3:6789/0 conn(0x7f460c15a560 legacy=0x7f460c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45f4003ab0 0x7f45f4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45f4004810 0x7f45f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4005210 0x7f45f400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c15d830 con 0x7f45f4003ab0 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f400c130 con 0x7f45f4005210 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f400c2f0 con 0x7f45f4004810 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460b7fe700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45f4003ab0 0x7f45f4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460affd700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45f4004810 0x7f45f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460bfff700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4005210 0x7f45f400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460b7fe700 1 -- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45f4004810 msgr2=0x7f45f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460b7fe700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45f4004810 0x7f45f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460b7fe700 1 -- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4005210 msgr2=0x7f45f400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:10.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460b7fe700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4005210 0x7f45f400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460b7fe700 1 -- 172.21.15.3:0/2503094632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4003570 con 0x7f45f4003ab0 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460bfff700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4005210 0x7f45f400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460bfff700 1 -- 172.21.15.3:0/2503094632 reap_dead start 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.833+0000 7f460b7fe700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45f4003ab0 0x7f45f4003ec0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009c00 tx=0x7f45fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f649c538f1594bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00e710 con 0x7f45f4003ab0 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc00e870 con 0x7f45f4003ab0 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f460a7fc700 1 -- 172.21.15.3:0/2503094632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00ec30 con 0x7f45f4003ab0 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f4611e94700 1 -- 172.21.15.3:0/2503094632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45f4003ab0 msgr2=0x7f45f4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f4611e94700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45f4003ab0 0x7f45f4003ec0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009c00 tx=0x7f45fc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f4611e94700 1 -- 172.21.15.3:0/2503094632 shutdown_connections 2023-10-06T04:58:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f4611e94700 1 --2- 172.21.15.3:0/2503094632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45f4003ab0 0x7f45f4003ec0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f4611e94700 1 -- 172.21.15.3:0/2503094632 >> 172.21.15.3:0/2503094632 conn(0x7f460c06f980 msgr2=0x7f460c070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:10.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f4611e94700 1 -- 172.21.15.3:0/2503094632 shutdown_connections 2023-10-06T04:58:10.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.834+0000 7f4611e94700 1 -- 172.21.15.3:0/2503094632 wait complete. 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.835+0000 7f4611e94700 1 Processor -- start 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f4611e94700 1 -- start start 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f4611e94700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4003ab0 0x7f460c1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f4611e94700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f460c1594b0 0x7f460c0c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f4611e94700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460c0b5570 0x7f460c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f4611e94700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0c8210 con 0x7f460c0b5570 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f4611e94700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0c7640 con 0x7f45f4003ab0 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f4611e94700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0c78e0 con 0x7f460c1594b0 2023-10-06T04:58:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f460bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460c0b5570 0x7f460c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f460bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460c0b5570 0x7f460c0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52996/0 (socket says 172.21.15.3:52996) 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f460bfff700 1 -- 172.21.15.3:0/3356384441 learned_addr learned my addr 172.21.15.3:0/3356384441 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f460b7fe700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4003ab0 0x7f460c1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.836+0000 7f460affd700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f460c1594b0 0x7f460c0c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f460bfff700 1 -- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f460c1594b0 msgr2=0x7f460c0c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f460bfff700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f460c1594b0 0x7f460c0c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f460bfff700 1 -- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4003ab0 msgr2=0x7f460c1520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f460bfff700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4003ab0 0x7f460c1520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f460bfff700 1 -- 172.21.15.3:0/3356384441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c0c7bc0 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f460bfff700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460c0b5570 0x7f460c0af190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f460c0cbc10 tx=0x7f4604009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f45f37fe700 1 -- 172.21.15.3:0/3356384441 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604012590 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f45f37fe700 1 -- 172.21.15.3:0/3356384441 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040126f0 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c0c7ea0 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f45f37fe700 1 -- 172.21.15.3:0/3356384441 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604012ab0 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.837+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c0c8be0 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.838+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c0c8380 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.839+0000 7f45f37fe700 1 -- 172.21.15.3:0/3356384441 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4604003030 con 0x7f460c0b5570 2023-10-06T04:58:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.839+0000 7f45f37fe700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f45f8066f30 0x7f45f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.839+0000 7f45f37fe700 1 -- 172.21.15.3:0/3356384441 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f46040842e0 con 0x7f460c0b5570 2023-10-06T04:58:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.839+0000 7f460b7fe700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f45f8066f30 0x7f45f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.840+0000 7f460b7fe700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f45f8066f30 0x7f45f80693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0091e0 tx=0x7f45fc01a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:10.843+0000 7f45f37fe700 1 -- 172.21.15.3:0/3356384441 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4604051550 con 0x7f460c0b5570 2023-10-06T04:58:11.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.037+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f460c071680 con 0x7f460c0b5570 2023-10-06T04:58:11.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.667+0000 7f45f37fe700 1 -- 172.21.15.3:0/3356384441 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4604054b70 con 0x7f460c0b5570 2023-10-06T04:58:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.671+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f45f8066f30 msgr2=0x7f45f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.671+0000 7f4611e94700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f45f8066f30 0x7f45f80693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0091e0 tx=0x7f45fc01a000 comp rx=0 tx=0).stop 2023-10-06T04:58:11.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460c0b5570 msgr2=0x7f460c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460c0b5570 0x7f460c0af190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f460c0cbc10 tx=0x7f4604009040 comp rx=0 tx=0).stop 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 shutdown_connections 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f45f8066f30 0x7f45f80693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45f4003ab0 0x7f460c1520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f460c0b5570 0x7f460c0af190 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 --2- 172.21.15.3:0/3356384441 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f460c1594b0 0x7f460c0c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 >> 172.21.15.3:0/3356384441 conn(0x7f460c06f980 msgr2=0x7f460c152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 shutdown_connections 2023-10-06T04:58:11.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.672+0000 7f4611e94700 1 -- 172.21.15.3:0/3356384441 wait complete. 2023-10-06T04:58:11.676 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T04:58:11.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-10-06T04:58:11.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.821+0000 7f3f69ca7700 1 Processor -- start 2023-10-06T04:58:11.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.821+0000 7f3f69ca7700 1 -- start start 2023-10-06T04:58:11.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.821+0000 7f3f69ca7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f64068190 0x7f3f6415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.821+0000 7f3f69ca7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f6415cd40 0x7f3f6406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.821+0000 7f3f69ca7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6415d130 con 0x7f3f64067dc0 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.821+0000 7f3f69ca7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640691c0 con 0x7f3f64068190 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.821+0000 7f3f69ca7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640694a0 con 0x7f3f6415cd40 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f64068190 0x7f3f6415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f64068190 0x7f3f6415c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53012/0 (socket says 172.21.15.3:53012) 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 -- 172.21.15.3:0/333043726 learned_addr learned my addr 172.21.15.3:0/333043726 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f63fff700 1 --2- 172.21.15.3:0/333043726 >> v2:172.21.15.153:3300/0 conn(0x7f3f6415cd40 0x7f3f6406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 -- 172.21.15.3:0/333043726 >> v2:172.21.15.153:3300/0 conn(0x7f3f6415cd40 msgr2=0x7f3f6406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 --2- 172.21.15.3:0/333043726 >> v2:172.21.15.153:3300/0 conn(0x7f3f6415cd40 0x7f3f6406d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 -- 172.21.15.3:0/333043726 >> v1:172.21.15.153:6789/0 conn(0x7f3f64067dc0 legacy=0x7f3f640b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 -- 172.21.15.3:0/333043726 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64069780 con 0x7f3f64068190 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f62ffd700 1 --2- 172.21.15.3:0/333043726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f64068190 0x7f3f6415c800 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3f64068db0 tx=0x7f3f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa951326b800ae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f627fc700 1 -- 172.21.15.3:0/333043726 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800b820 con 0x7f3f64068190 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f627fc700 1 -- 172.21.15.3:0/333043726 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5800b980 con 0x7f3f64068190 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.822+0000 7f3f627fc700 1 -- 172.21.15.3:0/333043726 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800bd20 con 0x7f3f64068190 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 -- 172.21.15.3:0/333043726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f64068190 msgr2=0x7f3f6415c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/333043726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f64068190 0x7f3f6415c800 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3f64068db0 tx=0x7f3f58009250 comp rx=0 tx=0).stop 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 -- 172.21.15.3:0/333043726 shutdown_connections 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/333043726 >> v2:172.21.15.153:3300/0 conn(0x7f3f6415cd40 0x7f3f6406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/333043726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f64068190 0x7f3f6415c800 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 -- 172.21.15.3:0/333043726 >> 172.21.15.3:0/333043726 conn(0x7f3f640ce270 msgr2=0x7f3f640ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:11.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 -- 172.21.15.3:0/333043726 shutdown_connections 2023-10-06T04:58:11.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.823+0000 7f3f69ca7700 1 -- 172.21.15.3:0/333043726 wait complete. 2023-10-06T04:58:11.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.824+0000 7f3f69ca7700 1 Processor -- start 2023-10-06T04:58:11.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.824+0000 7f3f69ca7700 1 -- start start 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f69ca7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f64067dc0 0x7f3f640afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f69ca7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f64068190 0x7f3f64153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f69ca7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f6415cd40 0x7f3f64154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f69ca7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6416bda0 con 0x7f3f6415cd40 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f69ca7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6416b790 con 0x7f3f64067dc0 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f69ca7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6416b090 con 0x7f3f64068190 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f6415cd40 0x7f3f64154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f6415cd40 0x7f3f64154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53026/0 (socket says 172.21.15.3:53026) 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 -- 172.21.15.3:0/667416143 learned_addr learned my addr 172.21.15.3:0/667416143 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f62ffd700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f64068190 0x7f3f64153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f637fe700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f64067dc0 0x7f3f640afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 -- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f64068190 msgr2=0x7f3f64153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f64068190 0x7f3f64153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 -- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f64067dc0 msgr2=0x7f3f640afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f64067dc0 0x7f3f640afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 -- 172.21.15.3:0/667416143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580094f0 con 0x7f3f6415cd40 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f63fff700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f6415cd40 0x7f3f64154100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00cdb0 tx=0x7f3f5c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f4b7fe700 1 -- 172.21.15.3:0/667416143 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00a9d0 con 0x7f3f6415cd40 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f4b7fe700 1 -- 172.21.15.3:0/667416143 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c00ab30 con 0x7f3f6415cd40 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6416b370 con 0x7f3f6415cd40 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.825+0000 7f3f4b7fe700 1 -- 172.21.15.3:0/667416143 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c017430 con 0x7f3f6415cd40 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.826+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f640b02d0 con 0x7f3f6415cd40 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.826+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f64154640 con 0x7f3f6415cd40 2023-10-06T04:58:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.827+0000 7f3f4b7fe700 1 -- 172.21.15.3:0/667416143 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f5c003020 con 0x7f3f6415cd40 2023-10-06T04:58:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.827+0000 7f3f4b7fe700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.827+0000 7f3f4b7fe700 1 -- 172.21.15.3:0/667416143 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3f5c013070 con 0x7f3f6415cd40 2023-10-06T04:58:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.828+0000 7f3f637fe700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.828+0000 7f3f637fe700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009b40 tx=0x7f3f540090f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:11.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:11.831+0000 7f3f4b7fe700 1 -- 172.21.15.3:0/667416143 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f5c051bc0 con 0x7f3f6415cd40 2023-10-06T04:58:12.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.027+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f3f6404c380 con 0x7f3f6415cd40 2023-10-06T04:58:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.031+0000 7f3f4b7fe700 1 -- 172.21.15.3:0/667416143 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v27) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3f5c0078f0 con 0x7f3f6415cd40 2023-10-06T04:58:12.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 msgr2=0x7f3f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009b40 tx=0x7f3f540090f0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f6415cd40 msgr2=0x7f3f64154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f6415cd40 0x7f3f64154100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00cdb0 tx=0x7f3f5c0077d0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 shutdown_connections 2023-10-06T04:58:12.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f64067dc0 0x7f3f640afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f6415cd40 0x7f3f64154100 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 --2- 172.21.15.3:0/667416143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f64068190 0x7f3f64153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 >> 172.21.15.3:0/667416143 conn(0x7f3f640ce270 msgr2=0x7f3f64158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 shutdown_connections 2023-10-06T04:58:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.033+0000 7f3f69ca7700 1 -- 172.21.15.3:0/667416143 wait complete. 2023-10-06T04:58:12.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:58:12.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.166+0000 7f31399f5700 1 Processor -- start 2023-10-06T04:58:12.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.166+0000 7f31399f5700 1 -- start start 2023-10-06T04:58:12.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31399f5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af290 0x7f31340ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31399f5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f31340af660 0x7f31340ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31399f5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340cb190 con 0x7f31340cd190 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31399f5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3134159950 con 0x7f31340af660 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31399f5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31340cd6a0 con 0x7f31340af290 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31391f4700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f31340cd190 0x7f31340c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48720/0 (socket says 172.21.15.3:48720) 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31391f4700 1 -- 172.21.15.3:0/2704473961 learned_addr learned my addr 172.21.15.3:0/2704473961 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f3133fff700 1 --2- 172.21.15.3:0/2704473961 >> v2:172.21.15.153:3300/0 conn(0x7f31340af660 0x7f31340ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.167+0000 7f31389f3700 1 --2- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af290 0x7f31340ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f31337fe700 1 -- 172.21.15.3:0/2704473961 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283291671 0 0) 0x7f31340cb190 con 0x7f31340cd190 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f31337fe700 1 -- 172.21.15.3:0/2704473961 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3120003570 con 0x7f31340cd190 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f3133fff700 1 -- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af290 msgr2=0x7f31340ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f3133fff700 1 --2- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af290 0x7f31340ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f3133fff700 1 -- 172.21.15.3:0/2704473961 >> v1:172.21.15.3:6789/0 conn(0x7f31340cd190 legacy=0x7f31340c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f3133fff700 1 -- 172.21.15.3:0/2704473961 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31340cb3f0 con 0x7f31340af660 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f31389f3700 1 --2- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af290 0x7f31340ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f3133fff700 1 --2- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af660 0x7f31340ccc50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3124009eb0 tx=0x7f3124009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a1e4702eb19ec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f31337fe700 1 -- 172.21.15.3:0/2704473961 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312400e070 con 0x7f31340af660 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f31337fe700 1 -- 172.21.15.3:0/2704473961 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312400bb80 con 0x7f31340af660 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.168+0000 7f31337fe700 1 -- 172.21.15.3:0/2704473961 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124012430 con 0x7f31340af660 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 -- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af660 msgr2=0x7f31340ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 --2- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af660 0x7f31340ccc50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3124009eb0 tx=0x7f3124009780 comp rx=0 tx=0).stop 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 -- 172.21.15.3:0/2704473961 shutdown_connections 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 --2- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af290 0x7f31340ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 --2- 172.21.15.3:0/2704473961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af660 0x7f31340ccc50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 -- 172.21.15.3:0/2704473961 >> 172.21.15.3:0/2704473961 conn(0x7f3134089dd0 msgr2=0x7f31340996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:12.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 -- 172.21.15.3:0/2704473961 shutdown_connections 2023-10-06T04:58:12.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.169+0000 7f31399f5700 1 -- 172.21.15.3:0/2704473961 wait complete. 2023-10-06T04:58:12.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.170+0000 7f31399f5700 1 Processor -- start 2023-10-06T04:58:12.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.170+0000 7f31399f5700 1 -- start start 2023-10-06T04:58:12.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af290 0x7f3134156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af660 0x7f31340b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31340cd190 0x7f31340b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31340c97d0 con 0x7f31340cd190 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31340c9210 con 0x7f31340af290 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31340c94f0 con 0x7f31340af660 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31340cd190 0x7f31340b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31340cd190 0x7f31340b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53028/0 (socket says 172.21.15.3:53028) 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 -- 172.21.15.3:0/2867755887 learned_addr learned my addr 172.21.15.3:0/2867755887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f3133fff700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af660 0x7f31340b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31389f3700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af290 0x7f3134156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 -- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af660 msgr2=0x7f31340b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af660 0x7f31340b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 -- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af290 msgr2=0x7f3134156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af290 0x7f3134156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 -- 172.21.15.3:0/2867755887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0093c0 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31391f4700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31340cd190 0x7f31340b2090 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c790 tx=0x7f311c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f3130ff9700 1 -- 172.21.15.3:0/2867755887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c005c00 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f3130ff9700 1 -- 172.21.15.3:0/2867755887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c005d60 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3124009a20 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f3130ff9700 1 -- 172.21.15.3:0/2867755887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0072e0 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.171+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31340afe80 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.172+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313404c380 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.173+0000 7f3130ff9700 1 -- 172.21.15.3:0/2867755887 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f311c01b3b0 con 0x7f31340cd190 2023-10-06T04:58:12.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.173+0000 7f3130ff9700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3114066f30 0x7f31140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.173+0000 7f3130ff9700 1 -- 172.21.15.3:0/2867755887 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f311c083750 con 0x7f31340cd190 2023-10-06T04:58:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.174+0000 7f31389f3700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3114066f30 0x7f31140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.174+0000 7f31389f3700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3114066f30 0x7f31140693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f312c009390 tx=0x7f312c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.177+0000 7f3130ff9700 1 -- 172.21.15.3:0/2867755887 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f311c04b5a0 con 0x7f31340cd190 2023-10-06T04:58:12.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.371+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3134159da0 con 0x7f31340cd190 2023-10-06T04:58:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.374+0000 7f3130ff9700 1 -- 172.21.15.3:0/2867755887 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f311c050c10 con 0x7f31340cd190 2023-10-06T04:58:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3114066f30 msgr2=0x7f31140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3114066f30 0x7f31140693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f312c009390 tx=0x7f312c008040 comp rx=0 tx=0).stop 2023-10-06T04:58:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31340cd190 msgr2=0x7f31340b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31340cd190 0x7f31340b2090 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c790 tx=0x7f311c007d90 comp rx=0 tx=0).stop 2023-10-06T04:58:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 shutdown_connections 2023-10-06T04:58:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3114066f30 0x7f31140693e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31340af290 0x7f3134156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31340cd190 0x7f31340b2090 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 --2- 172.21.15.3:0/2867755887 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31340af660 0x7f31340b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 >> 172.21.15.3:0/2867755887 conn(0x7f3134089dd0 msgr2=0x7f31340adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:12.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 shutdown_connections 2023-10-06T04:58:12.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.376+0000 7f31399f5700 1 -- 172.21.15.3:0/2867755887 wait complete. 2023-10-06T04:58:12.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-10-06T04:58:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.509+0000 7f1221ad6700 1 Processor -- start 2023-10-06T04:58:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.509+0000 7f1221ad6700 1 -- start start 2023-10-06T04:58:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1221ad6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f121c15cae0 0x7f121c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1221ad6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f121c06e010 0x7f121c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1221ad6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f121c0b1e70 con 0x7f121c06d480 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1221ad6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121c0b1860 con 0x7f121c15cae0 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1221ad6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121c06eaf0 con 0x7f121c06e010 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1220ad4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f121c15cae0 0x7f121c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f121bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f121c06d480 0x7f121c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44234/0 (socket says 172.21.15.3:44234) 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f121bfff700 1 -- 172.21.15.3:0/3438886189 learned_addr learned my addr 172.21.15.3:0/3438886189 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f12212d5700 1 --2- 172.21.15.3:0/3438886189 >> v2:172.21.15.153:3300/0 conn(0x7f121c06e010 0x7f121c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1220ad4700 1 -- 172.21.15.3:0/3438886189 >> v2:172.21.15.153:3300/0 conn(0x7f121c06e010 msgr2=0x7f121c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1220ad4700 1 --2- 172.21.15.3:0/3438886189 >> v2:172.21.15.153:3300/0 conn(0x7f121c06e010 0x7f121c0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1220ad4700 1 -- 172.21.15.3:0/3438886189 >> v1:172.21.15.3:6789/0 conn(0x7f121c06d480 legacy=0x7f121c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f1220ad4700 1 -- 172.21.15.3:0/3438886189 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c06e530 con 0x7f121c15cae0 2023-10-06T04:58:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f121b7fe700 1 -- 172.21.15.3:0/3438886189 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751656194 0 0) 0x7f121c0b1e70 con 0x7f121c06d480 2023-10-06T04:58:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.510+0000 7f12212d5700 1 --2- 172.21.15.3:0/3438886189 >> v2:172.21.15.153:3300/0 conn(0x7f121c06e010 0x7f121c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1220ad4700 1 --2- 172.21.15.3:0/3438886189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c15cae0 0x7f121c0b0430 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1214009eb0 tx=0x7f1214009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bd8fc87563fc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f121b7fe700 1 -- 172.21.15.3:0/3438886189 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121400e070 con 0x7f121c15cae0 2023-10-06T04:58:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f121b7fe700 1 -- 172.21.15.3:0/3438886189 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121400bbf0 con 0x7f121c15cae0 2023-10-06T04:58:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f121b7fe700 1 -- 172.21.15.3:0/3438886189 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214012430 con 0x7f121c15cae0 2023-10-06T04:58:12.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 -- 172.21.15.3:0/3438886189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c15cae0 msgr2=0x7f121c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 --2- 172.21.15.3:0/3438886189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c15cae0 0x7f121c0b0430 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1214009eb0 tx=0x7f1214009780 comp rx=0 tx=0).stop 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 -- 172.21.15.3:0/3438886189 shutdown_connections 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 --2- 172.21.15.3:0/3438886189 >> v2:172.21.15.153:3300/0 conn(0x7f121c06e010 0x7f121c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 --2- 172.21.15.3:0/3438886189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c15cae0 0x7f121c0b0430 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 -- 172.21.15.3:0/3438886189 >> 172.21.15.3:0/3438886189 conn(0x7f121c06cb50 msgr2=0x7f121c0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 -- 172.21.15.3:0/3438886189 shutdown_connections 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.511+0000 7f1221ad6700 1 -- 172.21.15.3:0/3438886189 wait complete. 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.512+0000 7f1221ad6700 1 Processor -- start 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.513+0000 7f1221ad6700 1 -- start start 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.513+0000 7f1221ad6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c06d480 0x7f121c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.513+0000 7f1221ad6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f121c06e010 0x7f121c09aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.513+0000 7f1221ad6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121c15cae0 0x7f121c09b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.513+0000 7f1221ad6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c099700 con 0x7f121c06d480 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.513+0000 7f1221ad6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c0986f0 con 0x7f121c15cae0 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.513+0000 7f1221ad6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121c099270 con 0x7f121c06e010 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c06d480 0x7f121c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c06d480 0x7f121c1586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54584/0 (socket says 172.21.15.3:54584) 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 -- 172.21.15.3:0/1067976051 learned_addr learned my addr 172.21.15.3:0/1067976051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 -- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f121c06e010 msgr2=0x7f121c09aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f121c06e010 0x7f121c09aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 -- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121c15cae0 msgr2=0x7f121c09b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121c15cae0 0x7f121c09b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 -- 172.21.15.3:0/1067976051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214009a20 con 0x7f121c06d480 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1220ad4700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c06d480 0x7f121c1586f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f121c06a820 tx=0x7f1214017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1218ff9700 1 -- 172.21.15.3:0/1067976051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214012e80 con 0x7f121c06d480 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1218ff9700 1 -- 172.21.15.3:0/1067976051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121401c610 con 0x7f121c06d480 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c158c30 con 0x7f121c06d480 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1218ff9700 1 -- 172.21.15.3:0/1067976051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121401c990 con 0x7f121c06d480 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.514+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121c098c20 con 0x7f121c06d480 2023-10-06T04:58:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.515+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121c04c380 con 0x7f121c06d480 2023-10-06T04:58:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.516+0000 7f1218ff9700 1 -- 172.21.15.3:0/1067976051 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f121401cd40 con 0x7f121c06d480 2023-10-06T04:58:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.516+0000 7f1218ff9700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1208066f30 0x7f12080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.516+0000 7f1218ff9700 1 -- 172.21.15.3:0/1067976051 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f12140864c0 con 0x7f121c06d480 2023-10-06T04:58:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.516+0000 7f121bfff700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1208066f30 0x7f12080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.517+0000 7f121bfff700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1208066f30 0x7f12080693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f12100011e0 tx=0x7f12100079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.520+0000 7f1218ff9700 1 -- 172.21.15.3:0/1067976051 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12140538d0 con 0x7f121c06d480 2023-10-06T04:58:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.715+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f121c06e810 con 0x7f121c06d480 2023-10-06T04:58:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.718+0000 7f1218ff9700 1 -- 172.21.15.3:0/1067976051 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v29) v1 ==== 107+0+0 (secure 0 0 0) 0x7f1214056ef0 con 0x7f121c06d480 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1208066f30 msgr2=0x7f12080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1208066f30 0x7f12080693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f12100011e0 tx=0x7f12100079d0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c06d480 msgr2=0x7f121c1586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c06d480 0x7f121c1586f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f121c06a820 tx=0x7f1214017000 comp rx=0 tx=0).stop 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 shutdown_connections 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1208066f30 0x7f12080693e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121c15cae0 0x7f121c09b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121c06d480 0x7f121c1586f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 --2- 172.21.15.3:0/1067976051 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f121c06e010 0x7f121c09aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 >> 172.21.15.3:0/1067976051 conn(0x7f121c06cb50 msgr2=0x7f121c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 shutdown_connections 2023-10-06T04:58:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.721+0000 7f1221ad6700 1 -- 172.21.15.3:0/1067976051 wait complete. 2023-10-06T04:58:12.735 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-10-06T04:58:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.853+0000 7feb2aa96700 1 Processor -- start 2023-10-06T04:58:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.853+0000 7feb2aa96700 1 -- start start 2023-10-06T04:58:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2aa96700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a820 0x7feb2415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2aa96700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb240c5a80 0x7feb240c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2aa96700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb240ce1b0 con 0x7feb2415a450 2023-10-06T04:58:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2aa96700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb24071200 con 0x7feb240c5a80 2023-10-06T04:58:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2aa96700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2415b3b0 con 0x7feb2415a820 2023-10-06T04:58:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2a295700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb240c5a80 0x7feb240c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2a295700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb240c5a80 0x7feb240c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54600/0 (socket says 172.21.15.3:54600) 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2a295700 1 -- 172.21.15.3:0/4050129391 learned_addr learned my addr 172.21.15.3:0/4050129391 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb29293700 1 --2- 172.21.15.3:0/4050129391 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a820 0x7feb2415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2a295700 1 -- 172.21.15.3:0/4050129391 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a820 msgr2=0x7feb2415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2a295700 1 --2- 172.21.15.3:0/4050129391 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a820 0x7feb2415ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2a295700 1 -- 172.21.15.3:0/4050129391 >> v1:172.21.15.153:6789/0 conn(0x7feb2415a450 legacy=0x7feb24070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.854+0000 7feb2a295700 1 -- 172.21.15.3:0/4050129391 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240c6420 con 0x7feb240c5a80 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2a295700 1 --2- 172.21.15.3:0/4050129391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240c5ee0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7feb14009f20 tx=0x7feb14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694ea395ae727dfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb28a92700 1 -- 172.21.15.3:0/4050129391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1400e070 con 0x7feb240c5a80 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb28a92700 1 -- 172.21.15.3:0/4050129391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1400ba80 con 0x7feb240c5a80 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb28a92700 1 -- 172.21.15.3:0/4050129391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140092e0 con 0x7feb240c5a80 2023-10-06T04:58:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 -- 172.21.15.3:0/4050129391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 msgr2=0x7feb240c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 --2- 172.21.15.3:0/4050129391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240c5ee0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7feb14009f20 tx=0x7feb14009840 comp rx=0 tx=0).stop 2023-10-06T04:58:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 -- 172.21.15.3:0/4050129391 shutdown_connections 2023-10-06T04:58:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 --2- 172.21.15.3:0/4050129391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240c5ee0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 --2- 172.21.15.3:0/4050129391 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a820 0x7feb2415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 -- 172.21.15.3:0/4050129391 >> 172.21.15.3:0/4050129391 conn(0x7feb24089dd0 msgr2=0x7feb240996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 -- 172.21.15.3:0/4050129391 shutdown_connections 2023-10-06T04:58:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.855+0000 7feb2aa96700 1 -- 172.21.15.3:0/4050129391 wait complete. 2023-10-06T04:58:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.856+0000 7feb2aa96700 1 Processor -- start 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.856+0000 7feb2aa96700 1 -- start start 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb2aa96700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb2aa96700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a450 0x7feb24147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb2aa96700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2415a820 0x7feb240b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb2aa96700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240cd710 con 0x7feb240c5a80 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb2aa96700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240b5f20 con 0x7feb2415a820 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb2aa96700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2416ab50 con 0x7feb2415a450 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb29a94700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb29a94700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54608/0 (socket says 172.21.15.3:54608) 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb29a94700 1 -- 172.21.15.3:0/2472425290 learned_addr learned my addr 172.21.15.3:0/2472425290 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb2a295700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2415a820 0x7feb240b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.857+0000 7feb29293700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a450 0x7feb24147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb29a94700 1 -- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a450 msgr2=0x7feb24147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb29a94700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a450 0x7feb24147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb29a94700 1 -- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2415a820 msgr2=0x7feb240b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb29a94700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2415a820 0x7feb240b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb29a94700 1 -- 172.21.15.3:0/2472425290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14009ae0 con 0x7feb240c5a80 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb29a94700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240b59e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c590 tx=0x7feb1c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb21ffb700 1 -- 172.21.15.3:0/2472425290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c011590 con 0x7feb240c5a80 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb21ffb700 1 -- 172.21.15.3:0/2472425290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c011710 con 0x7feb240c5a80 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb24148280 con 0x7feb240c5a80 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb21ffb700 1 -- 172.21.15.3:0/2472425290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c011ad0 con 0x7feb240c5a80 2023-10-06T04:58:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.858+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb24169f20 con 0x7feb240c5a80 2023-10-06T04:58:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.859+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2404c380 con 0x7feb240c5a80 2023-10-06T04:58:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.859+0000 7feb21ffb700 1 -- 172.21.15.3:0/2472425290 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb1c01b3b0 con 0x7feb240c5a80 2023-10-06T04:58:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.860+0000 7feb21ffb700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feb10066f30 0x7feb100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.860+0000 7feb21ffb700 1 -- 172.21.15.3:0/2472425290 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7feb1c083820 con 0x7feb240c5a80 2023-10-06T04:58:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.860+0000 7feb29293700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feb10066f30 0x7feb100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.861+0000 7feb29293700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feb10066f30 0x7feb100693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feb240b32f0 tx=0x7feb18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:12.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:12.864+0000 7feb21ffb700 1 -- 172.21.15.3:0/2472425290 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb1c050c30 con 0x7feb240c5a80 2023-10-06T04:58:13.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.060+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7feb240cdac0 con 0x7feb240c5a80 2023-10-06T04:58:13.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.064+0000 7feb21ffb700 1 -- 172.21.15.3:0/2472425290 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v30) v1 ==== 102+0+0 (secure 0 0 0) 0x7feb1c017070 con 0x7feb240c5a80 2023-10-06T04:58:13.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feb10066f30 msgr2=0x7feb100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feb10066f30 0x7feb100693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feb240b32f0 tx=0x7feb18008040 comp rx=0 tx=0).stop 2023-10-06T04:58:13.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 msgr2=0x7feb240b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240b59e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c590 tx=0x7feb1c00c8a0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 shutdown_connections 2023-10-06T04:58:13.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feb10066f30 0x7feb100693e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2415a820 0x7feb240b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb240c5a80 0x7feb240b59e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 --2- 172.21.15.3:0/2472425290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb2415a450 0x7feb24147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 >> 172.21.15.3:0/2472425290 conn(0x7feb24089dd0 msgr2=0x7feb240acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:13.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 shutdown_connections 2023-10-06T04:58:13.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.066+0000 7feb2aa96700 1 -- 172.21.15.3:0/2472425290 wait complete. 2023-10-06T04:58:13.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-10-06T04:58:13.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.199+0000 7f25a653f700 1 Processor -- start 2023-10-06T04:58:13.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.199+0000 7f25a653f700 1 -- start start 2023-10-06T04:58:13.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a653f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25a00af290 0x7f25a00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a653f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f25a00af660 0x7f25a00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a653f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a00cb190 con 0x7f25a00cd190 2023-10-06T04:58:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a653f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a0159950 con 0x7f25a00af290 2023-10-06T04:58:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a653f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00cd6a0 con 0x7f25a00af660 2023-10-06T04:58:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a553d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25a00af290 0x7f25a00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a5d3e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f25a00cd190 0x7f25a00c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44246/0 (socket says 172.21.15.3:44246) 2023-10-06T04:58:13.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a5d3e700 1 -- 172.21.15.3:0/964913330 learned_addr learned my addr 172.21.15.3:0/964913330 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.200+0000 7f25a4d3c700 1 --2- 172.21.15.3:0/964913330 >> v2:172.21.15.153:3300/0 conn(0x7f25a00af660 0x7f25a00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a553d700 1 -- 172.21.15.3:0/964913330 >> v2:172.21.15.153:3300/0 conn(0x7f25a00af660 msgr2=0x7f25a00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a553d700 1 --2- 172.21.15.3:0/964913330 >> v2:172.21.15.153:3300/0 conn(0x7f25a00af660 0x7f25a00ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a553d700 1 -- 172.21.15.3:0/964913330 >> v1:172.21.15.3:6789/0 conn(0x7f25a00cd190 legacy=0x7f25a00c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f259ffff700 1 -- 172.21.15.3:0/964913330 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855660268 0 0) 0x7f25a00cb190 con 0x7f25a00cd190 2023-10-06T04:58:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a553d700 1 -- 172.21.15.3:0/964913330 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00cb3f0 con 0x7f25a00af290 2023-10-06T04:58:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a553d700 1 --2- 172.21.15.3:0/964913330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af290 0x7f25a00ce1b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f25a00d12c0 tx=0x7f2590009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79560f7e1ccffac6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f259ffff700 1 -- 172.21.15.3:0/964913330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259000e070 con 0x7f25a00af290 2023-10-06T04:58:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f259ffff700 1 -- 172.21.15.3:0/964913330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259000bc30 con 0x7f25a00af290 2023-10-06T04:58:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f259ffff700 1 -- 172.21.15.3:0/964913330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590012430 con 0x7f25a00af290 2023-10-06T04:58:13.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a653f700 1 -- 172.21.15.3:0/964913330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af290 msgr2=0x7f25a00ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a653f700 1 --2- 172.21.15.3:0/964913330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af290 0x7f25a00ce1b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f25a00d12c0 tx=0x7f2590009780 comp rx=0 tx=0).stop 2023-10-06T04:58:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a653f700 1 -- 172.21.15.3:0/964913330 shutdown_connections 2023-10-06T04:58:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a653f700 1 --2- 172.21.15.3:0/964913330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af290 0x7f25a00ce1b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a653f700 1 --2- 172.21.15.3:0/964913330 >> v2:172.21.15.153:3300/0 conn(0x7f25a00af660 0x7f25a00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.201+0000 7f25a653f700 1 -- 172.21.15.3:0/964913330 >> 172.21.15.3:0/964913330 conn(0x7f25a0089dd0 msgr2=0x7f25a00996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.202+0000 7f25a653f700 1 -- 172.21.15.3:0/964913330 shutdown_connections 2023-10-06T04:58:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.202+0000 7f25a653f700 1 -- 172.21.15.3:0/964913330 wait complete. 2023-10-06T04:58:13.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 Processor -- start 2023-10-06T04:58:13.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 -- start start 2023-10-06T04:58:13.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f25a00af290 0x7f25a014fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af660 0x7f25a01692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25a00cd190 0x7f25a00b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00cf6b0 con 0x7f25a00af660 2023-10-06T04:58:13.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00cf380 con 0x7f25a00cd190 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.203+0000 7f25a653f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0169c60 con 0x7f25a00af290 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af660 0x7f25a01692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af660 0x7f25a01692f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54622/0 (socket says 172.21.15.3:54622) 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 -- 172.21.15.3:0/797514548 learned_addr learned my addr 172.21.15.3:0/797514548 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a553d700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f25a00af290 0x7f25a014fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a5d3e700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25a00cd190 0x7f25a00b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 -- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f25a00af290 msgr2=0x7f25a014fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f25a00af290 0x7f25a014fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 -- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25a00cd190 msgr2=0x7f25a00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25a00cd190 0x7f25a00b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 -- 172.21.15.3:0/797514548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590009a20 con 0x7f25a00af660 2023-10-06T04:58:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a553d700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f25a00af290 0x7f25a014fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a5d3e700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25a00cd190 0x7f25a00b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a4d3c700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af660 0x7f25a01692f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f259400d860 tx=0x7f2594008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f259d7fa700 1 -- 172.21.15.3:0/797514548 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594012630 con 0x7f25a00af660 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f259d7fa700 1 -- 172.21.15.3:0/797514548 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594012790 con 0x7f25a00af660 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a0169f40 con 0x7f25a00af660 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f259d7fa700 1 -- 172.21.15.3:0/797514548 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594012b50 con 0x7f25a00af660 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.204+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a016a220 con 0x7f25a00af660 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.205+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a00cf820 con 0x7f25a00af660 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.206+0000 7f259d7fa700 1 -- 172.21.15.3:0/797514548 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f259401d400 con 0x7f25a00af660 2023-10-06T04:58:13.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.206+0000 7f259d7fa700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f258c066f30 0x7f258c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.206+0000 7f259d7fa700 1 -- 172.21.15.3:0/797514548 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2594084870 con 0x7f25a00af660 2023-10-06T04:58:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.206+0000 7f25a553d700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f258c066f30 0x7f258c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.207+0000 7f25a553d700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f258c066f30 0x7f258c0693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f25a00d12c0 tx=0x7f2590019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.210+0000 7f259d7fa700 1 -- 172.21.15.3:0/797514548 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2594055710 con 0x7f25a00af660 2023-10-06T04:58:13.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.404+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f25a004c380 con 0x7f25a00af660 2023-10-06T04:58:13.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.407+0000 7f259d7fa700 1 -- 172.21.15.3:0/797514548 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v31) v1 ==== 111+0+0 (secure 0 0 0) 0x7f25940552a0 con 0x7f25a00af660 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f258c066f30 msgr2=0x7f258c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f258c066f30 0x7f258c0693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f25a00d12c0 tx=0x7f2590019000 comp rx=0 tx=0).stop 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af660 msgr2=0x7f25a01692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af660 0x7f25a01692f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f259400d860 tx=0x7f2594008040 comp rx=0 tx=0).stop 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 shutdown_connections 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f258c066f30 0x7f258c0693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25a00cd190 0x7f25a00b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f25a00af290 0x7f25a014fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 --2- 172.21.15.3:0/797514548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a00af660 0x7f25a01692f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 >> 172.21.15.3:0/797514548 conn(0x7f25a0089dd0 msgr2=0x7f25a00ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:13.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 shutdown_connections 2023-10-06T04:58:13.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.409+0000 7f25a653f700 1 -- 172.21.15.3:0/797514548 wait complete. 2023-10-06T04:58:13.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2023-10-06T04:58:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.540+0000 7f76added700 1 Processor -- start 2023-10-06T04:58:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.540+0000 7f76added700 1 -- start start 2023-10-06T04:58:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.541+0000 7f76added700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76a80cb3b0 0x7f76a80cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.541+0000 7f76added700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80cf130 con 0x7f76a81596d0 2023-10-06T04:58:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.541+0000 7f76added700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80ce520 con 0x7f76a80cb780 2023-10-06T04:58:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.541+0000 7f76added700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a80ce800 con 0x7f76a80cb3b0 2023-10-06T04:58:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.541+0000 7f76acdeb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76a80cb3b0 0x7f76a80cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.541+0000 7f76ad5ec700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f76a81596d0 0x7f76a80cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44250/0 (socket says 172.21.15.3:44250) 2023-10-06T04:58:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.541+0000 7f76ad5ec700 1 -- 172.21.15.3:0/1022272354 learned_addr learned my addr 172.21.15.3:0/1022272354 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76acdeb700 1 -- 172.21.15.3:0/1022272354 >> v1:172.21.15.153:6789/0 conn(0x7f76a80cb780 legacy=0x7f76a8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76acdeb700 1 -- 172.21.15.3:0/1022272354 >> v1:172.21.15.3:6789/0 conn(0x7f76a81596d0 legacy=0x7f76a80cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76acdeb700 1 -- 172.21.15.3:0/1022272354 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80ceae0 con 0x7f76a80cb3b0 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76a77fe700 1 -- 172.21.15.3:0/1022272354 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105948769 0 0) 0x7f76a80cf130 con 0x7f76a81596d0 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76acdeb700 1 --2- 172.21.15.3:0/1022272354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb3b0 0x7f76a80cf5c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f76980098e0 tx=0x7f76980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab164435d97dfd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76a77fe700 1 -- 172.21.15.3:0/1022272354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698009e80 con 0x7f76a80cb3b0 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76a77fe700 1 -- 172.21.15.3:0/1022272354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769800b860 con 0x7f76a80cb3b0 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76a77fe700 1 -- 172.21.15.3:0/1022272354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800bbe0 con 0x7f76a80cb3b0 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76added700 1 -- 172.21.15.3:0/1022272354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb3b0 msgr2=0x7f76a80cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76added700 1 --2- 172.21.15.3:0/1022272354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb3b0 0x7f76a80cf5c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f76980098e0 tx=0x7f76980091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76added700 1 -- 172.21.15.3:0/1022272354 shutdown_connections 2023-10-06T04:58:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76added700 1 --2- 172.21.15.3:0/1022272354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb3b0 0x7f76a80cf5c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76added700 1 -- 172.21.15.3:0/1022272354 >> 172.21.15.3:0/1022272354 conn(0x7f76a8070310 msgr2=0x7f76a8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.542+0000 7f76added700 1 -- 172.21.15.3:0/1022272354 shutdown_connections 2023-10-06T04:58:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.543+0000 7f76added700 1 -- 172.21.15.3:0/1022272354 wait complete. 2023-10-06T04:58:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.544+0000 7f76added700 1 Processor -- start 2023-10-06T04:58:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.544+0000 7f76added700 1 -- start start 2023-10-06T04:58:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.544+0000 7f76added700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76a80cb3b0 0x7f76a816adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76added700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb780 0x7f76a8155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76added700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a81596d0 0x7f76a8169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76added700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80c8330 con 0x7f76a80cb780 2023-10-06T04:58:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76added700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80c8bd0 con 0x7f76a81596d0 2023-10-06T04:58:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76added700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a816ab10 con 0x7f76a80cb3b0 2023-10-06T04:58:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb780 0x7f76a8155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb780 0x7f76a8155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54650/0 (socket says 172.21.15.3:54650) 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 -- 172.21.15.3:0/318749061 learned_addr learned my addr 172.21.15.3:0/318749061 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76acdeb700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76a80cb3b0 0x7f76a816adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76ad5ec700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a81596d0 0x7f76a8169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 -- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76a80cb3b0 msgr2=0x7f76a816adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76a80cb3b0 0x7f76a816adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 -- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a81596d0 msgr2=0x7f76a8169d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a81596d0 0x7f76a8169d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 -- 172.21.15.3:0/318749061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698009420 con 0x7f76a80cb780 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76acdeb700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76a80cb3b0 0x7f76a816adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a7fff700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb780 0x7f76a8155820 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c590 tx=0x7f769c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a4ff9700 1 -- 172.21.15.3:0/318749061 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c011590 con 0x7f76a80cb780 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a4ff9700 1 -- 172.21.15.3:0/318749061 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c0116f0 con 0x7f76a80cb780 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76a4ff9700 1 -- 172.21.15.3:0/318749061 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c011ab0 con 0x7f76a80cb780 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76added700 1 -- 172.21.15.3:0/318749061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a80c8580 con 0x7f76a80cb780 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.545+0000 7f76added700 1 -- 172.21.15.3:0/318749061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a816b330 con 0x7f76a80cb780 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.546+0000 7f76added700 1 -- 172.21.15.3:0/318749061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a80c8d80 con 0x7f76a80cb780 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.547+0000 7f76a4ff9700 1 -- 172.21.15.3:0/318749061 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f769c01b3b0 con 0x7f76a80cb780 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.547+0000 7f76a4ff9700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7694066f30 0x7f76940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.547+0000 7f76a4ff9700 1 -- 172.21.15.3:0/318749061 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f769c0838f0 con 0x7f76a80cb780 2023-10-06T04:58:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.547+0000 7f76acdeb700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7694066f30 0x7f76940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.548+0000 7f76acdeb700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7694066f30 0x7f76940693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f76a80c5440 tx=0x7f7698017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.552+0000 7f76a4ff9700 1 -- 172.21.15.3:0/318749061 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76a80c8d80 con 0x7f76a80cb780 2023-10-06T04:58:13.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:13.829+0000 7f76added700 1 -- 172.21.15.3:0/318749061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) v1 -- 0x7f76a80c8d40 con 0x7f76a80cb780 2023-10-06T04:58:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.673+0000 7f76a4ff9700 1 -- 172.21.15.3:0/318749061 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v290) v1 ==== 160+0+0 (secure 0 0 0) 0x7f769c017020 con 0x7f76a80cb780 2023-10-06T04:58:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 -- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7694066f30 msgr2=0x7f76940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7694066f30 0x7f76940693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f76a80c5440 tx=0x7f7698017000 comp rx=0 tx=0).stop 2023-10-06T04:58:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 -- 172.21.15.3:0/318749061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb780 msgr2=0x7f76a8155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb780 0x7f76a8155820 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c590 tx=0x7f769c00c950 comp rx=0 tx=0).stop 2023-10-06T04:58:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 -- 172.21.15.3:0/318749061 shutdown_connections 2023-10-06T04:58:14.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7694066f30 0x7f76940693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:14.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a81596d0 0x7f76a8169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:14.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76a80cb3b0 0x7f76a816adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:14.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 --2- 172.21.15.3:0/318749061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a80cb780 0x7f76a8155820 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:14.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 -- 172.21.15.3:0/318749061 >> 172.21.15.3:0/318749061 conn(0x7f76a8070310 msgr2=0x7f76a8154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:14.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 -- 172.21.15.3:0/318749061 shutdown_connections 2023-10-06T04:58:14.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.677+0000 7f76added700 1 -- 172.21.15.3:0/318749061 wait complete. 2023-10-06T04:58:14.694 INFO:tasks.cephfs_test_runner:When a client has inodes pinned in its cache (open files), that the MDS ... ok 2023-10-06T04:58:14.694 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) When a client does not advance its oldest tid, the MDS should notice that 2023-10-06T04:58:14.695 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T04:58:14.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2023-10-06T04:58:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.819+0000 7fdb6d4d9700 1 Processor -- start 2023-10-06T04:58:14.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.819+0000 7fdb6d4d9700 1 -- start start 2023-10-06T04:58:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.820+0000 7fdb6d4d9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb680cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.820+0000 7fdb6d4d9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb680cf130 con 0x7fdb680cb780 2023-10-06T04:58:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.820+0000 7fdb6d4d9700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb680ce520 con 0x7fdb680cb3b0 2023-10-06T04:58:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.820+0000 7fdb6d4d9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680ce800 con 0x7fdb681596d0 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.820+0000 7fdb677fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdb680cb780 0x7fdb68158fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44264/0 (socket says 172.21.15.3:44264) 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.820+0000 7fdb677fe700 1 -- 172.21.15.3:0/2319255864 learned_addr learned my addr 172.21.15.3:0/2319255864 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.820+0000 7fdb6ccd8700 1 --2- 172.21.15.3:0/2319255864 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb680cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242932211 0 0) 0x7fdb680cf130 con 0x7fdb680cb780 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb54003570 con 0x7fdb680cb780 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191222091 0 0) 0x7fdb680ce520 con 0x7fdb680cb3b0 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb680cf130 con 0x7fdb680cb3b0 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb6ccd8700 1 -- 172.21.15.3:0/2319255864 >> v1:172.21.15.153:6789/0 conn(0x7fdb680cb3b0 legacy=0x7fdb680cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb6ccd8700 1 -- 172.21.15.3:0/2319255864 >> v1:172.21.15.3:6789/0 conn(0x7fdb680cb780 legacy=0x7fdb68158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3136686454 0 0) 0x7fdb54003570 con 0x7fdb680cb780 2023-10-06T04:58:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb6ccd8700 1 -- 172.21.15.3:0/2319255864 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb680ceae0 con 0x7fdb681596d0 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb6ccd8700 1 --2- 172.21.15.3:0/2319255864 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb680cccc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009e90 tx=0x7fdb60009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe0020707ca6c8b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000e070 con 0x7fdb681596d0 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6000b870 con 0x7fdb681596d0 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.821+0000 7fdb66ffd700 1 -- 172.21.15.3:0/2319255864 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58001470 con 0x7fdb681596d0 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.823+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2319255864 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 msgr2=0x7fdb680cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.823+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2319255864 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb680cccc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009e90 tx=0x7fdb60009780 comp rx=0 tx=0).stop 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.823+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2319255864 shutdown_connections 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.823+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2319255864 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb680cccc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.823+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2319255864 >> 172.21.15.3:0/2319255864 conn(0x7fdb68070310 msgr2=0x7fdb68070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.823+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2319255864 shutdown_connections 2023-10-06T04:58:14.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.823+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2319255864 wait complete. 2023-10-06T04:58:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 Processor -- start 2023-10-06T04:58:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 -- start start 2023-10-06T04:58:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb680cb3b0 0x7fdb68143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680cb780 0x7fdb680b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb68153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680c9d90 con 0x7fdb680cb3b0 2023-10-06T04:58:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680c9180 con 0x7fdb680cb780 2023-10-06T04:58:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6d4d9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680c9460 con 0x7fdb681596d0 2023-10-06T04:58:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb680cb3b0 0x7fdb68143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb680cb3b0 0x7fdb68143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54654/0 (socket says 172.21.15.3:54654) 2023-10-06T04:58:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 -- 172.21.15.3:0/2321865579 learned_addr learned my addr 172.21.15.3:0/2321865579 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:14.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb6ccd8700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb68153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb677fe700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680cb780 0x7fdb680b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 -- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 msgr2=0x7fdb68153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb68153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 -- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680cb780 msgr2=0x7fdb680b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680cb780 0x7fdb680b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 -- 172.21.15.3:0/2321865579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60009a20 con 0x7fdb680cb3b0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.824+0000 7fdb67fff700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb680cb3b0 0x7fdb68143be0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c790 tx=0x7fdb50007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.826+0000 7fdb5ffff700 1 -- 172.21.15.3:0/2321865579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50010510 con 0x7fdb680cb3b0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.826+0000 7fdb5ffff700 1 -- 172.21.15.3:0/2321865579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50010670 con 0x7fdb680cb3b0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.826+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb680c9740 con 0x7fdb680cb3b0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.826+0000 7fdb5ffff700 1 -- 172.21.15.3:0/2321865579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50010a10 con 0x7fdb680cb3b0 2023-10-06T04:58:14.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.826+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb680c8e50 con 0x7fdb680cb3b0 2023-10-06T04:58:14.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.827+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb680ca070 con 0x7fdb680cb3b0 2023-10-06T04:58:14.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.827+0000 7fdb5ffff700 1 -- 172.21.15.3:0/2321865579 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb5001b400 con 0x7fdb680cb3b0 2023-10-06T04:58:14.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.827+0000 7fdb5ffff700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb48066f60 0x7fdb48069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:14.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.828+0000 7fdb5ffff700 1 -- 172.21.15.3:0/2321865579 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fdb500841e0 con 0x7fdb680cb3b0 2023-10-06T04:58:14.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.828+0000 7fdb677fe700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb48066f60 0x7fdb48069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:14.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.828+0000 7fdb677fe700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb48066f60 0x7fdb48069410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb6816cab0 tx=0x7fdb58000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:14.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:14.832+0000 7fdb5ffff700 1 -- 172.21.15.3:0/2321865579 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb50051550 con 0x7fdb680cb3b0 2023-10-06T04:58:15.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.106+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) v1 -- 0x7fdb6804c380 con 0x7fdb680cb3b0 2023-10-06T04:58:15.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.678+0000 7fdb5ffff700 1 -- 172.21.15.3:0/2321865579 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v291) v1 ==== 153+0+0 (secure 0 0 0) 0x7fdb50054b70 con 0x7fdb680cb3b0 2023-10-06T04:58:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb48066f60 msgr2=0x7fdb48069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb48066f60 0x7fdb48069410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb6816cab0 tx=0x7fdb58000bc0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb680cb3b0 msgr2=0x7fdb68143be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb680cb3b0 0x7fdb68143be0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c790 tx=0x7fdb50007e10 comp rx=0 tx=0).stop 2023-10-06T04:58:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 shutdown_connections 2023-10-06T04:58:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb48066f60 0x7fdb48069410 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb681596d0 0x7fdb68153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb680cb3b0 0x7fdb68143be0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 --2- 172.21.15.3:0/2321865579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680cb780 0x7fdb680b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 >> 172.21.15.3:0/2321865579 conn(0x7fdb68070310 msgr2=0x7fdb680b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:15.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 shutdown_connections 2023-10-06T04:58:15.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.681+0000 7fdb6d4d9700 1 -- 172.21.15.3:0/2321865579 wait complete. 2023-10-06T04:58:15.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:58:15.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.822+0000 7ff9f8e08700 1 Processor -- start 2023-10-06T04:58:15.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.822+0000 7ff9f8e08700 1 -- start start 2023-10-06T04:58:15.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.823+0000 7ff9f8e08700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 0x7ff9f4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.823+0000 7ff9f8e08700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff9f415d540 0x7ff9f40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.823+0000 7ff9f8e08700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f415a290 con 0x7ff9f40cba50 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.823+0000 7ff9f8e08700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f40d0020 con 0x7ff9f415d540 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.823+0000 7ff9f8e08700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4159f40 con 0x7ff9f4150690 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.823+0000 7ff9f1d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff9f40cba50 0x7ff9f415a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44270/0 (socket says 172.21.15.3:44270) 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f1d9b700 1 -- 172.21.15.3:0/2848463076 learned_addr learned my addr 172.21.15.3:0/2848463076 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f259c700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 0x7ff9f4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970869084 0 0) 0x7ff9f415a290 con 0x7ff9f40cba50 2023-10-06T04:58:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f2d9d700 1 --2- 172.21.15.3:0/2848463076 >> v2:172.21.15.153:3300/0 conn(0x7ff9f415d540 0x7ff9f40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e0003570 con 0x7ff9f40cba50 2023-10-06T04:58:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1556299668 0 0) 0x7ff9e0003570 con 0x7ff9f40cba50 2023-10-06T04:58:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 msgr2=0x7ff9f4150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 0x7ff9f4150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 >> v2:172.21.15.153:3300/0 conn(0x7ff9f415d540 msgr2=0x7ff9f40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 --2- 172.21.15.3:0/2848463076 >> v2:172.21.15.153:3300/0 conn(0x7ff9f415d540 0x7ff9f40af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f415dcb0 con 0x7ff9f40cba50 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7ff9dc001470 con 0x7ff9f40cba50 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 >> v1:172.21.15.3:6789/0 conn(0x7ff9f40cba50 legacy=0x7ff9f415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 0x7ff9e0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9e0004d50 0x7ff9e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40d0020 con 0x7ff9e0003fe0 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4159f40 con 0x7ff9e0004d50 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e000ba90 con 0x7ff9f4150690 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f259c700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 0x7ff9e0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.824+0000 7ff9f2d9d700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9e0004d50 0x7ff9e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f1d9b700 1 -- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 msgr2=0x7ff9e0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f4150690 0x7ff9e0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f1d9b700 1 -- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9e0004d50 msgr2=0x7ff9e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9e0004d50 0x7ff9e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f1d9b700 1 -- 172.21.15.3:0/2848463076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0003570 con 0x7ff9e0003fe0 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9e0004810 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00c3c0 tx=0x7ff9dc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4f7d917a3d8b64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00cc30 con 0x7ff9e0003fe0 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc00cdb0 con 0x7ff9e0003fe0 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f159a700 1 -- 172.21.15.3:0/2848463076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc005b70 con 0x7ff9e0003fe0 2023-10-06T04:58:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f8e08700 1 -- 172.21.15.3:0/2848463076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 msgr2=0x7ff9e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f8e08700 1 --2- 172.21.15.3:0/2848463076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9e0004810 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00c3c0 tx=0x7ff9dc001e90 comp rx=0 tx=0).stop 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f2d9d700 1 -- 172.21.15.3:0/2848463076 reap_dead start 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f8e08700 1 -- 172.21.15.3:0/2848463076 shutdown_connections 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.825+0000 7ff9f8e08700 1 -- 172.21.15.3:0/2848463076 >> 172.21.15.3:0/2848463076 conn(0x7ff9f4070310 msgr2=0x7ff9f4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.826+0000 7ff9f8e08700 1 -- 172.21.15.3:0/2848463076 shutdown_connections 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.826+0000 7ff9f8e08700 1 -- 172.21.15.3:0/2848463076 wait complete. 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 Processor -- start 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 -- start start 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9f4150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f40b57c0 0x7ff9f40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f414fe30 0x7ff9f416d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40c8bf0 con 0x7ff9e0003fe0 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40b6280 con 0x7ff9f40b57c0 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f8e08700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40c9190 con 0x7ff9f414fe30 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9f4150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9f4150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54674/0 (socket says 172.21.15.3:54674) 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.827+0000 7ff9f259c700 1 -- 172.21.15.3:0/1649274938 learned_addr learned my addr 172.21.15.3:0/1649274938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f2d9d700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f414fe30 0x7ff9f416d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f40b57c0 0x7ff9f40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f259c700 1 -- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f414fe30 msgr2=0x7ff9f416d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f259c700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f414fe30 0x7ff9f416d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f259c700 1 -- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f40b57c0 msgr2=0x7ff9f40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f259c700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f40b57c0 0x7ff9f40b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f259c700 1 -- 172.21.15.3:0/1649274938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f40b5cc0 con 0x7ff9e0003fe0 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f40b57c0 0x7ff9f40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f259c700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9f4150ea0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec011cf0 tx=0x7ff9ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9ea7fc700 1 -- 172.21.15.3:0/1649274938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00f940 con 0x7ff9e0003fe0 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9ea7fc700 1 -- 172.21.15.3:0/1649274938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec00faa0 con 0x7ff9e0003fe0 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f40b5fa0 con 0x7ff9e0003fe0 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9ea7fc700 1 -- 172.21.15.3:0/1649274938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00fe20 con 0x7ff9e0003fe0 2023-10-06T04:58:15.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.828+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f40b3720 con 0x7ff9e0003fe0 2023-10-06T04:58:15.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.829+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f404c380 con 0x7ff9e0003fe0 2023-10-06T04:58:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.830+0000 7ff9ea7fc700 1 -- 172.21.15.3:0/1649274938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9ec014650 con 0x7ff9e0003fe0 2023-10-06T04:58:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.830+0000 7ff9ea7fc700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9d4066f30 0x7ff9d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.830+0000 7ff9ea7fc700 1 -- 172.21.15.3:0/1649274938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ff9ec018030 con 0x7ff9e0003fe0 2023-10-06T04:58:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.830+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9d4066f30 0x7ff9d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.831+0000 7ff9f1d9b700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9d4066f30 0x7ff9d40693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:15.834+0000 7ff9ea7fc700 1 -- 172.21.15.3:0/1649274938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9ec0520d0 con 0x7ff9e0003fe0 2023-10-06T04:58:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.030+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff9f415df90 con 0x7ff9e0003fe0 2023-10-06T04:58:16.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.036+0000 7ff9ea7fc700 1 -- 172.21.15.3:0/1649274938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9ec0220c0 con 0x7ff9e0003fe0 2023-10-06T04:58:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9d4066f30 msgr2=0x7ff9d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9d4066f30 0x7ff9d40693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc015000 comp rx=0 tx=0).stop 2023-10-06T04:58:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 msgr2=0x7ff9f4150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9f4150ea0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec011cf0 tx=0x7ff9ec0091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 shutdown_connections 2023-10-06T04:58:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff9d4066f30 0x7ff9d40693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9f414fe30 0x7ff9f416d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0003fe0 0x7ff9f4150ea0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 --2- 172.21.15.3:0/1649274938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f40b57c0 0x7ff9f40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 >> 172.21.15.3:0/1649274938 conn(0x7ff9f4070310 msgr2=0x7ff9f415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:16.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 shutdown_connections 2023-10-06T04:58:16.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.038+0000 7ff9f8e08700 1 -- 172.21.15.3:0/1649274938 wait complete. 2023-10-06T04:58:16.052 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:58:16.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:16.095 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:16.137 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:58:16.138 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:58:16.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.255+0000 7f396a10b700 1 Processor -- start 2023-10-06T04:58:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.255+0000 7f396a10b700 1 -- start start 2023-10-06T04:58:16.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f396a10b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3964159350 0x7f3964159720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:16.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f396a10b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39640cac60 0x7f39640cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f396a10b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39640cb7b0 con 0x7f39640af050 2023-10-06T04:58:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f396a10b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396415e080 con 0x7f39640cac60 2023-10-06T04:58:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f396a10b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396415d2c0 con 0x7f3964159350 2023-10-06T04:58:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3968908700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39640cac60 0x7f39640cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3968908700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39640cac60 0x7f39640cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54684/0 (socket says 172.21.15.3:54684) 2023-10-06T04:58:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3968908700 1 -- 172.21.15.3:0/3947216431 learned_addr learned my addr 172.21.15.3:0/3947216431 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3969109700 1 --2- 172.21.15.3:0/3947216431 >> v2:172.21.15.153:3300/0 conn(0x7f3964159350 0x7f3964159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3968908700 1 -- 172.21.15.3:0/3947216431 >> v2:172.21.15.153:3300/0 conn(0x7f3964159350 msgr2=0x7f3964159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3968908700 1 --2- 172.21.15.3:0/3947216431 >> v2:172.21.15.153:3300/0 conn(0x7f3964159350 0x7f3964159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3968908700 1 -- 172.21.15.3:0/3947216431 >> v1:172.21.15.153:6789/0 conn(0x7f39640af050 legacy=0x7f396406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3968908700 1 -- 172.21.15.3:0/3947216431 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396415d5a0 con 0x7f39640cac60 2023-10-06T04:58:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.256+0000 7f3969109700 1 --2- 172.21.15.3:0/3947216431 >> v2:172.21.15.153:3300/0 conn(0x7f3964159350 0x7f3964159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f3968908700 1 --2- 172.21.15.3:0/3947216431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f39640cb270 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3964088420 tx=0x7f3958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6fe1a70cd25efb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f3963fff700 1 -- 172.21.15.3:0/3947216431 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800b820 con 0x7f39640cac60 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f3963fff700 1 -- 172.21.15.3:0/3947216431 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395800b980 con 0x7f39640cac60 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f3963fff700 1 -- 172.21.15.3:0/3947216431 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800bd20 con 0x7f39640cac60 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 -- 172.21.15.3:0/3947216431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 msgr2=0x7f39640cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 --2- 172.21.15.3:0/3947216431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f39640cb270 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3964088420 tx=0x7f3958009250 comp rx=0 tx=0).stop 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 -- 172.21.15.3:0/3947216431 shutdown_connections 2023-10-06T04:58:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 --2- 172.21.15.3:0/3947216431 >> v2:172.21.15.153:3300/0 conn(0x7f3964159350 0x7f3964159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 --2- 172.21.15.3:0/3947216431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f39640cb270 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 -- 172.21.15.3:0/3947216431 >> 172.21.15.3:0/3947216431 conn(0x7f396406ecc0 msgr2=0x7f396406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 -- 172.21.15.3:0/3947216431 shutdown_connections 2023-10-06T04:58:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.257+0000 7f396a10b700 1 -- 172.21.15.3:0/3947216431 wait complete. 2023-10-06T04:58:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.258+0000 7f396a10b700 1 Processor -- start 2023-10-06T04:58:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.258+0000 7f396a10b700 1 -- start start 2023-10-06T04:58:16.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f396a10b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39640af050 0x7f3964153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:16.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f396a10b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f3964155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f396a10b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3964159350 0x7f3964155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f396a10b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3964154520 con 0x7f39640cac60 2023-10-06T04:58:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f396a10b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39640b4970 con 0x7f39640af050 2023-10-06T04:58:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f396a10b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39640b2980 con 0x7f3964159350 2023-10-06T04:58:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f3968908700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f3964155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.259+0000 7f3968908700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f3964155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54694/0 (socket says 172.21.15.3:54694) 2023-10-06T04:58:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3968908700 1 -- 172.21.15.3:0/1734362772 learned_addr learned my addr 172.21.15.3:0/1734362772 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3969109700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39640af050 0x7f3964153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f396990a700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3964159350 0x7f3964155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3968908700 1 -- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3964159350 msgr2=0x7f3964155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3968908700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3964159350 0x7f3964155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3968908700 1 -- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39640af050 msgr2=0x7f3964153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3968908700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39640af050 0x7f3964153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3968908700 1 -- 172.21.15.3:0/1734362772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580094f0 con 0x7f39640cac60 2023-10-06T04:58:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f3968908700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f3964155590 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f39580030c0 tx=0x7f3958016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f39617fa700 1 -- 172.21.15.3:0/1734362772 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958011db0 con 0x7f39640cac60 2023-10-06T04:58:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f39617fa700 1 -- 172.21.15.3:0/1734362772 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3958020070 con 0x7f39640cac60 2023-10-06T04:58:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39640b4040 con 0x7f39640cac60 2023-10-06T04:58:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f39617fa700 1 -- 172.21.15.3:0/1734362772 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395801b6a0 con 0x7f39640cac60 2023-10-06T04:58:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.260+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39640b4600 con 0x7f39640cac60 2023-10-06T04:58:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.261+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396404c380 con 0x7f39640cac60 2023-10-06T04:58:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.262+0000 7f39617fa700 1 -- 172.21.15.3:0/1734362772 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3958022030 con 0x7f39640cac60 2023-10-06T04:58:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.262+0000 7f39617fa700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3950066f30 0x7f39500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.262+0000 7f39617fa700 1 -- 172.21.15.3:0/1734362772 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3958086360 con 0x7f39640cac60 2023-10-06T04:58:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.262+0000 7f3969109700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3950066f30 0x7f39500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.263+0000 7f3969109700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3950066f30 0x7f39500693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f395c006fd0 tx=0x7f395c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.267+0000 7f39617fa700 1 -- 172.21.15.3:0/1734362772 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3958053770 con 0x7f39640cac60 2023-10-06T04:58:16.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.501+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f396404c380 con 0x7f39640cac60 2023-10-06T04:58:16.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.502+0000 7f39617fa700 1 -- 172.21.15.3:0/1734362772 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f3958056d90 con 0x7f39640cac60 2023-10-06T04:58:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:16.504 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1669608527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1669608527},{"type":"v1","addr":"172.21.15.153:6837","nonce":1669608527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1477163678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1477163678},{"type":"v1","addr":"172.21.15.153:6836","nonce":1477163678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6105,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/246388454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":246388454},{"type":"v1","addr":"172.21.15.3:6833","nonce":246388454}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2291800447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2291800447},{"type":"v1","addr":"172.21.15.3:6835","nonce":2291800447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[]} 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3950066f30 msgr2=0x7f39500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3950066f30 0x7f39500693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f395c006fd0 tx=0x7f395c008040 comp rx=0 tx=0).stop 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 msgr2=0x7f3964155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f3964155590 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f39580030c0 tx=0x7f3958016000 comp rx=0 tx=0).stop 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 shutdown_connections 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3950066f30 0x7f39500693e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39640af050 0x7f3964153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3964159350 0x7f3964155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 --2- 172.21.15.3:0/1734362772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39640cac60 0x7f3964155590 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 >> 172.21.15.3:0/1734362772 conn(0x7f396406ecc0 msgr2=0x7f396406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 shutdown_connections 2023-10-06T04:58:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:16.504+0000 7f396a10b700 1 -- 172.21.15.3:0/1734362772 wait complete. 2023-10-06T04:58:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 44 2023-10-06T04:58:16.518 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:58:16.518 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:58:16.518 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:58:16.518 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:16.519 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:58:16.519 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:58:16.519 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:58:16.519 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:16.519 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:58:16.519 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:58:16.520 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:58:16.520 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:16.520 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:58:16.520 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:58:16.520 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:58:16.520 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:16.527 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:58:16.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:58:16.528 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:58:16.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:58:16.535 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:58:16.535 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:58:16.570 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:58:16.570 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:16.570 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:58:16.571 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:16.577 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:58:16.577 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:16.643 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:58:16.643 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:58:16.657 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:58:20.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T04:58:22.535 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:58:22.535 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:58:22.538 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:58:22.539 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:22.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:58:22.621 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:58:22.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.671+0000 7f7672fd3700 1 Processor -- start 2023-10-06T04:58:22.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.671+0000 7f7672fd3700 1 -- start start 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f7672fd3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f766c06b310 0x7f766c1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f7672fd3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c151600 0x7f766c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f7672fd3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c15b190 con 0x7f766c06b6e0 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f7672fd3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c158c00 con 0x7f766c06b310 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f7672fd3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c15ba30 con 0x7f766c151600 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f766bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f766c06b6e0 0x7f766c0af130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50016/0 (socket says 172.21.15.3:50016) 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f766bfff700 1 -- 172.21.15.3:0/2605083273 learned_addr learned my addr 172.21.15.3:0/2605083273 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f7671570700 1 --2- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c151600 0x7f766c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f7670d6f700 1 --2- 172.21.15.3:0/2605083273 >> v2:172.21.15.153:3300/0 conn(0x7f766c06b310 0x7f766c1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f766b7fe700 1 -- 172.21.15.3:0/2605083273 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900003190 0 0) 0x7f766c15b190 con 0x7f766c06b6e0 2023-10-06T04:58:22.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.672+0000 7f766b7fe700 1 -- 172.21.15.3:0/2605083273 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f765c003570 con 0x7f766c06b6e0 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f7670d6f700 1 -- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c151600 msgr2=0x7f766c151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f7670d6f700 1 --2- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c151600 0x7f766c151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f7670d6f700 1 -- 172.21.15.3:0/2605083273 >> v1:172.21.15.3:6789/0 conn(0x7f766c06b6e0 legacy=0x7f766c0af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f766b7fe700 1 -- 172.21.15.3:0/2605083273 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1223894043 0 0) 0x7f765c003570 con 0x7f766c06b6e0 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f7670d6f700 1 -- 172.21.15.3:0/2605083273 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0af960 con 0x7f766c06b310 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f7670d6f700 1 --2- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b310 0x7f766c1586c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f76600098e0 tx=0x7f76600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=904dd4768a3a6d51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f766b7fe700 1 -- 172.21.15.3:0/2605083273 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658001470 con 0x7f766c06b310 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f766b7fe700 1 -- 172.21.15.3:0/2605083273 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660011420 con 0x7f766c06b310 2023-10-06T04:58:22.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.673+0000 7f766b7fe700 1 -- 172.21.15.3:0/2605083273 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600117a0 con 0x7f766c06b310 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 -- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b310 msgr2=0x7f766c1586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 --2- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b310 0x7f766c1586c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f76600098e0 tx=0x7f76600091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 -- 172.21.15.3:0/2605083273 shutdown_connections 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 --2- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c151600 0x7f766c151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 --2- 172.21.15.3:0/2605083273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b310 0x7f766c1586c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 -- 172.21.15.3:0/2605083273 >> 172.21.15.3:0/2605083273 conn(0x7f766c0cdc30 msgr2=0x7f766c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 -- 172.21.15.3:0/2605083273 shutdown_connections 2023-10-06T04:58:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.674+0000 7f7672fd3700 1 -- 172.21.15.3:0/2605083273 wait complete. 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.675+0000 7f7672fd3700 1 Processor -- start 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.675+0000 7f7672fd3700 1 -- start start 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7672fd3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c06b310 0x7f766c14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7672fd3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b6e0 0x7f766c0c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7672fd3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f766c151600 0x7f766c0c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7672fd3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c156400 con 0x7f766c151600 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7672fd3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c167920 con 0x7f766c06b6e0 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7672fd3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0c26f0 con 0x7f766c06b310 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f766c151600 0x7f766c0c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f766c151600 0x7f766c0c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42574/0 (socket says 172.21.15.3:42574) 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 -- 172.21.15.3:0/561743493 learned_addr learned my addr 172.21.15.3:0/561743493 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:22.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f766bfff700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b6e0 0x7f766c0c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7670d6f700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c06b310 0x7f766c14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 -- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c06b310 msgr2=0x7f766c14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c06b310 0x7f766c14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 -- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b6e0 msgr2=0x7f766c0c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b6e0 0x7f766c0c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 -- 172.21.15.3:0/561743493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660009420 con 0x7f766c151600 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f766bfff700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b6e0 0x7f766c0c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.676+0000 7f7671570700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f766c151600 0x7f766c0c4610 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7664009df0 tx=0x7f766400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.677+0000 7f7668ff9700 1 -- 172.21.15.3:0/561743493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400cb50 con 0x7f766c151600 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.677+0000 7f7668ff9700 1 -- 172.21.15.3:0/561743493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400f040 con 0x7f766c151600 2023-10-06T04:58:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.677+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c156120 con 0x7f766c151600 2023-10-06T04:58:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.677+0000 7f7668ff9700 1 -- 172.21.15.3:0/561743493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664014480 con 0x7f766c151600 2023-10-06T04:58:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.677+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c0c2380 con 0x7f766c151600 2023-10-06T04:58:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.678+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c0c1f00 con 0x7f766c151600 2023-10-06T04:58:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.678+0000 7f7668ff9700 1 -- 172.21.15.3:0/561743493 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7664014830 con 0x7f766c151600 2023-10-06T04:58:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.678+0000 7f7668ff9700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.678+0000 7f7668ff9700 1 -- 172.21.15.3:0/561743493 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7664085000 con 0x7f766c151600 2023-10-06T04:58:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.679+0000 7f7670d6f700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.679+0000 7f7670d6f700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7650066f30 0x7f76500693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f76600091e0 tx=0x7f7660016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.683+0000 7f7668ff9700 1 -- 172.21.15.3:0/561743493 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f766c0c1f00 con 0x7f766c151600 2023-10-06T04:58:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:22.895+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f766c0c1f00 con 0x7f766c151600 2023-10-06T04:58:23.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.701+0000 7f7668ff9700 1 -- 172.21.15.3:0/561743493 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f766c0c1f00 con 0x7f766c151600 2023-10-06T04:58:23.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7650066f30 msgr2=0x7f76500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:23.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7650066f30 0x7f76500693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f76600091e0 tx=0x7f7660016000 comp rx=0 tx=0).stop 2023-10-06T04:58:23.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f766c151600 msgr2=0x7f766c0c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f766c151600 0x7f766c0c4610 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7664009df0 tx=0x7f766400b040 comp rx=0 tx=0).stop 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 shutdown_connections 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f766c151600 0x7f766c0c4610 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f766c06b310 0x7f766c14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 --2- 172.21.15.3:0/561743493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f766c06b6e0 0x7f766c0c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 >> 172.21.15.3:0/561743493 conn(0x7f766c0cdc30 msgr2=0x7f766c14ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 shutdown_connections 2023-10-06T04:58:23.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.703+0000 7f7672fd3700 1 -- 172.21.15.3:0/561743493 wait complete. 2023-10-06T04:58:23.707 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T04:58:23.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:58:23.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.844+0000 7fa768872700 1 Processor -- start 2023-10-06T04:58:23.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.844+0000 7fa768872700 1 -- start start 2023-10-06T04:58:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa768872700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa7640cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa768872700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7641596d0 0x7fa7640cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa768872700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7640cf130 con 0x7fa7640cb780 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa768872700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7640ce520 con 0x7fa7641596d0 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa768872700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640ce800 con 0x7fa7640cb3b0 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7641596d0 0x7fa7640cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7641596d0 0x7fa7640cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42586/0 (socket says 172.21.15.3:42586) 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 -- 172.21.15.3:0/31062693 learned_addr learned my addr 172.21.15.3:0/31062693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa762d9d700 1 --2- 172.21.15.3:0/31062693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa7640cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 -- 172.21.15.3:0/31062693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 msgr2=0x7fa7640cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 --2- 172.21.15.3:0/31062693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa7640cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 -- 172.21.15.3:0/31062693 >> v1:172.21.15.153:6789/0 conn(0x7fa7640cb780 legacy=0x7fa764158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 -- 172.21.15.3:0/31062693 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640ceae0 con 0x7fa7641596d0 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa76359e700 1 --2- 172.21.15.3:0/31062693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640cccc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009f20 tx=0x7fa75c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e8fd1142d98626 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa761d9b700 1 -- 172.21.15.3:0/31062693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00e070 con 0x7fa7641596d0 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa761d9b700 1 -- 172.21.15.3:0/31062693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c00ba40 con 0x7fa7641596d0 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.845+0000 7fa761d9b700 1 -- 172.21.15.3:0/31062693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0092e0 con 0x7fa7641596d0 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 -- 172.21.15.3:0/31062693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 msgr2=0x7fa7640cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 --2- 172.21.15.3:0/31062693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640cccc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009f20 tx=0x7fa75c009840 comp rx=0 tx=0).stop 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 -- 172.21.15.3:0/31062693 shutdown_connections 2023-10-06T04:58:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 --2- 172.21.15.3:0/31062693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640cccc0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 --2- 172.21.15.3:0/31062693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa7640cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 -- 172.21.15.3:0/31062693 >> 172.21.15.3:0/31062693 conn(0x7fa764070310 msgr2=0x7fa764070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 -- 172.21.15.3:0/31062693 shutdown_connections 2023-10-06T04:58:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.846+0000 7fa768872700 1 -- 172.21.15.3:0/31062693 wait complete. 2023-10-06T04:58:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.847+0000 7fa768872700 1 Processor -- start 2023-10-06T04:58:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.847+0000 7fa768872700 1 -- start start 2023-10-06T04:58:23.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa768872700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa764157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:23.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa768872700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7640cb780 0x7fa7640b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:23.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa768872700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa768872700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764149c50 con 0x7fa7641596d0 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa768872700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764158390 con 0x7fa7640cb780 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa768872700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c8ab0 con 0x7fa7640cb3b0 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42598/0 (socket says 172.21.15.3:42598) 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 -- 172.21.15.3:0/732908666 learned_addr learned my addr 172.21.15.3:0/732908666 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa762d9d700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa764157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76259c700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7640cb780 0x7fa7640b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 -- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 msgr2=0x7fa764157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:23.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa764157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 -- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7640cb780 msgr2=0x7fa7640b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7640cb780 0x7fa7640b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 -- 172.21.15.3:0/732908666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c009ae0 con 0x7fa7641596d0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa762d9d700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa764157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa76359e700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640b1620 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa7640893d0 tx=0x7fa75c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.848+0000 7fa74affd700 1 -- 172.21.15.3:0/732908666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0032f0 con 0x7fa7641596d0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.849+0000 7fa74affd700 1 -- 172.21.15.3:0/732908666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c012e60 con 0x7fa7641596d0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.849+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640b3590 con 0x7fa7641596d0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.849+0000 7fa74affd700 1 -- 172.21.15.3:0/732908666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c01d670 con 0x7fa7641596d0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.849+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640c8460 con 0x7fa7641596d0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.850+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76404c380 con 0x7fa7641596d0 2023-10-06T04:58:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.850+0000 7fa74affd700 1 -- 172.21.15.3:0/732908666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa75c01daa0 con 0x7fa7641596d0 2023-10-06T04:58:23.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.850+0000 7fa74affd700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa750066f30 0x7fa7500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:23.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.851+0000 7fa74affd700 1 -- 172.21.15.3:0/732908666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa75c085c40 con 0x7fa7641596d0 2023-10-06T04:58:23.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.851+0000 7fa762d9d700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa750066f30 0x7fa7500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:23.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.851+0000 7fa762d9d700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa750066f30 0x7fa7500693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa754006fd0 tx=0x7fa754008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:23.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:23.855+0000 7fa74affd700 1 -- 172.21.15.3:0/732908666 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa75c0530a0 con 0x7fa7641596d0 2023-10-06T04:58:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.125+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa7640cedc0 con 0x7fa7641596d0 2023-10-06T04:58:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.125+0000 7fa74affd700 1 -- 172.21.15.3:0/732908666 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa75c0093a0 con 0x7fa7641596d0 2023-10-06T04:58:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:24.127 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:58:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T04:58:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T04:58:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T04:58:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T04:58:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:58:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:58:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:58:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:58:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:58:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:58:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:58:24.144 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:58:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.127+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa750066f30 msgr2=0x7fa7500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.127+0000 7fa768872700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa750066f30 0x7fa7500693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa754006fd0 tx=0x7fa754008040 comp rx=0 tx=0).stop 2023-10-06T04:58:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.127+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 msgr2=0x7fa7640b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.127+0000 7fa768872700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640b1620 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa7640893d0 tx=0x7fa75c017000 comp rx=0 tx=0).stop 2023-10-06T04:58:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.127+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 shutdown_connections 2023-10-06T04:58:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.127+0000 7fa768872700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa750066f30 0x7fa7500693e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.128+0000 7fa768872700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7641596d0 0x7fa7640b1620 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.128+0000 7fa768872700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7640cb3b0 0x7fa764157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.128+0000 7fa768872700 1 --2- 172.21.15.3:0/732908666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7640cb780 0x7fa7640b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.128+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 >> 172.21.15.3:0/732908666 conn(0x7fa764070310 msgr2=0x7fa76415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.128+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 shutdown_connections 2023-10-06T04:58:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.128+0000 7fa768872700 1 -- 172.21.15.3:0/732908666 wait complete. 2023-10-06T04:58:24.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:24.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.266+0000 7f06eeb4b700 1 Processor -- start 2023-10-06T04:58:24.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.266+0000 7f06eeb4b700 1 -- start start 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.267+0000 7f06eeb4b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f06e8159500 0x7f06e80cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.267+0000 7f06eeb4b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e80b2930 0x7f06e80cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.267+0000 7f06eeb4b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e80cc5c0 con 0x7f06e81598d0 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.267+0000 7f06eeb4b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e80cb7b0 con 0x7f06e8159500 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.267+0000 7f06eeb4b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80cda60 con 0x7f06e80b2930 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ec8e7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f06e8159500 0x7f06e80cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e80b2930 0x7f06e80cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e80b2930 0x7f06e80cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:55872/0 (socket says 172.21.15.3:55872) 2023-10-06T04:58:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ec8e7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f06e8159500 0x7f06e80cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:39174/0 (socket says 172.21.15.3:39174) 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 -- 172.21.15.3:0/1165736830 learned_addr learned my addr 172.21.15.3:0/1165736830 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 -- 172.21.15.3:0/1165736830 >> v2:172.21.15.153:3300/0 conn(0x7f06e8159500 msgr2=0x7f06e80cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 --2- 172.21.15.3:0/1165736830 >> v2:172.21.15.153:3300/0 conn(0x7f06e8159500 0x7f06e80cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 -- 172.21.15.3:0/1165736830 >> v1:172.21.15.153:6789/0 conn(0x7f06e81598d0 legacy=0x7f06e80b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 -- 172.21.15.3:0/1165736830 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80cb9b0 con 0x7f06e80b2930 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06e77fe700 1 -- 172.21.15.3:0/1165736830 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527487298 0 0) 0x7f06e80cc5c0 con 0x7f06e81598d0 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ec8e7700 1 --2- 172.21.15.3:0/1165736830 >> v2:172.21.15.153:3300/0 conn(0x7f06e8159500 0x7f06e80cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06ed0e8700 1 --2- 172.21.15.3:0/1165736830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e80b2930 0x7f06e80cb270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06e00099b0 tx=0x7f06e0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d37f90ec6a0c534c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06e77fe700 1 -- 172.21.15.3:0/1165736830 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0009ea0 con 0x7f06e80b2930 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06e77fe700 1 -- 172.21.15.3:0/1165736830 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e000b860 con 0x7f06e80b2930 2023-10-06T04:58:24.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.268+0000 7f06e77fe700 1 -- 172.21.15.3:0/1165736830 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000bc00 con 0x7f06e80b2930 2023-10-06T04:58:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 -- 172.21.15.3:0/1165736830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e80b2930 msgr2=0x7f06e80cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/1165736830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e80b2930 0x7f06e80cb270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06e00099b0 tx=0x7f06e0009250 comp rx=0 tx=0).stop 2023-10-06T04:58:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 -- 172.21.15.3:0/1165736830 shutdown_connections 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/1165736830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e80b2930 0x7f06e80cb270 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/1165736830 >> v2:172.21.15.153:3300/0 conn(0x7f06e8159500 0x7f06e80cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 -- 172.21.15.3:0/1165736830 >> 172.21.15.3:0/1165736830 conn(0x7f06e8070310 msgr2=0x7f06e8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 -- 172.21.15.3:0/1165736830 shutdown_connections 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.269+0000 7f06eeb4b700 1 -- 172.21.15.3:0/1165736830 wait complete. 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.270+0000 7f06eeb4b700 1 Processor -- start 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.270+0000 7f06eeb4b700 1 -- start start 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06eeb4b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e80b2930 0x7f06e80b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06eeb4b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e8159500 0x7f06e81507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06eeb4b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06e81598d0 0x7f06e80b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06eeb4b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80b40a0 con 0x7f06e80b2930 2023-10-06T04:58:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06eeb4b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e815ac70 con 0x7f06e81598d0 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06eeb4b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e815b1f0 con 0x7f06e8159500 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e80b2930 0x7f06e80b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e80b2930 0x7f06e80b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42604/0 (socket says 172.21.15.3:42604) 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 -- 172.21.15.3:0/584364933 learned_addr learned my addr 172.21.15.3:0/584364933 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ed0e8700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06e81598d0 0x7f06e80b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06e7fff700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e8159500 0x7f06e81507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 -- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e8159500 msgr2=0x7f06e81507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e8159500 0x7f06e81507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 -- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06e81598d0 msgr2=0x7f06e80b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06e81598d0 0x7f06e80b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.271+0000 7f06ec8e7700 1 -- 172.21.15.3:0/584364933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0009420 con 0x7f06e80b2930 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.272+0000 7f06ec8e7700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e80b2930 0x7f06e80b31e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f06d000ec80 tx=0x7f06d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.272+0000 7f06e4ff9700 1 -- 172.21.15.3:0/584364933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d000c8f0 con 0x7f06e80b2930 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.272+0000 7f06e4ff9700 1 -- 172.21.15.3:0/584364933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d000ca50 con 0x7f06e80b2930 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.272+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e00094f0 con 0x7f06e80b2930 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.272+0000 7f06e4ff9700 1 -- 172.21.15.3:0/584364933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d000cdd0 con 0x7f06e80b2930 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.272+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e80b3e40 con 0x7f06e80b2930 2023-10-06T04:58:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.273+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e815afa0 con 0x7f06e80b2930 2023-10-06T04:58:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.273+0000 7f06e4ff9700 1 -- 172.21.15.3:0/584364933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06d0013680 con 0x7f06e80b2930 2023-10-06T04:58:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.274+0000 7f06e4ff9700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06c8066f30 0x7f06c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.274+0000 7f06e4ff9700 1 -- 172.21.15.3:0/584364933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f06d0084720 con 0x7f06e80b2930 2023-10-06T04:58:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.274+0000 7f06e7fff700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06c8066f30 0x7f06c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.274+0000 7f06e7fff700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06c8066f30 0x7f06c80693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f06e80712b0 tx=0x7f06d80016f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.278+0000 7f06e4ff9700 1 -- 172.21.15.3:0/584364933 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06d0051ba0 con 0x7f06e80b2930 2023-10-06T04:58:24.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.472+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06e8071680 con 0x7f06e80b2930 2023-10-06T04:58:24.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.473+0000 7f06e4ff9700 1 -- 172.21.15.3:0/584364933 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+13603 (secure 0 0 0) 0x7f06d00551c0 con 0x7f06e80b2930 2023-10-06T04:58:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:24.475 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":80,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:58:23.692050+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.474+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06c8066f30 msgr2=0x7f06c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.474+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06c8066f30 0x7f06c80693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f06e80712b0 tx=0x7f06d80016f0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.474+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e80b2930 msgr2=0x7f06e80b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.474+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e80b2930 0x7f06e80b31e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f06d000ec80 tx=0x7f06d00091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 shutdown_connections 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f06c8066f30 0x7f06c80693e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06e81598d0 0x7f06e80b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e80b2930 0x7f06e80b31e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 --2- 172.21.15.3:0/584364933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06e8159500 0x7f06e81507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 >> 172.21.15.3:0/584364933 conn(0x7f06e8070310 msgr2=0x7f06e80cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 shutdown_connections 2023-10-06T04:58:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.475+0000 7f06eeb4b700 1 -- 172.21.15.3:0/584364933 wait complete. 2023-10-06T04:58:24.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:58:24.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:58:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.610+0000 7fd356ddb700 1 Processor -- start 2023-10-06T04:58:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.610+0000 7fd356ddb700 1 -- start start 2023-10-06T04:58:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd356ddb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350150690 0x7fd350150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd356ddb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3500cba50 0x7fd35015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd356ddb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35015da80 con 0x7fd35015a720 2023-10-06T04:58:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd356ddb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35015a290 con 0x7fd3500cba50 2023-10-06T04:58:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd356ddb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500d0020 con 0x7fd350150690 2023-10-06T04:58:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3500cba50 0x7fd35015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3500cba50 0x7fd35015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42606/0 (socket says 172.21.15.3:42606) 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 -- 172.21.15.3:0/954190467 learned_addr learned my addr 172.21.15.3:0/954190467 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 -- 172.21.15.3:0/954190467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350150690 msgr2=0x7fd350150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd354b77700 1 --2- 172.21.15.3:0/954190467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350150690 0x7fd350150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 --2- 172.21.15.3:0/954190467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350150690 0x7fd350150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 -- 172.21.15.3:0/954190467 >> v1:172.21.15.153:6789/0 conn(0x7fd35015a720 legacy=0x7fd3500af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 -- 172.21.15.3:0/954190467 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd350159f40 con 0x7fd3500cba50 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34ffff700 1 --2- 172.21.15.3:0/954190467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cba50 0x7fd35015d540 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd3440099b0 tx=0x7fd344009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02e9112c1b56a67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34f7fe700 1 -- 172.21.15.3:0/954190467 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400b820 con 0x7fd3500cba50 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34f7fe700 1 -- 172.21.15.3:0/954190467 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34400b980 con 0x7fd3500cba50 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.611+0000 7fd34f7fe700 1 -- 172.21.15.3:0/954190467 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400bd20 con 0x7fd3500cba50 2023-10-06T04:58:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 -- 172.21.15.3:0/954190467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cba50 msgr2=0x7fd35015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 --2- 172.21.15.3:0/954190467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cba50 0x7fd35015d540 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd3440099b0 tx=0x7fd344009250 comp rx=0 tx=0).stop 2023-10-06T04:58:24.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 -- 172.21.15.3:0/954190467 shutdown_connections 2023-10-06T04:58:24.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 --2- 172.21.15.3:0/954190467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd350150690 0x7fd350150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 --2- 172.21.15.3:0/954190467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3500cba50 0x7fd35015d540 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 -- 172.21.15.3:0/954190467 >> 172.21.15.3:0/954190467 conn(0x7fd350070310 msgr2=0x7fd350070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:24.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 -- 172.21.15.3:0/954190467 shutdown_connections 2023-10-06T04:58:24.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.612+0000 7fd356ddb700 1 -- 172.21.15.3:0/954190467 wait complete. 2023-10-06T04:58:24.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.613+0000 7fd356ddb700 1 Processor -- start 2023-10-06T04:58:24.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.613+0000 7fd356ddb700 1 -- start start 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd356ddb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cba50 0x7fd3501474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd356ddb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd350150690 0x7fd3500c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd356ddb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd35015a720 0x7fd3500c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd356ddb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500b2b80 con 0x7fd350150690 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd356ddb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500b1f70 con 0x7fd3500cba50 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd356ddb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500b2250 con 0x7fd35015a720 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd350150690 0x7fd3500c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd350150690 0x7fd3500c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42610/0 (socket says 172.21.15.3:42610) 2023-10-06T04:58:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 -- 172.21.15.3:0/1496604979 learned_addr learned my addr 172.21.15.3:0/1496604979 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd354b77700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cba50 0x7fd3501474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd355378700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd35015a720 0x7fd3500c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 -- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd35015a720 msgr2=0x7fd3500c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd35015a720 0x7fd3500c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 -- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cba50 msgr2=0x7fd3501474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cba50 0x7fd3501474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.614+0000 7fd34ffff700 1 -- 172.21.15.3:0/1496604979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440094f0 con 0x7fd350150690 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.615+0000 7fd34ffff700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd350150690 0x7fd3500c56f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd344009290 tx=0x7fd344016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.615+0000 7fd34cff9700 1 -- 172.21.15.3:0/1496604979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344011d70 con 0x7fd350150690 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.615+0000 7fd34cff9700 1 -- 172.21.15.3:0/1496604979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd344020070 con 0x7fd350150690 2023-10-06T04:58:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.615+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3500b2530 con 0x7fd350150690 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.615+0000 7fd34cff9700 1 -- 172.21.15.3:0/1496604979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34401b6a0 con 0x7fd350150690 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.615+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3500b1220 con 0x7fd350150690 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.616+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3500b29d0 con 0x7fd350150690 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.616+0000 7fd34cff9700 1 -- 172.21.15.3:0/1496604979 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd344022030 con 0x7fd350150690 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.616+0000 7fd34cff9700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd33c066f30 0x7fd33c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.616+0000 7fd34cff9700 1 -- 172.21.15.3:0/1496604979 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd344086330 con 0x7fd350150690 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.617+0000 7fd354b77700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd33c066f30 0x7fd33c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.617+0000 7fd354b77700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd33c066f30 0x7fd33c0693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd34000ab30 tx=0x7fd340008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.621+0000 7fd34cff9700 1 -- 172.21.15.3:0/1496604979 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd344053790 con 0x7fd350150690 2023-10-06T04:58:24.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.817+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3500b29d0 con 0x7fd350150690 2023-10-06T04:58:24.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.818+0000 7fd34cff9700 1 -- 172.21.15.3:0/1496604979 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd344084e60 con 0x7fd350150690 2023-10-06T04:58:24.820 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:58:24.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd33c066f30 msgr2=0x7fd33c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd33c066f30 0x7fd33c0693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd34000ab30 tx=0x7fd340008040 comp rx=0 tx=0).stop 2023-10-06T04:58:24.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd350150690 msgr2=0x7fd3500c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd350150690 0x7fd3500c56f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd344009290 tx=0x7fd344016000 comp rx=0 tx=0).stop 2023-10-06T04:58:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 shutdown_connections 2023-10-06T04:58:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd33c066f30 0x7fd33c0693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3500cba50 0x7fd3501474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd35015a720 0x7fd3500c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 --2- 172.21.15.3:0/1496604979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd350150690 0x7fd3500c56f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 >> 172.21.15.3:0/1496604979 conn(0x7fd350070310 msgr2=0x7fd350157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 shutdown_connections 2023-10-06T04:58:24.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.820+0000 7fd356ddb700 1 -- 172.21.15.3:0/1496604979 wait complete. 2023-10-06T04:58:24.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:58:24.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.954+0000 7f5c7047b700 1 Processor -- start 2023-10-06T04:58:24.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.955+0000 7f5c7047b700 1 -- start start 2023-10-06T04:58:24.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.955+0000 7f5c7047b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c68150690 0x7f5c68150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.955+0000 7f5c7047b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5c6815d540 0x7f5c680af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.955+0000 7f5c7047b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6815a290 con 0x7f5c680cba50 2023-10-06T04:58:24.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.955+0000 7f5c7047b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c680d0020 con 0x7f5c6815d540 2023-10-06T04:58:24.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.955+0000 7f5c7047b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68159f40 con 0x7f5c68150690 2023-10-06T04:58:24.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6ea18700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5c6815d540 0x7f5c680af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6e217700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c68150690 0x7f5c68150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6ea18700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5c6815d540 0x7f5c680af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:39218/0 (socket says 172.21.15.3:39218) 2023-10-06T04:58:24.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6ea18700 1 -- 172.21.15.3:0/770134043 learned_addr learned my addr 172.21.15.3:0/770134043 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:24.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6e217700 1 -- 172.21.15.3:0/770134043 >> v2:172.21.15.153:3300/0 conn(0x7f5c6815d540 msgr2=0x7f5c680af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6e217700 1 --2- 172.21.15.3:0/770134043 >> v2:172.21.15.153:3300/0 conn(0x7f5c6815d540 0x7f5c680af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6e217700 1 -- 172.21.15.3:0/770134043 >> v1:172.21.15.153:6789/0 conn(0x7f5c680cba50 legacy=0x7f5c6815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.956+0000 7f5c6e217700 1 -- 172.21.15.3:0/770134043 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6815dcb0 con 0x7f5c68150690 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c6e217700 1 --2- 172.21.15.3:0/770134043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c68150690 0x7f5c68150a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009e60 tx=0x7f5c58009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a078eaafc30e5706 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c6d215700 1 -- 172.21.15.3:0/770134043 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800e070 con 0x7f5c68150690 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c6d215700 1 -- 172.21.15.3:0/770134043 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5800bb60 con 0x7f5c68150690 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c6d215700 1 -- 172.21.15.3:0/770134043 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58012430 con 0x7f5c68150690 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 -- 172.21.15.3:0/770134043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c68150690 msgr2=0x7f5c68150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 --2- 172.21.15.3:0/770134043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c68150690 0x7f5c68150a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009e60 tx=0x7f5c58009780 comp rx=0 tx=0).stop 2023-10-06T04:58:24.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 -- 172.21.15.3:0/770134043 shutdown_connections 2023-10-06T04:58:24.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 --2- 172.21.15.3:0/770134043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c68150690 0x7f5c68150a60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 --2- 172.21.15.3:0/770134043 >> v2:172.21.15.153:3300/0 conn(0x7f5c6815d540 0x7f5c680af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 -- 172.21.15.3:0/770134043 >> 172.21.15.3:0/770134043 conn(0x7f5c68070310 msgr2=0x7f5c68070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:24.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 -- 172.21.15.3:0/770134043 shutdown_connections 2023-10-06T04:58:24.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.957+0000 7f5c7047b700 1 -- 172.21.15.3:0/770134043 wait complete. 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.958+0000 7f5c7047b700 1 Processor -- start 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c7047b700 1 -- start start 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c7047b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c680cba50 0x7f5c6814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c7047b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68150690 0x7f5c680af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c7047b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c6815d540 0x7f5c680af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c7047b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680b5380 con 0x7f5c68150690 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c7047b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680b4ef0 con 0x7f5c6815d540 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c7047b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681504a0 con 0x7f5c680cba50 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c6da16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68150690 0x7f5c680af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c6da16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68150690 0x7f5c680af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42616/0 (socket says 172.21.15.3:42616) 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.959+0000 7f5c6da16700 1 -- 172.21.15.3:0/1871552164 learned_addr learned my addr 172.21.15.3:0/1871552164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6e217700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c680cba50 0x7f5c6814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6ea18700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c6815d540 0x7f5c680af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6da16700 1 -- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c680cba50 msgr2=0x7f5c6814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6da16700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c680cba50 0x7f5c6814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6da16700 1 -- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c6815d540 msgr2=0x7f5c680af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6da16700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c6815d540 0x7f5c680af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6da16700 1 -- 172.21.15.3:0/1871552164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c58009a20 con 0x7f5c68150690 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6e217700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c680cba50 0x7f5c6814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6ea18700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c6815d540 0x7f5c680af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c6da16700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68150690 0x7f5c680af140 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c510 tx=0x7f5c5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c667fc700 1 -- 172.21.15.3:0/1871552164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c011590 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c667fc700 1 -- 172.21.15.3:0/1871552164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c011710 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c680b42e0 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c667fc700 1 -- 172.21.15.3:0/1871552164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c011ad0 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.960+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c680b48a0 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.961+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6804c380 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.961+0000 7f5c667fc700 1 -- 172.21.15.3:0/1871552164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c5c01b3b0 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.962+0000 7f5c667fc700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c54066f30 0x7f5c540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.962+0000 7f5c667fc700 1 -- 172.21.15.3:0/1871552164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5c5c083900 con 0x7f5c68150690 2023-10-06T04:58:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.962+0000 7f5c6e217700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c54066f30 0x7f5c540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:24.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.963+0000 7f5c6e217700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c54066f30 0x7f5c540693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c580097c0 tx=0x7f5c58019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:24.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:24.966+0000 7f5c667fc700 1 -- 172.21.15.3:0/1871552164 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c5c050d60 con 0x7f5c68150690 2023-10-06T04:58:25.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.157+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c6815df90 con 0x7f5c68150690 2023-10-06T04:58:25.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.157+0000 7f5c667fc700 1 -- 172.21.15.3:0/1871552164 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c5c017070 con 0x7f5c68150690 2023-10-06T04:58:25.159 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:58:25.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c54066f30 msgr2=0x7f5c540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c54066f30 0x7f5c540693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c580097c0 tx=0x7f5c58019000 comp rx=0 tx=0).stop 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68150690 msgr2=0x7f5c680af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68150690 0x7f5c680af140 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c510 tx=0x7f5c5c00c8d0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 shutdown_connections 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5c54066f30 0x7f5c540693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c6815d540 0x7f5c680af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c680cba50 0x7f5c6814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 --2- 172.21.15.3:0/1871552164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68150690 0x7f5c680af140 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 >> 172.21.15.3:0/1871552164 conn(0x7f5c68070310 msgr2=0x7f5c6815c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 shutdown_connections 2023-10-06T04:58:25.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.160+0000 7f5c7047b700 1 -- 172.21.15.3:0/1871552164 wait complete. 2023-10-06T04:58:25.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:58:25.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:25.287 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:25.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.291+0000 7f06738f2700 1 Processor -- start 2023-10-06T04:58:25.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.291+0000 7f06738f2700 1 -- start start 2023-10-06T04:58:25.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.292+0000 7f06738f2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.292+0000 7f06738f2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f066c157400 0x7f066c06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.292+0000 7f06738f2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c15aeb0 con 0x7f066c15a220 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.292+0000 7f06738f2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c15a7a0 con 0x7f066c157400 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.292+0000 7f06738f2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c15aa80 con 0x7f066c157030 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.292+0000 7f0671e8f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f066c157400 0x7f066c06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0671e8f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f066c157400 0x7f066c06e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42618/0 (socket says 172.21.15.3:42618) 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0671e8f700 1 -- 172.21.15.3:0/729021982 learned_addr learned my addr 172.21.15.3:0/729021982 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f067168e700 1 --2- 172.21.15.3:0/729021982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0671e8f700 1 -- 172.21.15.3:0/729021982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 msgr2=0x7f066c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0671e8f700 1 --2- 172.21.15.3:0/729021982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0671e8f700 1 -- 172.21.15.3:0/729021982 >> v1:172.21.15.153:6789/0 conn(0x7f066c15a220 legacy=0x7f066c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0671e8f700 1 -- 172.21.15.3:0/729021982 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c06dba0 con 0x7f066c157400 2023-10-06T04:58:25.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f067168e700 1 --2- 172.21.15.3:0/729021982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0671e8f700 1 --2- 172.21.15.3:0/729021982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c06e3a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0664009f20 tx=0x7f0664009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4a569b75cff361 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0663fff700 1 -- 172.21.15.3:0/729021982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066400e070 con 0x7f066c157400 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0663fff700 1 -- 172.21.15.3:0/729021982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066400ba80 con 0x7f066c157400 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.293+0000 7f0663fff700 1 -- 172.21.15.3:0/729021982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640092e0 con 0x7f066c157400 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 -- 172.21.15.3:0/729021982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 msgr2=0x7f066c06e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 --2- 172.21.15.3:0/729021982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c06e3a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0664009f20 tx=0x7f0664009840 comp rx=0 tx=0).stop 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 -- 172.21.15.3:0/729021982 shutdown_connections 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 --2- 172.21.15.3:0/729021982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c06e3a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 --2- 172.21.15.3:0/729021982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 -- 172.21.15.3:0/729021982 >> 172.21.15.3:0/729021982 conn(0x7f066c0ce270 msgr2=0x7f066c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:25.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 -- 172.21.15.3:0/729021982 shutdown_connections 2023-10-06T04:58:25.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.294+0000 7f06738f2700 1 -- 172.21.15.3:0/729021982 wait complete. 2023-10-06T04:58:25.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.295+0000 7f06738f2700 1 Processor -- start 2023-10-06T04:58:25.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.295+0000 7f06738f2700 1 -- start start 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f06738f2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f06738f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f06738f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f066c15a220 0x7f066c15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f06738f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c153550 con 0x7f066c157400 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f06738f2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0c97b0 con 0x7f066c15a220 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f06738f2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0c8ba0 con 0x7f066c157030 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42628/0 (socket says 172.21.15.3:42628) 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 -- 172.21.15.3:0/4089115609 learned_addr learned my addr 172.21.15.3:0/4089115609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:25.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f067168e700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0671e8f700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f066c15a220 0x7f066c15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 -- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 msgr2=0x7f066c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c0c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 -- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f066c15a220 msgr2=0x7f066c15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f066c15a220 0x7f066c15bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.296+0000 7f0670e8d700 1 -- 172.21.15.3:0/4089115609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664009ae0 con 0x7f066c157400 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.297+0000 7f0670e8d700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c153010 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c4e0 tx=0x7f065c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.297+0000 7f06617fa700 1 -- 172.21.15.3:0/4089115609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c011590 con 0x7f066c157400 2023-10-06T04:58:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.297+0000 7f06617fa700 1 -- 172.21.15.3:0/4089115609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c011710 con 0x7f066c157400 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.297+0000 7f06617fa700 1 -- 172.21.15.3:0/4089115609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c011ad0 con 0x7f066c157400 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.297+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c0c8e80 con 0x7f066c157400 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.297+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c0c9440 con 0x7f066c157400 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.298+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c0644c0 con 0x7f066c157400 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.298+0000 7f06617fa700 1 -- 172.21.15.3:0/4089115609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f065c01b3b0 con 0x7f066c157400 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.299+0000 7f06617fa700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0650066f30 0x7f06500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.299+0000 7f06617fa700 1 -- 172.21.15.3:0/4089115609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f065c083700 con 0x7f066c157400 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.299+0000 7f067168e700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0650066f30 0x7f06500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.299+0000 7f067168e700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0650066f30 0x7f06500693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f066c0ae870 tx=0x7f0668008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:25.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.303+0000 7f06617fa700 1 -- 172.21.15.3:0/4089115609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f065c050b60 con 0x7f066c157400 2023-10-06T04:58:25.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.498+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f066c071680 con 0x7f066c157400 2023-10-06T04:58:25.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.499+0000 7f06617fa700 1 -- 172.21.15.3:0/4089115609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+126 (secure 0 0 0) 0x7f065c017020 con 0x7f066c157400 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:58:25.501 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:25.502 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0650066f30 msgr2=0x7f06500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0650066f30 0x7f06500693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f066c0ae870 tx=0x7f0668008040 comp rx=0 tx=0).stop 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 msgr2=0x7f066c153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c153010 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c4e0 tx=0x7f065c00c7f0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 shutdown_connections 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0650066f30 0x7f06500693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f066c15a220 0x7f066c15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f066c157030 0x7f066c0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 --2- 172.21.15.3:0/4089115609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c157400 0x7f066c153010 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 >> 172.21.15.3:0/4089115609 conn(0x7f066c0ce270 msgr2=0x7f066c0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 shutdown_connections 2023-10-06T04:58:25.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.501+0000 7f06738f2700 1 -- 172.21.15.3:0/4089115609 wait complete. 2023-10-06T04:58:25.514 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:58:25.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:58:25.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.636+0000 7fd38a034700 1 Processor -- start 2023-10-06T04:58:25.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.636+0000 7fd38a034700 1 -- start start 2023-10-06T04:58:25.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.637+0000 7fd38a034700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 0x7fd3840b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.637+0000 7fd38a034700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38406e8e0 con 0x7fd3840af980 2023-10-06T04:58:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.637+0000 7fd38a034700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38406dcd0 con 0x7fd3840a8b40 2023-10-06T04:58:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.637+0000 7fd38a034700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38406dfb0 con 0x7fd3840afd50 2023-10-06T04:58:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.637+0000 7fd3837fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3840af980 0x7fd38406ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50018/0 (socket says 172.21.15.3:50018) 2023-10-06T04:58:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.637+0000 7fd3837fe700 1 -- 172.21.15.3:0/1570993180 learned_addr learned my addr 172.21.15.3:0/1570993180 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd383fff700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 0x7fd3840b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357811969 0 0) 0x7fd38406e8e0 con 0x7fd3840af980 2023-10-06T04:58:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd36c003570 con 0x7fd3840af980 2023-10-06T04:58:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426676582 0 0) 0x7fd38406dcd0 con 0x7fd3840a8b40 2023-10-06T04:58:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd38406e8e0 con 0x7fd3840a8b40 2023-10-06T04:58:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3264854697 0 0) 0x7fd36c003570 con 0x7fd3840af980 2023-10-06T04:58:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 msgr2=0x7fd3840b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 0x7fd3840b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 >> v1:172.21.15.153:6789/0 conn(0x7fd3840a8b40 legacy=0x7fd38406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38406e290 con 0x7fd3840af980 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fd3740015f0 con 0x7fd3840af980 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 >> v1:172.21.15.3:6789/0 conn(0x7fd3840af980 legacy=0x7fd38406ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c003c20 0x7fd36c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 0x7fd36c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c004d50 0x7fd36c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38406dfb0 con 0x7fd36c003c20 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c00bb70 con 0x7fd36c004d50 2023-10-06T04:58:25.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c00bd30 con 0x7fd3840afd50 2023-10-06T04:58:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3837fe700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c003c20 0x7fd36c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd382ffd700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 0x7fd36c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd383fff700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c004d50 0x7fd36c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3837fe700 1 -- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 msgr2=0x7fd36c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3837fe700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 0x7fd36c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3837fe700 1 -- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c004d50 msgr2=0x7fd36c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3837fe700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c004d50 0x7fd36c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd3837fe700 1 -- 172.21.15.3:0/1570993180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c003570 con 0x7fd36c003c20 2023-10-06T04:58:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.638+0000 7fd382ffd700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3840afd50 0x7fd36c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd3837fe700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c003c20 0x7fd36c004010 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd37400c2a0 tx=0x7fd3740078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2bc18fe8b6058b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400cb70 con 0x7fd36c003c20 2023-10-06T04:58:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400ccf0 con 0x7fd36c003c20 2023-10-06T04:58:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd3827fc700 1 -- 172.21.15.3:0/1570993180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374007320 con 0x7fd36c003c20 2023-10-06T04:58:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd38a034700 1 -- 172.21.15.3:0/1570993180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c003c20 msgr2=0x7fd36c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd38a034700 1 --2- 172.21.15.3:0/1570993180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c003c20 0x7fd36c004010 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd37400c2a0 tx=0x7fd3740078a0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd383fff700 1 -- 172.21.15.3:0/1570993180 reap_dead start 2023-10-06T04:58:25.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd38a034700 1 -- 172.21.15.3:0/1570993180 shutdown_connections 2023-10-06T04:58:25.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd38a034700 1 -- 172.21.15.3:0/1570993180 >> 172.21.15.3:0/1570993180 conn(0x7fd38406c370 msgr2=0x7fd38406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:25.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd38a034700 1 -- 172.21.15.3:0/1570993180 shutdown_connections 2023-10-06T04:58:25.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.639+0000 7fd38a034700 1 -- 172.21.15.3:0/1570993180 wait complete. 2023-10-06T04:58:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 Processor -- start 2023-10-06T04:58:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 -- start start 2023-10-06T04:58:25.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c003c20 0x7fd38409afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd38409b510 0x7fd3841506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd384169a80 0x7fd384150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd384151150 con 0x7fd38409b510 2023-10-06T04:58:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd384099470 con 0x7fd36c003c20 2023-10-06T04:58:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.641+0000 7fd38a034700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd384099750 con 0x7fd384169a80 2023-10-06T04:58:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd38409b510 0x7fd3841506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd38409b510 0x7fd3841506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42648/0 (socket says 172.21.15.3:42648) 2023-10-06T04:58:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 -- 172.21.15.3:0/1301418044 learned_addr learned my addr 172.21.15.3:0/1301418044 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd3837fe700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c003c20 0x7fd38409afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd383fff700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd384169a80 0x7fd384150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 -- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd384169a80 msgr2=0x7fd384150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd384169a80 0x7fd384150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 -- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c003c20 msgr2=0x7fd38409afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:25.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c003c20 0x7fd38409afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:25.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 -- 172.21.15.3:0/1301418044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3841558f0 con 0x7fd38409b510 2023-10-06T04:58:25.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd383fff700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd384169a80 0x7fd384150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:25.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd382ffd700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd38409b510 0x7fd3841506d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e840 tx=0x7fd378009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:25.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd36b7fe700 1 -- 172.21.15.3:0/1301418044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378006820 con 0x7fd38409b510 2023-10-06T04:58:25.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd36b7fe700 1 -- 172.21.15.3:0/1301418044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd378006980 con 0x7fd38409b510 2023-10-06T04:58:25.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd36b7fe700 1 -- 172.21.15.3:0/1301418044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378006d40 con 0x7fd38409b510 2023-10-06T04:58:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd384155bd0 con 0x7fd38409b510 2023-10-06T04:58:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.642+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3840990d0 con 0x7fd38409b510 2023-10-06T04:58:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.643+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd384071680 con 0x7fd38409b510 2023-10-06T04:58:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.644+0000 7fd36b7fe700 1 -- 172.21.15.3:0/1301418044 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd378007230 con 0x7fd38409b510 2023-10-06T04:58:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.644+0000 7fd36b7fe700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd370066f30 0x7fd3700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.644+0000 7fd36b7fe700 1 -- 172.21.15.3:0/1301418044 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd378084540 con 0x7fd38409b510 2023-10-06T04:58:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.644+0000 7fd3837fe700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd370066f30 0x7fd3700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.645+0000 7fd3837fe700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd370066f30 0x7fd3700693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd374000ef0 tx=0x7fd374015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.648+0000 7fd36b7fe700 1 -- 172.21.15.3:0/1301418044 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3780519a0 con 0x7fd38409b510 2023-10-06T04:58:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:25.844+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd38406e570 con 0x7fd38409b510 2023-10-06T04:58:26.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.710+0000 7fd36b7fe700 1 -- 172.21.15.3:0/1301418044 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd378054fc0 con 0x7fd38409b510 2023-10-06T04:58:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd370066f30 msgr2=0x7fd3700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd370066f30 0x7fd3700693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd374000ef0 tx=0x7fd374015000 comp rx=0 tx=0).stop 2023-10-06T04:58:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd38409b510 msgr2=0x7fd3841506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd38409b510 0x7fd3841506d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e840 tx=0x7fd378009040 comp rx=0 tx=0).stop 2023-10-06T04:58:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 shutdown_connections 2023-10-06T04:58:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd370066f30 0x7fd3700693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c003c20 0x7fd38409afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd384169a80 0x7fd384150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 --2- 172.21.15.3:0/1301418044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd38409b510 0x7fd3841506d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 >> 172.21.15.3:0/1301418044 conn(0x7fd38406c370 msgr2=0x7fd384067700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 shutdown_connections 2023-10-06T04:58:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.716+0000 7fd38a034700 1 -- 172.21.15.3:0/1301418044 wait complete. 2023-10-06T04:58:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:58:26.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:58:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.870+0000 7f52f45b5700 1 Processor -- start 2023-10-06T04:58:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.870+0000 7f52f45b5700 1 -- start start 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.871+0000 7f52f45b5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52ec150690 0x7f52ec150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.871+0000 7f52f45b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec15d540 0x7f52ec0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.871+0000 7f52f45b5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ec15a290 con 0x7f52ec0cba50 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.871+0000 7f52f45b5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ec0d0020 con 0x7f52ec150690 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.871+0000 7f52f45b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec159f40 con 0x7f52ec15d540 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f1b50700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f52ec0cba50 0x7f52ec15a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50022/0 (socket says 172.21.15.3:50022) 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f1b50700 1 -- 172.21.15.3:0/659777603 learned_addr learned my addr 172.21.15.3:0/659777603 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f2351700 1 --2- 172.21.15.3:0/659777603 >> v2:172.21.15.153:3300/0 conn(0x7f52ec150690 0x7f52ec150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f2b52700 1 --2- 172.21.15.3:0/659777603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec15d540 0x7f52ec0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f134f700 1 -- 172.21.15.3:0/659777603 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256956232 0 0) 0x7f52ec15a290 con 0x7f52ec0cba50 2023-10-06T04:58:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f134f700 1 -- 172.21.15.3:0/659777603 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52d4003570 con 0x7f52ec0cba50 2023-10-06T04:58:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f134f700 1 -- 172.21.15.3:0/659777603 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127467743 0 0) 0x7f52d4003570 con 0x7f52ec0cba50 2023-10-06T04:58:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f2b52700 1 -- 172.21.15.3:0/659777603 >> v2:172.21.15.153:3300/0 conn(0x7f52ec150690 msgr2=0x7f52ec150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f2b52700 1 --2- 172.21.15.3:0/659777603 >> v2:172.21.15.153:3300/0 conn(0x7f52ec150690 0x7f52ec150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f2b52700 1 -- 172.21.15.3:0/659777603 >> v1:172.21.15.3:6789/0 conn(0x7f52ec0cba50 legacy=0x7f52ec15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f2b52700 1 -- 172.21.15.3:0/659777603 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec15dcb0 con 0x7f52ec15d540 2023-10-06T04:58:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.872+0000 7f52f2b52700 1 --2- 172.21.15.3:0/659777603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec15d540 0x7f52ec0af190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009e60 tx=0x7f52dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2215e4058e03605a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f134f700 1 -- 172.21.15.3:0/659777603 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00e070 con 0x7f52ec15d540 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f134f700 1 -- 172.21.15.3:0/659777603 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc00b7a0 con 0x7f52ec15d540 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f134f700 1 -- 172.21.15.3:0/659777603 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0001470 con 0x7f52ec15d540 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 -- 172.21.15.3:0/659777603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec15d540 msgr2=0x7f52ec0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 --2- 172.21.15.3:0/659777603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec15d540 0x7f52ec0af190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009e60 tx=0x7f52dc009780 comp rx=0 tx=0).stop 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 -- 172.21.15.3:0/659777603 shutdown_connections 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 --2- 172.21.15.3:0/659777603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec15d540 0x7f52ec0af190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 --2- 172.21.15.3:0/659777603 >> v2:172.21.15.153:3300/0 conn(0x7f52ec150690 0x7f52ec150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 -- 172.21.15.3:0/659777603 >> 172.21.15.3:0/659777603 conn(0x7f52ec070310 msgr2=0x7f52ec070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 -- 172.21.15.3:0/659777603 shutdown_connections 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.873+0000 7f52f45b5700 1 -- 172.21.15.3:0/659777603 wait complete. 2023-10-06T04:58:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.874+0000 7f52f45b5700 1 Processor -- start 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.874+0000 7f52f45b5700 1 -- start start 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f45b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52ec0cba50 0x7f52ec150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f45b5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec150690 0x7f52ec169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f45b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ec15d540 0x7f52ec16a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f45b5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec151c50 con 0x7f52ec15d540 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f45b5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec16ac00 con 0x7f52ec0cba50 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f45b5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec151040 con 0x7f52ec150690 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ec15d540 0x7f52ec16a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ec15d540 0x7f52ec16a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42656/0 (socket says 172.21.15.3:42656) 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 -- 172.21.15.3:0/1007196745 learned_addr learned my addr 172.21.15.3:0/1007196745 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2351700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52ec0cba50 0x7f52ec150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f1b50700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec150690 0x7f52ec169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 -- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec150690 msgr2=0x7f52ec169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec150690 0x7f52ec169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 -- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52ec0cba50 msgr2=0x7f52ec150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52ec0cba50 0x7f52ec150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.875+0000 7f52f2b52700 1 -- 172.21.15.3:0/1007196745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc009a20 con 0x7f52ec15d540 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.876+0000 7f52f2b52700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ec15d540 0x7f52ec16a6c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b610 tx=0x7f52dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.876+0000 7f52ea7fc700 1 -- 172.21.15.3:0/1007196745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc012d50 con 0x7f52ec15d540 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.876+0000 7f52ea7fc700 1 -- 172.21.15.3:0/1007196745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc01e430 con 0x7f52ec15d540 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.876+0000 7f52ea7fc700 1 -- 172.21.15.3:0/1007196745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc01e7f0 con 0x7f52ec15d540 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.876+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec151320 con 0x7f52ec15d540 2023-10-06T04:58:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.876+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1518e0 con 0x7f52ec15d540 2023-10-06T04:58:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.877+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec151e00 con 0x7f52ec15d540 2023-10-06T04:58:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.877+0000 7f52ea7fc700 1 -- 172.21.15.3:0/1007196745 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52dc01ebe0 con 0x7f52ec15d540 2023-10-06T04:58:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.878+0000 7f52ea7fc700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52d8066f30 0x7f52d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.878+0000 7f52ea7fc700 1 -- 172.21.15.3:0/1007196745 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f52dc086310 con 0x7f52ec15d540 2023-10-06T04:58:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.878+0000 7f52f2351700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52d8066f30 0x7f52d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.878+0000 7f52f2351700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52d8066f30 0x7f52d80693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009d40 tx=0x7f52e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:26.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:26.882+0000 7f52ea7fc700 1 -- 172.21.15.3:0/1007196745 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52ec151e00 con 0x7f52ec15d540 2023-10-06T04:58:27.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.076+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f52ec04c380 con 0x7f52ec15d540 2023-10-06T04:58:27.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T04:58:27.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.711+0000 7f52ea7fc700 1 -- 172.21.15.3:0/1007196745 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7f52dc053600 con 0x7f52ec15d540 2023-10-06T04:58:27.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.715+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52d8066f30 msgr2=0x7f52d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:27.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.715+0000 7f52f45b5700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52d8066f30 0x7f52d80693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009d40 tx=0x7f52e4008040 comp rx=0 tx=0).stop 2023-10-06T04:58:27.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.715+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ec15d540 msgr2=0x7f52ec16a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:27.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.715+0000 7f52f45b5700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ec15d540 0x7f52ec16a6c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b610 tx=0x7f52dc018000 comp rx=0 tx=0).stop 2023-10-06T04:58:27.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.716+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 shutdown_connections 2023-10-06T04:58:27.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.716+0000 7f52f45b5700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52d8066f30 0x7f52d80693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.716+0000 7f52f45b5700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52ec0cba50 0x7f52ec150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.716+0000 7f52f45b5700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ec15d540 0x7f52ec16a6c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.716+0000 7f52f45b5700 1 --2- 172.21.15.3:0/1007196745 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52ec150690 0x7f52ec169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.716+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 >> 172.21.15.3:0/1007196745 conn(0x7f52ec070310 msgr2=0x7f52ec0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:27.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.716+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 shutdown_connections 2023-10-06T04:58:27.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.717+0000 7f52f45b5700 1 -- 172.21.15.3:0/1007196745 wait complete. 2023-10-06T04:58:27.727 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:58:27.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:58:27.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.864+0000 7eff8c8be700 1 Processor -- start 2023-10-06T04:58:27.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.864+0000 7eff8c8be700 1 -- start start 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.865+0000 7eff8c8be700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7eff880cba50 0x7eff88150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.865+0000 7eff8c8be700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 0x7eff880af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.865+0000 7eff8c8be700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8815a290 con 0x7eff88150690 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.865+0000 7eff8c8be700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880d0020 con 0x7eff880cba50 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.865+0000 7eff8c8be700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff88159f40 con 0x7eff8815d540 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7eff88150690 0x7eff8815a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50034/0 (socket says 172.21.15.3:50034) 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8659c700 1 -- 172.21.15.3:0/2513745888 learned_addr learned my addr 172.21.15.3:0/2513745888 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff85d9b700 1 --2- 172.21.15.3:0/2513745888 >> v2:172.21.15.153:3300/0 conn(0x7eff880cba50 0x7eff88150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff86d9d700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 0x7eff880af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479739586 0 0) 0x7eff8815a290 con 0x7eff88150690 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff74003570 con 0x7eff88150690 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2839723517 0 0) 0x7eff74003570 con 0x7eff88150690 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 msgr2=0x7eff880af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 0x7eff880af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 >> v2:172.21.15.153:3300/0 conn(0x7eff880cba50 msgr2=0x7eff88150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 --2- 172.21.15.3:0/2513745888 >> v2:172.21.15.153:3300/0 conn(0x7eff880cba50 0x7eff88150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8815dcb0 con 0x7eff88150690 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7eff78001470 con 0x7eff88150690 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff85d9b700 1 --2- 172.21.15.3:0/2513745888 >> v2:172.21.15.153:3300/0 conn(0x7eff880cba50 0x7eff88150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 >> v1:172.21.15.3:6789/0 conn(0x7eff88150690 legacy=0x7eff8815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 0x7eff74003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74003fe0 0x7eff74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff74004d50 0x7eff740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff880d0020 con 0x7eff74004d50 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88159f40 con 0x7eff74003fe0 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7400ba90 con 0x7eff8815d540 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff86d9d700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff74004d50 0x7eff740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff8659c700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 0x7eff74003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.866+0000 7eff85d9b700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74003fe0 0x7eff74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff86d9d700 1 -- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 msgr2=0x7eff74003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff86d9d700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff8815d540 0x7eff74003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff86d9d700 1 -- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74003fe0 msgr2=0x7eff74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff86d9d700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74003fe0 0x7eff74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff86d9d700 1 -- 172.21.15.3:0/2513745888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff74003570 con 0x7eff74004d50 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff86d9d700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff74004d50 0x7eff740051c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7eff8000e210 tx=0x7eff8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8110ee0af8ede29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000ea60 con 0x7eff74004d50 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000f040 con 0x7eff74004d50 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff8559a700 1 -- 172.21.15.3:0/2513745888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000ee10 con 0x7eff74004d50 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff8c8be700 1 -- 172.21.15.3:0/2513745888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff74004d50 msgr2=0x7eff740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff8c8be700 1 --2- 172.21.15.3:0/2513745888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff74004d50 0x7eff740051c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7eff8000e210 tx=0x7eff8000b040 comp rx=0 tx=0).stop 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.867+0000 7eff86d9d700 1 -- 172.21.15.3:0/2513745888 reap_dead start 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.868+0000 7eff8c8be700 1 -- 172.21.15.3:0/2513745888 shutdown_connections 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.868+0000 7eff8c8be700 1 -- 172.21.15.3:0/2513745888 >> 172.21.15.3:0/2513745888 conn(0x7eff88070310 msgr2=0x7eff88070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.868+0000 7eff8c8be700 1 -- 172.21.15.3:0/2513745888 shutdown_connections 2023-10-06T04:58:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.868+0000 7eff8c8be700 1 -- 172.21.15.3:0/2513745888 wait complete. 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.869+0000 7eff8c8be700 1 Processor -- start 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.869+0000 7eff8c8be700 1 -- start start 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.869+0000 7eff8c8be700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74004d50 0x7eff8814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.869+0000 7eff8c8be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff880b4960 0x7eff881523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.869+0000 7eff8c8be700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff880af0b0 0x7eff880af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.869+0000 7eff8c8be700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff880afa90 con 0x7eff880b4960 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.869+0000 7eff8c8be700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881530b0 con 0x7eff74004d50 2023-10-06T04:58:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff8c8be700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880c7f70 con 0x7eff880af0b0 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff880b4960 0x7eff881523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff880b4960 0x7eff881523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42670/0 (socket says 172.21.15.3:42670) 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 -- 172.21.15.3:0/2903105720 learned_addr learned my addr 172.21.15.3:0/2903105720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff8659c700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74004d50 0x7eff8814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff86d9d700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff880af0b0 0x7eff880af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 -- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff880af0b0 msgr2=0x7eff880af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff880af0b0 0x7eff880af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 -- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74004d50 msgr2=0x7eff8814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74004d50 0x7eff8814fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 -- 172.21.15.3:0/2903105720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880b5970 con 0x7eff880b4960 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff86d9d700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff880af0b0 0x7eff880af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff8659c700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74004d50 0x7eff8814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff85d9b700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff880b4960 0x7eff881523f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7eff70011da0 tx=0x7eff700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff7e7fc700 1 -- 172.21.15.3:0/2903105720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7000fcd0 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff7e7fc700 1 -- 172.21.15.3:0/2903105720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7000fe30 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff7e7fc700 1 -- 172.21.15.3:0/2903105720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7001d6d0 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff880b4df0 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.870+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff880b5690 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.871+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff880b51e0 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.872+0000 7eff7e7fc700 1 -- 172.21.15.3:0/2903105720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff7001dac0 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.872+0000 7eff7e7fc700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff68066f30 0x7eff680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.872+0000 7eff7e7fc700 1 -- 172.21.15.3:0/2903105720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5958+0+0 (secure 0 0 0) 0x7eff70018070 con 0x7eff880b4960 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.872+0000 7eff8659c700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff68066f30 0x7eff680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.873+0000 7eff8659c700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff68066f30 0x7eff680693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff780079e0 tx=0x7eff78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:27.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:27.877+0000 7eff7e7fc700 1 -- 172.21.15.3:0/2903105720 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff70052680 con 0x7eff880b4960 2023-10-06T04:58:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.113+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7eff8804c380 con 0x7eff880b4960 2023-10-06T04:58:28.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.747+0000 7eff7e7fc700 1 -- 172.21.15.3:0/2903105720 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v47) v1 ==== 174+0+0 (secure 0 0 0) 0x7eff70055ca0 con 0x7eff880b4960 2023-10-06T04:58:28.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff68066f30 msgr2=0x7eff680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:28.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff68066f30 0x7eff680693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff780079e0 tx=0x7eff78009040 comp rx=0 tx=0).stop 2023-10-06T04:58:28.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff880b4960 msgr2=0x7eff881523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:28.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff880b4960 0x7eff881523f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7eff70011da0 tx=0x7eff700091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 shutdown_connections 2023-10-06T04:58:28.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff68066f30 0x7eff680693e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff74004d50 0x7eff8814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff880af0b0 0x7eff880af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 --2- 172.21.15.3:0/2903105720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff880b4960 0x7eff881523f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 >> 172.21.15.3:0/2903105720 conn(0x7eff88070310 msgr2=0x7eff880b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:28.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 shutdown_connections 2023-10-06T04:58:28.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.756+0000 7eff8c8be700 1 -- 172.21.15.3:0/2903105720 wait complete. 2023-10-06T04:58:28.763 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 13 and data pool 14 2023-10-06T04:58:28.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:28.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.900+0000 7feec14b7700 1 Processor -- start 2023-10-06T04:58:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.900+0000 7feec14b7700 1 -- start start 2023-10-06T04:58:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feec14b7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 0x7feebc0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feec14b7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7feebc0b2740 0x7feebc0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feec14b7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc0b0ce0 con 0x7feebc15cb50 2023-10-06T04:58:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feec14b7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0cb820 con 0x7feebc0b2740 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feec14b7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b0760 con 0x7feebc0cd950 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feebaffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feebc15cb50 0x7feebc0b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50044/0 (socket says 172.21.15.3:50044) 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feebaffd700 1 -- 172.21.15.3:0/1846703186 learned_addr learned my addr 172.21.15.3:0/1846703186 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeba7fc700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 0x7feebc0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1846703186 >> v2:172.21.15.153:3300/0 conn(0x7feebc0b2740 0x7feebc0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373584649 0 0) 0x7feebc0b0ce0 con 0x7feebc15cb50 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea8003570 con 0x7feebc15cb50 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3623235774 0 0) 0x7feea8003570 con 0x7feebc15cb50 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 msgr2=0x7feebc0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 0x7feebc0b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 >> v2:172.21.15.153:3300/0 conn(0x7feebc0b2740 msgr2=0x7feebc0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 --2- 172.21.15.3:0/1846703186 >> v2:172.21.15.153:3300/0 conn(0x7feebc0b2740 0x7feebc0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0b0a00 con 0x7feebc15cb50 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7feeac001970 con 0x7feebc15cb50 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.901+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 >> v1:172.21.15.3:6789/0 conn(0x7feebc15cb50 legacy=0x7feebc0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 0x7feea8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8003fe0 0x7feea8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea8004d50 0x7feea80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0cb820 con 0x7feea8004d50 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b0760 con 0x7feea8003fe0 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea800ba90 con 0x7feebc0cd950 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1846703186 >> v2:172.21.15.153:3300/0 conn(0x7feebc0b2740 0x7feebc0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea8004d50 0x7feea80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebaffd700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 0x7feea8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeba7fc700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8003fe0 0x7feea8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 -- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 msgr2=0x7feea8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0cd950 0x7feea8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 -- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8003fe0 msgr2=0x7feea8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8003fe0 0x7feea8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 -- 172.21.15.3:0/1846703186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8003570 con 0x7feea8004d50 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeba7fc700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8003fe0 0x7feea8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea8004d50 0x7feea80051c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009db0 tx=0x7feeb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8fb47db4e7dbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4009740 con 0x7feea8004d50 2023-10-06T04:58:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb400e8c0 con 0x7feea8004d50 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.902+0000 7feeb9ffb700 1 -- 172.21.15.3:0/1846703186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb400ec40 con 0x7feea8004d50 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.903+0000 7feec14b7700 1 -- 172.21.15.3:0/1846703186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea8004d50 msgr2=0x7feea80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.903+0000 7feec14b7700 1 --2- 172.21.15.3:0/1846703186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea8004d50 0x7feea80051c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009db0 tx=0x7feeb40091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.903+0000 7feebb7fe700 1 -- 172.21.15.3:0/1846703186 reap_dead start 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.903+0000 7feec14b7700 1 -- 172.21.15.3:0/1846703186 shutdown_connections 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.903+0000 7feec14b7700 1 -- 172.21.15.3:0/1846703186 >> 172.21.15.3:0/1846703186 conn(0x7feebc070310 msgr2=0x7feebc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.903+0000 7feec14b7700 1 -- 172.21.15.3:0/1846703186 shutdown_connections 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.903+0000 7feec14b7700 1 -- 172.21.15.3:0/1846703186 wait complete. 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.904+0000 7feec14b7700 1 Processor -- start 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.904+0000 7feec14b7700 1 -- start start 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feec14b7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8004d50 0x7feebc0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feec14b7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0b3e80 0x7feebc152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feec14b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feebc0c4220 0x7feebc0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feec14b7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b4dc0 con 0x7feebc0c4220 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feec14b7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0ce120 con 0x7feea8004d50 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feec14b7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0ce3c0 con 0x7feebc0b3e80 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feebc0c4220 0x7feebc0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feebc0c4220 0x7feebc0c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42682/0 (socket says 172.21.15.3:42682) 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 -- 172.21.15.3:0/1217380822 learned_addr learned my addr 172.21.15.3:0/1217380822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebaffd700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8004d50 0x7feebc0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feeba7fc700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0b3e80 0x7feebc152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 -- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0b3e80 msgr2=0x7feebc152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0b3e80 0x7feebc152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 -- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8004d50 msgr2=0x7feebc0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8004d50 0x7feebc0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feebb7fe700 1 -- 172.21.15.3:0/1217380822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0ce6a0 con 0x7feebc0c4220 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.905+0000 7feeba7fc700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0b3e80 0x7feebc152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.906+0000 7feebb7fe700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feebc0c4220 0x7feebc0c90b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feeb400f010 tx=0x7feeb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.906+0000 7feeb2ffd700 1 -- 172.21.15.3:0/1217380822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb400ea30 con 0x7feebc0c4220 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.906+0000 7feeb2ffd700 1 -- 172.21.15.3:0/1217380822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4021040 con 0x7feebc0c4220 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.906+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc0ce980 con 0x7feebc0c4220 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.906+0000 7feeb2ffd700 1 -- 172.21.15.3:0/1217380822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40253f0 con 0x7feebc0c4220 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.906+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc153280 con 0x7feebc0c4220 2023-10-06T04:58:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.907+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc071680 con 0x7feebc0c4220 2023-10-06T04:58:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.907+0000 7feeb2ffd700 1 -- 172.21.15.3:0/1217380822 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feeb40257e0 con 0x7feebc0c4220 2023-10-06T04:58:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.907+0000 7feeb2ffd700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.908+0000 7feeb2ffd700 1 -- 172.21.15.3:0/1217380822 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feeb40862c0 con 0x7feebc0c4220 2023-10-06T04:58:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.908+0000 7feebaffd700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.908+0000 7feebaffd700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feeac007db0 tx=0x7feeac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:28.912+0000 7feeb2ffd700 1 -- 172.21.15.3:0/1217380822 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeb40533d0 con 0x7feebc0c4220 2023-10-06T04:58:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.108+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feebc0cb1d0 con 0x7feebc0c4220 2023-10-06T04:58:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.109+0000 7feeb2ffd700 1 -- 172.21.15.3:0/1217380822 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16834 (secure 0 0 0) 0x7feeb40569f0 con 0x7feebc0c4220 2023-10-06T04:58:29.111 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:29.111 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:58:28.114399+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:58:25.874792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T04:58:27.107107+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee9c066f30 msgr2=0x7fee9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feeac007db0 tx=0x7feeac009040 comp rx=0 tx=0).stop 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feebc0c4220 msgr2=0x7feebc0c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feebc0c4220 0x7feebc0c90b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feeb400f010 tx=0x7feeb4019000 comp rx=0 tx=0).stop 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 shutdown_connections 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feea8004d50 0x7feebc0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feebc0c4220 0x7feebc0c90b0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 --2- 172.21.15.3:0/1217380822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feebc0b3e80 0x7feebc152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 >> 172.21.15.3:0/1217380822 conn(0x7feebc070310 msgr2=0x7feebc15cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 shutdown_connections 2023-10-06T04:58:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.111+0000 7feec14b7700 1 -- 172.21.15.3:0/1217380822 wait complete. 2023-10-06T04:58:29.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:29.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.244+0000 7f8b76936700 1 Processor -- start 2023-10-06T04:58:29.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.244+0000 7f8b76936700 1 -- start start 2023-10-06T04:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.244+0000 7f8b76936700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 0x7f8b700cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b76936700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8b700b2930 0x7f8b700cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b76936700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700cc5c0 con 0x7f8b701598d0 2023-10-06T04:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b76936700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b700cb7b0 con 0x7f8b700b2930 2023-10-06T04:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b76936700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700cda60 con 0x7f8b70159500 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8b701598d0 0x7f8b700b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50054/0 (socket says 172.21.15.3:50054) 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6f7fe700 1 -- 172.21.15.3:0/3069176295 learned_addr learned my addr 172.21.15.3:0/3069176295 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b74ed3700 1 --2- 172.21.15.3:0/3069176295 >> v2:172.21.15.153:3300/0 conn(0x7f8b700b2930 0x7f8b700cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6ffff700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 0x7f8b700cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163624795 0 0) 0x7f8b700cc5c0 con 0x7f8b701598d0 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b58003570 con 0x7f8b701598d0 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210167624 0 0) 0x7f8b58003570 con 0x7f8b701598d0 2023-10-06T04:58:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 msgr2=0x7f8b700cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 0x7f8b700cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 >> v2:172.21.15.153:3300/0 conn(0x7f8b700b2930 msgr2=0x7f8b700cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 --2- 172.21.15.3:0/3069176295 >> v2:172.21.15.153:3300/0 conn(0x7f8b700b2930 0x7f8b700cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700cb9b0 con 0x7f8b701598d0 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f8b64001470 con 0x7f8b701598d0 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 >> v1:172.21.15.3:6789/0 conn(0x7f8b701598d0 legacy=0x7f8b700b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 0x7f8b58003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b58003fe0 0x7f8b58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58004d50 0x7f8b580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700cb7b0 con 0x7f8b58003fe0 2023-10-06T04:58:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700cda60 con 0x7f8b58004d50 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5800ba90 con 0x7f8b70159500 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.245+0000 7f8b6f7fe700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b58003fe0 0x7f8b58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6ffff700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 0x7f8b58003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b74ed3700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58004d50 0x7f8b580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6f7fe700 1 -- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 msgr2=0x7f8b58003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6f7fe700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b70159500 0x7f8b58003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6f7fe700 1 -- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58004d50 msgr2=0x7f8b580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6f7fe700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58004d50 0x7f8b580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6f7fe700 1 -- 172.21.15.3:0/3069176295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b58003570 con 0x7f8b58003fe0 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6f7fe700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b58003fe0 0x7f8b58004810 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c3c0 tx=0x7f8b64001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682b33abb7ef652b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400cc30 con 0x7f8b58003fe0 2023-10-06T04:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6400cdb0 con 0x7f8b58003fe0 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b6effd700 1 -- 172.21.15.3:0/3069176295 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64005b70 con 0x7f8b58003fe0 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b76936700 1 -- 172.21.15.3:0/3069176295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b58003fe0 msgr2=0x7f8b58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b76936700 1 --2- 172.21.15.3:0/3069176295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b58003fe0 0x7f8b58004810 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c3c0 tx=0x7f8b64001e90 comp rx=0 tx=0).stop 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.246+0000 7f8b74ed3700 1 -- 172.21.15.3:0/3069176295 reap_dead start 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.247+0000 7f8b76936700 1 -- 172.21.15.3:0/3069176295 shutdown_connections 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.247+0000 7f8b76936700 1 -- 172.21.15.3:0/3069176295 >> 172.21.15.3:0/3069176295 conn(0x7f8b70070310 msgr2=0x7f8b70070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.247+0000 7f8b76936700 1 -- 172.21.15.3:0/3069176295 shutdown_connections 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.247+0000 7f8b76936700 1 -- 172.21.15.3:0/3069176295 wait complete. 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.248+0000 7f8b76936700 1 Processor -- start 2023-10-06T04:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.248+0000 7f8b76936700 1 -- start start 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.248+0000 7f8b76936700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58003fe0 0x7f8b700b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.248+0000 7f8b76936700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b7015b390 0x7f8b70145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.248+0000 7f8b76936700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7016cdb0 0x7f8b7016d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b76936700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7015a6d0 con 0x7f8b7016cdb0 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b76936700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7016d790 con 0x7f8b58003fe0 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b76936700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700b3840 con 0x7f8b7015b390 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7016cdb0 0x7f8b7016d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7016cdb0 0x7f8b7016d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42708/0 (socket says 172.21.15.3:42708) 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 -- 172.21.15.3:0/4177231096 learned_addr learned my addr 172.21.15.3:0/4177231096 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b6ffff700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58003fe0 0x7f8b700b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b6f7fe700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b7015b390 0x7f8b70145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 -- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b7015b390 msgr2=0x7f8b70145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b7015b390 0x7f8b70145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 -- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58003fe0 msgr2=0x7f8b700b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58003fe0 0x7f8b700b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 -- 172.21.15.3:0/4177231096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700c4250 con 0x7f8b7016cdb0 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b74ed3700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7016cdb0 0x7f8b7016d250 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8b680094d0 tx=0x7f8b6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b57fff700 1 -- 172.21.15.3:0/4177231096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800e040 con 0x7f8b7016cdb0 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b57fff700 1 -- 172.21.15.3:0/4177231096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800f040 con 0x7f8b7016cdb0 2023-10-06T04:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b57fff700 1 -- 172.21.15.3:0/4177231096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800fe50 con 0x7f8b7016cdb0 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7015a8b0 con 0x7f8b7016cdb0 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.249+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7015ae70 con 0x7f8b7016cdb0 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.250+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7015b060 con 0x7f8b7016cdb0 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.251+0000 7f8b57fff700 1 -- 172.21.15.3:0/4177231096 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b68015640 con 0x7f8b7016cdb0 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.251+0000 7f8b57fff700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b5c066f30 0x7f8b5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.251+0000 7f8b57fff700 1 -- 172.21.15.3:0/4177231096 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b68084f80 con 0x7f8b7016cdb0 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.251+0000 7f8b6ffff700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b5c066f30 0x7f8b5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.252+0000 7f8b6ffff700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b5c066f30 0x7f8b5c0693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006dc0 tx=0x7f8b60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.255+0000 7f8b57fff700 1 -- 172.21.15.3:0/4177231096 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b68055b20 con 0x7f8b7016cdb0 2023-10-06T04:58:29.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.449+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b70071680 con 0x7f8b7016cdb0 2023-10-06T04:58:29.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.450+0000 7f8b57fff700 1 -- 172.21.15.3:0/4177231096 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f8b680556b0 con 0x7f8b7016cdb0 2023-10-06T04:58:29.452 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:29.452 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:58:28.114399+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:58:25.874792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T04:58:27.107107+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b5c066f30 msgr2=0x7f8b5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b5c066f30 0x7f8b5c0693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006dc0 tx=0x7f8b60008040 comp rx=0 tx=0).stop 2023-10-06T04:58:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7016cdb0 msgr2=0x7f8b7016d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7016cdb0 0x7f8b7016d250 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8b680094d0 tx=0x7f8b6800b040 comp rx=0 tx=0).stop 2023-10-06T04:58:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 shutdown_connections 2023-10-06T04:58:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8b5c066f30 0x7f8b5c0693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b58003fe0 0x7f8b700b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7016cdb0 0x7f8b7016d250 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 --2- 172.21.15.3:0/4177231096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b7015b390 0x7f8b70145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 >> 172.21.15.3:0/4177231096 conn(0x7f8b70070310 msgr2=0x7f8b700709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 shutdown_connections 2023-10-06T04:58:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.452+0000 7f8b76936700 1 -- 172.21.15.3:0/4177231096 wait complete. 2023-10-06T04:58:29.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:58:29.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.588+0000 7f1b47337700 1 Processor -- start 2023-10-06T04:58:29.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.588+0000 7f1b47337700 1 -- start start 2023-10-06T04:58:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b47337700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1b40155f80 0x7f1b4006ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b47337700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40069720 0x7f1b400c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b47337700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b40156350 0x7f1b40159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b47337700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4006a7c0 con 0x7f1b40156350 2023-10-06T04:58:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b47337700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b40069bb0 con 0x7f1b40155f80 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b47337700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b40069e90 con 0x7f1b40069720 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b40156350 0x7f1b40159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b40156350 0x7f1b40159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42716/0 (socket says 172.21.15.3:42716) 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 -- 172.21.15.3:0/4242626405 learned_addr learned my addr 172.21.15.3:0/4242626405 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 -- 172.21.15.3:0/4242626405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40069720 msgr2=0x7f1b400c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b45b34700 1 --2- 172.21.15.3:0/4242626405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40069720 0x7f1b400c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46335700 1 --2- 172.21.15.3:0/4242626405 >> v2:172.21.15.153:3300/0 conn(0x7f1b40155f80 0x7f1b4006ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 --2- 172.21.15.3:0/4242626405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40069720 0x7f1b400c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 -- 172.21.15.3:0/4242626405 >> v2:172.21.15.153:3300/0 conn(0x7f1b40155f80 msgr2=0x7f1b4006ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 --2- 172.21.15.3:0/4242626405 >> v2:172.21.15.153:3300/0 conn(0x7f1b40155f80 0x7f1b4006ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.589+0000 7f1b46b36700 1 -- 172.21.15.3:0/4242626405 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4006a170 con 0x7f1b40156350 2023-10-06T04:58:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b46b36700 1 --2- 172.21.15.3:0/4242626405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40156350 0x7f1b40159d00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1b400693e0 tx=0x7f1b3000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d902e84ab7b8f5c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b45333700 1 -- 172.21.15.3:0/4242626405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30010070 con 0x7f1b40156350 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b45333700 1 -- 172.21.15.3:0/4242626405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3000de90 con 0x7f1b40156350 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b45333700 1 -- 172.21.15.3:0/4242626405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000b320 con 0x7f1b40156350 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 -- 172.21.15.3:0/4242626405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40156350 msgr2=0x7f1b40159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 --2- 172.21.15.3:0/4242626405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40156350 0x7f1b40159d00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1b400693e0 tx=0x7f1b3000ba50 comp rx=0 tx=0).stop 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 -- 172.21.15.3:0/4242626405 shutdown_connections 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 --2- 172.21.15.3:0/4242626405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40156350 0x7f1b40159d00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 --2- 172.21.15.3:0/4242626405 >> v2:172.21.15.153:3300/0 conn(0x7f1b40155f80 0x7f1b4006ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 --2- 172.21.15.3:0/4242626405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40069720 0x7f1b400c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 -- 172.21.15.3:0/4242626405 >> 172.21.15.3:0/4242626405 conn(0x7f1b4006c370 msgr2=0x7f1b4006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 -- 172.21.15.3:0/4242626405 shutdown_connections 2023-10-06T04:58:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.590+0000 7f1b47337700 1 -- 172.21.15.3:0/4242626405 wait complete. 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.591+0000 7f1b47337700 1 Processor -- start 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b47337700 1 -- start start 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b47337700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40069720 0x7f1b40154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b47337700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40155f80 0x7f1b400c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b47337700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b40156350 0x7f1b400aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b47337700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400c5b20 con 0x7f1b40069720 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b47337700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400a9860 con 0x7f1b40156350 2023-10-06T04:58:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b47337700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400aa3e0 con 0x7f1b40155f80 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b46335700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40069720 0x7f1b40154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b46335700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40069720 0x7f1b40154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42726/0 (socket says 172.21.15.3:42726) 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b46335700 1 -- 172.21.15.3:0/2940121360 learned_addr learned my addr 172.21.15.3:0/2940121360 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b45b34700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40155f80 0x7f1b400c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.592+0000 7f1b46b36700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b40156350 0x7f1b400aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b46335700 1 -- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40155f80 msgr2=0x7f1b400c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b46335700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40155f80 0x7f1b400c3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b46335700 1 -- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b40156350 msgr2=0x7f1b400aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b46335700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b40156350 0x7f1b400aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b46335700 1 -- 172.21.15.3:0/2940121360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3000bcf0 con 0x7f1b40069720 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b46b36700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b40156350 0x7f1b400aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b46335700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40069720 0x7f1b40154820 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c800 tx=0x7f1b38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b3e7fc700 1 -- 172.21.15.3:0/2940121360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38011600 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b3e7fc700 1 -- 172.21.15.3:0/2940121360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b38011780 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b400a9ab0 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b3e7fc700 1 -- 172.21.15.3:0/2940121360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38011b40 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.593+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b400aa070 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.594+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4004c380 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.594+0000 7f1b3e7fc700 1 -- 172.21.15.3:0/2940121360 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b380183a0 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.595+0000 7f1b3e7fc700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1b2c066f30 0x7f1b2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.595+0000 7f1b3e7fc700 1 -- 172.21.15.3:0/2940121360 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1b380843a0 con 0x7f1b40069720 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.595+0000 7f1b45b34700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1b2c066f30 0x7f1b2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.596+0000 7f1b45b34700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1b2c066f30 0x7f1b2c0693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1b40154d60 tx=0x7f1b34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:29.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.599+0000 7f1b3e7fc700 1 -- 172.21.15.3:0/2940121360 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b38051410 con 0x7f1b40069720 2023-10-06T04:58:29.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:29.844+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1b4006a450 con 0x7f1b40069720 2023-10-06T04:58:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.749+0000 7f1b3e7fc700 1 -- 172.21.15.3:0/2940121360 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1b38054a30 con 0x7f1b40069720 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1b2c066f30 msgr2=0x7f1b2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1b2c066f30 0x7f1b2c0693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1b40154d60 tx=0x7f1b34008040 comp rx=0 tx=0).stop 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40069720 msgr2=0x7f1b40154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40069720 0x7f1b40154820 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c800 tx=0x7f1b38008040 comp rx=0 tx=0).stop 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 shutdown_connections 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1b2c066f30 0x7f1b2c0693e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b40156350 0x7f1b400aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b40069720 0x7f1b40154820 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 --2- 172.21.15.3:0/2940121360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b40155f80 0x7f1b400c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 >> 172.21.15.3:0/2940121360 conn(0x7f1b4006c370 msgr2=0x7f1b400acb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:30.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 shutdown_connections 2023-10-06T04:58:30.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.752+0000 7f1b47337700 1 -- 172.21.15.3:0/2940121360 wait complete. 2023-10-06T04:58:30.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:58:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.896+0000 7f1545658700 1 Processor -- start 2023-10-06T04:58:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.896+0000 7f1545658700 1 -- start start 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.897+0000 7f1545658700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f15400b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.897+0000 7f1545658700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f15400b1b00 0x7f154015c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.897+0000 7f1545658700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15400691c0 con 0x7f1540068190 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.897+0000 7f1545658700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15400694a0 con 0x7f15400b1b00 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.897+0000 7f1545658700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1540069780 con 0x7f1540067dc0 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153e7fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f15400b1b00 0x7f154015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f15400b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1540068190 0x7f154015d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:33672/0 (socket says 172.21.15.3:33672) 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153f7fe700 1 -- 172.21.15.3:0/3303366842 learned_addr learned my addr 172.21.15.3:0/3303366842 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153dffb700 1 -- 172.21.15.3:0/3303366842 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512054316 0 0) 0x7f15400691c0 con 0x7f1540068190 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153dffb700 1 -- 172.21.15.3:0/3303366842 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f152c003570 con 0x7f1540068190 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153e7fc700 1 -- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 msgr2=0x7f15400b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153e7fc700 1 --2- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f15400b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153e7fc700 1 -- 172.21.15.3:0/3303366842 >> v1:172.21.15.153:6789/0 conn(0x7f1540068190 legacy=0x7f154015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153e7fc700 1 -- 172.21.15.3:0/3303366842 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540069a60 con 0x7f15400b1b00 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153effd700 1 --2- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f15400b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153e7fc700 1 --2- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15400b1b00 0x7f154015c800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f15280098e0 tx=0x7f15280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20530de5581ed7a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153dffb700 1 -- 172.21.15.3:0/3303366842 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528011630 con 0x7f15400b1b00 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153dffb700 1 -- 172.21.15.3:0/3303366842 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1528011790 con 0x7f15400b1b00 2023-10-06T04:58:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.898+0000 7f153dffb700 1 -- 172.21.15.3:0/3303366842 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528011b30 con 0x7f15400b1b00 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 -- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15400b1b00 msgr2=0x7f154015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 --2- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15400b1b00 0x7f154015c800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f15280098e0 tx=0x7f15280091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 -- 172.21.15.3:0/3303366842 shutdown_connections 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 --2- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f15400b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 --2- 172.21.15.3:0/3303366842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15400b1b00 0x7f154015c800 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 -- 172.21.15.3:0/3303366842 >> 172.21.15.3:0/3303366842 conn(0x7f15400ce270 msgr2=0x7f15400ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 -- 172.21.15.3:0/3303366842 shutdown_connections 2023-10-06T04:58:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.899+0000 7f1545658700 1 -- 172.21.15.3:0/3303366842 wait complete. 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.900+0000 7f1545658700 1 Processor -- start 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.900+0000 7f1545658700 1 -- start start 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f1545658700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f154013f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f1545658700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1540068190 0x7f15400c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f1545658700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15400b1b00 0x7f15400c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f1545658700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15400ca410 con 0x7f15400b1b00 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f1545658700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154013e1d0 con 0x7f1540068190 2023-10-06T04:58:30.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f1545658700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154013e400 con 0x7f1540067dc0 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15400b1b00 0x7f15400c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15400b1b00 0x7f15400c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42734/0 (socket says 172.21.15.3:42734) 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f153f7fe700 1 -- 172.21.15.3:0/1526825354 learned_addr learned my addr 172.21.15.3:0/1526825354 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f153e7fc700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1540068190 0x7f15400c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.901+0000 7f153effd700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f154013f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f153f7fe700 1 -- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 msgr2=0x7f154013f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f153f7fe700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f154013f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f153f7fe700 1 -- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1540068190 msgr2=0x7f15400c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f153f7fe700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1540068190 0x7f15400c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f153f7fe700 1 -- 172.21.15.3:0/1526825354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15380093c0 con 0x7f15400b1b00 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f153f7fe700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15400b1b00 0x7f15400c9ed0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1530001460 tx=0x7f1530001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f1536ffd700 1 -- 172.21.15.3:0/1526825354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000a890 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f1536ffd700 1 -- 172.21.15.3:0/1526825354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153000a9f0 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1528009420 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f1536ffd700 1 -- 172.21.15.3:0/1526825354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000ad70 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.902+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154013e9c0 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.903+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154004c380 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.903+0000 7f1536ffd700 1 -- 172.21.15.3:0/1526825354 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1530007300 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.904+0000 7f1536ffd700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.904+0000 7f1536ffd700 1 -- 172.21.15.3:0/1526825354 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1530084a80 con 0x7f15400b1b00 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.904+0000 7f153effd700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.905+0000 7f153effd700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1538009f30 tx=0x7f1538009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:30.908+0000 7f1536ffd700 1 -- 172.21.15.3:0/1526825354 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1530051b90 con 0x7f15400b1b00 2023-10-06T04:58:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.149+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1540069d40 con 0x7f15400b1b00 2023-10-06T04:58:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.757+0000 7f1536ffd700 1 -- 172.21.15.3:0/1526825354 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7f15300551b0 con 0x7f15400b1b00 2023-10-06T04:58:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 msgr2=0x7f15200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1538009f30 tx=0x7f1538009120 comp rx=0 tx=0).stop 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15400b1b00 msgr2=0x7f15400c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15400b1b00 0x7f15400c9ed0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1530001460 tx=0x7f1530001ae0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 shutdown_connections 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1520066f30 0x7f15200693e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15400b1b00 0x7f15400c9ed0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1540067dc0 0x7f154013f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 --2- 172.21.15.3:0/1526825354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1540068190 0x7f15400c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 >> 172.21.15.3:0/1526825354 conn(0x7f15400ce270 msgr2=0x7f1540156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 shutdown_connections 2023-10-06T04:58:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.760+0000 7f1545658700 1 -- 172.21.15.3:0/1526825354 wait complete. 2023-10-06T04:58:31.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.896+0000 7fa75054d700 1 Processor -- start 2023-10-06T04:58:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.896+0000 7fa75054d700 1 -- start start 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa75054d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa748153bd0 0x7fa74815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa75054d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa74815a4b0 0x7fa74806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa75054d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74815d9f0 con 0x7fa748153fa0 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa75054d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7480b0aa0 con 0x7fa74815a4b0 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa75054d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74815a980 con 0x7fa748153bd0 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74eaea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa74815a4b0 0x7fa74806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74dae8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa748153fa0 0x7fa748159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50066/0 (socket says 172.21.15.3:50066) 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74dae8700 1 -- 172.21.15.3:0/3918864031 learned_addr learned my addr 172.21.15.3:0/3918864031 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74eaea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa74815a4b0 0x7fa74806b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42748/0 (socket says 172.21.15.3:42748) 2023-10-06T04:58:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74e2e9700 1 --2- 172.21.15.3:0/3918864031 >> v2:172.21.15.153:3300/0 conn(0x7fa748153bd0 0x7fa74815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74eaea700 1 -- 172.21.15.3:0/3918864031 >> v2:172.21.15.153:3300/0 conn(0x7fa748153bd0 msgr2=0x7fa74815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74eaea700 1 --2- 172.21.15.3:0/3918864031 >> v2:172.21.15.153:3300/0 conn(0x7fa748153bd0 0x7fa74815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74eaea700 1 -- 172.21.15.3:0/3918864031 >> v1:172.21.15.3:6789/0 conn(0x7fa748153fa0 legacy=0x7fa748159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74eaea700 1 -- 172.21.15.3:0/3918864031 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74815ac60 con 0x7fa74815a4b0 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74eaea700 1 --2- 172.21.15.3:0/3918864031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa74806b0c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa748070d50 tx=0x7fa738009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddab68acca203ce0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74d2e7700 1 -- 172.21.15.3:0/3918864031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73800e070 con 0x7fa74815a4b0 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74d2e7700 1 -- 172.21.15.3:0/3918864031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73800bc30 con 0x7fa74815a4b0 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.897+0000 7fa74d2e7700 1 -- 172.21.15.3:0/3918864031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738012430 con 0x7fa74815a4b0 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 -- 172.21.15.3:0/3918864031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 msgr2=0x7fa74806b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 --2- 172.21.15.3:0/3918864031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa74806b0c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa748070d50 tx=0x7fa738009780 comp rx=0 tx=0).stop 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 -- 172.21.15.3:0/3918864031 shutdown_connections 2023-10-06T04:58:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 --2- 172.21.15.3:0/3918864031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa74806b0c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 --2- 172.21.15.3:0/3918864031 >> v2:172.21.15.153:3300/0 conn(0x7fa748153bd0 0x7fa74815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 -- 172.21.15.3:0/3918864031 >> 172.21.15.3:0/3918864031 conn(0x7fa7480cdc30 msgr2=0x7fa74806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 -- 172.21.15.3:0/3918864031 shutdown_connections 2023-10-06T04:58:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.898+0000 7fa75054d700 1 -- 172.21.15.3:0/3918864031 wait complete. 2023-10-06T04:58:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.899+0000 7fa75054d700 1 Processor -- start 2023-10-06T04:58:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.899+0000 7fa75054d700 1 -- start start 2023-10-06T04:58:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa75054d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa748153bd0 0x7fa748159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa75054d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa748153fa0 0x7fa748154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa75054d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa748154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa75054d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480aa4f0 con 0x7fa74815a4b0 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa75054d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480a9bc0 con 0x7fa748153bd0 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa75054d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480a9ea0 con 0x7fa748153fa0 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa748154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa748154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42760/0 (socket says 172.21.15.3:42760) 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 -- 172.21.15.3:0/3432277857 learned_addr learned my addr 172.21.15.3:0/3432277857 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74e2e9700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa748153bd0 0x7fa748159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74dae8700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa748153fa0 0x7fa748154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:31.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 -- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa748153fa0 msgr2=0x7fa748154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa748153fa0 0x7fa748154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 -- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa748153bd0 msgr2=0x7fa748159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa748153bd0 0x7fa748159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 -- 172.21.15.3:0/3432277857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738009a20 con 0x7fa74815a4b0 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74dae8700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa748153fa0 0x7fa748154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74e2e9700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa748153bd0 0x7fa748159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.900+0000 7fa74eaea700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa748154990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa738013000 tx=0x7fa738019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.901+0000 7fa7467fc700 1 -- 172.21.15.3:0/3432277857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738012e20 con 0x7fa74815a4b0 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.901+0000 7fa7467fc700 1 -- 172.21.15.3:0/3432277857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73801e630 con 0x7fa74815a4b0 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.901+0000 7fa7467fc700 1 -- 172.21.15.3:0/3432277857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73801e9d0 con 0x7fa74815a4b0 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.901+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7480aa180 con 0x7fa74815a4b0 2023-10-06T04:58:31.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.901+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74816d550 con 0x7fa74815a4b0 2023-10-06T04:58:31.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.901+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7480aa660 con 0x7fa74815a4b0 2023-10-06T04:58:31.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.902+0000 7fa7467fc700 1 -- 172.21.15.3:0/3432277857 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa738024070 con 0x7fa74815a4b0 2023-10-06T04:58:31.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.902+0000 7fa7467fc700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa734066f30 0x7fa7340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:31.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.902+0000 7fa7467fc700 1 -- 172.21.15.3:0/3432277857 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa738086de0 con 0x7fa74815a4b0 2023-10-06T04:58:31.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.903+0000 7fa74e2e9700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa734066f30 0x7fa7340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:31.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.903+0000 7fa74e2e9700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa734066f30 0x7fa7340693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa740006fd0 tx=0x7fa740008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:31.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:31.907+0000 7fa7467fc700 1 -- 172.21.15.3:0/3432277857 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa738053ef0 con 0x7fa74815a4b0 2023-10-06T04:58:32.043 INFO:tasks.ceph.mds.d.smithi153.stderr:2023-10-06T04:58:32.041+0000 7f72fcf62700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:58:32.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.144+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa748071680 con 0x7fa74815a4b0 2023-10-06T04:58:32.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.145+0000 7fa7467fc700 1 -- 172.21.15.3:0/3432277857 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fa73802c090 con 0x7fa74815a4b0 2023-10-06T04:58:32.147 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:32.147 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:58:28.114721+0000","modified":"2023-10-06T04:58:31.748997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.153:6835/2492546877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2492546877},{"type":"v1","addr":"172.21.15.153:6835","nonce":2492546877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa734066f30 msgr2=0x7fa7340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa734066f30 0x7fa7340693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa740006fd0 tx=0x7fa740008040 comp rx=0 tx=0).stop 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 msgr2=0x7fa748154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa748154990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa738013000 tx=0x7fa738019000 comp rx=0 tx=0).stop 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 shutdown_connections 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa734066f30 0x7fa7340693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa748153bd0 0x7fa748159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74815a4b0 0x7fa748154990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 --2- 172.21.15.3:0/3432277857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa748153fa0 0x7fa748154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 >> 172.21.15.3:0/3432277857 conn(0x7fa7480cdc30 msgr2=0x7fa748150830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:32.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 shutdown_connections 2023-10-06T04:58:32.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.147+0000 7fa75054d700 1 -- 172.21.15.3:0/3432277857 wait complete. 2023-10-06T04:58:32.150 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 50 2023-10-06T04:58:32.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:32.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.279+0000 7f3765041700 1 Processor -- start 2023-10-06T04:58:32.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.279+0000 7f3765041700 1 -- start start 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f3765041700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37600cb440 0x7f3760164510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f3765041700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37600cc5c0 con 0x7f37601598d0 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f3765041700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37600cb9b0 con 0x7f3760159500 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f3765041700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37600cda60 con 0x7f37600cb440 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37600cb440 0x7f3760164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f37601598d0 0x7f37600b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33448/0 (socket says 172.21.15.3:33448) 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375e59c700 1 -- 172.21.15.3:0/984463116 learned_addr learned my addr 172.21.15.3:0/984463116 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:32.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375f59e700 1 -- 172.21.15.3:0/984463116 >> v1:172.21.15.153:6789/0 conn(0x7f3760159500 legacy=0x7f37600cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375f59e700 1 -- 172.21.15.3:0/984463116 >> v1:172.21.15.3:6789/0 conn(0x7f37601598d0 legacy=0x7f37600b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375f59e700 1 -- 172.21.15.3:0/984463116 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37600cbc90 con 0x7f37600cb440 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375dd9b700 1 -- 172.21.15.3:0/984463116 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522524208 0 0) 0x7f37600cc5c0 con 0x7f37601598d0 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.280+0000 7f375f59e700 1 --2- 172.21.15.3:0/984463116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37600cb440 0x7f3760164510 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f37500098e0 tx=0x7f37500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74085529ce00a70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f375dd9b700 1 -- 172.21.15.3:0/984463116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750009e80 con 0x7f37600cb440 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f375dd9b700 1 -- 172.21.15.3:0/984463116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375000b820 con 0x7f37600cb440 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f375dd9b700 1 -- 172.21.15.3:0/984463116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375000bba0 con 0x7f37600cb440 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f3765041700 1 -- 172.21.15.3:0/984463116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37600cb440 msgr2=0x7f3760164510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f3765041700 1 --2- 172.21.15.3:0/984463116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37600cb440 0x7f3760164510 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f37500098e0 tx=0x7f37500091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f3765041700 1 -- 172.21.15.3:0/984463116 shutdown_connections 2023-10-06T04:58:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f3765041700 1 --2- 172.21.15.3:0/984463116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37600cb440 0x7f3760164510 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f3765041700 1 -- 172.21.15.3:0/984463116 >> 172.21.15.3:0/984463116 conn(0x7f3760070310 msgr2=0x7f3760070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f3765041700 1 -- 172.21.15.3:0/984463116 shutdown_connections 2023-10-06T04:58:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.281+0000 7f3765041700 1 -- 172.21.15.3:0/984463116 wait complete. 2023-10-06T04:58:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.282+0000 7f3765041700 1 Processor -- start 2023-10-06T04:58:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.282+0000 7f3765041700 1 -- start start 2023-10-06T04:58:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f3765041700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37600cb440 0x7f376014f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f3765041700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3760159500 0x7f376015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f3765041700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37601598d0 0x7f376015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f3765041700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37600b3410 con 0x7f3760159500 2023-10-06T04:58:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f3765041700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3760169ba0 con 0x7f37600cb440 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f3765041700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376015c2f0 con 0x7f37601598d0 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f375e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3760159500 0x7f376015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f375e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3760159500 0x7f376015b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51198/0 (socket says 172.21.15.3:51198) 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f375e59c700 1 -- 172.21.15.3:0/2477892208 learned_addr learned my addr 172.21.15.3:0/2477892208 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f375f59e700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37601598d0 0x7f376015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.283+0000 7f375ed9d700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37600cb440 0x7f376014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375e59c700 1 -- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37601598d0 msgr2=0x7f376015bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375e59c700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37601598d0 0x7f376015bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375e59c700 1 -- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37600cb440 msgr2=0x7f376014f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375e59c700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37600cb440 0x7f376014f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375e59c700 1 -- 172.21.15.3:0/2477892208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3750009420 con 0x7f3760159500 2023-10-06T04:58:32.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375f59e700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37601598d0 0x7f376015bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375ed9d700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37600cb440 0x7f376014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f375e59c700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3760159500 0x7f376015b780 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f374800c5b0 tx=0x7f3748001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f3756ffd700 1 -- 172.21.15.3:0/2477892208 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748005b30 con 0x7f3760159500 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f3756ffd700 1 -- 172.21.15.3:0/2477892208 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3748005c90 con 0x7f3760159500 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376015c5d0 con 0x7f3760159500 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f3756ffd700 1 -- 172.21.15.3:0/2477892208 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748007300 con 0x7f3760159500 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.284+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37600cf760 con 0x7f3760159500 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.285+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37600b35c0 con 0x7f3760159500 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.285+0000 7f3756ffd700 1 -- 172.21.15.3:0/2477892208 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f374801b3b0 con 0x7f3760159500 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.286+0000 7f3756ffd700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3740066f30 0x7f37400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.286+0000 7f3756ffd700 1 -- 172.21.15.3:0/2477892208 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3748083bd0 con 0x7f3760159500 2023-10-06T04:58:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.286+0000 7f375ed9d700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3740066f30 0x7f37400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.286+0000 7f375ed9d700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3740066f30 0x7f37400693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3758007fd0 tx=0x7f3758009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.290+0000 7f3756ffd700 1 -- 172.21.15.3:0/2477892208 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37600b35c0 con 0x7f3760159500 2023-10-06T04:58:32.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.488+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3760071680 con 0x7f3760159500 2023-10-06T04:58:32.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.489+0000 7f3756ffd700 1 -- 172.21.15.3:0/2477892208 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f3748017090 con 0x7f3760159500 2023-10-06T04:58:32.491 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:32.491 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:58:29.742286+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:58:25.874792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T04:58:27.107107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:32.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3740066f30 msgr2=0x7f37400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3740066f30 0x7f37400693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3758007fd0 tx=0x7f3758009040 comp rx=0 tx=0).stop 2023-10-06T04:58:32.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3760159500 msgr2=0x7f376015b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3760159500 0x7f376015b780 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f374800c5b0 tx=0x7f3748001070 comp rx=0 tx=0).stop 2023-10-06T04:58:32.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 shutdown_connections 2023-10-06T04:58:32.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3740066f30 0x7f37400693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37600cb440 0x7f376014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37601598d0 0x7f376015bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 --2- 172.21.15.3:0/2477892208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3760159500 0x7f376015b780 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 >> 172.21.15.3:0/2477892208 conn(0x7f3760070310 msgr2=0x7f3760150780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:32.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 shutdown_connections 2023-10-06T04:58:32.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.491+0000 7f3765041700 1 -- 172.21.15.3:0/2477892208 wait complete. 2023-10-06T04:58:32.506 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:58:32.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.623+0000 7fd045092700 1 Processor -- start 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.623+0000 7fd045092700 1 -- start start 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.624+0000 7fd045092700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0400b39d0 0x7fd0400cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.624+0000 7fd045092700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04015e4c0 con 0x7fd04006d480 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.624+0000 7fd045092700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04015d8b0 con 0x7fd0400cde10 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.624+0000 7fd045092700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04015db90 con 0x7fd0400b39d0 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.624+0000 7fd03ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0400b39d0 0x7fd0400cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.624+0000 7fd03f59e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd04006d480 0x7fd04015a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33452/0 (socket says 172.21.15.3:33452) 2023-10-06T04:58:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.624+0000 7fd03f59e700 1 -- 172.21.15.3:0/2824836102 learned_addr learned my addr 172.21.15.3:0/2824836102 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd03ed9d700 1 -- 172.21.15.3:0/2824836102 >> v1:172.21.15.153:6789/0 conn(0x7fd0400cde10 legacy=0x7fd0401571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd03ed9d700 1 -- 172.21.15.3:0/2824836102 >> v1:172.21.15.3:6789/0 conn(0x7fd04006d480 legacy=0x7fd04015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd03ed9d700 1 -- 172.21.15.3:0/2824836102 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04015de70 con 0x7fd0400b39d0 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd03ed9d700 1 --2- 172.21.15.3:0/2824836102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b39d0 0x7fd0400cd8d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd0300098e0 tx=0x7fd0300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af76fac01ab77cac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd03dd9b700 1 -- 172.21.15.3:0/2824836102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030009e80 con 0x7fd0400b39d0 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd03dd9b700 1 -- 172.21.15.3:0/2824836102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03000b820 con 0x7fd0400b39d0 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd03dd9b700 1 -- 172.21.15.3:0/2824836102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000bba0 con 0x7fd0400b39d0 2023-10-06T04:58:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd045092700 1 -- 172.21.15.3:0/2824836102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b39d0 msgr2=0x7fd0400cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd045092700 1 --2- 172.21.15.3:0/2824836102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b39d0 0x7fd0400cd8d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd0300098e0 tx=0x7fd0300091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd045092700 1 -- 172.21.15.3:0/2824836102 shutdown_connections 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd045092700 1 --2- 172.21.15.3:0/2824836102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b39d0 0x7fd0400cd8d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd045092700 1 -- 172.21.15.3:0/2824836102 >> 172.21.15.3:0/2824836102 conn(0x7fd0400ce2c0 msgr2=0x7fd0400ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd045092700 1 -- 172.21.15.3:0/2824836102 shutdown_connections 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.625+0000 7fd045092700 1 -- 172.21.15.3:0/2824836102 wait complete. 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 Processor -- start 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 -- start start 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd04006d480 0x7fd04015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0400b39d0 0x7fd04015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0400cde10 0x7fd040150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400ccf80 con 0x7fd04006d480 2023-10-06T04:58:32.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04015ac60 con 0x7fd0400cde10 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd045092700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd040154430 con 0x7fd0400b39d0 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd03ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd04006d480 0x7fd04015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd03ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd04006d480 0x7fd04015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51212/0 (socket says 172.21.15.3:51212) 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.627+0000 7fd03ed9d700 1 -- 172.21.15.3:0/3673559596 learned_addr learned my addr 172.21.15.3:0/3673559596 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03f59e700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0400cde10 0x7fd040150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03e59c700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0400b39d0 0x7fd04015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03ed9d700 1 -- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0400b39d0 msgr2=0x7fd04015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03ed9d700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0400b39d0 0x7fd04015c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03ed9d700 1 -- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0400cde10 msgr2=0x7fd040150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03ed9d700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0400cde10 0x7fd040150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03ed9d700 1 -- 172.21.15.3:0/3673559596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030009420 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03f59e700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0400cde10 0x7fd040150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03e59c700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0400b39d0 0x7fd04015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd03ed9d700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd04006d480 0x7fd04015a3a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd0300030c0 tx=0x7fd030016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd036ffd700 1 -- 172.21.15.3:0/3673559596 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03001b470 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd036ffd700 1 -- 172.21.15.3:0/3673559596 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03001b5f0 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04015b470 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd036ffd700 1 -- 172.21.15.3:0/3673559596 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03001b9b0 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.628+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04015b190 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.629+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0400644c0 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.629+0000 7fd036ffd700 1 -- 172.21.15.3:0/3673559596 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd03001c450 con 0x7fd04006d480 2023-10-06T04:58:32.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.630+0000 7fd036ffd700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd020066f30 0x7fd0200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:32.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.630+0000 7fd036ffd700 1 -- 172.21.15.3:0/3673559596 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd030086440 con 0x7fd04006d480 2023-10-06T04:58:32.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.630+0000 7fd03e59c700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd020066f30 0x7fd0200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:32.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.631+0000 7fd03e59c700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd020066f30 0x7fd0200693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd028006fd0 tx=0x7fd028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:32.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.634+0000 7fd036ffd700 1 -- 172.21.15.3:0/3673559596 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd030053550 con 0x7fd04006d480 2023-10-06T04:58:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.823+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd040071680 con 0x7fd020066f30 2023-10-06T04:58:32.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.833+0000 7fd036ffd700 1 -- 172.21.15.3:0/3673559596 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311703 (secure 0 0 0) 0x7fd040071680 con 0x7fd020066f30 2023-10-06T04:58:32.839 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd020066f30 msgr2=0x7fd0200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd020066f30 0x7fd0200693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd028006fd0 tx=0x7fd028008040 comp rx=0 tx=0).stop 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd04006d480 msgr2=0x7fd04015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd04006d480 0x7fd04015a3a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd0300030c0 tx=0x7fd030016000 comp rx=0 tx=0).stop 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 shutdown_connections 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd020066f30 0x7fd0200693e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0400cde10 0x7fd040150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd04006d480 0x7fd04015a3a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 --2- 172.21.15.3:0/3673559596 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0400b39d0 0x7fd04015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 >> 172.21.15.3:0/3673559596 conn(0x7fd0400ce2c0 msgr2=0x7fd0400b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 shutdown_connections 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:32.836+0000 7fd045092700 1 -- 172.21.15.3:0/3673559596 wait complete. 2023-10-06T04:58:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:58:32.849 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2023-10-06T04:58:31.515376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444900,"kb_used_data":9252,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752529820,"statfs":{"total":773094113280,"available":770590535680,"internally_reserved":0,"allocated":9474048,"data_stored":4874791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.806996"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760714+0000","last_change":"2023-10-06T04:58:28.757439+0000","last_active":"2023-10-06T04:58:28.760714+0000","last_peered":"2023-10-06T04:58:28.760714+0000","last_clean":"2023-10-06T04:58:28.760714+0000","last_became_active":"2023-10-06T04:58:28.757151+0000","last_became_peered":"2023-10-06T04:58:28.757151+0000","last_unstale":"2023-10-06T04:58:28.760714+0000","last_undegraded":"2023-10-06T04:58:28.760714+0000","last_fullsized":"2023-10-06T04:58:28.760714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:21.546515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761450+0000","last_change":"2023-10-06T04:58:28.760332+0000","last_active":"2023-10-06T04:58:28.761450+0000","last_peered":"2023-10-06T04:58:28.761450+0000","last_clean":"2023-10-06T04:58:28.761450+0000","last_became_active":"2023-10-06T04:58:28.760033+0000","last_became_peered":"2023-10-06T04:58:28.760033+0000","last_unstale":"2023-10-06T04:58:28.761450+0000","last_undegraded":"2023-10-06T04:58:28.761450+0000","last_fullsized":"2023-10-06T04:58:28.761450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:24.783070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758128+0000","last_change":"2023-10-06T04:58:28.754493+0000","last_active":"2023-10-06T04:58:28.758128+0000","last_peered":"2023-10-06T04:58:28.758128+0000","last_clean":"2023-10-06T04:58:28.758128+0000","last_became_active":"2023-10-06T04:58:28.754180+0000","last_became_peered":"2023-10-06T04:58:28.754180+0000","last_unstale":"2023-10-06T04:58:28.758128+0000","last_undegraded":"2023-10-06T04:58:28.758128+0000","last_fullsized":"2023-10-06T04:58:28.758128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.274854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766115+0000","last_change":"2023-10-06T04:58:28.759360+0000","last_active":"2023-10-06T04:58:28.766115+0000","last_peered":"2023-10-06T04:58:28.766115+0000","last_clean":"2023-10-06T04:58:28.766115+0000","last_became_active":"2023-10-06T04:58:28.759091+0000","last_became_peered":"2023-10-06T04:58:28.759091+0000","last_unstale":"2023-10-06T04:58:28.766115+0000","last_undegraded":"2023-10-06T04:58:28.766115+0000","last_fullsized":"2023-10-06T04:58:28.766115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:53.721225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762611+0000","last_change":"2023-10-06T04:58:28.760226+0000","last_active":"2023-10-06T04:58:28.762611+0000","last_peered":"2023-10-06T04:58:28.762611+0000","last_clean":"2023-10-06T04:58:28.762611+0000","last_became_active":"2023-10-06T04:58:28.759424+0000","last_became_peered":"2023-10-06T04:58:28.759424+0000","last_unstale":"2023-10-06T04:58:28.762611+0000","last_undegraded":"2023-10-06T04:58:28.762611+0000","last_fullsized":"2023-10-06T04:58:28.762611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:12.971917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.759817+0000","last_change":"2023-10-06T04:58:28.752846+0000","last_active":"2023-10-06T04:58:28.759817+0000","last_peered":"2023-10-06T04:58:28.759817+0000","last_clean":"2023-10-06T04:58:28.759817+0000","last_became_active":"2023-10-06T04:58:28.752373+0000","last_became_peered":"2023-10-06T04:58:28.752373+0000","last_unstale":"2023-10-06T04:58:28.759817+0000","last_undegraded":"2023-10-06T04:58:28.759817+0000","last_fullsized":"2023-10-06T04:58:28.759817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:23.907805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758369+0000","last_change":"2023-10-06T04:58:28.755509+0000","last_active":"2023-10-06T04:58:28.758369+0000","last_peered":"2023-10-06T04:58:28.758369+0000","last_clean":"2023-10-06T04:58:28.758369+0000","last_became_active":"2023-10-06T04:58:28.755199+0000","last_became_peered":"2023-10-06T04:58:28.755199+0000","last_unstale":"2023-10-06T04:58:28.758369+0000","last_undegraded":"2023-10-06T04:58:28.758369+0000","last_fullsized":"2023-10-06T04:58:28.758369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.274854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756818+0000","last_change":"2023-10-06T04:58:28.754092+0000","last_active":"2023-10-06T04:58:28.756818+0000","last_peered":"2023-10-06T04:58:28.756818+0000","last_clean":"2023-10-06T04:58:28.756818+0000","last_became_active":"2023-10-06T04:58:28.753789+0000","last_became_peered":"2023-10-06T04:58:28.753789+0000","last_unstale":"2023-10-06T04:58:28.756818+0000","last_undegraded":"2023-10-06T04:58:28.756818+0000","last_fullsized":"2023-10-06T04:58:28.756818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:35.605421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.759471+0000","last_change":"2023-10-06T04:58:28.752526+0000","last_active":"2023-10-06T04:58:28.759471+0000","last_peered":"2023-10-06T04:58:28.759471+0000","last_clean":"2023-10-06T04:58:28.759471+0000","last_became_active":"2023-10-06T04:58:28.752079+0000","last_became_peered":"2023-10-06T04:58:28.752079+0000","last_unstale":"2023-10-06T04:58:28.759471+0000","last_undegraded":"2023-10-06T04:58:28.759471+0000","last_fullsized":"2023-10-06T04:58:28.759471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:43.173543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755297+0000","last_change":"2023-10-06T04:58:28.751826+0000","last_active":"2023-10-06T04:58:28.755297+0000","last_peered":"2023-10-06T04:58:28.755297+0000","last_clean":"2023-10-06T04:58:28.755297+0000","last_became_active":"2023-10-06T04:58:28.751544+0000","last_became_peered":"2023-10-06T04:58:28.751544+0000","last_unstale":"2023-10-06T04:58:28.755297+0000","last_undegraded":"2023-10-06T04:58:28.755297+0000","last_fullsized":"2023-10-06T04:58:28.755297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:59.926203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761549+0000","last_change":"2023-10-06T04:58:28.755845+0000","last_active":"2023-10-06T04:58:28.761549+0000","last_peered":"2023-10-06T04:58:28.761549+0000","last_clean":"2023-10-06T04:58:28.761549+0000","last_became_active":"2023-10-06T04:58:28.755578+0000","last_became_peered":"2023-10-06T04:58:28.755578+0000","last_unstale":"2023-10-06T04:58:28.761549+0000","last_undegraded":"2023-10-06T04:58:28.761549+0000","last_fullsized":"2023-10-06T04:58:28.761549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:15.942873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758910+0000","last_change":"2023-10-06T04:58:28.757814+0000","last_active":"2023-10-06T04:58:28.758910+0000","last_peered":"2023-10-06T04:58:28.758910+0000","last_clean":"2023-10-06T04:58:28.758910+0000","last_became_active":"2023-10-06T04:58:28.757510+0000","last_became_peered":"2023-10-06T04:58:28.757510+0000","last_unstale":"2023-10-06T04:58:28.758910+0000","last_undegraded":"2023-10-06T04:58:28.758910+0000","last_fullsized":"2023-10-06T04:58:28.758910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:44.774012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756595+0000","last_change":"2023-10-06T04:58:28.753896+0000","last_active":"2023-10-06T04:58:28.756595+0000","last_peered":"2023-10-06T04:58:28.756595+0000","last_clean":"2023-10-06T04:58:28.756595+0000","last_became_active":"2023-10-06T04:58:28.753527+0000","last_became_peered":"2023-10-06T04:58:28.753527+0000","last_unstale":"2023-10-06T04:58:28.756595+0000","last_undegraded":"2023-10-06T04:58:28.756595+0000","last_fullsized":"2023-10-06T04:58:28.756595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:48.041072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762642+0000","last_change":"2023-10-06T04:58:28.760990+0000","last_active":"2023-10-06T04:58:28.762642+0000","last_peered":"2023-10-06T04:58:28.762642+0000","last_clean":"2023-10-06T04:58:28.762642+0000","last_became_active":"2023-10-06T04:58:28.760739+0000","last_became_peered":"2023-10-06T04:58:28.760739+0000","last_unstale":"2023-10-06T04:58:28.762642+0000","last_undegraded":"2023-10-06T04:58:28.762642+0000","last_fullsized":"2023-10-06T04:58:28.762642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:08.664175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766394+0000","last_change":"2023-10-06T04:58:28.761832+0000","last_active":"2023-10-06T04:58:28.766394+0000","last_peered":"2023-10-06T04:58:28.766394+0000","last_clean":"2023-10-06T04:58:28.766394+0000","last_became_active":"2023-10-06T04:58:28.760193+0000","last_became_peered":"2023-10-06T04:58:28.760193+0000","last_unstale":"2023-10-06T04:58:28.766394+0000","last_undegraded":"2023-10-06T04:58:28.766394+0000","last_fullsized":"2023-10-06T04:58:28.766394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:26.325538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761828+0000","last_change":"2023-10-06T04:58:28.754608+0000","last_active":"2023-10-06T04:58:28.761828+0000","last_peered":"2023-10-06T04:58:28.761828+0000","last_clean":"2023-10-06T04:58:28.761828+0000","last_became_active":"2023-10-06T04:58:28.754323+0000","last_became_peered":"2023-10-06T04:58:28.754323+0000","last_unstale":"2023-10-06T04:58:28.761828+0000","last_undegraded":"2023-10-06T04:58:28.761828+0000","last_fullsized":"2023-10-06T04:58:28.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:49.921914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.765128+0000","last_change":"2023-10-06T04:58:28.759939+0000","last_active":"2023-10-06T04:58:28.765128+0000","last_peered":"2023-10-06T04:58:28.765128+0000","last_clean":"2023-10-06T04:58:28.765128+0000","last_became_active":"2023-10-06T04:58:28.759440+0000","last_became_peered":"2023-10-06T04:58:28.759440+0000","last_unstale":"2023-10-06T04:58:28.765128+0000","last_undegraded":"2023-10-06T04:58:28.765128+0000","last_fullsized":"2023-10-06T04:58:28.765128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:17.753584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758167+0000","last_change":"2023-10-06T04:58:28.751501+0000","last_active":"2023-10-06T04:58:28.758167+0000","last_peered":"2023-10-06T04:58:28.758167+0000","last_clean":"2023-10-06T04:58:28.758167+0000","last_became_active":"2023-10-06T04:58:28.751208+0000","last_became_peered":"2023-10-06T04:58:28.751208+0000","last_unstale":"2023-10-06T04:58:28.758167+0000","last_undegraded":"2023-10-06T04:58:28.758167+0000","last_fullsized":"2023-10-06T04:58:28.758167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:28.360451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762598+0000","last_change":"2023-10-06T04:58:28.759856+0000","last_active":"2023-10-06T04:58:28.762598+0000","last_peered":"2023-10-06T04:58:28.762598+0000","last_clean":"2023-10-06T04:58:28.762598+0000","last_became_active":"2023-10-06T04:58:28.759574+0000","last_became_peered":"2023-10-06T04:58:28.759574+0000","last_unstale":"2023-10-06T04:58:28.762598+0000","last_undegraded":"2023-10-06T04:58:28.762598+0000","last_fullsized":"2023-10-06T04:58:28.762598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:15.912230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766179+0000","last_change":"2023-10-06T04:58:28.760834+0000","last_active":"2023-10-06T04:58:28.766179+0000","last_peered":"2023-10-06T04:58:28.766179+0000","last_clean":"2023-10-06T04:58:28.766179+0000","last_became_active":"2023-10-06T04:58:28.760406+0000","last_became_peered":"2023-10-06T04:58:28.760406+0000","last_unstale":"2023-10-06T04:58:28.766179+0000","last_undegraded":"2023-10-06T04:58:28.766179+0000","last_fullsized":"2023-10-06T04:58:28.766179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:17.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.753933+0000","last_change":"2023-10-06T04:58:28.750495+0000","last_active":"2023-10-06T04:58:28.753933+0000","last_peered":"2023-10-06T04:58:28.753933+0000","last_clean":"2023-10-06T04:58:28.753933+0000","last_became_active":"2023-10-06T04:58:28.750205+0000","last_became_peered":"2023-10-06T04:58:28.750205+0000","last_unstale":"2023-10-06T04:58:28.753933+0000","last_undegraded":"2023-10-06T04:58:28.753933+0000","last_fullsized":"2023-10-06T04:58:28.753933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:53.721225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762567+0000","last_change":"2023-10-06T04:58:28.759867+0000","last_active":"2023-10-06T04:58:28.762567+0000","last_peered":"2023-10-06T04:58:28.762567+0000","last_clean":"2023-10-06T04:58:28.762567+0000","last_became_active":"2023-10-06T04:58:28.759584+0000","last_became_peered":"2023-10-06T04:58:28.759584+0000","last_unstale":"2023-10-06T04:58:28.762567+0000","last_undegraded":"2023-10-06T04:58:28.762567+0000","last_fullsized":"2023-10-06T04:58:28.762567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:23.430458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762520+0000","last_change":"2023-10-06T04:58:28.758826+0000","last_active":"2023-10-06T04:58:28.762520+0000","last_peered":"2023-10-06T04:58:28.762520+0000","last_clean":"2023-10-06T04:58:28.762520+0000","last_became_active":"2023-10-06T04:58:28.758492+0000","last_became_peered":"2023-10-06T04:58:28.758492+0000","last_unstale":"2023-10-06T04:58:28.762520+0000","last_undegraded":"2023-10-06T04:58:28.762520+0000","last_fullsized":"2023-10-06T04:58:28.762520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:17.918940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758320+0000","last_change":"2023-10-06T04:58:28.754651+0000","last_active":"2023-10-06T04:58:28.758320+0000","last_peered":"2023-10-06T04:58:28.758320+0000","last_clean":"2023-10-06T04:58:28.758320+0000","last_became_active":"2023-10-06T04:58:28.754377+0000","last_became_peered":"2023-10-06T04:58:28.754377+0000","last_unstale":"2023-10-06T04:58:28.758320+0000","last_undegraded":"2023-10-06T04:58:28.758320+0000","last_fullsized":"2023-10-06T04:58:28.758320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:25.977793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761486+0000","last_change":"2023-10-06T04:58:28.757971+0000","last_active":"2023-10-06T04:58:28.761486+0000","last_peered":"2023-10-06T04:58:28.761486+0000","last_clean":"2023-10-06T04:58:28.761486+0000","last_became_active":"2023-10-06T04:58:28.757704+0000","last_became_peered":"2023-10-06T04:58:28.757704+0000","last_unstale":"2023-10-06T04:58:28.761486+0000","last_undegraded":"2023-10-06T04:58:28.761486+0000","last_fullsized":"2023-10-06T04:58:28.761486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:46.967172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762006+0000","last_change":"2023-10-06T04:58:28.759837+0000","last_active":"2023-10-06T04:58:28.762006+0000","last_peered":"2023-10-06T04:58:28.762006+0000","last_clean":"2023-10-06T04:58:28.762006+0000","last_became_active":"2023-10-06T04:58:28.759500+0000","last_became_peered":"2023-10-06T04:58:28.759500+0000","last_unstale":"2023-10-06T04:58:28.762006+0000","last_undegraded":"2023-10-06T04:58:28.762006+0000","last_fullsized":"2023-10-06T04:58:28.762006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.931049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757967+0000","last_change":"2023-10-06T04:58:28.752572+0000","last_active":"2023-10-06T04:58:28.757967+0000","last_peered":"2023-10-06T04:58:28.757967+0000","last_clean":"2023-10-06T04:58:28.757967+0000","last_became_active":"2023-10-06T04:58:28.752224+0000","last_became_peered":"2023-10-06T04:58:28.752224+0000","last_unstale":"2023-10-06T04:58:28.757967+0000","last_undegraded":"2023-10-06T04:58:28.757967+0000","last_fullsized":"2023-10-06T04:58:28.757967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:18.621699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755095+0000","last_change":"2023-10-06T04:58:28.747777+0000","last_active":"2023-10-06T04:58:28.755095+0000","last_peered":"2023-10-06T04:58:28.755095+0000","last_clean":"2023-10-06T04:58:28.755095+0000","last_became_active":"2023-10-06T04:58:28.747557+0000","last_became_peered":"2023-10-06T04:58:28.747557+0000","last_unstale":"2023-10-06T04:58:28.755095+0000","last_undegraded":"2023-10-06T04:58:28.755095+0000","last_fullsized":"2023-10-06T04:58:28.755095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:30.024992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.751394+0000","last_change":"2023-10-06T04:58:28.748820+0000","last_active":"2023-10-06T04:58:28.751394+0000","last_peered":"2023-10-06T04:58:28.751394+0000","last_clean":"2023-10-06T04:58:28.751394+0000","last_became_active":"2023-10-06T04:58:28.748582+0000","last_became_peered":"2023-10-06T04:58:28.748582+0000","last_unstale":"2023-10-06T04:58:28.751394+0000","last_undegraded":"2023-10-06T04:58:28.751394+0000","last_fullsized":"2023-10-06T04:58:28.751394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:21.546515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.752116+0000","last_change":"2023-10-06T04:58:28.747880+0000","last_active":"2023-10-06T04:58:28.752116+0000","last_peered":"2023-10-06T04:58:28.752116+0000","last_clean":"2023-10-06T04:58:28.752116+0000","last_became_active":"2023-10-06T04:58:28.747601+0000","last_became_peered":"2023-10-06T04:58:28.747601+0000","last_unstale":"2023-10-06T04:58:28.752116+0000","last_undegraded":"2023-10-06T04:58:28.752116+0000","last_fullsized":"2023-10-06T04:58:28.752116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:59.336531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767224+0000","last_change":"2023-10-06T04:58:28.766782+0000","last_active":"2023-10-06T04:58:28.767224+0000","last_peered":"2023-10-06T04:58:28.767224+0000","last_clean":"2023-10-06T04:58:28.767224+0000","last_became_active":"2023-10-06T04:58:28.766504+0000","last_became_peered":"2023-10-06T04:58:28.766504+0000","last_unstale":"2023-10-06T04:58:28.767224+0000","last_undegraded":"2023-10-06T04:58:28.767224+0000","last_fullsized":"2023-10-06T04:58:28.767224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:53.721225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756145+0000","last_change":"2023-10-06T04:58:28.748056+0000","last_active":"2023-10-06T04:58:28.756145+0000","last_peered":"2023-10-06T04:58:28.756145+0000","last_clean":"2023-10-06T04:58:28.756145+0000","last_became_active":"2023-10-06T04:58:28.747782+0000","last_became_peered":"2023-10-06T04:58:28.747782+0000","last_unstale":"2023-10-06T04:58:28.756145+0000","last_undegraded":"2023-10-06T04:58:28.756145+0000","last_fullsized":"2023-10-06T04:58:28.756145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:40.295631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757864+0000","last_change":"2023-10-06T04:58:28.748339+0000","last_active":"2023-10-06T04:58:28.757864+0000","last_peered":"2023-10-06T04:58:28.757864+0000","last_clean":"2023-10-06T04:58:28.757864+0000","last_became_active":"2023-10-06T04:58:28.747917+0000","last_became_peered":"2023-10-06T04:58:28.747917+0000","last_unstale":"2023-10-06T04:58:28.757864+0000","last_undegraded":"2023-10-06T04:58:28.757864+0000","last_fullsized":"2023-10-06T04:58:28.757864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:30.678346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"83'4","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.769666+0000","last_change":"2023-10-06T04:58:27.731642+0000","last_active":"2023-10-06T04:58:28.769666+0000","last_peered":"2023-10-06T04:58:28.769666+0000","last_clean":"2023-10-06T04:58:28.769666+0000","last_became_active":"2023-10-06T04:58:27.731213+0000","last_became_peered":"2023-10-06T04:58:27.731213+0000","last_unstale":"2023-10-06T04:58:28.769666+0000","last_undegraded":"2023-10-06T04:58:28.769666+0000","last_fullsized":"2023-10-06T04:58:28.769666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:31.747508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756058+0000","last_change":"2023-10-06T04:58:28.750187+0000","last_active":"2023-10-06T04:58:28.756058+0000","last_peered":"2023-10-06T04:58:28.756058+0000","last_clean":"2023-10-06T04:58:28.756058+0000","last_became_active":"2023-10-06T04:58:28.749946+0000","last_became_peered":"2023-10-06T04:58:28.749946+0000","last_unstale":"2023-10-06T04:58:28.756058+0000","last_undegraded":"2023-10-06T04:58:28.756058+0000","last_fullsized":"2023-10-06T04:58:28.756058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:50:06.145846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761186+0000","last_change":"2023-10-06T04:58:27.728013+0000","last_active":"2023-10-06T04:58:28.761186+0000","last_peered":"2023-10-06T04:58:28.761186+0000","last_clean":"2023-10-06T04:58:28.761186+0000","last_became_active":"2023-10-06T04:58:27.727758+0000","last_became_peered":"2023-10-06T04:58:27.727758+0000","last_unstale":"2023-10-06T04:58:28.761186+0000","last_undegraded":"2023-10-06T04:58:28.761186+0000","last_fullsized":"2023-10-06T04:58:28.761186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:01.177156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758153+0000","last_change":"2023-10-06T04:58:28.754370+0000","last_active":"2023-10-06T04:58:28.758153+0000","last_peered":"2023-10-06T04:58:28.758153+0000","last_clean":"2023-10-06T04:58:28.758153+0000","last_became_active":"2023-10-06T04:58:28.754112+0000","last_became_peered":"2023-10-06T04:58:28.754112+0000","last_unstale":"2023-10-06T04:58:28.758153+0000","last_undegraded":"2023-10-06T04:58:28.758153+0000","last_fullsized":"2023-10-06T04:58:28.758153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:45.917735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.765242+0000","last_change":"2023-10-06T04:58:27.733812+0000","last_active":"2023-10-06T04:58:28.765242+0000","last_peered":"2023-10-06T04:58:28.765242+0000","last_clean":"2023-10-06T04:58:28.765242+0000","last_became_active":"2023-10-06T04:58:27.733571+0000","last_became_peered":"2023-10-06T04:58:27.733571+0000","last_unstale":"2023-10-06T04:58:28.765242+0000","last_undegraded":"2023-10-06T04:58:28.765242+0000","last_fullsized":"2023-10-06T04:58:28.765242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:22.428467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761811+0000","last_change":"2023-10-06T04:58:28.756037+0000","last_active":"2023-10-06T04:58:28.761811+0000","last_peered":"2023-10-06T04:58:28.761811+0000","last_clean":"2023-10-06T04:58:28.761811+0000","last_became_active":"2023-10-06T04:58:28.755717+0000","last_became_peered":"2023-10-06T04:58:28.755717+0000","last_unstale":"2023-10-06T04:58:28.761811+0000","last_undegraded":"2023-10-06T04:58:28.761811+0000","last_fullsized":"2023-10-06T04:58:28.761811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:13.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767899+0000","last_change":"2023-10-06T04:58:27.729480+0000","last_active":"2023-10-06T04:58:28.767899+0000","last_peered":"2023-10-06T04:58:28.767899+0000","last_clean":"2023-10-06T04:58:28.767899+0000","last_became_active":"2023-10-06T04:58:27.729069+0000","last_became_peered":"2023-10-06T04:58:27.729069+0000","last_unstale":"2023-10-06T04:58:28.767899+0000","last_undegraded":"2023-10-06T04:58:28.767899+0000","last_fullsized":"2023-10-06T04:58:28.767899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:05.804921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766257+0000","last_change":"2023-10-06T04:58:28.761386+0000","last_active":"2023-10-06T04:58:28.766257+0000","last_peered":"2023-10-06T04:58:28.766257+0000","last_clean":"2023-10-06T04:58:28.766257+0000","last_became_active":"2023-10-06T04:58:28.760891+0000","last_became_peered":"2023-10-06T04:58:28.760891+0000","last_unstale":"2023-10-06T04:58:28.766257+0000","last_undegraded":"2023-10-06T04:58:28.766257+0000","last_fullsized":"2023-10-06T04:58:28.766257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:31.387712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761300+0000","last_change":"2023-10-06T04:58:27.732864+0000","last_active":"2023-10-06T04:58:28.761300+0000","last_peered":"2023-10-06T04:58:28.761300+0000","last_clean":"2023-10-06T04:58:28.761300+0000","last_became_active":"2023-10-06T04:58:27.732564+0000","last_became_peered":"2023-10-06T04:58:27.732564+0000","last_unstale":"2023-10-06T04:58:28.761300+0000","last_undegraded":"2023-10-06T04:58:28.761300+0000","last_fullsized":"2023-10-06T04:58:28.761300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:11.379057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758695+0000","last_change":"2023-10-06T04:58:28.752398+0000","last_active":"2023-10-06T04:58:28.758695+0000","last_peered":"2023-10-06T04:58:28.758695+0000","last_clean":"2023-10-06T04:58:28.758695+0000","last_became_active":"2023-10-06T04:58:28.752153+0000","last_became_peered":"2023-10-06T04:58:28.752153+0000","last_unstale":"2023-10-06T04:58:28.758695+0000","last_undegraded":"2023-10-06T04:58:28.758695+0000","last_fullsized":"2023-10-06T04:58:28.758695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:17.916940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.771472+0000","last_change":"2023-10-06T04:58:27.729212+0000","last_active":"2023-10-06T04:58:28.771472+0000","last_peered":"2023-10-06T04:58:28.771472+0000","last_clean":"2023-10-06T04:58:28.771472+0000","last_became_active":"2023-10-06T04:58:27.728925+0000","last_became_peered":"2023-10-06T04:58:27.728925+0000","last_unstale":"2023-10-06T04:58:28.771472+0000","last_undegraded":"2023-10-06T04:58:28.771472+0000","last_fullsized":"2023-10-06T04:58:28.771472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:25.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.759797+0000","last_change":"2023-10-06T04:58:28.752647+0000","last_active":"2023-10-06T04:58:28.759797+0000","last_peered":"2023-10-06T04:58:28.759797+0000","last_clean":"2023-10-06T04:58:28.759797+0000","last_became_active":"2023-10-06T04:58:28.751986+0000","last_became_peered":"2023-10-06T04:58:28.751986+0000","last_unstale":"2023-10-06T04:58:28.759797+0000","last_undegraded":"2023-10-06T04:58:28.759797+0000","last_fullsized":"2023-10-06T04:58:28.759797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:21.035597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.748122+0000","last_change":"2023-10-06T04:58:27.724367+0000","last_active":"2023-10-06T04:58:28.748122+0000","last_peered":"2023-10-06T04:58:28.748122+0000","last_clean":"2023-10-06T04:58:28.748122+0000","last_became_active":"2023-10-06T04:58:27.724102+0000","last_became_peered":"2023-10-06T04:58:27.724102+0000","last_unstale":"2023-10-06T04:58:28.748122+0000","last_undegraded":"2023-10-06T04:58:28.748122+0000","last_fullsized":"2023-10-06T04:58:28.748122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.969927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.764833+0000","last_change":"2023-10-06T04:58:28.759297+0000","last_active":"2023-10-06T04:58:28.764833+0000","last_peered":"2023-10-06T04:58:28.764833+0000","last_clean":"2023-10-06T04:58:28.764833+0000","last_became_active":"2023-10-06T04:58:28.759006+0000","last_became_peered":"2023-10-06T04:58:28.759006+0000","last_unstale":"2023-10-06T04:58:28.764833+0000","last_undegraded":"2023-10-06T04:58:28.764833+0000","last_fullsized":"2023-10-06T04:58:28.764833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:46.967172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.738645+0000","last_change":"2023-10-06T04:58:27.731171+0000","last_active":"2023-10-06T04:58:28.738645+0000","last_peered":"2023-10-06T04:58:28.738645+0000","last_clean":"2023-10-06T04:58:28.738645+0000","last_became_active":"2023-10-06T04:58:27.730898+0000","last_became_peered":"2023-10-06T04:58:27.730898+0000","last_unstale":"2023-10-06T04:58:28.738645+0000","last_undegraded":"2023-10-06T04:58:28.738645+0000","last_fullsized":"2023-10-06T04:58:28.738645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:14.910239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761144+0000","last_change":"2023-10-06T04:58:28.756189+0000","last_active":"2023-10-06T04:58:28.761144+0000","last_peered":"2023-10-06T04:58:28.761144+0000","last_clean":"2023-10-06T04:58:28.761144+0000","last_became_active":"2023-10-06T04:58:28.755922+0000","last_became_peered":"2023-10-06T04:58:28.755922+0000","last_unstale":"2023-10-06T04:58:28.761144+0000","last_undegraded":"2023-10-06T04:58:28.761144+0000","last_fullsized":"2023-10-06T04:58:28.761144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:18.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.744098+0000","last_change":"2023-10-06T04:58:27.728085+0000","last_active":"2023-10-06T04:58:28.744098+0000","last_peered":"2023-10-06T04:58:28.744098+0000","last_clean":"2023-10-06T04:58:28.744098+0000","last_became_active":"2023-10-06T04:58:27.727736+0000","last_became_peered":"2023-10-06T04:58:27.727736+0000","last_unstale":"2023-10-06T04:58:28.744098+0000","last_undegraded":"2023-10-06T04:58:28.744098+0000","last_fullsized":"2023-10-06T04:58:28.744098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:12.568372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758106+0000","last_change":"2023-10-06T04:58:28.753631+0000","last_active":"2023-10-06T04:58:28.758106+0000","last_peered":"2023-10-06T04:58:28.758106+0000","last_clean":"2023-10-06T04:58:28.758106+0000","last_became_active":"2023-10-06T04:58:28.753296+0000","last_became_peered":"2023-10-06T04:58:28.753296+0000","last_unstale":"2023-10-06T04:58:28.758106+0000","last_undegraded":"2023-10-06T04:58:28.758106+0000","last_fullsized":"2023-10-06T04:58:28.758106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:25.977793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.744112+0000","last_change":"2023-10-06T04:58:27.723143+0000","last_active":"2023-10-06T04:58:28.744112+0000","last_peered":"2023-10-06T04:58:28.744112+0000","last_clean":"2023-10-06T04:58:28.744112+0000","last_became_active":"2023-10-06T04:58:27.722847+0000","last_became_peered":"2023-10-06T04:58:27.722847+0000","last_unstale":"2023-10-06T04:58:28.744112+0000","last_undegraded":"2023-10-06T04:58:28.744112+0000","last_fullsized":"2023-10-06T04:58:28.744112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:47.252459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758188+0000","last_change":"2023-10-06T04:58:28.755224+0000","last_active":"2023-10-06T04:58:28.758188+0000","last_peered":"2023-10-06T04:58:28.758188+0000","last_clean":"2023-10-06T04:58:28.758188+0000","last_became_active":"2023-10-06T04:58:28.754886+0000","last_became_peered":"2023-10-06T04:58:28.754886+0000","last_unstale":"2023-10-06T04:58:28.758188+0000","last_undegraded":"2023-10-06T04:58:28.758188+0000","last_fullsized":"2023-10-06T04:58:28.758188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:21.608809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"83'3","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.765434+0000","last_change":"2023-10-06T04:58:27.722854+0000","last_active":"2023-10-06T04:58:28.765434+0000","last_peered":"2023-10-06T04:58:28.765434+0000","last_clean":"2023-10-06T04:58:28.765434+0000","last_became_active":"2023-10-06T04:58:27.722608+0000","last_became_peered":"2023-10-06T04:58:27.722608+0000","last_unstale":"2023-10-06T04:58:28.765434+0000","last_undegraded":"2023-10-06T04:58:28.765434+0000","last_fullsized":"2023-10-06T04:58:28.765434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:49.961948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755080+0000","last_change":"2023-10-06T04:58:28.748910+0000","last_active":"2023-10-06T04:58:28.755080+0000","last_peered":"2023-10-06T04:58:28.755080+0000","last_clean":"2023-10-06T04:58:28.755080+0000","last_became_active":"2023-10-06T04:58:28.748622+0000","last_became_peered":"2023-10-06T04:58:28.748622+0000","last_unstale":"2023-10-06T04:58:28.755080+0000","last_undegraded":"2023-10-06T04:58:28.755080+0000","last_fullsized":"2023-10-06T04:58:28.755080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:24.783070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"83'6","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.771542+0000","last_change":"2023-10-06T04:58:27.728654+0000","last_active":"2023-10-06T04:58:28.771542+0000","last_peered":"2023-10-06T04:58:28.771542+0000","last_clean":"2023-10-06T04:58:28.771542+0000","last_became_active":"2023-10-06T04:58:27.728399+0000","last_became_peered":"2023-10-06T04:58:27.728399+0000","last_unstale":"2023-10-06T04:58:28.771542+0000","last_undegraded":"2023-10-06T04:58:28.771542+0000","last_fullsized":"2023-10-06T04:58:28.771542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:42.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756797+0000","last_change":"2023-10-06T04:58:28.753923+0000","last_active":"2023-10-06T04:58:28.756797+0000","last_peered":"2023-10-06T04:58:28.756797+0000","last_clean":"2023-10-06T04:58:28.756797+0000","last_became_active":"2023-10-06T04:58:28.753624+0000","last_became_peered":"2023-10-06T04:58:28.753624+0000","last_unstale":"2023-10-06T04:58:28.756797+0000","last_undegraded":"2023-10-06T04:58:28.756797+0000","last_fullsized":"2023-10-06T04:58:28.756797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:19.297894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762811+0000","last_change":"2023-10-06T04:58:27.731366+0000","last_active":"2023-10-06T04:58:28.762811+0000","last_peered":"2023-10-06T04:58:28.762811+0000","last_clean":"2023-10-06T04:58:28.762811+0000","last_became_active":"2023-10-06T04:58:27.731111+0000","last_became_peered":"2023-10-06T04:58:27.731111+0000","last_unstale":"2023-10-06T04:58:28.762811+0000","last_undegraded":"2023-10-06T04:58:28.762811+0000","last_fullsized":"2023-10-06T04:58:28.762811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:14.940882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762669+0000","last_change":"2023-10-06T04:58:28.761402+0000","last_active":"2023-10-06T04:58:28.762669+0000","last_peered":"2023-10-06T04:58:28.762669+0000","last_clean":"2023-10-06T04:58:28.762669+0000","last_became_active":"2023-10-06T04:58:28.761126+0000","last_became_peered":"2023-10-06T04:58:28.761126+0000","last_unstale":"2023-10-06T04:58:28.762669+0000","last_undegraded":"2023-10-06T04:58:28.762669+0000","last_fullsized":"2023-10-06T04:58:28.762669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:48.254449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737548+0000","last_change":"2023-10-06T04:58:27.718613+0000","last_active":"2023-10-06T04:58:28.737548+0000","last_peered":"2023-10-06T04:58:28.737548+0000","last_clean":"2023-10-06T04:58:28.737548+0000","last_became_active":"2023-10-06T04:58:27.718037+0000","last_became_peered":"2023-10-06T04:58:27.718037+0000","last_unstale":"2023-10-06T04:58:28.737548+0000","last_undegraded":"2023-10-06T04:58:28.737548+0000","last_fullsized":"2023-10-06T04:58:28.737548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:22.428467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755261+0000","last_change":"2023-10-06T04:58:28.751739+0000","last_active":"2023-10-06T04:58:28.755261+0000","last_peered":"2023-10-06T04:58:28.755261+0000","last_clean":"2023-10-06T04:58:28.755261+0000","last_became_active":"2023-10-06T04:58:28.751457+0000","last_became_peered":"2023-10-06T04:58:28.751457+0000","last_unstale":"2023-10-06T04:58:28.755261+0000","last_undegraded":"2023-10-06T04:58:28.755261+0000","last_fullsized":"2023-10-06T04:58:28.755261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:05.122344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.746166+0000","last_change":"2023-10-06T04:58:27.728033+0000","last_active":"2023-10-06T04:58:28.746166+0000","last_peered":"2023-10-06T04:58:28.746166+0000","last_clean":"2023-10-06T04:58:28.746166+0000","last_became_active":"2023-10-06T04:58:27.727773+0000","last_became_peered":"2023-10-06T04:58:27.727773+0000","last_unstale":"2023-10-06T04:58:28.746166+0000","last_undegraded":"2023-10-06T04:58:28.746166+0000","last_fullsized":"2023-10-06T04:58:28.746166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:56.503521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.751355+0000","last_change":"2023-10-06T04:58:28.747125+0000","last_active":"2023-10-06T04:58:28.751355+0000","last_peered":"2023-10-06T04:58:28.751355+0000","last_clean":"2023-10-06T04:58:28.751355+0000","last_became_active":"2023-10-06T04:58:28.746820+0000","last_became_peered":"2023-10-06T04:58:28.746820+0000","last_unstale":"2023-10-06T04:58:28.751355+0000","last_undegraded":"2023-10-06T04:58:28.751355+0000","last_fullsized":"2023-10-06T04:58:28.751355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:49.696705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"83'8","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.774844+0000","last_change":"2023-10-06T04:58:27.729252+0000","last_active":"2023-10-06T04:58:28.774844+0000","last_peered":"2023-10-06T04:58:28.774844+0000","last_clean":"2023-10-06T04:58:28.774844+0000","last_became_active":"2023-10-06T04:58:27.728980+0000","last_became_peered":"2023-10-06T04:58:27.728980+0000","last_unstale":"2023-10-06T04:58:28.774844+0000","last_undegraded":"2023-10-06T04:58:28.774844+0000","last_fullsized":"2023-10-06T04:58:28.774844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:19.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756841+0000","last_change":"2023-10-06T04:58:28.751421+0000","last_active":"2023-10-06T04:58:28.756841+0000","last_peered":"2023-10-06T04:58:28.756841+0000","last_clean":"2023-10-06T04:58:28.756841+0000","last_became_active":"2023-10-06T04:58:28.751131+0000","last_became_peered":"2023-10-06T04:58:28.751131+0000","last_unstale":"2023-10-06T04:58:28.756841+0000","last_undegraded":"2023-10-06T04:58:28.756841+0000","last_fullsized":"2023-10-06T04:58:28.756841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:59.192317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762408+0000","last_change":"2023-10-06T04:58:27.728770+0000","last_active":"2023-10-06T04:58:28.762408+0000","last_peered":"2023-10-06T04:58:28.762408+0000","last_clean":"2023-10-06T04:58:28.762408+0000","last_became_active":"2023-10-06T04:58:27.728409+0000","last_became_peered":"2023-10-06T04:58:27.728409+0000","last_unstale":"2023-10-06T04:58:28.762408+0000","last_undegraded":"2023-10-06T04:58:28.762408+0000","last_fullsized":"2023-10-06T04:58:28.762408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.969927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760076+0000","last_change":"2023-10-06T04:58:28.752172+0000","last_active":"2023-10-06T04:58:28.760076+0000","last_peered":"2023-10-06T04:58:28.760076+0000","last_clean":"2023-10-06T04:58:28.760076+0000","last_became_active":"2023-10-06T04:58:28.751904+0000","last_became_peered":"2023-10-06T04:58:28.751904+0000","last_unstale":"2023-10-06T04:58:28.760076+0000","last_undegraded":"2023-10-06T04:58:28.760076+0000","last_fullsized":"2023-10-06T04:58:28.760076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:25.977793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.744582+0000","last_change":"2023-10-06T04:58:27.723780+0000","last_active":"2023-10-06T04:58:28.744582+0000","last_peered":"2023-10-06T04:58:28.744582+0000","last_clean":"2023-10-06T04:58:28.744582+0000","last_became_active":"2023-10-06T04:58:27.723510+0000","last_became_peered":"2023-10-06T04:58:27.723510+0000","last_unstale":"2023-10-06T04:58:28.744582+0000","last_undegraded":"2023-10-06T04:58:28.744582+0000","last_fullsized":"2023-10-06T04:58:28.744582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:55.908316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758047+0000","last_change":"2023-10-06T04:58:28.747634+0000","last_active":"2023-10-06T04:58:28.758047+0000","last_peered":"2023-10-06T04:58:28.758047+0000","last_clean":"2023-10-06T04:58:28.758047+0000","last_became_active":"2023-10-06T04:58:28.747327+0000","last_became_peered":"2023-10-06T04:58:28.747327+0000","last_unstale":"2023-10-06T04:58:28.758047+0000","last_undegraded":"2023-10-06T04:58:28.758047+0000","last_fullsized":"2023-10-06T04:58:28.758047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:49.382999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.750087+0000","last_change":"2023-10-06T04:58:27.718785+0000","last_active":"2023-10-06T04:58:28.750087+0000","last_peered":"2023-10-06T04:58:28.750087+0000","last_clean":"2023-10-06T04:58:28.750087+0000","last_became_active":"2023-10-06T04:58:27.717713+0000","last_became_peered":"2023-10-06T04:58:27.717713+0000","last_unstale":"2023-10-06T04:58:28.750087+0000","last_undegraded":"2023-10-06T04:58:28.750087+0000","last_fullsized":"2023-10-06T04:58:28.750087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:33.918728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.751642+0000","last_change":"2023-10-06T04:58:28.748505+0000","last_active":"2023-10-06T04:58:28.751642+0000","last_peered":"2023-10-06T04:58:28.751642+0000","last_clean":"2023-10-06T04:58:28.751642+0000","last_became_active":"2023-10-06T04:58:28.748176+0000","last_became_peered":"2023-10-06T04:58:28.748176+0000","last_unstale":"2023-10-06T04:58:28.751642+0000","last_undegraded":"2023-10-06T04:58:28.751642+0000","last_fullsized":"2023-10-06T04:58:28.751642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:26.325538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741779+0000","last_change":"2023-10-06T04:58:27.732758+0000","last_active":"2023-10-06T04:58:28.741779+0000","last_peered":"2023-10-06T04:58:28.741779+0000","last_clean":"2023-10-06T04:58:28.741779+0000","last_became_active":"2023-10-06T04:58:27.732487+0000","last_became_peered":"2023-10-06T04:58:27.732487+0000","last_unstale":"2023-10-06T04:58:28.741779+0000","last_undegraded":"2023-10-06T04:58:28.741779+0000","last_fullsized":"2023-10-06T04:58:28.741779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:42.667103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.754337+0000","last_change":"2023-10-06T04:58:28.750509+0000","last_active":"2023-10-06T04:58:28.754337+0000","last_peered":"2023-10-06T04:58:28.754337+0000","last_clean":"2023-10-06T04:58:28.754337+0000","last_became_active":"2023-10-06T04:58:28.750209+0000","last_became_peered":"2023-10-06T04:58:28.750209+0000","last_unstale":"2023-10-06T04:58:28.754337+0000","last_undegraded":"2023-10-06T04:58:28.754337+0000","last_fullsized":"2023-10-06T04:58:28.754337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:32.883243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.738829+0000","last_change":"2023-10-06T04:58:27.721729+0000","last_active":"2023-10-06T04:58:28.738829+0000","last_peered":"2023-10-06T04:58:28.738829+0000","last_clean":"2023-10-06T04:58:28.738829+0000","last_became_active":"2023-10-06T04:58:27.721436+0000","last_became_peered":"2023-10-06T04:58:27.721436+0000","last_unstale":"2023-10-06T04:58:28.738829+0000","last_undegraded":"2023-10-06T04:58:28.738829+0000","last_fullsized":"2023-10-06T04:58:28.738829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:07.662184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.763679+0000","last_change":"2023-10-06T04:58:28.759782+0000","last_active":"2023-10-06T04:58:28.763679+0000","last_peered":"2023-10-06T04:58:28.763679+0000","last_clean":"2023-10-06T04:58:28.763679+0000","last_became_active":"2023-10-06T04:58:28.759501+0000","last_became_peered":"2023-10-06T04:58:28.759501+0000","last_unstale":"2023-10-06T04:58:28.763679+0000","last_undegraded":"2023-10-06T04:58:28.763679+0000","last_fullsized":"2023-10-06T04:58:28.763679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:30.678346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.742140+0000","last_change":"2023-10-06T04:58:27.722076+0000","last_active":"2023-10-06T04:58:28.742140+0000","last_peered":"2023-10-06T04:58:28.742140+0000","last_clean":"2023-10-06T04:58:28.742140+0000","last_became_active":"2023-10-06T04:58:27.721806+0000","last_became_peered":"2023-10-06T04:58:27.721806+0000","last_unstale":"2023-10-06T04:58:28.742140+0000","last_undegraded":"2023-10-06T04:58:28.742140+0000","last_fullsized":"2023-10-06T04:58:28.742140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:41.996701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757203+0000","last_change":"2023-10-06T04:58:28.753628+0000","last_active":"2023-10-06T04:58:28.757203+0000","last_peered":"2023-10-06T04:58:28.757203+0000","last_clean":"2023-10-06T04:58:28.757203+0000","last_became_active":"2023-10-06T04:58:28.753310+0000","last_became_peered":"2023-10-06T04:58:28.753310+0000","last_unstale":"2023-10-06T04:58:28.757203+0000","last_undegraded":"2023-10-06T04:58:28.757203+0000","last_fullsized":"2023-10-06T04:58:28.757203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:51.801063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.742878+0000","last_change":"2023-10-06T04:58:27.732877+0000","last_active":"2023-10-06T04:58:28.742878+0000","last_peered":"2023-10-06T04:58:28.742878+0000","last_clean":"2023-10-06T04:58:28.742878+0000","last_became_active":"2023-10-06T04:58:27.732464+0000","last_became_peered":"2023-10-06T04:58:27.732464+0000","last_unstale":"2023-10-06T04:58:28.742878+0000","last_undegraded":"2023-10-06T04:58:28.742878+0000","last_fullsized":"2023-10-06T04:58:28.742878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:54.030943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760108+0000","last_change":"2023-10-06T04:58:28.754718+0000","last_active":"2023-10-06T04:58:28.760108+0000","last_peered":"2023-10-06T04:58:28.760108+0000","last_clean":"2023-10-06T04:58:28.760108+0000","last_became_active":"2023-10-06T04:58:28.754430+0000","last_became_peered":"2023-10-06T04:58:28.754430+0000","last_unstale":"2023-10-06T04:58:28.760108+0000","last_undegraded":"2023-10-06T04:58:28.760108+0000","last_fullsized":"2023-10-06T04:58:28.760108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:28.360451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737824+0000","last_change":"2023-10-06T04:58:27.727208+0000","last_active":"2023-10-06T04:58:28.737824+0000","last_peered":"2023-10-06T04:58:28.737824+0000","last_clean":"2023-10-06T04:58:28.737824+0000","last_became_active":"2023-10-06T04:58:27.726965+0000","last_became_peered":"2023-10-06T04:58:27.726965+0000","last_unstale":"2023-10-06T04:58:28.737824+0000","last_undegraded":"2023-10-06T04:58:28.737824+0000","last_fullsized":"2023-10-06T04:58:28.737824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:07.662184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767439+0000","last_change":"2023-10-06T04:58:28.766901+0000","last_active":"2023-10-06T04:58:28.767439+0000","last_peered":"2023-10-06T04:58:28.767439+0000","last_clean":"2023-10-06T04:58:28.767439+0000","last_became_active":"2023-10-06T04:58:28.766547+0000","last_became_peered":"2023-10-06T04:58:28.766547+0000","last_unstale":"2023-10-06T04:58:28.767439+0000","last_undegraded":"2023-10-06T04:58:28.767439+0000","last_fullsized":"2023-10-06T04:58:28.767439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:18.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741592+0000","last_change":"2023-10-06T04:58:27.725736+0000","last_active":"2023-10-06T04:58:28.741592+0000","last_peered":"2023-10-06T04:58:28.741592+0000","last_clean":"2023-10-06T04:58:28.741592+0000","last_became_active":"2023-10-06T04:58:27.725452+0000","last_became_peered":"2023-10-06T04:58:27.725452+0000","last_unstale":"2023-10-06T04:58:28.741592+0000","last_undegraded":"2023-10-06T04:58:28.741592+0000","last_fullsized":"2023-10-06T04:58:28.741592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:01.177156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.749932+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:58:28.749932+0000","last_peered":"2023-10-06T04:58:28.749932+0000","last_clean":"2023-10-06T04:58:28.749932+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:58:28.749932+0000","last_undegraded":"2023-10-06T04:58:28.749932+0000","last_fullsized":"2023-10-06T04:58:28.749932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758529+0000","last_change":"2023-10-06T04:58:28.747961+0000","last_active":"2023-10-06T04:58:28.758529+0000","last_peered":"2023-10-06T04:58:28.758529+0000","last_clean":"2023-10-06T04:58:28.758529+0000","last_became_active":"2023-10-06T04:58:28.747642+0000","last_became_peered":"2023-10-06T04:58:28.747642+0000","last_unstale":"2023-10-06T04:58:28.758529+0000","last_undegraded":"2023-10-06T04:58:28.758529+0000","last_fullsized":"2023-10-06T04:58:28.758529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:31.040866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.740415+0000","last_change":"2023-10-06T04:58:27.725960+0000","last_active":"2023-10-06T04:58:28.740415+0000","last_peered":"2023-10-06T04:58:28.740415+0000","last_clean":"2023-10-06T04:58:28.740415+0000","last_became_active":"2023-10-06T04:58:27.725520+0000","last_became_peered":"2023-10-06T04:58:27.725520+0000","last_unstale":"2023-10-06T04:58:28.740415+0000","last_undegraded":"2023-10-06T04:58:28.740415+0000","last_fullsized":"2023-10-06T04:58:28.740415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:54.181959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.749074+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:58:28.749074+0000","last_peered":"2023-10-06T04:58:28.749074+0000","last_clean":"2023-10-06T04:58:28.749074+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:58:28.749074+0000","last_undegraded":"2023-10-06T04:58:28.749074+0000","last_fullsized":"2023-10-06T04:58:28.749074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757842+0000","last_change":"2023-10-06T04:58:28.748173+0000","last_active":"2023-10-06T04:58:28.757842+0000","last_peered":"2023-10-06T04:58:28.757842+0000","last_clean":"2023-10-06T04:58:28.757842+0000","last_became_active":"2023-10-06T04:58:28.747234+0000","last_became_peered":"2023-10-06T04:58:28.747234+0000","last_unstale":"2023-10-06T04:58:28.757842+0000","last_undegraded":"2023-10-06T04:58:28.757842+0000","last_fullsized":"2023-10-06T04:58:28.757842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:21.546515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.763491+0000","last_change":"2023-10-06T04:58:27.727783+0000","last_active":"2023-10-06T04:58:28.763491+0000","last_peered":"2023-10-06T04:58:28.763491+0000","last_clean":"2023-10-06T04:58:28.763491+0000","last_became_active":"2023-10-06T04:58:27.727473+0000","last_became_peered":"2023-10-06T04:58:27.727473+0000","last_unstale":"2023-10-06T04:58:28.763491+0000","last_undegraded":"2023-10-06T04:58:28.763491+0000","last_fullsized":"2023-10-06T04:58:28.763491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:26.640364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.747317+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:58:28.747317+0000","last_peered":"2023-10-06T04:58:28.747317+0000","last_clean":"2023-10-06T04:58:28.747317+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:58:28.747317+0000","last_undegraded":"2023-10-06T04:58:28.747317+0000","last_fullsized":"2023-10-06T04:58:28.747317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":215,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737937+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:58:28.737937+0000","last_peered":"2023-10-06T04:58:28.737937+0000","last_clean":"2023-10-06T04:58:28.737937+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:58:28.737937+0000","last_undegraded":"2023-10-06T04:58:28.737937+0000","last_fullsized":"2023-10-06T04:58:28.737937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757305+0000","last_change":"2023-10-06T04:58:28.751394+0000","last_active":"2023-10-06T04:58:28.757305+0000","last_peered":"2023-10-06T04:58:28.757305+0000","last_clean":"2023-10-06T04:58:28.757305+0000","last_became_active":"2023-10-06T04:58:28.751148+0000","last_became_peered":"2023-10-06T04:58:28.751148+0000","last_unstale":"2023-10-06T04:58:28.757305+0000","last_undegraded":"2023-10-06T04:58:28.757305+0000","last_fullsized":"2023-10-06T04:58:28.757305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:55.102824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761641+0000","last_change":"2023-10-06T04:58:27.723817+0000","last_active":"2023-10-06T04:58:28.761641+0000","last_peered":"2023-10-06T04:58:28.761641+0000","last_clean":"2023-10-06T04:58:28.761641+0000","last_became_active":"2023-10-06T04:58:27.723533+0000","last_became_peered":"2023-10-06T04:58:27.723533+0000","last_unstale":"2023-10-06T04:58:28.761641+0000","last_undegraded":"2023-10-06T04:58:28.761641+0000","last_fullsized":"2023-10-06T04:58:28.761641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:42.171552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737849+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:58:28.737849+0000","last_peered":"2023-10-06T04:58:28.737849+0000","last_clean":"2023-10-06T04:58:28.737849+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:58:28.737849+0000","last_undegraded":"2023-10-06T04:58:28.737849+0000","last_fullsized":"2023-10-06T04:58:28.737849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760260+0000","last_change":"2023-10-06T04:58:28.759047+0000","last_active":"2023-10-06T04:58:28.760260+0000","last_peered":"2023-10-06T04:58:28.760260+0000","last_clean":"2023-10-06T04:58:28.760260+0000","last_became_active":"2023-10-06T04:58:28.758695+0000","last_became_peered":"2023-10-06T04:58:28.758695+0000","last_unstale":"2023-10-06T04:58:28.760260+0000","last_undegraded":"2023-10-06T04:58:28.760260+0000","last_fullsized":"2023-10-06T04:58:28.760260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:28.360451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.746772+0000","last_change":"2023-10-06T04:58:27.726169+0000","last_active":"2023-10-06T04:58:28.746772+0000","last_peered":"2023-10-06T04:58:28.746772+0000","last_clean":"2023-10-06T04:58:28.746772+0000","last_became_active":"2023-10-06T04:58:27.725663+0000","last_became_peered":"2023-10-06T04:58:27.725663+0000","last_unstale":"2023-10-06T04:58:28.746772+0000","last_undegraded":"2023-10-06T04:58:28.746772+0000","last_fullsized":"2023-10-06T04:58:28.746772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:33.983845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.738021+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:58:28.738021+0000","last_peered":"2023-10-06T04:58:28.738021+0000","last_clean":"2023-10-06T04:58:28.738021+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:58:28.738021+0000","last_undegraded":"2023-10-06T04:58:28.738021+0000","last_fullsized":"2023-10-06T04:58:28.738021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761268+0000","last_change":"2023-10-06T04:58:28.756810+0000","last_active":"2023-10-06T04:58:28.761268+0000","last_peered":"2023-10-06T04:58:28.761268+0000","last_clean":"2023-10-06T04:58:28.761268+0000","last_became_active":"2023-10-06T04:58:28.756551+0000","last_became_peered":"2023-10-06T04:58:28.756551+0000","last_unstale":"2023-10-06T04:58:28.761268+0000","last_undegraded":"2023-10-06T04:58:28.761268+0000","last_fullsized":"2023-10-06T04:58:28.761268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:55.440657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767735+0000","last_change":"2023-10-06T04:58:27.721320+0000","last_active":"2023-10-06T04:58:28.767735+0000","last_peered":"2023-10-06T04:58:28.767735+0000","last_clean":"2023-10-06T04:58:28.767735+0000","last_became_active":"2023-10-06T04:58:27.721062+0000","last_became_peered":"2023-10-06T04:58:27.721062+0000","last_unstale":"2023-10-06T04:58:28.767735+0000","last_undegraded":"2023-10-06T04:58:28.767735+0000","last_fullsized":"2023-10-06T04:58:28.767735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:06.202911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741689+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:58:28.741689+0000","last_peered":"2023-10-06T04:58:28.741689+0000","last_clean":"2023-10-06T04:58:28.741689+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:58:28.741689+0000","last_undegraded":"2023-10-06T04:58:28.741689+0000","last_fullsized":"2023-10-06T04:58:28.741689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761455+0000","last_change":"2023-10-06T04:58:28.760435+0000","last_active":"2023-10-06T04:58:28.761455+0000","last_peered":"2023-10-06T04:58:28.761455+0000","last_clean":"2023-10-06T04:58:28.761455+0000","last_became_active":"2023-10-06T04:58:28.760148+0000","last_became_peered":"2023-10-06T04:58:28.760148+0000","last_unstale":"2023-10-06T04:58:28.761455+0000","last_undegraded":"2023-10-06T04:58:28.761455+0000","last_fullsized":"2023-10-06T04:58:28.761455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:59.336531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.740584+0000","last_change":"2023-10-06T04:58:27.727369+0000","last_active":"2023-10-06T04:58:28.740584+0000","last_peered":"2023-10-06T04:58:28.740584+0000","last_clean":"2023-10-06T04:58:28.740584+0000","last_became_active":"2023-10-06T04:58:27.725301+0000","last_became_peered":"2023-10-06T04:58:27.725301+0000","last_unstale":"2023-10-06T04:58:28.740584+0000","last_undegraded":"2023-10-06T04:58:28.740584+0000","last_fullsized":"2023-10-06T04:58:28.740584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:59.163008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741153+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:58:28.741153+0000","last_peered":"2023-10-06T04:58:28.741153+0000","last_clean":"2023-10-06T04:58:28.741153+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:58:28.741153+0000","last_undegraded":"2023-10-06T04:58:28.741153+0000","last_fullsized":"2023-10-06T04:58:28.741153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760279+0000","last_change":"2023-10-06T04:58:28.752729+0000","last_active":"2023-10-06T04:58:28.760279+0000","last_peered":"2023-10-06T04:58:28.760279+0000","last_clean":"2023-10-06T04:58:28.760279+0000","last_became_active":"2023-10-06T04:58:28.752451+0000","last_became_peered":"2023-10-06T04:58:28.752451+0000","last_unstale":"2023-10-06T04:58:28.760279+0000","last_undegraded":"2023-10-06T04:58:28.760279+0000","last_fullsized":"2023-10-06T04:58:28.760279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:20.677651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.764064+0000","last_change":"2023-10-06T04:58:27.722935+0000","last_active":"2023-10-06T04:58:28.764064+0000","last_peered":"2023-10-06T04:58:28.764064+0000","last_clean":"2023-10-06T04:58:28.764064+0000","last_became_active":"2023-10-06T04:58:27.722662+0000","last_became_peered":"2023-10-06T04:58:27.722662+0000","last_unstale":"2023-10-06T04:58:28.764064+0000","last_undegraded":"2023-10-06T04:58:28.764064+0000","last_fullsized":"2023-10-06T04:58:28.764064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:23.234133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.743398+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:58:28.743398+0000","last_peered":"2023-10-06T04:58:28.743398+0000","last_clean":"2023-10-06T04:58:28.743398+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:58:28.743398+0000","last_undegraded":"2023-10-06T04:58:28.743398+0000","last_fullsized":"2023-10-06T04:58:28.743398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307060,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064780,"statfs":{"total":96636764160,"available":96322334720,"internally_reserved":0,"allocated":1617920,"data_stored":1045778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.73399999999999999},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73199999999999998,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":1.073},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.51600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":1.111},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.58999999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.503},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.41799999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":1.214},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302412,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069428,"statfs":{"total":96636764160,"available":96327094272,"internally_reserved":0,"allocated":1052672,"data_stored":465460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.52200000000000002},"min":{"1min":0.54900000000000004,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.81200000000000006,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.57199999999999995},"min":{"1min":0.56100000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.39800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":1.234,"15min":1.234},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.55200000000000005},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81000000000000005,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75800000000000001,"5min":3.669,"15min":3.669},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.88}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307068,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064772,"statfs":{"total":96636764160,"available":96322326528,"internally_reserved":0,"allocated":1626112,"data_stored":1052433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.23000000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.27900000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.501,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.70999999999999996},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.85499999999999998,"15min":0.92900000000000005},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.64100000000000001},"min":{"1min":0.47499999999999998,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.998,"15min":0.998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.65800000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":1.0900000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80600000000000005,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.026},"last":0.755},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.79400000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306512,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065328,"statfs":{"total":96636764160,"available":96322895872,"internally_reserved":0,"allocated":1056768,"data_stored":465562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.505,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57699999999999996,"5min":0.628,"15min":0.628},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.28899999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.32000000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38900000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90700000000000003,"5min":1.022,"15min":1.022},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.495,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.443}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.5,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96599999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.995,"15min":0.995},"last":0.51100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302392,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069448,"statfs":{"total":96636764160,"available":96327114752,"internally_reserved":0,"allocated":1032192,"data_stored":463348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.51200000000000001,"5min":0.747,"15min":1.048},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.42599999999999999,"15min":0.39000000000000001},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":0.61699999999999999,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.36099999999999999},"min":{"1min":0.29499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.5,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306488,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065352,"statfs":{"total":96636764160,"available":96322920448,"internally_reserved":0,"allocated":1032192,"data_stored":463348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38700000000000001,"15min":0.40600000000000003},"min":{"1min":0.251,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":2.008,"15min":2.008},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41299999999999998,"15min":0.56499999999999995},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.314}]},{"osd":1,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54500000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42399999999999999,"15min":0.46000000000000002},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.64600000000000002,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.443}]},{"osd":2,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":1.379,"15min":5.069},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":2.246,"15min":2.246},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.373}]},{"osd":5,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83199999999999996,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82199999999999995,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.79400000000000004},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.40400000000000003},"max":{"1min":0.76200000000000001,"5min":1.964,"15min":6.367},"last":0.502},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65000000000000002,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":1.3,"15min":5},"last":0.54600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306484,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065356,"statfs":{"total":96636764160,"available":96322924544,"internally_reserved":0,"allocated":1028096,"data_stored":456867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.78500000000000003},"min":{"1min":0.38200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.82099999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.80400000000000005,"15min":5.415},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.66700000000000004},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":4.202},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":1.044,"15min":4.2130000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.77600000000000002,"5min":0.86199999999999999,"15min":0.872},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.313},"max":{"1min":0.84399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.55600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306484,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065356,"statfs":{"total":96636764160,"available":96322924544,"internally_reserved":0,"allocated":1028096,"data_stored":461995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.67600000000000005},"min":{"1min":0.36899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.625},"min":{"1min":0.436,"5min":0.247,"15min":0.247},"max":{"1min":0.76000000000000001,"5min":0.80800000000000005,"15min":3.7679999999999998},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.70799999999999996},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":6.7949999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.75800000000000001},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.93799999999999994},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":9.7210000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.93500000000000005},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":9.7539999999999996},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.504},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45600000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.34200000000000003},"max":{"1min":0.66300000000000003,"5min":0.746,"15min":0.746},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80100000000000005},"last":0.432}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.49399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:58:32.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:58:33.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.008+0000 7f094e65a700 1 Processor -- start 2023-10-06T04:58:33.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f094e65a700 1 -- start start 2023-10-06T04:58:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f094e65a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f09481598d0 0x7f09480b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f094e65a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09480cb440 0x7f09480b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f094e65a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09480cc5c0 con 0x7f0948159500 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f094e65a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09480cda60 con 0x7f09480cb440 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f094e65a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09480b2ca0 con 0x7f09481598d0 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f0947fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0948159500 0x7f09480cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33468/0 (socket says 172.21.15.3:33468) 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f0947fff700 1 -- 172.21.15.3:0/3691843126 learned_addr learned my addr 172.21.15.3:0/3691843126 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.009+0000 7f094cbf7700 1 --2- 172.21.15.3:0/3691843126 >> v2:172.21.15.3:3300/0 conn(0x7f09480cb440 0x7f09480b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f09477fe700 1 --2- 172.21.15.3:0/3691843126 >> v2:172.21.15.153:3300/0 conn(0x7f09481598d0 0x7f09480b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f0946ffd700 1 -- 172.21.15.3:0/3691843126 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404266334 0 0) 0x7f09480cc5c0 con 0x7f0948159500 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f0946ffd700 1 -- 172.21.15.3:0/3691843126 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0930003570 con 0x7f0948159500 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f094cbf7700 1 -- 172.21.15.3:0/3691843126 >> v2:172.21.15.153:3300/0 conn(0x7f09481598d0 msgr2=0x7f09480b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f094cbf7700 1 --2- 172.21.15.3:0/3691843126 >> v2:172.21.15.153:3300/0 conn(0x7f09481598d0 0x7f09480b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f094cbf7700 1 -- 172.21.15.3:0/3691843126 >> v1:172.21.15.3:6789/0 conn(0x7f0948159500 legacy=0x7f09480cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f094cbf7700 1 -- 172.21.15.3:0/3691843126 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480cb9b0 con 0x7f09480cb440 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f094cbf7700 1 --2- 172.21.15.3:0/3691843126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09480cb440 0x7f09480b2760 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f094815d130 tx=0x7f0940009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9c1202c2573a77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f0946ffd700 1 -- 172.21.15.3:0/3691843126 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000e070 con 0x7f09480cb440 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f0946ffd700 1 -- 172.21.15.3:0/3691843126 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000bb60 con 0x7f09480cb440 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.010+0000 7f0946ffd700 1 -- 172.21.15.3:0/3691843126 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940012430 con 0x7f09480cb440 2023-10-06T04:58:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 -- 172.21.15.3:0/3691843126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09480cb440 msgr2=0x7f09480b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 --2- 172.21.15.3:0/3691843126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09480cb440 0x7f09480b2760 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f094815d130 tx=0x7f0940009780 comp rx=0 tx=0).stop 2023-10-06T04:58:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 -- 172.21.15.3:0/3691843126 shutdown_connections 2023-10-06T04:58:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 --2- 172.21.15.3:0/3691843126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09480cb440 0x7f09480b2760 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 --2- 172.21.15.3:0/3691843126 >> v2:172.21.15.153:3300/0 conn(0x7f09481598d0 0x7f09480b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 -- 172.21.15.3:0/3691843126 >> 172.21.15.3:0/3691843126 conn(0x7f0948070310 msgr2=0x7f0948070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 -- 172.21.15.3:0/3691843126 shutdown_connections 2023-10-06T04:58:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.011+0000 7f094e65a700 1 -- 172.21.15.3:0/3691843126 wait complete. 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.012+0000 7f094e65a700 1 Processor -- start 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.012+0000 7f094e65a700 1 -- start start 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094e65a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09480cb440 0x7f0948155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094e65a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0948159500 0x7f09480af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094e65a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481598d0 0x7f09480b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094e65a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0948155730 con 0x7f09481598d0 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094e65a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09481550e0 con 0x7f09480cb440 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094e65a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480b5390 con 0x7f0948159500 2023-10-06T04:58:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481598d0 0x7f09480b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481598d0 0x7f09480b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51232/0 (socket says 172.21.15.3:51232) 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 -- 172.21.15.3:0/2291489647 learned_addr learned my addr 172.21.15.3:0/2291489647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f0947fff700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09480cb440 0x7f0948155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f09477fe700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0948159500 0x7f09480af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 -- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0948159500 msgr2=0x7f09480af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0948159500 0x7f09480af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 -- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09480cb440 msgr2=0x7f0948155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09480cb440 0x7f0948155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 -- 172.21.15.3:0/2291489647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940009a20 con 0x7f09481598d0 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f09477fe700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0948159500 0x7f09480af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.013+0000 7f094cbf7700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481598d0 0x7f09480b4e50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f094806ea80 tx=0x7f0940019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.014+0000 7f092ffff700 1 -- 172.21.15.3:0/2291489647 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940012e00 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.014+0000 7f092ffff700 1 -- 172.21.15.3:0/2291489647 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094001e630 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.014+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094815c6b0 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.014+0000 7f092ffff700 1 -- 172.21.15.3:0/2291489647 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094001e9f0 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.014+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094815c990 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.015+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0948071680 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.015+0000 7f092ffff700 1 -- 172.21.15.3:0/2291489647 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0940024070 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.015+0000 7f092ffff700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0934066f30 0x7f09340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.016+0000 7f092ffff700 1 -- 172.21.15.3:0/2291489647 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0940086e70 con 0x7f09481598d0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.016+0000 7f0947fff700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0934066f30 0x7f09340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.016+0000 7f0947fff700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0934066f30 0x7f09340693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f09380058d0 tx=0x7f0938009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.020+0000 7f092ffff700 1 -- 172.21.15.3:0/2291489647 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0940053f80 con 0x7f09481598d0 2023-10-06T04:58:33.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.211+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09480cbc90 con 0x7f0934066f30 2023-10-06T04:58:33.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.220+0000 7f092ffff700 1 -- 172.21.15.3:0/2291489647 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311703 (secure 0 0 0) 0x7f09480cbc90 con 0x7f0934066f30 2023-10-06T04:58:33.225 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0934066f30 msgr2=0x7f09340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0934066f30 0x7f09340693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f09380058d0 tx=0x7f0938009040 comp rx=0 tx=0).stop 2023-10-06T04:58:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481598d0 msgr2=0x7f09480b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481598d0 0x7f09480b4e50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f094806ea80 tx=0x7f0940019000 comp rx=0 tx=0).stop 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 shutdown_connections 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0934066f30 0x7f09340693e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09480cb440 0x7f0948155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481598d0 0x7f09480b4e50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 --2- 172.21.15.3:0/2291489647 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0948159500 0x7f09480af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 >> 172.21.15.3:0/2291489647 conn(0x7f0948070310 msgr2=0x7f09480ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 shutdown_connections 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.223+0000 7f094e65a700 1 -- 172.21.15.3:0/2291489647 wait complete. 2023-10-06T04:58:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:58:33.237 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2023-10-06T04:58:31.515376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444900,"kb_used_data":9252,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752529820,"statfs":{"total":773094113280,"available":770590535680,"internally_reserved":0,"allocated":9474048,"data_stored":4874791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.806996"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760714+0000","last_change":"2023-10-06T04:58:28.757439+0000","last_active":"2023-10-06T04:58:28.760714+0000","last_peered":"2023-10-06T04:58:28.760714+0000","last_clean":"2023-10-06T04:58:28.760714+0000","last_became_active":"2023-10-06T04:58:28.757151+0000","last_became_peered":"2023-10-06T04:58:28.757151+0000","last_unstale":"2023-10-06T04:58:28.760714+0000","last_undegraded":"2023-10-06T04:58:28.760714+0000","last_fullsized":"2023-10-06T04:58:28.760714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:21.546515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761450+0000","last_change":"2023-10-06T04:58:28.760332+0000","last_active":"2023-10-06T04:58:28.761450+0000","last_peered":"2023-10-06T04:58:28.761450+0000","last_clean":"2023-10-06T04:58:28.761450+0000","last_became_active":"2023-10-06T04:58:28.760033+0000","last_became_peered":"2023-10-06T04:58:28.760033+0000","last_unstale":"2023-10-06T04:58:28.761450+0000","last_undegraded":"2023-10-06T04:58:28.761450+0000","last_fullsized":"2023-10-06T04:58:28.761450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:24.783070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758128+0000","last_change":"2023-10-06T04:58:28.754493+0000","last_active":"2023-10-06T04:58:28.758128+0000","last_peered":"2023-10-06T04:58:28.758128+0000","last_clean":"2023-10-06T04:58:28.758128+0000","last_became_active":"2023-10-06T04:58:28.754180+0000","last_became_peered":"2023-10-06T04:58:28.754180+0000","last_unstale":"2023-10-06T04:58:28.758128+0000","last_undegraded":"2023-10-06T04:58:28.758128+0000","last_fullsized":"2023-10-06T04:58:28.758128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.274854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766115+0000","last_change":"2023-10-06T04:58:28.759360+0000","last_active":"2023-10-06T04:58:28.766115+0000","last_peered":"2023-10-06T04:58:28.766115+0000","last_clean":"2023-10-06T04:58:28.766115+0000","last_became_active":"2023-10-06T04:58:28.759091+0000","last_became_peered":"2023-10-06T04:58:28.759091+0000","last_unstale":"2023-10-06T04:58:28.766115+0000","last_undegraded":"2023-10-06T04:58:28.766115+0000","last_fullsized":"2023-10-06T04:58:28.766115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:53.721225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762611+0000","last_change":"2023-10-06T04:58:28.760226+0000","last_active":"2023-10-06T04:58:28.762611+0000","last_peered":"2023-10-06T04:58:28.762611+0000","last_clean":"2023-10-06T04:58:28.762611+0000","last_became_active":"2023-10-06T04:58:28.759424+0000","last_became_peered":"2023-10-06T04:58:28.759424+0000","last_unstale":"2023-10-06T04:58:28.762611+0000","last_undegraded":"2023-10-06T04:58:28.762611+0000","last_fullsized":"2023-10-06T04:58:28.762611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:12.971917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.759817+0000","last_change":"2023-10-06T04:58:28.752846+0000","last_active":"2023-10-06T04:58:28.759817+0000","last_peered":"2023-10-06T04:58:28.759817+0000","last_clean":"2023-10-06T04:58:28.759817+0000","last_became_active":"2023-10-06T04:58:28.752373+0000","last_became_peered":"2023-10-06T04:58:28.752373+0000","last_unstale":"2023-10-06T04:58:28.759817+0000","last_undegraded":"2023-10-06T04:58:28.759817+0000","last_fullsized":"2023-10-06T04:58:28.759817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:23.907805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758369+0000","last_change":"2023-10-06T04:58:28.755509+0000","last_active":"2023-10-06T04:58:28.758369+0000","last_peered":"2023-10-06T04:58:28.758369+0000","last_clean":"2023-10-06T04:58:28.758369+0000","last_became_active":"2023-10-06T04:58:28.755199+0000","last_became_peered":"2023-10-06T04:58:28.755199+0000","last_unstale":"2023-10-06T04:58:28.758369+0000","last_undegraded":"2023-10-06T04:58:28.758369+0000","last_fullsized":"2023-10-06T04:58:28.758369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:01.274854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756818+0000","last_change":"2023-10-06T04:58:28.754092+0000","last_active":"2023-10-06T04:58:28.756818+0000","last_peered":"2023-10-06T04:58:28.756818+0000","last_clean":"2023-10-06T04:58:28.756818+0000","last_became_active":"2023-10-06T04:58:28.753789+0000","last_became_peered":"2023-10-06T04:58:28.753789+0000","last_unstale":"2023-10-06T04:58:28.756818+0000","last_undegraded":"2023-10-06T04:58:28.756818+0000","last_fullsized":"2023-10-06T04:58:28.756818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:35.605421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.759471+0000","last_change":"2023-10-06T04:58:28.752526+0000","last_active":"2023-10-06T04:58:28.759471+0000","last_peered":"2023-10-06T04:58:28.759471+0000","last_clean":"2023-10-06T04:58:28.759471+0000","last_became_active":"2023-10-06T04:58:28.752079+0000","last_became_peered":"2023-10-06T04:58:28.752079+0000","last_unstale":"2023-10-06T04:58:28.759471+0000","last_undegraded":"2023-10-06T04:58:28.759471+0000","last_fullsized":"2023-10-06T04:58:28.759471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:43.173543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755297+0000","last_change":"2023-10-06T04:58:28.751826+0000","last_active":"2023-10-06T04:58:28.755297+0000","last_peered":"2023-10-06T04:58:28.755297+0000","last_clean":"2023-10-06T04:58:28.755297+0000","last_became_active":"2023-10-06T04:58:28.751544+0000","last_became_peered":"2023-10-06T04:58:28.751544+0000","last_unstale":"2023-10-06T04:58:28.755297+0000","last_undegraded":"2023-10-06T04:58:28.755297+0000","last_fullsized":"2023-10-06T04:58:28.755297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:59.926203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761549+0000","last_change":"2023-10-06T04:58:28.755845+0000","last_active":"2023-10-06T04:58:28.761549+0000","last_peered":"2023-10-06T04:58:28.761549+0000","last_clean":"2023-10-06T04:58:28.761549+0000","last_became_active":"2023-10-06T04:58:28.755578+0000","last_became_peered":"2023-10-06T04:58:28.755578+0000","last_unstale":"2023-10-06T04:58:28.761549+0000","last_undegraded":"2023-10-06T04:58:28.761549+0000","last_fullsized":"2023-10-06T04:58:28.761549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:15.942873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758910+0000","last_change":"2023-10-06T04:58:28.757814+0000","last_active":"2023-10-06T04:58:28.758910+0000","last_peered":"2023-10-06T04:58:28.758910+0000","last_clean":"2023-10-06T04:58:28.758910+0000","last_became_active":"2023-10-06T04:58:28.757510+0000","last_became_peered":"2023-10-06T04:58:28.757510+0000","last_unstale":"2023-10-06T04:58:28.758910+0000","last_undegraded":"2023-10-06T04:58:28.758910+0000","last_fullsized":"2023-10-06T04:58:28.758910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:44.774012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756595+0000","last_change":"2023-10-06T04:58:28.753896+0000","last_active":"2023-10-06T04:58:28.756595+0000","last_peered":"2023-10-06T04:58:28.756595+0000","last_clean":"2023-10-06T04:58:28.756595+0000","last_became_active":"2023-10-06T04:58:28.753527+0000","last_became_peered":"2023-10-06T04:58:28.753527+0000","last_unstale":"2023-10-06T04:58:28.756595+0000","last_undegraded":"2023-10-06T04:58:28.756595+0000","last_fullsized":"2023-10-06T04:58:28.756595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:48.041072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762642+0000","last_change":"2023-10-06T04:58:28.760990+0000","last_active":"2023-10-06T04:58:28.762642+0000","last_peered":"2023-10-06T04:58:28.762642+0000","last_clean":"2023-10-06T04:58:28.762642+0000","last_became_active":"2023-10-06T04:58:28.760739+0000","last_became_peered":"2023-10-06T04:58:28.760739+0000","last_unstale":"2023-10-06T04:58:28.762642+0000","last_undegraded":"2023-10-06T04:58:28.762642+0000","last_fullsized":"2023-10-06T04:58:28.762642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:08.664175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766394+0000","last_change":"2023-10-06T04:58:28.761832+0000","last_active":"2023-10-06T04:58:28.766394+0000","last_peered":"2023-10-06T04:58:28.766394+0000","last_clean":"2023-10-06T04:58:28.766394+0000","last_became_active":"2023-10-06T04:58:28.760193+0000","last_became_peered":"2023-10-06T04:58:28.760193+0000","last_unstale":"2023-10-06T04:58:28.766394+0000","last_undegraded":"2023-10-06T04:58:28.766394+0000","last_fullsized":"2023-10-06T04:58:28.766394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:26.325538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761828+0000","last_change":"2023-10-06T04:58:28.754608+0000","last_active":"2023-10-06T04:58:28.761828+0000","last_peered":"2023-10-06T04:58:28.761828+0000","last_clean":"2023-10-06T04:58:28.761828+0000","last_became_active":"2023-10-06T04:58:28.754323+0000","last_became_peered":"2023-10-06T04:58:28.754323+0000","last_unstale":"2023-10-06T04:58:28.761828+0000","last_undegraded":"2023-10-06T04:58:28.761828+0000","last_fullsized":"2023-10-06T04:58:28.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:49.921914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.765128+0000","last_change":"2023-10-06T04:58:28.759939+0000","last_active":"2023-10-06T04:58:28.765128+0000","last_peered":"2023-10-06T04:58:28.765128+0000","last_clean":"2023-10-06T04:58:28.765128+0000","last_became_active":"2023-10-06T04:58:28.759440+0000","last_became_peered":"2023-10-06T04:58:28.759440+0000","last_unstale":"2023-10-06T04:58:28.765128+0000","last_undegraded":"2023-10-06T04:58:28.765128+0000","last_fullsized":"2023-10-06T04:58:28.765128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:17.753584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758167+0000","last_change":"2023-10-06T04:58:28.751501+0000","last_active":"2023-10-06T04:58:28.758167+0000","last_peered":"2023-10-06T04:58:28.758167+0000","last_clean":"2023-10-06T04:58:28.758167+0000","last_became_active":"2023-10-06T04:58:28.751208+0000","last_became_peered":"2023-10-06T04:58:28.751208+0000","last_unstale":"2023-10-06T04:58:28.758167+0000","last_undegraded":"2023-10-06T04:58:28.758167+0000","last_fullsized":"2023-10-06T04:58:28.758167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:28.360451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762598+0000","last_change":"2023-10-06T04:58:28.759856+0000","last_active":"2023-10-06T04:58:28.762598+0000","last_peered":"2023-10-06T04:58:28.762598+0000","last_clean":"2023-10-06T04:58:28.762598+0000","last_became_active":"2023-10-06T04:58:28.759574+0000","last_became_peered":"2023-10-06T04:58:28.759574+0000","last_unstale":"2023-10-06T04:58:28.762598+0000","last_undegraded":"2023-10-06T04:58:28.762598+0000","last_fullsized":"2023-10-06T04:58:28.762598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:15.912230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766179+0000","last_change":"2023-10-06T04:58:28.760834+0000","last_active":"2023-10-06T04:58:28.766179+0000","last_peered":"2023-10-06T04:58:28.766179+0000","last_clean":"2023-10-06T04:58:28.766179+0000","last_became_active":"2023-10-06T04:58:28.760406+0000","last_became_peered":"2023-10-06T04:58:28.760406+0000","last_unstale":"2023-10-06T04:58:28.766179+0000","last_undegraded":"2023-10-06T04:58:28.766179+0000","last_fullsized":"2023-10-06T04:58:28.766179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:17.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.753933+0000","last_change":"2023-10-06T04:58:28.750495+0000","last_active":"2023-10-06T04:58:28.753933+0000","last_peered":"2023-10-06T04:58:28.753933+0000","last_clean":"2023-10-06T04:58:28.753933+0000","last_became_active":"2023-10-06T04:58:28.750205+0000","last_became_peered":"2023-10-06T04:58:28.750205+0000","last_unstale":"2023-10-06T04:58:28.753933+0000","last_undegraded":"2023-10-06T04:58:28.753933+0000","last_fullsized":"2023-10-06T04:58:28.753933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:53.721225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762567+0000","last_change":"2023-10-06T04:58:28.759867+0000","last_active":"2023-10-06T04:58:28.762567+0000","last_peered":"2023-10-06T04:58:28.762567+0000","last_clean":"2023-10-06T04:58:28.762567+0000","last_became_active":"2023-10-06T04:58:28.759584+0000","last_became_peered":"2023-10-06T04:58:28.759584+0000","last_unstale":"2023-10-06T04:58:28.762567+0000","last_undegraded":"2023-10-06T04:58:28.762567+0000","last_fullsized":"2023-10-06T04:58:28.762567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:23.430458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762520+0000","last_change":"2023-10-06T04:58:28.758826+0000","last_active":"2023-10-06T04:58:28.762520+0000","last_peered":"2023-10-06T04:58:28.762520+0000","last_clean":"2023-10-06T04:58:28.762520+0000","last_became_active":"2023-10-06T04:58:28.758492+0000","last_became_peered":"2023-10-06T04:58:28.758492+0000","last_unstale":"2023-10-06T04:58:28.762520+0000","last_undegraded":"2023-10-06T04:58:28.762520+0000","last_fullsized":"2023-10-06T04:58:28.762520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:17.918940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758320+0000","last_change":"2023-10-06T04:58:28.754651+0000","last_active":"2023-10-06T04:58:28.758320+0000","last_peered":"2023-10-06T04:58:28.758320+0000","last_clean":"2023-10-06T04:58:28.758320+0000","last_became_active":"2023-10-06T04:58:28.754377+0000","last_became_peered":"2023-10-06T04:58:28.754377+0000","last_unstale":"2023-10-06T04:58:28.758320+0000","last_undegraded":"2023-10-06T04:58:28.758320+0000","last_fullsized":"2023-10-06T04:58:28.758320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:25.977793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761486+0000","last_change":"2023-10-06T04:58:28.757971+0000","last_active":"2023-10-06T04:58:28.761486+0000","last_peered":"2023-10-06T04:58:28.761486+0000","last_clean":"2023-10-06T04:58:28.761486+0000","last_became_active":"2023-10-06T04:58:28.757704+0000","last_became_peered":"2023-10-06T04:58:28.757704+0000","last_unstale":"2023-10-06T04:58:28.761486+0000","last_undegraded":"2023-10-06T04:58:28.761486+0000","last_fullsized":"2023-10-06T04:58:28.761486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:46.967172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762006+0000","last_change":"2023-10-06T04:58:28.759837+0000","last_active":"2023-10-06T04:58:28.762006+0000","last_peered":"2023-10-06T04:58:28.762006+0000","last_clean":"2023-10-06T04:58:28.762006+0000","last_became_active":"2023-10-06T04:58:28.759500+0000","last_became_peered":"2023-10-06T04:58:28.759500+0000","last_unstale":"2023-10-06T04:58:28.762006+0000","last_undegraded":"2023-10-06T04:58:28.762006+0000","last_fullsized":"2023-10-06T04:58:28.762006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.931049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757967+0000","last_change":"2023-10-06T04:58:28.752572+0000","last_active":"2023-10-06T04:58:28.757967+0000","last_peered":"2023-10-06T04:58:28.757967+0000","last_clean":"2023-10-06T04:58:28.757967+0000","last_became_active":"2023-10-06T04:58:28.752224+0000","last_became_peered":"2023-10-06T04:58:28.752224+0000","last_unstale":"2023-10-06T04:58:28.757967+0000","last_undegraded":"2023-10-06T04:58:28.757967+0000","last_fullsized":"2023-10-06T04:58:28.757967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:18.621699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755095+0000","last_change":"2023-10-06T04:58:28.747777+0000","last_active":"2023-10-06T04:58:28.755095+0000","last_peered":"2023-10-06T04:58:28.755095+0000","last_clean":"2023-10-06T04:58:28.755095+0000","last_became_active":"2023-10-06T04:58:28.747557+0000","last_became_peered":"2023-10-06T04:58:28.747557+0000","last_unstale":"2023-10-06T04:58:28.755095+0000","last_undegraded":"2023-10-06T04:58:28.755095+0000","last_fullsized":"2023-10-06T04:58:28.755095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:30.024992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.751394+0000","last_change":"2023-10-06T04:58:28.748820+0000","last_active":"2023-10-06T04:58:28.751394+0000","last_peered":"2023-10-06T04:58:28.751394+0000","last_clean":"2023-10-06T04:58:28.751394+0000","last_became_active":"2023-10-06T04:58:28.748582+0000","last_became_peered":"2023-10-06T04:58:28.748582+0000","last_unstale":"2023-10-06T04:58:28.751394+0000","last_undegraded":"2023-10-06T04:58:28.751394+0000","last_fullsized":"2023-10-06T04:58:28.751394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:21.546515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.752116+0000","last_change":"2023-10-06T04:58:28.747880+0000","last_active":"2023-10-06T04:58:28.752116+0000","last_peered":"2023-10-06T04:58:28.752116+0000","last_clean":"2023-10-06T04:58:28.752116+0000","last_became_active":"2023-10-06T04:58:28.747601+0000","last_became_peered":"2023-10-06T04:58:28.747601+0000","last_unstale":"2023-10-06T04:58:28.752116+0000","last_undegraded":"2023-10-06T04:58:28.752116+0000","last_fullsized":"2023-10-06T04:58:28.752116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:59.336531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767224+0000","last_change":"2023-10-06T04:58:28.766782+0000","last_active":"2023-10-06T04:58:28.767224+0000","last_peered":"2023-10-06T04:58:28.767224+0000","last_clean":"2023-10-06T04:58:28.767224+0000","last_became_active":"2023-10-06T04:58:28.766504+0000","last_became_peered":"2023-10-06T04:58:28.766504+0000","last_unstale":"2023-10-06T04:58:28.767224+0000","last_undegraded":"2023-10-06T04:58:28.767224+0000","last_fullsized":"2023-10-06T04:58:28.767224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:53.721225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756145+0000","last_change":"2023-10-06T04:58:28.748056+0000","last_active":"2023-10-06T04:58:28.756145+0000","last_peered":"2023-10-06T04:58:28.756145+0000","last_clean":"2023-10-06T04:58:28.756145+0000","last_became_active":"2023-10-06T04:58:28.747782+0000","last_became_peered":"2023-10-06T04:58:28.747782+0000","last_unstale":"2023-10-06T04:58:28.756145+0000","last_undegraded":"2023-10-06T04:58:28.756145+0000","last_fullsized":"2023-10-06T04:58:28.756145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:40.295631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757864+0000","last_change":"2023-10-06T04:58:28.748339+0000","last_active":"2023-10-06T04:58:28.757864+0000","last_peered":"2023-10-06T04:58:28.757864+0000","last_clean":"2023-10-06T04:58:28.757864+0000","last_became_active":"2023-10-06T04:58:28.747917+0000","last_became_peered":"2023-10-06T04:58:28.747917+0000","last_unstale":"2023-10-06T04:58:28.757864+0000","last_undegraded":"2023-10-06T04:58:28.757864+0000","last_fullsized":"2023-10-06T04:58:28.757864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:30.678346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"83'4","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.769666+0000","last_change":"2023-10-06T04:58:27.731642+0000","last_active":"2023-10-06T04:58:28.769666+0000","last_peered":"2023-10-06T04:58:28.769666+0000","last_clean":"2023-10-06T04:58:28.769666+0000","last_became_active":"2023-10-06T04:58:27.731213+0000","last_became_peered":"2023-10-06T04:58:27.731213+0000","last_unstale":"2023-10-06T04:58:28.769666+0000","last_undegraded":"2023-10-06T04:58:28.769666+0000","last_fullsized":"2023-10-06T04:58:28.769666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:31.747508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756058+0000","last_change":"2023-10-06T04:58:28.750187+0000","last_active":"2023-10-06T04:58:28.756058+0000","last_peered":"2023-10-06T04:58:28.756058+0000","last_clean":"2023-10-06T04:58:28.756058+0000","last_became_active":"2023-10-06T04:58:28.749946+0000","last_became_peered":"2023-10-06T04:58:28.749946+0000","last_unstale":"2023-10-06T04:58:28.756058+0000","last_undegraded":"2023-10-06T04:58:28.756058+0000","last_fullsized":"2023-10-06T04:58:28.756058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:50:06.145846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761186+0000","last_change":"2023-10-06T04:58:27.728013+0000","last_active":"2023-10-06T04:58:28.761186+0000","last_peered":"2023-10-06T04:58:28.761186+0000","last_clean":"2023-10-06T04:58:28.761186+0000","last_became_active":"2023-10-06T04:58:27.727758+0000","last_became_peered":"2023-10-06T04:58:27.727758+0000","last_unstale":"2023-10-06T04:58:28.761186+0000","last_undegraded":"2023-10-06T04:58:28.761186+0000","last_fullsized":"2023-10-06T04:58:28.761186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:01.177156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758153+0000","last_change":"2023-10-06T04:58:28.754370+0000","last_active":"2023-10-06T04:58:28.758153+0000","last_peered":"2023-10-06T04:58:28.758153+0000","last_clean":"2023-10-06T04:58:28.758153+0000","last_became_active":"2023-10-06T04:58:28.754112+0000","last_became_peered":"2023-10-06T04:58:28.754112+0000","last_unstale":"2023-10-06T04:58:28.758153+0000","last_undegraded":"2023-10-06T04:58:28.758153+0000","last_fullsized":"2023-10-06T04:58:28.758153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:45.917735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.765242+0000","last_change":"2023-10-06T04:58:27.733812+0000","last_active":"2023-10-06T04:58:28.765242+0000","last_peered":"2023-10-06T04:58:28.765242+0000","last_clean":"2023-10-06T04:58:28.765242+0000","last_became_active":"2023-10-06T04:58:27.733571+0000","last_became_peered":"2023-10-06T04:58:27.733571+0000","last_unstale":"2023-10-06T04:58:28.765242+0000","last_undegraded":"2023-10-06T04:58:28.765242+0000","last_fullsized":"2023-10-06T04:58:28.765242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:22.428467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761811+0000","last_change":"2023-10-06T04:58:28.756037+0000","last_active":"2023-10-06T04:58:28.761811+0000","last_peered":"2023-10-06T04:58:28.761811+0000","last_clean":"2023-10-06T04:58:28.761811+0000","last_became_active":"2023-10-06T04:58:28.755717+0000","last_became_peered":"2023-10-06T04:58:28.755717+0000","last_unstale":"2023-10-06T04:58:28.761811+0000","last_undegraded":"2023-10-06T04:58:28.761811+0000","last_fullsized":"2023-10-06T04:58:28.761811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:13.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767899+0000","last_change":"2023-10-06T04:58:27.729480+0000","last_active":"2023-10-06T04:58:28.767899+0000","last_peered":"2023-10-06T04:58:28.767899+0000","last_clean":"2023-10-06T04:58:28.767899+0000","last_became_active":"2023-10-06T04:58:27.729069+0000","last_became_peered":"2023-10-06T04:58:27.729069+0000","last_unstale":"2023-10-06T04:58:28.767899+0000","last_undegraded":"2023-10-06T04:58:28.767899+0000","last_fullsized":"2023-10-06T04:58:28.767899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:05.804921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.766257+0000","last_change":"2023-10-06T04:58:28.761386+0000","last_active":"2023-10-06T04:58:28.766257+0000","last_peered":"2023-10-06T04:58:28.766257+0000","last_clean":"2023-10-06T04:58:28.766257+0000","last_became_active":"2023-10-06T04:58:28.760891+0000","last_became_peered":"2023-10-06T04:58:28.760891+0000","last_unstale":"2023-10-06T04:58:28.766257+0000","last_undegraded":"2023-10-06T04:58:28.766257+0000","last_fullsized":"2023-10-06T04:58:28.766257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:31.387712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761300+0000","last_change":"2023-10-06T04:58:27.732864+0000","last_active":"2023-10-06T04:58:28.761300+0000","last_peered":"2023-10-06T04:58:28.761300+0000","last_clean":"2023-10-06T04:58:28.761300+0000","last_became_active":"2023-10-06T04:58:27.732564+0000","last_became_peered":"2023-10-06T04:58:27.732564+0000","last_unstale":"2023-10-06T04:58:28.761300+0000","last_undegraded":"2023-10-06T04:58:28.761300+0000","last_fullsized":"2023-10-06T04:58:28.761300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:11.379057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758695+0000","last_change":"2023-10-06T04:58:28.752398+0000","last_active":"2023-10-06T04:58:28.758695+0000","last_peered":"2023-10-06T04:58:28.758695+0000","last_clean":"2023-10-06T04:58:28.758695+0000","last_became_active":"2023-10-06T04:58:28.752153+0000","last_became_peered":"2023-10-06T04:58:28.752153+0000","last_unstale":"2023-10-06T04:58:28.758695+0000","last_undegraded":"2023-10-06T04:58:28.758695+0000","last_fullsized":"2023-10-06T04:58:28.758695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:17.916940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.771472+0000","last_change":"2023-10-06T04:58:27.729212+0000","last_active":"2023-10-06T04:58:28.771472+0000","last_peered":"2023-10-06T04:58:28.771472+0000","last_clean":"2023-10-06T04:58:28.771472+0000","last_became_active":"2023-10-06T04:58:27.728925+0000","last_became_peered":"2023-10-06T04:58:27.728925+0000","last_unstale":"2023-10-06T04:58:28.771472+0000","last_undegraded":"2023-10-06T04:58:28.771472+0000","last_fullsized":"2023-10-06T04:58:28.771472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:25.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.759797+0000","last_change":"2023-10-06T04:58:28.752647+0000","last_active":"2023-10-06T04:58:28.759797+0000","last_peered":"2023-10-06T04:58:28.759797+0000","last_clean":"2023-10-06T04:58:28.759797+0000","last_became_active":"2023-10-06T04:58:28.751986+0000","last_became_peered":"2023-10-06T04:58:28.751986+0000","last_unstale":"2023-10-06T04:58:28.759797+0000","last_undegraded":"2023-10-06T04:58:28.759797+0000","last_fullsized":"2023-10-06T04:58:28.759797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:21.035597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.748122+0000","last_change":"2023-10-06T04:58:27.724367+0000","last_active":"2023-10-06T04:58:28.748122+0000","last_peered":"2023-10-06T04:58:28.748122+0000","last_clean":"2023-10-06T04:58:28.748122+0000","last_became_active":"2023-10-06T04:58:27.724102+0000","last_became_peered":"2023-10-06T04:58:27.724102+0000","last_unstale":"2023-10-06T04:58:28.748122+0000","last_undegraded":"2023-10-06T04:58:28.748122+0000","last_fullsized":"2023-10-06T04:58:28.748122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.969927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.764833+0000","last_change":"2023-10-06T04:58:28.759297+0000","last_active":"2023-10-06T04:58:28.764833+0000","last_peered":"2023-10-06T04:58:28.764833+0000","last_clean":"2023-10-06T04:58:28.764833+0000","last_became_active":"2023-10-06T04:58:28.759006+0000","last_became_peered":"2023-10-06T04:58:28.759006+0000","last_unstale":"2023-10-06T04:58:28.764833+0000","last_undegraded":"2023-10-06T04:58:28.764833+0000","last_fullsized":"2023-10-06T04:58:28.764833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:46.967172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.738645+0000","last_change":"2023-10-06T04:58:27.731171+0000","last_active":"2023-10-06T04:58:28.738645+0000","last_peered":"2023-10-06T04:58:28.738645+0000","last_clean":"2023-10-06T04:58:28.738645+0000","last_became_active":"2023-10-06T04:58:27.730898+0000","last_became_peered":"2023-10-06T04:58:27.730898+0000","last_unstale":"2023-10-06T04:58:28.738645+0000","last_undegraded":"2023-10-06T04:58:28.738645+0000","last_fullsized":"2023-10-06T04:58:28.738645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:14.910239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761144+0000","last_change":"2023-10-06T04:58:28.756189+0000","last_active":"2023-10-06T04:58:28.761144+0000","last_peered":"2023-10-06T04:58:28.761144+0000","last_clean":"2023-10-06T04:58:28.761144+0000","last_became_active":"2023-10-06T04:58:28.755922+0000","last_became_peered":"2023-10-06T04:58:28.755922+0000","last_unstale":"2023-10-06T04:58:28.761144+0000","last_undegraded":"2023-10-06T04:58:28.761144+0000","last_fullsized":"2023-10-06T04:58:28.761144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:18.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.744098+0000","last_change":"2023-10-06T04:58:27.728085+0000","last_active":"2023-10-06T04:58:28.744098+0000","last_peered":"2023-10-06T04:58:28.744098+0000","last_clean":"2023-10-06T04:58:28.744098+0000","last_became_active":"2023-10-06T04:58:27.727736+0000","last_became_peered":"2023-10-06T04:58:27.727736+0000","last_unstale":"2023-10-06T04:58:28.744098+0000","last_undegraded":"2023-10-06T04:58:28.744098+0000","last_fullsized":"2023-10-06T04:58:28.744098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:12.568372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758106+0000","last_change":"2023-10-06T04:58:28.753631+0000","last_active":"2023-10-06T04:58:28.758106+0000","last_peered":"2023-10-06T04:58:28.758106+0000","last_clean":"2023-10-06T04:58:28.758106+0000","last_became_active":"2023-10-06T04:58:28.753296+0000","last_became_peered":"2023-10-06T04:58:28.753296+0000","last_unstale":"2023-10-06T04:58:28.758106+0000","last_undegraded":"2023-10-06T04:58:28.758106+0000","last_fullsized":"2023-10-06T04:58:28.758106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:25.977793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.744112+0000","last_change":"2023-10-06T04:58:27.723143+0000","last_active":"2023-10-06T04:58:28.744112+0000","last_peered":"2023-10-06T04:58:28.744112+0000","last_clean":"2023-10-06T04:58:28.744112+0000","last_became_active":"2023-10-06T04:58:27.722847+0000","last_became_peered":"2023-10-06T04:58:27.722847+0000","last_unstale":"2023-10-06T04:58:28.744112+0000","last_undegraded":"2023-10-06T04:58:28.744112+0000","last_fullsized":"2023-10-06T04:58:28.744112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:47.252459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758188+0000","last_change":"2023-10-06T04:58:28.755224+0000","last_active":"2023-10-06T04:58:28.758188+0000","last_peered":"2023-10-06T04:58:28.758188+0000","last_clean":"2023-10-06T04:58:28.758188+0000","last_became_active":"2023-10-06T04:58:28.754886+0000","last_became_peered":"2023-10-06T04:58:28.754886+0000","last_unstale":"2023-10-06T04:58:28.758188+0000","last_undegraded":"2023-10-06T04:58:28.758188+0000","last_fullsized":"2023-10-06T04:58:28.758188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:21.608809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"83'3","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.765434+0000","last_change":"2023-10-06T04:58:27.722854+0000","last_active":"2023-10-06T04:58:28.765434+0000","last_peered":"2023-10-06T04:58:28.765434+0000","last_clean":"2023-10-06T04:58:28.765434+0000","last_became_active":"2023-10-06T04:58:27.722608+0000","last_became_peered":"2023-10-06T04:58:27.722608+0000","last_unstale":"2023-10-06T04:58:28.765434+0000","last_undegraded":"2023-10-06T04:58:28.765434+0000","last_fullsized":"2023-10-06T04:58:28.765434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:49.961948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755080+0000","last_change":"2023-10-06T04:58:28.748910+0000","last_active":"2023-10-06T04:58:28.755080+0000","last_peered":"2023-10-06T04:58:28.755080+0000","last_clean":"2023-10-06T04:58:28.755080+0000","last_became_active":"2023-10-06T04:58:28.748622+0000","last_became_peered":"2023-10-06T04:58:28.748622+0000","last_unstale":"2023-10-06T04:58:28.755080+0000","last_undegraded":"2023-10-06T04:58:28.755080+0000","last_fullsized":"2023-10-06T04:58:28.755080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:24.783070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"83'6","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.771542+0000","last_change":"2023-10-06T04:58:27.728654+0000","last_active":"2023-10-06T04:58:28.771542+0000","last_peered":"2023-10-06T04:58:28.771542+0000","last_clean":"2023-10-06T04:58:28.771542+0000","last_became_active":"2023-10-06T04:58:27.728399+0000","last_became_peered":"2023-10-06T04:58:27.728399+0000","last_unstale":"2023-10-06T04:58:28.771542+0000","last_undegraded":"2023-10-06T04:58:28.771542+0000","last_fullsized":"2023-10-06T04:58:28.771542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:42.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756797+0000","last_change":"2023-10-06T04:58:28.753923+0000","last_active":"2023-10-06T04:58:28.756797+0000","last_peered":"2023-10-06T04:58:28.756797+0000","last_clean":"2023-10-06T04:58:28.756797+0000","last_became_active":"2023-10-06T04:58:28.753624+0000","last_became_peered":"2023-10-06T04:58:28.753624+0000","last_unstale":"2023-10-06T04:58:28.756797+0000","last_undegraded":"2023-10-06T04:58:28.756797+0000","last_fullsized":"2023-10-06T04:58:28.756797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:19.297894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762811+0000","last_change":"2023-10-06T04:58:27.731366+0000","last_active":"2023-10-06T04:58:28.762811+0000","last_peered":"2023-10-06T04:58:28.762811+0000","last_clean":"2023-10-06T04:58:28.762811+0000","last_became_active":"2023-10-06T04:58:27.731111+0000","last_became_peered":"2023-10-06T04:58:27.731111+0000","last_unstale":"2023-10-06T04:58:28.762811+0000","last_undegraded":"2023-10-06T04:58:28.762811+0000","last_fullsized":"2023-10-06T04:58:28.762811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:14.940882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762669+0000","last_change":"2023-10-06T04:58:28.761402+0000","last_active":"2023-10-06T04:58:28.762669+0000","last_peered":"2023-10-06T04:58:28.762669+0000","last_clean":"2023-10-06T04:58:28.762669+0000","last_became_active":"2023-10-06T04:58:28.761126+0000","last_became_peered":"2023-10-06T04:58:28.761126+0000","last_unstale":"2023-10-06T04:58:28.762669+0000","last_undegraded":"2023-10-06T04:58:28.762669+0000","last_fullsized":"2023-10-06T04:58:28.762669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:48.254449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737548+0000","last_change":"2023-10-06T04:58:27.718613+0000","last_active":"2023-10-06T04:58:28.737548+0000","last_peered":"2023-10-06T04:58:28.737548+0000","last_clean":"2023-10-06T04:58:28.737548+0000","last_became_active":"2023-10-06T04:58:27.718037+0000","last_became_peered":"2023-10-06T04:58:27.718037+0000","last_unstale":"2023-10-06T04:58:28.737548+0000","last_undegraded":"2023-10-06T04:58:28.737548+0000","last_fullsized":"2023-10-06T04:58:28.737548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:22.428467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.755261+0000","last_change":"2023-10-06T04:58:28.751739+0000","last_active":"2023-10-06T04:58:28.755261+0000","last_peered":"2023-10-06T04:58:28.755261+0000","last_clean":"2023-10-06T04:58:28.755261+0000","last_became_active":"2023-10-06T04:58:28.751457+0000","last_became_peered":"2023-10-06T04:58:28.751457+0000","last_unstale":"2023-10-06T04:58:28.755261+0000","last_undegraded":"2023-10-06T04:58:28.755261+0000","last_fullsized":"2023-10-06T04:58:28.755261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:05.122344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.746166+0000","last_change":"2023-10-06T04:58:27.728033+0000","last_active":"2023-10-06T04:58:28.746166+0000","last_peered":"2023-10-06T04:58:28.746166+0000","last_clean":"2023-10-06T04:58:28.746166+0000","last_became_active":"2023-10-06T04:58:27.727773+0000","last_became_peered":"2023-10-06T04:58:27.727773+0000","last_unstale":"2023-10-06T04:58:28.746166+0000","last_undegraded":"2023-10-06T04:58:28.746166+0000","last_fullsized":"2023-10-06T04:58:28.746166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:56.503521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.751355+0000","last_change":"2023-10-06T04:58:28.747125+0000","last_active":"2023-10-06T04:58:28.751355+0000","last_peered":"2023-10-06T04:58:28.751355+0000","last_clean":"2023-10-06T04:58:28.751355+0000","last_became_active":"2023-10-06T04:58:28.746820+0000","last_became_peered":"2023-10-06T04:58:28.746820+0000","last_unstale":"2023-10-06T04:58:28.751355+0000","last_undegraded":"2023-10-06T04:58:28.751355+0000","last_fullsized":"2023-10-06T04:58:28.751355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:49.696705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"83'8","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.774844+0000","last_change":"2023-10-06T04:58:27.729252+0000","last_active":"2023-10-06T04:58:28.774844+0000","last_peered":"2023-10-06T04:58:28.774844+0000","last_clean":"2023-10-06T04:58:28.774844+0000","last_became_active":"2023-10-06T04:58:27.728980+0000","last_became_peered":"2023-10-06T04:58:27.728980+0000","last_unstale":"2023-10-06T04:58:28.774844+0000","last_undegraded":"2023-10-06T04:58:28.774844+0000","last_fullsized":"2023-10-06T04:58:28.774844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:19.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.756841+0000","last_change":"2023-10-06T04:58:28.751421+0000","last_active":"2023-10-06T04:58:28.756841+0000","last_peered":"2023-10-06T04:58:28.756841+0000","last_clean":"2023-10-06T04:58:28.756841+0000","last_became_active":"2023-10-06T04:58:28.751131+0000","last_became_peered":"2023-10-06T04:58:28.751131+0000","last_unstale":"2023-10-06T04:58:28.756841+0000","last_undegraded":"2023-10-06T04:58:28.756841+0000","last_fullsized":"2023-10-06T04:58:28.756841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:59.192317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.762408+0000","last_change":"2023-10-06T04:58:27.728770+0000","last_active":"2023-10-06T04:58:28.762408+0000","last_peered":"2023-10-06T04:58:28.762408+0000","last_clean":"2023-10-06T04:58:28.762408+0000","last_became_active":"2023-10-06T04:58:27.728409+0000","last_became_peered":"2023-10-06T04:58:27.728409+0000","last_unstale":"2023-10-06T04:58:28.762408+0000","last_undegraded":"2023-10-06T04:58:28.762408+0000","last_fullsized":"2023-10-06T04:58:28.762408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.969927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760076+0000","last_change":"2023-10-06T04:58:28.752172+0000","last_active":"2023-10-06T04:58:28.760076+0000","last_peered":"2023-10-06T04:58:28.760076+0000","last_clean":"2023-10-06T04:58:28.760076+0000","last_became_active":"2023-10-06T04:58:28.751904+0000","last_became_peered":"2023-10-06T04:58:28.751904+0000","last_unstale":"2023-10-06T04:58:28.760076+0000","last_undegraded":"2023-10-06T04:58:28.760076+0000","last_fullsized":"2023-10-06T04:58:28.760076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:25.977793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.744582+0000","last_change":"2023-10-06T04:58:27.723780+0000","last_active":"2023-10-06T04:58:28.744582+0000","last_peered":"2023-10-06T04:58:28.744582+0000","last_clean":"2023-10-06T04:58:28.744582+0000","last_became_active":"2023-10-06T04:58:27.723510+0000","last_became_peered":"2023-10-06T04:58:27.723510+0000","last_unstale":"2023-10-06T04:58:28.744582+0000","last_undegraded":"2023-10-06T04:58:28.744582+0000","last_fullsized":"2023-10-06T04:58:28.744582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:55.908316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758047+0000","last_change":"2023-10-06T04:58:28.747634+0000","last_active":"2023-10-06T04:58:28.758047+0000","last_peered":"2023-10-06T04:58:28.758047+0000","last_clean":"2023-10-06T04:58:28.758047+0000","last_became_active":"2023-10-06T04:58:28.747327+0000","last_became_peered":"2023-10-06T04:58:28.747327+0000","last_unstale":"2023-10-06T04:58:28.758047+0000","last_undegraded":"2023-10-06T04:58:28.758047+0000","last_fullsized":"2023-10-06T04:58:28.758047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:49.382999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.750087+0000","last_change":"2023-10-06T04:58:27.718785+0000","last_active":"2023-10-06T04:58:28.750087+0000","last_peered":"2023-10-06T04:58:28.750087+0000","last_clean":"2023-10-06T04:58:28.750087+0000","last_became_active":"2023-10-06T04:58:27.717713+0000","last_became_peered":"2023-10-06T04:58:27.717713+0000","last_unstale":"2023-10-06T04:58:28.750087+0000","last_undegraded":"2023-10-06T04:58:28.750087+0000","last_fullsized":"2023-10-06T04:58:28.750087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:33.918728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.751642+0000","last_change":"2023-10-06T04:58:28.748505+0000","last_active":"2023-10-06T04:58:28.751642+0000","last_peered":"2023-10-06T04:58:28.751642+0000","last_clean":"2023-10-06T04:58:28.751642+0000","last_became_active":"2023-10-06T04:58:28.748176+0000","last_became_peered":"2023-10-06T04:58:28.748176+0000","last_unstale":"2023-10-06T04:58:28.751642+0000","last_undegraded":"2023-10-06T04:58:28.751642+0000","last_fullsized":"2023-10-06T04:58:28.751642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:26.325538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741779+0000","last_change":"2023-10-06T04:58:27.732758+0000","last_active":"2023-10-06T04:58:28.741779+0000","last_peered":"2023-10-06T04:58:28.741779+0000","last_clean":"2023-10-06T04:58:28.741779+0000","last_became_active":"2023-10-06T04:58:27.732487+0000","last_became_peered":"2023-10-06T04:58:27.732487+0000","last_unstale":"2023-10-06T04:58:28.741779+0000","last_undegraded":"2023-10-06T04:58:28.741779+0000","last_fullsized":"2023-10-06T04:58:28.741779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:42.667103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.754337+0000","last_change":"2023-10-06T04:58:28.750509+0000","last_active":"2023-10-06T04:58:28.754337+0000","last_peered":"2023-10-06T04:58:28.754337+0000","last_clean":"2023-10-06T04:58:28.754337+0000","last_became_active":"2023-10-06T04:58:28.750209+0000","last_became_peered":"2023-10-06T04:58:28.750209+0000","last_unstale":"2023-10-06T04:58:28.754337+0000","last_undegraded":"2023-10-06T04:58:28.754337+0000","last_fullsized":"2023-10-06T04:58:28.754337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:32.883243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.738829+0000","last_change":"2023-10-06T04:58:27.721729+0000","last_active":"2023-10-06T04:58:28.738829+0000","last_peered":"2023-10-06T04:58:28.738829+0000","last_clean":"2023-10-06T04:58:28.738829+0000","last_became_active":"2023-10-06T04:58:27.721436+0000","last_became_peered":"2023-10-06T04:58:27.721436+0000","last_unstale":"2023-10-06T04:58:28.738829+0000","last_undegraded":"2023-10-06T04:58:28.738829+0000","last_fullsized":"2023-10-06T04:58:28.738829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:07.662184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.763679+0000","last_change":"2023-10-06T04:58:28.759782+0000","last_active":"2023-10-06T04:58:28.763679+0000","last_peered":"2023-10-06T04:58:28.763679+0000","last_clean":"2023-10-06T04:58:28.763679+0000","last_became_active":"2023-10-06T04:58:28.759501+0000","last_became_peered":"2023-10-06T04:58:28.759501+0000","last_unstale":"2023-10-06T04:58:28.763679+0000","last_undegraded":"2023-10-06T04:58:28.763679+0000","last_fullsized":"2023-10-06T04:58:28.763679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:30.678346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.742140+0000","last_change":"2023-10-06T04:58:27.722076+0000","last_active":"2023-10-06T04:58:28.742140+0000","last_peered":"2023-10-06T04:58:28.742140+0000","last_clean":"2023-10-06T04:58:28.742140+0000","last_became_active":"2023-10-06T04:58:27.721806+0000","last_became_peered":"2023-10-06T04:58:27.721806+0000","last_unstale":"2023-10-06T04:58:28.742140+0000","last_undegraded":"2023-10-06T04:58:28.742140+0000","last_fullsized":"2023-10-06T04:58:28.742140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:41.996701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757203+0000","last_change":"2023-10-06T04:58:28.753628+0000","last_active":"2023-10-06T04:58:28.757203+0000","last_peered":"2023-10-06T04:58:28.757203+0000","last_clean":"2023-10-06T04:58:28.757203+0000","last_became_active":"2023-10-06T04:58:28.753310+0000","last_became_peered":"2023-10-06T04:58:28.753310+0000","last_unstale":"2023-10-06T04:58:28.757203+0000","last_undegraded":"2023-10-06T04:58:28.757203+0000","last_fullsized":"2023-10-06T04:58:28.757203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:51.801063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.742878+0000","last_change":"2023-10-06T04:58:27.732877+0000","last_active":"2023-10-06T04:58:28.742878+0000","last_peered":"2023-10-06T04:58:28.742878+0000","last_clean":"2023-10-06T04:58:28.742878+0000","last_became_active":"2023-10-06T04:58:27.732464+0000","last_became_peered":"2023-10-06T04:58:27.732464+0000","last_unstale":"2023-10-06T04:58:28.742878+0000","last_undegraded":"2023-10-06T04:58:28.742878+0000","last_fullsized":"2023-10-06T04:58:28.742878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:54.030943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760108+0000","last_change":"2023-10-06T04:58:28.754718+0000","last_active":"2023-10-06T04:58:28.760108+0000","last_peered":"2023-10-06T04:58:28.760108+0000","last_clean":"2023-10-06T04:58:28.760108+0000","last_became_active":"2023-10-06T04:58:28.754430+0000","last_became_peered":"2023-10-06T04:58:28.754430+0000","last_unstale":"2023-10-06T04:58:28.760108+0000","last_undegraded":"2023-10-06T04:58:28.760108+0000","last_fullsized":"2023-10-06T04:58:28.760108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:28.360451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737824+0000","last_change":"2023-10-06T04:58:27.727208+0000","last_active":"2023-10-06T04:58:28.737824+0000","last_peered":"2023-10-06T04:58:28.737824+0000","last_clean":"2023-10-06T04:58:28.737824+0000","last_became_active":"2023-10-06T04:58:27.726965+0000","last_became_peered":"2023-10-06T04:58:27.726965+0000","last_unstale":"2023-10-06T04:58:28.737824+0000","last_undegraded":"2023-10-06T04:58:28.737824+0000","last_fullsized":"2023-10-06T04:58:28.737824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:07.662184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767439+0000","last_change":"2023-10-06T04:58:28.766901+0000","last_active":"2023-10-06T04:58:28.767439+0000","last_peered":"2023-10-06T04:58:28.767439+0000","last_clean":"2023-10-06T04:58:28.767439+0000","last_became_active":"2023-10-06T04:58:28.766547+0000","last_became_peered":"2023-10-06T04:58:28.766547+0000","last_unstale":"2023-10-06T04:58:28.767439+0000","last_undegraded":"2023-10-06T04:58:28.767439+0000","last_fullsized":"2023-10-06T04:58:28.767439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:18.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741592+0000","last_change":"2023-10-06T04:58:27.725736+0000","last_active":"2023-10-06T04:58:28.741592+0000","last_peered":"2023-10-06T04:58:28.741592+0000","last_clean":"2023-10-06T04:58:28.741592+0000","last_became_active":"2023-10-06T04:58:27.725452+0000","last_became_peered":"2023-10-06T04:58:27.725452+0000","last_unstale":"2023-10-06T04:58:28.741592+0000","last_undegraded":"2023-10-06T04:58:28.741592+0000","last_fullsized":"2023-10-06T04:58:28.741592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:01.177156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.749932+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:58:28.749932+0000","last_peered":"2023-10-06T04:58:28.749932+0000","last_clean":"2023-10-06T04:58:28.749932+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:58:28.749932+0000","last_undegraded":"2023-10-06T04:58:28.749932+0000","last_fullsized":"2023-10-06T04:58:28.749932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.758529+0000","last_change":"2023-10-06T04:58:28.747961+0000","last_active":"2023-10-06T04:58:28.758529+0000","last_peered":"2023-10-06T04:58:28.758529+0000","last_clean":"2023-10-06T04:58:28.758529+0000","last_became_active":"2023-10-06T04:58:28.747642+0000","last_became_peered":"2023-10-06T04:58:28.747642+0000","last_unstale":"2023-10-06T04:58:28.758529+0000","last_undegraded":"2023-10-06T04:58:28.758529+0000","last_fullsized":"2023-10-06T04:58:28.758529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:31.040866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.740415+0000","last_change":"2023-10-06T04:58:27.725960+0000","last_active":"2023-10-06T04:58:28.740415+0000","last_peered":"2023-10-06T04:58:28.740415+0000","last_clean":"2023-10-06T04:58:28.740415+0000","last_became_active":"2023-10-06T04:58:27.725520+0000","last_became_peered":"2023-10-06T04:58:27.725520+0000","last_unstale":"2023-10-06T04:58:28.740415+0000","last_undegraded":"2023-10-06T04:58:28.740415+0000","last_fullsized":"2023-10-06T04:58:28.740415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:54.181959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.749074+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:58:28.749074+0000","last_peered":"2023-10-06T04:58:28.749074+0000","last_clean":"2023-10-06T04:58:28.749074+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:58:28.749074+0000","last_undegraded":"2023-10-06T04:58:28.749074+0000","last_fullsized":"2023-10-06T04:58:28.749074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757842+0000","last_change":"2023-10-06T04:58:28.748173+0000","last_active":"2023-10-06T04:58:28.757842+0000","last_peered":"2023-10-06T04:58:28.757842+0000","last_clean":"2023-10-06T04:58:28.757842+0000","last_became_active":"2023-10-06T04:58:28.747234+0000","last_became_peered":"2023-10-06T04:58:28.747234+0000","last_unstale":"2023-10-06T04:58:28.757842+0000","last_undegraded":"2023-10-06T04:58:28.757842+0000","last_fullsized":"2023-10-06T04:58:28.757842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:21.546515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.763491+0000","last_change":"2023-10-06T04:58:27.727783+0000","last_active":"2023-10-06T04:58:28.763491+0000","last_peered":"2023-10-06T04:58:28.763491+0000","last_clean":"2023-10-06T04:58:28.763491+0000","last_became_active":"2023-10-06T04:58:27.727473+0000","last_became_peered":"2023-10-06T04:58:27.727473+0000","last_unstale":"2023-10-06T04:58:28.763491+0000","last_undegraded":"2023-10-06T04:58:28.763491+0000","last_fullsized":"2023-10-06T04:58:28.763491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:26.640364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.747317+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:58:28.747317+0000","last_peered":"2023-10-06T04:58:28.747317+0000","last_clean":"2023-10-06T04:58:28.747317+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:58:28.747317+0000","last_undegraded":"2023-10-06T04:58:28.747317+0000","last_fullsized":"2023-10-06T04:58:28.747317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":215,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737937+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:58:28.737937+0000","last_peered":"2023-10-06T04:58:28.737937+0000","last_clean":"2023-10-06T04:58:28.737937+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:58:28.737937+0000","last_undegraded":"2023-10-06T04:58:28.737937+0000","last_fullsized":"2023-10-06T04:58:28.737937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.757305+0000","last_change":"2023-10-06T04:58:28.751394+0000","last_active":"2023-10-06T04:58:28.757305+0000","last_peered":"2023-10-06T04:58:28.757305+0000","last_clean":"2023-10-06T04:58:28.757305+0000","last_became_active":"2023-10-06T04:58:28.751148+0000","last_became_peered":"2023-10-06T04:58:28.751148+0000","last_unstale":"2023-10-06T04:58:28.757305+0000","last_undegraded":"2023-10-06T04:58:28.757305+0000","last_fullsized":"2023-10-06T04:58:28.757305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:55.102824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761641+0000","last_change":"2023-10-06T04:58:27.723817+0000","last_active":"2023-10-06T04:58:28.761641+0000","last_peered":"2023-10-06T04:58:28.761641+0000","last_clean":"2023-10-06T04:58:28.761641+0000","last_became_active":"2023-10-06T04:58:27.723533+0000","last_became_peered":"2023-10-06T04:58:27.723533+0000","last_unstale":"2023-10-06T04:58:28.761641+0000","last_undegraded":"2023-10-06T04:58:28.761641+0000","last_fullsized":"2023-10-06T04:58:28.761641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:42.171552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.737849+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:58:28.737849+0000","last_peered":"2023-10-06T04:58:28.737849+0000","last_clean":"2023-10-06T04:58:28.737849+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:58:28.737849+0000","last_undegraded":"2023-10-06T04:58:28.737849+0000","last_fullsized":"2023-10-06T04:58:28.737849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760260+0000","last_change":"2023-10-06T04:58:28.759047+0000","last_active":"2023-10-06T04:58:28.760260+0000","last_peered":"2023-10-06T04:58:28.760260+0000","last_clean":"2023-10-06T04:58:28.760260+0000","last_became_active":"2023-10-06T04:58:28.758695+0000","last_became_peered":"2023-10-06T04:58:28.758695+0000","last_unstale":"2023-10-06T04:58:28.760260+0000","last_undegraded":"2023-10-06T04:58:28.760260+0000","last_fullsized":"2023-10-06T04:58:28.760260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:28.360451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.746772+0000","last_change":"2023-10-06T04:58:27.726169+0000","last_active":"2023-10-06T04:58:28.746772+0000","last_peered":"2023-10-06T04:58:28.746772+0000","last_clean":"2023-10-06T04:58:28.746772+0000","last_became_active":"2023-10-06T04:58:27.725663+0000","last_became_peered":"2023-10-06T04:58:27.725663+0000","last_unstale":"2023-10-06T04:58:28.746772+0000","last_undegraded":"2023-10-06T04:58:28.746772+0000","last_fullsized":"2023-10-06T04:58:28.746772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:33.983845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.738021+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:58:28.738021+0000","last_peered":"2023-10-06T04:58:28.738021+0000","last_clean":"2023-10-06T04:58:28.738021+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:58:28.738021+0000","last_undegraded":"2023-10-06T04:58:28.738021+0000","last_fullsized":"2023-10-06T04:58:28.738021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761268+0000","last_change":"2023-10-06T04:58:28.756810+0000","last_active":"2023-10-06T04:58:28.761268+0000","last_peered":"2023-10-06T04:58:28.761268+0000","last_clean":"2023-10-06T04:58:28.761268+0000","last_became_active":"2023-10-06T04:58:28.756551+0000","last_became_peered":"2023-10-06T04:58:28.756551+0000","last_unstale":"2023-10-06T04:58:28.761268+0000","last_undegraded":"2023-10-06T04:58:28.761268+0000","last_fullsized":"2023-10-06T04:58:28.761268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:55.440657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.767735+0000","last_change":"2023-10-06T04:58:27.721320+0000","last_active":"2023-10-06T04:58:28.767735+0000","last_peered":"2023-10-06T04:58:28.767735+0000","last_clean":"2023-10-06T04:58:28.767735+0000","last_became_active":"2023-10-06T04:58:27.721062+0000","last_became_peered":"2023-10-06T04:58:27.721062+0000","last_unstale":"2023-10-06T04:58:28.767735+0000","last_undegraded":"2023-10-06T04:58:28.767735+0000","last_fullsized":"2023-10-06T04:58:28.767735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:06.202911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741689+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:58:28.741689+0000","last_peered":"2023-10-06T04:58:28.741689+0000","last_clean":"2023-10-06T04:58:28.741689+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:58:28.741689+0000","last_undegraded":"2023-10-06T04:58:28.741689+0000","last_fullsized":"2023-10-06T04:58:28.741689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.761455+0000","last_change":"2023-10-06T04:58:28.760435+0000","last_active":"2023-10-06T04:58:28.761455+0000","last_peered":"2023-10-06T04:58:28.761455+0000","last_clean":"2023-10-06T04:58:28.761455+0000","last_became_active":"2023-10-06T04:58:28.760148+0000","last_became_peered":"2023-10-06T04:58:28.760148+0000","last_unstale":"2023-10-06T04:58:28.761455+0000","last_undegraded":"2023-10-06T04:58:28.761455+0000","last_fullsized":"2023-10-06T04:58:28.761455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:59.336531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.740584+0000","last_change":"2023-10-06T04:58:27.727369+0000","last_active":"2023-10-06T04:58:28.740584+0000","last_peered":"2023-10-06T04:58:28.740584+0000","last_clean":"2023-10-06T04:58:28.740584+0000","last_became_active":"2023-10-06T04:58:27.725301+0000","last_became_peered":"2023-10-06T04:58:27.725301+0000","last_unstale":"2023-10-06T04:58:28.740584+0000","last_undegraded":"2023-10-06T04:58:28.740584+0000","last_fullsized":"2023-10-06T04:58:28.740584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:59.163008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.741153+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:58:28.741153+0000","last_peered":"2023-10-06T04:58:28.741153+0000","last_clean":"2023-10-06T04:58:28.741153+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:58:28.741153+0000","last_undegraded":"2023-10-06T04:58:28.741153+0000","last_fullsized":"2023-10-06T04:58:28.741153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.760279+0000","last_change":"2023-10-06T04:58:28.752729+0000","last_active":"2023-10-06T04:58:28.760279+0000","last_peered":"2023-10-06T04:58:28.760279+0000","last_clean":"2023-10-06T04:58:28.760279+0000","last_became_active":"2023-10-06T04:58:28.752451+0000","last_became_peered":"2023-10-06T04:58:28.752451+0000","last_unstale":"2023-10-06T04:58:28.760279+0000","last_undegraded":"2023-10-06T04:58:28.760279+0000","last_fullsized":"2023-10-06T04:58:28.760279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:27.707117+0000","last_clean_scrub_stamp":"2023-10-06T04:58:27.707117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:20.677651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.764064+0000","last_change":"2023-10-06T04:58:27.722935+0000","last_active":"2023-10-06T04:58:28.764064+0000","last_peered":"2023-10-06T04:58:28.764064+0000","last_clean":"2023-10-06T04:58:28.764064+0000","last_became_active":"2023-10-06T04:58:27.722662+0000","last_became_peered":"2023-10-06T04:58:27.722662+0000","last_unstale":"2023-10-06T04:58:28.764064+0000","last_undegraded":"2023-10-06T04:58:28.764064+0000","last_fullsized":"2023-10-06T04:58:28.764064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:58:26.705127+0000","last_clean_scrub_stamp":"2023-10-06T04:58:26.705127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:23.234133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T04:58:28.743398+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:58:28.743398+0000","last_peered":"2023-10-06T04:58:28.743398+0000","last_clean":"2023-10-06T04:58:28.743398+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:58:28.743398+0000","last_undegraded":"2023-10-06T04:58:28.743398+0000","last_fullsized":"2023-10-06T04:58:28.743398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307060,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064780,"statfs":{"total":96636764160,"available":96322334720,"internally_reserved":0,"allocated":1617920,"data_stored":1045778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.73399999999999999},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73199999999999998,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":1.073},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.51600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":1.111},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.58999999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.503},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.41799999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":1.214},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302412,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069428,"statfs":{"total":96636764160,"available":96327094272,"internally_reserved":0,"allocated":1052672,"data_stored":465460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.52200000000000002},"min":{"1min":0.54900000000000004,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.81200000000000006,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.57199999999999995},"min":{"1min":0.56100000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.39800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":1.234,"15min":1.234},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.55200000000000005},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81000000000000005,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75800000000000001,"5min":3.669,"15min":3.669},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.88}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307068,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064772,"statfs":{"total":96636764160,"available":96322326528,"internally_reserved":0,"allocated":1626112,"data_stored":1052433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.23000000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.27900000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.501,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.70999999999999996},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.85499999999999998,"15min":0.92900000000000005},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.64100000000000001},"min":{"1min":0.47499999999999998,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.998,"15min":0.998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.65800000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":1.0900000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80600000000000005,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.026},"last":0.755},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.79400000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306512,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065328,"statfs":{"total":96636764160,"available":96322895872,"internally_reserved":0,"allocated":1056768,"data_stored":465562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.505,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57699999999999996,"5min":0.628,"15min":0.628},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.28899999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.32000000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38900000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90700000000000003,"5min":1.022,"15min":1.022},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.495,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.443}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.5,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96599999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.995,"15min":0.995},"last":0.51100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302392,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069448,"statfs":{"total":96636764160,"available":96327114752,"internally_reserved":0,"allocated":1032192,"data_stored":463348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.51200000000000001,"5min":0.747,"15min":1.048},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.42599999999999999,"15min":0.39000000000000001},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":0.61699999999999999,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.36099999999999999},"min":{"1min":0.29499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.5,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306488,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065352,"statfs":{"total":96636764160,"available":96322920448,"internally_reserved":0,"allocated":1032192,"data_stored":463348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38700000000000001,"15min":0.40600000000000003},"min":{"1min":0.251,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":2.008,"15min":2.008},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41299999999999998,"15min":0.56499999999999995},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.314}]},{"osd":1,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54500000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42399999999999999,"15min":0.46000000000000002},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.64600000000000002,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.443}]},{"osd":2,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":1.379,"15min":5.069},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":2.246,"15min":2.246},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.373}]},{"osd":5,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83199999999999996,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82199999999999995,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.79400000000000004},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.40400000000000003},"max":{"1min":0.76200000000000001,"5min":1.964,"15min":6.367},"last":0.502},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65000000000000002,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":1.3,"15min":5},"last":0.54600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306484,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065356,"statfs":{"total":96636764160,"available":96322924544,"internally_reserved":0,"allocated":1028096,"data_stored":456867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.78500000000000003},"min":{"1min":0.38200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.82099999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.80400000000000005,"15min":5.415},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.66700000000000004},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":4.202},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":1.044,"15min":4.2130000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42599999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.77600000000000002,"5min":0.86199999999999999,"15min":0.872},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.313},"max":{"1min":0.84399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.55600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306484,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065356,"statfs":{"total":96636764160,"available":96322924544,"internally_reserved":0,"allocated":1028096,"data_stored":461995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.67600000000000005},"min":{"1min":0.36899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.625},"min":{"1min":0.436,"5min":0.247,"15min":0.247},"max":{"1min":0.76000000000000001,"5min":0.80800000000000005,"15min":3.7679999999999998},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.70799999999999996},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":6.7949999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.75800000000000001},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.93799999999999994},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":9.7210000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.93500000000000005},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":9.7539999999999996},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.504},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45600000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.34200000000000003},"max":{"1min":0.66300000000000003,"5min":0.746,"15min":0.746},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80100000000000005},"last":0.432}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.49399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:58:33.248 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:58:33.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:58:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.402+0000 7f5e55df6700 1 Processor -- start 2023-10-06T04:58:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.402+0000 7f5e55df6700 1 -- start start 2023-10-06T04:58:33.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e55df6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5e501539b0 0x7f5e5015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e55df6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e50153d80 0x7f5e500aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e55df6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e500ce390 con 0x7f5e5015d180 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e55df6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5015aaf0 con 0x7f5e50153d80 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e55df6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5015d680 con 0x7f5e501539b0 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e555f5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e50153d80 0x7f5e500aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e555f5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e50153d80 0x7f5e500aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51238/0 (socket says 172.21.15.3:51238) 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.403+0000 7f5e555f5700 1 -- 172.21.15.3:0/3196682606 learned_addr learned my addr 172.21.15.3:0/3196682606 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e54df4700 1 --2- 172.21.15.3:0/3196682606 >> v2:172.21.15.153:3300/0 conn(0x7f5e501539b0 0x7f5e5015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e555f5700 1 -- 172.21.15.3:0/3196682606 >> v2:172.21.15.153:3300/0 conn(0x7f5e501539b0 msgr2=0x7f5e5015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e555f5700 1 --2- 172.21.15.3:0/3196682606 >> v2:172.21.15.153:3300/0 conn(0x7f5e501539b0 0x7f5e5015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e555f5700 1 -- 172.21.15.3:0/3196682606 >> v1:172.21.15.153:6789/0 conn(0x7f5e5015d180 legacy=0x7f5e500cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e555f5700 1 -- 172.21.15.3:0/3196682606 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5015d960 con 0x7f5e50153d80 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e555f5700 1 --2- 172.21.15.3:0/3196682606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e50153d80 0x7f5e500aedf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5e50071310 tx=0x7f5e40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d67169100597b32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e4f7fe700 1 -- 172.21.15.3:0/3196682606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000b860 con 0x7f5e50153d80 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e4f7fe700 1 -- 172.21.15.3:0/3196682606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4000b9c0 con 0x7f5e50153d80 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.404+0000 7f5e4f7fe700 1 -- 172.21.15.3:0/3196682606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000bd60 con 0x7f5e50153d80 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 -- 172.21.15.3:0/3196682606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e50153d80 msgr2=0x7f5e500aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3196682606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e50153d80 0x7f5e500aedf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5e50071310 tx=0x7f5e40009250 comp rx=0 tx=0).stop 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 -- 172.21.15.3:0/3196682606 shutdown_connections 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3196682606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e50153d80 0x7f5e500aedf0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3196682606 >> v2:172.21.15.153:3300/0 conn(0x7f5e501539b0 0x7f5e5015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 -- 172.21.15.3:0/3196682606 >> 172.21.15.3:0/3196682606 conn(0x7f5e50070310 msgr2=0x7f5e50070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:33.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 -- 172.21.15.3:0/3196682606 shutdown_connections 2023-10-06T04:58:33.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.405+0000 7f5e55df6700 1 -- 172.21.15.3:0/3196682606 wait complete. 2023-10-06T04:58:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.406+0000 7f5e55df6700 1 Processor -- start 2023-10-06T04:58:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.406+0000 7f5e55df6700 1 -- start start 2023-10-06T04:58:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e501539b0 0x7f5e50159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e50153d80 0x7f5e500b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e5015d180 0x7f5e500b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500c8800 con 0x7f5e5015d180 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500c7950 con 0x7f5e50153d80 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500c7bf0 con 0x7f5e501539b0 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e5015d180 0x7f5e500b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e5015d180 0x7f5e500b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51246/0 (socket says 172.21.15.3:51246) 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 -- 172.21.15.3:0/3197489235 learned_addr learned my addr 172.21.15.3:0/3197489235 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 -- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e501539b0 msgr2=0x7f5e50159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e54df4700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e501539b0 0x7f5e50159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e4ffff700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e50153d80 0x7f5e500b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e501539b0 0x7f5e50159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 -- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e50153d80 msgr2=0x7f5e500b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e50153d80 0x7f5e500b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 -- 172.21.15.3:0/3197489235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400094f0 con 0x7f5e5015d180 2023-10-06T04:58:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e4ffff700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e50153d80 0x7f5e500b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e555f5700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e5015d180 0x7f5e500b3590 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009290 tx=0x7f5e40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e4cff9700 1 -- 172.21.15.3:0/3197489235 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4001a700 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e4cff9700 1 -- 172.21.15.3:0/3197489235 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4001a860 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e4cff9700 1 -- 172.21.15.3:0/3197489235 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4001ac20 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e500c7ed0 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.407+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e500c8490 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.408+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e500c8970 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.409+0000 7f5e4cff9700 1 -- 172.21.15.3:0/3197489235 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e4002d440 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.409+0000 7f5e4cff9700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e3c066f30 0x7f5e3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.409+0000 7f5e4cff9700 1 -- 172.21.15.3:0/3197489235 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e40086470 con 0x7f5e5015d180 2023-10-06T04:58:33.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.409+0000 7f5e54df4700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e3c066f30 0x7f5e3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.410+0000 7f5e54df4700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e3c066f30 0x7f5e3c0693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007460 tx=0x7f5e4800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.414+0000 7f5e4cff9700 1 -- 172.21.15.3:0/3197489235 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e40053580 con 0x7f5e5015d180 2023-10-06T04:58:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.688+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5e5004c380 con 0x7f5e5015d180 2023-10-06T04:58:33.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.692+0000 7f5e4cff9700 1 -- 172.21.15.3:0/3197489235 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5e40056ba0 con 0x7f5e5015d180 2023-10-06T04:58:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.693+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e3c066f30 msgr2=0x7f5e3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.693+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e3c066f30 0x7f5e3c0693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007460 tx=0x7f5e4800b040 comp rx=0 tx=0).stop 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.693+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e5015d180 msgr2=0x7f5e500b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.693+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e5015d180 0x7f5e500b3590 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009290 tx=0x7f5e40017000 comp rx=0 tx=0).stop 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 shutdown_connections 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5e3c066f30 0x7f5e3c0693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e5015d180 0x7f5e500b3590 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e501539b0 0x7f5e50159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 --2- 172.21.15.3:0/3197489235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e50153d80 0x7f5e500b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 >> 172.21.15.3:0/3197489235 conn(0x7f5e50070310 msgr2=0x7f5e5015acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 shutdown_connections 2023-10-06T04:58:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.694+0000 7f5e55df6700 1 -- 172.21.15.3:0/3197489235 wait complete. 2023-10-06T04:58:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T04:58:33.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.825+0000 7fe75a3ab700 1 Processor -- start 2023-10-06T04:58:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.825+0000 7fe75a3ab700 1 -- start start 2023-10-06T04:58:33.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe75a3ab700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe75406cfa0 0x7fe7540b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe75a3ab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe75406d370 0x7fe7540b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe75a3ab700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406c370 0x7fe75406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe75a3ab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe754158680 con 0x7fe75406d370 2023-10-06T04:58:33.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe75a3ab700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe754157a00 con 0x7fe75406cfa0 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe75a3ab700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe754157ce0 con 0x7fe75406c370 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe75406d370 0x7fe7540b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe75406d370 0x7fe7540b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51262/0 (socket says 172.21.15.3:51262) 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 -- 172.21.15.3:0/2755278811 learned_addr learned my addr 172.21.15.3:0/2755278811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe7593a9700 1 --2- 172.21.15.3:0/2755278811 >> v2:172.21.15.153:3300/0 conn(0x7fe75406cfa0 0x7fe7540b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe759baa700 1 --2- 172.21.15.3:0/2755278811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406c370 0x7fe75406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 -- 172.21.15.3:0/2755278811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406c370 msgr2=0x7fe75406d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 --2- 172.21.15.3:0/2755278811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406c370 0x7fe75406d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 -- 172.21.15.3:0/2755278811 >> v2:172.21.15.153:3300/0 conn(0x7fe75406cfa0 msgr2=0x7fe7540b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 --2- 172.21.15.3:0/2755278811 >> v2:172.21.15.153:3300/0 conn(0x7fe75406cfa0 0x7fe7540b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe758ba8700 1 -- 172.21.15.3:0/2755278811 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754157fc0 con 0x7fe75406d370 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.826+0000 7fe7593a9700 1 --2- 172.21.15.3:0/2755278811 >> v2:172.21.15.153:3300/0 conn(0x7fe75406cfa0 0x7fe7540b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe758ba8700 1 --2- 172.21.15.3:0/2755278811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540b2600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe75406c190 tx=0x7fe748009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5457264bf9cb5314 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe753fff700 1 -- 172.21.15.3:0/2755278811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74800b860 con 0x7fe75406d370 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe753fff700 1 -- 172.21.15.3:0/2755278811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74800b9c0 con 0x7fe75406d370 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe753fff700 1 -- 172.21.15.3:0/2755278811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74800bd60 con 0x7fe75406d370 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 -- 172.21.15.3:0/2755278811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 msgr2=0x7fe7540b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/2755278811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540b2600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe75406c190 tx=0x7fe748009250 comp rx=0 tx=0).stop 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 -- 172.21.15.3:0/2755278811 shutdown_connections 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/2755278811 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406c370 0x7fe75406d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/2755278811 >> v2:172.21.15.153:3300/0 conn(0x7fe75406cfa0 0x7fe7540b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/2755278811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540b2600 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 -- 172.21.15.3:0/2755278811 >> 172.21.15.3:0/2755278811 conn(0x7fe7540ccfd0 msgr2=0x7fe7540cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 -- 172.21.15.3:0/2755278811 shutdown_connections 2023-10-06T04:58:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.827+0000 7fe75a3ab700 1 -- 172.21.15.3:0/2755278811 wait complete. 2023-10-06T04:58:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.828+0000 7fe75a3ab700 1 Processor -- start 2023-10-06T04:58:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe75a3ab700 1 -- start start 2023-10-06T04:58:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe75a3ab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75406c370 0x7fe7541490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe75a3ab700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406cfa0 0x7fe754146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe75a3ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe75a3ab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75414b760 con 0x7fe75406d370 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe75a3ab700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75414bf00 con 0x7fe75406c370 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe75a3ab700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe754148cd0 con 0x7fe75406cfa0 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.829+0000 7fe759baa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51278/0 (socket says 172.21.15.3:51278) 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 -- 172.21.15.3:0/3078559129 learned_addr learned my addr 172.21.15.3:0/3078559129 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe758ba8700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406cfa0 0x7fe754146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe7593a9700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75406c370 0x7fe7541490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 -- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406cfa0 msgr2=0x7fe754146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406cfa0 0x7fe754146a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 -- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75406c370 msgr2=0x7fe7541490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75406c370 0x7fe7541490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 -- 172.21.15.3:0/3078559129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480094f0 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe759baa700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540ad9e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe744009fa0 tx=0x7fe744009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe7517fa700 1 -- 172.21.15.3:0/3078559129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400ed20 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe7517fa700 1 -- 172.21.15.3:0/3078559129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74400ee80 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75414b940 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe7517fa700 1 -- 172.21.15.3:0/3078559129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744014670 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.830+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe754149620 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.831+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75404c380 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.831+0000 7fe7517fa700 1 -- 172.21.15.3:0/3078559129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe744014a20 con 0x7fe75406d370 2023-10-06T04:58:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.832+0000 7fe7517fa700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe740066f30 0x7fe7400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.832+0000 7fe7517fa700 1 -- 172.21.15.3:0/3078559129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe744084df0 con 0x7fe75406d370 2023-10-06T04:58:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.832+0000 7fe7593a9700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe740066f30 0x7fe7400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.832+0000 7fe7593a9700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe740066f30 0x7fe7400693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe74c006f00 tx=0x7fe74c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:33.836+0000 7fe7517fa700 1 -- 172.21.15.3:0/3078559129 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe744051f00 con 0x7fe75406d370 2023-10-06T04:58:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.071+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7541582a0 con 0x7fe75406d370 2023-10-06T04:58:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.072+0000 7fe7517fa700 1 -- 172.21.15.3:0/3078559129 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe744055520 con 0x7fe75406d370 2023-10-06T04:58:34.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:34.074 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:58:28.114721+0000","modified":"2023-10-06T04:58:31.748997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.153:6835/2492546877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2492546877},{"type":"v1","addr":"172.21.15.153:6835","nonce":2492546877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T04:58:34.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe740066f30 msgr2=0x7fe7400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe740066f30 0x7fe7400693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe74c006f00 tx=0x7fe74c008040 comp rx=0 tx=0).stop 2023-10-06T04:58:34.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 msgr2=0x7fe7540ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540ad9e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe744009fa0 tx=0x7fe744009900 comp rx=0 tx=0).stop 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 shutdown_connections 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe740066f30 0x7fe7400693e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75406c370 0x7fe7541490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe75406d370 0x7fe7540ad9e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 --2- 172.21.15.3:0/3078559129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe75406cfa0 0x7fe754146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 >> 172.21.15.3:0/3078559129 conn(0x7fe7540ccfd0 msgr2=0x7fe75406d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 shutdown_connections 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.074+0000 7fe75a3ab700 1 -- 172.21.15.3:0/3078559129 wait complete. 2023-10-06T04:58:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 50 2023-10-06T04:58:34.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:58:28.114721+0000', 'modified': '2023-10-06T04:58:31.748997+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6717}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6717': {'gid': 6717, 'name': 'd', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.153:6835/2492546877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 2492546877}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 2492546877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:58:34.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:58:34.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T04:58:34.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.614+0000 7f44f4cb7700 1 Processor -- start 2023-10-06T04:58:34.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.614+0000 7f44f4cb7700 1 -- start start 2023-10-06T04:58:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44f4cb7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f0153bd0 0x7f44f015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44f4cb7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44f4cb7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f015d9f0 con 0x7f44f015a2e0 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44f4cb7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f015a710 con 0x7f44f0153bd0 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44f4cb7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f00b0aa0 con 0x7f44f0153fa0 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44ee59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f0153bd0 0x7f44f015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44ee59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f0153bd0 0x7f44f015de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51286/0 (socket says 172.21.15.3:51286) 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.615+0000 7f44ee59c700 1 -- 172.21.15.3:0/818148550 learned_addr learned my addr 172.21.15.3:0/818148550 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44edd9b700 1 --2- 172.21.15.3:0/818148550 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ee59c700 1 -- 172.21.15.3:0/818148550 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 msgr2=0x7f44f0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ee59c700 1 --2- 172.21.15.3:0/818148550 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ee59c700 1 -- 172.21.15.3:0/818148550 >> v1:172.21.15.153:6789/0 conn(0x7f44f015a2e0 legacy=0x7f44f006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ee59c700 1 -- 172.21.15.3:0/818148550 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f015a980 con 0x7f44f0153bd0 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ee59c700 1 --2- 172.21.15.3:0/818148550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f015de80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f44e80099b0 tx=0x7f44e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748598cb0bcfcb72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ed59a700 1 -- 172.21.15.3:0/818148550 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800b820 con 0x7f44f0153bd0 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ed59a700 1 -- 172.21.15.3:0/818148550 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e800b980 con 0x7f44f0153bd0 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.616+0000 7f44ed59a700 1 -- 172.21.15.3:0/818148550 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800bd20 con 0x7f44f0153bd0 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 -- 172.21.15.3:0/818148550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 msgr2=0x7f44f015de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/818148550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f015de80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f44e80099b0 tx=0x7f44e8009250 comp rx=0 tx=0).stop 2023-10-06T04:58:34.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 -- 172.21.15.3:0/818148550 shutdown_connections 2023-10-06T04:58:34.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/818148550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f015de80 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/818148550 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 -- 172.21.15.3:0/818148550 >> 172.21.15.3:0/818148550 conn(0x7f44f00cdc30 msgr2=0x7f44f006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 -- 172.21.15.3:0/818148550 shutdown_connections 2023-10-06T04:58:34.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.617+0000 7f44f4cb7700 1 -- 172.21.15.3:0/818148550 wait complete. 2023-10-06T04:58:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 Processor -- start 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 -- start start 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f00cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f015a2e0 0x7f44f0159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f0154ca0 con 0x7f44f0153bd0 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f0154a90 con 0x7f44f015a2e0 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.618+0000 7f44f4cb7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f01594a0 con 0x7f44f0153fa0 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f00cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f00cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51288/0 (socket says 172.21.15.3:51288) 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 -- 172.21.15.3:0/2464612746 learned_addr learned my addr 172.21.15.3:0/2464612746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44eed9d700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f015a2e0 0x7f44f0159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44edd9b700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 -- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 msgr2=0x7f44f0154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 -- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f015a2e0 msgr2=0x7f44f0159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f015a2e0 0x7f44f0159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 -- 172.21.15.3:0/2464612746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80094f0 con 0x7f44f0153bd0 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44ee59c700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f00cc9a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009290 tx=0x7f44e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44e67fc700 1 -- 172.21.15.3:0/2464612746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8011d70 con 0x7f44f0153bd0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44e67fc700 1 -- 172.21.15.3:0/2464612746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e8020070 con 0x7f44f0153bd0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44e67fc700 1 -- 172.21.15.3:0/2464612746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e801b6a0 con 0x7f44f0153bd0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f0158b70 con 0x7f44f0153bd0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.619+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f0159130 con 0x7f44f0153bd0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.621+0000 7f44e67fc700 1 -- 172.21.15.3:0/2464612746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44e8022030 con 0x7f44f0153bd0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.621+0000 7f44e67fc700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44d0066f30 0x7f44d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.621+0000 7f44e67fc700 1 -- 172.21.15.3:0/2464612746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44e8086690 con 0x7f44f0153bd0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.622+0000 7f44edd9b700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44d0066f30 0x7f44d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.622+0000 7f44edd9b700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44d0066f30 0x7f44d00693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f44f006b9f0 tx=0x7f44d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.623+0000 7f44fdfafb80 1 Processor -- start 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.623+0000 7f44fdfafb80 1 -- start start 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe94ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe90a9a0 0x5597fe907580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe908bd0 0x5597fe907ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe887c80 con 0x5597fe94ea90 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe85ea70 con 0x5597fe90a9a0 2023-10-06T04:58:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597fe85c270 con 0x5597fe908bd0 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe94ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe94ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51290/0 (socket says 172.21.15.3:51290) 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 -- 172.21.15.3:0/2346461261 learned_addr learned my addr 172.21.15.3:0/2346461261 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44edd9b700 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe90a9a0 0x5597fe907580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44eed9d700 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe908bd0 0x5597fe907ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 -- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe908bd0 msgr2=0x5597fe907ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe908bd0 0x5597fe907ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 -- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe90a9a0 msgr2=0x5597fe907580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe90a9a0 0x5597fe907580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 -- 172.21.15.3:0/2346461261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597fe6f2270 con 0x5597fe94ea90 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44eed9d700 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe908bd0 0x5597fe907ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.624+0000 7f44ee59c700 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe94ee60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f44e80538c0 tx=0x7f44e8084910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c542deb03e8008 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44cf7fe700 1 -- 172.21.15.3:0/2346461261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8053700 con 0x5597fe94ea90 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44cf7fe700 1 -- 172.21.15.3:0/2346461261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e8006e90 con 0x5597fe94ea90 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44cf7fe700 1 -- 172.21.15.3:0/2346461261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e805a610 con 0x5597fe94ea90 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 -- 172.21.15.3:0/2346461261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 msgr2=0x5597fe94ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe94ee60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f44e80538c0 tx=0x7f44e8084910 comp rx=0 tx=0).stop 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 -- 172.21.15.3:0/2346461261 shutdown_connections 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe908bd0 0x5597fe907ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe94ee60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/2346461261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe90a9a0 0x5597fe907580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 -- 172.21.15.3:0/2346461261 >> 172.21.15.3:0/2346461261 conn(0x5597fe909070 msgr2=0x5597fe909440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 -- 172.21.15.3:0/2346461261 shutdown_connections 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.625+0000 7f44fdfafb80 1 -- 172.21.15.3:0/2346461261 wait complete. 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.626+0000 7f44fdfafb80 1 -- start start 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.626+0000 7f44fdfafb80 1 Processor -- start 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe92f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe90a9a0 0x5597fe932590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe94ea90 0x5597fe933940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe858060 con 0x5597fe90a9a0 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe803890 con 0x5597fe908bd0 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597fe81eea0 con 0x5597fe94ea90 2023-10-06T04:58:34.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44fdfafb80 10 client.0 _ll_register_callbacks cb 0x5597fe6f4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe90a9a0 0x5597fe932590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe90a9a0 0x5597fe932590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51292/0 (socket says 172.21.15.3:51292) 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 -- 172.21.15.3:0/3393057949 learned_addr learned my addr 172.21.15.3:0/3393057949 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44eed9d700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe94ea90 0x5597fe933940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44ee59c700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe92f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 -- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe94ea90 msgr2=0x5597fe933940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe94ea90 0x5597fe933940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 -- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 msgr2=0x5597fe92f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe92f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44edd9b700 1 -- 172.21.15.3:0/3393057949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f0159130 con 0x5597fe90a9a0 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.627+0000 7f44ee59c700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe92f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44edd9b700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe90a9a0 0x5597fe932590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f44d80145e0 tx=0x7f44d8018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784995e46cfb052e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44ccff9700 1 -- 172.21.15.3:0/3393057949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8014e30 con 0x5597fe90a9a0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44ccff9700 1 -- 172.21.15.3:0/3393057949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44d8021630 con 0x5597fe90a9a0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597fe8fc690 con 0x5597fe90a9a0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44ccff9700 1 -- 172.21.15.3:0/3393057949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44d8005a00 con 0x5597fe90a9a0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 10 client.6852 fetch_fsmap learned FSMap version 50 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x5597fe694fe0 con 0x5597fe90a9a0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44ccff9700 1 -- 172.21.15.3:0/3393057949 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5597fe8fc690 con 0x5597fe90a9a0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44ccff9700 1 -- 172.21.15.3:0/3393057949 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f44d8005b60 con 0x5597fe90a9a0 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 10 client.6852 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T04:58:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 10 client.6852 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 20 client.6852 populate_metadata read hostname 'smithi003' 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe960290 0x5597fe960660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 4 client.6852 mds_command: new command op to 6717 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.628+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 --> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597fe8d95c0 con 0x5597fe960290 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.629+0000 7f44ee59c700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe960290 0x5597fe960660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.629+0000 7f44ccff9700 10 client.6852 ms_handle_connect on v2:172.21.15.153:6834/2492546877 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.629+0000 7f44ee59c700 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe960290 0x5597fe960660 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f9d70cf54601e server_cookie=4c5c614dd8805f79 in_seq=0 out_seq=0 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44ccff9700 1 -- 172.21.15.3:0/3393057949 <== mds.0 v2:172.21.15.153:6834/2492546877 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5597fe8d95c0 con 0x5597fe960290 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44ccff9700 10 client.6852 handle_command_reply: tid=0 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 client.6852 shutdown 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe90a9a0 msgr2=0x5597fe932590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe90a9a0 0x5597fe932590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f44d80145e0 tx=0x7f44d8018040 comp rx=0 tx=0).stop 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 shutdown_connections 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe960290 0x5597fe960660 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe92f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44ccff9700 0 client.6852 ms_handle_reset on v2:172.21.15.153:6834/2492546877 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe94ea90 0x5597fe933940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/3393057949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe90a9a0 0x5597fe932590 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 >> 172.21.15.3:0/3393057949 conn(0x5597fe909070 msgr2=0x5597fe9047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 shutdown_connections 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.630+0000 7f44fdfafb80 1 -- 172.21.15.3:0/3393057949 wait complete. 2023-10-06T04:58:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.631+0000 7f44fdfafb80 20 client.6852 trim_cache size 0 max 16384 2023-10-06T04:58:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 Processor -- start 2023-10-06T04:58:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 -- start start 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe908bd0 0x5597fe94b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe9047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe94ea90 0x5597fe904d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe6dae30 con 0x5597fe908bd0 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe951c80 con 0x5597fe94ea90 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597fe92f1b0 con 0x5597fe90a9a0 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe908bd0 0x5597fe94b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe908bd0 0x5597fe94b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51308/0 (socket says 172.21.15.3:51308) 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.648+0000 7f44ee59c700 1 -- 172.21.15.3:0/578431112 learned_addr learned my addr 172.21.15.3:0/578431112 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44edd9b700 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe9047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44eed9d700 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe94ea90 0x5597fe904d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44ee59c700 1 -- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 msgr2=0x5597fe9047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44ee59c700 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe9047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44ee59c700 1 -- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe94ea90 msgr2=0x5597fe904d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44ee59c700 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe94ea90 0x5597fe904d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44ee59c700 1 -- 172.21.15.3:0/578431112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597fe826780 con 0x5597fe908bd0 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44ee59c700 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe908bd0 0x5597fe94b0b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f44e80538c0 tx=0x7f44e8027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90263058b5c9a27c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44cffff700 1 -- 172.21.15.3:0/578431112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8073040 con 0x5597fe908bd0 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44cffff700 1 -- 172.21.15.3:0/578431112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e805e360 con 0x5597fe908bd0 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44cffff700 1 -- 172.21.15.3:0/578431112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e805a610 con 0x5597fe908bd0 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 -- 172.21.15.3:0/578431112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe908bd0 msgr2=0x5597fe94b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe908bd0 0x5597fe94b0b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f44e80538c0 tx=0x7f44e8027040 comp rx=0 tx=0).stop 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 -- 172.21.15.3:0/578431112 shutdown_connections 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe94ea90 0x5597fe904d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe908bd0 0x5597fe94b0b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/578431112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe9047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 -- 172.21.15.3:0/578431112 >> 172.21.15.3:0/578431112 conn(0x5597fe909070 msgr2=0x5597fe909440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 -- 172.21.15.3:0/578431112 shutdown_connections 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.649+0000 7f44fdfafb80 1 -- 172.21.15.3:0/578431112 wait complete. 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.650+0000 7f44fdfafb80 1 -- start start 2023-10-06T04:58:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.650+0000 7f44fdfafb80 1 Processor -- start 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe944260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe904bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44fdfafb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe92e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe9448a0 con 0x5597fe94ea90 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fe92f160 con 0x5597fe908bd0 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44fdfafb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597fe94d8e0 con 0x5597fe90a9a0 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44fdfafb80 10 client.0 _ll_register_callbacks cb 0x5597fe6f4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:58:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe92e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe92e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51312/0 (socket says 172.21.15.3:51312) 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 -- 172.21.15.3:0/513636666 learned_addr learned my addr 172.21.15.3:0/513636666 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44edd9b700 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe904bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44ee59c700 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe944260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 -- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 msgr2=0x5597fe904bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe904bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 -- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 msgr2=0x5597fe944260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe944260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.651+0000 7f44eed9d700 1 -- 172.21.15.3:0/513636666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597fe6f2270 con 0x5597fe94ea90 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44eed9d700 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe92e250 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f44e0016910 tx=0x7f44e000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7596a8b195e158ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:58:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44ccff9700 1 -- 172.21.15.3:0/513636666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e00186a0 con 0x5597fe94ea90 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44ccff9700 1 -- 172.21.15.3:0/513636666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44e0018800 con 0x5597fe94ea90 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44e8006e90 con 0x5597fe94ea90 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44ccff9700 1 -- 172.21.15.3:0/513636666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44e0018bd0 con 0x5597fe94ea90 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 10 client.6858 fetch_fsmap learned FSMap version 50 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x5597fe80d470 con 0x5597fe94ea90 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44ccff9700 1 -- 172.21.15.3:0/513636666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e8006e90 con 0x5597fe94ea90 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44ccff9700 1 -- 172.21.15.3:0/513636666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f44e0019460 con 0x5597fe94ea90 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 10 client.6858 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 10 client.6858 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 20 client.6858 populate_metadata read hostname 'smithi003' 2023-10-06T04:58:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe9b2f10 0x5597fe9b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 4 client.6858 mds_command: new command op to 6717 tid=0[{"prefix": "status"}] 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 --> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5597fe7fae00 con 0x5597fe9b2f10 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.652+0000 7f44ee59c700 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe9b2f10 0x5597fe9b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44ee59c700 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe9b2f10 0x5597fe9b32e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9becf52b542e73f server_cookie=d9785722f16388ad in_seq=0 out_seq=0 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44ccff9700 10 client.6858 ms_handle_connect on v2:172.21.15.153:6834/2492546877 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44ccff9700 1 -- 172.21.15.3:0/513636666 <== mds.0 v2:172.21.15.153:6834/2492546877 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5597fe7fae00 con 0x5597fe9b2f10 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44ccff9700 10 client.6858 handle_command_reply: tid=0 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 client.6858 shutdown 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 msgr2=0x5597fe92e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe92e250 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f44e0016910 tx=0x7f44e000e040 comp rx=0 tx=0).stop 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 shutdown_connections 2023-10-06T04:58:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:6834/2492546877,v1:172.21.15.153:6835/2492546877] conn(0x5597fe9b2f10 0x5597fe9b32e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597fe908bd0 0x5597fe944260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44ccff9700 0 client.6858 ms_handle_reset on v2:172.21.15.153:6834/2492546877 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5597fe94ea90 0x5597fe92e250 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 --2- 172.21.15.3:0/513636666 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5597fe90a9a0 0x5597fe904bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 >> 172.21.15.3:0/513636666 conn(0x5597fe909070 msgr2=0x5597fe931b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 shutdown_connections 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 1 -- 172.21.15.3:0/513636666 wait complete. 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.653+0000 7f44fdfafb80 20 client.6858 trim_cache size 0 max 16384 2023-10-06T04:58:34.662 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6717, 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 5.9106173350000004, 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 50, 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 84, 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 84, 2023-10-06T04:58:34.663 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 17.996371478 2023-10-06T04:58:34.664 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:58:34.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44d0066f30 msgr2=0x7f44d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44d0066f30 0x7f44d00693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f44f006b9f0 tx=0x7f44d8008040 comp rx=0 tx=0).stop 2023-10-06T04:58:34.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 msgr2=0x7f44f00cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f00cc9a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009290 tx=0x7f44e8016000 comp rx=0 tx=0).stop 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 shutdown_connections 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44d0066f30 0x7f44d00693e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f015a2e0 0x7f44f0159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f0153bd0 0x7f44f00cc9a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 --2- 172.21.15.3:0/2464612746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0153fa0 0x7f44f0154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 >> 172.21.15.3:0/2464612746 conn(0x7f44f00cdc30 msgr2=0x7f44f00af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 shutdown_connections 2023-10-06T04:58:34.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.654+0000 7f44f4cb7700 1 -- 172.21.15.3:0/2464612746 wait complete. 2023-10-06T04:58:34.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:58:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.785+0000 7fde9571a700 1 Processor -- start 2023-10-06T04:58:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.785+0000 7fde9571a700 1 -- start start 2023-10-06T04:58:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde9571a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157400 0x7fde900b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde9571a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9015aeb0 con 0x7fde900b3b70 2023-10-06T04:58:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde9571a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9015a7a0 con 0x7fde90157030 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde9571a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9015aa80 con 0x7fde90157400 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fde900b3b70 0x7fde9006e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33474/0 (socket says 172.21.15.3:33474) 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8f7fe700 1 -- 172.21.15.3:0/2644242655 learned_addr learned my addr 172.21.15.3:0/2644242655 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8e7fc700 1 --2- 172.21.15.3:0/2644242655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157400 0x7fde900b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021505036 0 0) 0x7fde9015aeb0 con 0x7fde900b3b70 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde7c003570 con 0x7fde900b3b70 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137855596 0 0) 0x7fde9015a7a0 con 0x7fde90157030 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde9015aeb0 con 0x7fde90157030 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8e7fc700 1 -- 172.21.15.3:0/2644242655 >> v1:172.21.15.153:6789/0 conn(0x7fde90157030 legacy=0x7fde90159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878271248 0 0) 0x7fde7c003570 con 0x7fde900b3b70 2023-10-06T04:58:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8e7fc700 1 -- 172.21.15.3:0/2644242655 >> v1:172.21.15.3:6789/0 conn(0x7fde900b3b70 legacy=0x7fde9006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.786+0000 7fde8e7fc700 1 -- 172.21.15.3:0/2644242655 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9006dba0 con 0x7fde90157400 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde8e7fc700 1 --2- 172.21.15.3:0/2644242655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157400 0x7fde900b3630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde78009910 tx=0x7fde780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbdd9de675e00a25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88001470 con 0x7fde90157400 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde78011420 con 0x7fde90157400 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde8dffb700 1 -- 172.21.15.3:0/2644242655 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde780117a0 con 0x7fde90157400 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde9571a700 1 -- 172.21.15.3:0/2644242655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157400 msgr2=0x7fde900b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde9571a700 1 --2- 172.21.15.3:0/2644242655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157400 0x7fde900b3630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde78009910 tx=0x7fde780091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde9571a700 1 -- 172.21.15.3:0/2644242655 shutdown_connections 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde9571a700 1 --2- 172.21.15.3:0/2644242655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157400 0x7fde900b3630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde9571a700 1 -- 172.21.15.3:0/2644242655 >> 172.21.15.3:0/2644242655 conn(0x7fde900ce270 msgr2=0x7fde900ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde9571a700 1 -- 172.21.15.3:0/2644242655 shutdown_connections 2023-10-06T04:58:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.787+0000 7fde9571a700 1 -- 172.21.15.3:0/2644242655 wait complete. 2023-10-06T04:58:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 Processor -- start 2023-10-06T04:58:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 -- start start 2023-10-06T04:58:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde900b3b70 0x7fde900c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157030 0x7fde90159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90157400 0x7fde90159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c92f0 con 0x7fde900b3b70 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900ca110 con 0x7fde90157400 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.789+0000 7fde9571a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c9500 con 0x7fde90157030 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde900b3b70 0x7fde900c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde900b3b70 0x7fde900c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51324/0 (socket says 172.21.15.3:51324) 2023-10-06T04:58:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 -- 172.21.15.3:0/1630999133 learned_addr learned my addr 172.21.15.3:0/1630999133 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8f7fe700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90157400 0x7fde90159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8e7fc700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157030 0x7fde90159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 -- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157030 msgr2=0x7fde90159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157030 0x7fde90159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 -- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90157400 msgr2=0x7fde90159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90157400 0x7fde90159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 -- 172.21.15.3:0/1630999133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde78009420 con 0x7fde900b3b70 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8f7fe700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90157400 0x7fde90159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde8effd700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde900b3b70 0x7fde900c8db0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fde8000cb40 tx=0x7fde80002810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde86ffd700 1 -- 172.21.15.3:0/1630999133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000a7e0 con 0x7fde900b3b70 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde86ffd700 1 -- 172.21.15.3:0/1630999133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000a960 con 0x7fde900b3b70 2023-10-06T04:58:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900c97e0 con 0x7fde900b3b70 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde86ffd700 1 -- 172.21.15.3:0/1630999133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000ad20 con 0x7fde900b3b70 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.790+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde900c9da0 con 0x7fde900b3b70 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.791+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde900644c0 con 0x7fde900b3b70 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.792+0000 7fde86ffd700 1 -- 172.21.15.3:0/1630999133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde80012630 con 0x7fde900b3b70 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.792+0000 7fde86ffd700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde70066f60 0x7fde70069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.792+0000 7fde86ffd700 1 -- 172.21.15.3:0/1630999133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde80084aa0 con 0x7fde900b3b70 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.792+0000 7fde8e7fc700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde70066f60 0x7fde70069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.793+0000 7fde8e7fc700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde70066f60 0x7fde70069410 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fde780091e0 tx=0x7fde78016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:34.796+0000 7fde86ffd700 1 -- 172.21.15.3:0/1630999133 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde80051bb0 con 0x7fde900b3b70 2023-10-06T04:58:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.036+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fde900cf200 con 0x7fde900b3b70 2023-10-06T04:58:35.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.036+0000 7fde86ffd700 1 -- 172.21.15.3:0/1630999133 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7fde800551d0 con 0x7fde900b3b70 2023-10-06T04:58:35.038 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:58:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde70066f60 msgr2=0x7fde70069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde70066f60 0x7fde70069410 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fde780091e0 tx=0x7fde78016000 comp rx=0 tx=0).stop 2023-10-06T04:58:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde900b3b70 msgr2=0x7fde900c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde900b3b70 0x7fde900c8db0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fde8000cb40 tx=0x7fde80002810 comp rx=0 tx=0).stop 2023-10-06T04:58:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 shutdown_connections 2023-10-06T04:58:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde70066f60 0x7fde70069410 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90157400 0x7fde90159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde900b3b70 0x7fde900c8db0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 --2- 172.21.15.3:0/1630999133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde90157030 0x7fde90159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 >> 172.21.15.3:0/1630999133 conn(0x7fde900ce270 msgr2=0x7fde901556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 shutdown_connections 2023-10-06T04:58:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:35.038+0000 7fde9571a700 1 -- 172.21.15.3:0/1630999133 wait complete. 2023-10-06T04:58:35.052 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:58:35.052 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:58:35.052 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:58:35.052 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:58:35.053 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:35.053 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:58:35.053 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:58:35.053 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:58:35.053 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:58:35.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:35.053 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:58:35.068 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:35.069 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42115sec preferred_lft 42115sec 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:58:35.070 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:58:35.071 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:35.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:35.072 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:58:35.133 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:58:35.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:35.135 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:58:35.191 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:58:35.191 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:58:35.191 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:58:35.191 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:58:35.191 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:58:35.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:35.192 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:58:35.192 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:58:35.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:58:35.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:58:35.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:58:35.192 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:58:35.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:35.294 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:58:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:58:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:58:35.310 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:58:35.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:35.310 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:58:35.374 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:58:35.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:58:35.374 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:35.429 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:58:35.429 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:58:35.430 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:35.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:58:35.550 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:58:35.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:58:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:58:35.615 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:58:35.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:35.616 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:58:35.671 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:58:35.671 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:58:35.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:58:35.740 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:58:35.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:58:35.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:58:35.798+0000 7fbe44e89540 -1 init, newargv = 0x55d9f4e3cb30 newargc=15 2023-10-06T04:58:35.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[117255]: starting ceph client 2023-10-06T04:58:35.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[117255]: starting fuse 2023-10-06T04:58:35.806 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:58:35.808 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:58:35.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:35.808 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:58:35.863 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:58:35.864 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:58:35.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:58:35.864 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:58:35.864 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:58:35.864 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:58:35.864 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:58:35.864 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:58:35.865 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:58:35.866 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:58:35.987 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.117255.asok 2023-10-06T04:58:36.169 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.117255.asok 2023-10-06T04:58:36.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:36.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.117255.asok status 2023-10-06T04:58:36.323 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:58:36.323 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:58:36.323 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "117255", 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6870, 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:58:36.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6870 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 4261756484 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:58:36.325 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 4261756484 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.6870 192.168.0.1:0/4261756484", 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/4261756484", 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 50, 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 84, 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:58:36.326 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:58:36.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:36.335 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:58:36.335 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:36.352 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:58:36.353 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:36.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:36.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:36.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:58:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.577+0000 7f96d19b6700 1 Processor -- start 2023-10-06T04:58:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.577+0000 7f96d19b6700 1 -- start start 2023-10-06T04:58:36.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96d19b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc1591a0 0x7f96cc0affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96d19b6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96cc15dbd0 0x7f96cc0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96d19b6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f96cc159570 0x7f96cc0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96d19b6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0b0a90 con 0x7f96cc15dbd0 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96d19b6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0b2a60 con 0x7f96cc159570 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96d19b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0b3110 con 0x7f96cc1591a0 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96ca7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96cc15dbd0 0x7f96cc0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96ca7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96cc15dbd0 0x7f96cc0b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51334/0 (socket says 172.21.15.3:51334) 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96ca7fc700 1 -- 172.21.15.3:0/2849746599 learned_addr learned my addr 172.21.15.3:0/2849746599 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96caffd700 1 --2- 172.21.15.3:0/2849746599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc1591a0 0x7f96cc0affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:36.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.578+0000 7f96cb7fe700 1 --2- 172.21.15.3:0/2849746599 >> v2:172.21.15.153:3300/0 conn(0x7f96cc159570 0x7f96cc0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96ca7fc700 1 -- 172.21.15.3:0/2849746599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc1591a0 msgr2=0x7f96cc0affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96ca7fc700 1 --2- 172.21.15.3:0/2849746599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc1591a0 0x7f96cc0affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96ca7fc700 1 -- 172.21.15.3:0/2849746599 >> v2:172.21.15.153:3300/0 conn(0x7f96cc159570 msgr2=0x7f96cc0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96ca7fc700 1 --2- 172.21.15.3:0/2849746599 >> v2:172.21.15.153:3300/0 conn(0x7f96cc159570 0x7f96cc0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96ca7fc700 1 -- 172.21.15.3:0/2849746599 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc0b33f0 con 0x7f96cc15dbd0 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96ca7fc700 1 --2- 172.21.15.3:0/2849746599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc0b1fe0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0b0cc0 tx=0x7f96c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9ba8fccc7963aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96c9ffb700 1 -- 172.21.15.3:0/2849746599 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000b640 con 0x7f96cc15dbd0 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96c9ffb700 1 -- 172.21.15.3:0/2849746599 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c000b7a0 con 0x7f96cc15dbd0 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96c9ffb700 1 -- 172.21.15.3:0/2849746599 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000bb60 con 0x7f96cc15dbd0 2023-10-06T04:58:36.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96d19b6700 1 -- 172.21.15.3:0/2849746599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 msgr2=0x7f96cc0b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96d19b6700 1 --2- 172.21.15.3:0/2849746599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc0b1fe0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0b0cc0 tx=0x7f96c0009250 comp rx=0 tx=0).stop 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96d19b6700 1 -- 172.21.15.3:0/2849746599 shutdown_connections 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96d19b6700 1 --2- 172.21.15.3:0/2849746599 >> v2:172.21.15.153:3300/0 conn(0x7f96cc159570 0x7f96cc0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96d19b6700 1 --2- 172.21.15.3:0/2849746599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc1591a0 0x7f96cc0affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96d19b6700 1 --2- 172.21.15.3:0/2849746599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc0b1fe0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.579+0000 7f96d19b6700 1 -- 172.21.15.3:0/2849746599 >> 172.21.15.3:0/2849746599 conn(0x7f96cc070310 msgr2=0x7f96cc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.580+0000 7f96d19b6700 1 -- 172.21.15.3:0/2849746599 shutdown_connections 2023-10-06T04:58:36.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.580+0000 7f96d19b6700 1 -- 172.21.15.3:0/2849746599 wait complete. 2023-10-06T04:58:36.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 Processor -- start 2023-10-06T04:58:36.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 -- start start 2023-10-06T04:58:36.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96cc1591a0 0x7f96cc150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:36.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc159570 0x7f96cc157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:36.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:36.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0c72b0 con 0x7f96cc15dbd0 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0cc600 con 0x7f96cc1591a0 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.581+0000 7f96d19b6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0cc8e0 con 0x7f96cc159570 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51336/0 (socket says 172.21.15.3:51336) 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 -- 172.21.15.3:0/1193017223 learned_addr learned my addr 172.21.15.3:0/1193017223 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96caffd700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96cc1591a0 0x7f96cc150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96ca7fc700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc159570 0x7f96cc157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 -- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc159570 msgr2=0x7f96cc157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc159570 0x7f96cc157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 -- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96cc1591a0 msgr2=0x7f96cc150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96cc1591a0 0x7f96cc150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 -- 172.21.15.3:0/1193017223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c00094f0 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96cb7fe700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc1582e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009e90 tx=0x7f96c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96b2ffd700 1 -- 172.21.15.3:0/1193017223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c400cb70 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96b2ffd700 1 -- 172.21.15.3:0/1193017223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c400f040 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc0c7570 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96b2ffd700 1 -- 172.21.15.3:0/1193017223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4014480 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.582+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc150b60 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.583+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc0c7a90 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.584+0000 7f96b2ffd700 1 -- 172.21.15.3:0/1193017223 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96c4014830 con 0x7f96cc15dbd0 2023-10-06T04:58:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.584+0000 7f96b2ffd700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b8066f30 0x7f96b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:36.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.584+0000 7f96b2ffd700 1 -- 172.21.15.3:0/1193017223 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f96c4085010 con 0x7f96cc15dbd0 2023-10-06T04:58:36.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.584+0000 7f96caffd700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b8066f30 0x7f96b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:36.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.585+0000 7f96caffd700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b8066f30 0x7f96b80693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009db0 tx=0x7f96bc0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:36.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.589+0000 7f96b2ffd700 1 -- 172.21.15.3:0/1193017223 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96c4052120 con 0x7f96cc15dbd0 2023-10-06T04:58:36.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.829+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96cc04c380 con 0x7f96cc15dbd0 2023-10-06T04:58:36.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.829+0000 7f96b2ffd700 1 -- 172.21.15.3:0/1193017223 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f96c4055740 con 0x7f96cc15dbd0 2023-10-06T04:58:36.831 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b8066f30 msgr2=0x7f96b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b8066f30 0x7f96b80693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009db0 tx=0x7f96bc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 msgr2=0x7f96cc1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc1582e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009e90 tx=0x7f96c400b040 comp rx=0 tx=0).stop 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 shutdown_connections 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96b8066f30 0x7f96b80693e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96cc1591a0 0x7f96cc150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96cc15dbd0 0x7f96cc1582e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 --2- 172.21.15.3:0/1193017223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96cc159570 0x7f96cc157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 >> 172.21.15.3:0/1193017223 conn(0x7f96cc070310 msgr2=0x7f96cc0b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:36.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 shutdown_connections 2023-10-06T04:58:36.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:36.831+0000 7f96d19b6700 1 -- 172.21.15.3:0/1193017223 wait complete. 2023-10-06T04:58:36.844 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:58:36.844 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:58:36.845 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:58:36.845 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:58:36.845 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:36.845 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:58:36.845 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:58:36.845 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:58:36.845 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:58:36.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:36.845 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:58:36.863 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:58:36.863 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:58:36.864 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42116sec preferred_lft 42116sec 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:36.865 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:36.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:36.867 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:58:36.931 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:58:36.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:36.933 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:58:36.991 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:58:36.991 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:58:36.991 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:58:36.991 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:58:36.991 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:58:36.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:36.992 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:58:36.992 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:58:36.992 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:58:36.992 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:58:36.992 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:58:36.992 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:58:37.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:37.096 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:58:37.111 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T04:58:37.111 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:58:37.111 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:58:37.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:37.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T04:58:37.177 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:58:37.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:58:37.177 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:37.231 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:58:37.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:58:37.232 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:37.286 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:58:37.352 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:58:37.352 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:58:37.417 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:58:37.418 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:58:37.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:37.418 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:58:37.473 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:58:37.474 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:58:37.516 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:58:37.540 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:58:37.540 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:58:37.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T04:58:37.595+0000 7f4a2edf0540 -1 init, newargv = 0x55f8cdb258e0 newargc=15 2023-10-06T04:58:37.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[100613]: starting ceph client 2023-10-06T04:58:37.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[100613]: starting fuse 2023-10-06T04:58:37.606 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:58:37.607 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:58:37.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:37.607 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:58:37.663 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:58:37.663 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:58:37.663 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:58:37.663 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:58:37.663 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:58:37.663 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:58:37.663 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:58:37.664 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T04:58:37.665 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:58:37.785 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.100613.asok 2023-10-06T04:58:37.966 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.100613.asok 2023-10-06T04:58:37.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:37.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.100613.asok status 2023-10-06T04:58:38.116 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:58:38.116 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "100613", 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:58:38.117 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 6514, 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 6514 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 1546087906 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:58:38.118 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 1546087906 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.6514 192.168.0.1:0/1546087906", 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/1546087906", 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 50, 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 84, 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:58:38.119 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:58:38.120 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:58:38.120 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:58:38.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:38.126 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:58:38.126 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:38.141 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:58:38.142 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:38.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:38.142 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:38.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:58:38.210 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:58:38.261 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-06T04:58:38.261 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-06T04:58:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.263 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:58:38.264 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-06T04:58:38.265 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-06T04:58:38.266 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-06T04:58:38.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-06T04:58:38.268 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-06T04:58:38.269 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:38.270 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-06T04:58:38.271 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.272 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.273 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-06T04:58:38.274 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout: client inject fixed oldest tid = true 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T04:58:38.275 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:58:38.276 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:58:38.290 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-10-06T04:58:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-10-06T04:58:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:58:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-10-06T04:58:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-10-06T04:58:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:58:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:58:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-10-06T04:58:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:58:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:58:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: client inject fixed oldest tid = true 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T04:58:38.302 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:58:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:58:38.304 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:58:38.304 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:58:38.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:38.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[117255]: fuse finished with error 0 and tester_r 0 2023-10-06T04:58:38.351 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:58:38.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:38.351 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:58:38.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:58:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.568+0000 7fc079c0c700 1 Processor -- start 2023-10-06T04:58:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.568+0000 7fc079c0c700 1 -- start start 2023-10-06T04:58:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc079c0c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc074159890 0x7fc0740b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc079c0c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0740b2ec0 0x7fc0740b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc079c0c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc0740b28c0 0x7fc0740b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc079c0c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0740b43e0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc079c0c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0740affe0 con 0x7fc0740b28c0 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc079c0c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740b45d0 con 0x7fc074159890 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0740b2ec0 0x7fc0740b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0740b2ec0 0x7fc0740b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51344/0 (socket says 172.21.15.3:51344) 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 -- 172.21.15.3:0/3309416814 learned_addr learned my addr 172.21.15.3:0/3309416814 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc073fff700 1 --2- 172.21.15.3:0/3309416814 >> v2:172.21.15.153:3300/0 conn(0x7fc0740b28c0 0x7fc0740b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc0737fe700 1 --2- 172.21.15.3:0/3309416814 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc074159890 0x7fc0740b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 -- 172.21.15.3:0/3309416814 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc074159890 msgr2=0x7fc0740b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 --2- 172.21.15.3:0/3309416814 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc074159890 0x7fc0740b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 -- 172.21.15.3:0/3309416814 >> v2:172.21.15.153:3300/0 conn(0x7fc0740b28c0 msgr2=0x7fc0740b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 --2- 172.21.15.3:0/3309416814 >> v2:172.21.15.153:3300/0 conn(0x7fc0740b28c0 0x7fc0740b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 -- 172.21.15.3:0/3309416814 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0740b48b0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc072ffd700 1 --2- 172.21.15.3:0/3309416814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc0740b2380 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc074063660 tx=0x7fc05c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf1c93e97e1d5be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.569+0000 7fc0727fc700 1 -- 172.21.15.3:0/3309416814 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00b860 con 0x7fc0740b2ec0 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc0727fc700 1 -- 172.21.15.3:0/3309416814 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c00b9c0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc0727fc700 1 -- 172.21.15.3:0/3309416814 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00bd60 con 0x7fc0740b2ec0 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 -- 172.21.15.3:0/3309416814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 msgr2=0x7fc0740b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 --2- 172.21.15.3:0/3309416814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc0740b2380 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc074063660 tx=0x7fc05c009250 comp rx=0 tx=0).stop 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 -- 172.21.15.3:0/3309416814 shutdown_connections 2023-10-06T04:58:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 --2- 172.21.15.3:0/3309416814 >> v2:172.21.15.153:3300/0 conn(0x7fc0740b28c0 0x7fc0740b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 --2- 172.21.15.3:0/3309416814 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc074159890 0x7fc0740b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 --2- 172.21.15.3:0/3309416814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc0740b2380 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 -- 172.21.15.3:0/3309416814 >> 172.21.15.3:0/3309416814 conn(0x7fc074099a90 msgr2=0x7fc07406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 -- 172.21.15.3:0/3309416814 shutdown_connections 2023-10-06T04:58:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.570+0000 7fc079c0c700 1 -- 172.21.15.3:0/3309416814 wait complete. 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.571+0000 7fc079c0c700 1 Processor -- start 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.571+0000 7fc079c0c700 1 -- start start 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc079c0c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0740b28c0 0x7fc07409d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc079c0c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc07409a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc079c0c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074159890 0x7fc07409ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc079c0c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07409b060 con 0x7fc0740b2ec0 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc079c0c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07409b310 con 0x7fc074159890 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc079c0c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07409b5f0 con 0x7fc0740b28c0 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc072ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc07409a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:38.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc072ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc07409a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51360/0 (socket says 172.21.15.3:51360) 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc072ffd700 1 -- 172.21.15.3:0/4145599501 learned_addr learned my addr 172.21.15.3:0/4145599501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc0737fe700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0740b28c0 0x7fc07409d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.572+0000 7fc073fff700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074159890 0x7fc07409ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc072ffd700 1 -- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0740b28c0 msgr2=0x7fc07409d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc072ffd700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0740b28c0 0x7fc07409d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc072ffd700 1 -- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074159890 msgr2=0x7fc07409ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc072ffd700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074159890 0x7fc07409ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc072ffd700 1 -- 172.21.15.3:0/4145599501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c0094f0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc072ffd700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc07409a5e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc074088c30 tx=0x7fc05c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc06b7fe700 1 -- 172.21.15.3:0/4145599501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c011dc0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc06b7fe700 1 -- 172.21.15.3:0/4145599501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c024440 con 0x7fc0740b2ec0 2023-10-06T04:58:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07409b8d0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc06b7fe700 1 -- 172.21.15.3:0/4145599501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c024800 con 0x7fc0740b2ec0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.573+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc074169290 con 0x7fc0740b2ec0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.574+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07409bdf0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.574+0000 7fc06b7fe700 1 -- 172.21.15.3:0/4145599501 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc05c01a420 con 0x7fc0740b2ec0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.575+0000 7fc06b7fe700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc054066f30 0x7fc0540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.575+0000 7fc06b7fe700 1 -- 172.21.15.3:0/4145599501 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc05c0864b0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.575+0000 7fc0737fe700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc054066f30 0x7fc0540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.575+0000 7fc0737fe700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc054066f30 0x7fc0540693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0640061f0 tx=0x7fc064008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.579+0000 7fc06b7fe700 1 -- 172.21.15.3:0/4145599501 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc07409bdf0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.821+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc07409bdf0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.821+0000 7fc06b7fe700 1 -- 172.21.15.3:0/4145599501 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc07409bdf0 con 0x7fc0740b2ec0 2023-10-06T04:58:38.823 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc054066f30 msgr2=0x7fc0540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc054066f30 0x7fc0540693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0640061f0 tx=0x7fc064008040 comp rx=0 tx=0).stop 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 msgr2=0x7fc07409a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc07409a5e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc074088c30 tx=0x7fc05c017000 comp rx=0 tx=0).stop 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 shutdown_connections 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc054066f30 0x7fc0540693e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074159890 0x7fc07409ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0740b28c0 0x7fc07409d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 --2- 172.21.15.3:0/4145599501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0740b2ec0 0x7fc07409a5e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 >> 172.21.15.3:0/4145599501 conn(0x7fc074099a90 msgr2=0x7fc07406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 shutdown_connections 2023-10-06T04:58:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:38.823+0000 7fc079c0c700 1 -- 172.21.15.3:0/4145599501 wait complete. 2023-10-06T04:58:38.838 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:58:38.838 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:58:38.838 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:58:38.838 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:58:38.838 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:38.839 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:58:38.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:58:38.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:58:38.839 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:58:38.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:38.839 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:38.855 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42112sec preferred_lft 42112sec 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:58:38.856 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:58:38.857 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:38.858 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:58:38.858 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:58:38.858 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:58:38.858 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:58:38.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:38.858 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:58:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:58:38.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:38.920 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:58:38.977 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:58:38.977 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:58:38.977 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:58:38.977 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:58:38.977 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:58:38.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:38.978 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:58:38.978 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:58:38.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:58:38.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:58:38.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:58:38.978 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:58:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:39.080 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:58:39.095 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:58:39.095 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:58:39.095 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:58:39.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:39.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:58:39.160 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:58:39.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:58:39.161 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:39.217 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:58:39.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:58:39.217 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:39.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:58:39.338 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:58:39.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:58:39.403 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:58:39.405 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:58:39.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:39.406 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:58:39.462 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:58:39.462 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:58:39.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:58:39.529 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:58:39.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:58:39.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:58:39.584+0000 7f8ba61ba540 -1 init, newargv = 0x55745bc00a70 newargc=15 2023-10-06T04:58:39.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[117770]: starting ceph client 2023-10-06T04:58:39.591 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[117770]: starting fuse 2023-10-06T04:58:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:58:39.596 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:58:39.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:39.597 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:58:39.652 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:58:39.653 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:58:39.653 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:58:39.653 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:58:39.653 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:58:39.653 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:58:39.653 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:58:39.654 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:58:39.655 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:58:39.777 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.117770.asok 2023-10-06T04:58:39.957 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.117770.asok 2023-10-06T04:58:39.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:39.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.117770.asok status 2023-10-06T04:58:40.112 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:58:40.112 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:58:40.112 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:58:40.112 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:58:40.112 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:58:40.112 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "117770", 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6894, 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6894 2023-10-06T04:58:40.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 123654855 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 123654855 2023-10-06T04:58:40.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.6894 192.168.0.1:0/123654855", 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/123654855", 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 50, 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 84, 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:58:40.115 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:58:40.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:40.122 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:58:40.122 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:40.138 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:58:40.138 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:40.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:58:40.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:40.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.360+0000 7fd8d82e3700 1 Processor -- start 2023-10-06T04:58:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.360+0000 7fd8d82e3700 1 -- start start 2023-10-06T04:58:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.361+0000 7fd8d82e3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d0157e50 0x7fd8d00c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.361+0000 7fd8d82e3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8d00ccae0 0x7fd8d00ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.361+0000 7fd8d82e3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00b0eb0 con 0x7fd8d00b0ae0 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.361+0000 7fd8d82e3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d01582a0 con 0x7fd8d00ccae0 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.361+0000 7fd8d82e3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0155930 con 0x7fd8d0157e50 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d6880700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8d00ccae0 0x7fd8d00ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d607f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd8d00b0ae0 0x7fd8d00c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33476/0 (socket says 172.21.15.3:33476) 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d6880700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8d00ccae0 0x7fd8d00ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51374/0 (socket says 172.21.15.3:51374) 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d607f700 1 -- 172.21.15.3:0/173248338 learned_addr learned my addr 172.21.15.3:0/173248338 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d587e700 1 --2- 172.21.15.3:0/173248338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d0157e50 0x7fd8d00c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d6880700 1 -- 172.21.15.3:0/173248338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d0157e50 msgr2=0x7fd8d00c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d6880700 1 --2- 172.21.15.3:0/173248338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d0157e50 0x7fd8d00c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d6880700 1 -- 172.21.15.3:0/173248338 >> v1:172.21.15.3:6789/0 conn(0x7fd8d00b0ae0 legacy=0x7fd8d00c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d6880700 1 -- 172.21.15.3:0/173248338 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00cd480 con 0x7fd8d00ccae0 2023-10-06T04:58:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d507d700 1 -- 172.21.15.3:0/173248338 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612258542 0 0) 0x7fd8c8001290 con 0x7fd8d00b0ae0 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d6880700 1 --2- 172.21.15.3:0/173248338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d00ccf40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009e90 tx=0x7fd8c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6763c051f21dd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d507d700 1 -- 172.21.15.3:0/173248338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000e070 con 0x7fd8d00ccae0 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d507d700 1 -- 172.21.15.3:0/173248338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c000bbf0 con 0x7fd8d00ccae0 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.362+0000 7fd8d507d700 1 -- 172.21.15.3:0/173248338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0012430 con 0x7fd8d00ccae0 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 -- 172.21.15.3:0/173248338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 msgr2=0x7fd8d00ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/173248338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d00ccf40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009e90 tx=0x7fd8c0009780 comp rx=0 tx=0).stop 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 -- 172.21.15.3:0/173248338 shutdown_connections 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/173248338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d00ccf40 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/173248338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d0157e50 0x7fd8d00c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 -- 172.21.15.3:0/173248338 >> 172.21.15.3:0/173248338 conn(0x7fd8d006ccb0 msgr2=0x7fd8d0087fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 -- 172.21.15.3:0/173248338 shutdown_connections 2023-10-06T04:58:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.363+0000 7fd8d82e3700 1 -- 172.21.15.3:0/173248338 wait complete. 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.364+0000 7fd8d82e3700 1 Processor -- start 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.364+0000 7fd8d82e3700 1 -- start start 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d00b0ae0 0x7fd8d01583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d0158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d0157e50 0x7fd8d00b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d016a1b0 con 0x7fd8d00ccae0 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d016afc0 con 0x7fd8d0157e50 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d016a3b0 con 0x7fd8d00b0ae0 2023-10-06T04:58:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d0158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51382/0 (socket says 172.21.15.3:51382) 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 -- 172.21.15.3:0/3073854054 learned_addr learned my addr 172.21.15.3:0/3073854054 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d6880700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d0157e50 0x7fd8d00b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d607f700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d00b0ae0 0x7fd8d01583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 -- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d00b0ae0 msgr2=0x7fd8d01583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d00b0ae0 0x7fd8d01583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 -- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d0157e50 msgr2=0x7fd8d00b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d0157e50 0x7fd8d00b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 -- 172.21.15.3:0/3073854054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c0009a20 con 0x7fd8d00ccae0 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d587e700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d0158920 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400d890 tx=0x7fd8c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8ce7fc700 1 -- 172.21.15.3:0/3073854054 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400b600 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8ce7fc700 1 -- 172.21.15.3:0/3073854054 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c400b760 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d016a690 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8ce7fc700 1 -- 172.21.15.3:0/3073854054 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400bae0 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.365+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d016ac50 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.366+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d006f790 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.367+0000 7fd8ce7fc700 1 -- 172.21.15.3:0/3073854054 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8c401e460 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.367+0000 7fd8ce7fc700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd8bc066f30 0x7fd8bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.367+0000 7fd8ce7fc700 1 -- 172.21.15.3:0/3073854054 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd8c4084d70 con 0x7fd8d00ccae0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.367+0000 7fd8d607f700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd8bc066f30 0x7fd8bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:40.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.368+0000 7fd8d607f700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd8bc066f30 0x7fd8bc0693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8002800 tx=0x7fd8c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:40.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.372+0000 7fd8ce7fc700 1 -- 172.21.15.3:0/3073854054 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8c4051e80 con 0x7fd8d00ccae0 2023-10-06T04:58:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.611+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8d0063f80 con 0x7fd8d00ccae0 2023-10-06T04:58:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.611+0000 7fd8ce7fc700 1 -- 172.21.15.3:0/3073854054 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fd8c40554a0 con 0x7fd8d00ccae0 2023-10-06T04:58:40.613 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:40.613 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:58:28.114721+0000","modified":"2023-10-06T04:58:31.748997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.153:6835/2492546877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2492546877},{"type":"v1","addr":"172.21.15.153:6835","nonce":2492546877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T04:58:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd8bc066f30 msgr2=0x7fd8bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd8bc066f30 0x7fd8bc0693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8002800 tx=0x7fd8c8009040 comp rx=0 tx=0).stop 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 msgr2=0x7fd8d0158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d0158920 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400d890 tx=0x7fd8c4008040 comp rx=0 tx=0).stop 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 shutdown_connections 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd8bc066f30 0x7fd8bc0693e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8d0157e50 0x7fd8d00b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8d00b0ae0 0x7fd8d01583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 --2- 172.21.15.3:0/3073854054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00ccae0 0x7fd8d0158920 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 >> 172.21.15.3:0/3073854054 conn(0x7fd8d006ccb0 msgr2=0x7fd8d00cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 shutdown_connections 2023-10-06T04:58:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:40.614+0000 7fd8d82e3700 1 -- 172.21.15.3:0/3073854054 wait complete. 2023-10-06T04:58:40.617 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 50 2023-10-06T04:58:40.627 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_max_completed_requests 1000 2023-10-06T04:58:40.770 INFO:teuthology.orchestra.run.smithi153.stdout:{"success":"mds_max_completed_requests = '1000' (not observed, change may require restart) "} 2023-10-06T04:58:40.770 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_max_completed_requests = '1000' (not observed, change may require restart) " } 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> n = 1100 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:58:40.772 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:58:40.773 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:58:43.031 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:58:43.031 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:58:43.031 DEBUG:teuthology.orchestra.run.smithi003:> import uuid 2023-10-06T04:58:43.031 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> n = 5 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = False 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> if 0 > 0: 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> create_hard_links = True 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> dpath = os.path.dirname(path) 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> fnameprefix = os.path.basename(path) 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(dpath, exist_ok=True) 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:43.032 DEBUG:teuthology.orchestra.run.smithi003:> try: 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(n): 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> with open(fpath, '"'"'w'"'"') as f: 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> f.write(f"{i}") 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> if True: 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> f.flush() 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(f.fileno()) 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:58:43.033 DEBUG:teuthology.orchestra.run.smithi003:> os.unlink(fpath) 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> if create_hard_links: 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> for j in range(0): 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> if False: 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> os.fsync(dirfd) 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> finally: 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> os.close(dirfd) 2023-10-06T04:58:43.034 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:58:43.326 INFO:tasks.ceph_test_case:waiting 100s for health warning matching MDS_CLIENT_OLDEST_TID 2023-10-06T04:58:43.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-06T04:58:43.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.484+0000 7f0878c9e700 1 Processor -- start 2023-10-06T04:58:43.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.484+0000 7f0878c9e700 1 -- start start 2023-10-06T04:58:43.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f0878c9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 0x7f08740b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f0878c9e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08740b18c0 con 0x7f087415e070 2023-10-06T04:58:43.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f0878c9e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08740b09d0 con 0x7f0874159740 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f0878c9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08740b0cb0 con 0x7f087415dbb0 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f0872d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f087415e070 0x7f08740b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50938/0 (socket says 172.21.15.3:50938) 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f0872d9d700 1 -- 172.21.15.3:0/2178758448 learned_addr learned my addr 172.21.15.3:0/2178758448 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f0871d9b700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 0x7f08740b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290034885 0 0) 0x7f08740b18c0 con 0x7f087415e070 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0860003570 con 0x7f087415e070 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2050826256 0 0) 0x7f0860003570 con 0x7f087415e070 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 msgr2=0x7f08740b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.485+0000 7f087159a700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 0x7f08740b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 >> v1:172.21.15.153:6789/0 conn(0x7f0874159740 legacy=0x7f087415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740b0f90 con 0x7f087415e070 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f086c0017f0 con 0x7f087415e070 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 >> v1:172.21.15.3:6789/0 conn(0x7f087415e070 legacy=0x7f08740b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f0860003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 0x7f0860004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0860004d50 0x7f08600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740b0cb0 con 0x7f0860003a90 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086000bb10 con 0x7f0860004d50 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086000bcd0 con 0x7f087415dbb0 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087259c700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f0860003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f0871d9b700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 0x7f0860004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f0872d9d700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0860004d50 0x7f08600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087259c700 1 -- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 msgr2=0x7f0860004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087259c700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 0x7f0860004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087259c700 1 -- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0860004d50 msgr2=0x7f08600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087259c700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0860004d50 0x7f08600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087259c700 1 -- 172.21.15.3:0/2178758448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860003570 con 0x7f0860003a90 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f0871d9b700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415dbb0 0x7f0860004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087259c700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f0860003e60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f08640019c0 tx=0x7f0864007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47983b019868b4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864001330 con 0x7f0860003a90 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086400c5d0 con 0x7f0860003a90 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.486+0000 7f087159a700 1 -- 172.21.15.3:0/2178758448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086400c970 con 0x7f0860003a90 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.487+0000 7f0878c9e700 1 -- 172.21.15.3:0/2178758448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 msgr2=0x7f0860003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.487+0000 7f0878c9e700 1 --2- 172.21.15.3:0/2178758448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f0860003e60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f08640019c0 tx=0x7f0864007d60 comp rx=0 tx=0).stop 2023-10-06T04:58:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.487+0000 7f0872d9d700 1 -- 172.21.15.3:0/2178758448 reap_dead start 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.487+0000 7f0878c9e700 1 -- 172.21.15.3:0/2178758448 shutdown_connections 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.487+0000 7f0878c9e700 1 -- 172.21.15.3:0/2178758448 >> 172.21.15.3:0/2178758448 conn(0x7f0874099a90 msgr2=0x7f087406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.487+0000 7f0878c9e700 1 -- 172.21.15.3:0/2178758448 shutdown_connections 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.487+0000 7f0878c9e700 1 -- 172.21.15.3:0/2178758448 wait complete. 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.488+0000 7f0878c9e700 1 Processor -- start 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.488+0000 7f0878c9e700 1 -- start start 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0878c9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f08740b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0878c9e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087415c3c0 0x7f08741559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0878c9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415c830 0x7f0874155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0878c9e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740cedf0 con 0x7f0860003a90 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0878c9e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740cf0d0 con 0x7f087415c3c0 2023-10-06T04:58:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0878c9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08740cf3b0 con 0x7f087415c830 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f08740b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f08740b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57144/0 (socket says 172.21.15.3:57144) 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 -- 172.21.15.3:0/3907077175 learned_addr learned my addr 172.21.15.3:0/3907077175 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0871d9b700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087415c3c0 0x7f08741559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0872d9d700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415c830 0x7f0874155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 -- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415c830 msgr2=0x7f0874155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415c830 0x7f0874155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 -- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087415c3c0 msgr2=0x7f08741559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087415c3c0 0x7f08741559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 -- 172.21.15.3:0/3907077175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740cf690 con 0x7f0860003a90 2023-10-06T04:58:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f0871d9b700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087415c3c0 0x7f08741559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.489+0000 7f087259c700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f08740b0200 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f086400c3c0 tx=0x7f0864015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.490+0000 7f086a7fc700 1 -- 172.21.15.3:0/3907077175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864001330 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.490+0000 7f086a7fc700 1 -- 172.21.15.3:0/3907077175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086400c740 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.490+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0874159c30 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.490+0000 7f086a7fc700 1 -- 172.21.15.3:0/3907077175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086401f410 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.490+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740ce9a0 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.490+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0874159fa0 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.491+0000 7f086a7fc700 1 -- 172.21.15.3:0/3907077175 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f086401f7c0 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.491+0000 7f086a7fc700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0854066f30 0x7f08540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.491+0000 7f086a7fc700 1 -- 172.21.15.3:0/3907077175 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0864086230 con 0x7f0860003a90 2023-10-06T04:58:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.492+0000 7f0871d9b700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0854066f30 0x7f08540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.492+0000 7f0871d9b700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0854066f30 0x7f08540693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f08741512e0 tx=0x7f085c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.496+0000 7f086a7fc700 1 -- 172.21.15.3:0/3907077175 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0864053340 con 0x7f0860003a90 2023-10-06T04:58:43.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.769+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f0874159dc0 con 0x7f0860003a90 2023-10-06T04:58:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.769+0000 7f086a7fc700 1 -- 172.21.15.3:0/3907077175 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+46 (secure 0 0 0) 0x7f0864056960 con 0x7f0860003a90 2023-10-06T04:58:43.771 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:43.771 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0854066f30 msgr2=0x7f08540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0854066f30 0x7f08540693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f08741512e0 tx=0x7f085c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 msgr2=0x7f08740b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f08740b0200 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f086400c3c0 tx=0x7f0864015000 comp rx=0 tx=0).stop 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 shutdown_connections 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0854066f30 0x7f08540693e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f087415c830 0x7f0874155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0860003a90 0x7f08740b0200 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 --2- 172.21.15.3:0/3907077175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087415c3c0 0x7f08741559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 >> 172.21.15.3:0/3907077175 conn(0x7f0874099a90 msgr2=0x7f087406e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 shutdown_connections 2023-10-06T04:58:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:43.771+0000 7f0878c9e700 1 -- 172.21.15.3:0/3907077175 wait complete. 2023-10-06T04:58:43.785 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T04:58:43.785 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-10-06T04:58:48.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-06T04:58:48.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.910+0000 7f69b8837700 1 Processor -- start 2023-10-06T04:58:48.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.911+0000 7f69b8837700 1 -- start start 2023-10-06T04:58:48.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.911+0000 7f69b8837700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40a8b80 0x7f69b40a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:48.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.911+0000 7f69b8837700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f69b40b0a40 0x7f69b40aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:48.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.911+0000 7f69b8837700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69b40af3b0 0x7f69b40af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:48.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.911+0000 7f69b8837700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b40b2560 con 0x7f69b40af3b0 2023-10-06T04:58:48.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.911+0000 7f69b8837700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b406e9c0 con 0x7f69b40b0a40 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.911+0000 7f69b8837700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b406eca0 con 0x7f69b40a8b80 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69b40af3b0 0x7f69b40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69b40af3b0 0x7f69b40af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57158/0 (socket says 172.21.15.3:57158) 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 -- 172.21.15.3:0/1561065562 learned_addr learned my addr 172.21.15.3:0/1561065562 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b1d9b700 1 --2- 172.21.15.3:0/1561065562 >> v2:172.21.15.153:3300/0 conn(0x7f69b40b0a40 0x7f69b40aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b259c700 1 --2- 172.21.15.3:0/1561065562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40a8b80 0x7f69b40a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 -- 172.21.15.3:0/1561065562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40a8b80 msgr2=0x7f69b40a8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 --2- 172.21.15.3:0/1561065562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40a8b80 0x7f69b40a8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 -- 172.21.15.3:0/1561065562 >> v2:172.21.15.153:3300/0 conn(0x7f69b40b0a40 msgr2=0x7f69b40aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:48.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 --2- 172.21.15.3:0/1561065562 >> v2:172.21.15.153:3300/0 conn(0x7f69b40b0a40 0x7f69b40aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 -- 172.21.15.3:0/1561065562 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40b21f0 con 0x7f69b40af3b0 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b2d9d700 1 --2- 172.21.15.3:0/1561065562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40af830 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f69b406aad0 tx=0x7f69ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d186773c6005e28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b159a700 1 -- 172.21.15.3:0/1561065562 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00e070 con 0x7f69b40af3b0 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b159a700 1 -- 172.21.15.3:0/1561065562 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac00ba50 con 0x7f69b40af3b0 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.912+0000 7f69b159a700 1 -- 172.21.15.3:0/1561065562 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0092e0 con 0x7f69b40af3b0 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 -- 172.21.15.3:0/1561065562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 msgr2=0x7f69b40af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 --2- 172.21.15.3:0/1561065562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40af830 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f69b406aad0 tx=0x7f69ac009840 comp rx=0 tx=0).stop 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 -- 172.21.15.3:0/1561065562 shutdown_connections 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 --2- 172.21.15.3:0/1561065562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40af830 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 --2- 172.21.15.3:0/1561065562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40a8b80 0x7f69b40a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 --2- 172.21.15.3:0/1561065562 >> v2:172.21.15.153:3300/0 conn(0x7f69b40b0a40 0x7f69b40aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 -- 172.21.15.3:0/1561065562 >> 172.21.15.3:0/1561065562 conn(0x7f69b4087fd0 msgr2=0x7f69b40ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:48.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 -- 172.21.15.3:0/1561065562 shutdown_connections 2023-10-06T04:58:48.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.913+0000 7f69b8837700 1 -- 172.21.15.3:0/1561065562 wait complete. 2023-10-06T04:58:48.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.914+0000 7f69b8837700 1 Processor -- start 2023-10-06T04:58:48.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.914+0000 7f69b8837700 1 -- start start 2023-10-06T04:58:48.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b40a8b80 0x7f69b4158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40b0a40 0x7f69b40cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40cc080 con 0x7f69b40af3b0 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40cb4b0 con 0x7f69b40a8b80 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40cb750 con 0x7f69b40b0a40 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57168/0 (socket says 172.21.15.3:57168) 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 -- 172.21.15.3:0/1354894848 learned_addr learned my addr 172.21.15.3:0/1354894848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b259c700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b40a8b80 0x7f69b4158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b2d9d700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40b0a40 0x7f69b40cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 -- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40b0a40 msgr2=0x7f69b40cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40b0a40 0x7f69b40cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 -- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b40a8b80 msgr2=0x7f69b4158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b40a8b80 0x7f69b4158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 -- 172.21.15.3:0/1354894848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac009ae0 con 0x7f69b40af3b0 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b1d9b700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40ce270 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f699c00d860 tx=0x7f699c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69aa7fc700 1 -- 172.21.15.3:0/1354894848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c0125f0 con 0x7f69b40af3b0 2023-10-06T04:58:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69aa7fc700 1 -- 172.21.15.3:0/1354894848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c012750 con 0x7f69b40af3b0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b40cba30 con 0x7f69b40af3b0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69aa7fc700 1 -- 172.21.15.3:0/1354894848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c012b10 con 0x7f69b40af3b0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.915+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b415b250 con 0x7f69b40af3b0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.916+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b415b470 con 0x7f69b40af3b0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.917+0000 7f69aa7fc700 1 -- 172.21.15.3:0/1354894848 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f699c01e460 con 0x7f69b40af3b0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.917+0000 7f69aa7fc700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.917+0000 7f69aa7fc700 1 -- 172.21.15.3:0/1354894848 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f699c084bc0 con 0x7f69b40af3b0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.917+0000 7f69b259c700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.918+0000 7f69b259c700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6994066f30 0x7f69940693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f69a4006aa0 tx=0x7f69a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:48.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:48.921+0000 7f69aa7fc700 1 -- 172.21.15.3:0/1354894848 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f699c051cd0 con 0x7f69b40af3b0 2023-10-06T04:58:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.197+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f69b4063f80 con 0x7f69b40af3b0 2023-10-06T04:58:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.197+0000 7f69aa7fc700 1 -- 172.21.15.3:0/1354894848 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+322 (secure 0 0 0) 0x7f699c0552f0 con 0x7f69b40af3b0 2023-10-06T04:58:49.199 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:49.199 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.d(mds.0): Client smithi003:0 failing to advance its oldest client/flush tid. client_id: 6894"}],"muted":false}},"mutes":[]} 2023-10-06T04:58:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6994066f30 msgr2=0x7f69940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6994066f30 0x7f69940693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f69a4006aa0 tx=0x7f69a4008040 comp rx=0 tx=0).stop 2023-10-06T04:58:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 msgr2=0x7f69b40ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40ce270 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f699c00d860 tx=0x7f699c008040 comp rx=0 tx=0).stop 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 shutdown_connections 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b40a8b80 0x7f69b4158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69b40b0a40 0x7f69b40cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 --2- 172.21.15.3:0/1354894848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69b40af3b0 0x7f69b40ce270 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 >> 172.21.15.3:0/1354894848 conn(0x7f69b4087fd0 msgr2=0x7f69b4068e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 shutdown_connections 2023-10-06T04:58:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.199+0000 7f69b8837700 1 -- 172.21.15.3:0/1354894848 wait complete. 2023-10-06T04:58:49.212 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.d(mds.0): Client smithi003:0 failing to advance its oldest client/flush tid. client_id: 6894'] 2023-10-06T04:58:49.213 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T04:58:49.213 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:58:49.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:49.256 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:49.295 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:58:49.295 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:58:49.296 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:58:49.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:58:49.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[117770]: fuse finished with error 0 and tester_r 0 2023-10-06T04:58:49.340 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:58:49.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:49.340 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:58:49.507 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:58:49.508 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:58:49.508 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:58:49.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[100613]: fuse finished with error 0 and tester_r 0 2023-10-06T04:58:49.541 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:58:49.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:49.542 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:58:49.604 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:58:49.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:49.604 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:58:49.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:58:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T04:58:49.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:58:49.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:58:49.620 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:58:49.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:58:49.659 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T04:58:49.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:49.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.778+0000 7f1075b02700 1 Processor -- start 2023-10-06T04:58:49.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.778+0000 7f1075b02700 1 -- start start 2023-10-06T04:58:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f1075b02700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10700b1760 0x7f107015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f1075b02700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107015b1b0 con 0x7f1070159890 2023-10-06T04:58:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f1075b02700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700affe0 con 0x7f10700b2ec0 2023-10-06T04:58:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f1075b02700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10700b43e0 con 0x7f10700b1760 2023-10-06T04:58:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1070159890 0x7f10700b1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50940/0 (socket says 172.21.15.3:50940) 2023-10-06T04:58:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106f7fe700 1 -- 172.21.15.3:0/1585802308 learned_addr learned my addr 172.21.15.3:0/1585802308 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106ffff700 1 --2- 172.21.15.3:0/1585802308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10700b1760 0x7f107015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106e7fc700 1 -- 172.21.15.3:0/1585802308 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145787607 0 0) 0x7f107015b1b0 con 0x7f1070159890 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106e7fc700 1 -- 172.21.15.3:0/1585802308 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f105c003570 con 0x7f1070159890 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106e7fc700 1 -- 172.21.15.3:0/1585802308 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095151683 0 0) 0x7f10700affe0 con 0x7f10700b2ec0 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106e7fc700 1 -- 172.21.15.3:0/1585802308 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107015b1b0 con 0x7f10700b2ec0 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106ffff700 1 -- 172.21.15.3:0/1585802308 >> v1:172.21.15.153:6789/0 conn(0x7f10700b2ec0 legacy=0x7f10700b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106ffff700 1 -- 172.21.15.3:0/1585802308 >> v1:172.21.15.3:6789/0 conn(0x7f1070159890 legacy=0x7f10700b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.779+0000 7f106ffff700 1 -- 172.21.15.3:0/1585802308 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700b2c60 con 0x7f10700b1760 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f106ffff700 1 --2- 172.21.15.3:0/1585802308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10700b1760 0x7f107015ac70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1068009910 tx=0x7f10680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed913465f9f03f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f106e7fc700 1 -- 172.21.15.3:0/1585802308 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068011630 con 0x7f10700b1760 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f106e7fc700 1 -- 172.21.15.3:0/1585802308 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068011790 con 0x7f10700b1760 2023-10-06T04:58:49.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f106e7fc700 1 -- 172.21.15.3:0/1585802308 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068011b30 con 0x7f10700b1760 2023-10-06T04:58:49.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f1075b02700 1 -- 172.21.15.3:0/1585802308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10700b1760 msgr2=0x7f107015ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:49.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f1075b02700 1 --2- 172.21.15.3:0/1585802308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10700b1760 0x7f107015ac70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1068009910 tx=0x7f10680091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f1075b02700 1 -- 172.21.15.3:0/1585802308 shutdown_connections 2023-10-06T04:58:49.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f1075b02700 1 --2- 172.21.15.3:0/1585802308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10700b1760 0x7f107015ac70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f1075b02700 1 -- 172.21.15.3:0/1585802308 >> 172.21.15.3:0/1585802308 conn(0x7f1070099a90 msgr2=0x7f107006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:49.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f1075b02700 1 -- 172.21.15.3:0/1585802308 shutdown_connections 2023-10-06T04:58:49.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.780+0000 7f1075b02700 1 -- 172.21.15.3:0/1585802308 wait complete. 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.781+0000 7f1075b02700 1 Processor -- start 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f1075b02700 1 -- start start 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f1075b02700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10700b1760 0x7f1070145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f1075b02700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10700b2ec0 0x7f10701691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f1075b02700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1070159890 0x7f10700ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f1075b02700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070145aa0 con 0x7f10700b1760 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f1075b02700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700ac020 con 0x7f10700b2ec0 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f1075b02700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10700ac300 con 0x7f1070159890 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f106f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10700b1760 0x7f1070145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f106f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10700b1760 0x7f1070145560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57182/0 (socket says 172.21.15.3:57182) 2023-10-06T04:58:49.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f106f7fe700 1 -- 172.21.15.3:0/1835158545 learned_addr learned my addr 172.21.15.3:0/1835158545 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f106ffff700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1070159890 0x7f10700ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.782+0000 7f106effd700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10700b2ec0 0x7f10701691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f106f7fe700 1 -- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1070159890 msgr2=0x7f10700ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f106f7fe700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1070159890 0x7f10700ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f106f7fe700 1 -- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10700b2ec0 msgr2=0x7f10701691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f106f7fe700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10700b2ec0 0x7f10701691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f106f7fe700 1 -- 172.21.15.3:0/1835158545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068009420 con 0x7f10700b1760 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f106f7fe700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10700b1760 0x7f1070145560 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f10600016e0 tx=0x7f1060002870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f10677fe700 1 -- 172.21.15.3:0/1835158545 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060005b70 con 0x7f10700b1760 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f10677fe700 1 -- 172.21.15.3:0/1835158545 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060005cd0 con 0x7f10700b1760 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10700ac5e0 con 0x7f10700b1760 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f10677fe700 1 -- 172.21.15.3:0/1835158545 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000a820 con 0x7f10700b1760 2023-10-06T04:58:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.783+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10700acba0 con 0x7f10700b1760 2023-10-06T04:58:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.784+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10700abd70 con 0x7f10700b1760 2023-10-06T04:58:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.784+0000 7f10677fe700 1 -- 172.21.15.3:0/1835158545 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f106000ac10 con 0x7f10700b1760 2023-10-06T04:58:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.785+0000 7f10677fe700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1050066f30 0x7f10500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.785+0000 7f10677fe700 1 -- 172.21.15.3:0/1835158545 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f10600843f0 con 0x7f10700b1760 2023-10-06T04:58:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.785+0000 7f106effd700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1050066f30 0x7f10500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.785+0000 7f106effd700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1050066f30 0x7f10500693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1058007fd0 tx=0x7f1058007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:49.789+0000 7f10677fe700 1 -- 172.21.15.3:0/1835158545 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1060051580 con 0x7f10700b1760 2023-10-06T04:58:50.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.026+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1070063f80 con 0x7f10700b1760 2023-10-06T04:58:50.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.027+0000 7f10677fe700 1 -- 172.21.15.3:0/1835158545 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f1060054ba0 con 0x7f10700b1760 2023-10-06T04:58:50.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:50.028 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:58:28.114721+0000","modified":"2023-10-06T04:58:48.790877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.153:6835/2492546877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2492546877},{"type":"v1","addr":"172.21.15.153:6835","nonce":2492546877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T04:58:50.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.028+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1050066f30 msgr2=0x7f10500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.028+0000 7f1075b02700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1050066f30 0x7f10500693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1058007fd0 tx=0x7f1058007d10 comp rx=0 tx=0).stop 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.028+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10700b1760 msgr2=0x7f1070145560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.028+0000 7f1075b02700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10700b1760 0x7f1070145560 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f10600016e0 tx=0x7f1060002870 comp rx=0 tx=0).stop 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.028+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 shutdown_connections 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.029+0000 7f1075b02700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1050066f30 0x7f10500693e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.029+0000 7f1075b02700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1070159890 0x7f10700ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.029+0000 7f1075b02700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10700b1760 0x7f1070145560 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.029+0000 7f1075b02700 1 --2- 172.21.15.3:0/1835158545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10700b2ec0 0x7f10701691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.029+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 >> 172.21.15.3:0/1835158545 conn(0x7f1070099a90 msgr2=0x7f107006e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.029+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 shutdown_connections 2023-10-06T04:58:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.029+0000 7f1075b02700 1 -- 172.21.15.3:0/1835158545 wait complete. 2023-10-06T04:58:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 51 2023-10-06T04:58:50.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.160+0000 7f468e5ee700 1 Processor -- start 2023-10-06T04:58:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f468e5ee700 1 -- start start 2023-10-06T04:58:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f468e5ee700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4688159aa0 0x7f468806c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f468e5ee700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 0x7f4688159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f468e5ee700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880683d0 con 0x7f46881596d0 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f468e5ee700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468806ce60 con 0x7f4688159aa0 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f468e5ee700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880677c0 con 0x7f468806ca10 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f4687fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f46881596d0 0x7f46880b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50942/0 (socket says 172.21.15.3:50942) 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.161+0000 7f4687fff700 1 -- 172.21.15.3:0/3351663585 learned_addr learned my addr 172.21.15.3:0/3351663585 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f468cb8b700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 0x7f4688159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597850226 0 0) 0x7f46880683d0 con 0x7f46881596d0 2023-10-06T04:58:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f46877fe700 1 --2- 172.21.15.3:0/3351663585 >> v2:172.21.15.153:3300/0 conn(0x7f4688159aa0 0x7f468806c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4674003570 con 0x7f46881596d0 2023-10-06T04:58:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1172605529 0 0) 0x7f4674003570 con 0x7f46881596d0 2023-10-06T04:58:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 msgr2=0x7f4688159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 0x7f4688159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 >> v2:172.21.15.153:3300/0 conn(0x7f4688159aa0 msgr2=0x7f468806c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 --2- 172.21.15.3:0/3351663585 >> v2:172.21.15.153:3300/0 conn(0x7f4688159aa0 0x7f468806c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688067aa0 con 0x7f46881596d0 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f4678001470 con 0x7f46881596d0 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 >> v1:172.21.15.3:6789/0 conn(0x7f46881596d0 legacy=0x7f46880b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4674004810 0x7f4674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 0x7f46740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468806ce60 con 0x7f4674004810 2023-10-06T04:58:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880677c0 con 0x7f4674003a60 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467400ba60 con 0x7f468806ca10 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f46877fe700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4674004810 0x7f4674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f4687fff700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.162+0000 7f468cb8b700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 0x7f46740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f46877fe700 1 -- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 msgr2=0x7f46740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f46877fe700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f468806ca10 0x7f46740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f46877fe700 1 -- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674003a60 msgr2=0x7f4674003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f46877fe700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f46877fe700 1 -- 172.21.15.3:0/3351663585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674003570 con 0x7f4674004810 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f4687fff700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f46877fe700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4674004810 0x7f4674004c80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f46880acf20 tx=0x7f46700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafe9da658499939 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467000e710 con 0x7f4674004810 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467000e870 con 0x7f4674004810 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f4686ffd700 1 -- 172.21.15.3:0/3351663585 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467000ec30 con 0x7f4674004810 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f468e5ee700 1 -- 172.21.15.3:0/3351663585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4674004810 msgr2=0x7f4674004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f468e5ee700 1 --2- 172.21.15.3:0/3351663585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4674004810 0x7f4674004c80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f46880acf20 tx=0x7f46700091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f468cb8b700 1 -- 172.21.15.3:0/3351663585 reap_dead start 2023-10-06T04:58:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f468e5ee700 1 -- 172.21.15.3:0/3351663585 shutdown_connections 2023-10-06T04:58:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f468e5ee700 1 -- 172.21.15.3:0/3351663585 >> 172.21.15.3:0/3351663585 conn(0x7f468806e0b0 msgr2=0x7f46880aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f468e5ee700 1 -- 172.21.15.3:0/3351663585 shutdown_connections 2023-10-06T04:58:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.163+0000 7f468e5ee700 1 -- 172.21.15.3:0/3351663585 wait complete. 2023-10-06T04:58:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.164+0000 7f468e5ee700 1 Processor -- start 2023-10-06T04:58:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f468e5ee700 1 -- start start 2023-10-06T04:58:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f468e5ee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674004810 0x7f468814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f468e5ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46880aa270 0x7f46880aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f468e5ee700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46881519f0 0x7f4688151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f468e5ee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881523a0 con 0x7f46880aa270 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f468e5ee700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4688154180 con 0x7f4674004810 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f468e5ee700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4688153e50 con 0x7f46881519f0 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.165+0000 7f46877fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46880aa270 0x7f46880aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46880aa270 0x7f46880aa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57194/0 (socket says 172.21.15.3:57194) 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 -- 172.21.15.3:0/2547015574 learned_addr learned my addr 172.21.15.3:0/2547015574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f4687fff700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674004810 0x7f468814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f468cb8b700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46881519f0 0x7f4688151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 -- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46881519f0 msgr2=0x7f4688151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46881519f0 0x7f4688151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 -- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674004810 msgr2=0x7f468814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674004810 0x7f468814f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 -- 172.21.15.3:0/2547015574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880aad80 con 0x7f46880aa270 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f4687fff700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674004810 0x7f468814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f46877fe700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46880aa270 0x7f46880aa6e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4674004700 tx=0x7f467001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f467ffff700 1 -- 172.21.15.3:0/2547015574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670023070 con 0x7f46880aa270 2023-10-06T04:58:50.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f467ffff700 1 -- 172.21.15.3:0/2547015574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467000e9e0 con 0x7f46880aa270 2023-10-06T04:58:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46880ab060 con 0x7f46880aa270 2023-10-06T04:58:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f467ffff700 1 -- 172.21.15.3:0/2547015574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467001e030 con 0x7f46880aa270 2023-10-06T04:58:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.166+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4688150e40 con 0x7f46880aa270 2023-10-06T04:58:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.167+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4688153cb0 con 0x7f46880aa270 2023-10-06T04:58:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.167+0000 7f467ffff700 1 -- 172.21.15.3:0/2547015574 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4670013470 con 0x7f46880aa270 2023-10-06T04:58:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.168+0000 7f467ffff700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4668066f60 0x7f4668069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.168+0000 7f467ffff700 1 -- 172.21.15.3:0/2547015574 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4670085d90 con 0x7f46880aa270 2023-10-06T04:58:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.168+0000 7f4687fff700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4668066f60 0x7f4668069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.168+0000 7f4687fff700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4668066f60 0x7f4668069410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4678000e10 tx=0x7f4678009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.172+0000 7f467ffff700 1 -- 172.21.15.3:0/2547015574 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4688153cb0 con 0x7f46880aa270 2023-10-06T04:58:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.367+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4688153cb0 con 0x7f46880aa270 2023-10-06T04:58:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.368+0000 7f467ffff700 1 -- 172.21.15.3:0/2547015574 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f4688153cb0 con 0x7f46880aa270 2023-10-06T04:58:50.369 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:50.370 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:58:29.742286+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:58:25.874792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T04:58:27.107107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4668066f60 msgr2=0x7f4668069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4668066f60 0x7f4668069410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4678000e10 tx=0x7f4678009040 comp rx=0 tx=0).stop 2023-10-06T04:58:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46880aa270 msgr2=0x7f46880aa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46880aa270 0x7f46880aa6e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4674004700 tx=0x7f467001a000 comp rx=0 tx=0).stop 2023-10-06T04:58:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 shutdown_connections 2023-10-06T04:58:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4668066f60 0x7f4668069410 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674004810 0x7f468814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46881519f0 0x7f4688151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 --2- 172.21.15.3:0/2547015574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46880aa270 0x7f46880aa6e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 >> 172.21.15.3:0/2547015574 conn(0x7f468806e0b0 msgr2=0x7f46880b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 shutdown_connections 2023-10-06T04:58:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.370+0000 7f468e5ee700 1 -- 172.21.15.3:0/2547015574 wait complete. 2023-10-06T04:58:50.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:58:25.874792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:58:27.107107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:58:50.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:58:50.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.506+0000 7fdd6ab63700 1 Processor -- start 2023-10-06T04:58:50.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.506+0000 7fdd6ab63700 1 -- start start 2023-10-06T04:58:50.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.506+0000 7fdd6ab63700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd640b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.506+0000 7fdd6ab63700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd64161690 0x7fdd64163a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.506+0000 7fdd6ab63700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd640b1ea0 con 0x7fdd640b09b0 2023-10-06T04:58:50.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.506+0000 7fdd6ab63700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd64159f40 con 0x7fdd64161690 2023-10-06T04:58:50.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.506+0000 7fdd6ab63700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640b6020 con 0x7fdd640b14f0 2023-10-06T04:58:50.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd64161690 0x7fdd64163a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd64161690 0x7fdd64163a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57208/0 (socket says 172.21.15.3:57208) 2023-10-06T04:58:50.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 -- 172.21.15.3:0/3433498527 learned_addr learned my addr 172.21.15.3:0/3433498527 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:50.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd63fff700 1 --2- 172.21.15.3:0/3433498527 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd640b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 -- 172.21.15.3:0/3433498527 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 msgr2=0x7fdd640b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:50.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 --2- 172.21.15.3:0/3433498527 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd640b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 -- 172.21.15.3:0/3433498527 >> v1:172.21.15.153:6789/0 conn(0x7fdd640b09b0 legacy=0x7fdd640b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 -- 172.21.15.3:0/3433498527 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640b3c80 con 0x7fdd64161690 2023-10-06T04:58:50.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd69100700 1 --2- 172.21.15.3:0/3433498527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64163a70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0099b0 tx=0x7fdd4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac33068c631c87f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd637fe700 1 -- 172.21.15.3:0/3433498527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c009ea0 con 0x7fdd64161690 2023-10-06T04:58:50.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd637fe700 1 -- 172.21.15.3:0/3433498527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00b860 con 0x7fdd64161690 2023-10-06T04:58:50.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.507+0000 7fdd637fe700 1 -- 172.21.15.3:0/3433498527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00bc00 con 0x7fdd64161690 2023-10-06T04:58:50.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 -- 172.21.15.3:0/3433498527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 msgr2=0x7fdd64163a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/3433498527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64163a70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0099b0 tx=0x7fdd4c009250 comp rx=0 tx=0).stop 2023-10-06T04:58:50.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 -- 172.21.15.3:0/3433498527 shutdown_connections 2023-10-06T04:58:50.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/3433498527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64163a70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/3433498527 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd640b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 -- 172.21.15.3:0/3433498527 >> 172.21.15.3:0/3433498527 conn(0x7fdd64099a90 msgr2=0x7fdd6406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:50.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 -- 172.21.15.3:0/3433498527 shutdown_connections 2023-10-06T04:58:50.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.508+0000 7fdd6ab63700 1 -- 172.21.15.3:0/3433498527 wait complete. 2023-10-06T04:58:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.509+0000 7fdd6ab63700 1 Processor -- start 2023-10-06T04:58:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.509+0000 7fdd6ab63700 1 -- start start 2023-10-06T04:58:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd6ab63700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd640b09b0 0x7fdd64152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd6ab63700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd64158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd6ab63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd6ab63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6415b230 con 0x7fdd64161690 2023-10-06T04:58:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd6ab63700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640cd650 con 0x7fdd640b09b0 2023-10-06T04:58:50.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd6ab63700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6415cfe0 con 0x7fdd640b14f0 2023-10-06T04:58:50.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57212/0 (socket says 172.21.15.3:57212) 2023-10-06T04:58:50.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 -- 172.21.15.3:0/989495489 learned_addr learned my addr 172.21.15.3:0/989495489 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd63fff700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd64158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd688ff700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd640b09b0 0x7fdd64152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 -- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 msgr2=0x7fdd64158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd64158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 -- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd640b09b0 msgr2=0x7fdd64152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd640b09b0 0x7fdd64152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.510+0000 7fdd69100700 1 -- 172.21.15.3:0/989495489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c0094f0 con 0x7fdd64161690 2023-10-06T04:58:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.511+0000 7fdd69100700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64158780 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdd6406d4b0 tx=0x7fdd4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.511+0000 7fdd60ff9700 1 -- 172.21.15.3:0/989495489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c01a450 con 0x7fdd64161690 2023-10-06T04:58:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.511+0000 7fdd60ff9700 1 -- 172.21.15.3:0/989495489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c01a5b0 con 0x7fdd64161690 2023-10-06T04:58:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.511+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6415ac70 con 0x7fdd64161690 2023-10-06T04:58:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.511+0000 7fdd60ff9700 1 -- 172.21.15.3:0/989495489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c01a970 con 0x7fdd64161690 2023-10-06T04:58:50.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.511+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd640cd2e0 con 0x7fdd64161690 2023-10-06T04:58:50.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.512+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd640cd140 con 0x7fdd64161690 2023-10-06T04:58:50.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.512+0000 7fdd60ff9700 1 -- 172.21.15.3:0/989495489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd4c01ace0 con 0x7fdd64161690 2023-10-06T04:58:50.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.512+0000 7fdd60ff9700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd44066f30 0x7fdd440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.512+0000 7fdd60ff9700 1 -- 172.21.15.3:0/989495489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd4c086490 con 0x7fdd64161690 2023-10-06T04:58:50.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.513+0000 7fdd688ff700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd44066f30 0x7fdd440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.513+0000 7fdd688ff700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd44066f30 0x7fdd440693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c007fd0 tx=0x7fdd5c001650 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.517+0000 7fdd60ff9700 1 -- 172.21.15.3:0/989495489 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd640cd140 con 0x7fdd64161690 2023-10-06T04:58:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.710+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdd640cd800 con 0x7fdd64161690 2023-10-06T04:58:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.710+0000 7fdd60ff9700 1 -- 172.21.15.3:0/989495489 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdd640cd800 con 0x7fdd64161690 2023-10-06T04:58:50.712 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:58:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd44066f30 msgr2=0x7fdd440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd44066f30 0x7fdd440693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c007fd0 tx=0x7fdd5c001650 comp rx=0 tx=0).stop 2023-10-06T04:58:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 msgr2=0x7fdd64158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64158780 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdd6406d4b0 tx=0x7fdd4c017000 comp rx=0 tx=0).stop 2023-10-06T04:58:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 shutdown_connections 2023-10-06T04:58:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd44066f30 0x7fdd440693e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd640b09b0 0x7fdd64152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd64161690 0x7fdd64158780 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 --2- 172.21.15.3:0/989495489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd640b14f0 0x7fdd64158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 >> 172.21.15.3:0/989495489 conn(0x7fdd64099a90 msgr2=0x7fdd640ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 shutdown_connections 2023-10-06T04:58:50.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.712+0000 7fdd6ab63700 1 -- 172.21.15.3:0/989495489 wait complete. 2023-10-06T04:58:50.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:58:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.846+0000 7f52cc19d700 1 Processor -- start 2023-10-06T04:58:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.846+0000 7f52cc19d700 1 -- start start 2023-10-06T04:58:50.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52cc19d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52c40b1800 0x7f52c415d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52cc19d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52c415dac0 0x7f52c415df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52cc19d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c40b6560 con 0x7f52c40b1430 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52cc19d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40b35e0 con 0x7f52c40b1800 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52cc19d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40b0640 con 0x7f52c415dac0 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52c9738700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52c40b1800 0x7f52c415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52c9738700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52c40b1800 0x7f52c415d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57216/0 (socket says 172.21.15.3:57216) 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52c9738700 1 -- 172.21.15.3:0/3877841711 learned_addr learned my addr 172.21.15.3:0/3877841711 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.847+0000 7f52ca73a700 1 --2- 172.21.15.3:0/3877841711 >> v2:172.21.15.153:3300/0 conn(0x7f52c415dac0 0x7f52c415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c9738700 1 -- 172.21.15.3:0/3877841711 >> v2:172.21.15.153:3300/0 conn(0x7f52c415dac0 msgr2=0x7f52c415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c9738700 1 --2- 172.21.15.3:0/3877841711 >> v2:172.21.15.153:3300/0 conn(0x7f52c415dac0 0x7f52c415df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c9738700 1 -- 172.21.15.3:0/3877841711 >> v1:172.21.15.153:6789/0 conn(0x7f52c40b1430 legacy=0x7f52c40aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c9738700 1 -- 172.21.15.3:0/3877841711 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40b08a0 con 0x7f52c40b1800 2023-10-06T04:58:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52ca73a700 1 --2- 172.21.15.3:0/3877841711 >> v2:172.21.15.153:3300/0 conn(0x7f52c415dac0 0x7f52c415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c9738700 1 --2- 172.21.15.3:0/3877841711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c40b1800 0x7f52c415d580 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f52c40b2850 tx=0x7f52b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff703557c1bf4b97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c8f37700 1 -- 172.21.15.3:0/3877841711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b800b860 con 0x7f52c40b1800 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c8f37700 1 -- 172.21.15.3:0/3877841711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b800b9c0 con 0x7f52c40b1800 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52c8f37700 1 -- 172.21.15.3:0/3877841711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b800bd60 con 0x7f52c40b1800 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52cc19d700 1 -- 172.21.15.3:0/3877841711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c40b1800 msgr2=0x7f52c415d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.848+0000 7f52cc19d700 1 --2- 172.21.15.3:0/3877841711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c40b1800 0x7f52c415d580 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f52c40b2850 tx=0x7f52b8009250 comp rx=0 tx=0).stop 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.849+0000 7f52cc19d700 1 -- 172.21.15.3:0/3877841711 shutdown_connections 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.849+0000 7f52cc19d700 1 --2- 172.21.15.3:0/3877841711 >> v2:172.21.15.153:3300/0 conn(0x7f52c415dac0 0x7f52c415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.849+0000 7f52cc19d700 1 --2- 172.21.15.3:0/3877841711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c40b1800 0x7f52c415d580 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.849+0000 7f52cc19d700 1 -- 172.21.15.3:0/3877841711 >> 172.21.15.3:0/3877841711 conn(0x7f52c4099a90 msgr2=0x7f52c406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.849+0000 7f52cc19d700 1 -- 172.21.15.3:0/3877841711 shutdown_connections 2023-10-06T04:58:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.849+0000 7f52cc19d700 1 -- 172.21.15.3:0/3877841711 wait complete. 2023-10-06T04:58:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.850+0000 7f52cc19d700 1 Processor -- start 2023-10-06T04:58:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.850+0000 7f52cc19d700 1 -- start start 2023-10-06T04:58:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.850+0000 7f52cc19d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c40b1430 0x7f52c4170ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.850+0000 7f52cc19d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c40b1800 0x7f52c416dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.850+0000 7f52cc19d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c415dac0 0x7f52c40ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.850+0000 7f52cc19d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c409a7e0 con 0x7f52c415dac0 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52cc19d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c416e850 con 0x7f52c40b1800 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52cc19d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c416eb30 con 0x7f52c40b1430 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c415dac0 0x7f52c40ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c415dac0 0x7f52c40ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57232/0 (socket says 172.21.15.3:57232) 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 -- 172.21.15.3:0/418067734 learned_addr learned my addr 172.21.15.3:0/418067734 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52c9f39700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c40b1430 0x7f52c4170ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52c9738700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c40b1800 0x7f52c416dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 -- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c40b1430 msgr2=0x7f52c4170ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c40b1430 0x7f52c4170ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 -- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c40b1800 msgr2=0x7f52c416dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c40b1800 0x7f52c416dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 -- 172.21.15.3:0/418067734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b80094f0 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52c9738700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c40b1800 0x7f52c416dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52ca73a700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c415dac0 0x7f52c40ae5a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009fc0 tx=0x7f52bc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52c1ffb700 1 -- 172.21.15.3:0/418067734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00dd40 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52c1ffb700 1 -- 172.21.15.3:0/418067734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc00f040 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c409a470 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.851+0000 7f52c1ffb700 1 -- 172.21.15.3:0/418067734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc015690 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.852+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c40cf870 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.852+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c409a950 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.853+0000 7f52c1ffb700 1 -- 172.21.15.3:0/418067734 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52bc01e3b0 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.853+0000 7f52c1ffb700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52b0066f30 0x7f52b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.853+0000 7f52c1ffb700 1 -- 172.21.15.3:0/418067734 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f52bc084c20 con 0x7f52c415dac0 2023-10-06T04:58:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.854+0000 7f52c9f39700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52b0066f30 0x7f52b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.854+0000 7f52c9f39700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52b0066f30 0x7f52b00693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f52b4003890 tx=0x7f52b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:50.858+0000 7f52c1ffb700 1 -- 172.21.15.3:0/418067734 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52bc051d30 con 0x7f52c415dac0 2023-10-06T04:58:51.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.047+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52c4063f80 con 0x7f52c415dac0 2023-10-06T04:58:51.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.048+0000 7f52c1ffb700 1 -- 172.21.15.3:0/418067734 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52bc01a0a0 con 0x7f52c415dac0 2023-10-06T04:58:51.050 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:58:51.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52b0066f30 msgr2=0x7f52b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52b0066f30 0x7f52b00693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f52b4003890 tx=0x7f52b4009040 comp rx=0 tx=0).stop 2023-10-06T04:58:51.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c415dac0 msgr2=0x7f52c40ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c415dac0 0x7f52c40ae5a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009fc0 tx=0x7f52bc0099b0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 shutdown_connections 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52b0066f30 0x7f52b00693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c415dac0 0x7f52c40ae5a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c40b1430 0x7f52c4170ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 --2- 172.21.15.3:0/418067734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c40b1800 0x7f52c416dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.049+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 >> 172.21.15.3:0/418067734 conn(0x7f52c4099a90 msgr2=0x7f52c40b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.050+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 shutdown_connections 2023-10-06T04:58:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.050+0000 7f52cc19d700 1 -- 172.21.15.3:0/418067734 wait complete. 2023-10-06T04:58:51.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:58:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.184+0000 7f40e8bda700 1 Processor -- start 2023-10-06T04:58:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.184+0000 7f40e8bda700 1 -- start start 2023-10-06T04:58:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e8bda700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40e40b1430 0x7f40e40b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e8bda700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e40b6560 con 0x7f40e40aff30 2023-10-06T04:58:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e8bda700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e40b08a0 con 0x7f40e415e590 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e8bda700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e40b35e0 con 0x7f40e40b1430 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e1d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40e40aff30 0x7f40e4159ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50950/0 (socket says 172.21.15.3:50950) 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e1d9b700 1 -- 172.21.15.3:0/393290666 learned_addr learned my addr 172.21.15.3:0/393290666 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e259c700 1 --2- 172.21.15.3:0/393290666 >> v2:172.21.15.153:3300/0 conn(0x7f40e40b1430 0x7f40e40b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e159a700 1 -- 172.21.15.3:0/393290666 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487395616 0 0) 0x7f40d8001110 con 0x7f40e40aff30 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e159a700 1 -- 172.21.15.3:0/393290666 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40cc003570 con 0x7f40e40aff30 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e159a700 1 -- 172.21.15.3:0/393290666 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1091383518 0 0) 0x7f40cc003570 con 0x7f40e40aff30 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e259c700 1 -- 172.21.15.3:0/393290666 >> v1:172.21.15.153:6789/0 conn(0x7f40e415e590 legacy=0x7f40e415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e259c700 1 -- 172.21.15.3:0/393290666 >> v1:172.21.15.3:6789/0 conn(0x7f40e40aff30 legacy=0x7f40e4159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.185+0000 7f40e259c700 1 -- 172.21.15.3:0/393290666 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40b0b80 con 0x7f40e40b1430 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e259c700 1 --2- 172.21.15.3:0/393290666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40b1430 0x7f40e40b1800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f40d40098d0 tx=0x7f40d4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e8a9c50020ee293 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e159a700 1 -- 172.21.15.3:0/393290666 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8001600 con 0x7f40e40b1430 2023-10-06T04:58:51.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e159a700 1 -- 172.21.15.3:0/393290666 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4011420 con 0x7f40e40b1430 2023-10-06T04:58:51.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e159a700 1 -- 172.21.15.3:0/393290666 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40117e0 con 0x7f40e40b1430 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e8bda700 1 -- 172.21.15.3:0/393290666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40b1430 msgr2=0x7f40e40b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e8bda700 1 --2- 172.21.15.3:0/393290666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40b1430 0x7f40e40b1800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f40d40098d0 tx=0x7f40d4009120 comp rx=0 tx=0).stop 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e8bda700 1 -- 172.21.15.3:0/393290666 shutdown_connections 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e8bda700 1 --2- 172.21.15.3:0/393290666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40b1430 0x7f40e40b1800 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e8bda700 1 -- 172.21.15.3:0/393290666 >> 172.21.15.3:0/393290666 conn(0x7f40e4099a90 msgr2=0x7f40e406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e8bda700 1 -- 172.21.15.3:0/393290666 shutdown_connections 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.186+0000 7f40e8bda700 1 -- 172.21.15.3:0/393290666 wait complete. 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.187+0000 7f40e8bda700 1 Processor -- start 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e8bda700 1 -- start start 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e8bda700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40aff30 0x7f40e40c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e8bda700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40e40b1430 0x7f40e40ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e8bda700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e415e590 0x7f40e40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e8bda700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4145e60 con 0x7f40e415e590 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e8bda700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e41575d0 con 0x7f40e40aff30 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e8bda700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e409c350 con 0x7f40e40b1430 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e415e590 0x7f40e40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e415e590 0x7f40e40ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57244/0 (socket says 172.21.15.3:57244) 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e2d9d700 1 -- 172.21.15.3:0/3385207349 learned_addr learned my addr 172.21.15.3:0/3385207349 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e259c700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40aff30 0x7f40e40c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.188+0000 7f40e1d9b700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40e40b1430 0x7f40e40ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e2d9d700 1 -- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40e40b1430 msgr2=0x7f40e40ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e2d9d700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40e40b1430 0x7f40e40ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e2d9d700 1 -- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40aff30 msgr2=0x7f40e40c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e2d9d700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40aff30 0x7f40e40c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e2d9d700 1 -- 172.21.15.3:0/3385207349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40093c0 con 0x7f40e415e590 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e2d9d700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e415e590 0x7f40e40ae690 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00c980 tx=0x7f40dc001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40d27fc700 1 -- 172.21.15.3:0/3385207349 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00a7e0 con 0x7f40e415e590 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40d27fc700 1 -- 172.21.15.3:0/3385207349 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc00a960 con 0x7f40e415e590 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e416e110 con 0x7f40e415e590 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40d27fc700 1 -- 172.21.15.3:0/3385207349 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00ad20 con 0x7f40e415e590 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.189+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e409c060 con 0x7f40e415e590 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.190+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e4063f80 con 0x7f40e415e590 2023-10-06T04:58:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.190+0000 7f40d27fc700 1 -- 172.21.15.3:0/3385207349 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40dc003030 con 0x7f40e415e590 2023-10-06T04:58:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.191+0000 7f40d27fc700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f40c4066f30 0x7f40c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.191+0000 7f40d27fc700 1 -- 172.21.15.3:0/3385207349 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f40dc084840 con 0x7f40e415e590 2023-10-06T04:58:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.191+0000 7f40e259c700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f40c4066f30 0x7f40c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.191+0000 7f40e259c700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f40c4066f30 0x7f40c40693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40d4003080 tx=0x7f40d4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.195+0000 7f40d27fc700 1 -- 172.21.15.3:0/3385207349 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40dc051950 con 0x7f40e415e590 2023-10-06T04:58:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.386+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40e406f9e0 con 0x7f40e415e590 2023-10-06T04:58:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.387+0000 7f40d27fc700 1 -- 172.21.15.3:0/3385207349 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40dc054f70 con 0x7f40e415e590 2023-10-06T04:58:51.388 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f40c4066f30 msgr2=0x7f40c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f40c4066f30 0x7f40c40693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40d4003080 tx=0x7f40d4017000 comp rx=0 tx=0).stop 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e415e590 msgr2=0x7f40e40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e415e590 0x7f40e40ae690 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00c980 tx=0x7f40dc001770 comp rx=0 tx=0).stop 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 shutdown_connections 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f40c4066f30 0x7f40c40693e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40e40aff30 0x7f40e40c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e415e590 0x7f40e40ae690 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 --2- 172.21.15.3:0/3385207349 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40e40b1430 0x7f40e40ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 >> 172.21.15.3:0/3385207349 conn(0x7f40e4099a90 msgr2=0x7f40e4157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 shutdown_connections 2023-10-06T04:58:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.389+0000 7f40e8bda700 1 -- 172.21.15.3:0/3385207349 wait complete. 2023-10-06T04:58:51.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:58:51.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.524+0000 7f298facb700 1 Processor -- start 2023-10-06T04:58:51.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.524+0000 7f298facb700 1 -- start start 2023-10-06T04:58:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298facb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2988159af0 0x7f29880b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298facb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2988067b10 0x7f298806c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298facb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298806ad00 con 0x7f298806ca10 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298facb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880ce290 con 0x7f2988159af0 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298facb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298806bb60 con 0x7f2988067b10 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2988159af0 0x7f29880b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2988159af0 0x7f29880b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57256/0 (socket says 172.21.15.3:57256) 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 -- 172.21.15.3:0/4095166699 learned_addr learned my addr 172.21.15.3:0/4095166699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 -- 172.21.15.3:0/4095166699 >> v2:172.21.15.153:3300/0 conn(0x7f2988067b10 msgr2=0x7f298806c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d066700 1 --2- 172.21.15.3:0/4095166699 >> v2:172.21.15.153:3300/0 conn(0x7f2988067b10 0x7f298806c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 --2- 172.21.15.3:0/4095166699 >> v2:172.21.15.153:3300/0 conn(0x7f2988067b10 0x7f298806c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 -- 172.21.15.3:0/4095166699 >> v1:172.21.15.153:6789/0 conn(0x7f298806ca10 legacy=0x7f29880a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 -- 172.21.15.3:0/4095166699 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298806ce50 con 0x7f2988159af0 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298d867700 1 --2- 172.21.15.3:0/4095166699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988159af0 0x7f29880b2c80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f29880ce490 tx=0x7f2984009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf52a5cdf8e926f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.525+0000 7f298c865700 1 -- 172.21.15.3:0/4095166699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984005c10 con 0x7f2988159af0 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298c865700 1 -- 172.21.15.3:0/4095166699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2984005d70 con 0x7f2988159af0 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298c865700 1 -- 172.21.15.3:0/4095166699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298400b840 con 0x7f2988159af0 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 -- 172.21.15.3:0/4095166699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988159af0 msgr2=0x7f29880b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 --2- 172.21.15.3:0/4095166699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988159af0 0x7f29880b2c80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f29880ce490 tx=0x7f2984009220 comp rx=0 tx=0).stop 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 -- 172.21.15.3:0/4095166699 shutdown_connections 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 --2- 172.21.15.3:0/4095166699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988159af0 0x7f29880b2c80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 --2- 172.21.15.3:0/4095166699 >> v2:172.21.15.153:3300/0 conn(0x7f2988067b10 0x7f298806c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 -- 172.21.15.3:0/4095166699 >> 172.21.15.3:0/4095166699 conn(0x7f298806d5e0 msgr2=0x7f298806d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:51.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 -- 172.21.15.3:0/4095166699 shutdown_connections 2023-10-06T04:58:51.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.526+0000 7f298facb700 1 -- 172.21.15.3:0/4095166699 wait complete. 2023-10-06T04:58:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.527+0000 7f298facb700 1 Processor -- start 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.527+0000 7f298facb700 1 -- start start 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298facb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988067b10 0x7f29880aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298facb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f298806ca10 0x7f29880af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298facb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2988159af0 0x7f29880af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298facb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988152150 con 0x7f2988067b10 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298facb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880aef70 con 0x7f2988159af0 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298facb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2988151540 con 0x7f298806ca10 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988067b10 0x7f29880aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988067b10 0x7f29880aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57268/0 (socket says 172.21.15.3:57268) 2023-10-06T04:58:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 -- 172.21.15.3:0/3712155514 learned_addr learned my addr 172.21.15.3:0/3712155514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d066700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f298806ca10 0x7f29880af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298e068700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2988159af0 0x7f29880af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 -- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f298806ca10 msgr2=0x7f29880af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f298806ca10 0x7f29880af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 -- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2988159af0 msgr2=0x7f29880af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2988159af0 0x7f29880af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298d867700 1 -- 172.21.15.3:0/3712155514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984009480 con 0x7f2988067b10 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.528+0000 7f298e068700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2988159af0 0x7f29880af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.529+0000 7f298d867700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988067b10 0x7f29880aea30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2984009260 tx=0x7f2984015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.529+0000 7f297dffb700 1 -- 172.21.15.3:0/3712155514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984011ae0 con 0x7f2988067b10 2023-10-06T04:58:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.529+0000 7f297dffb700 1 -- 172.21.15.3:0/3712155514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2984011c60 con 0x7f2988067b10 2023-10-06T04:58:51.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.529+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2988151820 con 0x7f2988067b10 2023-10-06T04:58:51.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.529+0000 7f297dffb700 1 -- 172.21.15.3:0/3712155514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298401c420 con 0x7f2988067b10 2023-10-06T04:58:51.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.529+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2988151de0 con 0x7f2988067b10 2023-10-06T04:58:51.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.530+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2988063f80 con 0x7f2988067b10 2023-10-06T04:58:51.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.530+0000 7f297dffb700 1 -- 172.21.15.3:0/3712155514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f298401c790 con 0x7f2988067b10 2023-10-06T04:58:51.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.531+0000 7f297dffb700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.531+0000 7f297dffb700 1 -- 172.21.15.3:0/3712155514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2984085f80 con 0x7f2988067b10 2023-10-06T04:58:51.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.531+0000 7f298d066700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.531+0000 7f298d066700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2988152a90 tx=0x7f2978008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.535+0000 7f297dffb700 1 -- 172.21.15.3:0/3712155514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2984052ff0 con 0x7f2988067b10 2023-10-06T04:58:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.729+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29880cd8b0 con 0x7f2988067b10 2023-10-06T04:58:51.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.729+0000 7f297dffb700 1 -- 172.21.15.3:0/3712155514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2984056610 con 0x7f2988067b10 2023-10-06T04:58:51.731 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:58:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 msgr2=0x7f296c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2988152a90 tx=0x7f2978008040 comp rx=0 tx=0).stop 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988067b10 msgr2=0x7f29880aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988067b10 0x7f29880aea30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2984009260 tx=0x7f2984015000 comp rx=0 tx=0).stop 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 shutdown_connections 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2988159af0 0x7f29880af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2988067b10 0x7f29880aea30 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 --2- 172.21.15.3:0/3712155514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f298806ca10 0x7f29880af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 >> 172.21.15.3:0/3712155514 conn(0x7f298806d5e0 msgr2=0x7f298815f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 shutdown_connections 2023-10-06T04:58:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.732+0000 7f298facb700 1 -- 172.21.15.3:0/3712155514 wait complete. 2023-10-06T04:58:51.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:51.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.873+0000 7f5a012f8700 1 Processor -- start 2023-10-06T04:58:51.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.873+0000 7f5a012f8700 1 -- start start 2023-10-06T04:58:51.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f5a012f8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f59fc159740 0x7f59fc15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f5a012f8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59fc15e070 0x7f59fc15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f5a012f8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc0b18c0 con 0x7f59fc159b10 2023-10-06T04:58:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f5a012f8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc0b09d0 con 0x7f59fc15e070 2023-10-06T04:58:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f5a012f8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc0b0cb0 con 0x7f59fc159740 2023-10-06T04:58:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59fc15e070 0x7f59fc15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59fc15e070 0x7f59fc15d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57280/0 (socket says 172.21.15.3:57280) 2023-10-06T04:58:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 -- 172.21.15.3:0/823551763 learned_addr learned my addr 172.21.15.3:0/823551763 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59faffd700 1 --2- 172.21.15.3:0/823551763 >> v2:172.21.15.153:3300/0 conn(0x7f59fc159740 0x7f59fc15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 -- 172.21.15.3:0/823551763 >> v2:172.21.15.153:3300/0 conn(0x7f59fc159740 msgr2=0x7f59fc15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 --2- 172.21.15.3:0/823551763 >> v2:172.21.15.153:3300/0 conn(0x7f59fc159740 0x7f59fc15d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 -- 172.21.15.3:0/823551763 >> v1:172.21.15.153:6789/0 conn(0x7f59fc159b10 legacy=0x7f59fc0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 -- 172.21.15.3:0/823551763 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc0b0f90 con 0x7f59fc15e070 2023-10-06T04:58:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.874+0000 7f59fa7fc700 1 --2- 172.21.15.3:0/823551763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15e070 0x7f59fc15d9e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0aff90 tx=0x7f59f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc1c9909a2bf3aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f59f9ffb700 1 -- 172.21.15.3:0/823551763 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000b820 con 0x7f59fc15e070 2023-10-06T04:58:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f59f9ffb700 1 -- 172.21.15.3:0/823551763 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f000b980 con 0x7f59fc15e070 2023-10-06T04:58:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f59f9ffb700 1 -- 172.21.15.3:0/823551763 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000bd20 con 0x7f59fc15e070 2023-10-06T04:58:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 -- 172.21.15.3:0/823551763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15e070 msgr2=0x7f59fc15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 --2- 172.21.15.3:0/823551763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15e070 0x7f59fc15d9e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0aff90 tx=0x7f59f0009250 comp rx=0 tx=0).stop 2023-10-06T04:58:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 -- 172.21.15.3:0/823551763 shutdown_connections 2023-10-06T04:58:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 --2- 172.21.15.3:0/823551763 >> v2:172.21.15.153:3300/0 conn(0x7f59fc159740 0x7f59fc15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 --2- 172.21.15.3:0/823551763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc15e070 0x7f59fc15d9e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 -- 172.21.15.3:0/823551763 >> 172.21.15.3:0/823551763 conn(0x7f59fc099a90 msgr2=0x7f59fc06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 -- 172.21.15.3:0/823551763 shutdown_connections 2023-10-06T04:58:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.875+0000 7f5a012f8700 1 -- 172.21.15.3:0/823551763 wait complete. 2023-10-06T04:58:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.876+0000 7f5a012f8700 1 Processor -- start 2023-10-06T04:58:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f5a012f8700 1 -- start start 2023-10-06T04:58:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f5a012f8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59fc159740 0x7f59fc14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f5a012f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc159b10 0x7f59fc06e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f5a012f8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59fc15e070 0x7f59fc0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f5a012f8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc169a80 con 0x7f59fc159b10 2023-10-06T04:58:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f5a012f8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc14df10 con 0x7f59fc15e070 2023-10-06T04:58:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f5a012f8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc16a590 con 0x7f59fc159740 2023-10-06T04:58:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f59fa7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc159b10 0x7f59fc06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f59fa7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc159b10 0x7f59fc06e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57286/0 (socket says 172.21.15.3:57286) 2023-10-06T04:58:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.877+0000 7f59fa7fc700 1 -- 172.21.15.3:0/3061140768 learned_addr learned my addr 172.21.15.3:0/3061140768 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59faffd700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59fc159740 0x7f59fc14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59fb7fe700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59fc15e070 0x7f59fc0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59fa7fc700 1 -- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59fc159740 msgr2=0x7f59fc14d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59fa7fc700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59fc159740 0x7f59fc14d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59fa7fc700 1 -- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59fc15e070 msgr2=0x7f59fc0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59fa7fc700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59fc15e070 0x7f59fc0b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59fa7fc700 1 -- 172.21.15.3:0/3061140768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00094f0 con 0x7f59fc159b10 2023-10-06T04:58:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59fa7fc700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc159b10 0x7f59fc06e750 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f59f0003080 tx=0x7f59f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59e2ffd700 1 -- 172.21.15.3:0/3061140768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0011e20 con 0x7f59fc159b10 2023-10-06T04:58:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59e2ffd700 1 -- 172.21.15.3:0/3061140768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f0020070 con 0x7f59fc159b10 2023-10-06T04:58:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc16aa20 con 0x7f59fc159b10 2023-10-06T04:58:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f59e2ffd700 1 -- 172.21.15.3:0/3061140768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f001b6a0 con 0x7f59fc159b10 2023-10-06T04:58:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.878+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc169f40 con 0x7f59fc159b10 2023-10-06T04:58:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.879+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc16a720 con 0x7f59fc159b10 2023-10-06T04:58:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.880+0000 7f59e2ffd700 1 -- 172.21.15.3:0/3061140768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59f0022030 con 0x7f59fc159b10 2023-10-06T04:58:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.880+0000 7f59e2ffd700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066f30 0x7f59e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.880+0000 7f59e2ffd700 1 -- 172.21.15.3:0/3061140768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f59f00866f0 con 0x7f59fc159b10 2023-10-06T04:58:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.880+0000 7f59faffd700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066f30 0x7f59e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.881+0000 7f59faffd700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066f30 0x7f59e80693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0b2790 tx=0x7f59ec009810 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:51.884+0000 7f59e2ffd700 1 -- 172.21.15.3:0/3061140768 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59f0053800 con 0x7f59fc159b10 2023-10-06T04:58:52.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.122+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59fc16a720 con 0x7f59fc159b10 2023-10-06T04:58:52.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.122+0000 7f59e2ffd700 1 -- 172.21.15.3:0/3061140768 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f59f0056e20 con 0x7f59fc159b10 2023-10-06T04:58:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:52.124 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:58:28.114721+0000","modified":"2023-10-06T04:58:48.790877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.153:6835/2492546877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2492546877},{"type":"v1","addr":"172.21.15.153:6835","nonce":2492546877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066f30 msgr2=0x7f59e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066f30 0x7f59e80693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0b2790 tx=0x7f59ec009810 comp rx=0 tx=0).stop 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc159b10 msgr2=0x7f59fc06e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc159b10 0x7f59fc06e750 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f59f0003080 tx=0x7f59f0016000 comp rx=0 tx=0).stop 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 shutdown_connections 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59e8066f30 0x7f59e80693e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59fc15e070 0x7f59fc0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59fc159740 0x7f59fc14d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 --2- 172.21.15.3:0/3061140768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59fc159b10 0x7f59fc06e750 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 >> 172.21.15.3:0/3061140768 conn(0x7f59fc099a90 msgr2=0x7f59fc0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 shutdown_connections 2023-10-06T04:58:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.125+0000 7f5a012f8700 1 -- 172.21.15.3:0/3061140768 wait complete. 2023-10-06T04:58:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 51 2023-10-06T04:58:52.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:52.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.256+0000 7fa5e359e700 1 Processor -- start 2023-10-06T04:58:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.256+0000 7fa5e359e700 1 -- start start 2023-10-06T04:58:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5e406d5e0 0x7fa5e406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e359e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40cd300 con 0x7fa5e406dfc0 2023-10-06T04:58:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e359e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40cc9d0 con 0x7fa5e415ba70 2023-10-06T04:58:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e359e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e40cf250 con 0x7fa5e406d5e0 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5e406d5e0 0x7fa5e406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa5e406dfc0 0x7fa5e415be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42826/0 (socket says 172.21.15.3:42826) 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e2d9d700 1 -- 172.21.15.3:0/1644616004 learned_addr learned my addr 172.21.15.3:0/1644616004 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e1d9b700 1 -- 172.21.15.3:0/1644616004 >> v1:172.21.15.153:6789/0 conn(0x7fa5e415ba70 legacy=0x7fa5e40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e1d9b700 1 -- 172.21.15.3:0/1644616004 >> v1:172.21.15.3:6789/0 conn(0x7fa5e406dfc0 legacy=0x7fa5e415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e1d9b700 1 -- 172.21.15.3:0/1644616004 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40cccb0 con 0x7fa5e406d5e0 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.257+0000 7fa5e159a700 1 -- 172.21.15.3:0/1644616004 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72085476 0 0) 0x7fa5e40cd300 con 0x7fa5e406dfc0 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e1d9b700 1 --2- 172.21.15.3:0/1644616004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406d5e0 0x7fa5e406da80 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0098e0 tx=0x7fa5cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b189825aaf9337f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e159a700 1 -- 172.21.15.3:0/1644616004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc009e80 con 0x7fa5e406d5e0 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e159a700 1 -- 172.21.15.3:0/1644616004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc00b860 con 0x7fa5e406d5e0 2023-10-06T04:58:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e159a700 1 -- 172.21.15.3:0/1644616004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00bbe0 con 0x7fa5e406d5e0 2023-10-06T04:58:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e359e700 1 -- 172.21.15.3:0/1644616004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406d5e0 msgr2=0x7fa5e406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e359e700 1 --2- 172.21.15.3:0/1644616004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406d5e0 0x7fa5e406da80 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0098e0 tx=0x7fa5cc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e359e700 1 -- 172.21.15.3:0/1644616004 shutdown_connections 2023-10-06T04:58:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e359e700 1 --2- 172.21.15.3:0/1644616004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406d5e0 0x7fa5e406da80 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e359e700 1 -- 172.21.15.3:0/1644616004 >> 172.21.15.3:0/1644616004 conn(0x7fa5e406ccb0 msgr2=0x7fa5e40ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e359e700 1 -- 172.21.15.3:0/1644616004 shutdown_connections 2023-10-06T04:58:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.258+0000 7fa5e359e700 1 -- 172.21.15.3:0/1644616004 wait complete. 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.259+0000 7fa5e359e700 1 Processor -- start 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.259+0000 7fa5e359e700 1 -- start start 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e359e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e406d5e0 0x7fa5e416a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e359e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5e406dfc0 0x7fa5e415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e415ba70 0x7fa5e40c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e415c530 con 0x7fa5e415ba70 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e359e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40c6560 con 0x7fa5e406d5e0 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e359e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e415bee0 con 0x7fa5e406dfc0 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e415ba70 0x7fa5e40c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e415ba70 0x7fa5e40c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33764/0 (socket says 172.21.15.3:33764) 2023-10-06T04:58:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 -- 172.21.15.3:0/3427769852 learned_addr learned my addr 172.21.15.3:0/3427769852 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e259c700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e406d5e0 0x7fa5e416a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e1d9b700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5e406dfc0 0x7fa5e415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 -- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5e406dfc0 msgr2=0x7fa5e415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5e406dfc0 0x7fa5e415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 -- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e406d5e0 msgr2=0x7fa5e416a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e406d5e0 0x7fa5e416a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e2d9d700 1 -- 172.21.15.3:0/3427769852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc009420 con 0x7fa5e415ba70 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e259c700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e406d5e0 0x7fa5e416a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.260+0000 7fa5e1d9b700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5e406dfc0 0x7fa5e415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.261+0000 7fa5e2d9d700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e415ba70 0x7fa5e40c5d10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400cc90 tx=0x7fa5d4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.261+0000 7fa5da7fc700 1 -- 172.21.15.3:0/3427769852 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400cea0 con 0x7fa5e415ba70 2023-10-06T04:58:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.261+0000 7fa5da7fc700 1 -- 172.21.15.3:0/3427769852 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d400ab70 con 0x7fa5e415ba70 2023-10-06T04:58:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.261+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e415c1c0 con 0x7fa5e415ba70 2023-10-06T04:58:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.261+0000 7fa5da7fc700 1 -- 172.21.15.3:0/3427769852 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40070f0 con 0x7fa5e415ba70 2023-10-06T04:58:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.261+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e40c6840 con 0x7fa5e415ba70 2023-10-06T04:58:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.262+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e415c350 con 0x7fa5e415ba70 2023-10-06T04:58:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.262+0000 7fa5da7fc700 1 -- 172.21.15.3:0/3427769852 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5d40074a0 con 0x7fa5e415ba70 2023-10-06T04:58:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.262+0000 7fa5da7fc700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa5c4066f30 0x7fa5c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.263+0000 7fa5da7fc700 1 -- 172.21.15.3:0/3427769852 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa5d4084b70 con 0x7fa5e415ba70 2023-10-06T04:58:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.263+0000 7fa5e259c700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa5c4066f30 0x7fa5c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.263+0000 7fa5e259c700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa5c4066f30 0x7fa5c40693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc007fd0 tx=0x7fa5dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.267+0000 7fa5da7fc700 1 -- 172.21.15.3:0/3427769852 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5d4051d00 con 0x7fa5e415ba70 2023-10-06T04:58:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.461+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5e4063f80 con 0x7fa5e415ba70 2023-10-06T04:58:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.462+0000 7fa5da7fc700 1 -- 172.21.15.3:0/3427769852 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16821 (secure 0 0 0) 0x7fa5d4055320 con 0x7fa5e415ba70 2023-10-06T04:58:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:52.464 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:58:29.742286+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:58:25.874792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T04:58:27.107107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa5c4066f30 msgr2=0x7fa5c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa5c4066f30 0x7fa5c40693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc007fd0 tx=0x7fa5dc009040 comp rx=0 tx=0).stop 2023-10-06T04:58:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e415ba70 msgr2=0x7fa5e40c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e415ba70 0x7fa5e40c5d10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400cc90 tx=0x7fa5d4001a80 comp rx=0 tx=0).stop 2023-10-06T04:58:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 shutdown_connections 2023-10-06T04:58:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa5c4066f30 0x7fa5c40693e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e406d5e0 0x7fa5e416a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e415ba70 0x7fa5e40c5d10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 --2- 172.21.15.3:0/3427769852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5e406dfc0 0x7fa5e415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 >> 172.21.15.3:0/3427769852 conn(0x7fa5e406ccb0 msgr2=0x7fa5e4159340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 shutdown_connections 2023-10-06T04:58:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.464+0000 7fa5e359e700 1 -- 172.21.15.3:0/3427769852 wait complete. 2023-10-06T04:58:52.478 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:58:52.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.596+0000 7f9c423de700 1 Processor -- start 2023-10-06T04:58:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.596+0000 7f9c423de700 1 -- start start 2023-10-06T04:58:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c423de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c3c15dbb0 0x7f9c3c0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c423de700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c15e070 0x7f9c3c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c423de700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c0b18c0 con 0x7f9c3c159740 2023-10-06T04:58:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c423de700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b09d0 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c423de700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b0cb0 con 0x7f9c3c15e070 2023-10-06T04:58:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c3c15dbb0 0x7f9c3c0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c3c15dbb0 0x7f9c3c0b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33770/0 (socket says 172.21.15.3:33770) 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/3731328599 learned_addr learned my addr 172.21.15.3:0/3731328599 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/3731328599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c15e070 msgr2=0x7f9c3c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c4097b700 1 --2- 172.21.15.3:0/3731328599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c15e070 0x7f9c3c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 --2- 172.21.15.3:0/3731328599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c15e070 0x7f9c3c0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/3731328599 >> v1:172.21.15.153:6789/0 conn(0x7f9c3c159740 legacy=0x7f9c3c15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/3731328599 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0b0f90 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c4097b700 1 --2- 172.21.15.3:0/3731328599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c15e070 0x7f9c3c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3b7fe700 1 --2- 172.21.15.3:0/3731328599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c0b3870 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0099b0 tx=0x7f9c2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2945aac814886372 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3affd700 1 -- 172.21.15.3:0/3731328599 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00b820 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3affd700 1 -- 172.21.15.3:0/3731328599 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c00b980 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.597+0000 7f9c3affd700 1 -- 172.21.15.3:0/3731328599 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00bd20 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 -- 172.21.15.3:0/3731328599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 msgr2=0x7f9c3c0b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 --2- 172.21.15.3:0/3731328599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c0b3870 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0099b0 tx=0x7f9c2c009250 comp rx=0 tx=0).stop 2023-10-06T04:58:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 -- 172.21.15.3:0/3731328599 shutdown_connections 2023-10-06T04:58:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 --2- 172.21.15.3:0/3731328599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c15e070 0x7f9c3c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 --2- 172.21.15.3:0/3731328599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c0b3870 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 -- 172.21.15.3:0/3731328599 >> 172.21.15.3:0/3731328599 conn(0x7f9c3c099a90 msgr2=0x7f9c3c06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 -- 172.21.15.3:0/3731328599 shutdown_connections 2023-10-06T04:58:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.598+0000 7f9c423de700 1 -- 172.21.15.3:0/3731328599 wait complete. 2023-10-06T04:58:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.599+0000 7f9c423de700 1 Processor -- start 2023-10-06T04:58:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.599+0000 7f9c423de700 1 -- start start 2023-10-06T04:58:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c423de700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c159740 0x7f9c3c09bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c423de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c423de700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c3c15e070 0x7f9c3c15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c423de700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c09cd70 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c423de700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c155f90 con 0x7f9c3c15e070 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c423de700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c09c140 con 0x7f9c3c159740 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33776/0 (socket says 172.21.15.3:33776) 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/4190338454 learned_addr learned my addr 172.21.15.3:0/4190338454 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c4097b700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c3c15e070 0x7f9c3c15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3bfff700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c159740 0x7f9c3c09bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c159740 msgr2=0x7f9c3c09bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c159740 0x7f9c3c09bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c3c15e070 msgr2=0x7f9c3c15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c3c15e070 0x7f9c3c15c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.600+0000 7f9c3b7fe700 1 -- 172.21.15.3:0/4190338454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0094f0 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.601+0000 7f9c3b7fe700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c155a50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009290 tx=0x7f9c2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.601+0000 7f9c33fff700 1 -- 172.21.15.3:0/4190338454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c011c10 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.601+0000 7f9c33fff700 1 -- 172.21.15.3:0/4190338454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c020070 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.601+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c09c720 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.601+0000 7f9c33fff700 1 -- 172.21.15.3:0/4190338454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c01b450 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.601+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c09c420 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.602+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c063f80 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.602+0000 7f9c33fff700 1 -- 172.21.15.3:0/4190338454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c2c022030 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.602+0000 7f9c33fff700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c1c066f30 0x7f9c1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.603+0000 7f9c33fff700 1 -- 172.21.15.3:0/4190338454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9c2c086520 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.603+0000 7f9c3bfff700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c1c066f30 0x7f9c1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.603+0000 7f9c3bfff700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c1c066f30 0x7f9c1c0693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0b0130 tx=0x7f9c24009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.607+0000 7f9c33fff700 1 -- 172.21.15.3:0/4190338454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c2c053630 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.845+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c3c063f80 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.845+0000 7f9c33fff700 1 -- 172.21.15.3:0/4190338454 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f9c2c056c50 con 0x7f9c3c15dbb0 2023-10-06T04:58:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:52.847 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:58:28.114721+0000","modified":"2023-10-06T04:58:48.790877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.153:6835/2492546877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2492546877},{"type":"v1","addr":"172.21.15.153:6835","nonce":2492546877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T04:58:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c1c066f30 msgr2=0x7f9c1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c1c066f30 0x7f9c1c0693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0b0130 tx=0x7f9c24009060 comp rx=0 tx=0).stop 2023-10-06T04:58:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 msgr2=0x7f9c3c155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c155a50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009290 tx=0x7f9c2c016000 comp rx=0 tx=0).stop 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 shutdown_connections 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9c1c066f30 0x7f9c1c0693e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c3c15e070 0x7f9c3c15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c3c159740 0x7f9c3c09bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 --2- 172.21.15.3:0/4190338454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c3c15dbb0 0x7f9c3c155a50 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.847+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 >> 172.21.15.3:0/4190338454 conn(0x7f9c3c099a90 msgr2=0x7f9c3c157430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.848+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 shutdown_connections 2023-10-06T04:58:52.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.848+0000 7f9c423de700 1 -- 172.21.15.3:0/4190338454 wait complete. 2023-10-06T04:58:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 51 2023-10-06T04:58:52.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.978+0000 7f33819a6700 1 Processor -- start 2023-10-06T04:58:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f33819a6700 1 -- start start 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f33819a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f337c159890 0x7f337c0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f33819a6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f337c0b2ec0 0x7f337c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f33819a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f33819a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337c0b43e0 con 0x7f337c159890 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f33819a6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337c0affe0 con 0x7f337c0b2ec0 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f33819a6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c0b45d0 con 0x7f337c0b28c0 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.979+0000 7f337affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f337c159890 0x7f337c0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f337c159890 0x7f337c0b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33784/0 (socket says 172.21.15.3:33784) 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 -- 172.21.15.3:0/3106090155 learned_addr learned my addr 172.21.15.3:0/3106090155 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337a7fc700 1 --2- 172.21.15.3:0/3106090155 >> v2:172.21.15.153:3300/0 conn(0x7f337c0b2ec0 0x7f337c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337b7fe700 1 --2- 172.21.15.3:0/3106090155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 -- 172.21.15.3:0/3106090155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 msgr2=0x7f337c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 --2- 172.21.15.3:0/3106090155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 -- 172.21.15.3:0/3106090155 >> v2:172.21.15.153:3300/0 conn(0x7f337c0b2ec0 msgr2=0x7f337c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 --2- 172.21.15.3:0/3106090155 >> v2:172.21.15.153:3300/0 conn(0x7f337c0b2ec0 0x7f337c0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 -- 172.21.15.3:0/3106090155 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c0b48b0 con 0x7f337c159890 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337a7fc700 1 --2- 172.21.15.3:0/3106090155 >> v2:172.21.15.153:3300/0 conn(0x7f337c0b2ec0 0x7f337c0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337b7fe700 1 --2- 172.21.15.3:0/3106090155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f337affd700 1 --2- 172.21.15.3:0/3106090155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c0b1e40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f337c0ab730 tx=0x7f3374009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232bcbeb1402da4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f3379ffb700 1 -- 172.21.15.3:0/3106090155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374005c10 con 0x7f337c159890 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f3379ffb700 1 -- 172.21.15.3:0/3106090155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3374005d70 con 0x7f337c159890 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.980+0000 7f3379ffb700 1 -- 172.21.15.3:0/3106090155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337400b880 con 0x7f337c159890 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 -- 172.21.15.3:0/3106090155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 msgr2=0x7f337c0b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 --2- 172.21.15.3:0/3106090155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c0b1e40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f337c0ab730 tx=0x7f3374009220 comp rx=0 tx=0).stop 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 -- 172.21.15.3:0/3106090155 shutdown_connections 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 --2- 172.21.15.3:0/3106090155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 --2- 172.21.15.3:0/3106090155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c0b1e40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 --2- 172.21.15.3:0/3106090155 >> v2:172.21.15.153:3300/0 conn(0x7f337c0b2ec0 0x7f337c0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 -- 172.21.15.3:0/3106090155 >> 172.21.15.3:0/3106090155 conn(0x7f337c099a90 msgr2=0x7f337c06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 -- 172.21.15.3:0/3106090155 shutdown_connections 2023-10-06T04:58:52.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.981+0000 7f33819a6700 1 -- 172.21.15.3:0/3106090155 wait complete. 2023-10-06T04:58:52.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.982+0000 7f33819a6700 1 Processor -- start 2023-10-06T04:58:52.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.982+0000 7f33819a6700 1 -- start start 2023-10-06T04:58:52.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f33819a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c09c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f33819a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c0b2ec0 0x7f337c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f33819a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c09a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f33819a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c09cd50 con 0x7f337c159890 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f33819a6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c09b760 con 0x7f337c0b2ec0 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f33819a6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c0ac730 con 0x7f337c0b28c0 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c09a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c09a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33794/0 (socket says 172.21.15.3:33794) 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 -- 172.21.15.3:0/2150738954 learned_addr learned my addr 172.21.15.3:0/2150738954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337affd700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c09c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337a7fc700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c0b2ec0 0x7f337c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 -- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 msgr2=0x7f337c09c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c09c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 -- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c0b2ec0 msgr2=0x7f337c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c0b2ec0 0x7f337c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 -- 172.21.15.3:0/2150738954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3374009480 con 0x7f337c159890 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337affd700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c09c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337a7fc700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c0b2ec0 0x7f337c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f337b7fe700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c09a8c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f336c00d180 tx=0x7f336c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f3372ffd700 1 -- 172.21.15.3:0/2150738954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c00de20 con 0x7f337c159890 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f3372ffd700 1 -- 172.21.15.3:0/2150738954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c00f040 con 0x7f337c159890 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.983+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c0accf0 con 0x7f337c159890 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.984+0000 7f3372ffd700 1 -- 172.21.15.3:0/2150738954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c015670 con 0x7f337c159890 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.984+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c09d2c0 con 0x7f337c159890 2023-10-06T04:58:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.984+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c0acaa0 con 0x7f337c159890 2023-10-06T04:58:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.985+0000 7f3372ffd700 1 -- 172.21.15.3:0/2150738954 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f336c01e3b0 con 0x7f337c159890 2023-10-06T04:58:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.985+0000 7f3372ffd700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f335c066f30 0x7f335c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.985+0000 7f3372ffd700 1 -- 172.21.15.3:0/2150738954 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f336c084c40 con 0x7f337c159890 2023-10-06T04:58:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.985+0000 7f337affd700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f335c066f30 0x7f335c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.986+0000 7f337affd700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f335c066f30 0x7f335c0693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3374009260 tx=0x7f3374016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:52.989+0000 7f3372ffd700 1 -- 172.21.15.3:0/2150738954 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f336c051d50 con 0x7f337c159890 2023-10-06T04:58:53.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.227+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f337c089780 con 0x7f337c159890 2023-10-06T04:58:53.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.228+0000 7f3372ffd700 1 -- 172.21.15.3:0/2150738954 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f336c01a090 con 0x7f337c159890 2023-10-06T04:58:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:53.229 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:58:28.114721+0000","modified":"2023-10-06T04:58:48.790877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.153:6835/2492546877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2492546877},{"type":"v1","addr":"172.21.15.153:6835","nonce":2492546877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f335c066f30 msgr2=0x7f335c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f335c066f30 0x7f335c0693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3374009260 tx=0x7f3374016000 comp rx=0 tx=0).stop 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 msgr2=0x7f337c09a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c09a8c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f336c00d180 tx=0x7f336c009a50 comp rx=0 tx=0).stop 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 shutdown_connections 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f335c066f30 0x7f335c0693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c159890 0x7f337c09a8c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f337c0b28c0 0x7f337c09c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 --2- 172.21.15.3:0/2150738954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c0b2ec0 0x7f337c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 >> 172.21.15.3:0/2150738954 conn(0x7f337c099a90 msgr2=0x7f337c15a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:53.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 shutdown_connections 2023-10-06T04:58:53.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.230+0000 7f33819a6700 1 -- 172.21.15.3:0/2150738954 wait complete. 2023-10-06T04:58:53.233 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 51 2023-10-06T04:58:53.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:58:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.361+0000 7f4500a86700 1 Processor -- start 2023-10-06T04:58:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.361+0000 7f4500a86700 1 -- start start 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f4500a86700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44fc0cee60 0x7f44fc0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f4500a86700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44fc0b1d60 0x7f44fc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f4500a86700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc0b18d0 con 0x7f44fc0c8a50 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f4500a86700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc0c9e90 con 0x7f44fc0cee60 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f4500a86700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc0ca170 con 0x7f44fc0b1d60 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f44f9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44fc0cee60 0x7f44fc0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f44f9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44fc0cee60 0x7f44fc0c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33810/0 (socket says 172.21.15.3:33810) 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f44f9d9b700 1 -- 172.21.15.3:0/4053319750 learned_addr learned my addr 172.21.15.3:0/4053319750 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.362+0000 7f44fa59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44fc0c8a50 0x7f44fc0c8e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42840/0 (socket says 172.21.15.3:42840) 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44fad9d700 1 --2- 172.21.15.3:0/4053319750 >> v2:172.21.15.153:3300/0 conn(0x7f44fc0b1d60 0x7f44fc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f9d9b700 1 -- 172.21.15.3:0/4053319750 >> v2:172.21.15.153:3300/0 conn(0x7f44fc0b1d60 msgr2=0x7f44fc0b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f9d9b700 1 --2- 172.21.15.3:0/4053319750 >> v2:172.21.15.153:3300/0 conn(0x7f44fc0b1d60 0x7f44fc0b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f9d9b700 1 -- 172.21.15.3:0/4053319750 >> v1:172.21.15.3:6789/0 conn(0x7f44fc0c8a50 legacy=0x7f44fc0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f9d9b700 1 -- 172.21.15.3:0/4053319750 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc0b1280 con 0x7f44fc0cee60 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f959a700 1 -- 172.21.15.3:0/4053319750 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058530880 0 0) 0x7f44fc0b18d0 con 0x7f44fc0c8a50 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44fad9d700 1 --2- 172.21.15.3:0/4053319750 >> v2:172.21.15.153:3300/0 conn(0x7f44fc0b1d60 0x7f44fc0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f9d9b700 1 --2- 172.21.15.3:0/4053319750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc0c9530 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0cc260 tx=0x7f44e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9062cea04c4e3b8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f959a700 1 -- 172.21.15.3:0/4053319750 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400b8f0 con 0x7f44fc0cee60 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f959a700 1 -- 172.21.15.3:0/4053319750 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e400ba50 con 0x7f44fc0cee60 2023-10-06T04:58:53.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.363+0000 7f44f959a700 1 -- 172.21.15.3:0/4053319750 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400bdf0 con 0x7f44fc0cee60 2023-10-06T04:58:53.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 -- 172.21.15.3:0/4053319750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 msgr2=0x7f44fc0c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 --2- 172.21.15.3:0/4053319750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc0c9530 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0cc260 tx=0x7f44e4009250 comp rx=0 tx=0).stop 2023-10-06T04:58:53.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 -- 172.21.15.3:0/4053319750 shutdown_connections 2023-10-06T04:58:53.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 --2- 172.21.15.3:0/4053319750 >> v2:172.21.15.153:3300/0 conn(0x7f44fc0b1d60 0x7f44fc0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 --2- 172.21.15.3:0/4053319750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc0c9530 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 -- 172.21.15.3:0/4053319750 >> 172.21.15.3:0/4053319750 conn(0x7f44fc06ccb0 msgr2=0x7f44fc0ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 -- 172.21.15.3:0/4053319750 shutdown_connections 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.364+0000 7f4500a86700 1 -- 172.21.15.3:0/4053319750 wait complete. 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.365+0000 7f4500a86700 1 Processor -- start 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.365+0000 7f4500a86700 1 -- start start 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f4500a86700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44fc0b1d60 0x7f44fc1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f4500a86700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44fc0c8a50 0x7f44fc15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f4500a86700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f4500a86700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc06e1f0 con 0x7f44fc0cee60 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f4500a86700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc155720 con 0x7f44fc0b1d60 2023-10-06T04:58:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f4500a86700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc06d5e0 con 0x7f44fc0c8a50 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc15c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33812/0 (socket says 172.21.15.3:33812) 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 -- 172.21.15.3:0/744281307 learned_addr learned my addr 172.21.15.3:0/744281307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44f9d9b700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44fc0c8a50 0x7f44fc15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fa59c700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44fc0b1d60 0x7f44fc1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 -- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44fc0c8a50 msgr2=0x7f44fc15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44fc0c8a50 0x7f44fc15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 -- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44fc0b1d60 msgr2=0x7f44fc1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44fc0b1d60 0x7f44fc1551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 -- 172.21.15.3:0/744281307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40094f0 con 0x7f44fc0cee60 2023-10-06T04:58:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44fad9d700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc15c900 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c850 tx=0x7f44ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44f27fc700 1 -- 172.21.15.3:0/744281307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec011600 con 0x7f44fc0cee60 2023-10-06T04:58:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.366+0000 7f44f27fc700 1 -- 172.21.15.3:0/744281307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec011760 con 0x7f44fc0cee60 2023-10-06T04:58:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.367+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc06d8c0 con 0x7f44fc0cee60 2023-10-06T04:58:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.367+0000 7f44f27fc700 1 -- 172.21.15.3:0/744281307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec011b20 con 0x7f44fc0cee60 2023-10-06T04:58:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.367+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc06de80 con 0x7f44fc0cee60 2023-10-06T04:58:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.367+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc06e360 con 0x7f44fc0cee60 2023-10-06T04:58:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.368+0000 7f44f27fc700 1 -- 172.21.15.3:0/744281307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44ec018350 con 0x7f44fc0cee60 2023-10-06T04:58:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.368+0000 7f44f27fc700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f30 0x7f44dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.368+0000 7f44f27fc700 1 -- 172.21.15.3:0/744281307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44ec084480 con 0x7f44fc0cee60 2023-10-06T04:58:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.368+0000 7f44fa59c700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f30 0x7f44dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.369+0000 7f44fa59c700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f30 0x7f44dc0693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f44f4001620 tx=0x7f44f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.372+0000 7f44f27fc700 1 -- 172.21.15.3:0/744281307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44ec055020 con 0x7f44fc0cee60 2023-10-06T04:58:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.566+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44fc06e580 con 0x7f44fc0cee60 2023-10-06T04:58:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.567+0000 7f44f27fc700 1 -- 172.21.15.3:0/744281307 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f44ec054bb0 con 0x7f44fc0cee60 2023-10-06T04:58:53.569 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:53.569 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:58:29.742286+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:58:25.874792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T04:58:27.107107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:58:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f30 msgr2=0x7f44dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f30 0x7f44dc0693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f44f4001620 tx=0x7f44f4009040 comp rx=0 tx=0).stop 2023-10-06T04:58:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 msgr2=0x7f44fc15c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc15c900 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c850 tx=0x7f44ec008040 comp rx=0 tx=0).stop 2023-10-06T04:58:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 shutdown_connections 2023-10-06T04:58:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f30 0x7f44dc0693e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44fc0b1d60 0x7f44fc1551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc0cee60 0x7f44fc15c900 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 --2- 172.21.15.3:0/744281307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44fc0c8a50 0x7f44fc15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 >> 172.21.15.3:0/744281307 conn(0x7f44fc06ccb0 msgr2=0x7f44fc0b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:53.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 shutdown_connections 2023-10-06T04:58:53.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.569+0000 7f4500a86700 1 -- 172.21.15.3:0/744281307 wait complete. 2023-10-06T04:58:53.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:58:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.702+0000 7f3b724b7700 1 Processor -- start 2023-10-06T04:58:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.702+0000 7f3b724b7700 1 -- start start 2023-10-06T04:58:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b724b7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c159890 0x7f3b6c0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b724b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b6c0b2ec0 0x7f3b6c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b724b7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b6c0affe0 con 0x7f3b6c0b28c0 2023-10-06T04:58:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b724b7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6c0b43e0 con 0x7f3b6c0b2ec0 2023-10-06T04:58:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b724b7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0b45d0 con 0x7f3b6c159890 2023-10-06T04:58:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b6b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b6c0b2ec0 0x7f3b6c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b6b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b6c0b2ec0 0x7f3b6c0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33824/0 (socket says 172.21.15.3:33824) 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.703+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/1406337586 learned_addr learned my addr 172.21.15.3:0/1406337586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/1406337586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c159890 0x7f3b6c0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/1406337586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c159890 msgr2=0x7f3b6c0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6b7fe700 1 --2- 172.21.15.3:0/1406337586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c159890 0x7f3b6c0b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/1406337586 >> v1:172.21.15.153:6789/0 conn(0x7f3b6c0b28c0 legacy=0x7f3b6c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6b7fe700 1 -- 172.21.15.3:0/1406337586 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c0b48b0 con 0x7f3b6c0b2ec0 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/1406337586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c159890 0x7f3b6c0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6b7fe700 1 --2- 172.21.15.3:0/1406337586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c0b2380 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0099b0 tx=0x7f3b5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e5664101a43185 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6affd700 1 -- 172.21.15.3:0/1406337586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00b820 con 0x7f3b6c0b2ec0 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6affd700 1 -- 172.21.15.3:0/1406337586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c00b980 con 0x7f3b6c0b2ec0 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b6affd700 1 -- 172.21.15.3:0/1406337586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00bd20 con 0x7f3b6c0b2ec0 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b724b7700 1 -- 172.21.15.3:0/1406337586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c0b2ec0 msgr2=0x7f3b6c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b724b7700 1 --2- 172.21.15.3:0/1406337586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c0b2380 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0099b0 tx=0x7f3b5c009250 comp rx=0 tx=0).stop 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b724b7700 1 -- 172.21.15.3:0/1406337586 shutdown_connections 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b724b7700 1 --2- 172.21.15.3:0/1406337586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c159890 0x7f3b6c0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b724b7700 1 --2- 172.21.15.3:0/1406337586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c0b2380 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.704+0000 7f3b724b7700 1 -- 172.21.15.3:0/1406337586 >> 172.21.15.3:0/1406337586 conn(0x7f3b6c099a90 msgr2=0x7f3b6c06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.705+0000 7f3b724b7700 1 -- 172.21.15.3:0/1406337586 shutdown_connections 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.705+0000 7f3b724b7700 1 -- 172.21.15.3:0/1406337586 wait complete. 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 Processor -- start 2023-10-06T04:58:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 -- start start 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c0b28c0 0x7f3b6c0ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c159890 0x7f3b6c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c158de0 con 0x7f3b6c159890 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0abdc0 con 0x7f3b6c0b2ec0 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.706+0000 7f3b724b7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0ac0a0 con 0x7f3b6c0b28c0 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c159890 0x7f3b6c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c159890 0x7f3b6c1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33826/0 (socket says 172.21.15.3:33826) 2023-10-06T04:58:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 -- 172.21.15.3:0/538767822 learned_addr learned my addr 172.21.15.3:0/538767822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b6b7fe700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c0b28c0 0x7f3b6c0ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 -- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c0b28c0 msgr2=0x7f3b6c0ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c0b28c0 0x7f3b6c0ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 -- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c0b2ec0 msgr2=0x7f3b6c158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 -- 172.21.15.3:0/538767822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0094f0 con 0x7f3b6c159890 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b6b7fe700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b70a54700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c159890 0x7f3b6c1588a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c930 tx=0x7f3b640026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b63fff700 1 -- 172.21.15.3:0/538767822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400ce70 con 0x7f3b6c159890 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b63fff700 1 -- 172.21.15.3:0/538767822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6400a7e0 con 0x7f3b6c159890 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6c0ac380 con 0x7f3b6c159890 2023-10-06T04:58:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b63fff700 1 -- 172.21.15.3:0/538767822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400aba0 con 0x7f3b6c159890 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.707+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6c0ac940 con 0x7f3b6c159890 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.708+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6c063f80 con 0x7f3b6c159890 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.709+0000 7f3b63fff700 1 -- 172.21.15.3:0/538767822 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b64012420 con 0x7f3b6c159890 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.709+0000 7f3b63fff700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b4c066f30 0x7f3b4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.709+0000 7f3b63fff700 1 -- 172.21.15.3:0/538767822 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b64084a60 con 0x7f3b6c159890 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.709+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b4c066f30 0x7f3b4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.710+0000 7f3b6bfff700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b4c066f30 0x7f3b4c0693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b54008990 tx=0x7f3b54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.713+0000 7f3b63fff700 1 -- 172.21.15.3:0/538767822 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b640556b0 con 0x7f3b6c159890 2023-10-06T04:58:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.953+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3b6c0cca20 con 0x7f3b6c159890 2023-10-06T04:58:53.955 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T04:58:53.954+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:58:53.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.959+0000 7f3b63fff700 1 -- 172.21.15.3:0/538767822 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3b64055240 con 0x7f3b6c159890 2023-10-06T04:58:53.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b4c066f30 msgr2=0x7f3b4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b4c066f30 0x7f3b4c0693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b54008990 tx=0x7f3b54008040 comp rx=0 tx=0).stop 2023-10-06T04:58:53.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c159890 msgr2=0x7f3b6c1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:53.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c159890 0x7f3b6c1588a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c930 tx=0x7f3b640026e0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 shutdown_connections 2023-10-06T04:58:53.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3b4c066f30 0x7f3b4c0693e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b6c159890 0x7f3b6c1588a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3b6c0b28c0 0x7f3b6c0ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 --2- 172.21.15.3:0/538767822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c0b2ec0 0x7f3b6c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 >> 172.21.15.3:0/538767822 conn(0x7f3b6c099a90 msgr2=0x7f3b6c0cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 shutdown_connections 2023-10-06T04:58:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:53.961+0000 7f3b724b7700 1 -- 172.21.15.3:0/538767822 wait complete. 2023-10-06T04:58:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:58:53.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:58:53.995 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:58:54.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.094+0000 7f0254223700 1 Processor -- start 2023-10-06T04:58:54.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.095+0000 7f0254223700 1 -- start start 2023-10-06T04:58:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.095+0000 7f0254223700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c0b2ec0 0x7f024c0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.095+0000 7f0254223700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f024c0b1760 0x7f024c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.095+0000 7f0254223700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c0affe0 con 0x7f024c159890 2023-10-06T04:58:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.095+0000 7f0254223700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024c0b43e0 con 0x7f024c0b1760 2023-10-06T04:58:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.095+0000 7f0254223700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c15b1b0 con 0x7f024c0b2ec0 2023-10-06T04:58:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f024c0b1760 0x7f024c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f024c0b1760 0x7f024c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33832/0 (socket says 172.21.15.3:33832) 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 -- 172.21.15.3:0/1805566604 learned_addr learned my addr 172.21.15.3:0/1805566604 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02517be700 1 --2- 172.21.15.3:0/1805566604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c0b2ec0 0x7f024c0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 -- 172.21.15.3:0/1805566604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c0b2ec0 msgr2=0x7f024c0b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 --2- 172.21.15.3:0/1805566604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c0b2ec0 0x7f024c0b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 -- 172.21.15.3:0/1805566604 >> v1:172.21.15.153:6789/0 conn(0x7f024c159890 legacy=0x7f024c0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 -- 172.21.15.3:0/1805566604 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0b45d0 con 0x7f024c0b1760 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f02527c0700 1 --2- 172.21.15.3:0/1805566604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c159c60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f023c0099b0 tx=0x7f023c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afafbf19b79a0b45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f0250fbd700 1 -- 172.21.15.3:0/1805566604 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c009ea0 con 0x7f024c0b1760 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f0250fbd700 1 -- 172.21.15.3:0/1805566604 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c00b860 con 0x7f024c0b1760 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.096+0000 7f0250fbd700 1 -- 172.21.15.3:0/1805566604 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c00bc00 con 0x7f024c0b1760 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 -- 172.21.15.3:0/1805566604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 msgr2=0x7f024c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 --2- 172.21.15.3:0/1805566604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c159c60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f023c0099b0 tx=0x7f023c009250 comp rx=0 tx=0).stop 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 -- 172.21.15.3:0/1805566604 shutdown_connections 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 --2- 172.21.15.3:0/1805566604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c159c60 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 --2- 172.21.15.3:0/1805566604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c0b2ec0 0x7f024c0b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 -- 172.21.15.3:0/1805566604 >> 172.21.15.3:0/1805566604 conn(0x7f024c099a90 msgr2=0x7f024c06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 -- 172.21.15.3:0/1805566604 shutdown_connections 2023-10-06T04:58:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.097+0000 7f0254223700 1 -- 172.21.15.3:0/1805566604 wait complete. 2023-10-06T04:58:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.098+0000 7f0254223700 1 Processor -- start 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.098+0000 7f0254223700 1 -- start start 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0254223700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0254223700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f024c0b2ec0 0x7f024c09ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0254223700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c159890 0x7f024c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0254223700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c09cd30 con 0x7f024c0b1760 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0254223700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c15d710 con 0x7f024c0b2ec0 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0254223700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c09c490 con 0x7f024c159890 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c09a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33840/0 (socket says 172.21.15.3:33840) 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 -- 172.21.15.3:0/4255319301 learned_addr learned my addr 172.21.15.3:0/4255319301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f02527c0700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c159890 0x7f024c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f02517be700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f024c0b2ec0 0x7f024c09ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 -- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c159890 msgr2=0x7f024c158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c159890 0x7f024c158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 -- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f024c0b2ec0 msgr2=0x7f024c09ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f024c0b2ec0 0x7f024c09ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 -- 172.21.15.3:0/4255319301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c0094f0 con 0x7f024c0b1760 2023-10-06T04:58:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f02517be700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f024c0b2ec0 0x7f024c09ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0251fbf700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c09a3b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f024400cd30 tx=0x7f0244001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.099+0000 7f0249ffb700 1 -- 172.21.15.3:0/4255319301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400ce30 con 0x7f024c0b1760 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.100+0000 7f0249ffb700 1 -- 172.21.15.3:0/4255319301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024400ab70 con 0x7f024c0b1760 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.100+0000 7f0249ffb700 1 -- 172.21.15.3:0/4255319301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244012420 con 0x7f024c0b1760 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.100+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c09c770 con 0x7f024c0b1760 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.100+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c16ce60 con 0x7f024c0b1760 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.101+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c063f80 con 0x7f024c0b1760 2023-10-06T04:58:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.101+0000 7f0249ffb700 1 -- 172.21.15.3:0/4255319301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0244012810 con 0x7f024c0b1760 2023-10-06T04:58:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.101+0000 7f0249ffb700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0238066f30 0x7f02380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.102+0000 7f0249ffb700 1 -- 172.21.15.3:0/4255319301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0244084e00 con 0x7f024c0b1760 2023-10-06T04:58:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.102+0000 7f02517be700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0238066f30 0x7f02380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.102+0000 7f02517be700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0238066f30 0x7f02380693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0240006f00 tx=0x7f0240008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.106+0000 7f0249ffb700 1 -- 172.21.15.3:0/4255319301 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0244051ec0 con 0x7f024c0b1760 2023-10-06T04:58:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.344+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f024c15d4e0 con 0x7f024c0b1760 2023-10-06T04:58:54.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.958+0000 7f0249ffb700 1 -- 172.21.15.3:0/4255319301 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02440554e0 con 0x7f024c0b1760 2023-10-06T04:58:54.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0238066f30 msgr2=0x7f02380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0238066f30 0x7f02380693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0240006f00 tx=0x7f0240008040 comp rx=0 tx=0).stop 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 msgr2=0x7f024c09a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c09a3b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f024400cd30 tx=0x7f0244001b40 comp rx=0 tx=0).stop 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 shutdown_connections 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0238066f30 0x7f02380693e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f024c159890 0x7f024c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f024c0b1760 0x7f024c09a3b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 --2- 172.21.15.3:0/4255319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f024c0b2ec0 0x7f024c09ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 >> 172.21.15.3:0/4255319301 conn(0x7f024c099a90 msgr2=0x7f024c0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:54.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 shutdown_connections 2023-10-06T04:58:54.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:54.961+0000 7f0254223700 1 -- 172.21.15.3:0/4255319301 wait complete. 2023-10-06T04:58:54.977 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:58:55.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.104+0000 7f82c6dc4700 1 Processor -- start 2023-10-06T04:58:55.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.104+0000 7f82c6dc4700 1 -- start start 2023-10-06T04:58:55.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.105+0000 7f82c6dc4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82c0150590 0x7f82c0150960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:55.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.105+0000 7f82c6dc4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f82c006c4d0 0x7f82c006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.105+0000 7f82c6dc4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c0159e50 con 0x7f82c00b2770 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.105+0000 7f82c6dc4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c00b3280 con 0x7f82c0150590 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.105+0000 7f82c6dc4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c006ce90 con 0x7f82c006c4d0 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.105+0000 7f82c4b60700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82c0150590 0x7f82c0150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82c4b60700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82c0150590 0x7f82c0150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33852/0 (socket says 172.21.15.3:33852) 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82c4b60700 1 -- 172.21.15.3:0/539410619 learned_addr learned my addr 172.21.15.3:0/539410619 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82bffff700 1 --2- 172.21.15.3:0/539410619 >> v2:172.21.15.153:3300/0 conn(0x7f82c006c4d0 0x7f82c006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82c4b60700 1 -- 172.21.15.3:0/539410619 >> v2:172.21.15.153:3300/0 conn(0x7f82c006c4d0 msgr2=0x7f82c006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82c4b60700 1 --2- 172.21.15.3:0/539410619 >> v2:172.21.15.153:3300/0 conn(0x7f82c006c4d0 0x7f82c006c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82c4b60700 1 -- 172.21.15.3:0/539410619 >> v1:172.21.15.153:6789/0 conn(0x7f82c00b2770 legacy=0x7f82c00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82c4b60700 1 -- 172.21.15.3:0/539410619 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c015acb0 con 0x7f82c0150590 2023-10-06T04:58:55.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82c4b60700 1 --2- 172.21.15.3:0/539410619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c0150590 0x7f82c0150960 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f82c00b0180 tx=0x7f82b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31a69a429b429e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82bf7fe700 1 -- 172.21.15.3:0/539410619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000b820 con 0x7f82c0150590 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82bf7fe700 1 -- 172.21.15.3:0/539410619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b000b980 con 0x7f82c0150590 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.106+0000 7f82bf7fe700 1 -- 172.21.15.3:0/539410619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000bd20 con 0x7f82c0150590 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 -- 172.21.15.3:0/539410619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c0150590 msgr2=0x7f82c0150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/539410619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c0150590 0x7f82c0150960 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f82c00b0180 tx=0x7f82b0009250 comp rx=0 tx=0).stop 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 -- 172.21.15.3:0/539410619 shutdown_connections 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/539410619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c0150590 0x7f82c0150960 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/539410619 >> v2:172.21.15.153:3300/0 conn(0x7f82c006c4d0 0x7f82c006c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 -- 172.21.15.3:0/539410619 >> 172.21.15.3:0/539410619 conn(0x7f82c0067620 msgr2=0x7f82c0067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 -- 172.21.15.3:0/539410619 shutdown_connections 2023-10-06T04:58:55.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.107+0000 7f82c6dc4700 1 -- 172.21.15.3:0/539410619 wait complete. 2023-10-06T04:58:55.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.108+0000 7f82c6dc4700 1 Processor -- start 2023-10-06T04:58:55.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.108+0000 7f82c6dc4700 1 -- start start 2023-10-06T04:58:55.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c6dc4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c006c4d0 0x7f82c015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:55.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c6dc4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82c00b2770 0x7f82c0156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:55.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c6dc4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82c0150590 0x7f82c00abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:55.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c6dc4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0169f00 con 0x7f82c006c4d0 2023-10-06T04:58:55.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c6dc4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c016b160 con 0x7f82c0150590 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c6dc4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c016acd0 con 0x7f82c00b2770 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c006c4d0 0x7f82c015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c006c4d0 0x7f82c015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33862/0 (socket says 172.21.15.3:33862) 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 -- 172.21.15.3:0/3837318566 learned_addr learned my addr 172.21.15.3:0/3837318566 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c5361700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82c0150590 0x7f82c00abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82bffff700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82c00b2770 0x7f82c0156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 -- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82c00b2770 msgr2=0x7f82c0156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82c00b2770 0x7f82c0156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 -- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82c0150590 msgr2=0x7f82c00abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82c0150590 0x7f82c00abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 -- 172.21.15.3:0/3837318566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00094f0 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82bffff700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82c00b2770 0x7f82c0156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.109+0000 7f82c4b60700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c006c4d0 0x7f82c015ba80 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009290 tx=0x7f82b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.110+0000 7f82bcff9700 1 -- 172.21.15.3:0/3837318566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000bd00 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.110+0000 7f82bcff9700 1 -- 172.21.15.3:0/3837318566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b0024440 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.110+0000 7f82bcff9700 1 -- 172.21.15.3:0/3837318566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0024800 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.110+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c016a3a0 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.110+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c016a960 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.111+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c0063f80 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.111+0000 7f82bcff9700 1 -- 172.21.15.3:0/3837318566 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82b001a420 con 0x7f82c006c4d0 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.111+0000 7f82bcff9700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f82ac066f30 0x7f82ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:55.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.111+0000 7f82bcff9700 1 -- 172.21.15.3:0/3837318566 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f82b0086590 con 0x7f82c006c4d0 2023-10-06T04:58:55.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.112+0000 7f82bffff700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f82ac066f30 0x7f82ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:55.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.112+0000 7f82bffff700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f82ac066f30 0x7f82ac0693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f82c015eae0 tx=0x7f82b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:55.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.116+0000 7f82bcff9700 1 -- 172.21.15.3:0/3837318566 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82b0053650 con 0x7f82c006c4d0 2023-10-06T04:58:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.307+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f82c015a0a0 con 0x7f82c006c4d0 2023-10-06T04:58:55.373 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:55.376 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.968+0000 7f82bcff9700 1 -- 172.21.15.3:0/3837318566 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f82b0056c70 con 0x7f82c006c4d0 2023-10-06T04:58:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.971+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f82ac066f30 msgr2=0x7f82ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.971+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f82ac066f30 0x7f82ac0693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f82c015eae0 tx=0x7f82b4008040 comp rx=0 tx=0).stop 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.971+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c006c4d0 msgr2=0x7f82c015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.971+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c006c4d0 0x7f82c015ba80 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009290 tx=0x7f82b0017000 comp rx=0 tx=0).stop 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 shutdown_connections 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f82ac066f30 0x7f82ac0693e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82c0150590 0x7f82c00abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82c006c4d0 0x7f82c015ba80 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 --2- 172.21.15.3:0/3837318566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82c00b2770 0x7f82c0156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 >> 172.21.15.3:0/3837318566 conn(0x7f82c0067620 msgr2=0x7f82c014fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 shutdown_connections 2023-10-06T04:58:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:55.972+0000 7f82c6dc4700 1 -- 172.21.15.3:0/3837318566 wait complete. 2023-10-06T04:58:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:58:55.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:58:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.116+0000 7ff3949d8700 1 Processor -- start 2023-10-06T04:58:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.116+0000 7ff3949d8700 1 -- start start 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff3949d8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff3900af7f0 0x7ff3900cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff3949d8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff3949d8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3900ced60 con 0x7ff39015d110 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff3949d8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3900cf980 con 0x7ff3900af7f0 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff3949d8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900c55a0 con 0x7ff3900cefc0 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff38dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff3900af7f0 0x7ff3900cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff38ed9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff38dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff3900af7f0 0x7ff3900cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:46026/0 (socket says 172.21.15.3:46026) 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff38e59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff39015d110 0x7ff3900ad0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53194/0 (socket says 172.21.15.3:53194) 2023-10-06T04:58:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.117+0000 7ff38dd9b700 1 -- 172.21.15.3:0/4061235574 learned_addr learned my addr 172.21.15.3:0/4061235574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38dd9b700 1 -- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 msgr2=0x7ff3900cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38dd9b700 1 --2- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38dd9b700 1 -- 172.21.15.3:0/4061235574 >> v1:172.21.15.153:6789/0 conn(0x7ff39015d110 legacy=0x7ff3900ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38dd9b700 1 -- 172.21.15.3:0/4061235574 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900c5880 con 0x7ff3900af7f0 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38ed9d700 1 --2- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38dd9b700 1 --2- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900cc690 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff378009990 tx=0x7ff378009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d82670022671582 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38d59a700 1 -- 172.21.15.3:0/4061235574 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378009e80 con 0x7ff3900af7f0 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38d59a700 1 -- 172.21.15.3:0/4061235574 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff378005c10 con 0x7ff3900af7f0 2023-10-06T04:58:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.118+0000 7ff38d59a700 1 -- 172.21.15.3:0/4061235574 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37800b650 con 0x7ff3900af7f0 2023-10-06T04:58:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 -- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 msgr2=0x7ff3900cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 --2- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900cc690 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff378009990 tx=0x7ff378009220 comp rx=0 tx=0).stop 2023-10-06T04:58:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 -- 172.21.15.3:0/4061235574 shutdown_connections 2023-10-06T04:58:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 --2- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 --2- 172.21.15.3:0/4061235574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900cc690 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 -- 172.21.15.3:0/4061235574 >> 172.21.15.3:0/4061235574 conn(0x7ff39006ccb0 msgr2=0x7ff3900d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 -- 172.21.15.3:0/4061235574 shutdown_connections 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.119+0000 7ff3949d8700 1 -- 172.21.15.3:0/4061235574 wait complete. 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.120+0000 7ff3949d8700 1 Processor -- start 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.120+0000 7ff3949d8700 1 -- start start 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff39015d110 0x7ff3900c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39016ae40 con 0x7ff39015d110 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390169ff0 con 0x7ff3900af7f0 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39016a230 con 0x7ff3900cefc0 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff39015d110 0x7ff3900c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff39015d110 0x7ff3900c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33872/0 (socket says 172.21.15.3:33872) 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 -- 172.21.15.3:0/146288480 learned_addr learned my addr 172.21.15.3:0/146288480 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38e59c700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38dd9b700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 -- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 msgr2=0x7ff3900c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 -- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 msgr2=0x7ff3900c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 -- 172.21.15.3:0/146288480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff388009420 con 0x7ff39015d110 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38e59c700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff38ed9d700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff39015d110 0x7ff3900c9300 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff38800ec80 tx=0x7ff3880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3867fc700 1 -- 172.21.15.3:0/146288480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800c9d0 con 0x7ff39015d110 2023-10-06T04:58:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3867fc700 1 -- 172.21.15.3:0/146288480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38800cb30 con 0x7ff39015d110 2023-10-06T04:58:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff378009480 con 0x7ff39015d110 2023-10-06T04:58:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3867fc700 1 -- 172.21.15.3:0/146288480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388013470 con 0x7ff39015d110 2023-10-06T04:58:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.121+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39016a7f0 con 0x7ff39015d110 2023-10-06T04:58:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.122+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3900cb140 con 0x7ff39015d110 2023-10-06T04:58:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.123+0000 7ff3867fc700 1 -- 172.21.15.3:0/146288480 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff388013860 con 0x7ff39015d110 2023-10-06T04:58:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.123+0000 7ff3867fc700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff370066f30 0x7ff3700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.123+0000 7ff3867fc700 1 -- 172.21.15.3:0/146288480 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5706+0+0 (secure 0 0 0) 0x7ff3880846d0 con 0x7ff39015d110 2023-10-06T04:58:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.123+0000 7ff38e59c700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff370066f30 0x7ff3700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.124+0000 7ff38e59c700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff370066f30 0x7ff3700693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff38000d220 tx=0x7ff380009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.127+0000 7ff3867fc700 1 -- 172.21.15.3:0/146288480 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff388051940 con 0x7ff39015d110 2023-10-06T04:58:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.327+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff39016aff0 con 0x7ff39015d110 2023-10-06T04:58:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.971+0000 7ff3867fc700 1 -- 172.21.15.3:0/146288480 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff388054f60 con 0x7ff39015d110 2023-10-06T04:58:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.973+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff370066f30 msgr2=0x7ff3700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff370066f30 0x7ff3700693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff38000d220 tx=0x7ff380009040 comp rx=0 tx=0).stop 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff39015d110 msgr2=0x7ff3900c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff39015d110 0x7ff3900c9300 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff38800ec80 tx=0x7ff3880091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 shutdown_connections 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff370066f30 0x7ff3700693e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3900af7f0 0x7ff3900c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff39015d110 0x7ff3900c9300 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 --2- 172.21.15.3:0/146288480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3900cefc0 0x7ff3900c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 >> 172.21.15.3:0/146288480 conn(0x7ff39006ccb0 msgr2=0x7ff3900cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 shutdown_connections 2023-10-06T04:58:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:56.974+0000 7ff3949d8700 1 -- 172.21.15.3:0/146288480 wait complete. 2023-10-06T04:58:56.977 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T04:58:56.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:58:56.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:58:57.043 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-06T04:58:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-06T04:58:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:58:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-06T04:58:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-06T04:58:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:58:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:58:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-06T04:58:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-06T04:58:57.054 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-06T04:58:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T04:58:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T04:58:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T04:58:57.056 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-10-06T04:58:57.056 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-10-06T04:58:57.056 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-10-06T04:58:57.057 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:58:57.058 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-10-06T04:58:57.059 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-10-06T04:58:57.060 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:58:57.061 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-10-06T04:58:57.062 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-10-06T04:58:57.063 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-10-06T04:58:57.064 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.065 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:58:57.066 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:58:57.067 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-10-06T04:58:57.068 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-10-06T04:58:57.069 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-10-06T04:58:57.069 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T04:58:57.069 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T04:58:57.069 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T04:58:57.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:58:57.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.191+0000 7f753094f700 1 Processor -- start 2023-10-06T04:58:57.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.191+0000 7f753094f700 1 -- start start 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f753094f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 0x7f752c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f753094f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c0691c0 con 0x7f752c0b1b00 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f753094f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c0694a0 con 0x7f752c15cf10 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f753094f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c069780 con 0x7f752c067dc0 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f7529d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f752c0b1b00 0x7f752c15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42848/0 (socket says 172.21.15.3:42848) 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f7529d9b700 1 -- 172.21.15.3:0/1359987114 learned_addr learned my addr 172.21.15.3:0/1359987114 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752a59c700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 0x7f752c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930634661 0 0) 0x7f752c0691c0 con 0x7f752c0b1b00 2023-10-06T04:58:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7514003570 con 0x7f752c0b1b00 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681409460 0 0) 0x7f7514003570 con 0x7f752c0b1b00 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 msgr2=0x7f752c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 0x7f752c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 >> v1:172.21.15.153:6789/0 conn(0x7f752c15cf10 legacy=0x7f752c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c069a60 con 0x7f752c0b1b00 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f7520001470 con 0x7f752c0b1b00 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 >> v1:172.21.15.3:6789/0 conn(0x7f752c0b1b00 legacy=0x7f752c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 0x7f7514003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514003fd0 0x7f7514004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7514004d50 0x7f75140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c069780 con 0x7f7514004d50 2023-10-06T04:58:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751400bb40 con 0x7f7514003fd0 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.192+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751400bd00 con 0x7f752c067dc0 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752ad9d700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7514004d50 0x7f75140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752a59c700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 0x7f7514003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f7529d9b700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514003fd0 0x7f7514004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752ad9d700 1 -- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 msgr2=0x7f7514003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752ad9d700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c067dc0 0x7f7514003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752ad9d700 1 -- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514003fd0 msgr2=0x7f7514004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752ad9d700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514003fd0 0x7f7514004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752ad9d700 1 -- 172.21.15.3:0/1359987114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514003570 con 0x7f7514004d50 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752ad9d700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7514004d50 0x7f75140051c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7524002070 tx=0x7f75240016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e516f12076091205 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75240019e0 con 0x7f7514004d50 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400c8b0 con 0x7f7514004d50 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.193+0000 7f752959a700 1 -- 172.21.15.3:0/1359987114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400cc50 con 0x7f7514004d50 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.194+0000 7f753094f700 1 -- 172.21.15.3:0/1359987114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7514004d50 msgr2=0x7f75140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.194+0000 7f753094f700 1 --2- 172.21.15.3:0/1359987114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7514004d50 0x7f75140051c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7524002070 tx=0x7f75240016d0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.194+0000 7f752ad9d700 1 -- 172.21.15.3:0/1359987114 reap_dead start 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.194+0000 7f753094f700 1 -- 172.21.15.3:0/1359987114 shutdown_connections 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.194+0000 7f753094f700 1 -- 172.21.15.3:0/1359987114 >> 172.21.15.3:0/1359987114 conn(0x7f752c0ce270 msgr2=0x7f752c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.194+0000 7f753094f700 1 -- 172.21.15.3:0/1359987114 shutdown_connections 2023-10-06T04:58:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.194+0000 7f753094f700 1 -- 172.21.15.3:0/1359987114 wait complete. 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.195+0000 7f753094f700 1 Processor -- start 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.195+0000 7f753094f700 1 -- start start 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514004d50 0x7f752c13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c15a100 0x7f752c1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c155140 0x7f752c1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c155b20 con 0x7f752c15a100 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0c7420 con 0x7f7514004d50 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c15aa90 con 0x7f752c155140 2023-10-06T04:58:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c15a100 0x7f752c1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c15a100 0x7f752c1596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33898/0 (socket says 172.21.15.3:33898) 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 -- 172.21.15.3:0/2026908337 learned_addr learned my addr 172.21.15.3:0/2026908337 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f752ad9d700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c155140 0x7f752c1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f752a59c700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514004d50 0x7f752c13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 -- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c155140 msgr2=0x7f752c1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c155140 0x7f752c1555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 -- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514004d50 msgr2=0x7f752c13bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514004d50 0x7f752c13bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 -- 172.21.15.3:0/2026908337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c169e70 con 0x7f752c15a100 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f7529d9b700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c15a100 0x7f752c1596c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f75200073f0 tx=0x7f7520001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f751a7fc700 1 -- 172.21.15.3:0/2026908337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520018030 con 0x7f752c15a100 2023-10-06T04:58:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f751a7fc700 1 -- 172.21.15.3:0/2026908337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752000f5f0 con 0x7f752c15a100 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f751a7fc700 1 -- 172.21.15.3:0/2026908337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752000f9b0 con 0x7f752c15a100 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c15a600 con 0x7f752c15a100 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.196+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c0c7950 con 0x7f752c15a100 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.197+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c04c380 con 0x7f752c15a100 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.198+0000 7f751a7fc700 1 -- 172.21.15.3:0/2026908337 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f752001b070 con 0x7f752c15a100 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.198+0000 7f751a7fc700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.198+0000 7f751a7fc700 1 -- 172.21.15.3:0/2026908337 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7520019070 con 0x7f752c15a100 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.198+0000 7f752a59c700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.199+0000 7f752a59c700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f750c066f30 0x7f750c0693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f752c0afbc0 tx=0x7f751c009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.203+0000 7f751a7fc700 1 -- 172.21.15.3:0/2026908337 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7520051580 con 0x7f752c15a100 2023-10-06T04:58:57.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.402+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f752c15abd0 con 0x7f752c15a100 2023-10-06T04:58:57.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.406+0000 7f751a7fc700 1 -- 172.21.15.3:0/2026908337 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7520054ba0 con 0x7f752c15a100 2023-10-06T04:58:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f750c066f30 msgr2=0x7f750c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f750c066f30 0x7f750c0693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f752c0afbc0 tx=0x7f751c009120 comp rx=0 tx=0).stop 2023-10-06T04:58:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c15a100 msgr2=0x7f752c1596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c15a100 0x7f752c1596c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f75200073f0 tx=0x7f7520001e70 comp rx=0 tx=0).stop 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 shutdown_connections 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7514004d50 0x7f752c13bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f752c155140 0x7f752c1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 --2- 172.21.15.3:0/2026908337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c15a100 0x7f752c1596c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 >> 172.21.15.3:0/2026908337 conn(0x7f752c0ce270 msgr2=0x7f752c15d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 shutdown_connections 2023-10-06T04:58:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.408+0000 7f753094f700 1 -- 172.21.15.3:0/2026908337 wait complete. 2023-10-06T04:58:57.422 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2023-10-06T04:58:57.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.539+0000 7f0a15ee7700 1 Processor -- start 2023-10-06T04:58:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.539+0000 7f0a15ee7700 1 -- start start 2023-10-06T04:58:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.539+0000 7f0a15ee7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a10157030 0x7f0a10159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.539+0000 7f0a15ee7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0a1015a220 0x7f0a100b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a15ee7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a10157400 0x7f0a100b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a15ee7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a100b40b0 con 0x7f0a10157400 2023-10-06T04:58:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a15ee7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1015aeb0 con 0x7f0a1015a220 2023-10-06T04:58:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a15ee7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1015a7a0 con 0x7f0a10157030 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a10157400 0x7f0a100b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a10157400 0x7f0a100b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33904/0 (socket says 172.21.15.3:33904) 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 -- 172.21.15.3:0/2155109344 learned_addr learned my addr 172.21.15.3:0/2155109344 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0f7fe700 1 --2- 172.21.15.3:0/2155109344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a10157030 0x7f0a10159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0effd700 1 --2- 172.21.15.3:0/2155109344 >> v2:172.21.15.153:3300/0 conn(0x7f0a1015a220 0x7f0a100b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 -- 172.21.15.3:0/2155109344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a10157030 msgr2=0x7f0a10159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 --2- 172.21.15.3:0/2155109344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a10157030 0x7f0a10159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 -- 172.21.15.3:0/2155109344 >> v2:172.21.15.153:3300/0 conn(0x7f0a1015a220 msgr2=0x7f0a100b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 --2- 172.21.15.3:0/2155109344 >> v2:172.21.15.153:3300/0 conn(0x7f0a1015a220 0x7f0a100b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 -- 172.21.15.3:0/2155109344 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1015aa80 con 0x7f0a10157400 2023-10-06T04:58:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0effd700 1 --2- 172.21.15.3:0/2155109344 >> v2:172.21.15.153:3300/0 conn(0x7f0a1015a220 0x7f0a100b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0f7fe700 1 --2- 172.21.15.3:0/2155109344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a10157030 0x7f0a10159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0ffff700 1 --2- 172.21.15.3:0/2155109344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a100b3b70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0a1006b690 tx=0x7f0a08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663a7ef27be07ab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0e7fc700 1 -- 172.21.15.3:0/2155109344 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800e070 con 0x7f0a10157400 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0e7fc700 1 -- 172.21.15.3:0/2155109344 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0800ba50 con 0x7f0a10157400 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.540+0000 7f0a0e7fc700 1 -- 172.21.15.3:0/2155109344 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080092e0 con 0x7f0a10157400 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 -- 172.21.15.3:0/2155109344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 msgr2=0x7f0a100b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/2155109344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a100b3b70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0a1006b690 tx=0x7f0a08009840 comp rx=0 tx=0).stop 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 -- 172.21.15.3:0/2155109344 shutdown_connections 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/2155109344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a100b3b70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/2155109344 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a10157030 0x7f0a10159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/2155109344 >> v2:172.21.15.153:3300/0 conn(0x7f0a1015a220 0x7f0a100b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 -- 172.21.15.3:0/2155109344 >> 172.21.15.3:0/2155109344 conn(0x7f0a100ce270 msgr2=0x7f0a100ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 -- 172.21.15.3:0/2155109344 shutdown_connections 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.541+0000 7f0a15ee7700 1 -- 172.21.15.3:0/2155109344 wait complete. 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.542+0000 7f0a15ee7700 1 Processor -- start 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.542+0000 7f0a15ee7700 1 -- start start 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a15ee7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a10157030 0x7f0a1006ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a15ee7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a10145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a15ee7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a1015a220 0x7f0a10145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a15ee7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100cd710 con 0x7f0a10157400 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a15ee7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1016cbe0 con 0x7f0a10157030 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a15ee7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1016d760 con 0x7f0a1015a220 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a10145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a10145990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33914/0 (socket says 172.21.15.3:33914) 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 -- 172.21.15.3:0/1562157032 learned_addr learned my addr 172.21.15.3:0/1562157032 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0f7fe700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a10157030 0x7f0a1006ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0ffff700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a1015a220 0x7f0a10145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 -- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a1015a220 msgr2=0x7f0a10145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a1015a220 0x7f0a10145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 -- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a10157030 msgr2=0x7f0a1006ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a10157030 0x7f0a1006ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 -- 172.21.15.3:0/1562157032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08009ae0 con 0x7f0a10157400 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0f7fe700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a10157030 0x7f0a1006ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.543+0000 7f0a0effd700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a10145990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000d860 tx=0x7f0a00008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.544+0000 7f09f77fe700 1 -- 172.21.15.3:0/1562157032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00009610 con 0x7f0a10157400 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.544+0000 7f09f77fe700 1 -- 172.21.15.3:0/1562157032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00009770 con 0x7f0a10157400 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.544+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1016ce30 con 0x7f0a10157400 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.544+0000 7f09f77fe700 1 -- 172.21.15.3:0/1562157032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00009b10 con 0x7f0a10157400 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.544+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1016d3f0 con 0x7f0a10157400 2023-10-06T04:58:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.545+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1016d910 con 0x7f0a10157400 2023-10-06T04:58:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.545+0000 7f09f77fe700 1 -- 172.21.15.3:0/1562157032 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a00014430 con 0x7f0a10157400 2023-10-06T04:58:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.545+0000 7f09f77fe700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f09fc066f30 0x7f09fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.546+0000 7f09f77fe700 1 -- 172.21.15.3:0/1562157032 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0a00084760 con 0x7f0a10157400 2023-10-06T04:58:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.546+0000 7f0a0f7fe700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f09fc066f30 0x7f09fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.546+0000 7f0a0f7fe700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f09fc066f30 0x7f09fc0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006f00 tx=0x7f0a04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.550+0000 7f09f77fe700 1 -- 172.21.15.3:0/1562157032 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a1016d910 con 0x7f0a10157400 2023-10-06T04:58:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.826+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) v1 -- 0x7f0a100644c0 con 0x7f0a10157400 2023-10-06T04:58:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.970+0000 7f09f77fe700 1 -- 172.21.15.3:0/1562157032 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v328) v1 ==== 150+0+0 (secure 0 0 0) 0x7f0a00051b70 con 0x7f0a10157400 2023-10-06T04:58:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.972+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f09fc066f30 msgr2=0x7f09fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.972+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f09fc066f30 0x7f09fc0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006f00 tx=0x7f0a04008040 comp rx=0 tx=0).stop 2023-10-06T04:58:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.972+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 msgr2=0x7f0a10145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.972+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a10145990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000d860 tx=0x7f0a00008040 comp rx=0 tx=0).stop 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 shutdown_connections 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f09fc066f30 0x7f09fc0693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a10157030 0x7f0a1006ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0a1015a220 0x7f0a10145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 --2- 172.21.15.3:0/1562157032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a10157400 0x7f0a10145990 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 >> 172.21.15.3:0/1562157032 conn(0x7f0a100ce270 msgr2=0x7f0a100b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 shutdown_connections 2023-10-06T04:58:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:57.973+0000 7f0a15ee7700 1 -- 172.21.15.3:0/1562157032 wait complete. 2023-10-06T04:58:57.987 INFO:tasks.cephfs_test_runner:When a client does not advance its oldest tid, the MDS should notice that ... ok 2023-10-06T04:58:57.987 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T04:58:57.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2023-10-06T04:58:58.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.105+0000 7ffb6e11f700 1 Processor -- start 2023-10-06T04:58:58.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.105+0000 7ffb6e11f700 1 -- start start 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb6e11f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb68159500 0x7ffb680cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb6e11f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680cc5c0 con 0x7ffb680b2930 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb6e11f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680cb9b0 con 0x7ffb681598d0 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb6e11f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb680cbc90 con 0x7ffb68159500 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb677fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb68159500 0x7ffb680cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb67fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffb680b2930 0x7ffb680cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42850/0 (socket says 172.21.15.3:42850) 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb67fff700 1 -- 172.21.15.3:0/1995437569 learned_addr learned my addr 172.21.15.3:0/1995437569 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb677fe700 1 -- 172.21.15.3:0/1995437569 >> v1:172.21.15.153:6789/0 conn(0x7ffb681598d0 legacy=0x7ffb680b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:58.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb677fe700 1 -- 172.21.15.3:0/1995437569 >> v1:172.21.15.3:6789/0 conn(0x7ffb680b2930 legacy=0x7ffb680cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb677fe700 1 -- 172.21.15.3:0/1995437569 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb680cbf70 con 0x7ffb68159500 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.106+0000 7ffb667fc700 1 -- 172.21.15.3:0/1995437569 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269689205 0 0) 0x7ffb680cc5c0 con 0x7ffb680b2930 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb677fe700 1 --2- 172.21.15.3:0/1995437569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680cad30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098e0 tx=0x7ffb580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a66cf7cbe18145 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb667fc700 1 -- 172.21.15.3:0/1995437569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58009e80 con 0x7ffb68159500 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb667fc700 1 -- 172.21.15.3:0/1995437569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5800b860 con 0x7ffb68159500 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb667fc700 1 -- 172.21.15.3:0/1995437569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800bbe0 con 0x7ffb68159500 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb6e11f700 1 -- 172.21.15.3:0/1995437569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 msgr2=0x7ffb680cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/1995437569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680cad30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098e0 tx=0x7ffb580091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb6e11f700 1 -- 172.21.15.3:0/1995437569 shutdown_connections 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/1995437569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680cad30 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb6e11f700 1 -- 172.21.15.3:0/1995437569 >> 172.21.15.3:0/1995437569 conn(0x7ffb68070310 msgr2=0x7ffb68070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:58.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb6e11f700 1 -- 172.21.15.3:0/1995437569 shutdown_connections 2023-10-06T04:58:58.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.107+0000 7ffb6e11f700 1 -- 172.21.15.3:0/1995437569 wait complete. 2023-10-06T04:58:58.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.108+0000 7ffb6e11f700 1 Processor -- start 2023-10-06T04:58:58.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb6e11f700 1 -- start start 2023-10-06T04:58:58.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb6e11f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb680b2930 0x7ffb6814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb6e11f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb6e11f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb681598d0 0x7ffb680ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb6e11f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680b31e0 con 0x7ffb68159500 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb6e11f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6814fe80 con 0x7ffb680b2930 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb6e11f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680b3fd0 con 0x7ffb681598d0 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb66ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb66ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33920/0 (socket says 172.21.15.3:33920) 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb66ffd700 1 -- 172.21.15.3:0/3625522836 learned_addr learned my addr 172.21.15.3:0/3625522836 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.109+0000 7ffb677fe700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb680b2930 0x7ffb6814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb67fff700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb681598d0 0x7ffb680ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb66ffd700 1 -- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb681598d0 msgr2=0x7ffb680ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb66ffd700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb681598d0 0x7ffb680ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb66ffd700 1 -- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb680b2930 msgr2=0x7ffb6814f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb66ffd700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb680b2930 0x7ffb6814f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb66ffd700 1 -- 172.21.15.3:0/3625522836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009420 con 0x7ffb68159500 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb67fff700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb681598d0 0x7ffb680ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb66ffd700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680ce030 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c5c0 tx=0x7ffb5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb4f7fe700 1 -- 172.21.15.3:0/3625522836 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c011590 con 0x7ffb68159500 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb4f7fe700 1 -- 172.21.15.3:0/3625522836 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c011710 con 0x7ffb68159500 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb680b33c0 con 0x7ffb68159500 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb4f7fe700 1 -- 172.21.15.3:0/3625522836 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c011ad0 con 0x7ffb68159500 2023-10-06T04:58:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.110+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb680b3980 con 0x7ffb68159500 2023-10-06T04:58:58.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.111+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb680b4180 con 0x7ffb68159500 2023-10-06T04:58:58.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.111+0000 7ffb4f7fe700 1 -- 172.21.15.3:0/3625522836 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb5c01b3b0 con 0x7ffb68159500 2023-10-06T04:58:58.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.112+0000 7ffb4f7fe700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb54066f30 0x7ffb540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:58.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.112+0000 7ffb4f7fe700 1 -- 172.21.15.3:0/3625522836 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ffb5c083860 con 0x7ffb68159500 2023-10-06T04:58:58.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.112+0000 7ffb677fe700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb54066f30 0x7ffb540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:58.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.113+0000 7ffb677fe700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb54066f30 0x7ffb540693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb580093f0 tx=0x7ffb58017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:58.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.116+0000 7ffb4f7fe700 1 -- 172.21.15.3:0/3625522836 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb680b4180 con 0x7ffb68159500 2023-10-06T04:58:58.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.389+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) v1 -- 0x7ffb680b4140 con 0x7ffb68159500 2023-10-06T04:58:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.971+0000 7ffb4f7fe700 1 -- 172.21.15.3:0/3625522836 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v329) v1 ==== 146+0+0 (secure 0 0 0) 0x7ffb5c017070 con 0x7ffb68159500 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb54066f30 msgr2=0x7ffb540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb54066f30 0x7ffb540693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb580093f0 tx=0x7ffb58017000 comp rx=0 tx=0).stop 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 msgr2=0x7ffb680ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680ce030 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c5c0 tx=0x7ffb5c00c8d0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 shutdown_connections 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffb54066f30 0x7ffb540693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb680b2930 0x7ffb6814f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb681598d0 0x7ffb680ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 --2- 172.21.15.3:0/3625522836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb68159500 0x7ffb680ce030 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:58.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 >> 172.21.15.3:0/3625522836 conn(0x7ffb68070310 msgr2=0x7ffb680cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:58.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 shutdown_connections 2023-10-06T04:58:58.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:58.974+0000 7ffb6e11f700 1 -- 172.21.15.3:0/3625522836 wait complete. 2023-10-06T04:58:58.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:58:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.111+0000 7fb2713e6700 1 Processor -- start 2023-10-06T04:58:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.111+0000 7fb2713e6700 1 -- start start 2023-10-06T04:58:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb2713e6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 0x7fb26c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb2713e6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c0cb190 con 0x7fb26c0af290 2023-10-06T04:58:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb2713e6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c159950 con 0x7fb26c0cd190 2023-10-06T04:58:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb2713e6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0cd6a0 con 0x7fb26c0af660 2023-10-06T04:58:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb26bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb26c0af290 0x7fb26c0ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42852/0 (socket says 172.21.15.3:42852) 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb26bfff700 1 -- 172.21.15.3:0/651083732 learned_addr learned my addr 172.21.15.3:0/651083732 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356379936 0 0) 0x7fb26c0cb190 con 0x7fb26c0af290 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 0x7fb26c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.112+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb258003570 con 0x7fb26c0af290 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 451925948 0 0) 0x7fb258003570 con 0x7fb26c0af290 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 msgr2=0x7fb26c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 0x7fb26c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 >> v1:172.21.15.153:6789/0 conn(0x7fb26c0cd190 legacy=0x7fb26c0c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0cb3f0 con 0x7fb26c0af290 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fb25c0015f0 con 0x7fb26c0af290 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 >> v1:172.21.15.3:6789/0 conn(0x7fb26c0af290 legacy=0x7fb26c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 0x7fb258003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb258004010 0x7fb258004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb2580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0cd6a0 con 0x7fb258004d50 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25800bbf0 con 0x7fb258004010 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25800bdb0 con 0x7fb26c0af660 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb270be5700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb2580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb258004010 0x7fb258004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26bfff700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 0x7fb258003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb270be5700 1 -- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 msgr2=0x7fb258003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb270be5700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0af660 0x7fb258003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb270be5700 1 -- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb258004010 msgr2=0x7fb258004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb270be5700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb258004010 0x7fb258004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb270be5700 1 -- 172.21.15.3:0/651083732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258003570 con 0x7fb258004d50 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb270be5700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb2580051c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb264001f90 tx=0x7fb264001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7826d47b19f6200b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.113+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264001950 con 0x7fb258004d50 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26400c8b0 con 0x7fb258004d50 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb26affd700 1 -- 172.21.15.3:0/651083732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26400cc70 con 0x7fb258004d50 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb2713e6700 1 -- 172.21.15.3:0/651083732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 msgr2=0x7fb2580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb2713e6700 1 --2- 172.21.15.3:0/651083732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb2580051c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb264001f90 tx=0x7fb264001640 comp rx=0 tx=0).stop 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb270be5700 1 -- 172.21.15.3:0/651083732 reap_dead start 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb2713e6700 1 -- 172.21.15.3:0/651083732 shutdown_connections 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb2713e6700 1 -- 172.21.15.3:0/651083732 >> 172.21.15.3:0/651083732 conn(0x7fb26c089dd0 msgr2=0x7fb26c0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb2713e6700 1 -- 172.21.15.3:0/651083732 shutdown_connections 2023-10-06T04:58:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.114+0000 7fb2713e6700 1 -- 172.21.15.3:0/651083732 wait complete. 2023-10-06T04:58:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.115+0000 7fb2713e6700 1 Processor -- start 2023-10-06T04:58:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.115+0000 7fb2713e6700 1 -- start start 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb2713e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb26c147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb2713e6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb26c0b1f60 0x7fb26c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb2713e6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0afa30 0x7fb26c0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb2713e6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0b2910 con 0x7fb258004d50 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb2713e6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c15d920 con 0x7fb26c0b1f60 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb2713e6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c15dbc0 con 0x7fb26c0afa30 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb26c147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb26c147c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33934/0 (socket says 172.21.15.3:33934) 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb26bfff700 1 -- 172.21.15.3:0/3350445436 learned_addr learned my addr 172.21.15.3:0/3350445436 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb270be5700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0afa30 0x7fb26c0afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.116+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb26c0b1f60 0x7fb26c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb26bfff700 1 -- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0afa30 msgr2=0x7fb26c0afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb26bfff700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0afa30 0x7fb26c0afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb26bfff700 1 -- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb26c0b1f60 msgr2=0x7fb26c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb26bfff700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb26c0b1f60 0x7fb26c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb26bfff700 1 -- 172.21.15.3:0/3350445436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c15dea0 con 0x7fb258004d50 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb26bfff700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb26c147c10 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb25c001360 tx=0x7fb25c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb263fff700 1 -- 172.21.15.3:0/3350445436 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0073b0 con 0x7fb258004d50 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb263fff700 1 -- 172.21.15.3:0/3350445436 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c007510 con 0x7fb258004d50 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c15e180 con 0x7fb258004d50 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb263fff700 1 -- 172.21.15.3:0/3350445436 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c005a00 con 0x7fb258004d50 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.117+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c148150 con 0x7fb258004d50 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.118+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c089bd0 con 0x7fb258004d50 2023-10-06T04:58:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.118+0000 7fb263fff700 1 -- 172.21.15.3:0/3350445436 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb25c00d610 con 0x7fb258004d50 2023-10-06T04:58:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.119+0000 7fb263fff700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb24c066f30 0x7fb24c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.119+0000 7fb263fff700 1 -- 172.21.15.3:0/3350445436 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb25c084610 con 0x7fb258004d50 2023-10-06T04:58:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.119+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb24c066f30 0x7fb24c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.120+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb24c066f30 0x7fb24c0693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb254006e50 tx=0x7fb254009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.123+0000 7fb263fff700 1 -- 172.21.15.3:0/3350445436 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb25c051a20 con 0x7fb258004d50 2023-10-06T04:58:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.319+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb26c04c380 con 0x7fb258004d50 2023-10-06T04:58:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.325+0000 7fb263fff700 1 -- 172.21.15.3:0/3350445436 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb25c005db0 con 0x7fb258004d50 2023-10-06T04:58:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb24c066f30 msgr2=0x7fb24c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb24c066f30 0x7fb24c0693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb254006e50 tx=0x7fb254009120 comp rx=0 tx=0).stop 2023-10-06T04:58:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 msgr2=0x7fb26c147c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb26c147c10 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb25c001360 tx=0x7fb25c009040 comp rx=0 tx=0).stop 2023-10-06T04:58:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 shutdown_connections 2023-10-06T04:58:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb24c066f30 0x7fb24c0693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb26c0afa30 0x7fb26c0afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb258004d50 0x7fb26c147c10 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 --2- 172.21.15.3:0/3350445436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb26c0b1f60 0x7fb26c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 >> 172.21.15.3:0/3350445436 conn(0x7fb26c089dd0 msgr2=0x7fb26c158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 shutdown_connections 2023-10-06T04:58:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.327+0000 7fb2713e6700 1 -- 172.21.15.3:0/3350445436 wait complete. 2023-10-06T04:58:59.341 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:58:59.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:59.381 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:58:59.424 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:58:59.425 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:58:59.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:58:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.543+0000 7f89cd3bc700 1 Processor -- start 2023-10-06T04:58:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.543+0000 7f89cd3bc700 1 -- start start 2023-10-06T04:58:59.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89cd3bc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb3b0 0x7f89c80cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89cd3bc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89c81596d0 0x7f89c80cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89cd3bc700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c80cf130 con 0x7f89c80cb780 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89cd3bc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c80ce520 con 0x7f89c81596d0 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89cd3bc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80ce800 con 0x7f89c80cb3b0 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89ccbbb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89c81596d0 0x7f89c80cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89ccbbb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89c81596d0 0x7f89c80cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33940/0 (socket says 172.21.15.3:33940) 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.544+0000 7f89ccbbb700 1 -- 172.21.15.3:0/2806031761 learned_addr learned my addr 172.21.15.3:0/2806031761 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89c7fff700 1 --2- 172.21.15.3:0/2806031761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb3b0 0x7f89c80cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89ccbbb700 1 -- 172.21.15.3:0/2806031761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb3b0 msgr2=0x7f89c80cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89ccbbb700 1 --2- 172.21.15.3:0/2806031761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb3b0 0x7f89c80cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89ccbbb700 1 -- 172.21.15.3:0/2806031761 >> v1:172.21.15.153:6789/0 conn(0x7f89c80cb780 legacy=0x7f89c8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89ccbbb700 1 -- 172.21.15.3:0/2806031761 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80ceae0 con 0x7f89c81596d0 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89ccbbb700 1 --2- 172.21.15.3:0/2806031761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80cccc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009f20 tx=0x7f89c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a908caf05fe29d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89c6ffd700 1 -- 172.21.15.3:0/2806031761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c000e070 con 0x7f89c81596d0 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89c6ffd700 1 -- 172.21.15.3:0/2806031761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c000ba40 con 0x7f89c81596d0 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.545+0000 7f89c6ffd700 1 -- 172.21.15.3:0/2806031761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c00092e0 con 0x7f89c81596d0 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 -- 172.21.15.3:0/2806031761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 msgr2=0x7f89c80cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/2806031761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80cccc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009f20 tx=0x7f89c0009840 comp rx=0 tx=0).stop 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 -- 172.21.15.3:0/2806031761 shutdown_connections 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/2806031761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80cccc0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/2806031761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb3b0 0x7f89c80cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 -- 172.21.15.3:0/2806031761 >> 172.21.15.3:0/2806031761 conn(0x7f89c8070310 msgr2=0x7f89c8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:59.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 -- 172.21.15.3:0/2806031761 shutdown_connections 2023-10-06T04:58:59.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.546+0000 7f89cd3bc700 1 -- 172.21.15.3:0/2806031761 wait complete. 2023-10-06T04:58:59.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.547+0000 7f89cd3bc700 1 Processor -- start 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.547+0000 7f89cd3bc700 1 -- start start 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89cd3bc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c80cb3b0 0x7f89c8157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89cd3bc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb780 0x7f89c80b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89cd3bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89cd3bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8149c50 con 0x7f89c81596d0 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89cd3bc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8158390 con 0x7f89c80cb3b0 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89cd3bc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80c8ab0 con 0x7f89c80cb780 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33956/0 (socket says 172.21.15.3:33956) 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 -- 172.21.15.3:0/1273641438 learned_addr learned my addr 172.21.15.3:0/1273641438 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:58:59.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89c7fff700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c80cb3b0 0x7f89c8157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89c77fe700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb780 0x7f89c80b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 -- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb780 msgr2=0x7f89c80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb780 0x7f89c80b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 -- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c80cb3b0 msgr2=0x7f89c8157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c80cb3b0 0x7f89c8157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 -- 172.21.15.3:0/1273641438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c0009ae0 con 0x7f89c81596d0 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89ccbbb700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80b1620 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f89c80893d0 tx=0x7f89c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.548+0000 7f89bffff700 1 -- 172.21.15.3:0/1273641438 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0012e60 con 0x7f89c81596d0 2023-10-06T04:58:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.549+0000 7f89bffff700 1 -- 172.21.15.3:0/1273641438 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c001d420 con 0x7f89c81596d0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.549+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c80b3590 con 0x7f89c81596d0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.549+0000 7f89bffff700 1 -- 172.21.15.3:0/1273641438 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c001d7e0 con 0x7f89c81596d0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.549+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c80c8460 con 0x7f89c81596d0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.549+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c804c380 con 0x7f89c81596d0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.550+0000 7f89bffff700 1 -- 172.21.15.3:0/1273641438 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89c001db90 con 0x7f89c81596d0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.550+0000 7f89bffff700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89a8066f30 0x7f89a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.550+0000 7f89bffff700 1 -- 172.21.15.3:0/1273641438 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f89c0085d50 con 0x7f89c81596d0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.551+0000 7f89c7fff700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89a8066f30 0x7f89a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.551+0000 7f89c7fff700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89a8066f30 0x7f89a80693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f89b000ab30 tx=0x7f89b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:59.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.555+0000 7f89bffff700 1 -- 172.21.15.3:0/1273641438 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89c0089030 con 0x7f89c81596d0 2023-10-06T04:58:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.791+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89c80cedc0 con 0x7f89c81596d0 2023-10-06T04:58:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.791+0000 7f89bffff700 1 -- 172.21.15.3:0/1273641438 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f89c002ade0 con 0x7f89c81596d0 2023-10-06T04:58:59.793 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:58:59.793 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2290041871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2290041871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2290041871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1529761607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1529761607},{"type":"v1","addr":"172.21.15.153:6835","nonce":1529761607}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1018218180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1018218180},{"type":"v1","addr":"172.21.15.3:6835","nonce":1018218180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6714,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2224049227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2224049227},{"type":"v1","addr":"172.21.15.3:6834","nonce":2224049227}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[]} 2023-10-06T04:58:59.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.793+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89a8066f30 msgr2=0x7f89a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.793+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89a8066f30 0x7f89a80693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f89b000ab30 tx=0x7f89b0008040 comp rx=0 tx=0).stop 2023-10-06T04:58:59.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.793+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 msgr2=0x7f89c80b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.793+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80b1620 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f89c80893d0 tx=0x7f89c0017000 comp rx=0 tx=0).stop 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 shutdown_connections 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89a8066f30 0x7f89a80693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c80cb3b0 0x7f89c8157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c81596d0 0x7f89c80b1620 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 --2- 172.21.15.3:0/1273641438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89c80cb780 0x7f89c80b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 >> 172.21.15.3:0/1273641438 conn(0x7f89c8070310 msgr2=0x7f89c815d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 shutdown_connections 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.794+0000 7f89cd3bc700 1 -- 172.21.15.3:0/1273641438 wait complete. 2023-10-06T04:58:59.796 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 53 2023-10-06T04:58:59.808 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:58:59.808 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:58:59.808 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:58:59.808 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:59.808 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:58:59.808 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:58:59.809 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:58:59.809 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:59.809 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:58:59.809 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:58:59.809 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:58:59.809 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:59.809 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:58:59.809 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:58:59.810 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:58:59.810 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:58:59.816 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:58:59.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:58:59.817 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:58:59.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:58:59.820 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:58:59.820 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:58:59.821 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:58:59.821 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:58:59.859 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:58:59.860 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:59.860 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:58:59.860 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:59.863 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:58:59.863 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:59.864 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:58:59.864 DEBUG:teuthology.parallel:result is None 2023-10-06T04:58:59.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:58:59.934 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:58:59.935 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:58:59.947 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:58:59.947 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:58:59.948 INFO:tasks.ceph.mds.d.smithi153.stderr:2023-10-06T04:58:59.946+0000 7f89d25e6700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T04:58:59.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.990+0000 7f3562330700 1 Processor -- start 2023-10-06T04:58:59.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.990+0000 7f3562330700 1 -- start start 2023-10-06T04:58:59.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.990+0000 7f3562330700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f355c159440 0x7f355c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.990+0000 7f3562330700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c15e0f0 con 0x7f355c06aaa0 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.990+0000 7f3562330700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c0c93f0 con 0x7f355c159810 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.990+0000 7f3562330700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c15d850 con 0x7f355c159440 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f35608cd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f355c06aaa0 0x7f355c0b1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42862/0 (socket says 172.21.15.3:42862) 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f35608cd700 1 -- 172.21.15.3:0/944291936 learned_addr learned my addr 172.21.15.3:0/944291936 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f355bfff700 1 --2- 172.21.15.3:0/944291936 >> v2:172.21.15.153:3300/0 conn(0x7f355c159440 0x7f355c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f355affd700 1 -- 172.21.15.3:0/944291936 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686892522 0 0) 0x7f355c15e0f0 con 0x7f355c06aaa0 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f355affd700 1 -- 172.21.15.3:0/944291936 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3544003570 con 0x7f355c06aaa0 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f355bfff700 1 -- 172.21.15.3:0/944291936 >> v1:172.21.15.153:6789/0 conn(0x7f355c159810 legacy=0x7f355c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f355bfff700 1 -- 172.21.15.3:0/944291936 >> v1:172.21.15.3:6789/0 conn(0x7f355c06aaa0 legacy=0x7f355c0b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f355bfff700 1 -- 172.21.15.3:0/944291936 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c15db30 con 0x7f355c159440 2023-10-06T04:58:59.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.991+0000 7f355bfff700 1 --2- 172.21.15.3:0/944291936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c159440 0x7f355c15d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f354c0098e0 tx=0x7f354c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4bf1e45ebda358 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:58:59.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f355affd700 1 -- 172.21.15.3:0/944291936 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c00b640 con 0x7f355c159440 2023-10-06T04:58:59.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f355affd700 1 -- 172.21.15.3:0/944291936 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c00b7a0 con 0x7f355c159440 2023-10-06T04:58:59.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f355affd700 1 -- 172.21.15.3:0/944291936 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c00bb40 con 0x7f355c159440 2023-10-06T04:58:59.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f3562330700 1 -- 172.21.15.3:0/944291936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c159440 msgr2=0x7f355c15d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:58:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f3562330700 1 --2- 172.21.15.3:0/944291936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c159440 0x7f355c15d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f354c0098e0 tx=0x7f354c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f3562330700 1 -- 172.21.15.3:0/944291936 shutdown_connections 2023-10-06T04:58:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f3562330700 1 --2- 172.21.15.3:0/944291936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c159440 0x7f355c15d190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:58:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.993+0000 7f3562330700 1 -- 172.21.15.3:0/944291936 >> 172.21.15.3:0/944291936 conn(0x7f355c066fb0 msgr2=0x7f355c0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:58:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.994+0000 7f3562330700 1 -- 172.21.15.3:0/944291936 shutdown_connections 2023-10-06T04:58:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.994+0000 7f3562330700 1 -- 172.21.15.3:0/944291936 wait complete. 2023-10-06T04:58:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.995+0000 7f3562330700 1 Processor -- start 2023-10-06T04:58:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.995+0000 7f3562330700 1 -- start start 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f3562330700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c06aaa0 0x7f355c16d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f3562330700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f355c159440 0x7f355c150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f3562330700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f355c159810 0x7f355c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f3562330700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c1517c0 con 0x7f355c159810 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f3562330700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c151aa0 con 0x7f355c06aaa0 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f3562330700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c151d80 con 0x7f355c159440 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f35608cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f355c159810 0x7f355c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f355bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c06aaa0 0x7f355c16d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:58:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f35608cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f355c159810 0x7f355c15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34004/0 (socket says 172.21.15.3:34004) 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f35608cd700 1 -- 172.21.15.3:0/2420933091 learned_addr learned my addr 172.21.15.3:0/2420933091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f355b7fe700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f355c159440 0x7f355c150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f35608cd700 1 -- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f355c159440 msgr2=0x7f355c150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f35608cd700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f355c159440 0x7f355c150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f35608cd700 1 -- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c06aaa0 msgr2=0x7f355c16d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.996+0000 7f35608cd700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c06aaa0 0x7f355c16d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.997+0000 7f35608cd700 1 -- 172.21.15.3:0/2420933091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c009420 con 0x7f355c159810 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.997+0000 7f35608cd700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f355c159810 0x7f355c15bd30 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3554001460 tx=0x7f3554001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.997+0000 7f3543fff700 1 -- 172.21.15.3:0/2420933091 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400a890 con 0x7f355c159810 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.997+0000 7f3543fff700 1 -- 172.21.15.3:0/2420933091 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355400a9f0 con 0x7f355c159810 2023-10-06T04:59:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.997+0000 7f3543fff700 1 -- 172.21.15.3:0/2420933091 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400ad70 con 0x7f355c159810 2023-10-06T04:59:00.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.997+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c152060 con 0x7f355c159810 2023-10-06T04:59:00.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.997+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c16d920 con 0x7f355c159810 2023-10-06T04:59:00.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.998+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c071680 con 0x7f355c159810 2023-10-06T04:59:00.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.998+0000 7f3543fff700 1 -- 172.21.15.3:0/2420933091 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3554007300 con 0x7f355c159810 2023-10-06T04:59:00.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.999+0000 7f3543fff700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3548066f30 0x7f35480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:00.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.999+0000 7f3543fff700 1 -- 172.21.15.3:0/2420933091 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3554084760 con 0x7f355c159810 2023-10-06T04:59:00.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.999+0000 7f355bfff700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3548066f30 0x7f35480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:00.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:58:59.999+0000 7f355bfff700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3548066f30 0x7f35480693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f354c0093f0 tx=0x7f354c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:00.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.003+0000 7f3543fff700 1 -- 172.21.15.3:0/2420933091 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3554051b60 con 0x7f355c159810 2023-10-06T04:59:00.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.211+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f355c15de10 con 0x7f355c159810 2023-10-06T04:59:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.995+0000 7f3543fff700 1 -- 172.21.15.3:0/2420933091 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3554055180 con 0x7f355c159810 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3548066f30 msgr2=0x7f35480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3548066f30 0x7f35480693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f354c0093f0 tx=0x7f354c017000 comp rx=0 tx=0).stop 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f355c159810 msgr2=0x7f355c15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f355c159810 0x7f355c15bd30 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3554001460 tx=0x7f3554001ae0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 shutdown_connections 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3548066f30 0x7f35480693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f355c06aaa0 0x7f355c16d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f355c159810 0x7f355c15bd30 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 --2- 172.21.15.3:0/2420933091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f355c159440 0x7f355c150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 >> 172.21.15.3:0/2420933091 conn(0x7f355c066fb0 msgr2=0x7f355c0cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 shutdown_connections 2023-10-06T04:59:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:00.998+0000 7f3562330700 1 -- 172.21.15.3:0/2420933091 wait complete. 2023-10-06T04:59:01.001 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T04:59:01.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:59:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.138+0000 7fa743965700 1 Processor -- start 2023-10-06T04:59:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa743965700 1 -- start start 2023-10-06T04:59:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa743965700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 0x7fa73c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa743965700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa73c0d06a0 0x7fa73c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa743965700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73c0cf6c0 con 0x7fa73c15c8f0 2023-10-06T04:59:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa743965700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0cfb50 con 0x7fa73c0d06a0 2023-10-06T04:59:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa743965700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0d0b20 con 0x7fa73c15ccc0 2023-10-06T04:59:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa741701700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa73c15c8f0 0x7fa73c15db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42868/0 (socket says 172.21.15.3:42868) 2023-10-06T04:59:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.139+0000 7fa741701700 1 -- 172.21.15.3:0/1372030925 learned_addr learned my addr 172.21.15.3:0/1372030925 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa741f02700 1 --2- 172.21.15.3:0/1372030925 >> v2:172.21.15.153:3300/0 conn(0x7fa73c0d06a0 0x7fa73c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa740f00700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 0x7fa73c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136134574 0 0) 0x7fa73c0cf6c0 con 0x7fa73c15c8f0 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa724003570 con 0x7fa73c15c8f0 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4214576939 0 0) 0x7fa724003570 con 0x7fa73c15c8f0 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 msgr2=0x7fa73c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 0x7fa73c0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 >> v2:172.21.15.153:3300/0 conn(0x7fa73c0d06a0 msgr2=0x7fa73c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 --2- 172.21.15.3:0/1372030925 >> v2:172.21.15.153:3300/0 conn(0x7fa73c0d06a0 0x7fa73c0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0ceab0 con 0x7fa73c15c8f0 2023-10-06T04:59:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fa72c0017f0 con 0x7fa73c15c8f0 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa741f02700 1 --2- 172.21.15.3:0/1372030925 >> v2:172.21.15.153:3300/0 conn(0x7fa73c0d06a0 0x7fa73c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 >> v1:172.21.15.3:6789/0 conn(0x7fa73c15c8f0 legacy=0x7fa73c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa724003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 0x7fa724004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa724004d50 0x7fa7240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0cfb50 con 0x7fa724003a60 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0d0b20 con 0x7fa724004d50 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72400ba60 con 0x7fa73c15ccc0 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa741701700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa724003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa741f02700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa724004d50 0x7fa7240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.140+0000 7fa740f00700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 0x7fa724004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa741701700 1 -- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 msgr2=0x7fa724004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa741701700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c15ccc0 0x7fa724004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa741701700 1 -- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa724004d50 msgr2=0x7fa7240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa741701700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa724004d50 0x7fa7240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa741701700 1 -- 172.21.15.3:0/1372030925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724003570 con 0x7fa724003a60 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa741701700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa724003e70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00a970 tx=0x7fa73c0d0c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0b52bb529e465 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0070d0 con 0x7fa724003a60 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c007270 con 0x7fa724003a60 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa73bfff700 1 -- 172.21.15.3:0/1372030925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c007630 con 0x7fa724003a60 2023-10-06T04:59:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa743965700 1 -- 172.21.15.3:0/1372030925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 msgr2=0x7fa724003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa743965700 1 --2- 172.21.15.3:0/1372030925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa724003e70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00a970 tx=0x7fa73c0d0c90 comp rx=0 tx=0).stop 2023-10-06T04:59:01.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.141+0000 7fa741f02700 1 -- 172.21.15.3:0/1372030925 reap_dead start 2023-10-06T04:59:01.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.142+0000 7fa743965700 1 -- 172.21.15.3:0/1372030925 shutdown_connections 2023-10-06T04:59:01.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.142+0000 7fa743965700 1 -- 172.21.15.3:0/1372030925 >> 172.21.15.3:0/1372030925 conn(0x7fa73c070310 msgr2=0x7fa73c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:01.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.142+0000 7fa743965700 1 -- 172.21.15.3:0/1372030925 shutdown_connections 2023-10-06T04:59:01.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.142+0000 7fa743965700 1 -- 172.21.15.3:0/1372030925 wait complete. 2023-10-06T04:59:01.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.143+0000 7fa743965700 1 Processor -- start 2023-10-06T04:59:01.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.143+0000 7fa743965700 1 -- start start 2023-10-06T04:59:01.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa73c148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c0b1470 0x7fa73c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa73c159b10 0x7fa73c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0b1e20 con 0x7fa724003a60 2023-10-06T04:59:01.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c15a860 con 0x7fa73c159b10 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0c5220 con 0x7fa73c0b1470 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa73c148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa73c148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34014/0 (socket says 172.21.15.3:34014) 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 -- 172.21.15.3:0/1635111251 learned_addr learned my addr 172.21.15.3:0/1635111251 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741f02700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa73c159b10 0x7fa73c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 -- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c0b1470 msgr2=0x7fa73c0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c0b1470 0x7fa73c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 -- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa73c159b10 msgr2=0x7fa73c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa73c159b10 0x7fa73c159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 -- 172.21.15.3:0/1635111251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0c5500 con 0x7fa724003a60 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741f02700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa73c159b10 0x7fa73c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:01.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa741701700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa73c148e00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa72c012000 tx=0x7fa72c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa7397fa700 1 -- 172.21.15.3:0/1635111251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c001e80 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa7397fa700 1 -- 172.21.15.3:0/1635111251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c01e440 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c0c57e0 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa7397fa700 1 -- 172.21.15.3:0/1635111251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c01e610 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.144+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c15a4f0 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.145+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c0c5e30 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.146+0000 7fa7397fa700 1 -- 172.21.15.3:0/1635111251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa72c025020 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.146+0000 7fa7397fa700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa728066f30 0x7fa7280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.146+0000 7fa7397fa700 1 -- 172.21.15.3:0/1635111251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa72c086800 con 0x7fa724003a60 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.146+0000 7fa740f00700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa728066f30 0x7fa7280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.147+0000 7fa740f00700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa728066f30 0x7fa7280693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa73c16cd40 tx=0x7fa7300091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.150+0000 7fa7397fa700 1 -- 172.21.15.3:0/1635111251 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa72c053c60 con 0x7fa724003a60 2023-10-06T04:59:01.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.421+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa73c071680 con 0x7fa724003a60 2023-10-06T04:59:01.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.422+0000 7fa7397fa700 1 -- 172.21.15.3:0/1635111251 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa72c057280 con 0x7fa724003a60 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:01.424 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.425 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.427 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.430 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T04:59:01.433 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:59:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:59:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:59:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:59:01.439 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:59:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa728066f30 msgr2=0x7fa7280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa728066f30 0x7fa7280693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa73c16cd40 tx=0x7fa7300091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 msgr2=0x7fa73c148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa73c148e00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa72c012000 tx=0x7fa72c016000 comp rx=0 tx=0).stop 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 shutdown_connections 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa728066f30 0x7fa7280693e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa73c159b10 0x7fa73c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa724003a60 0x7fa73c148e00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 --2- 172.21.15.3:0/1635111251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa73c0b1470 0x7fa73c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 >> 172.21.15.3:0/1635111251 conn(0x7fa73c070310 msgr2=0x7fa73c156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 shutdown_connections 2023-10-06T04:59:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.424+0000 7fa743965700 1 -- 172.21.15.3:0/1635111251 wait complete. 2023-10-06T04:59:01.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.560+0000 7fd705536700 1 Processor -- start 2023-10-06T04:59:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.560+0000 7fd705536700 1 -- start start 2023-10-06T04:59:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd705536700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 0x7fd7000b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd705536700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd700159570 0x7fd7000b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd705536700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7000b0a90 con 0x7fd7001591a0 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd705536700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7000b2a60 con 0x7fd700159570 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd705536700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000b3110 con 0x7fd70015dbd0 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd6feffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd7001591a0 0x7fd7000affd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42872/0 (socket says 172.21.15.3:42872) 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd6feffd700 1 -- 172.21.15.3:0/1365638337 learned_addr learned my addr 172.21.15.3:0/1365638337 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125783197 0 0) 0x7fd7000b0a90 con 0x7fd7001591a0 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd6fe7fc700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 0x7fd7000b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ec003570 con 0x7fd7001591a0 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.561+0000 7fd6ff7fe700 1 --2- 172.21.15.3:0/1365638337 >> v2:172.21.15.153:3300/0 conn(0x7fd700159570 0x7fd7000b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 727767194 0 0) 0x7fd6ec003570 con 0x7fd7001591a0 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 msgr2=0x7fd7000b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 0x7fd7000b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 >> v2:172.21.15.153:3300/0 conn(0x7fd700159570 msgr2=0x7fd7000b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 --2- 172.21.15.3:0/1365638337 >> v2:172.21.15.153:3300/0 conn(0x7fd700159570 0x7fd7000b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7000b33f0 con 0x7fd7001591a0 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fd6f8001470 con 0x7fd7001591a0 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 >> v1:172.21.15.3:6789/0 conn(0x7fd7001591a0 legacy=0x7fd7000affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec003a60 0x7fd6ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 0x7fd6ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec004d50 0x7fd6ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000b2a60 con 0x7fd6ec003a60 2023-10-06T04:59:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000b3110 con 0x7fd6ec004d50 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec00ba60 con 0x7fd70015dbd0 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6feffd700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec003a60 0x7fd6ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fe7fc700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 0x7fd6ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6ff7fe700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec004d50 0x7fd6ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6feffd700 1 -- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 msgr2=0x7fd6ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6feffd700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 0x7fd6ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6feffd700 1 -- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec004d50 msgr2=0x7fd6ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6feffd700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec004d50 0x7fd6ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6feffd700 1 -- 172.21.15.3:0/1365638337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec003570 con 0x7fd6ec003a60 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fe7fc700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd70015dbd0 0x7fd6ec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6feffd700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec003a60 0x7fd6ec003e70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c3c0 tx=0x7fd6f8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e57bd88cb7fc68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800cc30 con 0x7fd6ec003a60 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f800cdb0 con 0x7fd6ec003a60 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.562+0000 7fd6fdffb700 1 -- 172.21.15.3:0/1365638337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8005b70 con 0x7fd6ec003a60 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.563+0000 7fd705536700 1 -- 172.21.15.3:0/1365638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec003a60 msgr2=0x7fd6ec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.563+0000 7fd705536700 1 --2- 172.21.15.3:0/1365638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6ec003a60 0x7fd6ec003e70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c3c0 tx=0x7fd6f8001e90 comp rx=0 tx=0).stop 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.563+0000 7fd6ff7fe700 1 -- 172.21.15.3:0/1365638337 reap_dead start 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.563+0000 7fd705536700 1 -- 172.21.15.3:0/1365638337 shutdown_connections 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.563+0000 7fd705536700 1 -- 172.21.15.3:0/1365638337 >> 172.21.15.3:0/1365638337 conn(0x7fd700070310 msgr2=0x7fd700070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.563+0000 7fd705536700 1 -- 172.21.15.3:0/1365638337 shutdown_connections 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.563+0000 7fd705536700 1 -- 172.21.15.3:0/1365638337 wait complete. 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.564+0000 7fd705536700 1 Processor -- start 2023-10-06T04:59:01.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.564+0000 7fd705536700 1 -- start start 2023-10-06T04:59:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd705536700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec003a60 0x7fd7000c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd705536700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd700155060 0x7fd700154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd705536700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd70015f460 0x7fd70015f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd705536700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000c82b0 con 0x7fd700155060 2023-10-06T04:59:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd705536700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70015fe40 con 0x7fd70015f460 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd705536700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70016a200 con 0x7fd6ec003a60 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd700155060 0x7fd700154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd700155060 0x7fd700154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34036/0 (socket says 172.21.15.3:34036) 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 -- 172.21.15.3:0/390122351 learned_addr learned my addr 172.21.15.3:0/390122351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6feffd700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec003a60 0x7fd7000c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6ff7fe700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd70015f460 0x7fd70015f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 -- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec003a60 msgr2=0x7fd7000c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec003a60 0x7fd7000c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 -- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd70015f460 msgr2=0x7fd70015f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd70015f460 0x7fd70015f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 -- 172.21.15.3:0/390122351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700169900 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6ff7fe700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd70015f460 0x7fd70015f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.565+0000 7fd6fe7fc700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd700155060 0x7fd700154780 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80119c0 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.566+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/390122351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8011e50 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.566+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/390122351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800f8e0 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.566+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/390122351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800fca0 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.566+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7000ccf60 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.566+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd700169e90 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.567+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70004c380 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.567+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/390122351 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6e8015460 con 0x7fd700155060 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.567+0000 7fd6f6ffd700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f60 0x7fd6e0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.568+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/390122351 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd6e8084b70 con 0x7fd700155060 2023-10-06T04:59:01.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.568+0000 7fd6feffd700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f60 0x7fd6e0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.568+0000 7fd6feffd700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f60 0x7fd6e0069410 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8001ed0 tx=0x7fd6f8015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.572+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/390122351 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6e8051fd0 con 0x7fd700155060 2023-10-06T04:59:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.766+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd70016a340 con 0x7fd700155060 2023-10-06T04:59:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.766+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/390122351 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13603 (secure 0 0 0) 0x7fd6e80555f0 con 0x7fd700155060 2023-10-06T04:59:01.768 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:01.768 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:00.992293+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f60 msgr2=0x7fd6e0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f60 0x7fd6e0069410 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8001ed0 tx=0x7fd6f8015000 comp rx=0 tx=0).stop 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd700155060 msgr2=0x7fd700154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd700155060 0x7fd700154780 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80119c0 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 shutdown_connections 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f60 0x7fd6e0069410 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd70015f460 0x7fd70015f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec003a60 0x7fd7000c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 --2- 172.21.15.3:0/390122351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd700155060 0x7fd700154780 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 >> 172.21.15.3:0/390122351 conn(0x7fd700070310 msgr2=0x7fd700063570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:01.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 shutdown_connections 2023-10-06T04:59:01.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.769+0000 7fd705536700 1 -- 172.21.15.3:0/390122351 wait complete. 2023-10-06T04:59:01.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:59:01.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:59:01.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.904+0000 7f90fb643700 1 Processor -- start 2023-10-06T04:59:01.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.904+0000 7f90fb643700 1 -- start start 2023-10-06T04:59:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90fb643700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90f4150690 0x7f90f4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90fb643700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f90f40cba50 0x7f90f415d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90fb643700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f415da80 con 0x7f90f415a720 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90fb643700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f415a290 con 0x7f90f4150690 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90fb643700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f40d0020 con 0x7f90f40cba50 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90f93df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90f4150690 0x7f90f4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90f93df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90f4150690 0x7f90f4150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34046/0 (socket says 172.21.15.3:34046) 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90f93df700 1 -- 172.21.15.3:0/2808945699 learned_addr learned my addr 172.21.15.3:0/2808945699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90f8bde700 1 --2- 172.21.15.3:0/2808945699 >> v2:172.21.15.153:3300/0 conn(0x7f90f40cba50 0x7f90f415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.905+0000 7f90f93df700 1 -- 172.21.15.3:0/2808945699 >> v2:172.21.15.153:3300/0 conn(0x7f90f40cba50 msgr2=0x7f90f415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f93df700 1 --2- 172.21.15.3:0/2808945699 >> v2:172.21.15.153:3300/0 conn(0x7f90f40cba50 0x7f90f415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f93df700 1 -- 172.21.15.3:0/2808945699 >> v1:172.21.15.153:6789/0 conn(0x7f90f415a720 legacy=0x7f90f40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f93df700 1 -- 172.21.15.3:0/2808945699 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4159f40 con 0x7f90f4150690 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f8bde700 1 --2- 172.21.15.3:0/2808945699 >> v2:172.21.15.153:3300/0 conn(0x7f90f40cba50 0x7f90f415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f93df700 1 --2- 172.21.15.3:0/2808945699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f4150690 0x7f90f4150a60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ce1e0 tx=0x7f90ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378b145e39a0fb17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f3fff700 1 -- 172.21.15.3:0/2808945699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00b820 con 0x7f90f4150690 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f3fff700 1 -- 172.21.15.3:0/2808945699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec00b980 con 0x7f90f4150690 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90f3fff700 1 -- 172.21.15.3:0/2808945699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00bd20 con 0x7f90f4150690 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90fb643700 1 -- 172.21.15.3:0/2808945699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f4150690 msgr2=0x7f90f4150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90fb643700 1 --2- 172.21.15.3:0/2808945699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f4150690 0x7f90f4150a60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ce1e0 tx=0x7f90ec009250 comp rx=0 tx=0).stop 2023-10-06T04:59:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90fb643700 1 -- 172.21.15.3:0/2808945699 shutdown_connections 2023-10-06T04:59:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90fb643700 1 --2- 172.21.15.3:0/2808945699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f4150690 0x7f90f4150a60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90fb643700 1 --2- 172.21.15.3:0/2808945699 >> v2:172.21.15.153:3300/0 conn(0x7f90f40cba50 0x7f90f415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.906+0000 7f90fb643700 1 -- 172.21.15.3:0/2808945699 >> 172.21.15.3:0/2808945699 conn(0x7f90f4070310 msgr2=0x7f90f4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.907+0000 7f90fb643700 1 -- 172.21.15.3:0/2808945699 shutdown_connections 2023-10-06T04:59:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.907+0000 7f90fb643700 1 -- 172.21.15.3:0/2808945699 wait complete. 2023-10-06T04:59:01.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.908+0000 7f90fb643700 1 Processor -- start 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.908+0000 7f90fb643700 1 -- start start 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40cba50 0x7f90f414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90f4150690 0x7f90f40c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f415a720 0x7f90f4151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40c7b40 con 0x7f90f415a720 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41535c0 con 0x7f90f40cba50 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41538a0 con 0x7f90f4150690 2023-10-06T04:59:01.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f415a720 0x7f90f4151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f415a720 0x7f90f4151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34062/0 (socket says 172.21.15.3:34062) 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 -- 172.21.15.3:0/2931733082 learned_addr learned my addr 172.21.15.3:0/2931733082 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f93df700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40cba50 0x7f90f414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f8bde700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90f4150690 0x7f90f40c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 -- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90f4150690 msgr2=0x7f90f40c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90f4150690 0x7f90f40c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 -- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40cba50 msgr2=0x7f90f414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40cba50 0x7f90f414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 -- 172.21.15.3:0/2931733082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec0094f0 con 0x7f90f415a720 2023-10-06T04:59:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f8bde700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90f4150690 0x7f90f40c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f93df700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40cba50 0x7f90f414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f9be0700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f415a720 0x7f90f4151420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f90e400bbb0 tx=0x7f90e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f17fa700 1 -- 172.21.15.3:0/2931733082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e40092e0 con 0x7f90f415a720 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f17fa700 1 -- 172.21.15.3:0/2931733082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e400c7a0 con 0x7f90f415a720 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90f17fa700 1 -- 172.21.15.3:0/2931733082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e400cb40 con 0x7f90f415a720 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f40c74f0 con 0x7f90f415a720 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.909+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f40c9170 con 0x7f90f415a720 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.910+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f40c9390 con 0x7f90f415a720 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.911+0000 7f90f17fa700 1 -- 172.21.15.3:0/2931733082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90e4012070 con 0x7f90f415a720 2023-10-06T04:59:01.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.911+0000 7f90f17fa700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e0066f30 0x7f90e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:01.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.911+0000 7f90f17fa700 1 -- 172.21.15.3:0/2931733082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f90e40840b0 con 0x7f90f415a720 2023-10-06T04:59:01.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.911+0000 7f90f93df700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e0066f30 0x7f90e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:01.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.912+0000 7f90f93df700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e0066f30 0x7f90e00693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ce1e0 tx=0x7f90ec016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:01.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:01.915+0000 7f90f17fa700 1 -- 172.21.15.3:0/2931733082 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90e4051470 con 0x7f90f415a720 2023-10-06T04:59:02.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.111+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f90f404c380 con 0x7f90f415a720 2023-10-06T04:59:02.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.111+0000 7f90f17fa700 1 -- 172.21.15.3:0/2931733082 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90e4054a90 con 0x7f90f415a720 2023-10-06T04:59:02.113 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:59:02.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e0066f30 msgr2=0x7f90e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e0066f30 0x7f90e00693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ce1e0 tx=0x7f90ec016000 comp rx=0 tx=0).stop 2023-10-06T04:59:02.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f415a720 msgr2=0x7f90f4151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f415a720 0x7f90f4151420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f90e400bbb0 tx=0x7f90e4009840 comp rx=0 tx=0).stop 2023-10-06T04:59:02.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 shutdown_connections 2023-10-06T04:59:02.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90e0066f30 0x7f90e00693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40cba50 0x7f90f414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90f415a720 0x7f90f4151420 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 --2- 172.21.15.3:0/2931733082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90f4150690 0x7f90f40c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 >> 172.21.15.3:0/2931733082 conn(0x7f90f4070310 msgr2=0x7f90f4158040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:02.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 shutdown_connections 2023-10-06T04:59:02.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.113+0000 7f90fb643700 1 -- 172.21.15.3:0/2931733082 wait complete. 2023-10-06T04:59:02.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:59:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.248+0000 7f33e53d0700 1 Processor -- start 2023-10-06T04:59:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.248+0000 7f33e53d0700 1 -- start start 2023-10-06T04:59:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e53d0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e53d0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f33e00cd190 0x7f33e00c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e53d0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e00cb190 con 0x7f33e00af290 2023-10-06T04:59:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e53d0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e0159950 con 0x7f33e00cd190 2023-10-06T04:59:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e53d0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00cd6a0 con 0x7f33e00af660 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e4bcf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f33e00cd190 0x7f33e00c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33df7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e4bcf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f33e00cd190 0x7f33e00c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:41164/0 (socket says 172.21.15.3:41164) 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33df7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:59576/0 (socket says 172.21.15.3:59576) 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33e4bcf700 1 -- 172.21.15.3:0/2147911756 learned_addr learned my addr 172.21.15.3:0/2147911756 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33df7fe700 1 -- 172.21.15.3:0/2147911756 >> v2:172.21.15.153:3300/0 conn(0x7f33e00cd190 msgr2=0x7f33e00c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33df7fe700 1 --2- 172.21.15.3:0/2147911756 >> v2:172.21.15.153:3300/0 conn(0x7f33e00cd190 0x7f33e00c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.249+0000 7f33df7fe700 1 -- 172.21.15.3:0/2147911756 >> v1:172.21.15.153:6789/0 conn(0x7f33e00af290 legacy=0x7f33e00ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33df7fe700 1 -- 172.21.15.3:0/2147911756 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00cb3f0 con 0x7f33e00af660 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33e4bcf700 1 --2- 172.21.15.3:0/2147911756 >> v2:172.21.15.153:3300/0 conn(0x7f33e00cd190 0x7f33e00c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33df7fe700 1 --2- 172.21.15.3:0/2147911756 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00ccc50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f33d00098d0 tx=0x7f33d0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506c7abe422c4fa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33deffd700 1 -- 172.21.15.3:0/2147911756 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0005c10 con 0x7f33e00af660 2023-10-06T04:59:02.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33deffd700 1 -- 172.21.15.3:0/2147911756 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0005d70 con 0x7f33e00af660 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33deffd700 1 -- 172.21.15.3:0/2147911756 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0006280 con 0x7f33e00af660 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33e53d0700 1 -- 172.21.15.3:0/2147911756 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 msgr2=0x7f33e00ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33e53d0700 1 --2- 172.21.15.3:0/2147911756 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00ccc50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f33d00098d0 tx=0x7f33d0009120 comp rx=0 tx=0).stop 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33e53d0700 1 -- 172.21.15.3:0/2147911756 shutdown_connections 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33e53d0700 1 --2- 172.21.15.3:0/2147911756 >> v2:172.21.15.153:3300/0 conn(0x7f33e00cd190 0x7f33e00c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33e53d0700 1 --2- 172.21.15.3:0/2147911756 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00ccc50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.250+0000 7f33e53d0700 1 -- 172.21.15.3:0/2147911756 >> 172.21.15.3:0/2147911756 conn(0x7f33e0089dd0 msgr2=0x7f33e00996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.251+0000 7f33e53d0700 1 -- 172.21.15.3:0/2147911756 shutdown_connections 2023-10-06T04:59:02.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.251+0000 7f33e53d0700 1 -- 172.21.15.3:0/2147911756 wait complete. 2023-10-06T04:59:02.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.252+0000 7f33e53d0700 1 Processor -- start 2023-10-06T04:59:02.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.252+0000 7f33e53d0700 1 -- start start 2023-10-06T04:59:02.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.252+0000 7f33e53d0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33e00af290 0x7f33e00b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e53d0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e53d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e00cd190 0x7f33e00b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e53d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00d04e0 con 0x7f33e00cd190 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e53d0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00ce1b0 con 0x7f33e00af290 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e53d0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00cf010 con 0x7f33e00af660 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e00cd190 0x7f33e00b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e00cd190 0x7f33e00b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34242/0 (socket says 172.21.15.3:34242) 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 -- 172.21.15.3:0/542535691 learned_addr learned my addr 172.21.15.3:0/542535691 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 -- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 msgr2=0x7f33e00cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 -- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33e00af290 msgr2=0x7f33e00b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33e00af290 0x7f33e00b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 -- 172.21.15.3:0/542535691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8009ae0 con 0x7f33e00cd190 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e4bcf700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e00cd190 0x7f33e00b04d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f30 tx=0x7f33d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33d7fff700 1 -- 172.21.15.3:0/542535691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8018070 con 0x7f33e00cd190 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33d7fff700 1 -- 172.21.15.3:0/542535691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d800f040 con 0x7f33e00cd190 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33d7fff700 1 -- 172.21.15.3:0/542535691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d800ccd0 con 0x7f33e00cd190 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d00093c0 con 0x7f33e00cd190 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.253+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e00ce9c0 con 0x7f33e00cd190 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.254+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e004c380 con 0x7f33e00cd190 2023-10-06T04:59:02.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.255+0000 7f33d7fff700 1 -- 172.21.15.3:0/542535691 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33d8003020 con 0x7f33e00cd190 2023-10-06T04:59:02.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.255+0000 7f33d7fff700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33c0066f30 0x7f33c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.255+0000 7f33d7fff700 1 -- 172.21.15.3:0/542535691 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f33d8084420 con 0x7f33e00cd190 2023-10-06T04:59:02.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.255+0000 7f33dffff700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33c0066f30 0x7f33c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.256+0000 7f33dffff700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33c0066f30 0x7f33c00693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f33c80016f0 tx=0x7f33c8007af0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.260+0000 7f33d7fff700 1 -- 172.21.15.3:0/542535691 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33d8055270 con 0x7f33e00cd190 2023-10-06T04:59:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.452+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33e00ce400 con 0x7f33e00cd190 2023-10-06T04:59:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.452+0000 7f33d7fff700 1 -- 172.21.15.3:0/542535691 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33d8054e00 con 0x7f33e00cd190 2023-10-06T04:59:02.454 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:59:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33c0066f30 msgr2=0x7f33c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33c0066f30 0x7f33c00693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f33c80016f0 tx=0x7f33c8007af0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e00cd190 msgr2=0x7f33e00b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e00cd190 0x7f33e00b04d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f30 tx=0x7f33d800b040 comp rx=0 tx=0).stop 2023-10-06T04:59:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 shutdown_connections 2023-10-06T04:59:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f33c0066f30 0x7f33c00693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33e00af290 0x7f33e00b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e00cd190 0x7f33e00b04d0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 --2- 172.21.15.3:0/542535691 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33e00af660 0x7f33e00cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 >> 172.21.15.3:0/542535691 conn(0x7f33e0089dd0 msgr2=0x7f33e006ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 shutdown_connections 2023-10-06T04:59:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.454+0000 7f33e53d0700 1 -- 172.21.15.3:0/542535691 wait complete. 2023-10-06T04:59:02.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:59:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.585+0000 7f8a0453c700 1 Processor -- start 2023-10-06T04:59:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.585+0000 7f8a0453c700 1 -- start start 2023-10-06T04:59:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.586+0000 7f8a0453c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89fc09b500 0x7f89fc159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.586+0000 7f8a0453c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15e000 con 0x7f89fc15d0d0 2023-10-06T04:59:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.586+0000 7f8a0453c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc0b2520 con 0x7f89fc1536e0 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.586+0000 7f8a0453c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc15d550 con 0x7f89fc09b500 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.586+0000 7f8a02ad9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f89fc15d0d0 0x7f89fc0ac1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34046/0 (socket says 172.21.15.3:34046) 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.586+0000 7f8a02ad9700 1 -- 172.21.15.3:0/3135075623 learned_addr learned my addr 172.21.15.3:0/3135075623 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.586+0000 7f8a01ad7700 1 --2- 172.21.15.3:0/3135075623 >> v2:172.21.15.153:3300/0 conn(0x7f89fc09b500 0x7f89fc159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510231136 0 0) 0x7f89fc15e000 con 0x7f89fc15d0d0 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e4003570 con 0x7f89fc15d0d0 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233903003 0 0) 0x7f89fc0b2520 con 0x7f89fc1536e0 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89fc15e000 con 0x7f89fc1536e0 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1458097938 0 0) 0x7f89e4003570 con 0x7f89fc15d0d0 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 >> v2:172.21.15.153:3300/0 conn(0x7f89fc09b500 msgr2=0x7f89fc159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 --2- 172.21.15.3:0/3135075623 >> v2:172.21.15.153:3300/0 conn(0x7f89fc09b500 0x7f89fc159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 >> v1:172.21.15.153:6789/0 conn(0x7f89fc1536e0 legacy=0x7f89fc09adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc15d760 con 0x7f89fc15d0d0 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f89f4001470 con 0x7f89fc15d0d0 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 >> v1:172.21.15.3:6789/0 conn(0x7f89fc15d0d0 legacy=0x7f89fc0ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4003bf0 0x7f89e4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89e40051c0 0x7f89e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc15d550 con 0x7f89e4003bf0 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c160 con 0x7f89e4004810 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c320 con 0x7f89e40051c0 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a022d8700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4003bf0 0x7f89e4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a01ad7700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a02ad9700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89e40051c0 0x7f89e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a022d8700 1 -- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89e40051c0 msgr2=0x7f89e400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a022d8700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89e40051c0 0x7f89e400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a022d8700 1 -- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4004810 msgr2=0x7f89e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a022d8700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a022d8700 1 -- 172.21.15.3:0/3135075623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4003570 con 0x7f89e4003bf0 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a02ad9700 1 -- 172.21.15.3:0/3135075623 reap_dead start 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a01ad7700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.587+0000 7f8a022d8700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4003bf0 0x7f89e4003fe0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c390 tx=0x7f89ec001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c25ff9054c39244 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec001ee0 con 0x7f89e4003bf0 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec00cc80 con 0x7f89e4003bf0 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a012d6700 1 -- 172.21.15.3:0/3135075623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec012420 con 0x7f89e4003bf0 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a0453c700 1 -- 172.21.15.3:0/3135075623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4003bf0 msgr2=0x7f89e4003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a0453c700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4003bf0 0x7f89e4003fe0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c390 tx=0x7f89ec001bd0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a0453c700 1 -- 172.21.15.3:0/3135075623 shutdown_connections 2023-10-06T04:59:02.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a0453c700 1 --2- 172.21.15.3:0/3135075623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4003bf0 0x7f89e4003fe0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a0453c700 1 -- 172.21.15.3:0/3135075623 >> 172.21.15.3:0/3135075623 conn(0x7f89fc06c370 msgr2=0x7f89fc06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:02.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a0453c700 1 -- 172.21.15.3:0/3135075623 shutdown_connections 2023-10-06T04:59:02.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.588+0000 7f8a0453c700 1 -- 172.21.15.3:0/3135075623 wait complete. 2023-10-06T04:59:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.589+0000 7f8a0453c700 1 Processor -- start 2023-10-06T04:59:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.589+0000 7f8a0453c700 1 -- start start 2023-10-06T04:59:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a0453c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4003bf0 0x7f89fc13fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a0453c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89fc14ce70 0x7f89fc0b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a0453c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89fc0b0660 0x7f89fc15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a0453c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc14e790 con 0x7f89fc14ce70 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a0453c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc14ea70 con 0x7f89e4003bf0 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a0453c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0ae550 con 0x7f89fc0b0660 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a01ad7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89fc14ce70 0x7f89fc0b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a01ad7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89fc14ce70 0x7f89fc0b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34260/0 (socket says 172.21.15.3:34260) 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a01ad7700 1 -- 172.21.15.3:0/2759954583 learned_addr learned my addr 172.21.15.3:0/2759954583 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a022d8700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4003bf0 0x7f89fc13fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.590+0000 7f8a02ad9700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89fc0b0660 0x7f89fc15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a01ad7700 1 -- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89fc0b0660 msgr2=0x7f89fc15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a01ad7700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89fc0b0660 0x7f89fc15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a01ad7700 1 -- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4003bf0 msgr2=0x7f89fc13fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a01ad7700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4003bf0 0x7f89fc13fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a01ad7700 1 -- 172.21.15.3:0/2759954583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0ae830 con 0x7f89fc14ce70 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a02ad9700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89fc0b0660 0x7f89fc15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a022d8700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4003bf0 0x7f89fc13fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a01ad7700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89fc14ce70 0x7f89fc0b0120 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f89f0011740 tx=0x7f89f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f89fa7fc700 1 -- 172.21.15.3:0/2759954583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0015670 con 0x7f89fc14ce70 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f89fa7fc700 1 -- 172.21.15.3:0/2759954583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f00157d0 con 0x7f89fc14ce70 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc0aeb10 con 0x7f89fc14ce70 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f89fa7fc700 1 -- 172.21.15.3:0/2759954583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0015b70 con 0x7f89fc14ce70 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.591+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc169610 con 0x7f89fc14ce70 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.592+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc071680 con 0x7f89fc14ce70 2023-10-06T04:59:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.592+0000 7f89fa7fc700 1 -- 172.21.15.3:0/2759954583 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89f0013410 con 0x7f89fc14ce70 2023-10-06T04:59:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.593+0000 7f89fa7fc700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.593+0000 7f89fa7fc700 1 -- 172.21.15.3:0/2759954583 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f89f0084ae0 con 0x7f89fc14ce70 2023-10-06T04:59:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.593+0000 7f8a022d8700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.593+0000 7f8a022d8700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000f80 tx=0x7f89ec016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.597+0000 7f89fa7fc700 1 -- 172.21.15.3:0/2759954583 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89f0051ea0 con 0x7f89fc14ce70 2023-10-06T04:59:02.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.792+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f89fc15da40 con 0x7f89fc14ce70 2023-10-06T04:59:02.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.793+0000 7f89fa7fc700 1 -- 172.21.15.3:0/2759954583 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+126 (secure 0 0 0) 0x7f89f00554c0 con 0x7f89fc14ce70 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:59:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89e8066f30 msgr2=0x7f89e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000f80 tx=0x7f89ec016000 comp rx=0 tx=0).stop 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89fc14ce70 msgr2=0x7f89fc0b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89fc14ce70 0x7f89fc0b0120 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f89f0011740 tx=0x7f89f0008040 comp rx=0 tx=0).stop 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 shutdown_connections 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89e4003bf0 0x7f89fc13fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89fc0b0660 0x7f89fc15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 --2- 172.21.15.3:0/2759954583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89fc14ce70 0x7f89fc0b0120 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 >> 172.21.15.3:0/2759954583 conn(0x7f89fc06c370 msgr2=0x7f89fc0ac0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:02.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 shutdown_connections 2023-10-06T04:59:02.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.795+0000 7f8a0453c700 1 -- 172.21.15.3:0/2759954583 wait complete. 2023-10-06T04:59:02.809 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:59:02.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:59:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.930+0000 7faaa04e0700 1 Processor -- start 2023-10-06T04:59:02.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.931+0000 7faaa04e0700 1 -- start start 2023-10-06T04:59:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.931+0000 7faaa04e0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 0x7faa980cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faaa04e0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa980ccda0 con 0x7faa980d1170 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faaa04e0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa980cf7d0 con 0x7faa981598f0 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faaa04e0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9815e400 con 0x7faa9815dca0 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9ea7d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faa980d1170 0x7faa98159cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34058/0 (socket says 172.21.15.3:34058) 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9ea7d700 1 -- 172.21.15.3:0/854074709 learned_addr learned my addr 172.21.15.3:0/854074709 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9da7b700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 0x7faa980cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257669084 0 0) 0x7faa980ccda0 con 0x7faa980d1170 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa80003570 con 0x7faa980d1170 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148141972 0 0) 0x7faa80003570 con 0x7faa980d1170 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 msgr2=0x7faa980cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 0x7faa980cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 >> v1:172.21.15.153:6789/0 conn(0x7faa981598f0 legacy=0x7faa9815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980cf210 con 0x7faa980d1170 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7faa880017f0 con 0x7faa980d1170 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 >> v1:172.21.15.3:6789/0 conn(0x7faa980d1170 legacy=0x7faa98159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa80003a90 0x7faa80003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 0x7faa80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9815e400 con 0x7faa80004d50 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8000bb70 con 0x7faa80003a90 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8000bd30 con 0x7faa9815dca0 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.932+0000 7faa9ea7d700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9da7b700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 0x7faa80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9e27c700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa80003a90 0x7faa80003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9ea7d700 1 -- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 msgr2=0x7faa80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9ea7d700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa9815dca0 0x7faa80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9ea7d700 1 -- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa80003a90 msgr2=0x7faa80003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9ea7d700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa80003a90 0x7faa80003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9ea7d700 1 -- 172.21.15.3:0/854074709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80003570 con 0x7faa80004d50 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9ea7d700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa800051c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7faa8800a930 tx=0x7faa880021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbde07b22c4317d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880070d0 con 0x7faa80004d50 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa88007230 con 0x7faa80004d50 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.933+0000 7faa9d27a700 1 -- 172.21.15.3:0/854074709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880075d0 con 0x7faa80004d50 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.934+0000 7faaa04e0700 1 -- 172.21.15.3:0/854074709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 msgr2=0x7faa800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.934+0000 7faaa04e0700 1 --2- 172.21.15.3:0/854074709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa800051c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7faa8800a930 tx=0x7faa880021d0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.934+0000 7faa9ea7d700 1 -- 172.21.15.3:0/854074709 reap_dead start 2023-10-06T04:59:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.934+0000 7faaa04e0700 1 -- 172.21.15.3:0/854074709 shutdown_connections 2023-10-06T04:59:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.934+0000 7faaa04e0700 1 -- 172.21.15.3:0/854074709 >> 172.21.15.3:0/854074709 conn(0x7faa98070af0 msgr2=0x7faa980d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.934+0000 7faaa04e0700 1 -- 172.21.15.3:0/854074709 shutdown_connections 2023-10-06T04:59:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.934+0000 7faaa04e0700 1 -- 172.21.15.3:0/854074709 wait complete. 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.935+0000 7faaa04e0700 1 Processor -- start 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.935+0000 7faaa04e0700 1 -- start start 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.935+0000 7faaa04e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa9815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faaa04e0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9815c9b0 0x7faa980b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faaa04e0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa981527e0 0x7faa980b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faaa04e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9815da30 con 0x7faa80004d50 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faaa04e0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9815ce20 con 0x7faa9815c9b0 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faaa04e0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9815d100 con 0x7faa981527e0 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa9815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa9815c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34276/0 (socket says 172.21.15.3:34276) 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 -- 172.21.15.3:0/4144342721 learned_addr learned my addr 172.21.15.3:0/4144342721 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9ea7d700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa981527e0 0x7faa980b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9da7b700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9815c9b0 0x7faa980b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 -- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa981527e0 msgr2=0x7faa980b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa981527e0 0x7faa980b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 -- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9815c9b0 msgr2=0x7faa980b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9815c9b0 0x7faa980b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 -- 172.21.15.3:0/4144342721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9815d3e0 con 0x7faa80004d50 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9ea7d700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa981527e0 0x7faa980b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9da7b700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9815c9b0 0x7faa980b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa9e27c700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa9815c020 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7faa90002920 tx=0x7faa90007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa967fc700 1 -- 172.21.15.3:0/4144342721 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000f8d0 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa967fc700 1 -- 172.21.15.3:0/4144342721 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9000fa30 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9815d6c0 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faa967fc700 1 -- 172.21.15.3:0/4144342721 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000fdf0 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.936+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa98153990 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.937+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa981537e0 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.938+0000 7faa967fc700 1 -- 172.21.15.3:0/4144342721 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa90013600 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.938+0000 7faa967fc700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.938+0000 7faa967fc700 1 -- 172.21.15.3:0/4144342721 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5206+0+0 (secure 0 0 0) 0x7faa900842a0 con 0x7faa80004d50 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.938+0000 7faa9da7b700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.939+0000 7faa9da7b700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faa84066f30 0x7faa840693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0093f0 tx=0x7faa8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:02.942+0000 7faa967fc700 1 -- 172.21.15.3:0/4144342721 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa90051700 con 0x7faa80004d50 2023-10-06T04:59:03.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:03.137+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faa98071680 con 0x7faa80004d50 2023-10-06T04:59:04.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.006+0000 7faa967fc700 1 -- 172.21.15.3:0/4144342721 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7faa90054d20 con 0x7faa80004d50 2023-10-06T04:59:04.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faa84066f30 msgr2=0x7faa840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:04.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faa84066f30 0x7faa840693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0093f0 tx=0x7faa8c008040 comp rx=0 tx=0).stop 2023-10-06T04:59:04.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 msgr2=0x7faa9815c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa9815c020 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7faa90002920 tx=0x7faa90007d30 comp rx=0 tx=0).stop 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 shutdown_connections 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa981527e0 0x7faa980b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa80004d50 0x7faa9815c020 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 --2- 172.21.15.3:0/4144342721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9815c9b0 0x7faa980b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 >> 172.21.15.3:0/4144342721 conn(0x7faa98070af0 msgr2=0x7faa980d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 shutdown_connections 2023-10-06T04:59:04.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.010+0000 7faaa04e0700 1 -- 172.21.15.3:0/4144342721 wait complete. 2023-10-06T04:59:04.013 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:59:04.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:59:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.157+0000 7f4e3df2b700 1 Processor -- start 2023-10-06T04:59:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.157+0000 7f4e3df2b700 1 -- start start 2023-10-06T04:59:04.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.158+0000 7f4e3df2b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e3815ac90 0x7f4e380b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:04.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.158+0000 7f4e3df2b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e38070310 0x7f4e380b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:04.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.158+0000 7f4e3df2b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e380c9520 con 0x7f4e380ae9d0 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.158+0000 7f4e3df2b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e380c9fa0 con 0x7f4e3815ac90 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.158+0000 7f4e3df2b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3806aea0 con 0x7f4e38070310 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.158+0000 7f4e377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e3815ac90 0x7f4e380b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.158+0000 7f4e37fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4e380ae9d0 0x7f4e3806c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34060/0 (socket says 172.21.15.3:34060) 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e37fff700 1 -- 172.21.15.3:0/3490247310 learned_addr learned my addr 172.21.15.3:0/3490247310 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e36ffd700 1 --2- 172.21.15.3:0/3490247310 >> v2:172.21.15.153:3300/0 conn(0x7f4e38070310 0x7f4e380b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e377fe700 1 -- 172.21.15.3:0/3490247310 >> v2:172.21.15.153:3300/0 conn(0x7f4e38070310 msgr2=0x7f4e380b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e377fe700 1 --2- 172.21.15.3:0/3490247310 >> v2:172.21.15.153:3300/0 conn(0x7f4e38070310 0x7f4e380b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e377fe700 1 -- 172.21.15.3:0/3490247310 >> v1:172.21.15.3:6789/0 conn(0x7f4e380ae9d0 legacy=0x7f4e3806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e367fc700 1 -- 172.21.15.3:0/3490247310 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149532610 0 0) 0x7f4e380c9520 con 0x7f4e380ae9d0 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e377fe700 1 -- 172.21.15.3:0/3490247310 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3806a290 con 0x7f4e3815ac90 2023-10-06T04:59:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e36ffd700 1 --2- 172.21.15.3:0/3490247310 >> v2:172.21.15.153:3300/0 conn(0x7f4e38070310 0x7f4e380b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e377fe700 1 --2- 172.21.15.3:0/3490247310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e3815ac90 0x7f4e380b1960 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4e38071010 tx=0x7f4e30009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6dbdbdd3d47728 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e367fc700 1 -- 172.21.15.3:0/3490247310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30005c10 con 0x7f4e3815ac90 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e367fc700 1 -- 172.21.15.3:0/3490247310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e30005d70 con 0x7f4e3815ac90 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.159+0000 7f4e367fc700 1 -- 172.21.15.3:0/3490247310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30006260 con 0x7f4e3815ac90 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 -- 172.21.15.3:0/3490247310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e3815ac90 msgr2=0x7f4e380b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/3490247310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e3815ac90 0x7f4e380b1960 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4e38071010 tx=0x7f4e30009120 comp rx=0 tx=0).stop 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 -- 172.21.15.3:0/3490247310 shutdown_connections 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/3490247310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e3815ac90 0x7f4e380b1960 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/3490247310 >> v2:172.21.15.153:3300/0 conn(0x7f4e38070310 0x7f4e380b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 -- 172.21.15.3:0/3490247310 >> 172.21.15.3:0/3490247310 conn(0x7f4e38067a40 msgr2=0x7f4e380ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 -- 172.21.15.3:0/3490247310 shutdown_connections 2023-10-06T04:59:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.160+0000 7f4e3df2b700 1 -- 172.21.15.3:0/3490247310 wait complete. 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.161+0000 7f4e3df2b700 1 Processor -- start 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.161+0000 7f4e3df2b700 1 -- start start 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e38070310 0x7f4e380c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e380ae9d0 0x7f4e380c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e3815ac90 0x7f4e380c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38167b70 con 0x7f4e38070310 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380c5290 con 0x7f4e380ae9d0 2023-10-06T04:59:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380c2d20 con 0x7f4e3815ac90 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e38070310 0x7f4e380c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e38070310 0x7f4e380c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34288/0 (socket says 172.21.15.3:34288) 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 -- 172.21.15.3:0/802118159 learned_addr learned my addr 172.21.15.3:0/802118159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e36ffd700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e380ae9d0 0x7f4e380c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 -- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e3815ac90 msgr2=0x7f4e380c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e3815ac90 0x7f4e380c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 -- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e380ae9d0 msgr2=0x7f4e380c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e380ae9d0 0x7f4e380c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 -- 172.21.15.3:0/802118159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e300093c0 con 0x7f4e38070310 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e36ffd700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e380ae9d0 0x7f4e380c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e377fe700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e38070310 0x7f4e380c2740 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4e38068c40 tx=0x7f4e30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:04.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e2f7fe700 1 -- 172.21.15.3:0/802118159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000bd80 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e2f7fe700 1 -- 172.21.15.3:0/802118159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3001b480 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e380c3000 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e2f7fe700 1 -- 172.21.15.3:0/802118159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000bee0 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.162+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e380c35c0 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.163+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3806c000 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.164+0000 7f4e2f7fe700 1 -- 172.21.15.3:0/802118159 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e3001bad0 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.164+0000 7f4e2f7fe700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4e18066f30 0x7f4e180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.164+0000 7f4e2f7fe700 1 -- 172.21.15.3:0/802118159 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f4e30086ad0 con 0x7f4e38070310 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.164+0000 7f4e36ffd700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4e18066f30 0x7f4e180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:04.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.165+0000 7f4e36ffd700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4e18066f30 0x7f4e180693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4e380c8de0 tx=0x7f4e20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:04.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.169+0000 7f4e2f7fe700 1 -- 172.21.15.3:0/802118159 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e30053dc0 con 0x7f4e38070310 2023-10-06T04:59:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:04.361+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4e3804c380 con 0x7f4e38070310 2023-10-06T04:59:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.008+0000 7f4e2f7fe700 1 -- 172.21.15.3:0/802118159 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4e300573e0 con 0x7f4e38070310 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4e18066f30 msgr2=0x7f4e180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4e18066f30 0x7f4e180693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4e380c8de0 tx=0x7f4e20008040 comp rx=0 tx=0).stop 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e38070310 msgr2=0x7f4e380c2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e38070310 0x7f4e380c2740 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4e38068c40 tx=0x7f4e30016000 comp rx=0 tx=0).stop 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 shutdown_connections 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4e18066f30 0x7f4e180693e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e3815ac90 0x7f4e380c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e38070310 0x7f4e380c2740 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 --2- 172.21.15.3:0/802118159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e380ae9d0 0x7f4e380c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.013+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 >> 172.21.15.3:0/802118159 conn(0x7f4e38067a40 msgr2=0x7f4e38159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.015+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 shutdown_connections 2023-10-06T04:59:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.015+0000 7f4e3df2b700 1 -- 172.21.15.3:0/802118159 wait complete. 2023-10-06T04:59:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:59:05.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:59:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.162+0000 7f671479b700 1 Processor -- start 2023-10-06T04:59:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.163+0000 7f671479b700 1 -- start start 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.163+0000 7f671479b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f670c15dd90 0x7f670c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f671479b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15ce30 0x7f670c0aa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f671479b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670c0abdb0 con 0x7f670c0cc8b0 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f671479b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670c070930 con 0x7f670c15dd90 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f671479b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0ac090 con 0x7f670c15ce30 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6712537700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f670c0cc8b0 0x7f670c15c720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34070/0 (socket says 172.21.15.3:34070) 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6712537700 1 -- 172.21.15.3:0/3500643528 learned_addr learned my addr 172.21.15.3:0/3500643528 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711d36700 1 --2- 172.21.15.3:0/3500643528 >> v2:172.21.15.3:3300/0 conn(0x7f670c15dd90 0x7f670c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6712d38700 1 --2- 172.21.15.3:0/3500643528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15ce30 0x7f670c0aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711535700 1 -- 172.21.15.3:0/3500643528 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511664570 0 0) 0x7f670c0abdb0 con 0x7f670c0cc8b0 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711535700 1 -- 172.21.15.3:0/3500643528 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f4003570 con 0x7f670c0cc8b0 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711d36700 1 -- 172.21.15.3:0/3500643528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15ce30 msgr2=0x7f670c0aa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711d36700 1 --2- 172.21.15.3:0/3500643528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15ce30 0x7f670c0aa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711d36700 1 -- 172.21.15.3:0/3500643528 >> v1:172.21.15.3:6789/0 conn(0x7f670c0cc8b0 legacy=0x7f670c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711d36700 1 -- 172.21.15.3:0/3500643528 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c06c510 con 0x7f670c15dd90 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711d36700 1 --2- 172.21.15.3:0/3500643528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15dd90 0x7f670c0ccc80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f67000098e0 tx=0x7f67000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a54bcd57aee0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711535700 1 -- 172.21.15.3:0/3500643528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000b640 con 0x7f670c15dd90 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711535700 1 -- 172.21.15.3:0/3500643528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670000b7a0 con 0x7f670c15dd90 2023-10-06T04:59:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.164+0000 7f6711535700 1 -- 172.21.15.3:0/3500643528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000bb40 con 0x7f670c15dd90 2023-10-06T04:59:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 -- 172.21.15.3:0/3500643528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15dd90 msgr2=0x7f670c0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 --2- 172.21.15.3:0/3500643528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15dd90 0x7f670c0ccc80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f67000098e0 tx=0x7f67000091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 -- 172.21.15.3:0/3500643528 shutdown_connections 2023-10-06T04:59:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 --2- 172.21.15.3:0/3500643528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15ce30 0x7f670c0aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 --2- 172.21.15.3:0/3500643528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15dd90 0x7f670c0ccc80 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 -- 172.21.15.3:0/3500643528 >> 172.21.15.3:0/3500643528 conn(0x7f670c066fb0 msgr2=0x7f670c0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 -- 172.21.15.3:0/3500643528 shutdown_connections 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.165+0000 7f671479b700 1 -- 172.21.15.3:0/3500643528 wait complete. 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.166+0000 7f671479b700 1 Processor -- start 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.166+0000 7f671479b700 1 -- start start 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f671479b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c0cc8b0 0x7f670c153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f671479b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15ce30 0x7f670c14d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f671479b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15dd90 0x7f670c16a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f671479b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c154af0 con 0x7f670c15ce30 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f671479b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c154660 con 0x7f670c0cc8b0 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f671479b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c153a50 con 0x7f670c15dd90 2023-10-06T04:59:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15ce30 0x7f670c14d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15ce30 0x7f670c14d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34304/0 (socket says 172.21.15.3:34304) 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 -- 172.21.15.3:0/1663682781 learned_addr learned my addr 172.21.15.3:0/1663682781 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6712d38700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15dd90 0x7f670c16a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6712537700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c0cc8b0 0x7f670c153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 -- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15dd90 msgr2=0x7f670c16a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15dd90 0x7f670c16a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 -- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c0cc8b0 msgr2=0x7f670c153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c0cc8b0 0x7f670c153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 -- 172.21.15.3:0/1663682781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700009420 con 0x7f670c15ce30 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.167+0000 7f6711d36700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15ce30 0x7f670c14d1e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f670c06e890 tx=0x7f6700017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.168+0000 7f670a7fc700 1 -- 172.21.15.3:0/1663682781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670001bda0 con 0x7f670c15ce30 2023-10-06T04:59:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.168+0000 7f670a7fc700 1 -- 172.21.15.3:0/1663682781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700011420 con 0x7f670c15ce30 2023-10-06T04:59:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.168+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c153d30 con 0x7f670c15ce30 2023-10-06T04:59:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.168+0000 7f670a7fc700 1 -- 172.21.15.3:0/1663682781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000117e0 con 0x7f670c15ce30 2023-10-06T04:59:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.168+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c1542f0 con 0x7f670c15ce30 2023-10-06T04:59:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.169+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c1547f0 con 0x7f670c15ce30 2023-10-06T04:59:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.169+0000 7f670a7fc700 1 -- 172.21.15.3:0/1663682781 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6700011b50 con 0x7f670c15ce30 2023-10-06T04:59:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.170+0000 7f670a7fc700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66f8066f30 0x7f66f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.170+0000 7f670a7fc700 1 -- 172.21.15.3:0/1663682781 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f6700085cf0 con 0x7f670c15ce30 2023-10-06T04:59:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.170+0000 7f6712537700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66f8066f30 0x7f66f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.171+0000 7f6712537700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66f8066f30 0x7f66f80693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f67040058d0 tx=0x7f6704009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:05.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.175+0000 7f670a7fc700 1 -- 172.21.15.3:0/1663682781 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f670008a070 con 0x7f670c15ce30 2023-10-06T04:59:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:05.412+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f670c04c380 con 0x7f670c15ce30 2023-10-06T04:59:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.036+0000 7f670a7fc700 1 -- 172.21.15.3:0/1663682781 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v55) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6700052e60 con 0x7f670c15ce30 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66f8066f30 msgr2=0x7f66f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66f8066f30 0x7f66f80693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f67040058d0 tx=0x7f6704009040 comp rx=0 tx=0).stop 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15ce30 msgr2=0x7f670c14d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15ce30 0x7f670c14d1e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f670c06e890 tx=0x7f6700017000 comp rx=0 tx=0).stop 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 shutdown_connections 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66f8066f30 0x7f66f80693e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c0cc8b0 0x7f670c153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f670c15dd90 0x7f670c16a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 --2- 172.21.15.3:0/1663682781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f670c15ce30 0x7f670c14d1e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 >> 172.21.15.3:0/1663682781 conn(0x7f670c066fb0 msgr2=0x7f670c06d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 shutdown_connections 2023-10-06T04:59:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.043+0000 7f671479b700 1 -- 172.21.15.3:0/1663682781 wait complete. 2023-10-06T04:59:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 15 and data pool 16 2023-10-06T04:59:06.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:06.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.182+0000 7fd6031d6700 1 Processor -- start 2023-10-06T04:59:06.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.182+0000 7fd6031d6700 1 -- start start 2023-10-06T04:59:06.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.182+0000 7fd6031d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5fc06b310 0x7fd5fc1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd6031d6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc151600 0x7fd5fc151a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd6031d6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc15b190 con 0x7fd5fc06b6e0 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd6031d6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc158c00 con 0x7fd5fc06b310 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd6031d6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc15ba30 con 0x7fd5fc151600 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5fc06b310 0x7fd5fc1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5fc06b310 0x7fd5fc1586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34318/0 (socket says 172.21.15.3:34318) 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 -- 172.21.15.3:0/261243429 learned_addr learned my addr 172.21.15.3:0/261243429 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 -- 172.21.15.3:0/261243429 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc151600 msgr2=0x7fd5fc151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 --2- 172.21.15.3:0/261243429 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc151600 0x7fd5fc151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 -- 172.21.15.3:0/261243429 >> v1:172.21.15.153:6789/0 conn(0x7fd5fc06b6e0 legacy=0x7fd5fc0af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 -- 172.21.15.3:0/261243429 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0af960 con 0x7fd5fc06b310 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd600f72700 1 --2- 172.21.15.3:0/261243429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc06b310 0x7fd5fc1586c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099b0 tx=0x7fd5ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d167ddda2e90164d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd5fb7fe700 1 -- 172.21.15.3:0/261243429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00b820 con 0x7fd5fc06b310 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd5fb7fe700 1 -- 172.21.15.3:0/261243429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec00b980 con 0x7fd5fc06b310 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.183+0000 7fd5fb7fe700 1 -- 172.21.15.3:0/261243429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00bd20 con 0x7fd5fc06b310 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 -- 172.21.15.3:0/261243429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc06b310 msgr2=0x7fd5fc1586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 --2- 172.21.15.3:0/261243429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc06b310 0x7fd5fc1586c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099b0 tx=0x7fd5ec009250 comp rx=0 tx=0).stop 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 -- 172.21.15.3:0/261243429 shutdown_connections 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 --2- 172.21.15.3:0/261243429 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc151600 0x7fd5fc151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 --2- 172.21.15.3:0/261243429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc06b310 0x7fd5fc1586c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 -- 172.21.15.3:0/261243429 >> 172.21.15.3:0/261243429 conn(0x7fd5fc0cdc30 msgr2=0x7fd5fc06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 -- 172.21.15.3:0/261243429 shutdown_connections 2023-10-06T04:59:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.184+0000 7fd6031d6700 1 -- 172.21.15.3:0/261243429 wait complete. 2023-10-06T04:59:06.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.185+0000 7fd6031d6700 1 Processor -- start 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.185+0000 7fd6031d6700 1 -- start start 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd6031d6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc06b310 0x7fd5fc148c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd6031d6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5fc06b6e0 0x7fd5fc167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd6031d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc151600 0x7fd5fc144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd6031d6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc168420 con 0x7fd5fc151600 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd6031d6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc1458a0 con 0x7fd5fc06b6e0 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd6031d6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc145450 con 0x7fd5fc06b310 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc151600 0x7fd5fc144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc151600 0x7fd5fc144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34320/0 (socket says 172.21.15.3:34320) 2023-10-06T04:59:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 -- 172.21.15.3:0/1112420364 learned_addr learned my addr 172.21.15.3:0/1112420364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd5fbfff700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5fc06b6e0 0x7fd5fc167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd600f72700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc06b310 0x7fd5fc148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 -- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc06b310 msgr2=0x7fd5fc148c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc06b310 0x7fd5fc148c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 -- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5fc06b6e0 msgr2=0x7fd5fc167ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5fc06b6e0 0x7fd5fc167ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.186+0000 7fd601773700 1 -- 172.21.15.3:0/1112420364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0094f0 con 0x7fd5fc151600 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.187+0000 7fd601773700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc151600 0x7fd5fc144f10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400bc30 tx=0x7fd5f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.187+0000 7fd5f8ff9700 1 -- 172.21.15.3:0/1112420364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40092e0 con 0x7fd5fc151600 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.187+0000 7fd5f8ff9700 1 -- 172.21.15.3:0/1112420364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f400c7a0 con 0x7fd5fc151600 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.187+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc168650 con 0x7fd5fc151600 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.187+0000 7fd5f8ff9700 1 -- 172.21.15.3:0/1112420364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400cb80 con 0x7fd5fc151600 2023-10-06T04:59:06.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.187+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc168c10 con 0x7fd5fc151600 2023-10-06T04:59:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.188+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc145670 con 0x7fd5fc151600 2023-10-06T04:59:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.188+0000 7fd5f8ff9700 1 -- 172.21.15.3:0/1112420364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5f4012070 con 0x7fd5fc151600 2023-10-06T04:59:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.189+0000 7fd5f8ff9700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5e8066f30 0x7fd5e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.189+0000 7fd5f8ff9700 1 -- 172.21.15.3:0/1112420364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd5f4084330 con 0x7fd5fc151600 2023-10-06T04:59:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.189+0000 7fd600f72700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5e8066f30 0x7fd5e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.189+0000 7fd600f72700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5e8066f30 0x7fd5e80693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099b0 tx=0x7fd5ec016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.193+0000 7fd5f8ff9700 1 -- 172.21.15.3:0/1112420364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5fc145670 con 0x7fd5fc151600 2023-10-06T04:59:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.390+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5fc145670 con 0x7fd5fc151600 2023-10-06T04:59:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.391+0000 7fd5f8ff9700 1 -- 172.21.15.3:0/1112420364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16834 (secure 0 0 0) 0x7fd5fc145670 con 0x7fd5fc151600 2023-10-06T04:59:06.393 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:06.393 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":95,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:05.413626+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:03.167920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:04.391498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5e8066f30 msgr2=0x7fd5e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5e8066f30 0x7fd5e80693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099b0 tx=0x7fd5ec016000 comp rx=0 tx=0).stop 2023-10-06T04:59:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc151600 msgr2=0x7fd5fc144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc151600 0x7fd5fc144f10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400bc30 tx=0x7fd5f4009840 comp rx=0 tx=0).stop 2023-10-06T04:59:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 shutdown_connections 2023-10-06T04:59:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5e8066f30 0x7fd5e80693e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5fc151600 0x7fd5fc144f10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5fc06b310 0x7fd5fc148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 --2- 172.21.15.3:0/1112420364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5fc06b6e0 0x7fd5fc167ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 >> 172.21.15.3:0/1112420364 conn(0x7fd5fc0cdc30 msgr2=0x7fd5fc0b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 shutdown_connections 2023-10-06T04:59:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.393+0000 7fd6031d6700 1 -- 172.21.15.3:0/1112420364 wait complete. 2023-10-06T04:59:06.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:06.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.524+0000 7f2e491f9700 1 Processor -- start 2023-10-06T04:59:06.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.525+0000 7f2e491f9700 1 -- start start 2023-10-06T04:59:06.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.525+0000 7f2e491f9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e4415cb50 0x7f2e440b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.525+0000 7f2e491f9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2e440b0590 0x7f2e440b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.525+0000 7f2e491f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e440cd950 0x7f2e440b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.525+0000 7f2e491f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440b0ce0 con 0x7f2e440cd950 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.525+0000 7f2e491f9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440cb820 con 0x7f2e440b0590 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.525+0000 7f2e491f9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440b2c80 con 0x7f2e4415cb50 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e440cd950 0x7f2e440b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e440cd950 0x7f2e440b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34336/0 (socket says 172.21.15.3:34336) 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 -- 172.21.15.3:0/131291826 learned_addr learned my addr 172.21.15.3:0/131291826 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4259c700 1 --2- 172.21.15.3:0/131291826 >> v2:172.21.15.153:3300/0 conn(0x7f2e440b0590 0x7f2e440b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e42d9d700 1 --2- 172.21.15.3:0/131291826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e4415cb50 0x7f2e440b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 -- 172.21.15.3:0/131291826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e4415cb50 msgr2=0x7f2e440b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 --2- 172.21.15.3:0/131291826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e4415cb50 0x7f2e440b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 -- 172.21.15.3:0/131291826 >> v2:172.21.15.153:3300/0 conn(0x7f2e440b0590 msgr2=0x7f2e440b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 --2- 172.21.15.3:0/131291826 >> v2:172.21.15.153:3300/0 conn(0x7f2e440b0590 0x7f2e440b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 -- 172.21.15.3:0/131291826 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e440b0a00 con 0x7f2e440cd950 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e4359e700 1 --2- 172.21.15.3:0/131291826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440cd950 0x7f2e440b2740 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2e44089540 tx=0x7f2e3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d20506cd4805c38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e41d9b700 1 -- 172.21.15.3:0/131291826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00e070 con 0x7f2e440cd950 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e41d9b700 1 -- 172.21.15.3:0/131291826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c00b970 con 0x7f2e440cd950 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.526+0000 7f2e41d9b700 1 -- 172.21.15.3:0/131291826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0092e0 con 0x7f2e440cd950 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 -- 172.21.15.3:0/131291826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440cd950 msgr2=0x7f2e440b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 --2- 172.21.15.3:0/131291826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440cd950 0x7f2e440b2740 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2e44089540 tx=0x7f2e3c009840 comp rx=0 tx=0).stop 2023-10-06T04:59:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 -- 172.21.15.3:0/131291826 shutdown_connections 2023-10-06T04:59:06.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 --2- 172.21.15.3:0/131291826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440cd950 0x7f2e440b2740 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 --2- 172.21.15.3:0/131291826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e4415cb50 0x7f2e440b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 --2- 172.21.15.3:0/131291826 >> v2:172.21.15.153:3300/0 conn(0x7f2e440b0590 0x7f2e440b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 -- 172.21.15.3:0/131291826 >> 172.21.15.3:0/131291826 conn(0x7f2e44070310 msgr2=0x7f2e44070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:06.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 -- 172.21.15.3:0/131291826 shutdown_connections 2023-10-06T04:59:06.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.527+0000 7f2e491f9700 1 -- 172.21.15.3:0/131291826 wait complete. 2023-10-06T04:59:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.528+0000 7f2e491f9700 1 Processor -- start 2023-10-06T04:59:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.528+0000 7f2e491f9700 1 -- start start 2023-10-06T04:59:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e491f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440b0590 0x7f2e440d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e491f9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e440cd950 0x7f2e441582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e491f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e4415cb50 0x7f2e440c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e491f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440c9c90 con 0x7f2e440b0590 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e491f9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440d0b10 con 0x7f2e4415cb50 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e491f9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44158820 con 0x7f2e440cd950 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440b0590 0x7f2e440d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440b0590 0x7f2e440d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34350/0 (socket says 172.21.15.3:34350) 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 -- 172.21.15.3:0/2093997554 learned_addr learned my addr 172.21.15.3:0/2093997554 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e4259c700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e440cd950 0x7f2e441582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e4359e700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e4415cb50 0x7f2e440c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 -- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e440cd950 msgr2=0x7f2e441582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e440cd950 0x7f2e441582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 -- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e4415cb50 msgr2=0x7f2e440c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e4415cb50 0x7f2e440c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 -- 172.21.15.3:0/2093997554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c009ae0 con 0x7f2e440b0590 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e4359e700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e4415cb50 0x7f2e440c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.529+0000 7f2e42d9d700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440b0590 0x7f2e440d0eb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400ec80 tx=0x7f2e34009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.530+0000 7f2e2affd700 1 -- 172.21.15.3:0/2093997554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400c7f0 con 0x7f2e440b0590 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.530+0000 7f2e2affd700 1 -- 172.21.15.3:0/2093997554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3400c950 con 0x7f2e440b0590 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.530+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e440ce490 con 0x7f2e440b0590 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.530+0000 7f2e2affd700 1 -- 172.21.15.3:0/2093997554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400cd10 con 0x7f2e440b0590 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.530+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e440c99b0 con 0x7f2e440b0590 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.530+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e44071680 con 0x7f2e440b0590 2023-10-06T04:59:06.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.531+0000 7f2e2affd700 1 -- 172.21.15.3:0/2093997554 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e34013680 con 0x7f2e440b0590 2023-10-06T04:59:06.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.532+0000 7f2e2affd700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.532+0000 7f2e2affd700 1 -- 172.21.15.3:0/2093997554 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2e340851a0 con 0x7f2e440b0590 2023-10-06T04:59:06.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.532+0000 7f2e4259c700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.533+0000 7f2e4259c700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2e4415e800 tx=0x7f2e380091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.536+0000 7f2e2affd700 1 -- 172.21.15.3:0/2093997554 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e34052290 con 0x7f2e440b0590 2023-10-06T04:59:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.727+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e44070c40 con 0x7f2e440b0590 2023-10-06T04:59:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.728+0000 7f2e2affd700 1 -- 172.21.15.3:0/2093997554 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f2e340558b0 con 0x7f2e440b0590 2023-10-06T04:59:06.730 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:06.730 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":95,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:05.413626+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:03.167920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:04.391498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:06.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 msgr2=0x7f2e300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2e4415e800 tx=0x7f2e380091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440b0590 msgr2=0x7f2e440d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440b0590 0x7f2e440d0eb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400ec80 tx=0x7f2e34009120 comp rx=0 tx=0).stop 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 shutdown_connections 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e4415cb50 0x7f2e440c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e440b0590 0x7f2e440d0eb0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 --2- 172.21.15.3:0/2093997554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e440cd950 0x7f2e441582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 >> 172.21.15.3:0/2093997554 conn(0x7f2e44070310 msgr2=0x7f2e4415cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 shutdown_connections 2023-10-06T04:59:06.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.730+0000 7f2e491f9700 1 -- 172.21.15.3:0/2093997554 wait complete. 2023-10-06T04:59:06.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:59:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.865+0000 7f2d0eb14700 1 Processor -- start 2023-10-06T04:59:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.865+0000 7f2d0eb14700 1 -- start start 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0eb14700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d08150690 0x7f2d08150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0eb14700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2d080cba50 0x7f2d0815d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0eb14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0eb14700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0815da80 con 0x7f2d08150690 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0eb14700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0815a290 con 0x7f2d080cba50 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0eb14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080d0020 con 0x7f2d0815a720 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d08150690 0x7f2d08150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d08150690 0x7f2d08150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34364/0 (socket says 172.21.15.3:34364) 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/1710815999 learned_addr learned my addr 172.21.15.3:0/1710815999 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/1710815999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 msgr2=0x7f2d080af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0d0b1700 1 --2- 172.21.15.3:0/1710815999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 --2- 172.21.15.3:0/1710815999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d07fff700 1 --2- 172.21.15.3:0/1710815999 >> v2:172.21.15.153:3300/0 conn(0x7f2d080cba50 0x7f2d0815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/1710815999 >> v2:172.21.15.153:3300/0 conn(0x7f2d080cba50 msgr2=0x7f2d0815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 --2- 172.21.15.3:0/1710815999 >> v2:172.21.15.153:3300/0 conn(0x7f2d080cba50 0x7f2d0815d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/1710815999 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08159f40 con 0x7f2d08150690 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d0c8b0700 1 --2- 172.21.15.3:0/1710815999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d08150690 0x7f2d08150a60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2d080ce1e0 tx=0x7f2cf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f06922954d5c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d077fe700 1 -- 172.21.15.3:0/1710815999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800b860 con 0x7f2d08150690 2023-10-06T04:59:06.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d077fe700 1 -- 172.21.15.3:0/1710815999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf800b9c0 con 0x7f2d08150690 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.866+0000 7f2d077fe700 1 -- 172.21.15.3:0/1710815999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800bd60 con 0x7f2d08150690 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 -- 172.21.15.3:0/1710815999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d08150690 msgr2=0x7f2d08150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/1710815999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d08150690 0x7f2d08150a60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2d080ce1e0 tx=0x7f2cf8009250 comp rx=0 tx=0).stop 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 -- 172.21.15.3:0/1710815999 shutdown_connections 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/1710815999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/1710815999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d08150690 0x7f2d08150a60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/1710815999 >> v2:172.21.15.153:3300/0 conn(0x7f2d080cba50 0x7f2d0815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 -- 172.21.15.3:0/1710815999 >> 172.21.15.3:0/1710815999 conn(0x7f2d08070310 msgr2=0x7f2d08070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 -- 172.21.15.3:0/1710815999 shutdown_connections 2023-10-06T04:59:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.867+0000 7f2d0eb14700 1 -- 172.21.15.3:0/1710815999 wait complete. 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.868+0000 7f2d0eb14700 1 Processor -- start 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.868+0000 7f2d0eb14700 1 -- start start 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0eb14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d080cba50 0x7f2d08153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0eb14700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d08150690 0x7f2d0814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0eb14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0eb14700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080c84a0 con 0x7f2d080cba50 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0eb14700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080c7890 con 0x7f2d08150690 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0eb14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080c7b70 con 0x7f2d0815a720 2023-10-06T04:59:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d080cba50 0x7f2d08153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d080cba50 0x7f2d08153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34366/0 (socket says 172.21.15.3:34366) 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/3879929176 learned_addr learned my addr 172.21.15.3:0/3879929176 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0d0b1700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d07fff700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d08150690 0x7f2d0814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 msgr2=0x7f2d080cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d08150690 msgr2=0x7f2d0814fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d08150690 0x7f2d0814fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.869+0000 7f2d0c8b0700 1 -- 172.21.15.3:0/3879929176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80094f0 con 0x7f2d080cba50 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.870+0000 7f2d0c8b0700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d080cba50 0x7f2d08153060 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2d080c71f0 tx=0x7f2cf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.870+0000 7f2d04ff9700 1 -- 172.21.15.3:0/3879929176 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8024440 con 0x7f2d080cba50 2023-10-06T04:59:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.870+0000 7f2d04ff9700 1 -- 172.21.15.3:0/3879929176 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf80245a0 con 0x7f2d080cba50 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.870+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d080c7e50 con 0x7f2d080cba50 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.870+0000 7f2d04ff9700 1 -- 172.21.15.3:0/3879929176 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8024960 con 0x7f2d080cba50 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.870+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d080c74f0 con 0x7f2d080cba50 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.871+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0804c380 con 0x7f2d080cba50 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.871+0000 7f2d04ff9700 1 -- 172.21.15.3:0/3879929176 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cf801a420 con 0x7f2d080cba50 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.872+0000 7f2d04ff9700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2cf4066f30 0x7f2cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.872+0000 7f2d04ff9700 1 -- 172.21.15.3:0/3879929176 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2cf80865a0 con 0x7f2d080cba50 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.872+0000 7f2d07fff700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2cf4066f30 0x7f2cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.872+0000 7f2d07fff700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2cf4066f30 0x7f2cf40693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2d0816d0c0 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:06.876+0000 7f2d04ff9700 1 -- 172.21.15.3:0/3879929176 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cf80536b0 con 0x7f2d080cba50 2023-10-06T04:59:07.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:07.121+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2d0815dcb0 con 0x7f2d080cba50 2023-10-06T04:59:08.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.043+0000 7f2d04ff9700 1 -- 172.21.15.3:0/3879929176 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2cf8056cd0 con 0x7f2d080cba50 2023-10-06T04:59:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2cf4066f30 msgr2=0x7f2cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2cf4066f30 0x7f2cf40693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2d0816d0c0 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2023-10-06T04:59:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d080cba50 msgr2=0x7f2d08153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d080cba50 0x7f2d08153060 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2d080c71f0 tx=0x7f2cf8017000 comp rx=0 tx=0).stop 2023-10-06T04:59:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 shutdown_connections 2023-10-06T04:59:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2cf4066f30 0x7f2cf40693e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d0815a720 0x7f2d080cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d080cba50 0x7f2d08153060 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 --2- 172.21.15.3:0/3879929176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d08150690 0x7f2d0814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 >> 172.21.15.3:0/3879929176 conn(0x7f2d08070310 msgr2=0x7f2d080af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 shutdown_connections 2023-10-06T04:59:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.045+0000 7f2d0eb14700 1 -- 172.21.15.3:0/3879929176 wait complete. 2023-10-06T04:59:08.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:59:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.183+0000 7f06a8f42700 1 Processor -- start 2023-10-06T04:59:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.183+0000 7f06a8f42700 1 -- start start 2023-10-06T04:59:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.184+0000 7f06a8f42700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f06a4142d70 0x7f06a40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.184+0000 7f06a8f42700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06a4143140 0x7f06a4067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.184+0000 7f06a8f42700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a4069a80 con 0x7f06a40cc320 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.184+0000 7f06a8f42700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40cc880 con 0x7f06a4143140 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.184+0000 7f06a8f42700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a4069f10 con 0x7f06a4142d70 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.184+0000 7f06a1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06a4143140 0x7f06a4067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06a4143140 0x7f06a4067ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34380/0 (socket says 172.21.15.3:34380) 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a1d9b700 1 -- 172.21.15.3:0/2279973718 learned_addr learned my addr 172.21.15.3:0/2279973718 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a259c700 1 --2- 172.21.15.3:0/2279973718 >> v2:172.21.15.153:3300/0 conn(0x7f06a4142d70 0x7f06a40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a1d9b700 1 -- 172.21.15.3:0/2279973718 >> v2:172.21.15.153:3300/0 conn(0x7f06a4142d70 msgr2=0x7f06a40cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a1d9b700 1 --2- 172.21.15.3:0/2279973718 >> v2:172.21.15.153:3300/0 conn(0x7f06a4142d70 0x7f06a40cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a1d9b700 1 -- 172.21.15.3:0/2279973718 >> v1:172.21.15.153:6789/0 conn(0x7f06a40cc320 legacy=0x7f06a415d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a1d9b700 1 -- 172.21.15.3:0/2279973718 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4069150 con 0x7f06a4143140 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a259c700 1 --2- 172.21.15.3:0/2279973718 >> v2:172.21.15.153:3300/0 conn(0x7f06a4142d70 0x7f06a40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a1d9b700 1 --2- 172.21.15.3:0/2279973718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a4143140 0x7f06a4067ab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ce170 tx=0x7f0698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6457eada01046e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a159a700 1 -- 172.21.15.3:0/2279973718 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800b820 con 0x7f06a4143140 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a159a700 1 -- 172.21.15.3:0/2279973718 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069800b980 con 0x7f06a4143140 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.185+0000 7f06a159a700 1 -- 172.21.15.3:0/2279973718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800bd20 con 0x7f06a4143140 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 -- 172.21.15.3:0/2279973718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a4143140 msgr2=0x7f06a4067ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 --2- 172.21.15.3:0/2279973718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a4143140 0x7f06a4067ab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ce170 tx=0x7f0698009250 comp rx=0 tx=0).stop 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 -- 172.21.15.3:0/2279973718 shutdown_connections 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 --2- 172.21.15.3:0/2279973718 >> v2:172.21.15.153:3300/0 conn(0x7f06a4142d70 0x7f06a40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 --2- 172.21.15.3:0/2279973718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a4143140 0x7f06a4067ab0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 -- 172.21.15.3:0/2279973718 >> 172.21.15.3:0/2279973718 conn(0x7f06a40ce210 msgr2=0x7f06a40ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 -- 172.21.15.3:0/2279973718 shutdown_connections 2023-10-06T04:59:08.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.186+0000 7f06a8f42700 1 -- 172.21.15.3:0/2279973718 wait complete. 2023-10-06T04:59:08.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.187+0000 7f06a8f42700 1 Processor -- start 2023-10-06T04:59:08.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.187+0000 7f06a8f42700 1 -- start start 2023-10-06T04:59:08.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a40cc320 0x7f06a416a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06a4142d70 0x7f06a4158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06a4143140 0x7f06a40b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a416dc70 con 0x7f06a40cc320 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a416df50 con 0x7f06a4143140 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c53f0 con 0x7f06a4142d70 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a40cc320 0x7f06a416a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a40cc320 0x7f06a416a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34396/0 (socket says 172.21.15.3:34396) 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 -- 172.21.15.3:0/1017776872 learned_addr learned my addr 172.21.15.3:0/1017776872 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a1d9b700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06a4142d70 0x7f06a4158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a2d9d700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06a4143140 0x7f06a40b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:08.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 -- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06a4142d70 msgr2=0x7f06a4158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06a4142d70 0x7f06a4158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 -- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06a4143140 msgr2=0x7f06a40b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06a4143140 0x7f06a40b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 -- 172.21.15.3:0/1017776872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980094f0 con 0x7f06a40cc320 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a2d9d700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06a4143140 0x7f06a40b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a259c700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a40cc320 0x7f06a416a830 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f069400d860 tx=0x7f0694008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06927fc700 1 -- 172.21.15.3:0/1017776872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940125f0 con 0x7f06a40cc320 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06927fc700 1 -- 172.21.15.3:0/1017776872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694012750 con 0x7f06a40cc320 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a40c56d0 con 0x7f06a40cc320 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06927fc700 1 -- 172.21.15.3:0/1017776872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694012ad0 con 0x7f06a40cc320 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.188+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a416d370 con 0x7f06a40cc320 2023-10-06T04:59:08.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.189+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a404c380 con 0x7f06a40cc320 2023-10-06T04:59:08.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.190+0000 7f06927fc700 1 -- 172.21.15.3:0/1017776872 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0694014430 con 0x7f06a40cc320 2023-10-06T04:59:08.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.190+0000 7f06927fc700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:08.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.190+0000 7f06927fc700 1 -- 172.21.15.3:0/1017776872 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0694084ae0 con 0x7f06a40cc320 2023-10-06T04:59:08.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.190+0000 7f06a1d9b700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:08.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.191+0000 7f06a1d9b700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0684066f30 0x7f06840693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f06a4143590 tx=0x7f0698017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:08.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.194+0000 7f06927fc700 1 -- 172.21.15.3:0/1017776872 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0694055680 con 0x7f06a40cc320 2023-10-06T04:59:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:08.435+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f06a40c5a40 con 0x7f06a40cc320 2023-10-06T04:59:09.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.047+0000 7f06927fc700 1 -- 172.21.15.3:0/1017776872 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0694055210 con 0x7f06a40cc320 2023-10-06T04:59:09.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0684066f30 msgr2=0x7f06840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0684066f30 0x7f06840693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f06a4143590 tx=0x7f0698017000 comp rx=0 tx=0).stop 2023-10-06T04:59:09.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a40cc320 msgr2=0x7f06a416a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a40cc320 0x7f06a416a830 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f069400d860 tx=0x7f0694008040 comp rx=0 tx=0).stop 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 shutdown_connections 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06a4143140 0x7f06a40b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.049+0000 7f06a8f42700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a40cc320 0x7f06a416a830 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.050+0000 7f06a8f42700 1 --2- 172.21.15.3:0/1017776872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06a4142d70 0x7f06a4158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.050+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 >> 172.21.15.3:0/1017776872 conn(0x7f06a40ce210 msgr2=0x7f06a413cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.050+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 shutdown_connections 2023-10-06T04:59:09.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.050+0000 7f06a8f42700 1 -- 172.21.15.3:0/1017776872 wait complete. 2023-10-06T04:59:09.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:09.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.187+0000 7f3a12976700 1 Processor -- start 2023-10-06T04:59:09.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.187+0000 7f3a12976700 1 -- start start 2023-10-06T04:59:09.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a12976700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a0c15c8f0 0x7f3a0c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a12976700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15ccc0 0x7f3a0c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a12976700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a0c0d06a0 0x7f3a0c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a12976700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c0cf6c0 con 0x7f3a0c15c8f0 2023-10-06T04:59:09.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a12976700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c0cfb50 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a12976700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c15e050 con 0x7f3a0c15ccc0 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a0c15c8f0 0x7f3a0c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a0c15c8f0 0x7f3a0c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34398/0 (socket says 172.21.15.3:34398) 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 -- 172.21.15.3:0/1823067491 learned_addr learned my addr 172.21.15.3:0/1823067491 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0b7fe700 1 --2- 172.21.15.3:0/1823067491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15ccc0 0x7f3a0c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a10f13700 1 --2- 172.21.15.3:0/1823067491 >> v2:172.21.15.153:3300/0 conn(0x7f3a0c0d06a0 0x7f3a0c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 -- 172.21.15.3:0/1823067491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15ccc0 msgr2=0x7f3a0c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 --2- 172.21.15.3:0/1823067491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15ccc0 0x7f3a0c0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 -- 172.21.15.3:0/1823067491 >> v2:172.21.15.153:3300/0 conn(0x7f3a0c0d06a0 msgr2=0x7f3a0c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 --2- 172.21.15.3:0/1823067491 >> v2:172.21.15.153:3300/0 conn(0x7f3a0c0d06a0 0x7f3a0c06d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 -- 172.21.15.3:0/1823067491 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c0d0af0 con 0x7f3a0c15c8f0 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0bfff700 1 --2- 172.21.15.3:0/1823067491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c15c8f0 0x7f3a0c15db10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0af8a0 tx=0x7f39fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd569d772da0cc83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.188+0000 7f3a0affd700 1 -- 172.21.15.3:0/1823067491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00b860 con 0x7f3a0c15c8f0 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a0affd700 1 -- 172.21.15.3:0/1823067491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc00b9c0 con 0x7f3a0c15c8f0 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a0affd700 1 -- 172.21.15.3:0/1823067491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00bd60 con 0x7f3a0c15c8f0 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 -- 172.21.15.3:0/1823067491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c15c8f0 msgr2=0x7f3a0c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 --2- 172.21.15.3:0/1823067491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c15c8f0 0x7f3a0c15db10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0af8a0 tx=0x7f39fc009250 comp rx=0 tx=0).stop 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 -- 172.21.15.3:0/1823067491 shutdown_connections 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 --2- 172.21.15.3:0/1823067491 >> v2:172.21.15.153:3300/0 conn(0x7f3a0c0d06a0 0x7f3a0c06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 --2- 172.21.15.3:0/1823067491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c15c8f0 0x7f3a0c15db10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 --2- 172.21.15.3:0/1823067491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15ccc0 0x7f3a0c0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 -- 172.21.15.3:0/1823067491 >> 172.21.15.3:0/1823067491 conn(0x7f3a0c070310 msgr2=0x7f3a0c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 -- 172.21.15.3:0/1823067491 shutdown_connections 2023-10-06T04:59:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.189+0000 7f3a12976700 1 -- 172.21.15.3:0/1823067491 wait complete. 2023-10-06T04:59:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.190+0000 7f3a12976700 1 Processor -- start 2023-10-06T04:59:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.190+0000 7f3a12976700 1 -- start start 2023-10-06T04:59:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a12976700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c0d06a0 0x7f3a0c14d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a12976700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15c8f0 0x7f3a0c14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a12976700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a0c15ccc0 0x7f3a0c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a12976700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c14d590 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a12976700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c14cb00 con 0x7f3a0c15ccc0 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a12976700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c14df90 con 0x7f3a0c15c8f0 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c0d06a0 0x7f3a0c14d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c0d06a0 0x7f3a0c14d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34410/0 (socket says 172.21.15.3:34410) 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a0bfff700 1 -- 172.21.15.3:0/3454629669 learned_addr learned my addr 172.21.15.3:0/3454629669 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a0b7fe700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15c8f0 0x7f3a0c14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.191+0000 7f3a10f13700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a0c15ccc0 0x7f3a0c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a0bfff700 1 -- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15c8f0 msgr2=0x7f3a0c14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a0bfff700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15c8f0 0x7f3a0c14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a0bfff700 1 -- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a0c15ccc0 msgr2=0x7f3a0c1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a0bfff700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a0c15ccc0 0x7f3a0c1525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a0bfff700 1 -- 172.21.15.3:0/3454629669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0094f0 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a0bfff700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c0d06a0 0x7f3a0c14d8b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c06eaa0 tx=0x7f39fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f39f3fff700 1 -- 172.21.15.3:0/3454629669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc024440 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f39f3fff700 1 -- 172.21.15.3:0/3454629669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc0245a0 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c14e270 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f39f3fff700 1 -- 172.21.15.3:0/3454629669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc024940 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.192+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c14e830 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.193+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c071680 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.193+0000 7f39f3fff700 1 -- 172.21.15.3:0/3454629669 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f39fc01a420 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.194+0000 7f39f3fff700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39f8066f30 0x7f39f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.194+0000 7f39f3fff700 1 -- 172.21.15.3:0/3454629669 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f39fc086530 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.194+0000 7f3a0b7fe700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39f8066f30 0x7f39f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.194+0000 7f3a0b7fe700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39f8066f30 0x7f39f80693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c155a70 tx=0x7f3a00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.198+0000 7f39f3fff700 1 -- 172.21.15.3:0/3454629669 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39fc0536f0 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.437+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a0c0b35a0 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.437+0000 7f39f3fff700 1 -- 172.21.15.3:0/3454629669 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f39fc056d10 con 0x7f3a0c0d06a0 2023-10-06T04:59:09.440 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:09.440 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:09.043192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:09.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39f8066f30 msgr2=0x7f39f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39f8066f30 0x7f39f80693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c155a70 tx=0x7f3a00008040 comp rx=0 tx=0).stop 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c0d06a0 msgr2=0x7f3a0c14d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c0d06a0 0x7f3a0c14d8b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c06eaa0 tx=0x7f39fc017000 comp rx=0 tx=0).stop 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 shutdown_connections 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39f8066f30 0x7f39f80693e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a0c15ccc0 0x7f3a0c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a0c0d06a0 0x7f3a0c14d8b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 --2- 172.21.15.3:0/3454629669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a0c15c8f0 0x7f3a0c14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 >> 172.21.15.3:0/3454629669 conn(0x7f3a0c070310 msgr2=0x7f3a0c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 shutdown_connections 2023-10-06T04:59:09.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.440+0000 7f3a12976700 1 -- 172.21.15.3:0/3454629669 wait complete. 2023-10-06T04:59:09.444 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 59 2023-10-06T04:59:09.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.571+0000 7f7cb9d6a700 1 Processor -- start 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.571+0000 7f7cb9d6a700 1 -- start start 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb9d6a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb406aa80 0x7f7cb406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb9d6a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7cb406ea70 0x7f7cb4157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb9d6a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb406aed0 con 0x7f7cb4155980 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb9d6a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40ae9e0 con 0x7f7cb406ea70 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb9d6a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb406eef0 con 0x7f7cb406aa80 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7cb4155980 0x7f7cb4156a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34082/0 (socket says 172.21.15.3:34082) 2023-10-06T04:59:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb37fe700 1 -- 172.21.15.3:0/1121429013 learned_addr learned my addr 172.21.15.3:0/1121429013 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb2ffd700 1 --2- 172.21.15.3:0/1121429013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb406aa80 0x7f7cb406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb3fff700 1 --2- 172.21.15.3:0/1121429013 >> v2:172.21.15.153:3300/0 conn(0x7f7cb406ea70 0x7f7cb4157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.572+0000 7f7cb27fc700 1 -- 172.21.15.3:0/1121429013 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680199933 0 0) 0x7f7cb406aed0 con 0x7f7cb4155980 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb27fc700 1 -- 172.21.15.3:0/1121429013 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c9c003570 con 0x7f7cb4155980 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb27fc700 1 -- 172.21.15.3:0/1121429013 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305522400 0 0) 0x7f7c9c003570 con 0x7f7cb4155980 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb2ffd700 1 -- 172.21.15.3:0/1121429013 >> v2:172.21.15.153:3300/0 conn(0x7f7cb406ea70 msgr2=0x7f7cb4157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb2ffd700 1 --2- 172.21.15.3:0/1121429013 >> v2:172.21.15.153:3300/0 conn(0x7f7cb406ea70 0x7f7cb4157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb2ffd700 1 -- 172.21.15.3:0/1121429013 >> v1:172.21.15.3:6789/0 conn(0x7f7cb4155980 legacy=0x7f7cb4156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb2ffd700 1 -- 172.21.15.3:0/1121429013 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb4069790 con 0x7f7cb406aa80 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb3fff700 1 --2- 172.21.15.3:0/1121429013 >> v2:172.21.15.153:3300/0 conn(0x7f7cb406ea70 0x7f7cb4157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb2ffd700 1 --2- 172.21.15.3:0/1121429013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb406aa80 0x7f7cb406e530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80098e0 tx=0x7f7ca80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd64faf36bc42a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb27fc700 1 -- 172.21.15.3:0/1121429013 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8011630 con 0x7f7cb406aa80 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb27fc700 1 -- 172.21.15.3:0/1121429013 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca8011790 con 0x7f7cb406aa80 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.573+0000 7f7cb27fc700 1 -- 172.21.15.3:0/1121429013 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8011b30 con 0x7f7cb406aa80 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/1121429013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb406aa80 msgr2=0x7f7cb406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/1121429013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb406aa80 0x7f7cb406e530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80098e0 tx=0x7f7ca80091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/1121429013 shutdown_connections 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/1121429013 >> v2:172.21.15.153:3300/0 conn(0x7f7cb406ea70 0x7f7cb4157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/1121429013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb406aa80 0x7f7cb406e530 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/1121429013 >> 172.21.15.3:0/1121429013 conn(0x7f7cb406c370 msgr2=0x7f7cb406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/1121429013 shutdown_connections 2023-10-06T04:59:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.574+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/1121429013 wait complete. 2023-10-06T04:59:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.575+0000 7f7cb9d6a700 1 Processor -- start 2023-10-06T04:59:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.575+0000 7f7cb9d6a700 1 -- start start 2023-10-06T04:59:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb9d6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cb406aa80 0x7f7cb414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb9d6a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb406ea70 0x7f7cb40aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb9d6a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb4155980 0x7f7cb40ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb9d6a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4169910 con 0x7f7cb406aa80 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb9d6a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4169b80 con 0x7f7cb406ea70 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb9d6a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4169e60 con 0x7f7cb4155980 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cb406aa80 0x7f7cb414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cb406aa80 0x7f7cb414d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34416/0 (socket says 172.21.15.3:34416) 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 -- 172.21.15.3:0/376910145 learned_addr learned my addr 172.21.15.3:0/376910145 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb3fff700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb4155980 0x7f7cb40ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb2ffd700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb406ea70 0x7f7cb40aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 -- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb4155980 msgr2=0x7f7cb40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb4155980 0x7f7cb40ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 -- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb406ea70 msgr2=0x7f7cb40aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb406ea70 0x7f7cb40aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 -- 172.21.15.3:0/376910145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac009a20 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb37fe700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cb406aa80 0x7f7cb414d210 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c5d0 tx=0x7f7ca4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7c9b7fe700 1 -- 172.21.15.3:0/376910145 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4007360 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7c9b7fe700 1 -- 172.21.15.3:0/376910145 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca40074c0 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca8009420 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.576+0000 7f7c9b7fe700 1 -- 172.21.15.3:0/376910145 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004830 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.577+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb416a420 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.577+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb416a640 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.578+0000 7f7c9b7fe700 1 -- 172.21.15.3:0/376910145 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ca4004be0 con 0x7f7cb406aa80 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.578+0000 7f7c9b7fe700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ca0066f30 0x7f7ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.578+0000 7f7c9b7fe700 1 -- 172.21.15.3:0/376910145 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7ca4084450 con 0x7f7cb406aa80 2023-10-06T04:59:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.579+0000 7f7cb2ffd700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ca0066f30 0x7f7ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.579+0000 7f7cb2ffd700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ca0066f30 0x7f7ca00693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80093a0 tx=0x7f7ca8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.583+0000 7f7c9b7fe700 1 -- 172.21.15.3:0/376910145 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ca40515e0 con 0x7f7cb406aa80 2023-10-06T04:59:09.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.776+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cb4071680 con 0x7f7cb406aa80 2023-10-06T04:59:09.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.777+0000 7f7c9b7fe700 1 -- 172.21.15.3:0/376910145 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f7ca4054c00 con 0x7f7cb406aa80 2023-10-06T04:59:09.780 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:09.780 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:07.019162+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:03.167920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:04.391498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ca0066f30 msgr2=0x7f7ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ca0066f30 0x7f7ca00693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80093a0 tx=0x7f7ca8017000 comp rx=0 tx=0).stop 2023-10-06T04:59:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cb406aa80 msgr2=0x7f7cb414d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cb406aa80 0x7f7cb414d210 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c5d0 tx=0x7f7ca4007df0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 shutdown_connections 2023-10-06T04:59:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ca0066f30 0x7f7ca00693e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cb4155980 0x7f7cb40ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cb406aa80 0x7f7cb414d210 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 --2- 172.21.15.3:0/376910145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb406ea70 0x7f7cb40aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 >> 172.21.15.3:0/376910145 conn(0x7f7cb406c370 msgr2=0x7f7cb4154bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 shutdown_connections 2023-10-06T04:59:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.779+0000 7f7cb9d6a700 1 -- 172.21.15.3:0/376910145 wait complete. 2023-10-06T04:59:09.794 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:59:09.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:59:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.911+0000 7fb73c31f700 1 Processor -- start 2023-10-06T04:59:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.911+0000 7fb73c31f700 1 -- start start 2023-10-06T04:59:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb73c31f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb734157400 0x7fb7340b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb73c31f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb7340b3b70 0x7fb73406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb73c31f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73415aeb0 con 0x7fb734157030 2023-10-06T04:59:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb73c31f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73415a7a0 con 0x7fb734157400 2023-10-06T04:59:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb73c31f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73415aa80 con 0x7fb7340b3b70 2023-10-06T04:59:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb734157400 0x7fb7340b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb734157400 0x7fb7340b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34418/0 (socket says 172.21.15.3:34418) 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 -- 172.21.15.3:0/1097162837 learned_addr learned my addr 172.21.15.3:0/1097162837 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb73a8bc700 1 --2- 172.21.15.3:0/1097162837 >> v2:172.21.15.153:3300/0 conn(0x7fb7340b3b70 0x7fb73406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 -- 172.21.15.3:0/1097162837 >> v2:172.21.15.153:3300/0 conn(0x7fb7340b3b70 msgr2=0x7fb73406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 --2- 172.21.15.3:0/1097162837 >> v2:172.21.15.153:3300/0 conn(0x7fb7340b3b70 0x7fb73406e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 -- 172.21.15.3:0/1097162837 >> v1:172.21.15.153:6789/0 conn(0x7fb734157030 legacy=0x7fb734159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 -- 172.21.15.3:0/1097162837 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73406dba0 con 0x7fb734157400 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7398ba700 1 --2- 172.21.15.3:0/1097162837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157400 0x7fb7340b3630 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb7340687c0 tx=0x7fb724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982e7fab48c811b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.912+0000 7fb7390b9700 1 -- 172.21.15.3:0/1097162837 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400b820 con 0x7fb734157400 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb7390b9700 1 -- 172.21.15.3:0/1097162837 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400b980 con 0x7fb734157400 2023-10-06T04:59:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb7390b9700 1 -- 172.21.15.3:0/1097162837 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400bd20 con 0x7fb734157400 2023-10-06T04:59:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 -- 172.21.15.3:0/1097162837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157400 msgr2=0x7fb7340b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 --2- 172.21.15.3:0/1097162837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157400 0x7fb7340b3630 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb7340687c0 tx=0x7fb724009250 comp rx=0 tx=0).stop 2023-10-06T04:59:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 -- 172.21.15.3:0/1097162837 shutdown_connections 2023-10-06T04:59:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 --2- 172.21.15.3:0/1097162837 >> v2:172.21.15.153:3300/0 conn(0x7fb7340b3b70 0x7fb73406e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 --2- 172.21.15.3:0/1097162837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157400 0x7fb7340b3630 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 -- 172.21.15.3:0/1097162837 >> 172.21.15.3:0/1097162837 conn(0x7fb7340ce270 msgr2=0x7fb7340ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 -- 172.21.15.3:0/1097162837 shutdown_connections 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.913+0000 7fb73c31f700 1 -- 172.21.15.3:0/1097162837 wait complete. 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.914+0000 7fb73c31f700 1 Processor -- start 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.914+0000 7fb73c31f700 1 -- start start 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73c31f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7340b3b70 0x7fb7340cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73c31f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157030 0x7fb7340b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73c31f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb734157400 0x7fb734151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73c31f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73415f0a0 con 0x7fb734157030 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73c31f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734159780 con 0x7fb7340b3b70 2023-10-06T04:59:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73c31f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb734158bb0 con 0x7fb734157400 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157030 0x7fb7340b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157030 0x7fb7340b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34434/0 (socket says 172.21.15.3:34434) 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 -- 172.21.15.3:0/2579228358 learned_addr learned my addr 172.21.15.3:0/2579228358 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73a0bb700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7340b3b70 0x7fb7340cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb73a8bc700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb734157400 0x7fb734151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 -- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb734157400 msgr2=0x7fb734151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb734157400 0x7fb734151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 -- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7340b3b70 msgr2=0x7fb7340cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7340b3b70 0x7fb7340cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.915+0000 7fb7398ba700 1 -- 172.21.15.3:0/2579228358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240094f0 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.916+0000 7fb7398ba700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157030 0x7fb7340b06a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb724009290 tx=0x7fb724016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.916+0000 7fb72a7fc700 1 -- 172.21.15.3:0/2579228358 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724011d70 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.916+0000 7fb72a7fc700 1 -- 172.21.15.3:0/2579228358 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb724020070 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.916+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb734158e50 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.916+0000 7fb72a7fc700 1 -- 172.21.15.3:0/2579228358 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72401b6a0 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.916+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb734159410 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.917+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb734159930 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.917+0000 7fb72a7fc700 1 -- 172.21.15.3:0/2579228358 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb724022030 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.918+0000 7fb72a7fc700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb718066f30 0x7fb7180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.918+0000 7fb72a7fc700 1 -- 172.21.15.3:0/2579228358 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb724086420 con 0x7fb734157030 2023-10-06T04:59:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.918+0000 7fb73a0bb700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb718066f30 0x7fb7180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.918+0000 7fb73a0bb700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb718066f30 0x7fb7180693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb730006fd0 tx=0x7fb7300090f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:09.922+0000 7fb72a7fc700 1 -- 172.21.15.3:0/2579228358 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb734159930 con 0x7fb734157030 2023-10-06T04:59:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.114+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7340644c0 con 0x7fb718066f30 2023-10-06T04:59:10.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.124+0000 7fb72a7fc700 1 -- 172.21.15.3:0/2579228358 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308708 (secure 0 0 0) 0x7fb7340644c0 con 0x7fb718066f30 2023-10-06T04:59:10.130 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb718066f30 msgr2=0x7fb7180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb718066f30 0x7fb7180693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb730006fd0 tx=0x7fb7300090f0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157030 msgr2=0x7fb7340b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157030 0x7fb7340b06a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb724009290 tx=0x7fb724016000 comp rx=0 tx=0).stop 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 shutdown_connections 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb718066f30 0x7fb7180693e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7340b3b70 0x7fb7340cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb734157400 0x7fb734151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 --2- 172.21.15.3:0/2579228358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734157030 0x7fb7340b06a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.127+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 >> 172.21.15.3:0/2579228358 conn(0x7fb7340ce270 msgr2=0x7fb734157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.128+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 shutdown_connections 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.128+0000 7fb73c31f700 1 -- 172.21.15.3:0/2579228358 wait complete. 2023-10-06T04:59:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:59:10.142 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-06T04:59:09.524507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445824,"kb_used_data":10176,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752528896,"statfs":{"total":773094113280,"available":770589589504,"internally_reserved":0,"allocated":10420224,"data_stored":5321181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.518620"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.021652+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:40.965728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.018066+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:31.156638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017806+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:55.185806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016696+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.104668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016366+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:24.260605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014934+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:49.295778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.019752+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:36.653367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.019396+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:27.219788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017920+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:44.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014935+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:01.732144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016919+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011338+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:13.415770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014693+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016598+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.018596+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017837+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015755+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:26.205356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015002+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:58.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012429+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:41.792782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.010492+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:49.048000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014331+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:02.587278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013637+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013020+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:36.271837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015735+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011571+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:21.883224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012207+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:22.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016324+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:58.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015319+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:10.609306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013796+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:07.976219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011941+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:59.866311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015126+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:22.404213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016613+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:01.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012301+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:17.016243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030540+0000","last_change":"2023-10-06T04:59:05.030540+0000","last_active":"2023-10-06T04:59:05.030540+0000","last_peered":"2023-10-06T04:59:05.030540+0000","last_clean":"2023-10-06T04:59:05.030540+0000","last_became_active":"2023-10-06T04:59:05.030260+0000","last_became_peered":"2023-10-06T04:59:05.030260+0000","last_unstale":"2023-10-06T04:59:05.030540+0000","last_undegraded":"2023-10-06T04:59:05.030540+0000","last_fullsized":"2023-10-06T04:59:05.030540+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012428+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:23.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026990+0000","last_change":"2023-10-06T04:59:05.026990+0000","last_active":"2023-10-06T04:59:05.026990+0000","last_peered":"2023-10-06T04:59:05.026990+0000","last_clean":"2023-10-06T04:59:05.026990+0000","last_became_active":"2023-10-06T04:59:05.026693+0000","last_became_peered":"2023-10-06T04:59:05.026693+0000","last_unstale":"2023-10-06T04:59:05.026990+0000","last_undegraded":"2023-10-06T04:59:05.026990+0000","last_fullsized":"2023-10-06T04:59:05.026990+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:49.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011347+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:28.217935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021204+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:05.021204+0000","last_peered":"2023-10-06T04:59:05.021204+0000","last_clean":"2023-10-06T04:59:05.021204+0000","last_became_active":"2023-10-06T04:59:05.020908+0000","last_became_peered":"2023-10-06T04:59:05.020908+0000","last_unstale":"2023-10-06T04:59:05.021204+0000","last_undegraded":"2023-10-06T04:59:05.021204+0000","last_fullsized":"2023-10-06T04:59:05.021204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.255698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014483+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:19.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023617+0000","last_change":"2023-10-06T04:59:05.023617+0000","last_active":"2023-10-06T04:59:05.023617+0000","last_peered":"2023-10-06T04:59:05.023617+0000","last_clean":"2023-10-06T04:59:05.023617+0000","last_became_active":"2023-10-06T04:59:05.023394+0000","last_became_peered":"2023-10-06T04:59:05.023394+0000","last_unstale":"2023-10-06T04:59:05.023617+0000","last_undegraded":"2023-10-06T04:59:05.023617+0000","last_fullsized":"2023-10-06T04:59:05.023617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.562815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014939+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030809+0000","last_change":"2023-10-06T04:59:05.030809+0000","last_active":"2023-10-06T04:59:05.030809+0000","last_peered":"2023-10-06T04:59:05.030809+0000","last_clean":"2023-10-06T04:59:05.030809+0000","last_became_active":"2023-10-06T04:59:05.030550+0000","last_became_peered":"2023-10-06T04:59:05.030550+0000","last_unstale":"2023-10-06T04:59:05.030809+0000","last_undegraded":"2023-10-06T04:59:05.030809+0000","last_fullsized":"2023-10-06T04:59:05.030809+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011439+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:19.436490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024000+0000","last_change":"2023-10-06T04:59:05.024000+0000","last_active":"2023-10-06T04:59:05.024000+0000","last_peered":"2023-10-06T04:59:05.024000+0000","last_clean":"2023-10-06T04:59:05.024000+0000","last_became_active":"2023-10-06T04:59:05.023706+0000","last_became_peered":"2023-10-06T04:59:05.023706+0000","last_unstale":"2023-10-06T04:59:05.024000+0000","last_undegraded":"2023-10-06T04:59:05.024000+0000","last_fullsized":"2023-10-06T04:59:05.024000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014554+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:50:43.443720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021204+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:05.021204+0000","last_peered":"2023-10-06T04:59:05.021204+0000","last_clean":"2023-10-06T04:59:05.021204+0000","last_became_active":"2023-10-06T04:59:05.020967+0000","last_became_peered":"2023-10-06T04:59:05.020967+0000","last_unstale":"2023-10-06T04:59:05.021204+0000","last_undegraded":"2023-10-06T04:59:05.021204+0000","last_fullsized":"2023-10-06T04:59:05.021204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:54.137958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016972+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025368+0000","last_change":"2023-10-06T04:59:05.025368+0000","last_active":"2023-10-06T04:59:05.025368+0000","last_peered":"2023-10-06T04:59:05.025368+0000","last_clean":"2023-10-06T04:59:05.025368+0000","last_became_active":"2023-10-06T04:59:05.025101+0000","last_became_peered":"2023-10-06T04:59:05.025101+0000","last_unstale":"2023-10-06T04:59:05.025368+0000","last_undegraded":"2023-10-06T04:59:05.025368+0000","last_fullsized":"2023-10-06T04:59:05.025368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012737+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.571901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021703+0000","last_change":"2023-10-06T04:59:05.021703+0000","last_active":"2023-10-06T04:59:05.021703+0000","last_peered":"2023-10-06T04:59:05.021703+0000","last_clean":"2023-10-06T04:59:05.021703+0000","last_became_active":"2023-10-06T04:59:05.021413+0000","last_became_peered":"2023-10-06T04:59:05.021413+0000","last_unstale":"2023-10-06T04:59:05.021703+0000","last_undegraded":"2023-10-06T04:59:05.021703+0000","last_fullsized":"2023-10-06T04:59:05.021703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:15.699249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013498+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.016394+0000","last_change":"2023-10-06T04:59:05.016394+0000","last_active":"2023-10-06T04:59:05.016394+0000","last_peered":"2023-10-06T04:59:05.016394+0000","last_clean":"2023-10-06T04:59:05.016394+0000","last_became_active":"2023-10-06T04:59:05.016106+0000","last_became_peered":"2023-10-06T04:59:05.016106+0000","last_unstale":"2023-10-06T04:59:05.016394+0000","last_undegraded":"2023-10-06T04:59:05.016394+0000","last_fullsized":"2023-10-06T04:59:05.016394+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:10.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014320+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:50.435166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026950+0000","last_change":"2023-10-06T04:59:05.026950+0000","last_active":"2023-10-06T04:59:05.026950+0000","last_peered":"2023-10-06T04:59:05.026950+0000","last_clean":"2023-10-06T04:59:05.026950+0000","last_became_active":"2023-10-06T04:59:05.026723+0000","last_became_peered":"2023-10-06T04:59:05.026723+0000","last_unstale":"2023-10-06T04:59:05.026950+0000","last_undegraded":"2023-10-06T04:59:05.026950+0000","last_fullsized":"2023-10-06T04:59:05.026950+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014381+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025160+0000","last_change":"2023-10-06T04:59:05.025160+0000","last_active":"2023-10-06T04:59:05.025160+0000","last_peered":"2023-10-06T04:59:05.025160+0000","last_clean":"2023-10-06T04:59:05.025160+0000","last_became_active":"2023-10-06T04:59:05.024909+0000","last_became_peered":"2023-10-06T04:59:05.024909+0000","last_unstale":"2023-10-06T04:59:05.025160+0000","last_undegraded":"2023-10-06T04:59:05.025160+0000","last_fullsized":"2023-10-06T04:59:05.025160+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:00.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016608+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023984+0000","last_change":"2023-10-06T04:59:05.023984+0000","last_active":"2023-10-06T04:59:05.023984+0000","last_peered":"2023-10-06T04:59:05.023984+0000","last_clean":"2023-10-06T04:59:05.023984+0000","last_became_active":"2023-10-06T04:59:05.023641+0000","last_became_peered":"2023-10-06T04:59:05.023641+0000","last_unstale":"2023-10-06T04:59:05.023984+0000","last_undegraded":"2023-10-06T04:59:05.023984+0000","last_fullsized":"2023-10-06T04:59:05.023984+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:54.810243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015507+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.531534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025232+0000","last_change":"2023-10-06T04:59:05.025232+0000","last_active":"2023-10-06T04:59:05.025232+0000","last_peered":"2023-10-06T04:59:05.025232+0000","last_clean":"2023-10-06T04:59:05.025232+0000","last_became_active":"2023-10-06T04:59:05.024850+0000","last_became_peered":"2023-10-06T04:59:05.024850+0000","last_unstale":"2023-10-06T04:59:05.025232+0000","last_undegraded":"2023-10-06T04:59:05.025232+0000","last_fullsized":"2023-10-06T04:59:05.025232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012744+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:58.844389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.020554+0000","last_change":"2023-10-06T04:59:05.020554+0000","last_active":"2023-10-06T04:59:05.020554+0000","last_peered":"2023-10-06T04:59:05.020554+0000","last_clean":"2023-10-06T04:59:05.020554+0000","last_became_active":"2023-10-06T04:59:05.020294+0000","last_became_peered":"2023-10-06T04:59:05.020294+0000","last_unstale":"2023-10-06T04:59:05.020554+0000","last_undegraded":"2023-10-06T04:59:05.020554+0000","last_fullsized":"2023-10-06T04:59:05.020554+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:24.547416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011224+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.022660+0000","last_change":"2023-10-06T04:59:05.022660+0000","last_active":"2023-10-06T04:59:05.022660+0000","last_peered":"2023-10-06T04:59:05.022660+0000","last_clean":"2023-10-06T04:59:05.022660+0000","last_became_active":"2023-10-06T04:59:05.021894+0000","last_became_peered":"2023-10-06T04:59:05.021894+0000","last_unstale":"2023-10-06T04:59:05.022660+0000","last_undegraded":"2023-10-06T04:59:05.022660+0000","last_fullsized":"2023-10-06T04:59:05.022660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.654900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014857+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:41.853665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.022693+0000","last_change":"2023-10-06T04:59:05.022693+0000","last_active":"2023-10-06T04:59:05.022693+0000","last_peered":"2023-10-06T04:59:05.022693+0000","last_clean":"2023-10-06T04:59:05.022693+0000","last_became_active":"2023-10-06T04:59:05.022421+0000","last_became_peered":"2023-10-06T04:59:05.022421+0000","last_unstale":"2023-10-06T04:59:05.022693+0000","last_undegraded":"2023-10-06T04:59:05.022693+0000","last_fullsized":"2023-10-06T04:59:05.022693+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:59.701496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.010770+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:47.264143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024903+0000","last_change":"2023-10-06T04:59:05.024903+0000","last_active":"2023-10-06T04:59:05.024903+0000","last_peered":"2023-10-06T04:59:05.024903+0000","last_clean":"2023-10-06T04:59:05.024903+0000","last_became_active":"2023-10-06T04:59:05.024608+0000","last_became_peered":"2023-10-06T04:59:05.024608+0000","last_unstale":"2023-10-06T04:59:05.024903+0000","last_undegraded":"2023-10-06T04:59:05.024903+0000","last_fullsized":"2023-10-06T04:59:05.024903+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:48.099761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016917+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:11.903244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030722+0000","last_change":"2023-10-06T04:59:05.030722+0000","last_active":"2023-10-06T04:59:05.030722+0000","last_peered":"2023-10-06T04:59:05.030722+0000","last_clean":"2023-10-06T04:59:05.030722+0000","last_became_active":"2023-10-06T04:59:05.030276+0000","last_became_peered":"2023-10-06T04:59:05.030276+0000","last_unstale":"2023-10-06T04:59:05.030722+0000","last_undegraded":"2023-10-06T04:59:05.030722+0000","last_fullsized":"2023-10-06T04:59:05.030722+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:57.328563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017462+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024303+0000","last_change":"2023-10-06T04:59:05.024303+0000","last_active":"2023-10-06T04:59:05.024303+0000","last_peered":"2023-10-06T04:59:05.024303+0000","last_clean":"2023-10-06T04:59:05.024303+0000","last_became_active":"2023-10-06T04:59:05.024038+0000","last_became_peered":"2023-10-06T04:59:05.024038+0000","last_unstale":"2023-10-06T04:59:05.024303+0000","last_undegraded":"2023-10-06T04:59:05.024303+0000","last_fullsized":"2023-10-06T04:59:05.024303+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012621+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:05.658324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023856+0000","last_change":"2023-10-06T04:59:05.023856+0000","last_active":"2023-10-06T04:59:05.023856+0000","last_peered":"2023-10-06T04:59:05.023856+0000","last_clean":"2023-10-06T04:59:05.023856+0000","last_became_active":"2023-10-06T04:59:05.023612+0000","last_became_peered":"2023-10-06T04:59:05.023612+0000","last_unstale":"2023-10-06T04:59:05.023856+0000","last_undegraded":"2023-10-06T04:59:05.023856+0000","last_fullsized":"2023-10-06T04:59:05.023856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:04.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012763+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:58.906683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023970+0000","last_change":"2023-10-06T04:59:05.023970+0000","last_active":"2023-10-06T04:59:05.023970+0000","last_peered":"2023-10-06T04:59:05.023970+0000","last_clean":"2023-10-06T04:59:05.023970+0000","last_became_active":"2023-10-06T04:59:05.023712+0000","last_became_peered":"2023-10-06T04:59:05.023712+0000","last_unstale":"2023-10-06T04:59:05.023970+0000","last_undegraded":"2023-10-06T04:59:05.023970+0000","last_fullsized":"2023-10-06T04:59:05.023970+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:31.733624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017238+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028495+0000","last_change":"2023-10-06T04:59:05.028495+0000","last_active":"2023-10-06T04:59:05.028495+0000","last_peered":"2023-10-06T04:59:05.028495+0000","last_clean":"2023-10-06T04:59:05.028495+0000","last_became_active":"2023-10-06T04:59:05.028219+0000","last_became_peered":"2023-10-06T04:59:05.028219+0000","last_unstale":"2023-10-06T04:59:05.028495+0000","last_undegraded":"2023-10-06T04:59:05.028495+0000","last_fullsized":"2023-10-06T04:59:05.028495+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:54.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016324+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023535+0000","last_change":"2023-10-06T04:59:05.023535+0000","last_active":"2023-10-06T04:59:05.023535+0000","last_peered":"2023-10-06T04:59:05.023535+0000","last_clean":"2023-10-06T04:59:05.023535+0000","last_became_active":"2023-10-06T04:59:05.023261+0000","last_became_peered":"2023-10-06T04:59:05.023261+0000","last_unstale":"2023-10-06T04:59:05.023535+0000","last_undegraded":"2023-10-06T04:59:05.023535+0000","last_fullsized":"2023-10-06T04:59:05.023535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:47.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011797+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.065649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026906+0000","last_change":"2023-10-06T04:59:05.026906+0000","last_active":"2023-10-06T04:59:05.026906+0000","last_peered":"2023-10-06T04:59:05.026906+0000","last_clean":"2023-10-06T04:59:05.026906+0000","last_became_active":"2023-10-06T04:59:05.026633+0000","last_became_peered":"2023-10-06T04:59:05.026633+0000","last_unstale":"2023-10-06T04:59:05.026906+0000","last_undegraded":"2023-10-06T04:59:05.026906+0000","last_fullsized":"2023-10-06T04:59:05.026906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:56.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.013419+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:59:04.013419+0000","last_peered":"2023-10-06T04:59:04.013419+0000","last_clean":"2023-10-06T04:59:04.013419+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:59:04.013419+0000","last_undegraded":"2023-10-06T04:59:04.013419+0000","last_fullsized":"2023-10-06T04:59:04.013419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011969+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:31.872736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028380+0000","last_change":"2023-10-06T04:59:05.028380+0000","last_active":"2023-10-06T04:59:05.028380+0000","last_peered":"2023-10-06T04:59:05.028380+0000","last_clean":"2023-10-06T04:59:05.028380+0000","last_became_active":"2023-10-06T04:59:05.028098+0000","last_became_peered":"2023-10-06T04:59:05.028098+0000","last_unstale":"2023-10-06T04:59:05.028380+0000","last_undegraded":"2023-10-06T04:59:05.028380+0000","last_fullsized":"2023-10-06T04:59:05.028380+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:22.276266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.007290+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:59:04.007290+0000","last_peered":"2023-10-06T04:59:04.007290+0000","last_clean":"2023-10-06T04:59:04.007290+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:59:04.007290+0000","last_undegraded":"2023-10-06T04:59:04.007290+0000","last_fullsized":"2023-10-06T04:59:04.007290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014304+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:08.338740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024110+0000","last_change":"2023-10-06T04:59:05.024110+0000","last_active":"2023-10-06T04:59:05.024110+0000","last_peered":"2023-10-06T04:59:05.024110+0000","last_clean":"2023-10-06T04:59:05.024110+0000","last_became_active":"2023-10-06T04:59:05.023840+0000","last_became_peered":"2023-10-06T04:59:05.023840+0000","last_unstale":"2023-10-06T04:59:05.024110+0000","last_undegraded":"2023-10-06T04:59:05.024110+0000","last_fullsized":"2023-10-06T04:59:05.024110+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.014748+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:59:04.014748+0000","last_peered":"2023-10-06T04:59:04.014748+0000","last_clean":"2023-10-06T04:59:04.014748+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:59:04.014748+0000","last_undegraded":"2023-10-06T04:59:04.014748+0000","last_fullsized":"2023-10-06T04:59:04.014748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":222,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010191+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:59:04.010191+0000","last_peered":"2023-10-06T04:59:04.010191+0000","last_clean":"2023-10-06T04:59:04.010191+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:59:04.010191+0000","last_undegraded":"2023-10-06T04:59:04.010191+0000","last_fullsized":"2023-10-06T04:59:04.010191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015895+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:16.704156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030905+0000","last_change":"2023-10-06T04:59:05.030905+0000","last_active":"2023-10-06T04:59:05.030905+0000","last_peered":"2023-10-06T04:59:05.030905+0000","last_clean":"2023-10-06T04:59:05.030905+0000","last_became_active":"2023-10-06T04:59:05.030530+0000","last_became_peered":"2023-10-06T04:59:05.030530+0000","last_unstale":"2023-10-06T04:59:05.030905+0000","last_undegraded":"2023-10-06T04:59:05.030905+0000","last_fullsized":"2023-10-06T04:59:05.030905+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:15.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":72,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010382+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:59:04.010382+0000","last_peered":"2023-10-06T04:59:04.010382+0000","last_clean":"2023-10-06T04:59:04.010382+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:59:04.010382+0000","last_undegraded":"2023-10-06T04:59:04.010382+0000","last_fullsized":"2023-10-06T04:59:04.010382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016145+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.815150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028526+0000","last_change":"2023-10-06T04:59:05.028526+0000","last_active":"2023-10-06T04:59:05.028526+0000","last_peered":"2023-10-06T04:59:05.028526+0000","last_clean":"2023-10-06T04:59:05.028526+0000","last_became_active":"2023-10-06T04:59:05.028134+0000","last_became_peered":"2023-10-06T04:59:05.028134+0000","last_unstale":"2023-10-06T04:59:05.028526+0000","last_undegraded":"2023-10-06T04:59:05.028526+0000","last_fullsized":"2023-10-06T04:59:05.028526+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:16.407534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010554+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:59:04.010554+0000","last_peered":"2023-10-06T04:59:04.010554+0000","last_clean":"2023-10-06T04:59:04.010554+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:59:04.010554+0000","last_undegraded":"2023-10-06T04:59:04.010554+0000","last_fullsized":"2023-10-06T04:59:04.010554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016082+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:58.333470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023805+0000","last_change":"2023-10-06T04:59:05.023805+0000","last_active":"2023-10-06T04:59:05.023805+0000","last_peered":"2023-10-06T04:59:05.023805+0000","last_clean":"2023-10-06T04:59:05.023805+0000","last_became_active":"2023-10-06T04:59:05.023560+0000","last_became_peered":"2023-10-06T04:59:05.023560+0000","last_unstale":"2023-10-06T04:59:05.023805+0000","last_undegraded":"2023-10-06T04:59:05.023805+0000","last_fullsized":"2023-10-06T04:59:05.023805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:16.588598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010472+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:59:04.010472+0000","last_peered":"2023-10-06T04:59:04.010472+0000","last_clean":"2023-10-06T04:59:04.010472+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:59:04.010472+0000","last_undegraded":"2023-10-06T04:59:04.010472+0000","last_fullsized":"2023-10-06T04:59:04.010472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013631+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:38.572727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.020425+0000","last_change":"2023-10-06T04:59:05.020425+0000","last_active":"2023-10-06T04:59:05.020425+0000","last_peered":"2023-10-06T04:59:05.020425+0000","last_clean":"2023-10-06T04:59:05.020425+0000","last_became_active":"2023-10-06T04:59:05.020172+0000","last_became_peered":"2023-10-06T04:59:05.020172+0000","last_unstale":"2023-10-06T04:59:05.020425+0000","last_undegraded":"2023-10-06T04:59:05.020425+0000","last_fullsized":"2023-10-06T04:59:05.020425+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:25.681695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.011548+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:59:04.011548+0000","last_peered":"2023-10-06T04:59:04.011548+0000","last_clean":"2023-10-06T04:59:04.011548+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:59:04.011548+0000","last_undegraded":"2023-10-06T04:59:04.011548+0000","last_fullsized":"2023-10-06T04:59:04.011548+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014748+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028402+0000","last_change":"2023-10-06T04:59:05.028402+0000","last_active":"2023-10-06T04:59:05.028402+0000","last_peered":"2023-10-06T04:59:05.028402+0000","last_clean":"2023-10-06T04:59:05.028402+0000","last_became_active":"2023-10-06T04:59:05.028139+0000","last_became_peered":"2023-10-06T04:59:05.028139+0000","last_unstale":"2023-10-06T04:59:05.028402+0000","last_undegraded":"2023-10-06T04:59:05.028402+0000","last_fullsized":"2023-10-06T04:59:05.028402+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:56.970617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.011518+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:59:04.011518+0000","last_peered":"2023-10-06T04:59:04.011518+0000","last_clean":"2023-10-06T04:59:04.011518+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:59:04.011518+0000","last_undegraded":"2023-10-06T04:59:04.011518+0000","last_fullsized":"2023-10-06T04:59:04.011518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":1104597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.73399999999999999},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73199999999999998,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.495}]},{"osd":1,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":1.073},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.51600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":1.111},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.58999999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.503},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.441}]},{"osd":5,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.502},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":1.214},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302520,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069320,"statfs":{"total":96636764160,"available":96326983680,"internally_reserved":0,"allocated":1163264,"data_stored":522198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.52200000000000002},"min":{"1min":0.54900000000000004,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.81200000000000006,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.57199999999999995},"min":{"1min":0.56100000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.39800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":1.234,"15min":1.234},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.55200000000000005},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81000000000000005,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75800000000000001,"5min":3.669,"15min":3.669},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.495},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":1104597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.23000000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.27900000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.376}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.501,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.316},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.70999999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.85499999999999998,"15min":0.92900000000000005},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.64100000000000001},"min":{"1min":0.47499999999999998,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.998,"15min":0.998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.65800000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":1.0900000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80600000000000005,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.026},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.79400000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.505,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57699999999999996,"5min":0.628,"15min":0.628},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.441}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90700000000000003,"5min":1.022,"15min":1.022},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.495,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.5,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96599999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.995,"15min":0.995},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302520,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069320,"statfs":{"total":96636764160,"available":96326983680,"internally_reserved":0,"allocated":1163264,"data_stored":522198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.51200000000000001,"5min":0.747,"15min":1.048},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.42599999999999999,"15min":0.39000000000000001},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.375}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":0.61699999999999999,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.309},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.36099999999999999},"min":{"1min":0.29499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.439}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.5,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38700000000000001,"15min":0.40600000000000003},"min":{"1min":0.251,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":2.008,"15min":2.008},"last":0.316},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41299999999999998,"15min":0.56499999999999995},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54500000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42399999999999999,"15min":0.46000000000000002},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.64600000000000002,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":1.379,"15min":5.069},"last":0.443},{"interface":"front","average":{"1min":0.443,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":2.246,"15min":2.246},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83199999999999996,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82199999999999995,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.79400000000000004},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.40400000000000003},"max":{"1min":0.76200000000000001,"5min":1.964,"15min":6.367},"last":0.623},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65000000000000002,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":1.3,"15min":5},"last":0.754}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.379},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.78500000000000003},"min":{"1min":0.38200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.82099999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.80400000000000005,"15min":5.415},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.66700000000000004},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":4.202},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":1.044,"15min":4.2130000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40699999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.42499999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.77600000000000002,"5min":0.86199999999999999,"15min":0.872},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.313},"max":{"1min":0.84399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306612,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065228,"statfs":{"total":96636764160,"available":96322793472,"internally_reserved":0,"allocated":1159168,"data_stored":520845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.67600000000000005},"min":{"1min":0.36899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.625},"min":{"1min":0.436,"5min":0.247,"15min":0.247},"max":{"1min":0.76000000000000001,"5min":0.80800000000000005,"15min":3.7679999999999998},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.70799999999999996},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":6.7949999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.75800000000000001},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.93799999999999994},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":9.7210000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.93500000000000005},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":9.7539999999999996},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.504},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.34200000000000003},"max":{"1min":0.66300000000000003,"5min":0.746,"15min":0.746},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80100000000000005},"last":0.46200000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:59:10.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:59:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.299+0000 7fce4a4ef700 1 Processor -- start 2023-10-06T04:59:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.299+0000 7fce4a4ef700 1 -- start start 2023-10-06T04:59:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce4a4ef700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fce440b2530 0x7fce440b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce4a4ef700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce44159080 0x7fce44159500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce4a4ef700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4415e0d0 con 0x7fce440b2900 2023-10-06T04:59:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce4a4ef700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce440b0e00 con 0x7fce440b2530 2023-10-06T04:59:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce4a4ef700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440cbd70 con 0x7fce44159080 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce43fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fce440b2530 0x7fce440b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce48a8c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce44159080 0x7fce44159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce437fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fce440b2900 0x7fce4415d9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34578/0 (socket says 172.21.15.3:34578) 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.300+0000 7fce437fe700 1 -- 172.21.15.3:0/1593511804 learned_addr learned my addr 172.21.15.3:0/1593511804 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce48a8c700 1 -- 172.21.15.3:0/1593511804 >> v2:172.21.15.153:3300/0 conn(0x7fce440b2530 msgr2=0x7fce440b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce48a8c700 1 --2- 172.21.15.3:0/1593511804 >> v2:172.21.15.153:3300/0 conn(0x7fce440b2530 0x7fce440b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce48a8c700 1 -- 172.21.15.3:0/1593511804 >> v1:172.21.15.153:6789/0 conn(0x7fce440b2900 legacy=0x7fce4415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce42ffd700 1 -- 172.21.15.3:0/1593511804 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145976740 0 0) 0x7fce38001110 con 0x7fce440b2900 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce48a8c700 1 -- 172.21.15.3:0/1593511804 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440cb160 con 0x7fce44159080 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce43fff700 1 --2- 172.21.15.3:0/1593511804 >> v2:172.21.15.153:3300/0 conn(0x7fce440b2530 0x7fce440b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce48a8c700 1 --2- 172.21.15.3:0/1593511804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce44159080 0x7fce44159500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009e60 tx=0x7fce3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be09e9f2819787b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce42ffd700 1 -- 172.21.15.3:0/1593511804 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00e070 con 0x7fce44159080 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce42ffd700 1 -- 172.21.15.3:0/1593511804 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c00bb60 con 0x7fce44159080 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.301+0000 7fce42ffd700 1 -- 172.21.15.3:0/1593511804 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c012430 con 0x7fce44159080 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1593511804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce44159080 msgr2=0x7fce44159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1593511804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce44159080 0x7fce44159500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009e60 tx=0x7fce3c009780 comp rx=0 tx=0).stop 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1593511804 shutdown_connections 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1593511804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce44159080 0x7fce44159500 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1593511804 >> v2:172.21.15.153:3300/0 conn(0x7fce440b2530 0x7fce440b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1593511804 >> 172.21.15.3:0/1593511804 conn(0x7fce44070310 msgr2=0x7fce44070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1593511804 shutdown_connections 2023-10-06T04:59:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.302+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1593511804 wait complete. 2023-10-06T04:59:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.303+0000 7fce4a4ef700 1 Processor -- start 2023-10-06T04:59:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.303+0000 7fce4a4ef700 1 -- start start 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce4a4ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce440b2530 0x7fce440c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce4a4ef700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce440b2900 0x7fce440c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce4a4ef700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce44159080 0x7fce440c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce4a4ef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440c41c0 con 0x7fce440b2530 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce4a4ef700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440c8ff0 con 0x7fce44159080 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce4a4ef700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440c83e0 con 0x7fce440b2900 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce440b2530 0x7fce440c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce440b2530 0x7fce440c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34440/0 (socket says 172.21.15.3:34440) 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 -- 172.21.15.3:0/1179777496 learned_addr learned my addr 172.21.15.3:0/1179777496 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce437fe700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce440b2900 0x7fce440c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce48a8c700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce44159080 0x7fce440c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 -- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce440b2900 msgr2=0x7fce440c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce440b2900 0x7fce440c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 -- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce44159080 msgr2=0x7fce440c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce44159080 0x7fce440c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 -- 172.21.15.3:0/1179777496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009420 con 0x7fce440b2530 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.304+0000 7fce43fff700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce440b2530 0x7fce440c9c10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fce3400ec80 tx=0x7fce340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.305+0000 7fce2bfff700 1 -- 172.21.15.3:0/1179777496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400c9d0 con 0x7fce440b2530 2023-10-06T04:59:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.305+0000 7fce2bfff700 1 -- 172.21.15.3:0/1179777496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3400cb30 con 0x7fce440b2530 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.305+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c009a20 con 0x7fce440b2530 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.305+0000 7fce2bfff700 1 -- 172.21.15.3:0/1179777496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400ceb0 con 0x7fce440b2530 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.305+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce440c89a0 con 0x7fce440b2530 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.306+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce44071680 con 0x7fce440b2530 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.306+0000 7fce2bfff700 1 -- 172.21.15.3:0/1179777496 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce34013680 con 0x7fce440b2530 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.306+0000 7fce2bfff700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fce30066f30 0x7fce300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.306+0000 7fce2bfff700 1 -- 172.21.15.3:0/1179777496 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce34084f00 con 0x7fce440b2530 2023-10-06T04:59:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.307+0000 7fce437fe700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fce30066f30 0x7fce300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.307+0000 7fce437fe700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fce30066f30 0x7fce300693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce4416cd30 tx=0x7fce38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:10.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.311+0000 7fce2bfff700 1 -- 172.21.15.3:0/1179777496 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce34052010 con 0x7fce440b2530 2023-10-06T04:59:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.497+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce440cb440 con 0x7fce30066f30 2023-10-06T04:59:10.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.507+0000 7fce2bfff700 1 -- 172.21.15.3:0/1179777496 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308708 (secure 0 0 0) 0x7fce440cb440 con 0x7fce30066f30 2023-10-06T04:59:10.513 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:10.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fce30066f30 msgr2=0x7fce300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fce30066f30 0x7fce300693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce4416cd30 tx=0x7fce38009040 comp rx=0 tx=0).stop 2023-10-06T04:59:10.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce440b2530 msgr2=0x7fce440c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce440b2530 0x7fce440c9c10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fce3400ec80 tx=0x7fce340091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 shutdown_connections 2023-10-06T04:59:10.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fce30066f30 0x7fce300693e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce44159080 0x7fce440c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce440b2530 0x7fce440c9c10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 --2- 172.21.15.3:0/1179777496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce440b2900 0x7fce440c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 >> 172.21.15.3:0/1179777496 conn(0x7fce44070310 msgr2=0x7fce440cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 shutdown_connections 2023-10-06T04:59:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.510+0000 7fce4a4ef700 1 -- 172.21.15.3:0/1179777496 wait complete. 2023-10-06T04:59:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:59:10.525 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-06T04:59:09.524507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445824,"kb_used_data":10176,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752528896,"statfs":{"total":773094113280,"available":770589589504,"internally_reserved":0,"allocated":10420224,"data_stored":5321181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.518620"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.021652+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:40.965728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.018066+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:31.156638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017806+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:55.185806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016696+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.104668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016366+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:24.260605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014934+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:49.295778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.019752+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:36.653367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.019396+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:27.219788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017920+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:44.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014935+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:01.732144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016919+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011338+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:13.415770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014693+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016598+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.018596+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017837+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015755+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:26.205356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015002+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:58.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012429+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:41.792782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.010492+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:49.048000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014331+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:02.587278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013637+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013020+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:36.271837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015735+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011571+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:21.883224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012207+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:22.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016324+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:58.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015319+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:10.609306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013796+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:07.976219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011941+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:59.866311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015126+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:22.404213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016613+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:01.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012301+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:17.016243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030540+0000","last_change":"2023-10-06T04:59:05.030540+0000","last_active":"2023-10-06T04:59:05.030540+0000","last_peered":"2023-10-06T04:59:05.030540+0000","last_clean":"2023-10-06T04:59:05.030540+0000","last_became_active":"2023-10-06T04:59:05.030260+0000","last_became_peered":"2023-10-06T04:59:05.030260+0000","last_unstale":"2023-10-06T04:59:05.030540+0000","last_undegraded":"2023-10-06T04:59:05.030540+0000","last_fullsized":"2023-10-06T04:59:05.030540+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012428+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:23.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026990+0000","last_change":"2023-10-06T04:59:05.026990+0000","last_active":"2023-10-06T04:59:05.026990+0000","last_peered":"2023-10-06T04:59:05.026990+0000","last_clean":"2023-10-06T04:59:05.026990+0000","last_became_active":"2023-10-06T04:59:05.026693+0000","last_became_peered":"2023-10-06T04:59:05.026693+0000","last_unstale":"2023-10-06T04:59:05.026990+0000","last_undegraded":"2023-10-06T04:59:05.026990+0000","last_fullsized":"2023-10-06T04:59:05.026990+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:49.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011347+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:28.217935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021204+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:05.021204+0000","last_peered":"2023-10-06T04:59:05.021204+0000","last_clean":"2023-10-06T04:59:05.021204+0000","last_became_active":"2023-10-06T04:59:05.020908+0000","last_became_peered":"2023-10-06T04:59:05.020908+0000","last_unstale":"2023-10-06T04:59:05.021204+0000","last_undegraded":"2023-10-06T04:59:05.021204+0000","last_fullsized":"2023-10-06T04:59:05.021204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.255698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014483+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:19.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023617+0000","last_change":"2023-10-06T04:59:05.023617+0000","last_active":"2023-10-06T04:59:05.023617+0000","last_peered":"2023-10-06T04:59:05.023617+0000","last_clean":"2023-10-06T04:59:05.023617+0000","last_became_active":"2023-10-06T04:59:05.023394+0000","last_became_peered":"2023-10-06T04:59:05.023394+0000","last_unstale":"2023-10-06T04:59:05.023617+0000","last_undegraded":"2023-10-06T04:59:05.023617+0000","last_fullsized":"2023-10-06T04:59:05.023617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.562815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014939+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030809+0000","last_change":"2023-10-06T04:59:05.030809+0000","last_active":"2023-10-06T04:59:05.030809+0000","last_peered":"2023-10-06T04:59:05.030809+0000","last_clean":"2023-10-06T04:59:05.030809+0000","last_became_active":"2023-10-06T04:59:05.030550+0000","last_became_peered":"2023-10-06T04:59:05.030550+0000","last_unstale":"2023-10-06T04:59:05.030809+0000","last_undegraded":"2023-10-06T04:59:05.030809+0000","last_fullsized":"2023-10-06T04:59:05.030809+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011439+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:19.436490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024000+0000","last_change":"2023-10-06T04:59:05.024000+0000","last_active":"2023-10-06T04:59:05.024000+0000","last_peered":"2023-10-06T04:59:05.024000+0000","last_clean":"2023-10-06T04:59:05.024000+0000","last_became_active":"2023-10-06T04:59:05.023706+0000","last_became_peered":"2023-10-06T04:59:05.023706+0000","last_unstale":"2023-10-06T04:59:05.024000+0000","last_undegraded":"2023-10-06T04:59:05.024000+0000","last_fullsized":"2023-10-06T04:59:05.024000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014554+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:50:43.443720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021204+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:05.021204+0000","last_peered":"2023-10-06T04:59:05.021204+0000","last_clean":"2023-10-06T04:59:05.021204+0000","last_became_active":"2023-10-06T04:59:05.020967+0000","last_became_peered":"2023-10-06T04:59:05.020967+0000","last_unstale":"2023-10-06T04:59:05.021204+0000","last_undegraded":"2023-10-06T04:59:05.021204+0000","last_fullsized":"2023-10-06T04:59:05.021204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:54.137958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016972+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025368+0000","last_change":"2023-10-06T04:59:05.025368+0000","last_active":"2023-10-06T04:59:05.025368+0000","last_peered":"2023-10-06T04:59:05.025368+0000","last_clean":"2023-10-06T04:59:05.025368+0000","last_became_active":"2023-10-06T04:59:05.025101+0000","last_became_peered":"2023-10-06T04:59:05.025101+0000","last_unstale":"2023-10-06T04:59:05.025368+0000","last_undegraded":"2023-10-06T04:59:05.025368+0000","last_fullsized":"2023-10-06T04:59:05.025368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012737+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.571901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021703+0000","last_change":"2023-10-06T04:59:05.021703+0000","last_active":"2023-10-06T04:59:05.021703+0000","last_peered":"2023-10-06T04:59:05.021703+0000","last_clean":"2023-10-06T04:59:05.021703+0000","last_became_active":"2023-10-06T04:59:05.021413+0000","last_became_peered":"2023-10-06T04:59:05.021413+0000","last_unstale":"2023-10-06T04:59:05.021703+0000","last_undegraded":"2023-10-06T04:59:05.021703+0000","last_fullsized":"2023-10-06T04:59:05.021703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:15.699249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013498+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.016394+0000","last_change":"2023-10-06T04:59:05.016394+0000","last_active":"2023-10-06T04:59:05.016394+0000","last_peered":"2023-10-06T04:59:05.016394+0000","last_clean":"2023-10-06T04:59:05.016394+0000","last_became_active":"2023-10-06T04:59:05.016106+0000","last_became_peered":"2023-10-06T04:59:05.016106+0000","last_unstale":"2023-10-06T04:59:05.016394+0000","last_undegraded":"2023-10-06T04:59:05.016394+0000","last_fullsized":"2023-10-06T04:59:05.016394+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:10.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014320+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:50.435166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026950+0000","last_change":"2023-10-06T04:59:05.026950+0000","last_active":"2023-10-06T04:59:05.026950+0000","last_peered":"2023-10-06T04:59:05.026950+0000","last_clean":"2023-10-06T04:59:05.026950+0000","last_became_active":"2023-10-06T04:59:05.026723+0000","last_became_peered":"2023-10-06T04:59:05.026723+0000","last_unstale":"2023-10-06T04:59:05.026950+0000","last_undegraded":"2023-10-06T04:59:05.026950+0000","last_fullsized":"2023-10-06T04:59:05.026950+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014381+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025160+0000","last_change":"2023-10-06T04:59:05.025160+0000","last_active":"2023-10-06T04:59:05.025160+0000","last_peered":"2023-10-06T04:59:05.025160+0000","last_clean":"2023-10-06T04:59:05.025160+0000","last_became_active":"2023-10-06T04:59:05.024909+0000","last_became_peered":"2023-10-06T04:59:05.024909+0000","last_unstale":"2023-10-06T04:59:05.025160+0000","last_undegraded":"2023-10-06T04:59:05.025160+0000","last_fullsized":"2023-10-06T04:59:05.025160+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:00.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016608+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023984+0000","last_change":"2023-10-06T04:59:05.023984+0000","last_active":"2023-10-06T04:59:05.023984+0000","last_peered":"2023-10-06T04:59:05.023984+0000","last_clean":"2023-10-06T04:59:05.023984+0000","last_became_active":"2023-10-06T04:59:05.023641+0000","last_became_peered":"2023-10-06T04:59:05.023641+0000","last_unstale":"2023-10-06T04:59:05.023984+0000","last_undegraded":"2023-10-06T04:59:05.023984+0000","last_fullsized":"2023-10-06T04:59:05.023984+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:54.810243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015507+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.531534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025232+0000","last_change":"2023-10-06T04:59:05.025232+0000","last_active":"2023-10-06T04:59:05.025232+0000","last_peered":"2023-10-06T04:59:05.025232+0000","last_clean":"2023-10-06T04:59:05.025232+0000","last_became_active":"2023-10-06T04:59:05.024850+0000","last_became_peered":"2023-10-06T04:59:05.024850+0000","last_unstale":"2023-10-06T04:59:05.025232+0000","last_undegraded":"2023-10-06T04:59:05.025232+0000","last_fullsized":"2023-10-06T04:59:05.025232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012744+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:58.844389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.020554+0000","last_change":"2023-10-06T04:59:05.020554+0000","last_active":"2023-10-06T04:59:05.020554+0000","last_peered":"2023-10-06T04:59:05.020554+0000","last_clean":"2023-10-06T04:59:05.020554+0000","last_became_active":"2023-10-06T04:59:05.020294+0000","last_became_peered":"2023-10-06T04:59:05.020294+0000","last_unstale":"2023-10-06T04:59:05.020554+0000","last_undegraded":"2023-10-06T04:59:05.020554+0000","last_fullsized":"2023-10-06T04:59:05.020554+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:24.547416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011224+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.022660+0000","last_change":"2023-10-06T04:59:05.022660+0000","last_active":"2023-10-06T04:59:05.022660+0000","last_peered":"2023-10-06T04:59:05.022660+0000","last_clean":"2023-10-06T04:59:05.022660+0000","last_became_active":"2023-10-06T04:59:05.021894+0000","last_became_peered":"2023-10-06T04:59:05.021894+0000","last_unstale":"2023-10-06T04:59:05.022660+0000","last_undegraded":"2023-10-06T04:59:05.022660+0000","last_fullsized":"2023-10-06T04:59:05.022660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.654900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014857+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:41.853665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.022693+0000","last_change":"2023-10-06T04:59:05.022693+0000","last_active":"2023-10-06T04:59:05.022693+0000","last_peered":"2023-10-06T04:59:05.022693+0000","last_clean":"2023-10-06T04:59:05.022693+0000","last_became_active":"2023-10-06T04:59:05.022421+0000","last_became_peered":"2023-10-06T04:59:05.022421+0000","last_unstale":"2023-10-06T04:59:05.022693+0000","last_undegraded":"2023-10-06T04:59:05.022693+0000","last_fullsized":"2023-10-06T04:59:05.022693+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:59.701496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.010770+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:47.264143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024903+0000","last_change":"2023-10-06T04:59:05.024903+0000","last_active":"2023-10-06T04:59:05.024903+0000","last_peered":"2023-10-06T04:59:05.024903+0000","last_clean":"2023-10-06T04:59:05.024903+0000","last_became_active":"2023-10-06T04:59:05.024608+0000","last_became_peered":"2023-10-06T04:59:05.024608+0000","last_unstale":"2023-10-06T04:59:05.024903+0000","last_undegraded":"2023-10-06T04:59:05.024903+0000","last_fullsized":"2023-10-06T04:59:05.024903+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:48.099761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016917+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:11.903244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030722+0000","last_change":"2023-10-06T04:59:05.030722+0000","last_active":"2023-10-06T04:59:05.030722+0000","last_peered":"2023-10-06T04:59:05.030722+0000","last_clean":"2023-10-06T04:59:05.030722+0000","last_became_active":"2023-10-06T04:59:05.030276+0000","last_became_peered":"2023-10-06T04:59:05.030276+0000","last_unstale":"2023-10-06T04:59:05.030722+0000","last_undegraded":"2023-10-06T04:59:05.030722+0000","last_fullsized":"2023-10-06T04:59:05.030722+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:57.328563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017462+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024303+0000","last_change":"2023-10-06T04:59:05.024303+0000","last_active":"2023-10-06T04:59:05.024303+0000","last_peered":"2023-10-06T04:59:05.024303+0000","last_clean":"2023-10-06T04:59:05.024303+0000","last_became_active":"2023-10-06T04:59:05.024038+0000","last_became_peered":"2023-10-06T04:59:05.024038+0000","last_unstale":"2023-10-06T04:59:05.024303+0000","last_undegraded":"2023-10-06T04:59:05.024303+0000","last_fullsized":"2023-10-06T04:59:05.024303+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012621+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:05.658324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023856+0000","last_change":"2023-10-06T04:59:05.023856+0000","last_active":"2023-10-06T04:59:05.023856+0000","last_peered":"2023-10-06T04:59:05.023856+0000","last_clean":"2023-10-06T04:59:05.023856+0000","last_became_active":"2023-10-06T04:59:05.023612+0000","last_became_peered":"2023-10-06T04:59:05.023612+0000","last_unstale":"2023-10-06T04:59:05.023856+0000","last_undegraded":"2023-10-06T04:59:05.023856+0000","last_fullsized":"2023-10-06T04:59:05.023856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:04.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012763+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:58.906683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023970+0000","last_change":"2023-10-06T04:59:05.023970+0000","last_active":"2023-10-06T04:59:05.023970+0000","last_peered":"2023-10-06T04:59:05.023970+0000","last_clean":"2023-10-06T04:59:05.023970+0000","last_became_active":"2023-10-06T04:59:05.023712+0000","last_became_peered":"2023-10-06T04:59:05.023712+0000","last_unstale":"2023-10-06T04:59:05.023970+0000","last_undegraded":"2023-10-06T04:59:05.023970+0000","last_fullsized":"2023-10-06T04:59:05.023970+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:31.733624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017238+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028495+0000","last_change":"2023-10-06T04:59:05.028495+0000","last_active":"2023-10-06T04:59:05.028495+0000","last_peered":"2023-10-06T04:59:05.028495+0000","last_clean":"2023-10-06T04:59:05.028495+0000","last_became_active":"2023-10-06T04:59:05.028219+0000","last_became_peered":"2023-10-06T04:59:05.028219+0000","last_unstale":"2023-10-06T04:59:05.028495+0000","last_undegraded":"2023-10-06T04:59:05.028495+0000","last_fullsized":"2023-10-06T04:59:05.028495+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:54.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016324+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023535+0000","last_change":"2023-10-06T04:59:05.023535+0000","last_active":"2023-10-06T04:59:05.023535+0000","last_peered":"2023-10-06T04:59:05.023535+0000","last_clean":"2023-10-06T04:59:05.023535+0000","last_became_active":"2023-10-06T04:59:05.023261+0000","last_became_peered":"2023-10-06T04:59:05.023261+0000","last_unstale":"2023-10-06T04:59:05.023535+0000","last_undegraded":"2023-10-06T04:59:05.023535+0000","last_fullsized":"2023-10-06T04:59:05.023535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:47.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011797+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.065649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026906+0000","last_change":"2023-10-06T04:59:05.026906+0000","last_active":"2023-10-06T04:59:05.026906+0000","last_peered":"2023-10-06T04:59:05.026906+0000","last_clean":"2023-10-06T04:59:05.026906+0000","last_became_active":"2023-10-06T04:59:05.026633+0000","last_became_peered":"2023-10-06T04:59:05.026633+0000","last_unstale":"2023-10-06T04:59:05.026906+0000","last_undegraded":"2023-10-06T04:59:05.026906+0000","last_fullsized":"2023-10-06T04:59:05.026906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:56.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.013419+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:59:04.013419+0000","last_peered":"2023-10-06T04:59:04.013419+0000","last_clean":"2023-10-06T04:59:04.013419+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:59:04.013419+0000","last_undegraded":"2023-10-06T04:59:04.013419+0000","last_fullsized":"2023-10-06T04:59:04.013419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011969+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:31.872736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028380+0000","last_change":"2023-10-06T04:59:05.028380+0000","last_active":"2023-10-06T04:59:05.028380+0000","last_peered":"2023-10-06T04:59:05.028380+0000","last_clean":"2023-10-06T04:59:05.028380+0000","last_became_active":"2023-10-06T04:59:05.028098+0000","last_became_peered":"2023-10-06T04:59:05.028098+0000","last_unstale":"2023-10-06T04:59:05.028380+0000","last_undegraded":"2023-10-06T04:59:05.028380+0000","last_fullsized":"2023-10-06T04:59:05.028380+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:22.276266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.007290+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:59:04.007290+0000","last_peered":"2023-10-06T04:59:04.007290+0000","last_clean":"2023-10-06T04:59:04.007290+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:59:04.007290+0000","last_undegraded":"2023-10-06T04:59:04.007290+0000","last_fullsized":"2023-10-06T04:59:04.007290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014304+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:08.338740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024110+0000","last_change":"2023-10-06T04:59:05.024110+0000","last_active":"2023-10-06T04:59:05.024110+0000","last_peered":"2023-10-06T04:59:05.024110+0000","last_clean":"2023-10-06T04:59:05.024110+0000","last_became_active":"2023-10-06T04:59:05.023840+0000","last_became_peered":"2023-10-06T04:59:05.023840+0000","last_unstale":"2023-10-06T04:59:05.024110+0000","last_undegraded":"2023-10-06T04:59:05.024110+0000","last_fullsized":"2023-10-06T04:59:05.024110+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.014748+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:59:04.014748+0000","last_peered":"2023-10-06T04:59:04.014748+0000","last_clean":"2023-10-06T04:59:04.014748+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:59:04.014748+0000","last_undegraded":"2023-10-06T04:59:04.014748+0000","last_fullsized":"2023-10-06T04:59:04.014748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":222,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010191+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:59:04.010191+0000","last_peered":"2023-10-06T04:59:04.010191+0000","last_clean":"2023-10-06T04:59:04.010191+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:59:04.010191+0000","last_undegraded":"2023-10-06T04:59:04.010191+0000","last_fullsized":"2023-10-06T04:59:04.010191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015895+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:16.704156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030905+0000","last_change":"2023-10-06T04:59:05.030905+0000","last_active":"2023-10-06T04:59:05.030905+0000","last_peered":"2023-10-06T04:59:05.030905+0000","last_clean":"2023-10-06T04:59:05.030905+0000","last_became_active":"2023-10-06T04:59:05.030530+0000","last_became_peered":"2023-10-06T04:59:05.030530+0000","last_unstale":"2023-10-06T04:59:05.030905+0000","last_undegraded":"2023-10-06T04:59:05.030905+0000","last_fullsized":"2023-10-06T04:59:05.030905+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:15.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":72,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010382+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:59:04.010382+0000","last_peered":"2023-10-06T04:59:04.010382+0000","last_clean":"2023-10-06T04:59:04.010382+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:59:04.010382+0000","last_undegraded":"2023-10-06T04:59:04.010382+0000","last_fullsized":"2023-10-06T04:59:04.010382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016145+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.815150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028526+0000","last_change":"2023-10-06T04:59:05.028526+0000","last_active":"2023-10-06T04:59:05.028526+0000","last_peered":"2023-10-06T04:59:05.028526+0000","last_clean":"2023-10-06T04:59:05.028526+0000","last_became_active":"2023-10-06T04:59:05.028134+0000","last_became_peered":"2023-10-06T04:59:05.028134+0000","last_unstale":"2023-10-06T04:59:05.028526+0000","last_undegraded":"2023-10-06T04:59:05.028526+0000","last_fullsized":"2023-10-06T04:59:05.028526+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:16.407534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010554+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:59:04.010554+0000","last_peered":"2023-10-06T04:59:04.010554+0000","last_clean":"2023-10-06T04:59:04.010554+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:59:04.010554+0000","last_undegraded":"2023-10-06T04:59:04.010554+0000","last_fullsized":"2023-10-06T04:59:04.010554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016082+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:58.333470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023805+0000","last_change":"2023-10-06T04:59:05.023805+0000","last_active":"2023-10-06T04:59:05.023805+0000","last_peered":"2023-10-06T04:59:05.023805+0000","last_clean":"2023-10-06T04:59:05.023805+0000","last_became_active":"2023-10-06T04:59:05.023560+0000","last_became_peered":"2023-10-06T04:59:05.023560+0000","last_unstale":"2023-10-06T04:59:05.023805+0000","last_undegraded":"2023-10-06T04:59:05.023805+0000","last_fullsized":"2023-10-06T04:59:05.023805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:16.588598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010472+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:59:04.010472+0000","last_peered":"2023-10-06T04:59:04.010472+0000","last_clean":"2023-10-06T04:59:04.010472+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:59:04.010472+0000","last_undegraded":"2023-10-06T04:59:04.010472+0000","last_fullsized":"2023-10-06T04:59:04.010472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013631+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:38.572727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.020425+0000","last_change":"2023-10-06T04:59:05.020425+0000","last_active":"2023-10-06T04:59:05.020425+0000","last_peered":"2023-10-06T04:59:05.020425+0000","last_clean":"2023-10-06T04:59:05.020425+0000","last_became_active":"2023-10-06T04:59:05.020172+0000","last_became_peered":"2023-10-06T04:59:05.020172+0000","last_unstale":"2023-10-06T04:59:05.020425+0000","last_undegraded":"2023-10-06T04:59:05.020425+0000","last_fullsized":"2023-10-06T04:59:05.020425+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:25.681695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.011548+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:59:04.011548+0000","last_peered":"2023-10-06T04:59:04.011548+0000","last_clean":"2023-10-06T04:59:04.011548+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:59:04.011548+0000","last_undegraded":"2023-10-06T04:59:04.011548+0000","last_fullsized":"2023-10-06T04:59:04.011548+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014748+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028402+0000","last_change":"2023-10-06T04:59:05.028402+0000","last_active":"2023-10-06T04:59:05.028402+0000","last_peered":"2023-10-06T04:59:05.028402+0000","last_clean":"2023-10-06T04:59:05.028402+0000","last_became_active":"2023-10-06T04:59:05.028139+0000","last_became_peered":"2023-10-06T04:59:05.028139+0000","last_unstale":"2023-10-06T04:59:05.028402+0000","last_undegraded":"2023-10-06T04:59:05.028402+0000","last_fullsized":"2023-10-06T04:59:05.028402+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:56.970617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.011518+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:59:04.011518+0000","last_peered":"2023-10-06T04:59:04.011518+0000","last_clean":"2023-10-06T04:59:04.011518+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:59:04.011518+0000","last_undegraded":"2023-10-06T04:59:04.011518+0000","last_fullsized":"2023-10-06T04:59:04.011518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":1104597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.73399999999999999},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73199999999999998,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.495}]},{"osd":1,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":1.073},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.51600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":1.111},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.58999999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.503},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.441}]},{"osd":5,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.502},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":1.214},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302520,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069320,"statfs":{"total":96636764160,"available":96326983680,"internally_reserved":0,"allocated":1163264,"data_stored":522198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.52200000000000002},"min":{"1min":0.54900000000000004,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.81200000000000006,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.57199999999999995},"min":{"1min":0.56100000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.39800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":1.234,"15min":1.234},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.55200000000000005},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81000000000000005,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75800000000000001,"5min":3.669,"15min":3.669},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.495},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":1104597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.23000000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.27900000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.376}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.501,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.316},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.70999999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.85499999999999998,"15min":0.92900000000000005},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.64100000000000001},"min":{"1min":0.47499999999999998,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.998,"15min":0.998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.65800000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":1.0900000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80600000000000005,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.026},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.79400000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.505,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57699999999999996,"5min":0.628,"15min":0.628},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.441}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90700000000000003,"5min":1.022,"15min":1.022},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.495,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.5,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96599999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.995,"15min":0.995},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302520,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069320,"statfs":{"total":96636764160,"available":96326983680,"internally_reserved":0,"allocated":1163264,"data_stored":522198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.51200000000000001,"5min":0.747,"15min":1.048},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.42599999999999999,"15min":0.39000000000000001},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.375}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":0.61699999999999999,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.309},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.36099999999999999},"min":{"1min":0.29499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.439}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.5,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38700000000000001,"15min":0.40600000000000003},"min":{"1min":0.251,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":2.008,"15min":2.008},"last":0.316},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41299999999999998,"15min":0.56499999999999995},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54500000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42399999999999999,"15min":0.46000000000000002},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.64600000000000002,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":1.379,"15min":5.069},"last":0.443},{"interface":"front","average":{"1min":0.443,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":2.246,"15min":2.246},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83199999999999996,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82199999999999995,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.79400000000000004},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.40400000000000003},"max":{"1min":0.76200000000000001,"5min":1.964,"15min":6.367},"last":0.623},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65000000000000002,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":1.3,"15min":5},"last":0.754}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.379},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.78500000000000003},"min":{"1min":0.38200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.82099999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.80400000000000005,"15min":5.415},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.66700000000000004},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":4.202},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":1.044,"15min":4.2130000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40699999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.42499999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.77600000000000002,"5min":0.86199999999999999,"15min":0.872},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.313},"max":{"1min":0.84399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306612,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065228,"statfs":{"total":96636764160,"available":96322793472,"internally_reserved":0,"allocated":1159168,"data_stored":520845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.67600000000000005},"min":{"1min":0.36899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.625},"min":{"1min":0.436,"5min":0.247,"15min":0.247},"max":{"1min":0.76000000000000001,"5min":0.80800000000000005,"15min":3.7679999999999998},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.70799999999999996},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":6.7949999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.75800000000000001},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.93799999999999994},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":9.7210000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.93500000000000005},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":9.7539999999999996},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.504},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.34200000000000003},"max":{"1min":0.66300000000000003,"5min":0.746,"15min":0.746},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80100000000000005},"last":0.46200000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:59:10.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:59:10.571 INFO:tasks.ceph.mds.d.smithi153.stderr:2023-10-06T04:59:10.569+0000 7f89cc5da700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:59:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.681+0000 7f7f49114700 1 Processor -- start 2023-10-06T04:59:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.681+0000 7f7f49114700 1 -- start start 2023-10-06T04:59:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.682+0000 7f7f49114700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7f4415ccc0 0x7f7f440d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.682+0000 7f7f49114700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 0x7f7f440b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.682+0000 7f7f49114700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f440cf6c0 con 0x7f7f4415c8f0 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.682+0000 7f7f49114700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f440cfb50 con 0x7f7f4415ccc0 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.682+0000 7f7f49114700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440d0ad0 con 0x7f7f440d06a0 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.682+0000 7f7f42d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7f4415c8f0 0x7f7f4415db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34084/0 (socket says 172.21.15.3:34084) 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.682+0000 7f7f42d9d700 1 -- 172.21.15.3:0/2083347613 learned_addr learned my addr 172.21.15.3:0/2083347613 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4259c700 1 --2- 172.21.15.3:0/2083347613 >> v2:172.21.15.153:3300/0 conn(0x7f7f4415ccc0 0x7f7f440d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4359e700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 0x7f7f440b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158560003 0 0) 0x7f7f440cf6c0 con 0x7f7f4415c8f0 2023-10-06T04:59:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2c003570 con 0x7f7f4415c8f0 2023-10-06T04:59:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368877580 0 0) 0x7f7f2c003570 con 0x7f7f4415c8f0 2023-10-06T04:59:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 msgr2=0x7f7f440b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 0x7f7f440b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 >> v2:172.21.15.153:3300/0 conn(0x7f7f4415ccc0 msgr2=0x7f7f440d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 --2- 172.21.15.3:0/2083347613 >> v2:172.21.15.153:3300/0 conn(0x7f7f4415ccc0 0x7f7f440d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440ceab0 con 0x7f7f4415c8f0 2023-10-06T04:59:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f7f340017f0 con 0x7f7f4415c8f0 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 >> v1:172.21.15.3:6789/0 conn(0x7f7f4415c8f0 legacy=0x7f7f4415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 0x7f7f2c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f2c003fe0 0x7f7f2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f2c004d50 0x7f7f2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440cfb50 con 0x7f7f2c003fe0 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440d0ad0 con 0x7f7f2c004d50 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c00ba90 con 0x7f7f440d06a0 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4259c700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f2c003fe0 0x7f7f2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f42d9d700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 0x7f7f2c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4359e700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f2c004d50 0x7f7f2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4259c700 1 -- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 msgr2=0x7f7f2c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4259c700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440d06a0 0x7f7f2c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4259c700 1 -- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f2c004d50 msgr2=0x7f7f2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4259c700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f2c004d50 0x7f7f2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4259c700 1 -- 172.21.15.3:0/2083347613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c003570 con 0x7f7f2c003fe0 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.683+0000 7f7f4359e700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f2c004d50 0x7f7f2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f4259c700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f2c003fe0 0x7f7f2c004810 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009c00 tx=0x7f7f380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c48b24fa04442db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800e6b0 con 0x7f7f2c003fe0 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3800e810 con 0x7f7f2c003fe0 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f41d9b700 1 -- 172.21.15.3:0/2083347613 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800ebd0 con 0x7f7f2c003fe0 2023-10-06T04:59:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f49114700 1 -- 172.21.15.3:0/2083347613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f2c003fe0 msgr2=0x7f7f2c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f49114700 1 --2- 172.21.15.3:0/2083347613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f2c003fe0 0x7f7f2c004810 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009c00 tx=0x7f7f380091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f4359e700 1 -- 172.21.15.3:0/2083347613 reap_dead start 2023-10-06T04:59:10.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f49114700 1 -- 172.21.15.3:0/2083347613 shutdown_connections 2023-10-06T04:59:10.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f49114700 1 -- 172.21.15.3:0/2083347613 >> 172.21.15.3:0/2083347613 conn(0x7f7f44070310 msgr2=0x7f7f44070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:10.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f49114700 1 -- 172.21.15.3:0/2083347613 shutdown_connections 2023-10-06T04:59:10.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.684+0000 7f7f49114700 1 -- 172.21.15.3:0/2083347613 wait complete. 2023-10-06T04:59:10.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.685+0000 7f7f49114700 1 Processor -- start 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f49114700 1 -- start start 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f49114700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f441483d0 0x7f7f4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f49114700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440c5c30 0x7f7f44154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f49114700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f44155240 0x7f7f441588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f49114700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441556e0 con 0x7f7f44155240 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f49114700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440c6d00 con 0x7f7f441483d0 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f49114700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440c60f0 con 0x7f7f440c5c30 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f4359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f44155240 0x7f7f441588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f4359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f44155240 0x7f7f441588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34456/0 (socket says 172.21.15.3:34456) 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f4359e700 1 -- 172.21.15.3:0/3474363086 learned_addr learned my addr 172.21.15.3:0/3474363086 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f42d9d700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f441483d0 0x7f7f4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.686+0000 7f7f4259c700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440c5c30 0x7f7f44154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f4359e700 1 -- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440c5c30 msgr2=0x7f7f44154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f4359e700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440c5c30 0x7f7f44154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f4359e700 1 -- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f441483d0 msgr2=0x7f7f4415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f4359e700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f441483d0 0x7f7f4415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f4359e700 1 -- 172.21.15.3:0/3474363086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440c63d0 con 0x7f7f44155240 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f42d9d700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f441483d0 0x7f7f4415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f4259c700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440c5c30 0x7f7f44154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f4359e700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f44155240 0x7f7f441588c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f440af8a0 tx=0x7f7f3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f2affd700 1 -- 172.21.15.3:0/3474363086 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c015670 con 0x7f7f44155240 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f2affd700 1 -- 172.21.15.3:0/3474363086 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c0157d0 con 0x7f7f44155240 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f440c66b0 con 0x7f7f44155240 2023-10-06T04:59:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f2affd700 1 -- 172.21.15.3:0/3474363086 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c015b90 con 0x7f7f44155240 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.687+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4415b110 con 0x7f7f44155240 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.688+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f440c7070 con 0x7f7f44155240 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.688+0000 7f7f2affd700 1 -- 172.21.15.3:0/3474363086 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f3c01e400 con 0x7f7f44155240 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.689+0000 7f7f2affd700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f30066f30 0x7f7f300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.689+0000 7f7f2affd700 1 -- 172.21.15.3:0/3474363086 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f3c0849f0 con 0x7f7f44155240 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.689+0000 7f7f42d9d700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f30066f30 0x7f7f300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.690+0000 7f7f42d9d700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f30066f30 0x7f7f300693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7f34004c10 tx=0x7f7f34006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.693+0000 7f7f2affd700 1 -- 172.21.15.3:0/3474363086 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f3c051b00 con 0x7f7f44155240 2023-10-06T04:59:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.883+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f44071680 con 0x7f7f30066f30 2023-10-06T04:59:10.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.892+0000 7f7f2affd700 1 -- 172.21.15.3:0/3474363086 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308708 (secure 0 0 0) 0x7f7f44071680 con 0x7f7f30066f30 2023-10-06T04:59:10.898 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:10.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.895+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f30066f30 msgr2=0x7f7f300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.895+0000 7f7f49114700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f30066f30 0x7f7f300693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7f34004c10 tx=0x7f7f34006040 comp rx=0 tx=0).stop 2023-10-06T04:59:10.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.895+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f44155240 msgr2=0x7f7f441588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:10.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.895+0000 7f7f49114700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f44155240 0x7f7f441588c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f440af8a0 tx=0x7f7f3c008040 comp rx=0 tx=0).stop 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 shutdown_connections 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f30066f30 0x7f7f300693e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f441483d0 0x7f7f4415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f44155240 0x7f7f441588c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 --2- 172.21.15.3:0/3474363086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f440c5c30 0x7f7f44154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 >> 172.21.15.3:0/3474363086 conn(0x7f7f44070310 msgr2=0x7f7f440b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 shutdown_connections 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:10.896+0000 7f7f49114700 1 -- 172.21.15.3:0/3474363086 wait complete. 2023-10-06T04:59:10.901 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:59:10.910 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-06T04:59:09.524507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445824,"kb_used_data":10176,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752528896,"statfs":{"total":773094113280,"available":770589589504,"internally_reserved":0,"allocated":10420224,"data_stored":5321181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.518620"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.021652+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:40.965728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.018066+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:31.156638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017806+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:55.185806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016696+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.104668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016366+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:24.260605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014934+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:49.295778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.019752+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:36.653367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.019396+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:27.219788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017920+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:44.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014935+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:01.732144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016919+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011338+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:13.415770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014693+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016598+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.018596+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017837+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015755+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:26.205356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015002+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:58.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012429+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:41.792782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.010492+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:49.048000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014331+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:02.587278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013637+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013020+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:36.271837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015735+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011571+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:21.883224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012207+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:22.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016324+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:58.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015319+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:10.609306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013796+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:07.976219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011941+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:59.866311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015126+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:22.404213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016613+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:01.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012301+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:17.016243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030540+0000","last_change":"2023-10-06T04:59:05.030540+0000","last_active":"2023-10-06T04:59:05.030540+0000","last_peered":"2023-10-06T04:59:05.030540+0000","last_clean":"2023-10-06T04:59:05.030540+0000","last_became_active":"2023-10-06T04:59:05.030260+0000","last_became_peered":"2023-10-06T04:59:05.030260+0000","last_unstale":"2023-10-06T04:59:05.030540+0000","last_undegraded":"2023-10-06T04:59:05.030540+0000","last_fullsized":"2023-10-06T04:59:05.030540+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012428+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:23.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026990+0000","last_change":"2023-10-06T04:59:05.026990+0000","last_active":"2023-10-06T04:59:05.026990+0000","last_peered":"2023-10-06T04:59:05.026990+0000","last_clean":"2023-10-06T04:59:05.026990+0000","last_became_active":"2023-10-06T04:59:05.026693+0000","last_became_peered":"2023-10-06T04:59:05.026693+0000","last_unstale":"2023-10-06T04:59:05.026990+0000","last_undegraded":"2023-10-06T04:59:05.026990+0000","last_fullsized":"2023-10-06T04:59:05.026990+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:49.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011347+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:28.217935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021204+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:05.021204+0000","last_peered":"2023-10-06T04:59:05.021204+0000","last_clean":"2023-10-06T04:59:05.021204+0000","last_became_active":"2023-10-06T04:59:05.020908+0000","last_became_peered":"2023-10-06T04:59:05.020908+0000","last_unstale":"2023-10-06T04:59:05.021204+0000","last_undegraded":"2023-10-06T04:59:05.021204+0000","last_fullsized":"2023-10-06T04:59:05.021204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.255698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014483+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:19.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023617+0000","last_change":"2023-10-06T04:59:05.023617+0000","last_active":"2023-10-06T04:59:05.023617+0000","last_peered":"2023-10-06T04:59:05.023617+0000","last_clean":"2023-10-06T04:59:05.023617+0000","last_became_active":"2023-10-06T04:59:05.023394+0000","last_became_peered":"2023-10-06T04:59:05.023394+0000","last_unstale":"2023-10-06T04:59:05.023617+0000","last_undegraded":"2023-10-06T04:59:05.023617+0000","last_fullsized":"2023-10-06T04:59:05.023617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.562815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014939+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030809+0000","last_change":"2023-10-06T04:59:05.030809+0000","last_active":"2023-10-06T04:59:05.030809+0000","last_peered":"2023-10-06T04:59:05.030809+0000","last_clean":"2023-10-06T04:59:05.030809+0000","last_became_active":"2023-10-06T04:59:05.030550+0000","last_became_peered":"2023-10-06T04:59:05.030550+0000","last_unstale":"2023-10-06T04:59:05.030809+0000","last_undegraded":"2023-10-06T04:59:05.030809+0000","last_fullsized":"2023-10-06T04:59:05.030809+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011439+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:19.436490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024000+0000","last_change":"2023-10-06T04:59:05.024000+0000","last_active":"2023-10-06T04:59:05.024000+0000","last_peered":"2023-10-06T04:59:05.024000+0000","last_clean":"2023-10-06T04:59:05.024000+0000","last_became_active":"2023-10-06T04:59:05.023706+0000","last_became_peered":"2023-10-06T04:59:05.023706+0000","last_unstale":"2023-10-06T04:59:05.024000+0000","last_undegraded":"2023-10-06T04:59:05.024000+0000","last_fullsized":"2023-10-06T04:59:05.024000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014554+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:50:43.443720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021204+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:05.021204+0000","last_peered":"2023-10-06T04:59:05.021204+0000","last_clean":"2023-10-06T04:59:05.021204+0000","last_became_active":"2023-10-06T04:59:05.020967+0000","last_became_peered":"2023-10-06T04:59:05.020967+0000","last_unstale":"2023-10-06T04:59:05.021204+0000","last_undegraded":"2023-10-06T04:59:05.021204+0000","last_fullsized":"2023-10-06T04:59:05.021204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:54.137958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016972+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025368+0000","last_change":"2023-10-06T04:59:05.025368+0000","last_active":"2023-10-06T04:59:05.025368+0000","last_peered":"2023-10-06T04:59:05.025368+0000","last_clean":"2023-10-06T04:59:05.025368+0000","last_became_active":"2023-10-06T04:59:05.025101+0000","last_became_peered":"2023-10-06T04:59:05.025101+0000","last_unstale":"2023-10-06T04:59:05.025368+0000","last_undegraded":"2023-10-06T04:59:05.025368+0000","last_fullsized":"2023-10-06T04:59:05.025368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012737+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:43.571901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.021703+0000","last_change":"2023-10-06T04:59:05.021703+0000","last_active":"2023-10-06T04:59:05.021703+0000","last_peered":"2023-10-06T04:59:05.021703+0000","last_clean":"2023-10-06T04:59:05.021703+0000","last_became_active":"2023-10-06T04:59:05.021413+0000","last_became_peered":"2023-10-06T04:59:05.021413+0000","last_unstale":"2023-10-06T04:59:05.021703+0000","last_undegraded":"2023-10-06T04:59:05.021703+0000","last_fullsized":"2023-10-06T04:59:05.021703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:15.699249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013498+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.016394+0000","last_change":"2023-10-06T04:59:05.016394+0000","last_active":"2023-10-06T04:59:05.016394+0000","last_peered":"2023-10-06T04:59:05.016394+0000","last_clean":"2023-10-06T04:59:05.016394+0000","last_became_active":"2023-10-06T04:59:05.016106+0000","last_became_peered":"2023-10-06T04:59:05.016106+0000","last_unstale":"2023-10-06T04:59:05.016394+0000","last_undegraded":"2023-10-06T04:59:05.016394+0000","last_fullsized":"2023-10-06T04:59:05.016394+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:10.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014320+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:50.435166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026950+0000","last_change":"2023-10-06T04:59:05.026950+0000","last_active":"2023-10-06T04:59:05.026950+0000","last_peered":"2023-10-06T04:59:05.026950+0000","last_clean":"2023-10-06T04:59:05.026950+0000","last_became_active":"2023-10-06T04:59:05.026723+0000","last_became_peered":"2023-10-06T04:59:05.026723+0000","last_unstale":"2023-10-06T04:59:05.026950+0000","last_undegraded":"2023-10-06T04:59:05.026950+0000","last_fullsized":"2023-10-06T04:59:05.026950+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014381+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025160+0000","last_change":"2023-10-06T04:59:05.025160+0000","last_active":"2023-10-06T04:59:05.025160+0000","last_peered":"2023-10-06T04:59:05.025160+0000","last_clean":"2023-10-06T04:59:05.025160+0000","last_became_active":"2023-10-06T04:59:05.024909+0000","last_became_peered":"2023-10-06T04:59:05.024909+0000","last_unstale":"2023-10-06T04:59:05.025160+0000","last_undegraded":"2023-10-06T04:59:05.025160+0000","last_fullsized":"2023-10-06T04:59:05.025160+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:00.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016608+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023984+0000","last_change":"2023-10-06T04:59:05.023984+0000","last_active":"2023-10-06T04:59:05.023984+0000","last_peered":"2023-10-06T04:59:05.023984+0000","last_clean":"2023-10-06T04:59:05.023984+0000","last_became_active":"2023-10-06T04:59:05.023641+0000","last_became_peered":"2023-10-06T04:59:05.023641+0000","last_unstale":"2023-10-06T04:59:05.023984+0000","last_undegraded":"2023-10-06T04:59:05.023984+0000","last_fullsized":"2023-10-06T04:59:05.023984+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:54.810243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015507+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.531534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.025232+0000","last_change":"2023-10-06T04:59:05.025232+0000","last_active":"2023-10-06T04:59:05.025232+0000","last_peered":"2023-10-06T04:59:05.025232+0000","last_clean":"2023-10-06T04:59:05.025232+0000","last_became_active":"2023-10-06T04:59:05.024850+0000","last_became_peered":"2023-10-06T04:59:05.024850+0000","last_unstale":"2023-10-06T04:59:05.025232+0000","last_undegraded":"2023-10-06T04:59:05.025232+0000","last_fullsized":"2023-10-06T04:59:05.025232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012744+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:58.844389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.020554+0000","last_change":"2023-10-06T04:59:05.020554+0000","last_active":"2023-10-06T04:59:05.020554+0000","last_peered":"2023-10-06T04:59:05.020554+0000","last_clean":"2023-10-06T04:59:05.020554+0000","last_became_active":"2023-10-06T04:59:05.020294+0000","last_became_peered":"2023-10-06T04:59:05.020294+0000","last_unstale":"2023-10-06T04:59:05.020554+0000","last_undegraded":"2023-10-06T04:59:05.020554+0000","last_fullsized":"2023-10-06T04:59:05.020554+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:24.547416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011224+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.022660+0000","last_change":"2023-10-06T04:59:05.022660+0000","last_active":"2023-10-06T04:59:05.022660+0000","last_peered":"2023-10-06T04:59:05.022660+0000","last_clean":"2023-10-06T04:59:05.022660+0000","last_became_active":"2023-10-06T04:59:05.021894+0000","last_became_peered":"2023-10-06T04:59:05.021894+0000","last_unstale":"2023-10-06T04:59:05.022660+0000","last_undegraded":"2023-10-06T04:59:05.022660+0000","last_fullsized":"2023-10-06T04:59:05.022660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.654900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014857+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:41.853665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.022693+0000","last_change":"2023-10-06T04:59:05.022693+0000","last_active":"2023-10-06T04:59:05.022693+0000","last_peered":"2023-10-06T04:59:05.022693+0000","last_clean":"2023-10-06T04:59:05.022693+0000","last_became_active":"2023-10-06T04:59:05.022421+0000","last_became_peered":"2023-10-06T04:59:05.022421+0000","last_unstale":"2023-10-06T04:59:05.022693+0000","last_undegraded":"2023-10-06T04:59:05.022693+0000","last_fullsized":"2023-10-06T04:59:05.022693+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:59.701496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.010770+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:47.264143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024903+0000","last_change":"2023-10-06T04:59:05.024903+0000","last_active":"2023-10-06T04:59:05.024903+0000","last_peered":"2023-10-06T04:59:05.024903+0000","last_clean":"2023-10-06T04:59:05.024903+0000","last_became_active":"2023-10-06T04:59:05.024608+0000","last_became_peered":"2023-10-06T04:59:05.024608+0000","last_unstale":"2023-10-06T04:59:05.024903+0000","last_undegraded":"2023-10-06T04:59:05.024903+0000","last_fullsized":"2023-10-06T04:59:05.024903+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:48.099761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016917+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:11.903244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030722+0000","last_change":"2023-10-06T04:59:05.030722+0000","last_active":"2023-10-06T04:59:05.030722+0000","last_peered":"2023-10-06T04:59:05.030722+0000","last_clean":"2023-10-06T04:59:05.030722+0000","last_became_active":"2023-10-06T04:59:05.030276+0000","last_became_peered":"2023-10-06T04:59:05.030276+0000","last_unstale":"2023-10-06T04:59:05.030722+0000","last_undegraded":"2023-10-06T04:59:05.030722+0000","last_fullsized":"2023-10-06T04:59:05.030722+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:57.328563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017462+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024303+0000","last_change":"2023-10-06T04:59:05.024303+0000","last_active":"2023-10-06T04:59:05.024303+0000","last_peered":"2023-10-06T04:59:05.024303+0000","last_clean":"2023-10-06T04:59:05.024303+0000","last_became_active":"2023-10-06T04:59:05.024038+0000","last_became_peered":"2023-10-06T04:59:05.024038+0000","last_unstale":"2023-10-06T04:59:05.024303+0000","last_undegraded":"2023-10-06T04:59:05.024303+0000","last_fullsized":"2023-10-06T04:59:05.024303+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012621+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:05.658324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023856+0000","last_change":"2023-10-06T04:59:05.023856+0000","last_active":"2023-10-06T04:59:05.023856+0000","last_peered":"2023-10-06T04:59:05.023856+0000","last_clean":"2023-10-06T04:59:05.023856+0000","last_became_active":"2023-10-06T04:59:05.023612+0000","last_became_peered":"2023-10-06T04:59:05.023612+0000","last_unstale":"2023-10-06T04:59:05.023856+0000","last_undegraded":"2023-10-06T04:59:05.023856+0000","last_fullsized":"2023-10-06T04:59:05.023856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:04.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.012763+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:58.906683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023970+0000","last_change":"2023-10-06T04:59:05.023970+0000","last_active":"2023-10-06T04:59:05.023970+0000","last_peered":"2023-10-06T04:59:05.023970+0000","last_clean":"2023-10-06T04:59:05.023970+0000","last_became_active":"2023-10-06T04:59:05.023712+0000","last_became_peered":"2023-10-06T04:59:05.023712+0000","last_unstale":"2023-10-06T04:59:05.023970+0000","last_undegraded":"2023-10-06T04:59:05.023970+0000","last_fullsized":"2023-10-06T04:59:05.023970+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:31.733624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.017238+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028495+0000","last_change":"2023-10-06T04:59:05.028495+0000","last_active":"2023-10-06T04:59:05.028495+0000","last_peered":"2023-10-06T04:59:05.028495+0000","last_clean":"2023-10-06T04:59:05.028495+0000","last_became_active":"2023-10-06T04:59:05.028219+0000","last_became_peered":"2023-10-06T04:59:05.028219+0000","last_unstale":"2023-10-06T04:59:05.028495+0000","last_undegraded":"2023-10-06T04:59:05.028495+0000","last_fullsized":"2023-10-06T04:59:05.028495+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:54.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016324+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023535+0000","last_change":"2023-10-06T04:59:05.023535+0000","last_active":"2023-10-06T04:59:05.023535+0000","last_peered":"2023-10-06T04:59:05.023535+0000","last_clean":"2023-10-06T04:59:05.023535+0000","last_became_active":"2023-10-06T04:59:05.023261+0000","last_became_peered":"2023-10-06T04:59:05.023261+0000","last_unstale":"2023-10-06T04:59:05.023535+0000","last_undegraded":"2023-10-06T04:59:05.023535+0000","last_fullsized":"2023-10-06T04:59:05.023535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:47.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011797+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.065649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.026906+0000","last_change":"2023-10-06T04:59:05.026906+0000","last_active":"2023-10-06T04:59:05.026906+0000","last_peered":"2023-10-06T04:59:05.026906+0000","last_clean":"2023-10-06T04:59:05.026906+0000","last_became_active":"2023-10-06T04:59:05.026633+0000","last_became_peered":"2023-10-06T04:59:05.026633+0000","last_unstale":"2023-10-06T04:59:05.026906+0000","last_undegraded":"2023-10-06T04:59:05.026906+0000","last_fullsized":"2023-10-06T04:59:05.026906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:56.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.013419+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:59:04.013419+0000","last_peered":"2023-10-06T04:59:04.013419+0000","last_clean":"2023-10-06T04:59:04.013419+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:59:04.013419+0000","last_undegraded":"2023-10-06T04:59:04.013419+0000","last_fullsized":"2023-10-06T04:59:04.013419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.011969+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:31.872736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028380+0000","last_change":"2023-10-06T04:59:05.028380+0000","last_active":"2023-10-06T04:59:05.028380+0000","last_peered":"2023-10-06T04:59:05.028380+0000","last_clean":"2023-10-06T04:59:05.028380+0000","last_became_active":"2023-10-06T04:59:05.028098+0000","last_became_peered":"2023-10-06T04:59:05.028098+0000","last_unstale":"2023-10-06T04:59:05.028380+0000","last_undegraded":"2023-10-06T04:59:05.028380+0000","last_fullsized":"2023-10-06T04:59:05.028380+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:22.276266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.007290+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:59:04.007290+0000","last_peered":"2023-10-06T04:59:04.007290+0000","last_clean":"2023-10-06T04:59:04.007290+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:59:04.007290+0000","last_undegraded":"2023-10-06T04:59:04.007290+0000","last_fullsized":"2023-10-06T04:59:04.007290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014304+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:08.338740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.024110+0000","last_change":"2023-10-06T04:59:05.024110+0000","last_active":"2023-10-06T04:59:05.024110+0000","last_peered":"2023-10-06T04:59:05.024110+0000","last_clean":"2023-10-06T04:59:05.024110+0000","last_became_active":"2023-10-06T04:59:05.023840+0000","last_became_peered":"2023-10-06T04:59:05.023840+0000","last_unstale":"2023-10-06T04:59:05.024110+0000","last_undegraded":"2023-10-06T04:59:05.024110+0000","last_fullsized":"2023-10-06T04:59:05.024110+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.014748+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:59:04.014748+0000","last_peered":"2023-10-06T04:59:04.014748+0000","last_clean":"2023-10-06T04:59:04.014748+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:59:04.014748+0000","last_undegraded":"2023-10-06T04:59:04.014748+0000","last_fullsized":"2023-10-06T04:59:04.014748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":222,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010191+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:59:04.010191+0000","last_peered":"2023-10-06T04:59:04.010191+0000","last_clean":"2023-10-06T04:59:04.010191+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:59:04.010191+0000","last_undegraded":"2023-10-06T04:59:04.010191+0000","last_fullsized":"2023-10-06T04:59:04.010191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.015895+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:16.704156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.030905+0000","last_change":"2023-10-06T04:59:05.030905+0000","last_active":"2023-10-06T04:59:05.030905+0000","last_peered":"2023-10-06T04:59:05.030905+0000","last_clean":"2023-10-06T04:59:05.030905+0000","last_became_active":"2023-10-06T04:59:05.030530+0000","last_became_peered":"2023-10-06T04:59:05.030530+0000","last_unstale":"2023-10-06T04:59:05.030905+0000","last_undegraded":"2023-10-06T04:59:05.030905+0000","last_fullsized":"2023-10-06T04:59:05.030905+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:15.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":72,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010382+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:59:04.010382+0000","last_peered":"2023-10-06T04:59:04.010382+0000","last_clean":"2023-10-06T04:59:04.010382+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:59:04.010382+0000","last_undegraded":"2023-10-06T04:59:04.010382+0000","last_fullsized":"2023-10-06T04:59:04.010382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016145+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.815150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028526+0000","last_change":"2023-10-06T04:59:05.028526+0000","last_active":"2023-10-06T04:59:05.028526+0000","last_peered":"2023-10-06T04:59:05.028526+0000","last_clean":"2023-10-06T04:59:05.028526+0000","last_became_active":"2023-10-06T04:59:05.028134+0000","last_became_peered":"2023-10-06T04:59:05.028134+0000","last_unstale":"2023-10-06T04:59:05.028526+0000","last_undegraded":"2023-10-06T04:59:05.028526+0000","last_fullsized":"2023-10-06T04:59:05.028526+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:16.407534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010554+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:59:04.010554+0000","last_peered":"2023-10-06T04:59:04.010554+0000","last_clean":"2023-10-06T04:59:04.010554+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:59:04.010554+0000","last_undegraded":"2023-10-06T04:59:04.010554+0000","last_fullsized":"2023-10-06T04:59:04.010554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.016082+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:58.333470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.023805+0000","last_change":"2023-10-06T04:59:05.023805+0000","last_active":"2023-10-06T04:59:05.023805+0000","last_peered":"2023-10-06T04:59:05.023805+0000","last_clean":"2023-10-06T04:59:05.023805+0000","last_became_active":"2023-10-06T04:59:05.023560+0000","last_became_peered":"2023-10-06T04:59:05.023560+0000","last_unstale":"2023-10-06T04:59:05.023805+0000","last_undegraded":"2023-10-06T04:59:05.023805+0000","last_fullsized":"2023-10-06T04:59:05.023805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:16.588598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.010472+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:59:04.010472+0000","last_peered":"2023-10-06T04:59:04.010472+0000","last_clean":"2023-10-06T04:59:04.010472+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:59:04.010472+0000","last_undegraded":"2023-10-06T04:59:04.010472+0000","last_fullsized":"2023-10-06T04:59:04.010472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.013631+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:38.572727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.020425+0000","last_change":"2023-10-06T04:59:05.020425+0000","last_active":"2023-10-06T04:59:05.020425+0000","last_peered":"2023-10-06T04:59:05.020425+0000","last_clean":"2023-10-06T04:59:05.020425+0000","last_became_active":"2023-10-06T04:59:05.020172+0000","last_became_peered":"2023-10-06T04:59:05.020172+0000","last_unstale":"2023-10-06T04:59:05.020425+0000","last_undegraded":"2023-10-06T04:59:05.020425+0000","last_fullsized":"2023-10-06T04:59:05.020425+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:25.681695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.011548+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:59:04.011548+0000","last_peered":"2023-10-06T04:59:04.011548+0000","last_clean":"2023-10-06T04:59:04.011548+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:59:04.011548+0000","last_undegraded":"2023-10-06T04:59:04.011548+0000","last_fullsized":"2023-10-06T04:59:04.011548+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:59:05.004991+0000","last_change":"2023-10-06T04:59:05.014748+0000","last_active":"2023-10-06T04:59:05.004991+0000","last_peered":"2023-10-06T04:59:05.004991+0000","last_clean":"2023-10-06T04:59:05.004991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:59:05.004991+0000","last_undegraded":"2023-10-06T04:59:05.004991+0000","last_fullsized":"2023-10-06T04:59:05.004991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":8,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-06T04:59:05.028402+0000","last_change":"2023-10-06T04:59:05.028402+0000","last_active":"2023-10-06T04:59:05.028402+0000","last_peered":"2023-10-06T04:59:05.028402+0000","last_clean":"2023-10-06T04:59:05.028402+0000","last_became_active":"2023-10-06T04:59:05.028139+0000","last_became_peered":"2023-10-06T04:59:05.028139+0000","last_unstale":"2023-10-06T04:59:05.028402+0000","last_undegraded":"2023-10-06T04:59:05.028402+0000","last_fullsized":"2023-10-06T04:59:05.028402+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:56.970617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-06T04:59:04.011518+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:59:04.011518+0000","last_peered":"2023-10-06T04:59:04.011518+0000","last_clean":"2023-10-06T04:59:04.011518+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:59:04.011518+0000","last_undegraded":"2023-10-06T04:59:04.011518+0000","last_fullsized":"2023-10-06T04:59:04.011518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":1104597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.73399999999999999},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73199999999999998,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.495}]},{"osd":1,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":1.073},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.51600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":1.111},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.58999999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.503},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.441}]},{"osd":5,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.502},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":1.214},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302520,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069320,"statfs":{"total":96636764160,"available":96326983680,"internally_reserved":0,"allocated":1163264,"data_stored":522198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.52200000000000002},"min":{"1min":0.54900000000000004,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.81200000000000006,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.57199999999999995},"min":{"1min":0.56100000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.39800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":1.234,"15min":1.234},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.55200000000000005},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81000000000000005,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75800000000000001,"5min":3.669,"15min":3.669},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.495},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":1104597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.23000000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.27900000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.376}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.501,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.316},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.70999999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.85499999999999998,"15min":0.92900000000000005},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.64100000000000001},"min":{"1min":0.47499999999999998,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.998,"15min":0.998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.65800000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":1.0900000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80600000000000005,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.026},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.79400000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.505,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57699999999999996,"5min":0.628,"15min":0.628},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.441}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90700000000000003,"5min":1.022,"15min":1.022},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.495,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.5,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96599999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.995,"15min":0.995},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302520,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069320,"statfs":{"total":96636764160,"available":96326983680,"internally_reserved":0,"allocated":1163264,"data_stored":522198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.51200000000000001,"5min":0.747,"15min":1.048},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.42599999999999999,"15min":0.39000000000000001},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.375}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":0.61699999999999999,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.309},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.36099999999999999},"min":{"1min":0.29499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.439}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.5,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38700000000000001,"15min":0.40600000000000003},"min":{"1min":0.251,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":2.008,"15min":2.008},"last":0.316},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41299999999999998,"15min":0.56499999999999995},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54500000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42399999999999999,"15min":0.46000000000000002},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.64600000000000002,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":1.379,"15min":5.069},"last":0.443},{"interface":"front","average":{"1min":0.443,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":2.246,"15min":2.246},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83199999999999996,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82199999999999995,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.79400000000000004},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.40400000000000003},"max":{"1min":0.76200000000000001,"5min":1.964,"15min":6.367},"last":0.623},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65000000000000002,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":1.3,"15min":5},"last":0.754}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306604,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065236,"statfs":{"total":96636764160,"available":96322801664,"internally_reserved":0,"allocated":1150976,"data_stored":515582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.379},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.78500000000000003},"min":{"1min":0.38200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.82099999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.80400000000000005,"15min":5.415},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.66700000000000004},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":4.202},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":1.044,"15min":4.2130000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40699999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.42499999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.77600000000000002,"5min":0.86199999999999999,"15min":0.872},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.313},"max":{"1min":0.84399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306612,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065228,"statfs":{"total":96636764160,"available":96322793472,"internally_reserved":0,"allocated":1159168,"data_stored":520845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.67600000000000005},"min":{"1min":0.36899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.625},"min":{"1min":0.436,"5min":0.247,"15min":0.247},"max":{"1min":0.76000000000000001,"5min":0.80800000000000005,"15min":3.7679999999999998},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.70799999999999996},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":6.7949999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.75800000000000001},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.93799999999999994},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":9.7210000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.93500000000000005},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":9.7539999999999996},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.504},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.34200000000000003},"max":{"1min":0.66300000000000003,"5min":0.746,"15min":0.746},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80100000000000005},"last":0.46200000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:59:10.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2023-10-06T04:59:10.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.021652+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:11:40.965728+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2023-10-06T04:59:10.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.018066+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:30:31.156638+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.017806+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:29:55.185806+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016696+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:07:49.104668+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016366+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:04:24.260605+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014934+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:37:49.295778+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2023-10-06T04:59:10.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.019752+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:41:36.653367+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.019396+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:37:27.219788+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.017920+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:13:44.174946+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014935+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:14:01.732144+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016919+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-10-06T04:59:10.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011338+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:54:13.415770+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014693+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016598+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.018596+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2023-10-06T04:59:10.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.017837+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.015755+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:13:26.205356+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.015002+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:55:58.910125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012429+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:28:41.792782+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.010492+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:31:49.048000+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2023-10-06T04:59:10.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014331+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:23:02.587278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.013637+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:04:54.489379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.013020+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:39:36.271837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.015735+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011571+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:40:21.883224+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:59:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012207+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:00:22.143287+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016324+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:16:58.288670+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.015319+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:56:10.609306+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.013796+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:25:07.976219+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-10-06T04:59:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011941+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:28:59.866311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.015126+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:20:22.404213+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016613+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:47:01.205679+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012301+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:34:17.016243+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012428+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:10:23.215609+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011347+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:46:28.217935+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014483+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:26:19.567722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014939+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:18:10.116830+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011439+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:11:19.436490+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2023-10-06T04:59:10.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014554+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:50:43.443720+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016972+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012737+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:15:43.571901+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.013498+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014320+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:15:50.435166+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-10-06T04:59:10.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014381+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:59:10.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-10-06T04:59:10.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016608+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:51:49.143630+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:59:10.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2023-10-06T04:59:10.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.015507+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:28:37.531534+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-10-06T04:59:10.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012744+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:07:58.844389+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011224+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014857+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:01:41.853665+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.010770+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:22:47.264143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016917+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:19:11.903244+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:59:10.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.017462+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:22:02.584723+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012621+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:35:05.658324+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.012763+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:44:58.906683+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.017238+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016324+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:51:12.538256+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:59:10.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011797+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:13:45.065649+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.011969+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:31:31.872736+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014304+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:17:08.338740+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.015895+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:43:16.704156+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-10-06T04:59:10.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016145+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:01:55.815150+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T04:59:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-10-06T04:59:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.016082+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:48:58.333470+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-10-06T04:59:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.013631+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:25:38.572727+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T04:59:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-10-06T04:59:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T04:59:05.004991+0000', 'last_change': '2023-10-06T04:59:05.014748+0000', 'last_active': '2023-10-06T04:59:05.004991+0000', 'last_peered': '2023-10-06T04:59:05.004991+0000', 'last_clean': '2023-10-06T04:59:05.004991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:59:05.004991+0000', 'last_undegraded': '2023-10-06T04:59:05.004991+0000', 'last_fullsized': '2023-10-06T04:59:05.004991+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'last_clean_scrub_stamp': '2023-10-06T04:59:05.004991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:51:57.340513+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T04:59:10.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T04:59:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.066+0000 7effbcd44700 1 Processor -- start 2023-10-06T04:59:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.066+0000 7effbcd44700 1 -- start start 2023-10-06T04:59:11.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.067+0000 7effbcd44700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 0x7effb815dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.067+0000 7effbcd44700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80cf2a0 con 0x7effb80afed0 2023-10-06T04:59:11.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.067+0000 7effbcd44700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80ce970 con 0x7effb815c7b0 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.067+0000 7effbcd44700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80cc7d0 con 0x7effb815cb80 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.067+0000 7effb6d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7effb80afed0 0x7effb80cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34094/0 (socket says 172.21.15.3:34094) 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb6d9d700 1 -- 172.21.15.3:0/3173977706 learned_addr learned my addr 172.21.15.3:0/3173977706 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207282287 0 0) 0x7effb80cf2a0 con 0x7effb80afed0 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb5d9b700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 0x7effb815dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa4003570 con 0x7effb80afed0 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 802398115 0 0) 0x7effa4003570 con 0x7effb80afed0 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 msgr2=0x7effb815dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 0x7effb815dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 >> v1:172.21.15.153:6789/0 conn(0x7effb815c7b0 legacy=0x7effb80cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80cec50 con 0x7effb80afed0 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7effb0001470 con 0x7effb80afed0 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 >> v1:172.21.15.3:6789/0 conn(0x7effb80afed0 legacy=0x7effb80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa4003a90 0x7effa4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 0x7effa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa4004d50 0x7effa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb80cc7d0 con 0x7effa4004d50 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa400bba0 con 0x7effa4003a90 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa400bd60 con 0x7effb815cb80 2023-10-06T04:59:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb6d9d700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa4004d50 0x7effa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb5d9b700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 0x7effa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.068+0000 7effb659c700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa4003a90 0x7effa4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb6d9d700 1 -- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 msgr2=0x7effa4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb6d9d700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effb815cb80 0x7effa4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb6d9d700 1 -- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa4003a90 msgr2=0x7effa4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb6d9d700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa4003a90 0x7effa4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb6d9d700 1 -- 172.21.15.3:0/3173977706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4003570 con 0x7effa4004d50 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb6d9d700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa4004d50 0x7effa40051c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7effb000c3c0 tx=0x7effb0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f625d59a03415b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000cc90 con 0x7effa4004d50 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb000cdf0 con 0x7effa4004d50 2023-10-06T04:59:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb559a700 1 -- 172.21.15.3:0/3173977706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0005b50 con 0x7effa4004d50 2023-10-06T04:59:11.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effbcd44700 1 -- 172.21.15.3:0/3173977706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa4004d50 msgr2=0x7effa40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effbcd44700 1 --2- 172.21.15.3:0/3173977706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa4004d50 0x7effa40051c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7effb000c3c0 tx=0x7effb0001e50 comp rx=0 tx=0).stop 2023-10-06T04:59:11.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effb6d9d700 1 -- 172.21.15.3:0/3173977706 reap_dead start 2023-10-06T04:59:11.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effbcd44700 1 -- 172.21.15.3:0/3173977706 shutdown_connections 2023-10-06T04:59:11.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.069+0000 7effbcd44700 1 -- 172.21.15.3:0/3173977706 >> 172.21.15.3:0/3173977706 conn(0x7effb8070310 msgr2=0x7effb8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:11.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.070+0000 7effbcd44700 1 -- 172.21.15.3:0/3173977706 shutdown_connections 2023-10-06T04:59:11.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.070+0000 7effbcd44700 1 -- 172.21.15.3:0/3173977706 wait complete. 2023-10-06T04:59:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 Processor -- start 2023-10-06T04:59:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 -- start start 2023-10-06T04:59:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effa4004d50 0x7effb814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb80c7550 0x7effb80c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effb8158080 0x7effb815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb80c7bf0 con 0x7effb80c7550 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb814b4b0 con 0x7effb8158080 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effbcd44700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80c7e40 con 0x7effa4004d50 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effb5d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb80c7550 0x7effb80c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effb5d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb80c7550 0x7effb80c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34468/0 (socket says 172.21.15.3:34468) 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.071+0000 7effb5d9b700 1 -- 172.21.15.3:0/3968340653 learned_addr learned my addr 172.21.15.3:0/3968340653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb6d9d700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effb8158080 0x7effb815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb659c700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effa4004d50 0x7effb814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb5d9b700 1 -- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effa4004d50 msgr2=0x7effb814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb5d9b700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effa4004d50 0x7effb814af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb5d9b700 1 -- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effb8158080 msgr2=0x7effb815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb5d9b700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effb8158080 0x7effb815f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb5d9b700 1 -- 172.21.15.3:0/3968340653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80c8120 con 0x7effb80c7550 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb6d9d700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effb8158080 0x7effb815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effb5d9b700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb80c7550 0x7effb80c8c00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7effa0006ac0 tx=0x7effa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effae7fc700 1 -- 172.21.15.3:0/3968340653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000fa10 con 0x7effb80c7550 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effae7fc700 1 -- 172.21.15.3:0/3968340653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa000fb70 con 0x7effb80c7550 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb80c8400 con 0x7effb80c7550 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effae7fc700 1 -- 172.21.15.3:0/3968340653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0014440 con 0x7effb80c7550 2023-10-06T04:59:11.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.072+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb80c9140 con 0x7effb80c7550 2023-10-06T04:59:11.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.073+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb8071680 con 0x7effb80c7550 2023-10-06T04:59:11.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.074+0000 7effae7fc700 1 -- 172.21.15.3:0/3968340653 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effa0014870 con 0x7effb80c7550 2023-10-06T04:59:11.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.074+0000 7effae7fc700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff98066f30 0x7eff980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.074+0000 7effae7fc700 1 -- 172.21.15.3:0/3968340653 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7effa0018030 con 0x7effb80c7550 2023-10-06T04:59:11.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.074+0000 7effb659c700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff98066f30 0x7eff980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.075+0000 7effb659c700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff98066f30 0x7eff980693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7effa8001750 tx=0x7effa8001640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:11.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.078+0000 7effae7fc700 1 -- 172.21.15.3:0/3968340653 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effa00522b0 con 0x7effb80c7550 2023-10-06T04:59:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.352+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7effb80cef30 con 0x7effb80c7550 2023-10-06T04:59:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.352+0000 7effae7fc700 1 -- 172.21.15.3:0/3968340653 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1037 (secure 0 0 0) 0x7effa0022420 con 0x7effb80c7550 2023-10-06T04:59:11.355 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:11.355 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":431,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1696567930,"num_in_osds":8,"osd_in_since":1696567922,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2504523776,"bytes_avail":770589589504,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":59,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7071}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-10-06T04:56:37.486481+0000","services":{}},"progress_events":{}} 2023-10-06T04:59:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff98066f30 msgr2=0x7eff980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff98066f30 0x7eff980693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7effa8001750 tx=0x7effa8001640 comp rx=0 tx=0).stop 2023-10-06T04:59:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb80c7550 msgr2=0x7effb80c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb80c7550 0x7effb80c8c00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7effa0006ac0 tx=0x7effa00091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 shutdown_connections 2023-10-06T04:59:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff98066f30 0x7eff980693e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effb8158080 0x7effb815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effa4004d50 0x7effb814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 --2- 172.21.15.3:0/3968340653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb80c7550 0x7effb80c8c00 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 >> 172.21.15.3:0/3968340653 conn(0x7effb8070310 msgr2=0x7effb815d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 shutdown_connections 2023-10-06T04:59:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.354+0000 7effbcd44700 1 -- 172.21.15.3:0/3968340653 wait complete. 2023-10-06T04:59:11.369 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T04:59:11.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:59:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.485+0000 7f44a9dd5700 1 Processor -- start 2023-10-06T04:59:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.485+0000 7f44a9dd5700 1 -- start start 2023-10-06T04:59:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.486+0000 7f44a9dd5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44a415c8f0 0x7f44a415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.486+0000 7f44a9dd5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 0x7f44a40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.486+0000 7f44a9dd5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a40cf6c0 con 0x7f44a415ccc0 2023-10-06T04:59:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.486+0000 7f44a9dd5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a40cfb50 con 0x7f44a415c8f0 2023-10-06T04:59:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.486+0000 7f44a9dd5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a415e050 con 0x7f44a40d0870 2023-10-06T04:59:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.486+0000 7f44a2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44a415ccc0 0x7f44a40d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34100/0 (socket says 172.21.15.3:34100) 2023-10-06T04:59:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.486+0000 7f44a2ffd700 1 -- 172.21.15.3:0/4219094631 learned_addr learned my addr 172.21.15.3:0/4219094631 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a37fe700 1 --2- 172.21.15.3:0/4219094631 >> v2:172.21.15.153:3300/0 conn(0x7f44a415c8f0 0x7f44a415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a3fff700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 0x7f44a40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231647925 0 0) 0x7f44a40cf6c0 con 0x7f44a415ccc0 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f448c003570 con 0x7f44a415ccc0 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981873641 0 0) 0x7f448c003570 con 0x7f44a415ccc0 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 msgr2=0x7f44a40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 0x7f44a40b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 >> v2:172.21.15.153:3300/0 conn(0x7f44a415c8f0 msgr2=0x7f44a415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 --2- 172.21.15.3:0/4219094631 >> v2:172.21.15.153:3300/0 conn(0x7f44a415c8f0 0x7f44a415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40ceab0 con 0x7f44a415ccc0 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f4498001470 con 0x7f44a415ccc0 2023-10-06T04:59:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 >> v1:172.21.15.3:6789/0 conn(0x7f44a415ccc0 legacy=0x7f44a40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f448c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 0x7f448c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c004d50 0x7f448c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40cfb50 con 0x7f448c003a60 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a415e050 con 0x7f448c004d50 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c00ba60 con 0x7f44a40d0870 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a37fe700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f448c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a3fff700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c004d50 0x7f448c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a2ffd700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 0x7f448c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a37fe700 1 -- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 msgr2=0x7f448c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:11.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a37fe700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a40d0870 0x7f448c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a37fe700 1 -- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c004d50 msgr2=0x7f448c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a37fe700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c004d50 0x7f448c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a37fe700 1 -- 172.21.15.3:0/4219094631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c003570 con 0x7f448c003a60 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.487+0000 7f44a3fff700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c004d50 0x7f448c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a37fe700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f448c003e70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4494009c00 tx=0x7f44940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4335123615aa3125 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400e6b0 con 0x7f448c003a60 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449400e810 con 0x7f448c003a60 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a27fc700 1 -- 172.21.15.3:0/4219094631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400ebd0 con 0x7f448c003a60 2023-10-06T04:59:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a9dd5700 1 -- 172.21.15.3:0/4219094631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 msgr2=0x7f448c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a9dd5700 1 --2- 172.21.15.3:0/4219094631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f448c003e70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4494009c00 tx=0x7f44940091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a3fff700 1 -- 172.21.15.3:0/4219094631 reap_dead start 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a9dd5700 1 -- 172.21.15.3:0/4219094631 shutdown_connections 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a9dd5700 1 -- 172.21.15.3:0/4219094631 >> 172.21.15.3:0/4219094631 conn(0x7f44a4070310 msgr2=0x7f44a4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a9dd5700 1 -- 172.21.15.3:0/4219094631 shutdown_connections 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.488+0000 7f44a9dd5700 1 -- 172.21.15.3:0/4219094631 wait complete. 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 Processor -- start 2023-10-06T04:59:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 -- start start 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f44a415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a4147e90 0x7f44a4157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4157d60 0x7f44a41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a4154370 con 0x7f448c003a60 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a415a340 con 0x7f44a4157d60 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a9dd5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40c5cd0 con 0x7f44a4147e90 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f44a415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f44a415bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34482/0 (socket says 172.21.15.3:34482) 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a37fe700 1 -- 172.21.15.3:0/2015208691 learned_addr learned my addr 172.21.15.3:0/2015208691 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.490+0000 7f44a3fff700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4157d60 0x7f44a41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a2ffd700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a4147e90 0x7f44a4157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a37fe700 1 -- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a4147e90 msgr2=0x7f44a4157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a37fe700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a4147e90 0x7f44a4157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a37fe700 1 -- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4157d60 msgr2=0x7f44a41581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a37fe700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4157d60 0x7f44a41581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a37fe700 1 -- 172.21.15.3:0/2015208691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40c5fb0 con 0x7f448c003a60 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a3fff700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4157d60 0x7f44a41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a37fe700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f44a415bce0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f448c004730 tx=0x7f449401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f448b7fe700 1 -- 172.21.15.3:0/2015208691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494022070 con 0x7f448c003a60 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f448b7fe700 1 -- 172.21.15.3:0/2015208691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449400e980 con 0x7f448c003a60 2023-10-06T04:59:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a4159cf0 con 0x7f448c003a60 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f448b7fe700 1 -- 172.21.15.3:0/2015208691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494027470 con 0x7f448c003a60 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.491+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a4148300 con 0x7f448c003a60 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.492+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a4154370 con 0x7f448c003a60 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.492+0000 7f448b7fe700 1 -- 172.21.15.3:0/2015208691 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f449401c450 con 0x7f448c003a60 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.493+0000 7f448b7fe700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4490066f30 0x7f44900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.493+0000 7f448b7fe700 1 -- 172.21.15.3:0/2015208691 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4494086200 con 0x7f448c003a60 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.493+0000 7f44a2ffd700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4490066f30 0x7f44900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.493+0000 7f44a2ffd700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4490066f30 0x7f44900693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4498000e10 tx=0x7f4498009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.497+0000 7f448b7fe700 1 -- 172.21.15.3:0/2015208691 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44940533d0 con 0x7f448c003a60 2023-10-06T04:59:11.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.685+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44a415a190 con 0x7f4490066f30 2023-10-06T04:59:11.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.694+0000 7f448b7fe700 1 -- 172.21.15.3:0/2015208691 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311633 (secure 0 0 0) 0x7f44a415a190 con 0x7f4490066f30 2023-10-06T04:59:11.701 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4490066f30 msgr2=0x7f44900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4490066f30 0x7f44900693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4498000e10 tx=0x7f4498009040 comp rx=0 tx=0).stop 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 msgr2=0x7f44a415bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f44a415bce0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f448c004730 tx=0x7f449401a000 comp rx=0 tx=0).stop 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 shutdown_connections 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4490066f30 0x7f44900693e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4157d60 0x7f44a41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448c003a60 0x7f44a415bce0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 --2- 172.21.15.3:0/2015208691 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44a4147e90 0x7f44a4157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 >> 172.21.15.3:0/2015208691 conn(0x7f44a4070310 msgr2=0x7f44a4155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 shutdown_connections 2023-10-06T04:59:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:11.698+0000 7f44a9dd5700 1 -- 172.21.15.3:0/2015208691 wait complete. 2023-10-06T04:59:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:59:11.712 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-10-06T04:59:11.524977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2446140,"kb_used_data":10492,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752528580,"statfs":{"total":773094113280,"available":770589265920,"internally_reserved":0,"allocated":10743808,"data_stored":5468929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516812"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042247+0000","last_change":"2023-10-06T04:59:06.038068+0000","last_active":"2023-10-06T04:59:06.042247+0000","last_peered":"2023-10-06T04:59:06.042247+0000","last_clean":"2023-10-06T04:59:06.042247+0000","last_became_active":"2023-10-06T04:59:06.037779+0000","last_became_peered":"2023-10-06T04:59:06.037779+0000","last_unstale":"2023-10-06T04:59:06.042247+0000","last_undegraded":"2023-10-06T04:59:06.042247+0000","last_fullsized":"2023-10-06T04:59:06.042247+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:30.027634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043940+0000","last_change":"2023-10-06T04:59:06.034963+0000","last_active":"2023-10-06T04:59:06.043940+0000","last_peered":"2023-10-06T04:59:06.043940+0000","last_clean":"2023-10-06T04:59:06.043940+0000","last_became_active":"2023-10-06T04:59:06.034699+0000","last_became_peered":"2023-10-06T04:59:06.034699+0000","last_unstale":"2023-10-06T04:59:06.043940+0000","last_undegraded":"2023-10-06T04:59:06.043940+0000","last_fullsized":"2023-10-06T04:59:06.043940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:16.596514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.041916+0000","last_change":"2023-10-06T04:59:06.037800+0000","last_active":"2023-10-06T04:59:06.041916+0000","last_peered":"2023-10-06T04:59:06.041916+0000","last_clean":"2023-10-06T04:59:06.041916+0000","last_became_active":"2023-10-06T04:59:06.037521+0000","last_became_peered":"2023-10-06T04:59:06.037521+0000","last_unstale":"2023-10-06T04:59:06.041916+0000","last_undegraded":"2023-10-06T04:59:06.041916+0000","last_fullsized":"2023-10-06T04:59:06.041916+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:24.018318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.051282+0000","last_change":"2023-10-06T04:59:06.047011+0000","last_active":"2023-10-06T04:59:06.051282+0000","last_peered":"2023-10-06T04:59:06.051282+0000","last_clean":"2023-10-06T04:59:06.051282+0000","last_became_active":"2023-10-06T04:59:06.046495+0000","last_became_peered":"2023-10-06T04:59:06.046495+0000","last_unstale":"2023-10-06T04:59:06.051282+0000","last_undegraded":"2023-10-06T04:59:06.051282+0000","last_fullsized":"2023-10-06T04:59:06.051282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046019+0000","last_change":"2023-10-06T04:59:06.041490+0000","last_active":"2023-10-06T04:59:06.046019+0000","last_peered":"2023-10-06T04:59:06.046019+0000","last_clean":"2023-10-06T04:59:06.046019+0000","last_became_active":"2023-10-06T04:59:06.041177+0000","last_became_peered":"2023-10-06T04:59:06.041177+0000","last_unstale":"2023-10-06T04:59:06.046019+0000","last_undegraded":"2023-10-06T04:59:06.046019+0000","last_fullsized":"2023-10-06T04:59:06.046019+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:41.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044000+0000","last_change":"2023-10-06T04:59:06.037886+0000","last_active":"2023-10-06T04:59:06.044000+0000","last_peered":"2023-10-06T04:59:06.044000+0000","last_clean":"2023-10-06T04:59:06.044000+0000","last_became_active":"2023-10-06T04:59:06.037642+0000","last_became_peered":"2023-10-06T04:59:06.037642+0000","last_unstale":"2023-10-06T04:59:06.044000+0000","last_undegraded":"2023-10-06T04:59:06.044000+0000","last_fullsized":"2023-10-06T04:59:06.044000+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:33.236974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043474+0000","last_change":"2023-10-06T04:59:06.034089+0000","last_active":"2023-10-06T04:59:06.043474+0000","last_peered":"2023-10-06T04:59:06.043474+0000","last_clean":"2023-10-06T04:59:06.043474+0000","last_became_active":"2023-10-06T04:59:06.033808+0000","last_became_peered":"2023-10-06T04:59:06.033808+0000","last_unstale":"2023-10-06T04:59:06.043474+0000","last_undegraded":"2023-10-06T04:59:06.043474+0000","last_fullsized":"2023-10-06T04:59:06.043474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:16.545577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043070+0000","last_change":"2023-10-06T04:59:06.041845+0000","last_active":"2023-10-06T04:59:06.043070+0000","last_peered":"2023-10-06T04:59:06.043070+0000","last_clean":"2023-10-06T04:59:06.043070+0000","last_became_active":"2023-10-06T04:59:06.041514+0000","last_became_peered":"2023-10-06T04:59:06.041514+0000","last_unstale":"2023-10-06T04:59:06.043070+0000","last_undegraded":"2023-10-06T04:59:06.043070+0000","last_fullsized":"2023-10-06T04:59:06.043070+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:10.609306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.035114+0000","last_change":"2023-10-06T04:59:06.031918+0000","last_active":"2023-10-06T04:59:06.035114+0000","last_peered":"2023-10-06T04:59:06.035114+0000","last_clean":"2023-10-06T04:59:06.035114+0000","last_became_active":"2023-10-06T04:59:06.031641+0000","last_became_peered":"2023-10-06T04:59:06.031641+0000","last_unstale":"2023-10-06T04:59:06.035114+0000","last_undegraded":"2023-10-06T04:59:06.035114+0000","last_fullsized":"2023-10-06T04:59:06.035114+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:10.384995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039501+0000","last_change":"2023-10-06T04:59:06.036932+0000","last_active":"2023-10-06T04:59:06.039501+0000","last_peered":"2023-10-06T04:59:06.039501+0000","last_clean":"2023-10-06T04:59:06.039501+0000","last_became_active":"2023-10-06T04:59:06.036651+0000","last_became_peered":"2023-10-06T04:59:06.036651+0000","last_unstale":"2023-10-06T04:59:06.039501+0000","last_undegraded":"2023-10-06T04:59:06.039501+0000","last_fullsized":"2023-10-06T04:59:06.039501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:49.356661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044695+0000","last_change":"2023-10-06T04:59:06.034544+0000","last_active":"2023-10-06T04:59:06.044695+0000","last_peered":"2023-10-06T04:59:06.044695+0000","last_clean":"2023-10-06T04:59:06.044695+0000","last_became_active":"2023-10-06T04:59:06.034240+0000","last_became_peered":"2023-10-06T04:59:06.034240+0000","last_unstale":"2023-10-06T04:59:06.044695+0000","last_undegraded":"2023-10-06T04:59:06.044695+0000","last_fullsized":"2023-10-06T04:59:06.044695+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:16.704156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045766+0000","last_change":"2023-10-06T04:59:06.042730+0000","last_active":"2023-10-06T04:59:06.045766+0000","last_peered":"2023-10-06T04:59:06.045766+0000","last_clean":"2023-10-06T04:59:06.045766+0000","last_became_active":"2023-10-06T04:59:06.042188+0000","last_became_peered":"2023-10-06T04:59:06.042188+0000","last_unstale":"2023-10-06T04:59:06.045766+0000","last_undegraded":"2023-10-06T04:59:06.045766+0000","last_fullsized":"2023-10-06T04:59:06.045766+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:53.196280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039919+0000","last_change":"2023-10-06T04:59:06.035450+0000","last_active":"2023-10-06T04:59:06.039919+0000","last_peered":"2023-10-06T04:59:06.039919+0000","last_clean":"2023-10-06T04:59:06.039919+0000","last_became_active":"2023-10-06T04:59:06.035166+0000","last_became_peered":"2023-10-06T04:59:06.035166+0000","last_unstale":"2023-10-06T04:59:06.039919+0000","last_undegraded":"2023-10-06T04:59:06.039919+0000","last_fullsized":"2023-10-06T04:59:06.039919+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:19.436490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044413+0000","last_change":"2023-10-06T04:59:06.037669+0000","last_active":"2023-10-06T04:59:06.044413+0000","last_peered":"2023-10-06T04:59:06.044413+0000","last_clean":"2023-10-06T04:59:06.044413+0000","last_became_active":"2023-10-06T04:59:06.037389+0000","last_became_peered":"2023-10-06T04:59:06.037389+0000","last_unstale":"2023-10-06T04:59:06.044413+0000","last_undegraded":"2023-10-06T04:59:06.044413+0000","last_fullsized":"2023-10-06T04:59:06.044413+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.104668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046491+0000","last_change":"2023-10-06T04:59:06.044897+0000","last_active":"2023-10-06T04:59:06.046491+0000","last_peered":"2023-10-06T04:59:06.046491+0000","last_clean":"2023-10-06T04:59:06.046491+0000","last_became_active":"2023-10-06T04:59:06.044608+0000","last_became_peered":"2023-10-06T04:59:06.044608+0000","last_unstale":"2023-10-06T04:59:06.046491+0000","last_undegraded":"2023-10-06T04:59:06.046491+0000","last_fullsized":"2023-10-06T04:59:06.046491+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.035125+0000","last_change":"2023-10-06T04:59:06.033266+0000","last_active":"2023-10-06T04:59:06.035125+0000","last_peered":"2023-10-06T04:59:06.035125+0000","last_clean":"2023-10-06T04:59:06.035125+0000","last_became_active":"2023-10-06T04:59:06.032982+0000","last_became_peered":"2023-10-06T04:59:06.032982+0000","last_unstale":"2023-10-06T04:59:06.035125+0000","last_undegraded":"2023-10-06T04:59:06.035125+0000","last_fullsized":"2023-10-06T04:59:06.035125+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:11.903244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039147+0000","last_change":"2023-10-06T04:59:06.036068+0000","last_active":"2023-10-06T04:59:06.039147+0000","last_peered":"2023-10-06T04:59:06.039147+0000","last_clean":"2023-10-06T04:59:06.039147+0000","last_became_active":"2023-10-06T04:59:06.035798+0000","last_became_peered":"2023-10-06T04:59:06.035798+0000","last_unstale":"2023-10-06T04:59:06.039147+0000","last_undegraded":"2023-10-06T04:59:06.039147+0000","last_fullsized":"2023-10-06T04:59:06.039147+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:08.561731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044126+0000","last_change":"2023-10-06T04:59:06.038074+0000","last_active":"2023-10-06T04:59:06.044126+0000","last_peered":"2023-10-06T04:59:06.044126+0000","last_clean":"2023-10-06T04:59:06.044126+0000","last_became_active":"2023-10-06T04:59:06.037756+0000","last_became_peered":"2023-10-06T04:59:06.037756+0000","last_unstale":"2023-10-06T04:59:06.044126+0000","last_undegraded":"2023-10-06T04:59:06.044126+0000","last_fullsized":"2023-10-06T04:59:06.044126+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:58.334959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038972+0000","last_change":"2023-10-06T04:59:06.034401+0000","last_active":"2023-10-06T04:59:06.038972+0000","last_peered":"2023-10-06T04:59:06.038972+0000","last_clean":"2023-10-06T04:59:06.038972+0000","last_became_active":"2023-10-06T04:59:06.033982+0000","last_became_peered":"2023-10-06T04:59:06.033982+0000","last_unstale":"2023-10-06T04:59:06.038972+0000","last_undegraded":"2023-10-06T04:59:06.038972+0000","last_fullsized":"2023-10-06T04:59:06.038972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:21.963167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045747+0000","last_change":"2023-10-06T04:59:06.042530+0000","last_active":"2023-10-06T04:59:06.045747+0000","last_peered":"2023-10-06T04:59:06.045747+0000","last_clean":"2023-10-06T04:59:06.045747+0000","last_became_active":"2023-10-06T04:59:06.041470+0000","last_became_peered":"2023-10-06T04:59:06.041470+0000","last_unstale":"2023-10-06T04:59:06.045747+0000","last_undegraded":"2023-10-06T04:59:06.045747+0000","last_fullsized":"2023-10-06T04:59:06.045747+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:45.748990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034853+0000","last_change":"2023-10-06T04:59:06.032101+0000","last_active":"2023-10-06T04:59:06.034853+0000","last_peered":"2023-10-06T04:59:06.034853+0000","last_clean":"2023-10-06T04:59:06.034853+0000","last_became_active":"2023-10-06T04:59:06.031813+0000","last_became_peered":"2023-10-06T04:59:06.031813+0000","last_unstale":"2023-10-06T04:59:06.034853+0000","last_undegraded":"2023-10-06T04:59:06.034853+0000","last_fullsized":"2023-10-06T04:59:06.034853+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:17.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.052178+0000","last_change":"2023-10-06T04:59:06.045745+0000","last_active":"2023-10-06T04:59:06.052178+0000","last_peered":"2023-10-06T04:59:06.052178+0000","last_clean":"2023-10-06T04:59:06.052178+0000","last_became_active":"2023-10-06T04:59:06.045502+0000","last_became_peered":"2023-10-06T04:59:06.045502+0000","last_unstale":"2023-10-06T04:59:06.052178+0000","last_undegraded":"2023-10-06T04:59:06.052178+0000","last_fullsized":"2023-10-06T04:59:06.052178+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:05.065894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038999+0000","last_change":"2023-10-06T04:59:06.035681+0000","last_active":"2023-10-06T04:59:06.038999+0000","last_peered":"2023-10-06T04:59:06.038999+0000","last_clean":"2023-10-06T04:59:06.038999+0000","last_became_active":"2023-10-06T04:59:06.035410+0000","last_became_peered":"2023-10-06T04:59:06.035410+0000","last_unstale":"2023-10-06T04:59:06.038999+0000","last_undegraded":"2023-10-06T04:59:06.038999+0000","last_fullsized":"2023-10-06T04:59:06.038999+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:00.221865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039851+0000","last_change":"2023-10-06T04:59:06.036403+0000","last_active":"2023-10-06T04:59:06.039851+0000","last_peered":"2023-10-06T04:59:06.039851+0000","last_clean":"2023-10-06T04:59:06.039851+0000","last_became_active":"2023-10-06T04:59:06.036136+0000","last_became_peered":"2023-10-06T04:59:06.036136+0000","last_unstale":"2023-10-06T04:59:06.039851+0000","last_undegraded":"2023-10-06T04:59:06.039851+0000","last_fullsized":"2023-10-06T04:59:06.039851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:47.264143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.050425+0000","last_change":"2023-10-06T04:59:06.044784+0000","last_active":"2023-10-06T04:59:06.050425+0000","last_peered":"2023-10-06T04:59:06.050425+0000","last_clean":"2023-10-06T04:59:06.050425+0000","last_became_active":"2023-10-06T04:59:06.044308+0000","last_became_peered":"2023-10-06T04:59:06.044308+0000","last_unstale":"2023-10-06T04:59:06.050425+0000","last_undegraded":"2023-10-06T04:59:06.050425+0000","last_fullsized":"2023-10-06T04:59:06.050425+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042470+0000","last_change":"2023-10-06T04:59:06.040545+0000","last_active":"2023-10-06T04:59:06.042470+0000","last_peered":"2023-10-06T04:59:06.042470+0000","last_clean":"2023-10-06T04:59:06.042470+0000","last_became_active":"2023-10-06T04:59:06.040199+0000","last_became_peered":"2023-10-06T04:59:06.040199+0000","last_unstale":"2023-10-06T04:59:06.042470+0000","last_undegraded":"2023-10-06T04:59:06.042470+0000","last_fullsized":"2023-10-06T04:59:06.042470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044724+0000","last_change":"2023-10-06T04:59:06.036141+0000","last_active":"2023-10-06T04:59:06.044724+0000","last_peered":"2023-10-06T04:59:06.044724+0000","last_clean":"2023-10-06T04:59:06.044724+0000","last_became_active":"2023-10-06T04:59:06.035373+0000","last_became_peered":"2023-10-06T04:59:06.035373+0000","last_unstale":"2023-10-06T04:59:06.044724+0000","last_undegraded":"2023-10-06T04:59:06.044724+0000","last_fullsized":"2023-10-06T04:59:06.044724+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:05.065894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043944+0000","last_change":"2023-10-06T04:59:06.035866+0000","last_active":"2023-10-06T04:59:06.043944+0000","last_peered":"2023-10-06T04:59:06.043944+0000","last_clean":"2023-10-06T04:59:06.043944+0000","last_became_active":"2023-10-06T04:59:06.035611+0000","last_became_peered":"2023-10-06T04:59:06.035611+0000","last_unstale":"2023-10-06T04:59:06.043944+0000","last_undegraded":"2023-10-06T04:59:06.043944+0000","last_fullsized":"2023-10-06T04:59:06.043944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:06.804711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.031235+0000","last_change":"2023-10-06T04:59:06.030767+0000","last_active":"2023-10-06T04:59:06.031235+0000","last_peered":"2023-10-06T04:59:06.031235+0000","last_clean":"2023-10-06T04:59:06.031235+0000","last_became_active":"2023-10-06T04:59:06.029903+0000","last_became_peered":"2023-10-06T04:59:06.029903+0000","last_unstale":"2023-10-06T04:59:06.031235+0000","last_undegraded":"2023-10-06T04:59:06.031235+0000","last_fullsized":"2023-10-06T04:59:06.031235+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:26.680872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046166+0000","last_change":"2023-10-06T04:59:06.044577+0000","last_active":"2023-10-06T04:59:06.046166+0000","last_peered":"2023-10-06T04:59:06.046166+0000","last_clean":"2023-10-06T04:59:06.046166+0000","last_became_active":"2023-10-06T04:59:06.044307+0000","last_became_peered":"2023-10-06T04:59:06.044307+0000","last_unstale":"2023-10-06T04:59:06.046166+0000","last_undegraded":"2023-10-06T04:59:06.046166+0000","last_fullsized":"2023-10-06T04:59:06.046166+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:23.371571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043495+0000","last_change":"2023-10-06T04:59:06.037209+0000","last_active":"2023-10-06T04:59:06.043495+0000","last_peered":"2023-10-06T04:59:06.043495+0000","last_clean":"2023-10-06T04:59:06.043495+0000","last_became_active":"2023-10-06T04:59:06.036973+0000","last_became_peered":"2023-10-06T04:59:06.036973+0000","last_unstale":"2023-10-06T04:59:06.043495+0000","last_undegraded":"2023-10-06T04:59:06.043495+0000","last_fullsized":"2023-10-06T04:59:06.043495+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:17.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043049+0000","last_change":"2023-10-06T04:59:06.041834+0000","last_active":"2023-10-06T04:59:06.043049+0000","last_peered":"2023-10-06T04:59:06.043049+0000","last_clean":"2023-10-06T04:59:06.043049+0000","last_became_active":"2023-10-06T04:59:06.041130+0000","last_became_peered":"2023-10-06T04:59:06.041130+0000","last_unstale":"2023-10-06T04:59:06.043049+0000","last_undegraded":"2023-10-06T04:59:06.043049+0000","last_fullsized":"2023-10-06T04:59:06.043049+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:55.051458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034743+0000","last_change":"2023-10-06T04:59:06.032570+0000","last_active":"2023-10-06T04:59:06.034743+0000","last_peered":"2023-10-06T04:59:06.034743+0000","last_clean":"2023-10-06T04:59:06.034743+0000","last_became_active":"2023-10-06T04:59:06.032311+0000","last_became_peered":"2023-10-06T04:59:06.032311+0000","last_unstale":"2023-10-06T04:59:06.034743+0000","last_undegraded":"2023-10-06T04:59:06.034743+0000","last_fullsized":"2023-10-06T04:59:06.034743+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:33.709775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"95'4","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054300+0000","last_change":"2023-10-06T04:59:05.030540+0000","last_active":"2023-10-06T04:59:06.054300+0000","last_peered":"2023-10-06T04:59:06.054300+0000","last_clean":"2023-10-06T04:59:06.054300+0000","last_became_active":"2023-10-06T04:59:05.030260+0000","last_became_peered":"2023-10-06T04:59:05.030260+0000","last_unstale":"2023-10-06T04:59:06.054300+0000","last_undegraded":"2023-10-06T04:59:06.054300+0000","last_fullsized":"2023-10-06T04:59:06.054300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038511+0000","last_change":"2023-10-06T04:59:06.034197+0000","last_active":"2023-10-06T04:59:06.038511+0000","last_peered":"2023-10-06T04:59:06.038511+0000","last_clean":"2023-10-06T04:59:06.038511+0000","last_became_active":"2023-10-06T04:59:06.033834+0000","last_became_peered":"2023-10-06T04:59:06.033834+0000","last_unstale":"2023-10-06T04:59:06.038511+0000","last_undegraded":"2023-10-06T04:59:06.038511+0000","last_fullsized":"2023-10-06T04:59:06.038511+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:22.404213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.053565+0000","last_change":"2023-10-06T04:59:05.026990+0000","last_active":"2023-10-06T04:59:06.053565+0000","last_peered":"2023-10-06T04:59:06.053565+0000","last_clean":"2023-10-06T04:59:06.053565+0000","last_became_active":"2023-10-06T04:59:05.026693+0000","last_became_peered":"2023-10-06T04:59:05.026693+0000","last_unstale":"2023-10-06T04:59:06.053565+0000","last_undegraded":"2023-10-06T04:59:06.053565+0000","last_fullsized":"2023-10-06T04:59:06.053565+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:49.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045669+0000","last_change":"2023-10-06T04:59:06.041542+0000","last_active":"2023-10-06T04:59:06.045669+0000","last_peered":"2023-10-06T04:59:06.045669+0000","last_clean":"2023-10-06T04:59:06.045669+0000","last_became_active":"2023-10-06T04:59:06.041152+0000","last_became_peered":"2023-10-06T04:59:06.041152+0000","last_unstale":"2023-10-06T04:59:06.045669+0000","last_undegraded":"2023-10-06T04:59:06.045669+0000","last_fullsized":"2023-10-06T04:59:06.045669+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:34.810780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054088+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:06.054088+0000","last_peered":"2023-10-06T04:59:06.054088+0000","last_clean":"2023-10-06T04:59:06.054088+0000","last_became_active":"2023-10-06T04:59:05.020908+0000","last_became_peered":"2023-10-06T04:59:05.020908+0000","last_unstale":"2023-10-06T04:59:06.054088+0000","last_undegraded":"2023-10-06T04:59:06.054088+0000","last_fullsized":"2023-10-06T04:59:06.054088+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.255698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045933+0000","last_change":"2023-10-06T04:59:06.043586+0000","last_active":"2023-10-06T04:59:06.045933+0000","last_peered":"2023-10-06T04:59:06.045933+0000","last_clean":"2023-10-06T04:59:06.045933+0000","last_became_active":"2023-10-06T04:59:06.043273+0000","last_became_peered":"2023-10-06T04:59:06.043273+0000","last_unstale":"2023-10-06T04:59:06.045933+0000","last_undegraded":"2023-10-06T04:59:06.045933+0000","last_fullsized":"2023-10-06T04:59:06.045933+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:01.732144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054886+0000","last_change":"2023-10-06T04:59:05.023617+0000","last_active":"2023-10-06T04:59:06.054886+0000","last_peered":"2023-10-06T04:59:06.054886+0000","last_clean":"2023-10-06T04:59:06.054886+0000","last_became_active":"2023-10-06T04:59:05.023394+0000","last_became_peered":"2023-10-06T04:59:05.023394+0000","last_unstale":"2023-10-06T04:59:06.054886+0000","last_undegraded":"2023-10-06T04:59:06.054886+0000","last_fullsized":"2023-10-06T04:59:06.054886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.562815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.041211+0000","last_change":"2023-10-06T04:59:06.038723+0000","last_active":"2023-10-06T04:59:06.041211+0000","last_peered":"2023-10-06T04:59:06.041211+0000","last_clean":"2023-10-06T04:59:06.041211+0000","last_became_active":"2023-10-06T04:59:06.038383+0000","last_became_peered":"2023-10-06T04:59:06.038383+0000","last_unstale":"2023-10-06T04:59:06.041211+0000","last_undegraded":"2023-10-06T04:59:06.041211+0000","last_fullsized":"2023-10-06T04:59:06.041211+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.065649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.050600+0000","last_change":"2023-10-06T04:59:05.030809+0000","last_active":"2023-10-06T04:59:06.050600+0000","last_peered":"2023-10-06T04:59:06.050600+0000","last_clean":"2023-10-06T04:59:06.050600+0000","last_became_active":"2023-10-06T04:59:05.030550+0000","last_became_peered":"2023-10-06T04:59:05.030550+0000","last_unstale":"2023-10-06T04:59:06.050600+0000","last_undegraded":"2023-10-06T04:59:06.050600+0000","last_fullsized":"2023-10-06T04:59:06.050600+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038993+0000","last_change":"2023-10-06T04:59:06.034665+0000","last_active":"2023-10-06T04:59:06.038993+0000","last_peered":"2023-10-06T04:59:06.038993+0000","last_clean":"2023-10-06T04:59:06.038993+0000","last_became_active":"2023-10-06T04:59:06.034343+0000","last_became_peered":"2023-10-06T04:59:06.034343+0000","last_unstale":"2023-10-06T04:59:06.038993+0000","last_undegraded":"2023-10-06T04:59:06.038993+0000","last_fullsized":"2023-10-06T04:59:06.038993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:35.827029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.053879+0000","last_change":"2023-10-06T04:59:05.024000+0000","last_active":"2023-10-06T04:59:06.053879+0000","last_peered":"2023-10-06T04:59:06.053879+0000","last_clean":"2023-10-06T04:59:06.053879+0000","last_became_active":"2023-10-06T04:59:05.023706+0000","last_became_peered":"2023-10-06T04:59:05.023706+0000","last_unstale":"2023-10-06T04:59:06.053879+0000","last_undegraded":"2023-10-06T04:59:06.053879+0000","last_fullsized":"2023-10-06T04:59:06.053879+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043959+0000","last_change":"2023-10-06T04:59:06.036931+0000","last_active":"2023-10-06T04:59:06.043959+0000","last_peered":"2023-10-06T04:59:06.043959+0000","last_clean":"2023-10-06T04:59:06.043959+0000","last_became_active":"2023-10-06T04:59:06.036339+0000","last_became_peered":"2023-10-06T04:59:06.036339+0000","last_unstale":"2023-10-06T04:59:06.043959+0000","last_undegraded":"2023-10-06T04:59:06.043959+0000","last_fullsized":"2023-10-06T04:59:06.043959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:15.943200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.031443+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:06.031443+0000","last_peered":"2023-10-06T04:59:06.031443+0000","last_clean":"2023-10-06T04:59:06.031443+0000","last_became_active":"2023-10-06T04:59:05.020967+0000","last_became_peered":"2023-10-06T04:59:05.020967+0000","last_unstale":"2023-10-06T04:59:06.031443+0000","last_undegraded":"2023-10-06T04:59:06.031443+0000","last_fullsized":"2023-10-06T04:59:06.031443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:54.137958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039952+0000","last_change":"2023-10-06T04:59:06.035514+0000","last_active":"2023-10-06T04:59:06.039952+0000","last_peered":"2023-10-06T04:59:06.039952+0000","last_clean":"2023-10-06T04:59:06.039952+0000","last_became_active":"2023-10-06T04:59:06.035260+0000","last_became_peered":"2023-10-06T04:59:06.035260+0000","last_unstale":"2023-10-06T04:59:06.039952+0000","last_undegraded":"2023-10-06T04:59:06.039952+0000","last_fullsized":"2023-10-06T04:59:06.039952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:00.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.022944+0000","last_change":"2023-10-06T04:59:05.025368+0000","last_active":"2023-10-06T04:59:06.022944+0000","last_peered":"2023-10-06T04:59:06.022944+0000","last_clean":"2023-10-06T04:59:06.022944+0000","last_became_active":"2023-10-06T04:59:05.025101+0000","last_became_peered":"2023-10-06T04:59:05.025101+0000","last_unstale":"2023-10-06T04:59:06.022944+0000","last_undegraded":"2023-10-06T04:59:06.022944+0000","last_fullsized":"2023-10-06T04:59:06.022944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042741+0000","last_change":"2023-10-06T04:59:06.038836+0000","last_active":"2023-10-06T04:59:06.042741+0000","last_peered":"2023-10-06T04:59:06.042741+0000","last_clean":"2023-10-06T04:59:06.042741+0000","last_became_active":"2023-10-06T04:59:06.038271+0000","last_became_peered":"2023-10-06T04:59:06.038271+0000","last_unstale":"2023-10-06T04:59:06.042741+0000","last_undegraded":"2023-10-06T04:59:06.042741+0000","last_fullsized":"2023-10-06T04:59:06.042741+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:12.926654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.031200+0000","last_change":"2023-10-06T04:59:05.021703+0000","last_active":"2023-10-06T04:59:06.031200+0000","last_peered":"2023-10-06T04:59:06.031200+0000","last_clean":"2023-10-06T04:59:06.031200+0000","last_became_active":"2023-10-06T04:59:05.021413+0000","last_became_peered":"2023-10-06T04:59:05.021413+0000","last_unstale":"2023-10-06T04:59:06.031200+0000","last_undegraded":"2023-10-06T04:59:06.031200+0000","last_fullsized":"2023-10-06T04:59:06.031200+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:15.699249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.037345+0000","last_change":"2023-10-06T04:59:06.033225+0000","last_active":"2023-10-06T04:59:06.037345+0000","last_peered":"2023-10-06T04:59:06.037345+0000","last_clean":"2023-10-06T04:59:06.037345+0000","last_became_active":"2023-10-06T04:59:06.032769+0000","last_became_peered":"2023-10-06T04:59:06.032769+0000","last_unstale":"2023-10-06T04:59:06.037345+0000","last_undegraded":"2023-10-06T04:59:06.037345+0000","last_fullsized":"2023-10-06T04:59:06.037345+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:49.048000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.025061+0000","last_change":"2023-10-06T04:59:05.016394+0000","last_active":"2023-10-06T04:59:06.025061+0000","last_peered":"2023-10-06T04:59:06.025061+0000","last_clean":"2023-10-06T04:59:06.025061+0000","last_became_active":"2023-10-06T04:59:05.016106+0000","last_became_peered":"2023-10-06T04:59:05.016106+0000","last_unstale":"2023-10-06T04:59:06.025061+0000","last_undegraded":"2023-10-06T04:59:06.025061+0000","last_fullsized":"2023-10-06T04:59:06.025061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:10.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043794+0000","last_change":"2023-10-06T04:59:06.038121+0000","last_active":"2023-10-06T04:59:06.043794+0000","last_peered":"2023-10-06T04:59:06.043794+0000","last_clean":"2023-10-06T04:59:06.043794+0000","last_became_active":"2023-10-06T04:59:06.037844+0000","last_became_peered":"2023-10-06T04:59:06.037844+0000","last_unstale":"2023-10-06T04:59:06.043794+0000","last_undegraded":"2023-10-06T04:59:06.043794+0000","last_fullsized":"2023-10-06T04:59:06.043794+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:58.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"95'3","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.057717+0000","last_change":"2023-10-06T04:59:05.026950+0000","last_active":"2023-10-06T04:59:06.057717+0000","last_peered":"2023-10-06T04:59:06.057717+0000","last_clean":"2023-10-06T04:59:06.057717+0000","last_became_active":"2023-10-06T04:59:05.026723+0000","last_became_peered":"2023-10-06T04:59:05.026723+0000","last_unstale":"2023-10-06T04:59:06.057717+0000","last_undegraded":"2023-10-06T04:59:06.057717+0000","last_fullsized":"2023-10-06T04:59:06.057717+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039188+0000","last_change":"2023-10-06T04:59:06.035752+0000","last_active":"2023-10-06T04:59:06.039188+0000","last_peered":"2023-10-06T04:59:06.039188+0000","last_clean":"2023-10-06T04:59:06.039188+0000","last_became_active":"2023-10-06T04:59:06.035165+0000","last_became_peered":"2023-10-06T04:59:06.035165+0000","last_unstale":"2023-10-06T04:59:06.039188+0000","last_undegraded":"2023-10-06T04:59:06.039188+0000","last_fullsized":"2023-10-06T04:59:06.039188+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:21.883224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"95'6","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.060375+0000","last_change":"2023-10-06T04:59:05.025160+0000","last_active":"2023-10-06T04:59:06.060375+0000","last_peered":"2023-10-06T04:59:06.060375+0000","last_clean":"2023-10-06T04:59:06.060375+0000","last_became_active":"2023-10-06T04:59:05.024909+0000","last_became_peered":"2023-10-06T04:59:05.024909+0000","last_unstale":"2023-10-06T04:59:06.060375+0000","last_undegraded":"2023-10-06T04:59:06.060375+0000","last_fullsized":"2023-10-06T04:59:06.060375+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:00.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044934+0000","last_change":"2023-10-06T04:59:06.041960+0000","last_active":"2023-10-06T04:59:06.044934+0000","last_peered":"2023-10-06T04:59:06.044934+0000","last_clean":"2023-10-06T04:59:06.044934+0000","last_became_active":"2023-10-06T04:59:06.041649+0000","last_became_peered":"2023-10-06T04:59:06.041649+0000","last_unstale":"2023-10-06T04:59:06.044934+0000","last_undegraded":"2023-10-06T04:59:06.044934+0000","last_fullsized":"2023-10-06T04:59:06.044934+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:55.142865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.057437+0000","last_change":"2023-10-06T04:59:05.023984+0000","last_active":"2023-10-06T04:59:06.057437+0000","last_peered":"2023-10-06T04:59:06.057437+0000","last_clean":"2023-10-06T04:59:06.057437+0000","last_became_active":"2023-10-06T04:59:05.023641+0000","last_became_peered":"2023-10-06T04:59:05.023641+0000","last_unstale":"2023-10-06T04:59:06.057437+0000","last_undegraded":"2023-10-06T04:59:06.057437+0000","last_fullsized":"2023-10-06T04:59:06.057437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:54.810243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039722+0000","last_change":"2023-10-06T04:59:06.032585+0000","last_active":"2023-10-06T04:59:06.039722+0000","last_peered":"2023-10-06T04:59:06.039722+0000","last_clean":"2023-10-06T04:59:06.039722+0000","last_became_active":"2023-10-06T04:59:06.032306+0000","last_became_peered":"2023-10-06T04:59:06.032306+0000","last_unstale":"2023-10-06T04:59:06.039722+0000","last_undegraded":"2023-10-06T04:59:06.039722+0000","last_fullsized":"2023-10-06T04:59:06.039722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:05.265074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.023480+0000","last_change":"2023-10-06T04:59:05.025232+0000","last_active":"2023-10-06T04:59:06.023480+0000","last_peered":"2023-10-06T04:59:06.023480+0000","last_clean":"2023-10-06T04:59:06.023480+0000","last_became_active":"2023-10-06T04:59:05.024850+0000","last_became_peered":"2023-10-06T04:59:05.024850+0000","last_unstale":"2023-10-06T04:59:06.023480+0000","last_undegraded":"2023-10-06T04:59:06.023480+0000","last_fullsized":"2023-10-06T04:59:06.023480+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043059+0000","last_change":"2023-10-06T04:59:06.042021+0000","last_active":"2023-10-06T04:59:06.043059+0000","last_peered":"2023-10-06T04:59:06.043059+0000","last_clean":"2023-10-06T04:59:06.043059+0000","last_became_active":"2023-10-06T04:59:06.041759+0000","last_became_peered":"2023-10-06T04:59:06.041759+0000","last_unstale":"2023-10-06T04:59:06.043059+0000","last_undegraded":"2023-10-06T04:59:06.043059+0000","last_fullsized":"2023-10-06T04:59:06.043059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:57.975524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021678+0000","last_change":"2023-10-06T04:59:05.020554+0000","last_active":"2023-10-06T04:59:06.021678+0000","last_peered":"2023-10-06T04:59:06.021678+0000","last_clean":"2023-10-06T04:59:06.021678+0000","last_became_active":"2023-10-06T04:59:05.020294+0000","last_became_peered":"2023-10-06T04:59:05.020294+0000","last_unstale":"2023-10-06T04:59:06.021678+0000","last_undegraded":"2023-10-06T04:59:06.021678+0000","last_fullsized":"2023-10-06T04:59:06.021678+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:24.547416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039127+0000","last_change":"2023-10-06T04:59:06.035847+0000","last_active":"2023-10-06T04:59:06.039127+0000","last_peered":"2023-10-06T04:59:06.039127+0000","last_clean":"2023-10-06T04:59:06.039127+0000","last_became_active":"2023-10-06T04:59:06.035462+0000","last_became_peered":"2023-10-06T04:59:06.035462+0000","last_unstale":"2023-10-06T04:59:06.039127+0000","last_undegraded":"2023-10-06T04:59:06.039127+0000","last_fullsized":"2023-10-06T04:59:06.039127+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:03.596586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"95'8","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054885+0000","last_change":"2023-10-06T04:59:05.022660+0000","last_active":"2023-10-06T04:59:06.054885+0000","last_peered":"2023-10-06T04:59:06.054885+0000","last_clean":"2023-10-06T04:59:06.054885+0000","last_became_active":"2023-10-06T04:59:05.021894+0000","last_became_peered":"2023-10-06T04:59:05.021894+0000","last_unstale":"2023-10-06T04:59:06.054885+0000","last_undegraded":"2023-10-06T04:59:06.054885+0000","last_fullsized":"2023-10-06T04:59:06.054885+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.654900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034711+0000","last_change":"2023-10-06T04:59:06.031277+0000","last_active":"2023-10-06T04:59:06.034711+0000","last_peered":"2023-10-06T04:59:06.034711+0000","last_clean":"2023-10-06T04:59:06.034711+0000","last_became_active":"2023-10-06T04:59:06.030991+0000","last_became_peered":"2023-10-06T04:59:06.030991+0000","last_unstale":"2023-10-06T04:59:06.034711+0000","last_undegraded":"2023-10-06T04:59:06.034711+0000","last_fullsized":"2023-10-06T04:59:06.034711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:43.083569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054900+0000","last_change":"2023-10-06T04:59:05.022693+0000","last_active":"2023-10-06T04:59:06.054900+0000","last_peered":"2023-10-06T04:59:06.054900+0000","last_clean":"2023-10-06T04:59:06.054900+0000","last_became_active":"2023-10-06T04:59:05.022421+0000","last_became_peered":"2023-10-06T04:59:05.022421+0000","last_unstale":"2023-10-06T04:59:06.054900+0000","last_undegraded":"2023-10-06T04:59:06.054900+0000","last_fullsized":"2023-10-06T04:59:06.054900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:59.701496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042234+0000","last_change":"2023-10-06T04:59:06.041119+0000","last_active":"2023-10-06T04:59:06.042234+0000","last_peered":"2023-10-06T04:59:06.042234+0000","last_clean":"2023-10-06T04:59:06.042234+0000","last_became_active":"2023-10-06T04:59:06.040816+0000","last_became_peered":"2023-10-06T04:59:06.040816+0000","last_unstale":"2023-10-06T04:59:06.042234+0000","last_undegraded":"2023-10-06T04:59:06.042234+0000","last_fullsized":"2023-10-06T04:59:06.042234+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:51.603555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"96'1","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-10-06T04:59:07.058308+0000","last_change":"2023-10-06T04:59:05.024903+0000","last_active":"2023-10-06T04:59:07.058308+0000","last_peered":"2023-10-06T04:59:07.058308+0000","last_clean":"2023-10-06T04:59:07.058308+0000","last_became_active":"2023-10-06T04:59:05.024608+0000","last_became_peered":"2023-10-06T04:59:05.024608+0000","last_unstale":"2023-10-06T04:59:07.058308+0000","last_undegraded":"2023-10-06T04:59:07.058308+0000","last_fullsized":"2023-10-06T04:59:07.058308+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:48.099761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039701+0000","last_change":"2023-10-06T04:59:06.033420+0000","last_active":"2023-10-06T04:59:06.039701+0000","last_peered":"2023-10-06T04:59:06.039701+0000","last_clean":"2023-10-06T04:59:06.039701+0000","last_became_active":"2023-10-06T04:59:06.033107+0000","last_became_peered":"2023-10-06T04:59:06.033107+0000","last_unstale":"2023-10-06T04:59:06.039701+0000","last_undegraded":"2023-10-06T04:59:06.039701+0000","last_fullsized":"2023-10-06T04:59:06.039701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:31.872736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026818+0000","last_change":"2023-10-06T04:59:05.030722+0000","last_active":"2023-10-06T04:59:06.026818+0000","last_peered":"2023-10-06T04:59:06.026818+0000","last_clean":"2023-10-06T04:59:06.026818+0000","last_became_active":"2023-10-06T04:59:05.030276+0000","last_became_peered":"2023-10-06T04:59:05.030276+0000","last_unstale":"2023-10-06T04:59:06.026818+0000","last_undegraded":"2023-10-06T04:59:06.026818+0000","last_fullsized":"2023-10-06T04:59:06.026818+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:57.328563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044890+0000","last_change":"2023-10-06T04:59:06.042074+0000","last_active":"2023-10-06T04:59:06.044890+0000","last_peered":"2023-10-06T04:59:06.044890+0000","last_clean":"2023-10-06T04:59:06.044890+0000","last_became_active":"2023-10-06T04:59:06.038267+0000","last_became_peered":"2023-10-06T04:59:06.038267+0000","last_unstale":"2023-10-06T04:59:06.044890+0000","last_undegraded":"2023-10-06T04:59:06.044890+0000","last_fullsized":"2023-10-06T04:59:06.044890+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:49.356661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.033122+0000","last_change":"2023-10-06T04:59:05.024303+0000","last_active":"2023-10-06T04:59:06.033122+0000","last_peered":"2023-10-06T04:59:06.033122+0000","last_clean":"2023-10-06T04:59:06.033122+0000","last_became_active":"2023-10-06T04:59:05.024038+0000","last_became_peered":"2023-10-06T04:59:05.024038+0000","last_unstale":"2023-10-06T04:59:06.033122+0000","last_undegraded":"2023-10-06T04:59:06.033122+0000","last_fullsized":"2023-10-06T04:59:06.033122+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044804+0000","last_change":"2023-10-06T04:59:06.042312+0000","last_active":"2023-10-06T04:59:06.044804+0000","last_peered":"2023-10-06T04:59:06.044804+0000","last_clean":"2023-10-06T04:59:06.044804+0000","last_became_active":"2023-10-06T04:59:06.041490+0000","last_became_peered":"2023-10-06T04:59:06.041490+0000","last_unstale":"2023-10-06T04:59:06.044804+0000","last_undegraded":"2023-10-06T04:59:06.044804+0000","last_fullsized":"2023-10-06T04:59:06.044804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:08.338740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034069+0000","last_change":"2023-10-06T04:59:05.023856+0000","last_active":"2023-10-06T04:59:06.034069+0000","last_peered":"2023-10-06T04:59:06.034069+0000","last_clean":"2023-10-06T04:59:06.034069+0000","last_became_active":"2023-10-06T04:59:05.023612+0000","last_became_peered":"2023-10-06T04:59:05.023612+0000","last_unstale":"2023-10-06T04:59:06.034069+0000","last_undegraded":"2023-10-06T04:59:06.034069+0000","last_fullsized":"2023-10-06T04:59:06.034069+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:04.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042994+0000","last_change":"2023-10-06T04:59:06.041624+0000","last_active":"2023-10-06T04:59:06.042994+0000","last_peered":"2023-10-06T04:59:06.042994+0000","last_clean":"2023-10-06T04:59:06.042994+0000","last_became_active":"2023-10-06T04:59:06.041360+0000","last_became_peered":"2023-10-06T04:59:06.041360+0000","last_unstale":"2023-10-06T04:59:06.042994+0000","last_undegraded":"2023-10-06T04:59:06.042994+0000","last_fullsized":"2023-10-06T04:59:06.042994+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:50:43.443720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021745+0000","last_change":"2023-10-06T04:59:05.023970+0000","last_active":"2023-10-06T04:59:06.021745+0000","last_peered":"2023-10-06T04:59:06.021745+0000","last_clean":"2023-10-06T04:59:06.021745+0000","last_became_active":"2023-10-06T04:59:05.023712+0000","last_became_peered":"2023-10-06T04:59:05.023712+0000","last_unstale":"2023-10-06T04:59:06.021745+0000","last_undegraded":"2023-10-06T04:59:06.021745+0000","last_fullsized":"2023-10-06T04:59:06.021745+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:31.733624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046432+0000","last_change":"2023-10-06T04:59:06.044979+0000","last_active":"2023-10-06T04:59:06.046432+0000","last_peered":"2023-10-06T04:59:06.046432+0000","last_clean":"2023-10-06T04:59:06.046432+0000","last_became_active":"2023-10-06T04:59:06.044703+0000","last_became_peered":"2023-10-06T04:59:06.044703+0000","last_unstale":"2023-10-06T04:59:06.046432+0000","last_undegraded":"2023-10-06T04:59:06.046432+0000","last_fullsized":"2023-10-06T04:59:06.046432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:00.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026891+0000","last_change":"2023-10-06T04:59:05.028495+0000","last_active":"2023-10-06T04:59:06.026891+0000","last_peered":"2023-10-06T04:59:06.026891+0000","last_clean":"2023-10-06T04:59:06.026891+0000","last_became_active":"2023-10-06T04:59:05.028219+0000","last_became_peered":"2023-10-06T04:59:05.028219+0000","last_unstale":"2023-10-06T04:59:06.026891+0000","last_undegraded":"2023-10-06T04:59:06.026891+0000","last_fullsized":"2023-10-06T04:59:06.026891+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:54.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045137+0000","last_change":"2023-10-06T04:59:06.035823+0000","last_active":"2023-10-06T04:59:06.045137+0000","last_peered":"2023-10-06T04:59:06.045137+0000","last_clean":"2023-10-06T04:59:06.045137+0000","last_became_active":"2023-10-06T04:59:06.035551+0000","last_became_peered":"2023-10-06T04:59:06.035551+0000","last_unstale":"2023-10-06T04:59:06.045137+0000","last_undegraded":"2023-10-06T04:59:06.045137+0000","last_fullsized":"2023-10-06T04:59:06.045137+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:24.260605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021954+0000","last_change":"2023-10-06T04:59:05.023535+0000","last_active":"2023-10-06T04:59:06.021954+0000","last_peered":"2023-10-06T04:59:06.021954+0000","last_clean":"2023-10-06T04:59:06.021954+0000","last_became_active":"2023-10-06T04:59:05.023261+0000","last_became_peered":"2023-10-06T04:59:05.023261+0000","last_unstale":"2023-10-06T04:59:06.021954+0000","last_undegraded":"2023-10-06T04:59:06.021954+0000","last_fullsized":"2023-10-06T04:59:06.021954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:47.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.036062+0000","last_change":"2023-10-06T04:59:06.032000+0000","last_active":"2023-10-06T04:59:06.036062+0000","last_peered":"2023-10-06T04:59:06.036062+0000","last_clean":"2023-10-06T04:59:06.036062+0000","last_became_active":"2023-10-06T04:59:06.031738+0000","last_became_peered":"2023-10-06T04:59:06.031738+0000","last_unstale":"2023-10-06T04:59:06.036062+0000","last_undegraded":"2023-10-06T04:59:06.036062+0000","last_fullsized":"2023-10-06T04:59:06.036062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:17.752764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.052121+0000","last_change":"2023-10-06T04:59:05.026906+0000","last_active":"2023-10-06T04:59:06.052121+0000","last_peered":"2023-10-06T04:59:06.052121+0000","last_clean":"2023-10-06T04:59:06.052121+0000","last_became_active":"2023-10-06T04:59:05.026633+0000","last_became_peered":"2023-10-06T04:59:05.026633+0000","last_unstale":"2023-10-06T04:59:06.052121+0000","last_undegraded":"2023-10-06T04:59:06.052121+0000","last_fullsized":"2023-10-06T04:59:06.052121+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:56.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.029616+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:59:06.029616+0000","last_peered":"2023-10-06T04:59:06.029616+0000","last_clean":"2023-10-06T04:59:06.029616+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:59:06.029616+0000","last_undegraded":"2023-10-06T04:59:06.029616+0000","last_fullsized":"2023-10-06T04:59:06.029616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043881+0000","last_change":"2023-10-06T04:59:06.041830+0000","last_active":"2023-10-06T04:59:06.043881+0000","last_peered":"2023-10-06T04:59:06.043881+0000","last_clean":"2023-10-06T04:59:06.043881+0000","last_became_active":"2023-10-06T04:59:06.041558+0000","last_became_peered":"2023-10-06T04:59:06.041558+0000","last_unstale":"2023-10-06T04:59:06.043881+0000","last_undegraded":"2023-10-06T04:59:06.043881+0000","last_fullsized":"2023-10-06T04:59:06.043881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:10.771633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021346+0000","last_change":"2023-10-06T04:59:05.028380+0000","last_active":"2023-10-06T04:59:06.021346+0000","last_peered":"2023-10-06T04:59:06.021346+0000","last_clean":"2023-10-06T04:59:06.021346+0000","last_became_active":"2023-10-06T04:59:05.028098+0000","last_became_peered":"2023-10-06T04:59:05.028098+0000","last_unstale":"2023-10-06T04:59:06.021346+0000","last_undegraded":"2023-10-06T04:59:06.021346+0000","last_fullsized":"2023-10-06T04:59:06.021346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:22.276266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.023345+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:59:06.023345+0000","last_peered":"2023-10-06T04:59:06.023345+0000","last_clean":"2023-10-06T04:59:06.023345+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:59:06.023345+0000","last_undegraded":"2023-10-06T04:59:06.023345+0000","last_fullsized":"2023-10-06T04:59:06.023345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.040333+0000","last_change":"2023-10-06T04:59:06.033951+0000","last_active":"2023-10-06T04:59:06.040333+0000","last_peered":"2023-10-06T04:59:06.040333+0000","last_clean":"2023-10-06T04:59:06.040333+0000","last_became_active":"2023-10-06T04:59:06.033626+0000","last_became_peered":"2023-10-06T04:59:06.033626+0000","last_unstale":"2023-10-06T04:59:06.040333+0000","last_undegraded":"2023-10-06T04:59:06.040333+0000","last_fullsized":"2023-10-06T04:59:06.040333+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:14.960791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054103+0000","last_change":"2023-10-06T04:59:05.024110+0000","last_active":"2023-10-06T04:59:06.054103+0000","last_peered":"2023-10-06T04:59:06.054103+0000","last_clean":"2023-10-06T04:59:06.054103+0000","last_became_active":"2023-10-06T04:59:05.023840+0000","last_became_peered":"2023-10-06T04:59:05.023840+0000","last_unstale":"2023-10-06T04:59:06.054103+0000","last_undegraded":"2023-10-06T04:59:06.054103+0000","last_fullsized":"2023-10-06T04:59:06.054103+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.032795+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:59:06.032795+0000","last_peered":"2023-10-06T04:59:06.032795+0000","last_clean":"2023-10-06T04:59:06.032795+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:59:06.032795+0000","last_undegraded":"2023-10-06T04:59:06.032795+0000","last_fullsized":"2023-10-06T04:59:06.032795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":224,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.025871+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:59:06.025871+0000","last_peered":"2023-10-06T04:59:06.025871+0000","last_clean":"2023-10-06T04:59:06.025871+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:59:06.025871+0000","last_undegraded":"2023-10-06T04:59:06.025871+0000","last_fullsized":"2023-10-06T04:59:06.025871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039490+0000","last_change":"2023-10-06T04:59:06.036003+0000","last_active":"2023-10-06T04:59:06.039490+0000","last_peered":"2023-10-06T04:59:06.039490+0000","last_clean":"2023-10-06T04:59:06.039490+0000","last_became_active":"2023-10-06T04:59:06.035458+0000","last_became_peered":"2023-10-06T04:59:06.035458+0000","last_unstale":"2023-10-06T04:59:06.039490+0000","last_undegraded":"2023-10-06T04:59:06.039490+0000","last_fullsized":"2023-10-06T04:59:06.039490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:55.142865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.027365+0000","last_change":"2023-10-06T04:59:05.030905+0000","last_active":"2023-10-06T04:59:06.027365+0000","last_peered":"2023-10-06T04:59:06.027365+0000","last_clean":"2023-10-06T04:59:06.027365+0000","last_became_active":"2023-10-06T04:59:05.030530+0000","last_became_peered":"2023-10-06T04:59:05.030530+0000","last_unstale":"2023-10-06T04:59:06.027365+0000","last_undegraded":"2023-10-06T04:59:06.027365+0000","last_fullsized":"2023-10-06T04:59:06.027365+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:15.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026199+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:59:06.026199+0000","last_peered":"2023-10-06T04:59:06.026199+0000","last_clean":"2023-10-06T04:59:06.026199+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:59:06.026199+0000","last_undegraded":"2023-10-06T04:59:06.026199+0000","last_fullsized":"2023-10-06T04:59:06.026199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044118+0000","last_change":"2023-10-06T04:59:06.035806+0000","last_active":"2023-10-06T04:59:06.044118+0000","last_peered":"2023-10-06T04:59:06.044118+0000","last_clean":"2023-10-06T04:59:06.044118+0000","last_became_active":"2023-10-06T04:59:06.035465+0000","last_became_peered":"2023-10-06T04:59:06.035465+0000","last_unstale":"2023-10-06T04:59:06.044118+0000","last_undegraded":"2023-10-06T04:59:06.044118+0000","last_fullsized":"2023-10-06T04:59:06.044118+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:41.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.051300+0000","last_change":"2023-10-06T04:59:05.028526+0000","last_active":"2023-10-06T04:59:06.051300+0000","last_peered":"2023-10-06T04:59:06.051300+0000","last_clean":"2023-10-06T04:59:06.051300+0000","last_became_active":"2023-10-06T04:59:05.028134+0000","last_became_peered":"2023-10-06T04:59:05.028134+0000","last_unstale":"2023-10-06T04:59:06.051300+0000","last_undegraded":"2023-10-06T04:59:06.051300+0000","last_fullsized":"2023-10-06T04:59:06.051300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:16.407534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.025959+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:59:06.025959+0000","last_peered":"2023-10-06T04:59:06.025959+0000","last_clean":"2023-10-06T04:59:06.025959+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:59:06.025959+0000","last_undegraded":"2023-10-06T04:59:06.025959+0000","last_fullsized":"2023-10-06T04:59:06.025959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038814+0000","last_change":"2023-10-06T04:59:06.034490+0000","last_active":"2023-10-06T04:59:06.038814+0000","last_peered":"2023-10-06T04:59:06.038814+0000","last_clean":"2023-10-06T04:59:06.038814+0000","last_became_active":"2023-10-06T04:59:06.034228+0000","last_became_peered":"2023-10-06T04:59:06.034228+0000","last_unstale":"2023-10-06T04:59:06.038814+0000","last_undegraded":"2023-10-06T04:59:06.038814+0000","last_fullsized":"2023-10-06T04:59:06.038814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.531534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054134+0000","last_change":"2023-10-06T04:59:05.023805+0000","last_active":"2023-10-06T04:59:06.054134+0000","last_peered":"2023-10-06T04:59:06.054134+0000","last_clean":"2023-10-06T04:59:06.054134+0000","last_became_active":"2023-10-06T04:59:05.023560+0000","last_became_peered":"2023-10-06T04:59:05.023560+0000","last_unstale":"2023-10-06T04:59:06.054134+0000","last_undegraded":"2023-10-06T04:59:06.054134+0000","last_fullsized":"2023-10-06T04:59:06.054134+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:16.588598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.027734+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:59:06.027734+0000","last_peered":"2023-10-06T04:59:06.027734+0000","last_clean":"2023-10-06T04:59:06.027734+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:59:06.027734+0000","last_undegraded":"2023-10-06T04:59:06.027734+0000","last_fullsized":"2023-10-06T04:59:06.027734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042442+0000","last_change":"2023-10-06T04:59:06.038092+0000","last_active":"2023-10-06T04:59:06.042442+0000","last_peered":"2023-10-06T04:59:06.042442+0000","last_clean":"2023-10-06T04:59:06.042442+0000","last_became_active":"2023-10-06T04:59:06.037773+0000","last_became_peered":"2023-10-06T04:59:06.037773+0000","last_unstale":"2023-10-06T04:59:06.042442+0000","last_undegraded":"2023-10-06T04:59:06.042442+0000","last_fullsized":"2023-10-06T04:59:06.042442+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:49.295778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021849+0000","last_change":"2023-10-06T04:59:05.020425+0000","last_active":"2023-10-06T04:59:06.021849+0000","last_peered":"2023-10-06T04:59:06.021849+0000","last_clean":"2023-10-06T04:59:06.021849+0000","last_became_active":"2023-10-06T04:59:05.020172+0000","last_became_peered":"2023-10-06T04:59:05.020172+0000","last_unstale":"2023-10-06T04:59:06.021849+0000","last_undegraded":"2023-10-06T04:59:06.021849+0000","last_fullsized":"2023-10-06T04:59:06.021849+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:25.681695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.027873+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:59:06.027873+0000","last_peered":"2023-10-06T04:59:06.027873+0000","last_clean":"2023-10-06T04:59:06.027873+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:59:06.027873+0000","last_undegraded":"2023-10-06T04:59:06.027873+0000","last_fullsized":"2023-10-06T04:59:06.027873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039817+0000","last_change":"2023-10-06T04:59:06.034739+0000","last_active":"2023-10-06T04:59:06.039817+0000","last_peered":"2023-10-06T04:59:06.039817+0000","last_clean":"2023-10-06T04:59:06.039817+0000","last_became_active":"2023-10-06T04:59:06.034438+0000","last_became_peered":"2023-10-06T04:59:06.034438+0000","last_unstale":"2023-10-06T04:59:06.039817+0000","last_undegraded":"2023-10-06T04:59:06.039817+0000","last_fullsized":"2023-10-06T04:59:06.039817+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:17.016243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026061+0000","last_change":"2023-10-06T04:59:05.028402+0000","last_active":"2023-10-06T04:59:06.026061+0000","last_peered":"2023-10-06T04:59:06.026061+0000","last_clean":"2023-10-06T04:59:06.026061+0000","last_became_active":"2023-10-06T04:59:05.028139+0000","last_became_peered":"2023-10-06T04:59:05.028139+0000","last_unstale":"2023-10-06T04:59:06.026061+0000","last_undegraded":"2023-10-06T04:59:06.026061+0000","last_fullsized":"2023-10-06T04:59:06.026061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:56.970617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026721+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:59:06.026721+0000","last_peered":"2023-10-06T04:59:06.026721+0000","last_clean":"2023-10-06T04:59:06.026721+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:59:06.026721+0000","last_undegraded":"2023-10-06T04:59:06.026721+0000","last_fullsized":"2023-10-06T04:59:06.026721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307224,"kb_used_data":1744,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064616,"statfs":{"total":96636764160,"available":96322166784,"internally_reserved":0,"allocated":1785856,"data_stored":1125130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.73399999999999999},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73199999999999998,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.497}]},{"osd":1,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":1.073},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.51600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":1.111},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.58999999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.503},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.315}]},{"osd":5,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":1.214},"last":0.33200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.372},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1208320,"data_stored":538067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.52200000000000002},"min":{"1min":0.54900000000000004,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.81200000000000006,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.57199999999999995},"min":{"1min":0.56100000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.39800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":1.234,"15min":1.234},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.55200000000000005},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81000000000000005,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75800000000000001,"5min":3.669,"15min":3.669},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.747},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307220,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064620,"statfs":{"total":96636764160,"available":96322170880,"internally_reserved":0,"allocated":1781760,"data_stored":1125004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.23000000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.27900000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.501,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.377}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.70999999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.85499999999999998,"15min":0.92900000000000005},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.64100000000000001},"min":{"1min":0.47499999999999998,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.998,"15min":0.998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.65800000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":1.0900000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80600000000000005,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.026},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.79400000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306656,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065184,"statfs":{"total":96636764160,"available":96322748416,"internally_reserved":0,"allocated":1204224,"data_stored":537581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.505,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57699999999999996,"5min":0.628,"15min":0.628},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.873},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90700000000000003,"5min":1.022,"15min":1.022},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.495,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.5,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96599999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.995,"15min":0.995},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302548,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069292,"statfs":{"total":96636764160,"available":96326955008,"internally_reserved":0,"allocated":1191936,"data_stored":535989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.51200000000000001,"5min":0.747,"15min":1.048},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.42599999999999999,"15min":0.39000000000000001},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":0.61699999999999999,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.36099999999999999},"min":{"1min":0.29499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.31900000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40999999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.37},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.5,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306644,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065196,"statfs":{"total":96636764160,"available":96322760704,"internally_reserved":0,"allocated":1191936,"data_stored":536509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38700000000000001,"15min":0.40600000000000003},"min":{"1min":0.251,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":2.008,"15min":2.008},"last":0.307},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41299999999999998,"15min":0.56499999999999995},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54500000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42399999999999999,"15min":0.46000000000000002},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.64600000000000002,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":1.379,"15min":5.069},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":2.246,"15min":2.246},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83199999999999996,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82199999999999995,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.79400000000000004},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.40400000000000003},"max":{"1min":0.76200000000000001,"5min":1.964,"15min":6.367},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65000000000000002,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":1.3,"15min":5},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306648,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065192,"statfs":{"total":96636764160,"available":96322756608,"internally_reserved":0,"allocated":1196032,"data_stored":536047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.372},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.78500000000000003},"min":{"1min":0.38200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.82099999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.80400000000000005,"15min":5.415},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.66700000000000004},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":4.202},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":1.044,"15min":4.2130000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.745}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.77600000000000002,"5min":0.86199999999999999,"15min":0.872},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.313},"max":{"1min":0.84399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306636,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065204,"statfs":{"total":96636764160,"available":96322768896,"internally_reserved":0,"allocated":1183744,"data_stored":534602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.67600000000000005},"min":{"1min":0.36899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.625},"min":{"1min":0.436,"5min":0.247,"15min":0.247},"max":{"1min":0.76000000000000001,"5min":0.80800000000000005,"15min":3.7679999999999998},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.70799999999999996},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":6.7949999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.75800000000000001},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.93799999999999994},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":9.7210000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.93500000000000005},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":9.7539999999999996},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.504},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.34200000000000003},"max":{"1min":0.66300000000000003,"5min":0.746,"15min":0.746},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80100000000000005},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:59:14.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:59:14.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.852+0000 7fe826e4c700 1 Processor -- start 2023-10-06T04:59:14.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.852+0000 7fe826e4c700 1 -- start start 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe826e4c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe82015ccc0 0x7fe8200d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe826e4c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 0x7fe8200b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe826e4c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8200cf6c0 con 0x7fe82015c8f0 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe826e4c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8200cfb50 con 0x7fe82015ccc0 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe826e4c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200d0ad0 con 0x7fe8200d06a0 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe824be8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe82015c8f0 0x7fe82015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58948/0 (socket says 172.21.15.3:58948) 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe824be8700 1 -- 172.21.15.3:0/1170030528 learned_addr learned my addr 172.21.15.3:0/1170030528 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733357801 0 0) 0x7fe8200cf6c0 con 0x7fe82015c8f0 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe810003570 con 0x7fe82015c8f0 2023-10-06T04:59:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe81ffff700 1 --2- 172.21.15.3:0/1170030528 >> v2:172.21.15.153:3300/0 conn(0x7fe82015ccc0 0x7fe8200d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.853+0000 7fe8253e9700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 0x7fe8200b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464187073 0 0) 0x7fe810003570 con 0x7fe82015c8f0 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 msgr2=0x7fe8200b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 0x7fe8200b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 >> v2:172.21.15.153:3300/0 conn(0x7fe82015ccc0 msgr2=0x7fe8200d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 --2- 172.21.15.3:0/1170030528 >> v2:172.21.15.153:3300/0 conn(0x7fe82015ccc0 0x7fe8200d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200ceab0 con 0x7fe82015c8f0 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fe814001470 con 0x7fe82015c8f0 2023-10-06T04:59:14.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 >> v1:172.21.15.3:6789/0 conn(0x7fe82015c8f0 legacy=0x7fe82015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810003a60 0x7fe810003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 0x7fe810004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe810004d50 0x7fe8100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200cfb50 con 0x7fe810003a60 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200d0ad0 con 0x7fe810004d50 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81000ba60 con 0x7fe8200d06a0 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe824be8700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810003a60 0x7fe810003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe81ffff700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 0x7fe810004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe8253e9700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe810004d50 0x7fe8100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe824be8700 1 -- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 msgr2=0x7fe810004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe824be8700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8200d06a0 0x7fe810004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:14.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe824be8700 1 -- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe810004d50 msgr2=0x7fe8100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe824be8700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe810004d50 0x7fe8100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.854+0000 7fe824be8700 1 -- 172.21.15.3:0/1170030528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810003570 con 0x7fe810003a60 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe824be8700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810003a60 0x7fe810003e70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c3c0 tx=0x7fe814001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8723abbed13476ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81400cc30 con 0x7fe810003a60 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81400cdb0 con 0x7fe810003a60 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe81f7fe700 1 -- 172.21.15.3:0/1170030528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814005b70 con 0x7fe810003a60 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe826e4c700 1 -- 172.21.15.3:0/1170030528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810003a60 msgr2=0x7fe810003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe826e4c700 1 --2- 172.21.15.3:0/1170030528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810003a60 0x7fe810003e70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c3c0 tx=0x7fe814001e90 comp rx=0 tx=0).stop 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe8253e9700 1 -- 172.21.15.3:0/1170030528 reap_dead start 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe826e4c700 1 -- 172.21.15.3:0/1170030528 shutdown_connections 2023-10-06T04:59:14.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe826e4c700 1 -- 172.21.15.3:0/1170030528 >> 172.21.15.3:0/1170030528 conn(0x7fe820070310 msgr2=0x7fe820070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe826e4c700 1 -- 172.21.15.3:0/1170030528 shutdown_connections 2023-10-06T04:59:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.855+0000 7fe826e4c700 1 -- 172.21.15.3:0/1170030528 wait complete. 2023-10-06T04:59:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.856+0000 7fe826e4c700 1 Processor -- start 2023-10-06T04:59:14.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.857+0000 7fe826e4c700 1 -- start start 2023-10-06T04:59:14.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.857+0000 7fe826e4c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe810003a60 0x7fe820155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.857+0000 7fe826e4c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe820169da0 0x7fe8200c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.857+0000 7fe826e4c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82016a210 0x7fe8200c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.857+0000 7fe826e4c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82015a090 con 0x7fe82016a210 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.857+0000 7fe826e4c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820156520 con 0x7fe820169da0 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.857+0000 7fe826e4c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820159480 con 0x7fe810003a60 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82016a210 0x7fe8200c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82016a210 0x7fe8200c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37168/0 (socket says 172.21.15.3:37168) 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 -- 172.21.15.3:0/3336830780 learned_addr learned my addr 172.21.15.3:0/3336830780 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe824be8700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe810003a60 0x7fe820155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe81ffff700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe820169da0 0x7fe8200c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 -- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe810003a60 msgr2=0x7fe820155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe810003a60 0x7fe820155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 -- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe820169da0 msgr2=0x7fe8200c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe820169da0 0x7fe8200c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 -- 172.21.15.3:0/3336830780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820159760 con 0x7fe82016a210 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe8253e9700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82016a210 0x7fe8200c7860 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe8200890f0 tx=0x7fe81800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe81cff9700 1 -- 172.21.15.3:0/3336830780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800e040 con 0x7fe82016a210 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe81cff9700 1 -- 172.21.15.3:0/3336830780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81800f040 con 0x7fe82016a210 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe820159a40 con 0x7fe82016a210 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe81cff9700 1 -- 172.21.15.3:0/3336830780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800fd70 con 0x7fe82016a210 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.858+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82015a520 con 0x7fe82016a210 2023-10-06T04:59:14.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.859+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82015afa0 con 0x7fe82016a210 2023-10-06T04:59:14.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.860+0000 7fe81cff9700 1 -- 172.21.15.3:0/3336830780 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe818015640 con 0x7fe82016a210 2023-10-06T04:59:14.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.860+0000 7fe81cff9700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe804066f30 0x7fe8040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:14.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.860+0000 7fe81cff9700 1 -- 172.21.15.3:0/3336830780 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe818084ec0 con 0x7fe82016a210 2023-10-06T04:59:14.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.860+0000 7fe824be8700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe804066f30 0x7fe8040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:14.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.861+0000 7fe824be8700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe804066f30 0x7fe8040693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe814001ed0 tx=0x7fe814015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:14.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:14.865+0000 7fe81cff9700 1 -- 172.21.15.3:0/3336830780 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe818051f30 con 0x7fe82016a210 2023-10-06T04:59:15.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.054+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe820071680 con 0x7fe804066f30 2023-10-06T04:59:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.064+0000 7fe81cff9700 1 -- 172.21.15.3:0/3336830780 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311633 (secure 0 0 0) 0x7fe820071680 con 0x7fe804066f30 2023-10-06T04:59:15.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.066+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe804066f30 msgr2=0x7fe8040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.066+0000 7fe826e4c700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe804066f30 0x7fe8040693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe814001ed0 tx=0x7fe814015000 comp rx=0 tx=0).stop 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.066+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82016a210 msgr2=0x7fe8200c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.066+0000 7fe826e4c700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82016a210 0x7fe8200c7860 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe8200890f0 tx=0x7fe81800b040 comp rx=0 tx=0).stop 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 shutdown_connections 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe804066f30 0x7fe8040693e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82016a210 0x7fe8200c7860 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe810003a60 0x7fe820155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 --2- 172.21.15.3:0/3336830780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe820169da0 0x7fe8200c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 >> 172.21.15.3:0/3336830780 conn(0x7fe820070310 msgr2=0x7fe8200afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 shutdown_connections 2023-10-06T04:59:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.067+0000 7fe826e4c700 1 -- 172.21.15.3:0/3336830780 wait complete. 2023-10-06T04:59:15.073 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:59:15.081 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-10-06T04:59:13.525793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2446140,"kb_used_data":10492,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752528580,"statfs":{"total":773094113280,"available":770589265920,"internally_reserved":0,"allocated":10743808,"data_stored":5468929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002380"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042247+0000","last_change":"2023-10-06T04:59:06.038068+0000","last_active":"2023-10-06T04:59:06.042247+0000","last_peered":"2023-10-06T04:59:06.042247+0000","last_clean":"2023-10-06T04:59:06.042247+0000","last_became_active":"2023-10-06T04:59:06.037779+0000","last_became_peered":"2023-10-06T04:59:06.037779+0000","last_unstale":"2023-10-06T04:59:06.042247+0000","last_undegraded":"2023-10-06T04:59:06.042247+0000","last_fullsized":"2023-10-06T04:59:06.042247+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:30.027634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043940+0000","last_change":"2023-10-06T04:59:06.034963+0000","last_active":"2023-10-06T04:59:06.043940+0000","last_peered":"2023-10-06T04:59:06.043940+0000","last_clean":"2023-10-06T04:59:06.043940+0000","last_became_active":"2023-10-06T04:59:06.034699+0000","last_became_peered":"2023-10-06T04:59:06.034699+0000","last_unstale":"2023-10-06T04:59:06.043940+0000","last_undegraded":"2023-10-06T04:59:06.043940+0000","last_fullsized":"2023-10-06T04:59:06.043940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:16.596514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.041916+0000","last_change":"2023-10-06T04:59:06.037800+0000","last_active":"2023-10-06T04:59:06.041916+0000","last_peered":"2023-10-06T04:59:06.041916+0000","last_clean":"2023-10-06T04:59:06.041916+0000","last_became_active":"2023-10-06T04:59:06.037521+0000","last_became_peered":"2023-10-06T04:59:06.037521+0000","last_unstale":"2023-10-06T04:59:06.041916+0000","last_undegraded":"2023-10-06T04:59:06.041916+0000","last_fullsized":"2023-10-06T04:59:06.041916+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:24.018318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.051282+0000","last_change":"2023-10-06T04:59:06.047011+0000","last_active":"2023-10-06T04:59:06.051282+0000","last_peered":"2023-10-06T04:59:06.051282+0000","last_clean":"2023-10-06T04:59:06.051282+0000","last_became_active":"2023-10-06T04:59:06.046495+0000","last_became_peered":"2023-10-06T04:59:06.046495+0000","last_unstale":"2023-10-06T04:59:06.051282+0000","last_undegraded":"2023-10-06T04:59:06.051282+0000","last_fullsized":"2023-10-06T04:59:06.051282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046019+0000","last_change":"2023-10-06T04:59:06.041490+0000","last_active":"2023-10-06T04:59:06.046019+0000","last_peered":"2023-10-06T04:59:06.046019+0000","last_clean":"2023-10-06T04:59:06.046019+0000","last_became_active":"2023-10-06T04:59:06.041177+0000","last_became_peered":"2023-10-06T04:59:06.041177+0000","last_unstale":"2023-10-06T04:59:06.046019+0000","last_undegraded":"2023-10-06T04:59:06.046019+0000","last_fullsized":"2023-10-06T04:59:06.046019+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:41.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044000+0000","last_change":"2023-10-06T04:59:06.037886+0000","last_active":"2023-10-06T04:59:06.044000+0000","last_peered":"2023-10-06T04:59:06.044000+0000","last_clean":"2023-10-06T04:59:06.044000+0000","last_became_active":"2023-10-06T04:59:06.037642+0000","last_became_peered":"2023-10-06T04:59:06.037642+0000","last_unstale":"2023-10-06T04:59:06.044000+0000","last_undegraded":"2023-10-06T04:59:06.044000+0000","last_fullsized":"2023-10-06T04:59:06.044000+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:33.236974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043474+0000","last_change":"2023-10-06T04:59:06.034089+0000","last_active":"2023-10-06T04:59:06.043474+0000","last_peered":"2023-10-06T04:59:06.043474+0000","last_clean":"2023-10-06T04:59:06.043474+0000","last_became_active":"2023-10-06T04:59:06.033808+0000","last_became_peered":"2023-10-06T04:59:06.033808+0000","last_unstale":"2023-10-06T04:59:06.043474+0000","last_undegraded":"2023-10-06T04:59:06.043474+0000","last_fullsized":"2023-10-06T04:59:06.043474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:16.545577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043070+0000","last_change":"2023-10-06T04:59:06.041845+0000","last_active":"2023-10-06T04:59:06.043070+0000","last_peered":"2023-10-06T04:59:06.043070+0000","last_clean":"2023-10-06T04:59:06.043070+0000","last_became_active":"2023-10-06T04:59:06.041514+0000","last_became_peered":"2023-10-06T04:59:06.041514+0000","last_unstale":"2023-10-06T04:59:06.043070+0000","last_undegraded":"2023-10-06T04:59:06.043070+0000","last_fullsized":"2023-10-06T04:59:06.043070+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:10.609306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.035114+0000","last_change":"2023-10-06T04:59:06.031918+0000","last_active":"2023-10-06T04:59:06.035114+0000","last_peered":"2023-10-06T04:59:06.035114+0000","last_clean":"2023-10-06T04:59:06.035114+0000","last_became_active":"2023-10-06T04:59:06.031641+0000","last_became_peered":"2023-10-06T04:59:06.031641+0000","last_unstale":"2023-10-06T04:59:06.035114+0000","last_undegraded":"2023-10-06T04:59:06.035114+0000","last_fullsized":"2023-10-06T04:59:06.035114+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:10.384995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039501+0000","last_change":"2023-10-06T04:59:06.036932+0000","last_active":"2023-10-06T04:59:06.039501+0000","last_peered":"2023-10-06T04:59:06.039501+0000","last_clean":"2023-10-06T04:59:06.039501+0000","last_became_active":"2023-10-06T04:59:06.036651+0000","last_became_peered":"2023-10-06T04:59:06.036651+0000","last_unstale":"2023-10-06T04:59:06.039501+0000","last_undegraded":"2023-10-06T04:59:06.039501+0000","last_fullsized":"2023-10-06T04:59:06.039501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:49.356661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044695+0000","last_change":"2023-10-06T04:59:06.034544+0000","last_active":"2023-10-06T04:59:06.044695+0000","last_peered":"2023-10-06T04:59:06.044695+0000","last_clean":"2023-10-06T04:59:06.044695+0000","last_became_active":"2023-10-06T04:59:06.034240+0000","last_became_peered":"2023-10-06T04:59:06.034240+0000","last_unstale":"2023-10-06T04:59:06.044695+0000","last_undegraded":"2023-10-06T04:59:06.044695+0000","last_fullsized":"2023-10-06T04:59:06.044695+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:16.704156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045766+0000","last_change":"2023-10-06T04:59:06.042730+0000","last_active":"2023-10-06T04:59:06.045766+0000","last_peered":"2023-10-06T04:59:06.045766+0000","last_clean":"2023-10-06T04:59:06.045766+0000","last_became_active":"2023-10-06T04:59:06.042188+0000","last_became_peered":"2023-10-06T04:59:06.042188+0000","last_unstale":"2023-10-06T04:59:06.045766+0000","last_undegraded":"2023-10-06T04:59:06.045766+0000","last_fullsized":"2023-10-06T04:59:06.045766+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:53.196280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039919+0000","last_change":"2023-10-06T04:59:06.035450+0000","last_active":"2023-10-06T04:59:06.039919+0000","last_peered":"2023-10-06T04:59:06.039919+0000","last_clean":"2023-10-06T04:59:06.039919+0000","last_became_active":"2023-10-06T04:59:06.035166+0000","last_became_peered":"2023-10-06T04:59:06.035166+0000","last_unstale":"2023-10-06T04:59:06.039919+0000","last_undegraded":"2023-10-06T04:59:06.039919+0000","last_fullsized":"2023-10-06T04:59:06.039919+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:19.436490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044413+0000","last_change":"2023-10-06T04:59:06.037669+0000","last_active":"2023-10-06T04:59:06.044413+0000","last_peered":"2023-10-06T04:59:06.044413+0000","last_clean":"2023-10-06T04:59:06.044413+0000","last_became_active":"2023-10-06T04:59:06.037389+0000","last_became_peered":"2023-10-06T04:59:06.037389+0000","last_unstale":"2023-10-06T04:59:06.044413+0000","last_undegraded":"2023-10-06T04:59:06.044413+0000","last_fullsized":"2023-10-06T04:59:06.044413+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.104668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046491+0000","last_change":"2023-10-06T04:59:06.044897+0000","last_active":"2023-10-06T04:59:06.046491+0000","last_peered":"2023-10-06T04:59:06.046491+0000","last_clean":"2023-10-06T04:59:06.046491+0000","last_became_active":"2023-10-06T04:59:06.044608+0000","last_became_peered":"2023-10-06T04:59:06.044608+0000","last_unstale":"2023-10-06T04:59:06.046491+0000","last_undegraded":"2023-10-06T04:59:06.046491+0000","last_fullsized":"2023-10-06T04:59:06.046491+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:42.388368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.035125+0000","last_change":"2023-10-06T04:59:06.033266+0000","last_active":"2023-10-06T04:59:06.035125+0000","last_peered":"2023-10-06T04:59:06.035125+0000","last_clean":"2023-10-06T04:59:06.035125+0000","last_became_active":"2023-10-06T04:59:06.032982+0000","last_became_peered":"2023-10-06T04:59:06.032982+0000","last_unstale":"2023-10-06T04:59:06.035125+0000","last_undegraded":"2023-10-06T04:59:06.035125+0000","last_fullsized":"2023-10-06T04:59:06.035125+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:11.903244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039147+0000","last_change":"2023-10-06T04:59:06.036068+0000","last_active":"2023-10-06T04:59:06.039147+0000","last_peered":"2023-10-06T04:59:06.039147+0000","last_clean":"2023-10-06T04:59:06.039147+0000","last_became_active":"2023-10-06T04:59:06.035798+0000","last_became_peered":"2023-10-06T04:59:06.035798+0000","last_unstale":"2023-10-06T04:59:06.039147+0000","last_undegraded":"2023-10-06T04:59:06.039147+0000","last_fullsized":"2023-10-06T04:59:06.039147+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:08.561731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044126+0000","last_change":"2023-10-06T04:59:06.038074+0000","last_active":"2023-10-06T04:59:06.044126+0000","last_peered":"2023-10-06T04:59:06.044126+0000","last_clean":"2023-10-06T04:59:06.044126+0000","last_became_active":"2023-10-06T04:59:06.037756+0000","last_became_peered":"2023-10-06T04:59:06.037756+0000","last_unstale":"2023-10-06T04:59:06.044126+0000","last_undegraded":"2023-10-06T04:59:06.044126+0000","last_fullsized":"2023-10-06T04:59:06.044126+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:58.334959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038972+0000","last_change":"2023-10-06T04:59:06.034401+0000","last_active":"2023-10-06T04:59:06.038972+0000","last_peered":"2023-10-06T04:59:06.038972+0000","last_clean":"2023-10-06T04:59:06.038972+0000","last_became_active":"2023-10-06T04:59:06.033982+0000","last_became_peered":"2023-10-06T04:59:06.033982+0000","last_unstale":"2023-10-06T04:59:06.038972+0000","last_undegraded":"2023-10-06T04:59:06.038972+0000","last_fullsized":"2023-10-06T04:59:06.038972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:21.963167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045747+0000","last_change":"2023-10-06T04:59:06.042530+0000","last_active":"2023-10-06T04:59:06.045747+0000","last_peered":"2023-10-06T04:59:06.045747+0000","last_clean":"2023-10-06T04:59:06.045747+0000","last_became_active":"2023-10-06T04:59:06.041470+0000","last_became_peered":"2023-10-06T04:59:06.041470+0000","last_unstale":"2023-10-06T04:59:06.045747+0000","last_undegraded":"2023-10-06T04:59:06.045747+0000","last_fullsized":"2023-10-06T04:59:06.045747+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:45.748990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034853+0000","last_change":"2023-10-06T04:59:06.032101+0000","last_active":"2023-10-06T04:59:06.034853+0000","last_peered":"2023-10-06T04:59:06.034853+0000","last_clean":"2023-10-06T04:59:06.034853+0000","last_became_active":"2023-10-06T04:59:06.031813+0000","last_became_peered":"2023-10-06T04:59:06.031813+0000","last_unstale":"2023-10-06T04:59:06.034853+0000","last_undegraded":"2023-10-06T04:59:06.034853+0000","last_fullsized":"2023-10-06T04:59:06.034853+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:17.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.052178+0000","last_change":"2023-10-06T04:59:06.045745+0000","last_active":"2023-10-06T04:59:06.052178+0000","last_peered":"2023-10-06T04:59:06.052178+0000","last_clean":"2023-10-06T04:59:06.052178+0000","last_became_active":"2023-10-06T04:59:06.045502+0000","last_became_peered":"2023-10-06T04:59:06.045502+0000","last_unstale":"2023-10-06T04:59:06.052178+0000","last_undegraded":"2023-10-06T04:59:06.052178+0000","last_fullsized":"2023-10-06T04:59:06.052178+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:05.065894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038999+0000","last_change":"2023-10-06T04:59:06.035681+0000","last_active":"2023-10-06T04:59:06.038999+0000","last_peered":"2023-10-06T04:59:06.038999+0000","last_clean":"2023-10-06T04:59:06.038999+0000","last_became_active":"2023-10-06T04:59:06.035410+0000","last_became_peered":"2023-10-06T04:59:06.035410+0000","last_unstale":"2023-10-06T04:59:06.038999+0000","last_undegraded":"2023-10-06T04:59:06.038999+0000","last_fullsized":"2023-10-06T04:59:06.038999+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:00.221865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039851+0000","last_change":"2023-10-06T04:59:06.036403+0000","last_active":"2023-10-06T04:59:06.039851+0000","last_peered":"2023-10-06T04:59:06.039851+0000","last_clean":"2023-10-06T04:59:06.039851+0000","last_became_active":"2023-10-06T04:59:06.036136+0000","last_became_peered":"2023-10-06T04:59:06.036136+0000","last_unstale":"2023-10-06T04:59:06.039851+0000","last_undegraded":"2023-10-06T04:59:06.039851+0000","last_fullsized":"2023-10-06T04:59:06.039851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:47.264143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.050425+0000","last_change":"2023-10-06T04:59:06.044784+0000","last_active":"2023-10-06T04:59:06.050425+0000","last_peered":"2023-10-06T04:59:06.050425+0000","last_clean":"2023-10-06T04:59:06.050425+0000","last_became_active":"2023-10-06T04:59:06.044308+0000","last_became_peered":"2023-10-06T04:59:06.044308+0000","last_unstale":"2023-10-06T04:59:06.050425+0000","last_undegraded":"2023-10-06T04:59:06.050425+0000","last_fullsized":"2023-10-06T04:59:06.050425+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:04.736845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042470+0000","last_change":"2023-10-06T04:59:06.040545+0000","last_active":"2023-10-06T04:59:06.042470+0000","last_peered":"2023-10-06T04:59:06.042470+0000","last_clean":"2023-10-06T04:59:06.042470+0000","last_became_active":"2023-10-06T04:59:06.040199+0000","last_became_peered":"2023-10-06T04:59:06.040199+0000","last_unstale":"2023-10-06T04:59:06.042470+0000","last_undegraded":"2023-10-06T04:59:06.042470+0000","last_fullsized":"2023-10-06T04:59:06.042470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:53.403275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044724+0000","last_change":"2023-10-06T04:59:06.036141+0000","last_active":"2023-10-06T04:59:06.044724+0000","last_peered":"2023-10-06T04:59:06.044724+0000","last_clean":"2023-10-06T04:59:06.044724+0000","last_became_active":"2023-10-06T04:59:06.035373+0000","last_became_peered":"2023-10-06T04:59:06.035373+0000","last_unstale":"2023-10-06T04:59:06.044724+0000","last_undegraded":"2023-10-06T04:59:06.044724+0000","last_fullsized":"2023-10-06T04:59:06.044724+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:05.065894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043944+0000","last_change":"2023-10-06T04:59:06.035866+0000","last_active":"2023-10-06T04:59:06.043944+0000","last_peered":"2023-10-06T04:59:06.043944+0000","last_clean":"2023-10-06T04:59:06.043944+0000","last_became_active":"2023-10-06T04:59:06.035611+0000","last_became_peered":"2023-10-06T04:59:06.035611+0000","last_unstale":"2023-10-06T04:59:06.043944+0000","last_undegraded":"2023-10-06T04:59:06.043944+0000","last_fullsized":"2023-10-06T04:59:06.043944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:06.804711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.031235+0000","last_change":"2023-10-06T04:59:06.030767+0000","last_active":"2023-10-06T04:59:06.031235+0000","last_peered":"2023-10-06T04:59:06.031235+0000","last_clean":"2023-10-06T04:59:06.031235+0000","last_became_active":"2023-10-06T04:59:06.029903+0000","last_became_peered":"2023-10-06T04:59:06.029903+0000","last_unstale":"2023-10-06T04:59:06.031235+0000","last_undegraded":"2023-10-06T04:59:06.031235+0000","last_fullsized":"2023-10-06T04:59:06.031235+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:26.680872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046166+0000","last_change":"2023-10-06T04:59:06.044577+0000","last_active":"2023-10-06T04:59:06.046166+0000","last_peered":"2023-10-06T04:59:06.046166+0000","last_clean":"2023-10-06T04:59:06.046166+0000","last_became_active":"2023-10-06T04:59:06.044307+0000","last_became_peered":"2023-10-06T04:59:06.044307+0000","last_unstale":"2023-10-06T04:59:06.046166+0000","last_undegraded":"2023-10-06T04:59:06.046166+0000","last_fullsized":"2023-10-06T04:59:06.046166+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:23.371571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043495+0000","last_change":"2023-10-06T04:59:06.037209+0000","last_active":"2023-10-06T04:59:06.043495+0000","last_peered":"2023-10-06T04:59:06.043495+0000","last_clean":"2023-10-06T04:59:06.043495+0000","last_became_active":"2023-10-06T04:59:06.036973+0000","last_became_peered":"2023-10-06T04:59:06.036973+0000","last_unstale":"2023-10-06T04:59:06.043495+0000","last_undegraded":"2023-10-06T04:59:06.043495+0000","last_fullsized":"2023-10-06T04:59:06.043495+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:17.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043049+0000","last_change":"2023-10-06T04:59:06.041834+0000","last_active":"2023-10-06T04:59:06.043049+0000","last_peered":"2023-10-06T04:59:06.043049+0000","last_clean":"2023-10-06T04:59:06.043049+0000","last_became_active":"2023-10-06T04:59:06.041130+0000","last_became_peered":"2023-10-06T04:59:06.041130+0000","last_unstale":"2023-10-06T04:59:06.043049+0000","last_undegraded":"2023-10-06T04:59:06.043049+0000","last_fullsized":"2023-10-06T04:59:06.043049+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:55.051458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034743+0000","last_change":"2023-10-06T04:59:06.032570+0000","last_active":"2023-10-06T04:59:06.034743+0000","last_peered":"2023-10-06T04:59:06.034743+0000","last_clean":"2023-10-06T04:59:06.034743+0000","last_became_active":"2023-10-06T04:59:06.032311+0000","last_became_peered":"2023-10-06T04:59:06.032311+0000","last_unstale":"2023-10-06T04:59:06.034743+0000","last_undegraded":"2023-10-06T04:59:06.034743+0000","last_fullsized":"2023-10-06T04:59:06.034743+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:33.709775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"95'4","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054300+0000","last_change":"2023-10-06T04:59:05.030540+0000","last_active":"2023-10-06T04:59:06.054300+0000","last_peered":"2023-10-06T04:59:06.054300+0000","last_clean":"2023-10-06T04:59:06.054300+0000","last_became_active":"2023-10-06T04:59:05.030260+0000","last_became_peered":"2023-10-06T04:59:05.030260+0000","last_unstale":"2023-10-06T04:59:06.054300+0000","last_undegraded":"2023-10-06T04:59:06.054300+0000","last_fullsized":"2023-10-06T04:59:06.054300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038511+0000","last_change":"2023-10-06T04:59:06.034197+0000","last_active":"2023-10-06T04:59:06.038511+0000","last_peered":"2023-10-06T04:59:06.038511+0000","last_clean":"2023-10-06T04:59:06.038511+0000","last_became_active":"2023-10-06T04:59:06.033834+0000","last_became_peered":"2023-10-06T04:59:06.033834+0000","last_unstale":"2023-10-06T04:59:06.038511+0000","last_undegraded":"2023-10-06T04:59:06.038511+0000","last_fullsized":"2023-10-06T04:59:06.038511+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:22.404213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.053565+0000","last_change":"2023-10-06T04:59:05.026990+0000","last_active":"2023-10-06T04:59:06.053565+0000","last_peered":"2023-10-06T04:59:06.053565+0000","last_clean":"2023-10-06T04:59:06.053565+0000","last_became_active":"2023-10-06T04:59:05.026693+0000","last_became_peered":"2023-10-06T04:59:05.026693+0000","last_unstale":"2023-10-06T04:59:06.053565+0000","last_undegraded":"2023-10-06T04:59:06.053565+0000","last_fullsized":"2023-10-06T04:59:06.053565+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:49.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045669+0000","last_change":"2023-10-06T04:59:06.041542+0000","last_active":"2023-10-06T04:59:06.045669+0000","last_peered":"2023-10-06T04:59:06.045669+0000","last_clean":"2023-10-06T04:59:06.045669+0000","last_became_active":"2023-10-06T04:59:06.041152+0000","last_became_peered":"2023-10-06T04:59:06.041152+0000","last_unstale":"2023-10-06T04:59:06.045669+0000","last_undegraded":"2023-10-06T04:59:06.045669+0000","last_fullsized":"2023-10-06T04:59:06.045669+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:34.810780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054088+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:06.054088+0000","last_peered":"2023-10-06T04:59:06.054088+0000","last_clean":"2023-10-06T04:59:06.054088+0000","last_became_active":"2023-10-06T04:59:05.020908+0000","last_became_peered":"2023-10-06T04:59:05.020908+0000","last_unstale":"2023-10-06T04:59:06.054088+0000","last_undegraded":"2023-10-06T04:59:06.054088+0000","last_fullsized":"2023-10-06T04:59:06.054088+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.255698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045933+0000","last_change":"2023-10-06T04:59:06.043586+0000","last_active":"2023-10-06T04:59:06.045933+0000","last_peered":"2023-10-06T04:59:06.045933+0000","last_clean":"2023-10-06T04:59:06.045933+0000","last_became_active":"2023-10-06T04:59:06.043273+0000","last_became_peered":"2023-10-06T04:59:06.043273+0000","last_unstale":"2023-10-06T04:59:06.045933+0000","last_undegraded":"2023-10-06T04:59:06.045933+0000","last_fullsized":"2023-10-06T04:59:06.045933+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:01.732144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054886+0000","last_change":"2023-10-06T04:59:05.023617+0000","last_active":"2023-10-06T04:59:06.054886+0000","last_peered":"2023-10-06T04:59:06.054886+0000","last_clean":"2023-10-06T04:59:06.054886+0000","last_became_active":"2023-10-06T04:59:05.023394+0000","last_became_peered":"2023-10-06T04:59:05.023394+0000","last_unstale":"2023-10-06T04:59:06.054886+0000","last_undegraded":"2023-10-06T04:59:06.054886+0000","last_fullsized":"2023-10-06T04:59:06.054886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.562815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.041211+0000","last_change":"2023-10-06T04:59:06.038723+0000","last_active":"2023-10-06T04:59:06.041211+0000","last_peered":"2023-10-06T04:59:06.041211+0000","last_clean":"2023-10-06T04:59:06.041211+0000","last_became_active":"2023-10-06T04:59:06.038383+0000","last_became_peered":"2023-10-06T04:59:06.038383+0000","last_unstale":"2023-10-06T04:59:06.041211+0000","last_undegraded":"2023-10-06T04:59:06.041211+0000","last_fullsized":"2023-10-06T04:59:06.041211+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:45.065649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.050600+0000","last_change":"2023-10-06T04:59:05.030809+0000","last_active":"2023-10-06T04:59:06.050600+0000","last_peered":"2023-10-06T04:59:06.050600+0000","last_clean":"2023-10-06T04:59:06.050600+0000","last_became_active":"2023-10-06T04:59:05.030550+0000","last_became_peered":"2023-10-06T04:59:05.030550+0000","last_unstale":"2023-10-06T04:59:06.050600+0000","last_undegraded":"2023-10-06T04:59:06.050600+0000","last_fullsized":"2023-10-06T04:59:06.050600+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038993+0000","last_change":"2023-10-06T04:59:06.034665+0000","last_active":"2023-10-06T04:59:06.038993+0000","last_peered":"2023-10-06T04:59:06.038993+0000","last_clean":"2023-10-06T04:59:06.038993+0000","last_became_active":"2023-10-06T04:59:06.034343+0000","last_became_peered":"2023-10-06T04:59:06.034343+0000","last_unstale":"2023-10-06T04:59:06.038993+0000","last_undegraded":"2023-10-06T04:59:06.038993+0000","last_fullsized":"2023-10-06T04:59:06.038993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:35.827029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.053879+0000","last_change":"2023-10-06T04:59:05.024000+0000","last_active":"2023-10-06T04:59:06.053879+0000","last_peered":"2023-10-06T04:59:06.053879+0000","last_clean":"2023-10-06T04:59:06.053879+0000","last_became_active":"2023-10-06T04:59:05.023706+0000","last_became_peered":"2023-10-06T04:59:05.023706+0000","last_unstale":"2023-10-06T04:59:06.053879+0000","last_undegraded":"2023-10-06T04:59:06.053879+0000","last_fullsized":"2023-10-06T04:59:06.053879+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043959+0000","last_change":"2023-10-06T04:59:06.036931+0000","last_active":"2023-10-06T04:59:06.043959+0000","last_peered":"2023-10-06T04:59:06.043959+0000","last_clean":"2023-10-06T04:59:06.043959+0000","last_became_active":"2023-10-06T04:59:06.036339+0000","last_became_peered":"2023-10-06T04:59:06.036339+0000","last_unstale":"2023-10-06T04:59:06.043959+0000","last_undegraded":"2023-10-06T04:59:06.043959+0000","last_fullsized":"2023-10-06T04:59:06.043959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:15.943200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.031443+0000","last_change":"2023-10-06T04:59:05.021204+0000","last_active":"2023-10-06T04:59:06.031443+0000","last_peered":"2023-10-06T04:59:06.031443+0000","last_clean":"2023-10-06T04:59:06.031443+0000","last_became_active":"2023-10-06T04:59:05.020967+0000","last_became_peered":"2023-10-06T04:59:05.020967+0000","last_unstale":"2023-10-06T04:59:06.031443+0000","last_undegraded":"2023-10-06T04:59:06.031443+0000","last_fullsized":"2023-10-06T04:59:06.031443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:54.137958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039952+0000","last_change":"2023-10-06T04:59:06.035514+0000","last_active":"2023-10-06T04:59:06.039952+0000","last_peered":"2023-10-06T04:59:06.039952+0000","last_clean":"2023-10-06T04:59:06.039952+0000","last_became_active":"2023-10-06T04:59:06.035260+0000","last_became_peered":"2023-10-06T04:59:06.035260+0000","last_unstale":"2023-10-06T04:59:06.039952+0000","last_undegraded":"2023-10-06T04:59:06.039952+0000","last_fullsized":"2023-10-06T04:59:06.039952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:00.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.022944+0000","last_change":"2023-10-06T04:59:05.025368+0000","last_active":"2023-10-06T04:59:06.022944+0000","last_peered":"2023-10-06T04:59:06.022944+0000","last_clean":"2023-10-06T04:59:06.022944+0000","last_became_active":"2023-10-06T04:59:05.025101+0000","last_became_peered":"2023-10-06T04:59:05.025101+0000","last_unstale":"2023-10-06T04:59:06.022944+0000","last_undegraded":"2023-10-06T04:59:06.022944+0000","last_fullsized":"2023-10-06T04:59:06.022944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042741+0000","last_change":"2023-10-06T04:59:06.038836+0000","last_active":"2023-10-06T04:59:06.042741+0000","last_peered":"2023-10-06T04:59:06.042741+0000","last_clean":"2023-10-06T04:59:06.042741+0000","last_became_active":"2023-10-06T04:59:06.038271+0000","last_became_peered":"2023-10-06T04:59:06.038271+0000","last_unstale":"2023-10-06T04:59:06.042741+0000","last_undegraded":"2023-10-06T04:59:06.042741+0000","last_fullsized":"2023-10-06T04:59:06.042741+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:12.926654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.031200+0000","last_change":"2023-10-06T04:59:05.021703+0000","last_active":"2023-10-06T04:59:06.031200+0000","last_peered":"2023-10-06T04:59:06.031200+0000","last_clean":"2023-10-06T04:59:06.031200+0000","last_became_active":"2023-10-06T04:59:05.021413+0000","last_became_peered":"2023-10-06T04:59:05.021413+0000","last_unstale":"2023-10-06T04:59:06.031200+0000","last_undegraded":"2023-10-06T04:59:06.031200+0000","last_fullsized":"2023-10-06T04:59:06.031200+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:15.699249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.037345+0000","last_change":"2023-10-06T04:59:06.033225+0000","last_active":"2023-10-06T04:59:06.037345+0000","last_peered":"2023-10-06T04:59:06.037345+0000","last_clean":"2023-10-06T04:59:06.037345+0000","last_became_active":"2023-10-06T04:59:06.032769+0000","last_became_peered":"2023-10-06T04:59:06.032769+0000","last_unstale":"2023-10-06T04:59:06.037345+0000","last_undegraded":"2023-10-06T04:59:06.037345+0000","last_fullsized":"2023-10-06T04:59:06.037345+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:49.048000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.025061+0000","last_change":"2023-10-06T04:59:05.016394+0000","last_active":"2023-10-06T04:59:06.025061+0000","last_peered":"2023-10-06T04:59:06.025061+0000","last_clean":"2023-10-06T04:59:06.025061+0000","last_became_active":"2023-10-06T04:59:05.016106+0000","last_became_peered":"2023-10-06T04:59:05.016106+0000","last_unstale":"2023-10-06T04:59:06.025061+0000","last_undegraded":"2023-10-06T04:59:06.025061+0000","last_fullsized":"2023-10-06T04:59:06.025061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:10.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043794+0000","last_change":"2023-10-06T04:59:06.038121+0000","last_active":"2023-10-06T04:59:06.043794+0000","last_peered":"2023-10-06T04:59:06.043794+0000","last_clean":"2023-10-06T04:59:06.043794+0000","last_became_active":"2023-10-06T04:59:06.037844+0000","last_became_peered":"2023-10-06T04:59:06.037844+0000","last_unstale":"2023-10-06T04:59:06.043794+0000","last_undegraded":"2023-10-06T04:59:06.043794+0000","last_fullsized":"2023-10-06T04:59:06.043794+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:58.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"95'3","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.057717+0000","last_change":"2023-10-06T04:59:05.026950+0000","last_active":"2023-10-06T04:59:06.057717+0000","last_peered":"2023-10-06T04:59:06.057717+0000","last_clean":"2023-10-06T04:59:06.057717+0000","last_became_active":"2023-10-06T04:59:05.026723+0000","last_became_peered":"2023-10-06T04:59:05.026723+0000","last_unstale":"2023-10-06T04:59:06.057717+0000","last_undegraded":"2023-10-06T04:59:06.057717+0000","last_fullsized":"2023-10-06T04:59:06.057717+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:52.191373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039188+0000","last_change":"2023-10-06T04:59:06.035752+0000","last_active":"2023-10-06T04:59:06.039188+0000","last_peered":"2023-10-06T04:59:06.039188+0000","last_clean":"2023-10-06T04:59:06.039188+0000","last_became_active":"2023-10-06T04:59:06.035165+0000","last_became_peered":"2023-10-06T04:59:06.035165+0000","last_unstale":"2023-10-06T04:59:06.039188+0000","last_undegraded":"2023-10-06T04:59:06.039188+0000","last_fullsized":"2023-10-06T04:59:06.039188+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:21.883224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"95'6","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.060375+0000","last_change":"2023-10-06T04:59:05.025160+0000","last_active":"2023-10-06T04:59:06.060375+0000","last_peered":"2023-10-06T04:59:06.060375+0000","last_clean":"2023-10-06T04:59:06.060375+0000","last_became_active":"2023-10-06T04:59:05.024909+0000","last_became_peered":"2023-10-06T04:59:05.024909+0000","last_unstale":"2023-10-06T04:59:06.060375+0000","last_undegraded":"2023-10-06T04:59:06.060375+0000","last_fullsized":"2023-10-06T04:59:06.060375+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:00.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044934+0000","last_change":"2023-10-06T04:59:06.041960+0000","last_active":"2023-10-06T04:59:06.044934+0000","last_peered":"2023-10-06T04:59:06.044934+0000","last_clean":"2023-10-06T04:59:06.044934+0000","last_became_active":"2023-10-06T04:59:06.041649+0000","last_became_peered":"2023-10-06T04:59:06.041649+0000","last_unstale":"2023-10-06T04:59:06.044934+0000","last_undegraded":"2023-10-06T04:59:06.044934+0000","last_fullsized":"2023-10-06T04:59:06.044934+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:55.142865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.057437+0000","last_change":"2023-10-06T04:59:05.023984+0000","last_active":"2023-10-06T04:59:06.057437+0000","last_peered":"2023-10-06T04:59:06.057437+0000","last_clean":"2023-10-06T04:59:06.057437+0000","last_became_active":"2023-10-06T04:59:05.023641+0000","last_became_peered":"2023-10-06T04:59:05.023641+0000","last_unstale":"2023-10-06T04:59:06.057437+0000","last_undegraded":"2023-10-06T04:59:06.057437+0000","last_fullsized":"2023-10-06T04:59:06.057437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:54.810243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039722+0000","last_change":"2023-10-06T04:59:06.032585+0000","last_active":"2023-10-06T04:59:06.039722+0000","last_peered":"2023-10-06T04:59:06.039722+0000","last_clean":"2023-10-06T04:59:06.039722+0000","last_became_active":"2023-10-06T04:59:06.032306+0000","last_became_peered":"2023-10-06T04:59:06.032306+0000","last_unstale":"2023-10-06T04:59:06.039722+0000","last_undegraded":"2023-10-06T04:59:06.039722+0000","last_fullsized":"2023-10-06T04:59:06.039722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:05.265074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.023480+0000","last_change":"2023-10-06T04:59:05.025232+0000","last_active":"2023-10-06T04:59:06.023480+0000","last_peered":"2023-10-06T04:59:06.023480+0000","last_clean":"2023-10-06T04:59:06.023480+0000","last_became_active":"2023-10-06T04:59:05.024850+0000","last_became_peered":"2023-10-06T04:59:05.024850+0000","last_unstale":"2023-10-06T04:59:06.023480+0000","last_undegraded":"2023-10-06T04:59:06.023480+0000","last_fullsized":"2023-10-06T04:59:06.023480+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:33.805873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043059+0000","last_change":"2023-10-06T04:59:06.042021+0000","last_active":"2023-10-06T04:59:06.043059+0000","last_peered":"2023-10-06T04:59:06.043059+0000","last_clean":"2023-10-06T04:59:06.043059+0000","last_became_active":"2023-10-06T04:59:06.041759+0000","last_became_peered":"2023-10-06T04:59:06.041759+0000","last_unstale":"2023-10-06T04:59:06.043059+0000","last_undegraded":"2023-10-06T04:59:06.043059+0000","last_fullsized":"2023-10-06T04:59:06.043059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:57.975524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021678+0000","last_change":"2023-10-06T04:59:05.020554+0000","last_active":"2023-10-06T04:59:06.021678+0000","last_peered":"2023-10-06T04:59:06.021678+0000","last_clean":"2023-10-06T04:59:06.021678+0000","last_became_active":"2023-10-06T04:59:05.020294+0000","last_became_peered":"2023-10-06T04:59:05.020294+0000","last_unstale":"2023-10-06T04:59:06.021678+0000","last_undegraded":"2023-10-06T04:59:06.021678+0000","last_fullsized":"2023-10-06T04:59:06.021678+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:24.547416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039127+0000","last_change":"2023-10-06T04:59:06.035847+0000","last_active":"2023-10-06T04:59:06.039127+0000","last_peered":"2023-10-06T04:59:06.039127+0000","last_clean":"2023-10-06T04:59:06.039127+0000","last_became_active":"2023-10-06T04:59:06.035462+0000","last_became_peered":"2023-10-06T04:59:06.035462+0000","last_unstale":"2023-10-06T04:59:06.039127+0000","last_undegraded":"2023-10-06T04:59:06.039127+0000","last_fullsized":"2023-10-06T04:59:06.039127+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:03.596586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"95'8","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054885+0000","last_change":"2023-10-06T04:59:05.022660+0000","last_active":"2023-10-06T04:59:06.054885+0000","last_peered":"2023-10-06T04:59:06.054885+0000","last_clean":"2023-10-06T04:59:06.054885+0000","last_became_active":"2023-10-06T04:59:05.021894+0000","last_became_peered":"2023-10-06T04:59:05.021894+0000","last_unstale":"2023-10-06T04:59:06.054885+0000","last_undegraded":"2023-10-06T04:59:06.054885+0000","last_fullsized":"2023-10-06T04:59:06.054885+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.654900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034711+0000","last_change":"2023-10-06T04:59:06.031277+0000","last_active":"2023-10-06T04:59:06.034711+0000","last_peered":"2023-10-06T04:59:06.034711+0000","last_clean":"2023-10-06T04:59:06.034711+0000","last_became_active":"2023-10-06T04:59:06.030991+0000","last_became_peered":"2023-10-06T04:59:06.030991+0000","last_unstale":"2023-10-06T04:59:06.034711+0000","last_undegraded":"2023-10-06T04:59:06.034711+0000","last_fullsized":"2023-10-06T04:59:06.034711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:43.083569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054900+0000","last_change":"2023-10-06T04:59:05.022693+0000","last_active":"2023-10-06T04:59:06.054900+0000","last_peered":"2023-10-06T04:59:06.054900+0000","last_clean":"2023-10-06T04:59:06.054900+0000","last_became_active":"2023-10-06T04:59:05.022421+0000","last_became_peered":"2023-10-06T04:59:05.022421+0000","last_unstale":"2023-10-06T04:59:06.054900+0000","last_undegraded":"2023-10-06T04:59:06.054900+0000","last_fullsized":"2023-10-06T04:59:06.054900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:59.701496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042234+0000","last_change":"2023-10-06T04:59:06.041119+0000","last_active":"2023-10-06T04:59:06.042234+0000","last_peered":"2023-10-06T04:59:06.042234+0000","last_clean":"2023-10-06T04:59:06.042234+0000","last_became_active":"2023-10-06T04:59:06.040816+0000","last_became_peered":"2023-10-06T04:59:06.040816+0000","last_unstale":"2023-10-06T04:59:06.042234+0000","last_undegraded":"2023-10-06T04:59:06.042234+0000","last_fullsized":"2023-10-06T04:59:06.042234+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:51.603555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"96'1","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-10-06T04:59:07.058308+0000","last_change":"2023-10-06T04:59:05.024903+0000","last_active":"2023-10-06T04:59:07.058308+0000","last_peered":"2023-10-06T04:59:07.058308+0000","last_clean":"2023-10-06T04:59:07.058308+0000","last_became_active":"2023-10-06T04:59:05.024608+0000","last_became_peered":"2023-10-06T04:59:05.024608+0000","last_unstale":"2023-10-06T04:59:07.058308+0000","last_undegraded":"2023-10-06T04:59:07.058308+0000","last_fullsized":"2023-10-06T04:59:07.058308+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:48.099761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039701+0000","last_change":"2023-10-06T04:59:06.033420+0000","last_active":"2023-10-06T04:59:06.039701+0000","last_peered":"2023-10-06T04:59:06.039701+0000","last_clean":"2023-10-06T04:59:06.039701+0000","last_became_active":"2023-10-06T04:59:06.033107+0000","last_became_peered":"2023-10-06T04:59:06.033107+0000","last_unstale":"2023-10-06T04:59:06.039701+0000","last_undegraded":"2023-10-06T04:59:06.039701+0000","last_fullsized":"2023-10-06T04:59:06.039701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:31.872736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026818+0000","last_change":"2023-10-06T04:59:05.030722+0000","last_active":"2023-10-06T04:59:06.026818+0000","last_peered":"2023-10-06T04:59:06.026818+0000","last_clean":"2023-10-06T04:59:06.026818+0000","last_became_active":"2023-10-06T04:59:05.030276+0000","last_became_peered":"2023-10-06T04:59:05.030276+0000","last_unstale":"2023-10-06T04:59:06.026818+0000","last_undegraded":"2023-10-06T04:59:06.026818+0000","last_fullsized":"2023-10-06T04:59:06.026818+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:57.328563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044890+0000","last_change":"2023-10-06T04:59:06.042074+0000","last_active":"2023-10-06T04:59:06.044890+0000","last_peered":"2023-10-06T04:59:06.044890+0000","last_clean":"2023-10-06T04:59:06.044890+0000","last_became_active":"2023-10-06T04:59:06.038267+0000","last_became_peered":"2023-10-06T04:59:06.038267+0000","last_unstale":"2023-10-06T04:59:06.044890+0000","last_undegraded":"2023-10-06T04:59:06.044890+0000","last_fullsized":"2023-10-06T04:59:06.044890+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:49.356661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.033122+0000","last_change":"2023-10-06T04:59:05.024303+0000","last_active":"2023-10-06T04:59:06.033122+0000","last_peered":"2023-10-06T04:59:06.033122+0000","last_clean":"2023-10-06T04:59:06.033122+0000","last_became_active":"2023-10-06T04:59:05.024038+0000","last_became_peered":"2023-10-06T04:59:05.024038+0000","last_unstale":"2023-10-06T04:59:06.033122+0000","last_undegraded":"2023-10-06T04:59:06.033122+0000","last_fullsized":"2023-10-06T04:59:06.033122+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:09.380088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044804+0000","last_change":"2023-10-06T04:59:06.042312+0000","last_active":"2023-10-06T04:59:06.044804+0000","last_peered":"2023-10-06T04:59:06.044804+0000","last_clean":"2023-10-06T04:59:06.044804+0000","last_became_active":"2023-10-06T04:59:06.041490+0000","last_became_peered":"2023-10-06T04:59:06.041490+0000","last_unstale":"2023-10-06T04:59:06.044804+0000","last_undegraded":"2023-10-06T04:59:06.044804+0000","last_fullsized":"2023-10-06T04:59:06.044804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:08.338740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.034069+0000","last_change":"2023-10-06T04:59:05.023856+0000","last_active":"2023-10-06T04:59:06.034069+0000","last_peered":"2023-10-06T04:59:06.034069+0000","last_clean":"2023-10-06T04:59:06.034069+0000","last_became_active":"2023-10-06T04:59:05.023612+0000","last_became_peered":"2023-10-06T04:59:05.023612+0000","last_unstale":"2023-10-06T04:59:06.034069+0000","last_undegraded":"2023-10-06T04:59:06.034069+0000","last_fullsized":"2023-10-06T04:59:06.034069+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:04.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042994+0000","last_change":"2023-10-06T04:59:06.041624+0000","last_active":"2023-10-06T04:59:06.042994+0000","last_peered":"2023-10-06T04:59:06.042994+0000","last_clean":"2023-10-06T04:59:06.042994+0000","last_became_active":"2023-10-06T04:59:06.041360+0000","last_became_peered":"2023-10-06T04:59:06.041360+0000","last_unstale":"2023-10-06T04:59:06.042994+0000","last_undegraded":"2023-10-06T04:59:06.042994+0000","last_fullsized":"2023-10-06T04:59:06.042994+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:50:43.443720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021745+0000","last_change":"2023-10-06T04:59:05.023970+0000","last_active":"2023-10-06T04:59:06.021745+0000","last_peered":"2023-10-06T04:59:06.021745+0000","last_clean":"2023-10-06T04:59:06.021745+0000","last_became_active":"2023-10-06T04:59:05.023712+0000","last_became_peered":"2023-10-06T04:59:05.023712+0000","last_unstale":"2023-10-06T04:59:06.021745+0000","last_undegraded":"2023-10-06T04:59:06.021745+0000","last_fullsized":"2023-10-06T04:59:06.021745+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:31.733624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.046432+0000","last_change":"2023-10-06T04:59:06.044979+0000","last_active":"2023-10-06T04:59:06.046432+0000","last_peered":"2023-10-06T04:59:06.046432+0000","last_clean":"2023-10-06T04:59:06.046432+0000","last_became_active":"2023-10-06T04:59:06.044703+0000","last_became_peered":"2023-10-06T04:59:06.044703+0000","last_unstale":"2023-10-06T04:59:06.046432+0000","last_undegraded":"2023-10-06T04:59:06.046432+0000","last_fullsized":"2023-10-06T04:59:06.046432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:00.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026891+0000","last_change":"2023-10-06T04:59:05.028495+0000","last_active":"2023-10-06T04:59:06.026891+0000","last_peered":"2023-10-06T04:59:06.026891+0000","last_clean":"2023-10-06T04:59:06.026891+0000","last_became_active":"2023-10-06T04:59:05.028219+0000","last_became_peered":"2023-10-06T04:59:05.028219+0000","last_unstale":"2023-10-06T04:59:06.026891+0000","last_undegraded":"2023-10-06T04:59:06.026891+0000","last_fullsized":"2023-10-06T04:59:06.026891+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:54.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.045137+0000","last_change":"2023-10-06T04:59:06.035823+0000","last_active":"2023-10-06T04:59:06.045137+0000","last_peered":"2023-10-06T04:59:06.045137+0000","last_clean":"2023-10-06T04:59:06.045137+0000","last_became_active":"2023-10-06T04:59:06.035551+0000","last_became_peered":"2023-10-06T04:59:06.035551+0000","last_unstale":"2023-10-06T04:59:06.045137+0000","last_undegraded":"2023-10-06T04:59:06.045137+0000","last_fullsized":"2023-10-06T04:59:06.045137+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:24.260605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021954+0000","last_change":"2023-10-06T04:59:05.023535+0000","last_active":"2023-10-06T04:59:06.021954+0000","last_peered":"2023-10-06T04:59:06.021954+0000","last_clean":"2023-10-06T04:59:06.021954+0000","last_became_active":"2023-10-06T04:59:05.023261+0000","last_became_peered":"2023-10-06T04:59:05.023261+0000","last_unstale":"2023-10-06T04:59:06.021954+0000","last_undegraded":"2023-10-06T04:59:06.021954+0000","last_fullsized":"2023-10-06T04:59:06.021954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:47.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.036062+0000","last_change":"2023-10-06T04:59:06.032000+0000","last_active":"2023-10-06T04:59:06.036062+0000","last_peered":"2023-10-06T04:59:06.036062+0000","last_clean":"2023-10-06T04:59:06.036062+0000","last_became_active":"2023-10-06T04:59:06.031738+0000","last_became_peered":"2023-10-06T04:59:06.031738+0000","last_unstale":"2023-10-06T04:59:06.036062+0000","last_undegraded":"2023-10-06T04:59:06.036062+0000","last_fullsized":"2023-10-06T04:59:06.036062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:17.752764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.052121+0000","last_change":"2023-10-06T04:59:05.026906+0000","last_active":"2023-10-06T04:59:06.052121+0000","last_peered":"2023-10-06T04:59:06.052121+0000","last_clean":"2023-10-06T04:59:06.052121+0000","last_became_active":"2023-10-06T04:59:05.026633+0000","last_became_peered":"2023-10-06T04:59:05.026633+0000","last_unstale":"2023-10-06T04:59:06.052121+0000","last_undegraded":"2023-10-06T04:59:06.052121+0000","last_fullsized":"2023-10-06T04:59:06.052121+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:56.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.029616+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:59:06.029616+0000","last_peered":"2023-10-06T04:59:06.029616+0000","last_clean":"2023-10-06T04:59:06.029616+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:59:06.029616+0000","last_undegraded":"2023-10-06T04:59:06.029616+0000","last_fullsized":"2023-10-06T04:59:06.029616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.043881+0000","last_change":"2023-10-06T04:59:06.041830+0000","last_active":"2023-10-06T04:59:06.043881+0000","last_peered":"2023-10-06T04:59:06.043881+0000","last_clean":"2023-10-06T04:59:06.043881+0000","last_became_active":"2023-10-06T04:59:06.041558+0000","last_became_peered":"2023-10-06T04:59:06.041558+0000","last_unstale":"2023-10-06T04:59:06.043881+0000","last_undegraded":"2023-10-06T04:59:06.043881+0000","last_fullsized":"2023-10-06T04:59:06.043881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:10.771633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021346+0000","last_change":"2023-10-06T04:59:05.028380+0000","last_active":"2023-10-06T04:59:06.021346+0000","last_peered":"2023-10-06T04:59:06.021346+0000","last_clean":"2023-10-06T04:59:06.021346+0000","last_became_active":"2023-10-06T04:59:05.028098+0000","last_became_peered":"2023-10-06T04:59:05.028098+0000","last_unstale":"2023-10-06T04:59:06.021346+0000","last_undegraded":"2023-10-06T04:59:06.021346+0000","last_fullsized":"2023-10-06T04:59:06.021346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:22.276266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.023345+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:59:06.023345+0000","last_peered":"2023-10-06T04:59:06.023345+0000","last_clean":"2023-10-06T04:59:06.023345+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:59:06.023345+0000","last_undegraded":"2023-10-06T04:59:06.023345+0000","last_fullsized":"2023-10-06T04:59:06.023345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.040333+0000","last_change":"2023-10-06T04:59:06.033951+0000","last_active":"2023-10-06T04:59:06.040333+0000","last_peered":"2023-10-06T04:59:06.040333+0000","last_clean":"2023-10-06T04:59:06.040333+0000","last_became_active":"2023-10-06T04:59:06.033626+0000","last_became_peered":"2023-10-06T04:59:06.033626+0000","last_unstale":"2023-10-06T04:59:06.040333+0000","last_undegraded":"2023-10-06T04:59:06.040333+0000","last_fullsized":"2023-10-06T04:59:06.040333+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:14.960791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054103+0000","last_change":"2023-10-06T04:59:05.024110+0000","last_active":"2023-10-06T04:59:06.054103+0000","last_peered":"2023-10-06T04:59:06.054103+0000","last_clean":"2023-10-06T04:59:06.054103+0000","last_became_active":"2023-10-06T04:59:05.023840+0000","last_became_peered":"2023-10-06T04:59:05.023840+0000","last_unstale":"2023-10-06T04:59:06.054103+0000","last_undegraded":"2023-10-06T04:59:06.054103+0000","last_fullsized":"2023-10-06T04:59:06.054103+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:50.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.032795+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:59:06.032795+0000","last_peered":"2023-10-06T04:59:06.032795+0000","last_clean":"2023-10-06T04:59:06.032795+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:59:06.032795+0000","last_undegraded":"2023-10-06T04:59:06.032795+0000","last_fullsized":"2023-10-06T04:59:06.032795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":224,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.025871+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:59:06.025871+0000","last_peered":"2023-10-06T04:59:06.025871+0000","last_clean":"2023-10-06T04:59:06.025871+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:59:06.025871+0000","last_undegraded":"2023-10-06T04:59:06.025871+0000","last_fullsized":"2023-10-06T04:59:06.025871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039490+0000","last_change":"2023-10-06T04:59:06.036003+0000","last_active":"2023-10-06T04:59:06.039490+0000","last_peered":"2023-10-06T04:59:06.039490+0000","last_clean":"2023-10-06T04:59:06.039490+0000","last_became_active":"2023-10-06T04:59:06.035458+0000","last_became_peered":"2023-10-06T04:59:06.035458+0000","last_unstale":"2023-10-06T04:59:06.039490+0000","last_undegraded":"2023-10-06T04:59:06.039490+0000","last_fullsized":"2023-10-06T04:59:06.039490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:55.142865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.027365+0000","last_change":"2023-10-06T04:59:05.030905+0000","last_active":"2023-10-06T04:59:06.027365+0000","last_peered":"2023-10-06T04:59:06.027365+0000","last_clean":"2023-10-06T04:59:06.027365+0000","last_became_active":"2023-10-06T04:59:05.030530+0000","last_became_peered":"2023-10-06T04:59:05.030530+0000","last_unstale":"2023-10-06T04:59:06.027365+0000","last_undegraded":"2023-10-06T04:59:06.027365+0000","last_fullsized":"2023-10-06T04:59:06.027365+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:15.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026199+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:59:06.026199+0000","last_peered":"2023-10-06T04:59:06.026199+0000","last_clean":"2023-10-06T04:59:06.026199+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:59:06.026199+0000","last_undegraded":"2023-10-06T04:59:06.026199+0000","last_fullsized":"2023-10-06T04:59:06.026199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.044118+0000","last_change":"2023-10-06T04:59:06.035806+0000","last_active":"2023-10-06T04:59:06.044118+0000","last_peered":"2023-10-06T04:59:06.044118+0000","last_clean":"2023-10-06T04:59:06.044118+0000","last_became_active":"2023-10-06T04:59:06.035465+0000","last_became_peered":"2023-10-06T04:59:06.035465+0000","last_unstale":"2023-10-06T04:59:06.044118+0000","last_undegraded":"2023-10-06T04:59:06.044118+0000","last_fullsized":"2023-10-06T04:59:06.044118+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:41.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.051300+0000","last_change":"2023-10-06T04:59:05.028526+0000","last_active":"2023-10-06T04:59:06.051300+0000","last_peered":"2023-10-06T04:59:06.051300+0000","last_clean":"2023-10-06T04:59:06.051300+0000","last_became_active":"2023-10-06T04:59:05.028134+0000","last_became_peered":"2023-10-06T04:59:05.028134+0000","last_unstale":"2023-10-06T04:59:06.051300+0000","last_undegraded":"2023-10-06T04:59:06.051300+0000","last_fullsized":"2023-10-06T04:59:06.051300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:16.407534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.025959+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:59:06.025959+0000","last_peered":"2023-10-06T04:59:06.025959+0000","last_clean":"2023-10-06T04:59:06.025959+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:59:06.025959+0000","last_undegraded":"2023-10-06T04:59:06.025959+0000","last_fullsized":"2023-10-06T04:59:06.025959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.038814+0000","last_change":"2023-10-06T04:59:06.034490+0000","last_active":"2023-10-06T04:59:06.038814+0000","last_peered":"2023-10-06T04:59:06.038814+0000","last_clean":"2023-10-06T04:59:06.038814+0000","last_became_active":"2023-10-06T04:59:06.034228+0000","last_became_peered":"2023-10-06T04:59:06.034228+0000","last_unstale":"2023-10-06T04:59:06.038814+0000","last_undegraded":"2023-10-06T04:59:06.038814+0000","last_fullsized":"2023-10-06T04:59:06.038814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.531534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.054134+0000","last_change":"2023-10-06T04:59:05.023805+0000","last_active":"2023-10-06T04:59:06.054134+0000","last_peered":"2023-10-06T04:59:06.054134+0000","last_clean":"2023-10-06T04:59:06.054134+0000","last_became_active":"2023-10-06T04:59:05.023560+0000","last_became_peered":"2023-10-06T04:59:05.023560+0000","last_unstale":"2023-10-06T04:59:06.054134+0000","last_undegraded":"2023-10-06T04:59:06.054134+0000","last_fullsized":"2023-10-06T04:59:06.054134+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:16.588598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.027734+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:59:06.027734+0000","last_peered":"2023-10-06T04:59:06.027734+0000","last_clean":"2023-10-06T04:59:06.027734+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:59:06.027734+0000","last_undegraded":"2023-10-06T04:59:06.027734+0000","last_fullsized":"2023-10-06T04:59:06.027734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.042442+0000","last_change":"2023-10-06T04:59:06.038092+0000","last_active":"2023-10-06T04:59:06.042442+0000","last_peered":"2023-10-06T04:59:06.042442+0000","last_clean":"2023-10-06T04:59:06.042442+0000","last_became_active":"2023-10-06T04:59:06.037773+0000","last_became_peered":"2023-10-06T04:59:06.037773+0000","last_unstale":"2023-10-06T04:59:06.042442+0000","last_undegraded":"2023-10-06T04:59:06.042442+0000","last_fullsized":"2023-10-06T04:59:06.042442+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:49.295778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.021849+0000","last_change":"2023-10-06T04:59:05.020425+0000","last_active":"2023-10-06T04:59:06.021849+0000","last_peered":"2023-10-06T04:59:06.021849+0000","last_clean":"2023-10-06T04:59:06.021849+0000","last_became_active":"2023-10-06T04:59:05.020172+0000","last_became_peered":"2023-10-06T04:59:05.020172+0000","last_unstale":"2023-10-06T04:59:06.021849+0000","last_undegraded":"2023-10-06T04:59:06.021849+0000","last_fullsized":"2023-10-06T04:59:06.021849+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:25.681695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.027873+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:59:06.027873+0000","last_peered":"2023-10-06T04:59:06.027873+0000","last_clean":"2023-10-06T04:59:06.027873+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:59:06.027873+0000","last_undegraded":"2023-10-06T04:59:06.027873+0000","last_fullsized":"2023-10-06T04:59:06.027873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.039817+0000","last_change":"2023-10-06T04:59:06.034739+0000","last_active":"2023-10-06T04:59:06.039817+0000","last_peered":"2023-10-06T04:59:06.039817+0000","last_clean":"2023-10-06T04:59:06.039817+0000","last_became_active":"2023-10-06T04:59:06.034438+0000","last_became_peered":"2023-10-06T04:59:06.034438+0000","last_unstale":"2023-10-06T04:59:06.039817+0000","last_undegraded":"2023-10-06T04:59:06.039817+0000","last_fullsized":"2023-10-06T04:59:06.039817+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:05.004991+0000","last_clean_scrub_stamp":"2023-10-06T04:59:05.004991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:17.016243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026061+0000","last_change":"2023-10-06T04:59:05.028402+0000","last_active":"2023-10-06T04:59:06.026061+0000","last_peered":"2023-10-06T04:59:06.026061+0000","last_clean":"2023-10-06T04:59:06.026061+0000","last_became_active":"2023-10-06T04:59:05.028139+0000","last_became_peered":"2023-10-06T04:59:05.028139+0000","last_unstale":"2023-10-06T04:59:06.026061+0000","last_undegraded":"2023-10-06T04:59:06.026061+0000","last_fullsized":"2023-10-06T04:59:06.026061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:04.000084+0000","last_clean_scrub_stamp":"2023-10-06T04:59:04.000084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:56.970617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-06T04:59:06.026721+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:59:06.026721+0000","last_peered":"2023-10-06T04:59:06.026721+0000","last_clean":"2023-10-06T04:59:06.026721+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:59:06.026721+0000","last_undegraded":"2023-10-06T04:59:06.026721+0000","last_fullsized":"2023-10-06T04:59:06.026721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307224,"kb_used_data":1744,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064616,"statfs":{"total":96636764160,"available":96322166784,"internally_reserved":0,"allocated":1785856,"data_stored":1125130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.73399999999999999},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73199999999999998,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.497}]},{"osd":1,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":1.073},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.51600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":1.111},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.58999999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.503},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.315}]},{"osd":5,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":1.214},"last":0.33200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.372},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1208320,"data_stored":538067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.52200000000000002},"min":{"1min":0.54900000000000004,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.81200000000000006,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.57199999999999995},"min":{"1min":0.56100000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.39800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":1.234,"15min":1.234},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.55200000000000005},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81000000000000005,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75800000000000001,"5min":3.669,"15min":3.669},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.747},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307220,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064620,"statfs":{"total":96636764160,"available":96322170880,"internally_reserved":0,"allocated":1781760,"data_stored":1125004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.23000000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.27900000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.501,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.377}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.70999999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.85499999999999998,"15min":0.92900000000000005},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.64100000000000001},"min":{"1min":0.47499999999999998,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.998,"15min":0.998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.65800000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":1.0900000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80600000000000005,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.026},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.79400000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306656,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065184,"statfs":{"total":96636764160,"available":96322748416,"internally_reserved":0,"allocated":1204224,"data_stored":537581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.505,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57699999999999996,"5min":0.628,"15min":0.628},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.873},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90700000000000003,"5min":1.022,"15min":1.022},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.495,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.5,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96599999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.995,"15min":0.995},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302548,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069292,"statfs":{"total":96636764160,"available":96326955008,"internally_reserved":0,"allocated":1191936,"data_stored":535989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.51200000000000001,"5min":0.747,"15min":1.048},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.42599999999999999,"15min":0.39000000000000001},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":0.61699999999999999,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.36099999999999999},"min":{"1min":0.29499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.31900000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40999999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.37},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.5,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306644,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065196,"statfs":{"total":96636764160,"available":96322760704,"internally_reserved":0,"allocated":1191936,"data_stored":536509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38700000000000001,"15min":0.40600000000000003},"min":{"1min":0.251,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":2.008,"15min":2.008},"last":0.307},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41299999999999998,"15min":0.56499999999999995},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54500000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42399999999999999,"15min":0.46000000000000002},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.64600000000000002,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.627,"5min":1.379,"15min":5.069},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":2.246,"15min":2.246},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83199999999999996,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82199999999999995,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 04:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.79400000000000004},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.40400000000000003},"max":{"1min":0.76200000000000001,"5min":1.964,"15min":6.367},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65000000000000002,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":1.3,"15min":5},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306648,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065192,"statfs":{"total":96636764160,"available":96322756608,"internally_reserved":0,"allocated":1196032,"data_stored":536047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.372},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.78500000000000003},"min":{"1min":0.38200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.82099999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.80400000000000005,"15min":5.415},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.66700000000000004},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":4.202},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":1.044,"15min":4.2130000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.504},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.745}]},{"osd":6,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:58:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.77600000000000002,"5min":0.86199999999999999,"15min":0.872},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.313},"max":{"1min":0.84399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306636,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065204,"statfs":{"total":96636764160,"available":96322768896,"internally_reserved":0,"allocated":1183744,"data_stored":534602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.67600000000000005},"min":{"1min":0.36899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.625},"min":{"1min":0.436,"5min":0.247,"15min":0.247},"max":{"1min":0.76000000000000001,"5min":0.80800000000000005,"15min":3.7679999999999998},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.70799999999999996},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":6.7949999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.75800000000000001},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.93799999999999994},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":9.7210000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.93500000000000005},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":9.7539999999999996},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.504},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.34200000000000003},"max":{"1min":0.66300000000000003,"5min":0.746,"15min":0.746},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80100000000000005},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:59:15.088 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:59:15.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:59:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.245+0000 7f63c9d13700 1 Processor -- start 2023-10-06T04:59:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.245+0000 7f63c9d13700 1 -- start start 2023-10-06T04:59:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c9d13700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c4159eb0 0x7f63c40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c9d13700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63c40cef10 0x7f63c415da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c9d13700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c40afca0 con 0x7f63c40cfb80 2023-10-06T04:59:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c9d13700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c415d510 con 0x7f63c40cef10 2023-10-06T04:59:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c9d13700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c415a480 con 0x7f63c4159eb0 2023-10-06T04:59:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f63c40cfb80 0x7f63c40af590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58952/0 (socket says 172.21.15.3:58952) 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63c40cef10 0x7f63c415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c37fe700 1 -- 172.21.15.3:0/3257618634 learned_addr learned my addr 172.21.15.3:0/3257618634 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c2ffd700 1 --2- 172.21.15.3:0/3257618634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c4159eb0 0x7f63c40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c3fff700 1 -- 172.21.15.3:0/3257618634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c4159eb0 msgr2=0x7f63c40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c3fff700 1 --2- 172.21.15.3:0/3257618634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c4159eb0 0x7f63c40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c3fff700 1 -- 172.21.15.3:0/3257618634 >> v1:172.21.15.3:6789/0 conn(0x7f63c40cfb80 legacy=0x7f63c40af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c27fc700 1 -- 172.21.15.3:0/3257618634 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842279379 0 0) 0x7f63c40afca0 con 0x7f63c40cfb80 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c3fff700 1 -- 172.21.15.3:0/3257618634 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c415a760 con 0x7f63c40cef10 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c3fff700 1 --2- 172.21.15.3:0/3257618634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c40cef10 0x7f63c415da10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009e60 tx=0x7f63bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262534b610bb0909 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c27fc700 1 -- 172.21.15.3:0/3257618634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00e070 con 0x7f63c40cef10 2023-10-06T04:59:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c27fc700 1 -- 172.21.15.3:0/3257618634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc00bc30 con 0x7f63c40cef10 2023-10-06T04:59:15.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.246+0000 7f63c27fc700 1 -- 172.21.15.3:0/3257618634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc012430 con 0x7f63c40cef10 2023-10-06T04:59:15.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 -- 172.21.15.3:0/3257618634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c40cef10 msgr2=0x7f63c415da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 --2- 172.21.15.3:0/3257618634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c40cef10 0x7f63c415da10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009e60 tx=0x7f63bc009780 comp rx=0 tx=0).stop 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 -- 172.21.15.3:0/3257618634 shutdown_connections 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 --2- 172.21.15.3:0/3257618634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c40cef10 0x7f63c415da10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 --2- 172.21.15.3:0/3257618634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c4159eb0 0x7f63c40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 -- 172.21.15.3:0/3257618634 >> 172.21.15.3:0/3257618634 conn(0x7f63c4070310 msgr2=0x7f63c4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 -- 172.21.15.3:0/3257618634 shutdown_connections 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.247+0000 7f63c9d13700 1 -- 172.21.15.3:0/3257618634 wait complete. 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.248+0000 7f63c9d13700 1 Processor -- start 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.248+0000 7f63c9d13700 1 -- start start 2023-10-06T04:59:15.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c9d13700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c40cef10 0x7f63c40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c9d13700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c40cfb80 0x7f63c4154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c9d13700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c4159eb0 0x7f63c40c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c9d13700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40b4cc0 con 0x7f63c4159eb0 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c9d13700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40b1fc0 con 0x7f63c40cfb80 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c9d13700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40b4f30 con 0x7f63c40cef10 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c4159eb0 0x7f63c40c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c4159eb0 0x7f63c40c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37190/0 (socket says 172.21.15.3:37190) 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 -- 172.21.15.3:0/521573900 learned_addr learned my addr 172.21.15.3:0/521573900 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c37fe700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c40cef10 0x7f63c40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c2ffd700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c40cfb80 0x7f63c4154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 -- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c40cef10 msgr2=0x7f63c40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c40cef10 0x7f63c40c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 -- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c40cfb80 msgr2=0x7f63c4154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c40cfb80 0x7f63c4154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.249+0000 7f63c3fff700 1 -- 172.21.15.3:0/521573900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc009a20 con 0x7f63c4159eb0 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.250+0000 7f63c3fff700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c4159eb0 0x7f63c40c74f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f63bc013000 tx=0x7f63bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.250+0000 7f63ab7fe700 1 -- 172.21.15.3:0/521573900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc027470 con 0x7f63c4159eb0 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.250+0000 7f63ab7fe700 1 -- 172.21.15.3:0/521573900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc0275f0 con 0x7f63c4159eb0 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.250+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c40b5210 con 0x7f63c4159eb0 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.250+0000 7f63ab7fe700 1 -- 172.21.15.3:0/521573900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0279b0 con 0x7f63c4159eb0 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.250+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c40b33e0 con 0x7f63c4159eb0 2023-10-06T04:59:15.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.251+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c40b5420 con 0x7f63c4159eb0 2023-10-06T04:59:15.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.251+0000 7f63ab7fe700 1 -- 172.21.15.3:0/521573900 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63bc024070 con 0x7f63c4159eb0 2023-10-06T04:59:15.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.251+0000 7f63ab7fe700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b0066f30 0x7f63b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.252+0000 7f63ab7fe700 1 -- 172.21.15.3:0/521573900 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f63bc086e40 con 0x7f63c4159eb0 2023-10-06T04:59:15.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.252+0000 7f63c37fe700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b0066f30 0x7f63b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.252+0000 7f63c37fe700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b0066f30 0x7f63b00693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f63b4007fd0 tx=0x7f63b4007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:15.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.256+0000 7f63ab7fe700 1 -- 172.21.15.3:0/521573900 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63bc053f50 con 0x7f63c4159eb0 2023-10-06T04:59:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.527+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f63c404c380 con 0x7f63c4159eb0 2023-10-06T04:59:15.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.531+0000 7f63ab7fe700 1 -- 172.21.15.3:0/521573900 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f63bc057570 con 0x7f63c4159eb0 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b0066f30 msgr2=0x7f63b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b0066f30 0x7f63b00693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f63b4007fd0 tx=0x7f63b4007d30 comp rx=0 tx=0).stop 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c4159eb0 msgr2=0x7f63c40c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c4159eb0 0x7f63c40c74f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f63bc013000 tx=0x7f63bc019000 comp rx=0 tx=0).stop 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 shutdown_connections 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f63b0066f30 0x7f63b00693e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63c4159eb0 0x7f63c40c74f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c40cef10 0x7f63c40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 --2- 172.21.15.3:0/521573900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c40cfb80 0x7f63c4154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 >> 172.21.15.3:0/521573900 conn(0x7f63c4070310 msgr2=0x7f63c4158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 shutdown_connections 2023-10-06T04:59:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.533+0000 7f63c9d13700 1 -- 172.21.15.3:0/521573900 wait complete. 2023-10-06T04:59:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T04:59:15.548 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:15.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.665+0000 7f8df99c0700 1 Processor -- start 2023-10-06T04:59:15.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.665+0000 7f8df99c0700 1 -- start start 2023-10-06T04:59:15.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df99c0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df99c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8df415ccc0 0x7f8df40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df99c0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8df40d06a0 0x7f8df406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df99c0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df40cf6c0 con 0x7f8df415ccc0 2023-10-06T04:59:15.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df99c0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df40cfb50 con 0x7f8df40d06a0 2023-10-06T04:59:15.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df99c0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df415e050 con 0x7f8df415c8f0 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8df415ccc0 0x7f8df40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8df415ccc0 0x7f8df40d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37206/0 (socket says 172.21.15.3:37206) 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.666+0000 7f8df27fc700 1 -- 172.21.15.3:0/227109823 learned_addr learned my addr 172.21.15.3:0/227109823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df2ffd700 1 --2- 172.21.15.3:0/227109823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df37fe700 1 --2- 172.21.15.3:0/227109823 >> v2:172.21.15.153:3300/0 conn(0x7f8df40d06a0 0x7f8df406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df27fc700 1 -- 172.21.15.3:0/227109823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 msgr2=0x7f8df415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df27fc700 1 --2- 172.21.15.3:0/227109823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df27fc700 1 -- 172.21.15.3:0/227109823 >> v2:172.21.15.153:3300/0 conn(0x7f8df40d06a0 msgr2=0x7f8df406d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df27fc700 1 --2- 172.21.15.3:0/227109823 >> v2:172.21.15.153:3300/0 conn(0x7f8df40d06a0 0x7f8df406d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df27fc700 1 -- 172.21.15.3:0/227109823 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40d0af0 con 0x7f8df415ccc0 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df2ffd700 1 --2- 172.21.15.3:0/227109823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:15.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df37fe700 1 --2- 172.21.15.3:0/227109823 >> v2:172.21.15.153:3300/0 conn(0x7f8df40d06a0 0x7f8df406d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df27fc700 1 --2- 172.21.15.3:0/227109823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40d0160 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8df40af8a0 tx=0x7f8ddc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649b3a00038e7371 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df1ffb700 1 -- 172.21.15.3:0/227109823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00b860 con 0x7f8df415ccc0 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df1ffb700 1 -- 172.21.15.3:0/227109823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc00b9c0 con 0x7f8df415ccc0 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df1ffb700 1 -- 172.21.15.3:0/227109823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00bd60 con 0x7f8df415ccc0 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 -- 172.21.15.3:0/227109823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 msgr2=0x7f8df40d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 --2- 172.21.15.3:0/227109823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40d0160 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8df40af8a0 tx=0x7f8ddc009250 comp rx=0 tx=0).stop 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 -- 172.21.15.3:0/227109823 shutdown_connections 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 --2- 172.21.15.3:0/227109823 >> v2:172.21.15.153:3300/0 conn(0x7f8df40d06a0 0x7f8df406d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 --2- 172.21.15.3:0/227109823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 --2- 172.21.15.3:0/227109823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40d0160 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 -- 172.21.15.3:0/227109823 >> 172.21.15.3:0/227109823 conn(0x7f8df4070310 msgr2=0x7f8df4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:15.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.667+0000 7f8df99c0700 1 -- 172.21.15.3:0/227109823 shutdown_connections 2023-10-06T04:59:15.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.668+0000 7f8df99c0700 1 -- 172.21.15.3:0/227109823 wait complete. 2023-10-06T04:59:15.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 Processor -- start 2023-10-06T04:59:15.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 -- start start 2023-10-06T04:59:15.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df40d06a0 0x7f8df415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df4156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df415a830 con 0x7f8df415ccc0 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4159c20 con 0x7f8df40d06a0 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.669+0000 7f8df99c0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4159f00 con 0x7f8df415c8f0 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37208/0 (socket says 172.21.15.3:37208) 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 -- 172.21.15.3:0/865645198 learned_addr learned my addr 172.21.15.3:0/865645198 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df27fc700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df4156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df2ffd700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df40d06a0 0x7f8df415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 -- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 msgr2=0x7f8df4156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df4156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 -- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df40d06a0 msgr2=0x7f8df415acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df40d06a0 0x7f8df415acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 -- 172.21.15.3:0/865645198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0094f0 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df37fe700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40cb400 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8de400ce40 tx=0x7f8de40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8deaffd700 1 -- 172.21.15.3:0/865645198 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de400adf0 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8deaffd700 1 -- 172.21.15.3:0/865645198 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de4016420 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df415a1e0 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8deaffd700 1 -- 172.21.15.3:0/865645198 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40167e0 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.670+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df41573a0 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.671+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df4071680 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.672+0000 7f8deaffd700 1 -- 172.21.15.3:0/865645198 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8de4016b50 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.672+0000 7f8deaffd700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8dd4066f30 0x7f8dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.672+0000 7f8deaffd700 1 -- 172.21.15.3:0/865645198 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8de4012070 con 0x7f8df415ccc0 2023-10-06T04:59:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.672+0000 7f8df2ffd700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8dd4066f30 0x7f8dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:15.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.673+0000 7f8df2ffd700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8dd4066f30 0x7f8dd40693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8df4064a40 tx=0x7f8dec006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:15.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.676+0000 7f8deaffd700 1 -- 172.21.15.3:0/865645198 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8de404e5a0 con 0x7f8df415ccc0 2023-10-06T04:59:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.916+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8df40cdf90 con 0x7f8df415ccc0 2023-10-06T04:59:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.917+0000 7f8deaffd700 1 -- 172.21.15.3:0/865645198 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f8de4052120 con 0x7f8df415ccc0 2023-10-06T04:59:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:15.920 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:09.043192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8dd4066f30 msgr2=0x7f8dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8dd4066f30 0x7f8dd40693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8df4064a40 tx=0x7f8dec006f90 comp rx=0 tx=0).stop 2023-10-06T04:59:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 msgr2=0x7f8df40cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40cb400 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8de400ce40 tx=0x7f8de40077d0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 shutdown_connections 2023-10-06T04:59:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8dd4066f30 0x7f8dd40693e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df40d06a0 0x7f8df415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8df415ccc0 0x7f8df40cb400 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 --2- 172.21.15.3:0/865645198 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8df415c8f0 0x7f8df4156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 >> 172.21.15.3:0/865645198 conn(0x7f8df4070310 msgr2=0x7f8df40afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 shutdown_connections 2023-10-06T04:59:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:15.919+0000 7f8df99c0700 1 -- 172.21.15.3:0/865645198 wait complete. 2023-10-06T04:59:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 59 2023-10-06T04:59:16.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:59:05.413962+0000', 'modified': '2023-10-06T04:59:09.043192+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7071}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7071': {'gid': 7071, 'name': 'd', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6837/1968748272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6835', 'nonce': 1968748272}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 1968748272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:59:16.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:59:16.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T04:59:16.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.459+0000 7fce12c2a700 1 Processor -- start 2023-10-06T04:59:16.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.459+0000 7fce12c2a700 1 -- start start 2023-10-06T04:59:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce12c2a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce12c2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce0c15a220 0x7fce0c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce12c2a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c15aeb0 con 0x7fce0c157400 2023-10-06T04:59:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce12c2a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c15a7a0 con 0x7fce0c15a220 2023-10-06T04:59:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce12c2a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c15aa80 con 0x7fce0c157030 2023-10-06T04:59:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce0c15a220 0x7fce0c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce111c7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fce0c157400 0x7fce0c069340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58968/0 (socket says 172.21.15.3:58968) 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce0c15a220 0x7fce0c0b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37214/0 (socket says 172.21.15.3:37214) 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce111c7700 1 -- 172.21.15.3:0/425044286 learned_addr learned my addr 172.21.15.3:0/425044286 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce109c6700 1 --2- 172.21.15.3:0/425044286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0bfff700 1 -- 172.21.15.3:0/425044286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 msgr2=0x7fce0c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0bfff700 1 --2- 172.21.15.3:0/425044286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0bfff700 1 -- 172.21.15.3:0/425044286 >> v1:172.21.15.3:6789/0 conn(0x7fce0c157400 legacy=0x7fce0c069340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0bfff700 1 -- 172.21.15.3:0/425044286 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c06dba0 con 0x7fce0c15a220 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0b7fe700 1 -- 172.21.15.3:0/425044286 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618539367 0 0) 0x7fce0c15aeb0 con 0x7fce0c157400 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce109c6700 1 --2- 172.21.15.3:0/425044286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.460+0000 7fce0bfff700 1 --2- 172.21.15.3:0/425044286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c15a220 0x7fce0c0b3630 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fce000098e0 tx=0x7fce000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cca9af3602dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce0b7fe700 1 -- 172.21.15.3:0/425044286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00009e80 con 0x7fce0c15a220 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce0b7fe700 1 -- 172.21.15.3:0/425044286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0000b860 con 0x7fce0c15a220 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce0b7fe700 1 -- 172.21.15.3:0/425044286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0000bbe0 con 0x7fce0c15a220 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 -- 172.21.15.3:0/425044286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c15a220 msgr2=0x7fce0c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 --2- 172.21.15.3:0/425044286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c15a220 0x7fce0c0b3630 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fce000098e0 tx=0x7fce000091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 -- 172.21.15.3:0/425044286 shutdown_connections 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 --2- 172.21.15.3:0/425044286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 --2- 172.21.15.3:0/425044286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c15a220 0x7fce0c0b3630 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 -- 172.21.15.3:0/425044286 >> 172.21.15.3:0/425044286 conn(0x7fce0c0ce270 msgr2=0x7fce0c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 -- 172.21.15.3:0/425044286 shutdown_connections 2023-10-06T04:59:16.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.461+0000 7fce12c2a700 1 -- 172.21.15.3:0/425044286 wait complete. 2023-10-06T04:59:16.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.462+0000 7fce12c2a700 1 Processor -- start 2023-10-06T04:59:16.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.462+0000 7fce12c2a700 1 -- start start 2023-10-06T04:59:16.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce12c2a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce12c2a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce0c157400 0x7fce0c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce12c2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c151810 0x7fce0c1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce12c2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0c8d20 con 0x7fce0c151810 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce12c2a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0aefe0 con 0x7fce0c157400 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce12c2a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0af2c0 con 0x7fce0c157030 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce111c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c151810 0x7fce0c1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce111c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c151810 0x7fce0c1523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37218/0 (socket says 172.21.15.3:37218) 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce111c7700 1 -- 172.21.15.3:0/1226871656 learned_addr learned my addr 172.21.15.3:0/1226871656 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce109c6700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.463+0000 7fce0bfff700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce0c157400 0x7fce0c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce111c7700 1 -- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 msgr2=0x7fce0c0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce111c7700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c0c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce111c7700 1 -- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce0c157400 msgr2=0x7fce0c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce111c7700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce0c157400 0x7fce0c151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce111c7700 1 -- 172.21.15.3:0/1226871656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce00009420 con 0x7fce0c151810 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce111c7700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c151810 0x7fce0c1523f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fce0400cc20 tx=0x7fce040019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce08ff9700 1 -- 172.21.15.3:0/1226871656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400a870 con 0x7fce0c151810 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce08ff9700 1 -- 172.21.15.3:0/1226871656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0400a9d0 con 0x7fce0c151810 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c0c8480 con 0x7fce0c151810 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce08ff9700 1 -- 172.21.15.3:0/1226871656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400ad90 con 0x7fce0c151810 2023-10-06T04:59:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.464+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c0c8a40 con 0x7fce0c151810 2023-10-06T04:59:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.466+0000 7fce08ff9700 1 -- 172.21.15.3:0/1226871656 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce04007300 con 0x7fce0c151810 2023-10-06T04:59:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.466+0000 7fce08ff9700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcdf8066f30 0x7fcdf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.466+0000 7fce08ff9700 1 -- 172.21.15.3:0/1226871656 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce04084960 con 0x7fce0c151810 2023-10-06T04:59:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.466+0000 7fce109c6700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcdf8066f30 0x7fcdf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.467+0000 7fce109c6700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcdf8066f30 0x7fcdf80693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00adc0 tx=0x7fcdfc006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.468+0000 7fce1bf22b80 1 Processor -- start 2023-10-06T04:59:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.468+0000 7fce1bf22b80 1 -- start start 2023-10-06T04:59:16.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a48816d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4880edc80 con 0x55a4881b4a90 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4880c4a70 con 0x55a48816ebd0 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4880c2270 con 0x55a4881709a0 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37230/0 (socket says 172.21.15.3:37230) 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 -- 172.21.15.3:0/1848948855 learned_addr learned my addr 172.21.15.3:0/1848948855 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce0bfff700 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a48816d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce111c7700 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 -- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 msgr2=0x55a48816d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a48816d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 -- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 msgr2=0x55a48816dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 -- 172.21.15.3:0/1848948855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a487f58270 con 0x55a4881b4a90 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce0bfff700 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a48816d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce111c7700 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fce109c6700 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881b4e60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00ff10 tx=0x7fcdfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c9d6da8b4cf30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fcdf1ffb700 1 -- 172.21.15.3:0/1848948855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc013d50 con 0x55a4881b4a90 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fcdf1ffb700 1 -- 172.21.15.3:0/1848948855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc019070 con 0x55a4881b4a90 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.469+0000 7fcdf1ffb700 1 -- 172.21.15.3:0/1848948855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc015680 con 0x55a4881b4a90 2023-10-06T04:59:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 -- 172.21.15.3:0/1848948855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 msgr2=0x55a4881b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881b4e60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00ff10 tx=0x7fcdfc014000 comp rx=0 tx=0).stop 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 -- 172.21.15.3:0/1848948855 shutdown_connections 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881b4e60 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/1848948855 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a48816d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 -- 172.21.15.3:0/1848948855 >> 172.21.15.3:0/1848948855 conn(0x55a48816f070 msgr2=0x55a48816f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 -- 172.21.15.3:0/1848948855 shutdown_connections 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 -- 172.21.15.3:0/1848948855 wait complete. 2023-10-06T04:59:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.470+0000 7fce1bf22b80 1 -- start start 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.471+0000 7fce1bf22b80 1 Processor -- start 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.471+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a48816ebd0 0x55a488198030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.471+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a488195a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.471+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881b4a90 0x55a488195f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.471+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4880be060 con 0x55a48816ebd0 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a488069890 con 0x55a4881709a0 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a488084ea0 con 0x55a4881b4a90 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce1bf22b80 10 client.0 _ll_register_callbacks cb 0x55a487f5af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a48816ebd0 0x55a488198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a48816ebd0 0x55a488198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37244/0 (socket says 172.21.15.3:37244) 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 -- 172.21.15.3:0/2228222187 learned_addr learned my addr 172.21.15.3:0/2228222187 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce0bfff700 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a488195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce111c7700 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881b4a90 0x55a488195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 -- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881b4a90 msgr2=0x55a488195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881b4a90 0x55a488195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 -- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 msgr2=0x55a488195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a488195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 -- 172.21.15.3:0/2228222187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc00fb10 con 0x55a48816ebd0 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce109c6700 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a48816ebd0 0x55a488198030 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00f8b0 tx=0x7fcdfc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b379998b1acc355 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fcdeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:59:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fcdeaffd700 1 -- 172.21.15.3:0/2228222187 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc015b80 con 0x55a48816ebd0 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fcdeaffd700 1 -- 172.21.15.3:0/2228222187 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcdfc015ce0 con 0x55a48816ebd0 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a488162690 con 0x55a48816ebd0 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fcdeaffd700 1 -- 172.21.15.3:0/2228222187 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdfc015ea0 con 0x55a48816ebd0 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce1bf22b80 10 client.7230 fetch_fsmap learned FSMap version 59 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.472+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55a487efafe0 con 0x55a48816ebd0 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fcdeaffd700 1 -- 172.21.15.3:0/2228222187 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a488162690 con 0x55a48816ebd0 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fcdeaffd700 1 -- 172.21.15.3:0/2228222187 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcdfc005a00 con 0x55a48816ebd0 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce1bf22b80 10 client.7230 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce1bf22b80 10 client.7230 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce1bf22b80 20 client.7230 populate_metadata read hostname 'smithi003' 2023-10-06T04:59:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881b1610 0x55a4881b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce1bf22b80 4 client.7230 mds_command: new command op to 7071 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:59:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 --> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a48813f5c0 con 0x55a4881b1610 2023-10-06T04:59:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce0bfff700 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881b1610 0x55a4881b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fcdeaffd700 10 client.7230 ms_handle_connect on v2:172.21.15.153:6835/1968748272 2023-10-06T04:59:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.473+0000 7fce0bfff700 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881b1610 0x55a4881b19e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b15228fc4cd2210 server_cookie=a0ad7377343e87f in_seq=0 out_seq=0 2023-10-06T04:59:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fcdeaffd700 1 -- 172.21.15.3:0/2228222187 <== mds.0 v2:172.21.15.153:6835/1968748272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a48813f5c0 con 0x55a4881b1610 2023-10-06T04:59:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fcdeaffd700 10 client.7230 handle_command_reply: tid=0 2023-10-06T04:59:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 client.7230 shutdown 2023-10-06T04:59:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a48816ebd0 msgr2=0x55a488198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a48816ebd0 0x55a488198030 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00f8b0 tx=0x7fcdfc019000 comp rx=0 tx=0).stop 2023-10-06T04:59:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 shutdown_connections 2023-10-06T04:59:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881b1610 0x55a4881b19e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881b4a90 0x55a488195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a48816ebd0 0x55a488198030 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fcdeaffd700 0 client.7230 ms_handle_reset on v2:172.21.15.153:6835/1968748272 2023-10-06T04:59:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/2228222187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a488195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 >> 172.21.15.3:0/2228222187 conn(0x55a48816f070 msgr2=0x55a4881b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 shutdown_connections 2023-10-06T04:59:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.474+0000 7fce1bf22b80 1 -- 172.21.15.3:0/2228222187 wait complete. 2023-10-06T04:59:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.475+0000 7fce1bf22b80 20 client.7230 trim_cache size 0 max 16384 2023-10-06T04:59:16.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.491+0000 7fce1bf22b80 1 Processor -- start 2023-10-06T04:59:16.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce1bf22b80 1 -- start start 2023-10-06T04:59:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a4881aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a487f40e30 con 0x55a4881b4a90 2023-10-06T04:59:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48816b7a0 con 0x55a48816ebd0 2023-10-06T04:59:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a488135b20 con 0x55a4881709a0 2023-10-06T04:59:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce111c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce111c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37260/0 (socket says 172.21.15.3:37260) 2023-10-06T04:59:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce111c7700 1 -- 172.21.15.3:0/421286358 learned_addr learned my addr 172.21.15.3:0/421286358 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce109c6700 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.492+0000 7fce0bfff700 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a4881aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce111c7700 1 -- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 msgr2=0x55a4881aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce111c7700 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a4881aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce111c7700 1 -- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 msgr2=0x55a48816aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce111c7700 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce111c7700 1 -- 172.21.15.3:0/421286358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4881ab7b0 con 0x55a4881b4a90 2023-10-06T04:59:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce111c7700 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881aa7a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fce0400a340 tx=0x7fce0401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2b4fa53a48196e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fcdf27fc700 1 -- 172.21.15.3:0/421286358 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0405c750 con 0x55a4881b4a90 2023-10-06T04:59:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fcdf27fc700 1 -- 172.21.15.3:0/421286358 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce04084040 con 0x55a4881b4a90 2023-10-06T04:59:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fcdf27fc700 1 -- 172.21.15.3:0/421286358 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04053b50 con 0x55a4881b4a90 2023-10-06T04:59:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 -- 172.21.15.3:0/421286358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 msgr2=0x55a4881aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881aa7a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fce0400a340 tx=0x7fce0401d040 comp rx=0 tx=0).stop 2023-10-06T04:59:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 -- 172.21.15.3:0/421286358 shutdown_connections 2023-10-06T04:59:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a48816ebd0 0x55a48816aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881aa7a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/421286358 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a4881709a0 0x55a4881aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 -- 172.21.15.3:0/421286358 >> 172.21.15.3:0/421286358 conn(0x55a48816f070 msgr2=0x55a4881b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.493+0000 7fce1bf22b80 1 -- 172.21.15.3:0/421286358 shutdown_connections 2023-10-06T04:59:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.494+0000 7fce1bf22b80 1 -- 172.21.15.3:0/421286358 wait complete. 2023-10-06T04:59:16.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.494+0000 7fce1bf22b80 1 -- start start 2023-10-06T04:59:16.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.494+0000 7fce1bf22b80 1 Processor -- start 2023-10-06T04:59:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a48816ebd0 0x55a48821cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a4881cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce1bf22b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48816fa20 con 0x55a4881b4a90 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48821d590 con 0x55a4881709a0 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce1bf22b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a48816f7d0 con 0x55a48816ebd0 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce1bf22b80 10 client.0 _ll_register_callbacks cb 0x55a487f5af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37268/0 (socket says 172.21.15.3:37268) 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 -- 172.21.15.3:0/3735183123 learned_addr learned my addr 172.21.15.3:0/3735183123 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce0bfff700 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a4881cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce109c6700 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a48816ebd0 0x55a48821cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 -- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a48816ebd0 msgr2=0x55a48821cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a48816ebd0 0x55a48821cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 -- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 msgr2=0x55a4881cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a4881cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.495+0000 7fce111c7700 1 -- 172.21.15.3:0/3735183123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c0c8a40 con 0x55a4881b4a90 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce111c7700 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881cdac0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fce0400cbf0 tx=0x7fce04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7562b6857227314 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fcdeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fcdeaffd700 1 -- 172.21.15.3:0/3735183123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0405c750 con 0x55a4881b4a90 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fcdeaffd700 1 -- 172.21.15.3:0/3735183123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce04058a00 con 0x55a4881b4a90 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdfc019070 con 0x55a4881b4a90 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fcdeaffd700 1 -- 172.21.15.3:0/3735183123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce04053b50 con 0x55a4881b4a90 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 10 client.7236 fetch_fsmap learned FSMap version 59 2023-10-06T04:59:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55a48808c780 con 0x55a4881b4a90 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fcdeaffd700 1 -- 172.21.15.3:0/3735183123 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc019070 con 0x55a4881b4a90 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fcdeaffd700 1 -- 172.21.15.3:0/3735183123 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fce04065020 con 0x55a4881b4a90 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 10 client.7236 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 10 client.7236 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 20 client.7236 populate_metadata read hostname 'smithi003' 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881976c0 0x55a488197a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 4 client.7236 mds_command: new command op to 7071 tid=0[{"prefix": "status"}] 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 --> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a488060e00 con 0x55a4881976c0 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.496+0000 7fce109c6700 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881976c0 0x55a488197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce109c6700 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881976c0 0x55a488197a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c805374c7ce123e server_cookie=6c6898eddc7fb63e in_seq=0 out_seq=0 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fcdeaffd700 10 client.7236 ms_handle_connect on v2:172.21.15.153:6835/1968748272 2023-10-06T04:59:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fcdeaffd700 1 -- 172.21.15.3:0/3735183123 <== mds.0 v2:172.21.15.153:6835/1968748272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a488060e00 con 0x55a4881976c0 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fcdeaffd700 10 client.7236 handle_command_reply: tid=0 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 client.7236 shutdown 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 msgr2=0x55a4881cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881cdac0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fce0400cbf0 tx=0x7fce04014000 comp rx=0 tx=0).stop 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 shutdown_connections 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:6835/1968748272,v1:172.21.15.153:6837/1968748272] conn(0x55a4881976c0 0x55a488197a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4881b4a90 0x55a4881cdac0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fcdeaffd700 0 client.7236 ms_handle_reset on v2:172.21.15.153:6835/1968748272 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a48816ebd0 0x55a48821cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 --2- 172.21.15.3:0/3735183123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4881709a0 0x55a4881cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 >> 172.21.15.3:0/3735183123 conn(0x55a48816f070 msgr2=0x55a488193710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 shutdown_connections 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 1 -- 172.21.15.3:0/3735183123 wait complete. 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.497+0000 7fce1bf22b80 20 client.7236 trim_cache size 0 max 16384 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7071, 2023-10-06T04:59:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.464556921, 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 59, 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 96, 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 96, 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.550296155000002 2023-10-06T04:59:16.515 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcdf8066f30 msgr2=0x7fcdf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcdf8066f30 0x7fcdf80693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00adc0 tx=0x7fcdfc006f90 comp rx=0 tx=0).stop 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c151810 msgr2=0x7fce0c1523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c151810 0x7fce0c1523f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fce0400cc20 tx=0x7fce040019b0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 shutdown_connections 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcdf8066f30 0x7fcdf80693e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce0c151810 0x7fce0c1523f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce0c157030 0x7fce0c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 --2- 172.21.15.3:0/1226871656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce0c157400 0x7fce0c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 >> 172.21.15.3:0/1226871656 conn(0x7fce0c0ce270 msgr2=0x7fce0c1556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 shutdown_connections 2023-10-06T04:59:16.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.498+0000 7fce12c2a700 1 -- 172.21.15.3:0/1226871656 wait complete. 2023-10-06T04:59:16.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:59:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.635+0000 7fdd3b1f4700 1 Processor -- start 2023-10-06T04:59:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.635+0000 7fdd3b1f4700 1 -- start start 2023-10-06T04:59:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd3b1f4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdd3406b1a0 0x7fdd340c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd3b1f4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd341564e0 0x7fdd340c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd3b1f4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd340acbb0 con 0x7fdd34156110 2023-10-06T04:59:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd3b1f4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd340c7640 con 0x7fdd341564e0 2023-10-06T04:59:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd3b1f4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd340c7920 con 0x7fdd3406b1a0 2023-10-06T04:59:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd39791700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd341564e0 0x7fdd340c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd39791700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd341564e0 0x7fdd340c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37270/0 (socket says 172.21.15.3:37270) 2023-10-06T04:59:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd39791700 1 -- 172.21.15.3:0/3839387643 learned_addr learned my addr 172.21.15.3:0/3839387643 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.636+0000 7fdd33fff700 1 --2- 172.21.15.3:0/3839387643 >> v2:172.21.15.153:3300/0 conn(0x7fdd3406b1a0 0x7fdd340c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd39791700 1 -- 172.21.15.3:0/3839387643 >> v2:172.21.15.153:3300/0 conn(0x7fdd3406b1a0 msgr2=0x7fdd340c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd39791700 1 --2- 172.21.15.3:0/3839387643 >> v2:172.21.15.153:3300/0 conn(0x7fdd3406b1a0 0x7fdd340c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd39791700 1 -- 172.21.15.3:0/3839387643 >> v1:172.21.15.153:6789/0 conn(0x7fdd34156110 legacy=0x7fdd3406aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd39791700 1 -- 172.21.15.3:0/3839387643 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340c7c00 con 0x7fdd341564e0 2023-10-06T04:59:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd33fff700 1 --2- 172.21.15.3:0/3839387643 >> v2:172.21.15.153:3300/0 conn(0x7fdd3406b1a0 0x7fdd340c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd39791700 1 --2- 172.21.15.3:0/3839387643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd341564e0 0x7fdd340c6e10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdd340b2100 tx=0x7fdd2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a827fc0511879ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd337fe700 1 -- 172.21.15.3:0/3839387643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00e070 con 0x7fdd341564e0 2023-10-06T04:59:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd337fe700 1 -- 172.21.15.3:0/3839387643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c00ba50 con 0x7fdd341564e0 2023-10-06T04:59:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd337fe700 1 -- 172.21.15.3:0/3839387643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c0092e0 con 0x7fdd341564e0 2023-10-06T04:59:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/3839387643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd341564e0 msgr2=0x7fdd340c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.637+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/3839387643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd341564e0 0x7fdd340c6e10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdd340b2100 tx=0x7fdd2c009840 comp rx=0 tx=0).stop 2023-10-06T04:59:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.638+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/3839387643 shutdown_connections 2023-10-06T04:59:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.638+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/3839387643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd341564e0 0x7fdd340c6e10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.638+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/3839387643 >> v2:172.21.15.153:3300/0 conn(0x7fdd3406b1a0 0x7fdd340c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.638+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/3839387643 >> 172.21.15.3:0/3839387643 conn(0x7fdd3406c370 msgr2=0x7fdd3406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.638+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/3839387643 shutdown_connections 2023-10-06T04:59:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.638+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/3839387643 wait complete. 2023-10-06T04:59:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.639+0000 7fdd3b1f4700 1 Processor -- start 2023-10-06T04:59:16.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.639+0000 7fdd3b1f4700 1 -- start start 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.639+0000 7fdd3b1f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd3406b1a0 0x7fdd340b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.639+0000 7fdd3b1f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd34156110 0x7fdd34151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.639+0000 7fdd3b1f4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd341564e0 0x7fdd34167e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd3b1f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341574a0 con 0x7fdd3406b1a0 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd3b1f4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341568b0 con 0x7fdd34156110 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd3b1f4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34156b70 con 0x7fdd341564e0 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd3406b1a0 0x7fdd340b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd3406b1a0 0x7fdd340b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37280/0 (socket says 172.21.15.3:37280) 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 -- 172.21.15.3:0/1094000255 learned_addr learned my addr 172.21.15.3:0/1094000255 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd33fff700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd34156110 0x7fdd34151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd39791700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd341564e0 0x7fdd34167e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 -- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd341564e0 msgr2=0x7fdd34167e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd341564e0 0x7fdd34167e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 -- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd34156110 msgr2=0x7fdd34151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd34156110 0x7fdd34151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 -- 172.21.15.3:0/1094000255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c009ae0 con 0x7fdd3406b1a0 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd33fff700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd34156110 0x7fdd34151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd38f90700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd3406b1a0 0x7fdd340b16d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c590 tx=0x7fdd2400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd30ff9700 1 -- 172.21.15.3:0/1094000255 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24011590 con 0x7fdd3406b1a0 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd30ff9700 1 -- 172.21.15.3:0/1094000255 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd240116f0 con 0x7fdd3406b1a0 2023-10-06T04:59:16.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd34156e50 con 0x7fdd3406b1a0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd30ff9700 1 -- 172.21.15.3:0/1094000255 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24011ab0 con 0x7fdd3406b1a0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.640+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd340b1c10 con 0x7fdd3406b1a0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.641+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd34157780 con 0x7fdd3406b1a0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.642+0000 7fdd30ff9700 1 -- 172.21.15.3:0/1094000255 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd2401b3b0 con 0x7fdd3406b1a0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.642+0000 7fdd30ff9700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd20066f30 0x7fdd200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.642+0000 7fdd30ff9700 1 -- 172.21.15.3:0/1094000255 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd24083ce0 con 0x7fdd3406b1a0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.642+0000 7fdd33fff700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd20066f30 0x7fdd200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.643+0000 7fdd33fff700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd20066f30 0x7fdd200693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdd28006f00 tx=0x7fdd28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:16.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.646+0000 7fdd30ff9700 1 -- 172.21.15.3:0/1094000255 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd24050d50 con 0x7fdd3406b1a0 2023-10-06T04:59:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.885+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdd34071680 con 0x7fdd3406b1a0 2023-10-06T04:59:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.885+0000 7fdd30ff9700 1 -- 172.21.15.3:0/1094000255 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdd24017070 con 0x7fdd3406b1a0 2023-10-06T04:59:16.888 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd20066f30 msgr2=0x7fdd200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd20066f30 0x7fdd200693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdd28006f00 tx=0x7fdd28008040 comp rx=0 tx=0).stop 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd3406b1a0 msgr2=0x7fdd340b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd3406b1a0 0x7fdd340b16d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c590 tx=0x7fdd2400c950 comp rx=0 tx=0).stop 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 shutdown_connections 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdd20066f30 0x7fdd200693e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd341564e0 0x7fdd34167e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd3406b1a0 0x7fdd340b16d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 --2- 172.21.15.3:0/1094000255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd34156110 0x7fdd34151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:16.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 >> 172.21.15.3:0/1094000255 conn(0x7fdd3406c370 msgr2=0x7fdd340c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:16.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 shutdown_connections 2023-10-06T04:59:16.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:16.887+0000 7fdd3b1f4700 1 -- 172.21.15.3:0/1094000255 wait complete. 2023-10-06T04:59:16.902 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:59:16.902 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:59:16.902 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T04:59:16.902 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:59:16.902 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:59:16.902 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:59:16.903 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:59:16.903 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:59:16.903 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:59:16.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:16.903 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T04:59:16.919 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42074sec preferred_lft 42074sec 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:59:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:59:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T04:59:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:16.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:16.922 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:59:16.983 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T04:59:16.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:16.985 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T04:59:17.042 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T04:59:17.042 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:59:17.042 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:59:17.042 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:59:17.043 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:59:17.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:17.043 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T04:59:17.043 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T04:59:17.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:59:17.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:59:17.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:59:17.044 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T04:59:17.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:17.145 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T04:59:17.159 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:59:17.159 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:59:17.160 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:59:17.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:17.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T04:59:17.225 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:59:17.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:59:17.225 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:59:17.280 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:59:17.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:59:17.281 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:59:17.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:59:17.401 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:59:17.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:59:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:59:17.467 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:59:17.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:17.467 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:59:17.523 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:59:17.571 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:59:17.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T04:59:17.601 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:59:17.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:59:17.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T04:59:17.654+0000 7f8822fc6540 -1 init, newargv = 0x5643447cc860 newargc=15 2023-10-06T04:59:17.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[120433]: starting ceph client 2023-10-06T04:59:17.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[120433]: starting fuse 2023-10-06T04:59:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:59:17.668 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:59:17.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:17.669 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T04:59:17.724 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T04:59:17.725 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:59:17.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:59:17.725 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:59:17.725 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:59:17.725 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:59:17.725 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:59:17.725 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:17.725 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:17.726 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:59:17.727 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:59:17.847 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.120433.asok 2023-10-06T04:59:18.029 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.120433.asok 2023-10-06T04:59:18.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:18.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.120433.asok status 2023-10-06T04:59:18.184 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:59:18.184 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T04:59:18.184 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:59:18.184 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "120433", 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7248, 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:59:18.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 7248 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1770037939 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1770037939 2023-10-06T04:59:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.7248 192.168.0.1:0/1770037939", 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/1770037939", 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 59, 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 96, 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T04:59:18.187 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:59:18.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:18.194 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T04:59:18.194 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:59:18.209 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T04:59:18.210 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:59:18.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:18.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:59:18.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:59:18.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.432+0000 7f94a3b9d700 1 Processor -- start 2023-10-06T04:59:18.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.433+0000 7f94a3b9d700 1 -- start start 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.433+0000 7f94a3b9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f949c159500 0x7f949c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.433+0000 7f94a3b9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f949c1598d0 0x7f949c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.433+0000 7f94a3b9d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c0cc5c0 con 0x7f949c0b2220 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.433+0000 7f94a3b9d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c0cb7b0 con 0x7f949c1598d0 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.433+0000 7f94a3b9d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c0cda60 con 0x7f949c159500 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.433+0000 7f94a1138700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f949c1598d0 0x7f949c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1138700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f949c1598d0 0x7f949c0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37288/0 (socket says 172.21.15.3:37288) 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1138700 1 -- 172.21.15.3:0/1958693168 learned_addr learned my addr 172.21.15.3:0/1958693168 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:18.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1939700 1 --2- 172.21.15.3:0/1958693168 >> v2:172.21.15.153:3300/0 conn(0x7f949c159500 0x7f949c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1138700 1 -- 172.21.15.3:0/1958693168 >> v2:172.21.15.153:3300/0 conn(0x7f949c159500 msgr2=0x7f949c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1138700 1 --2- 172.21.15.3:0/1958693168 >> v2:172.21.15.153:3300/0 conn(0x7f949c159500 0x7f949c0cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1138700 1 -- 172.21.15.3:0/1958693168 >> v1:172.21.15.153:6789/0 conn(0x7f949c0b2220 legacy=0x7f949c0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1138700 1 -- 172.21.15.3:0/1958693168 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0cb9b0 con 0x7f949c1598d0 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1939700 1 --2- 172.21.15.3:0/1958693168 >> v2:172.21.15.153:3300/0 conn(0x7f949c159500 0x7f949c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a1138700 1 --2- 172.21.15.3:0/1958693168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0cb270 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f949c089390 tx=0x7f9490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4f83ca93707e9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a0937700 1 -- 172.21.15.3:0/1958693168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000b860 con 0x7f949c1598d0 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a0937700 1 -- 172.21.15.3:0/1958693168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949000b9c0 con 0x7f949c1598d0 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.434+0000 7f94a0937700 1 -- 172.21.15.3:0/1958693168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000bd60 con 0x7f949c1598d0 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1958693168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 msgr2=0x7f949c0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1958693168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0cb270 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f949c089390 tx=0x7f9490009250 comp rx=0 tx=0).stop 2023-10-06T04:59:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1958693168 shutdown_connections 2023-10-06T04:59:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1958693168 >> v2:172.21.15.153:3300/0 conn(0x7f949c159500 0x7f949c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1958693168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0cb270 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1958693168 >> 172.21.15.3:0/1958693168 conn(0x7f949c070310 msgr2=0x7f949c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1958693168 shutdown_connections 2023-10-06T04:59:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.435+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1958693168 wait complete. 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.436+0000 7f94a3b9d700 1 Processor -- start 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.436+0000 7f94a3b9d700 1 -- start start 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a3b9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f949c0b2220 0x7f949c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a3b9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f949c159500 0x7f949c14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a3b9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a3b9d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c159cc0 con 0x7f949c1598d0 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a3b9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c159fa0 con 0x7f949c159500 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a3b9d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15a280 con 0x7f949c0b2220 2023-10-06T04:59:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37302/0 (socket says 172.21.15.3:37302) 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 -- 172.21.15.3:0/1571522369 learned_addr learned my addr 172.21.15.3:0/1571522369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a1138700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f949c159500 0x7f949c14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a1939700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f949c0b2220 0x7f949c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 -- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f949c0b2220 msgr2=0x7f949c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f949c0b2220 0x7f949c15b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 -- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f949c159500 msgr2=0x7f949c14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f949c159500 0x7f949c14cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 -- 172.21.15.3:0/1571522369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94900094f0 con 0x7f949c1598d0 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a1939700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f949c0b2220 0x7f949c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a213a700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0c9990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f949400c560 tx=0x7f9494007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f9499ffb700 1 -- 172.21.15.3:0/1571522369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494011590 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f9499ffb700 1 -- 172.21.15.3:0/1571522369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94940116f0 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c15a560 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.437+0000 7f9499ffb700 1 -- 172.21.15.3:0/1571522369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494011ab0 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.438+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c15ab20 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.438+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c071680 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.439+0000 7f9499ffb700 1 -- 172.21.15.3:0/1571522369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f949401b3b0 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.439+0000 7f9499ffb700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9488066f30 0x7f94880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.439+0000 7f9499ffb700 1 -- 172.21.15.3:0/1571522369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9494083b10 con 0x7f949c1598d0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.440+0000 7f94a1939700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9488066f30 0x7f94880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.440+0000 7f94a1939700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9488066f30 0x7f94880693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f948c0076e0 tx=0x7f948c0098f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:18.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.443+0000 7f9499ffb700 1 -- 172.21.15.3:0/1571522369 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9494050c20 con 0x7f949c1598d0 2023-10-06T04:59:18.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.681+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f949c0cab10 con 0x7f949c1598d0 2023-10-06T04:59:18.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.681+0000 7f9499ffb700 1 -- 172.21.15.3:0/1571522369 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9494017070 con 0x7f949c1598d0 2023-10-06T04:59:18.684 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9488066f30 msgr2=0x7f94880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9488066f30 0x7f94880693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f948c0076e0 tx=0x7f948c0098f0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 msgr2=0x7f949c0c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0c9990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f949400c560 tx=0x7f9494007ef0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 shutdown_connections 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9488066f30 0x7f94880693e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949c1598d0 0x7f949c0c9990 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f949c0b2220 0x7f949c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 --2- 172.21.15.3:0/1571522369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f949c159500 0x7f949c14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:18.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 >> 172.21.15.3:0/1571522369 conn(0x7f949c070310 msgr2=0x7f949c0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:18.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 shutdown_connections 2023-10-06T04:59:18.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:18.683+0000 7f94a3b9d700 1 -- 172.21.15.3:0/1571522369 wait complete. 2023-10-06T04:59:18.697 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:59:18.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:59:18.697 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T04:59:18.698 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:59:18.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:59:18.698 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:59:18.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:59:18.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:59:18.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:59:18.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:18.698 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:59:18.717 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42074sec preferred_lft 42074sec 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:59:18.718 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T04:59:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:59:18.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:18.720 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:59:18.785 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T04:59:18.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:18.786 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T04:59:18.844 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T04:59:18.844 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:59:18.844 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:59:18.845 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:59:18.845 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:59:18.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:18.845 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T04:59:18.845 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T04:59:18.845 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:59:18.845 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:59:18.845 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:59:18.845 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T04:59:18.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:18.950 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T04:59:18.966 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T04:59:18.966 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:59:18.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:59:18.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:18.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T04:59:19.031 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:59:19.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:59:19.031 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:59:19.087 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:59:19.087 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:59:19.087 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:59:19.141 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:59:19.207 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:59:19.207 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:59:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:59:19.272 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:59:19.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:19.272 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:59:19.327 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:59:19.328 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:59:19.371 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T04:59:19.395 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:59:19.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:59:19.451 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T04:59:19.449+0000 7f0b1123a540 -1 init, newargv = 0x56325858fe30 newargc=15 2023-10-06T04:59:19.451 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[101273]: starting ceph client 2023-10-06T04:59:19.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[101273]: starting fuse 2023-10-06T04:59:19.461 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:59:19.462 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:59:19.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:19.463 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T04:59:19.517 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T04:59:19.518 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:59:19.518 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:59:19.518 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T04:59:19.518 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T04:59:19.518 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T04:59:19.518 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T04:59:19.518 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:59:19.518 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:59:19.519 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:59:19.520 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T04:59:19.520 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T04:59:19.520 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T04:59:19.520 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:59:19.520 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T04:59:19.520 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T04:59:19.520 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T04:59:19.639 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.101273.asok 2023-10-06T04:59:19.821 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.101273.asok 2023-10-06T04:59:19.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:19.821 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.101273.asok status 2023-10-06T04:59:19.971 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "101273", 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T04:59:19.972 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 6802, 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 6802 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2791282291 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:59:19.973 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2791282291 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.6802 192.168.0.1:0/2791282291", 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/2791282291", 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 59, 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 96, 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T04:59:19.974 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T04:59:19.975 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T04:59:19.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:19.982 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T04:59:19.982 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:59:19.998 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T04:59:19.998 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:59:19.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:19.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:59:20.062 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-10-06T04:59:20.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.193+0000 7f0445407700 1 Processor -- start 2023-10-06T04:59:20.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.193+0000 7f0445407700 1 -- start start 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.194+0000 7f0445407700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400af5f0 0x7f0440164260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.194+0000 7f0445407700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04400b1e70 con 0x7f044015cae0 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.194+0000 7f0445407700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04400b1860 con 0x7f04400aea60 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.194+0000 7f0445407700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044006eaf0 con 0x7f04400af5f0 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.194+0000 7f043ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f044015cae0 0x7f04400b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58984/0 (socket says 172.21.15.3:58984) 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.194+0000 7f043ffff700 1 -- 172.21.15.3:0/3263585270 learned_addr learned my addr 172.21.15.3:0/3263585270 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.194+0000 7f0444c06700 1 --2- 172.21.15.3:0/3263585270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400af5f0 0x7f0440164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283518351 0 0) 0x7f04400b1e70 con 0x7f044015cae0 2023-10-06T04:59:20.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f042c003570 con 0x7f044015cae0 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075398897 0 0) 0x7f04400b1860 con 0x7f04400aea60 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04400b1e70 con 0x7f04400aea60 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f0444c06700 1 -- 172.21.15.3:0/3263585270 >> v1:172.21.15.153:6789/0 conn(0x7f04400aea60 legacy=0x7f04400aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3892084342 0 0) 0x7f042c003570 con 0x7f044015cae0 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f0444c06700 1 -- 172.21.15.3:0/3263585270 >> v1:172.21.15.3:6789/0 conn(0x7f044015cae0 legacy=0x7f04400b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f0444c06700 1 -- 172.21.15.3:0/3263585270 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044006e530 con 0x7f04400af5f0 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f0444c06700 1 --2- 172.21.15.3:0/3263585270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400af5f0 0x7f0440164260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0438009ec0 tx=0x7f0438009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c6eb887006f1ce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043800e070 con 0x7f04400af5f0 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043800b870 con 0x7f04400af5f0 2023-10-06T04:59:20.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.195+0000 7f043effd700 1 -- 172.21.15.3:0/3263585270 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300015f0 con 0x7f04400af5f0 2023-10-06T04:59:20.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.196+0000 7f0445407700 1 -- 172.21.15.3:0/3263585270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400af5f0 msgr2=0x7f0440164260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.196+0000 7f0445407700 1 --2- 172.21.15.3:0/3263585270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400af5f0 0x7f0440164260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0438009ec0 tx=0x7f0438009780 comp rx=0 tx=0).stop 2023-10-06T04:59:20.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.196+0000 7f0445407700 1 -- 172.21.15.3:0/3263585270 shutdown_connections 2023-10-06T04:59:20.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.196+0000 7f0445407700 1 --2- 172.21.15.3:0/3263585270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400af5f0 0x7f0440164260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.196+0000 7f0445407700 1 -- 172.21.15.3:0/3263585270 >> 172.21.15.3:0/3263585270 conn(0x7f044006cb50 msgr2=0x7f04400ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:20.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.196+0000 7f0445407700 1 -- 172.21.15.3:0/3263585270 shutdown_connections 2023-10-06T04:59:20.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.196+0000 7f0445407700 1 -- 172.21.15.3:0/3263585270 wait complete. 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.197+0000 7f0445407700 1 Processor -- start 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.197+0000 7f0445407700 1 -- start start 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f0445407700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400aea60 0x7f0440158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f0445407700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400af5f0 0x7f0440169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f0445407700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f044015cae0 0x7f044009b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f0445407700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0440158750 con 0x7f04400af5f0 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f0445407700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044009afa0 con 0x7f044015cae0 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f0445407700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044016a060 con 0x7f04400aea60 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400af5f0 0x7f0440169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400af5f0 0x7f0440169b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37310/0 (socket says 172.21.15.3:37310) 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 -- 172.21.15.3:0/2443097587 learned_addr learned my addr 172.21.15.3:0/2443097587 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043ffff700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400aea60 0x7f0440158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f0444c06700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f044015cae0 0x7f044009b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 -- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400aea60 msgr2=0x7f0440158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400aea60 0x7f0440158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 -- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f044015cae0 msgr2=0x7f044009b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f044015cae0 0x7f044009b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.198+0000 7f043f7fe700 1 -- 172.21.15.3:0/2443097587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438009a20 con 0x7f04400af5f0 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.199+0000 7f043f7fe700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400af5f0 0x7f0440169b20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f042800c790 tx=0x7f0428007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.199+0000 7f0437fff700 1 -- 172.21.15.3:0/2443097587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428010510 con 0x7f04400af5f0 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.199+0000 7f0437fff700 1 -- 172.21.15.3:0/2443097587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0428010670 con 0x7f04400af5f0 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.199+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0440098f00 con 0x7f04400af5f0 2023-10-06T04:59:20.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.199+0000 7f0437fff700 1 -- 172.21.15.3:0/2443097587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428010a10 con 0x7f04400af5f0 2023-10-06T04:59:20.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.199+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044009a950 con 0x7f04400af5f0 2023-10-06T04:59:20.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.200+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044009b130 con 0x7f04400af5f0 2023-10-06T04:59:20.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.200+0000 7f0437fff700 1 -- 172.21.15.3:0/2443097587 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f042801b400 con 0x7f04400af5f0 2023-10-06T04:59:20.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.201+0000 7f0437fff700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0420066f00 0x7f04200693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.201+0000 7f0437fff700 1 -- 172.21.15.3:0/2443097587 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f04280844e0 con 0x7f04400af5f0 2023-10-06T04:59:20.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.201+0000 7f043ffff700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0420066f00 0x7f04200693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.201+0000 7f043ffff700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0420066f00 0x7f04200693b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0430001360 tx=0x7f0430009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.205+0000 7f0437fff700 1 -- 172.21.15.3:0/2443097587 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04280515f0 con 0x7f04400af5f0 2023-10-06T04:59:20.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.397+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f044004c380 con 0x7f04400af5f0 2023-10-06T04:59:20.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.401+0000 7f0437fff700 1 -- 172.21.15.3:0/2443097587 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v35)=0 v35) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0428054c10 con 0x7f04400af5f0 2023-10-06T04:59:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0420066f00 msgr2=0x7f04200693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0420066f00 0x7f04200693b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0430001360 tx=0x7f0430009040 comp rx=0 tx=0).stop 2023-10-06T04:59:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400af5f0 msgr2=0x7f0440169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400af5f0 0x7f0440169b20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f042800c790 tx=0x7f0428007e10 comp rx=0 tx=0).stop 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 shutdown_connections 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0420066f00 0x7f04200693b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f044015cae0 0x7f044009b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04400aea60 0x7f0440158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 --2- 172.21.15.3:0/2443097587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400af5f0 0x7f0440169b20 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 >> 172.21.15.3:0/2443097587 conn(0x7f044006cb50 msgr2=0x7f04400b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.403+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 shutdown_connections 2023-10-06T04:59:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.404+0000 7f0445407700 1 -- 172.21.15.3:0/2443097587 wait complete. 2023-10-06T04:59:20.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2023-10-06T04:59:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.537+0000 7fa4834fb700 1 Processor -- start 2023-10-06T04:59:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.538+0000 7fa4834fb700 1 -- start start 2023-10-06T04:59:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.538+0000 7fa4834fb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.538+0000 7fa4834fb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c15a290 con 0x7fa47c15d540 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.538+0000 7fa4834fb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0d0020 con 0x7fa47c150690 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.538+0000 7fa4834fb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c159f40 con 0x7fa47c0cba50 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa481a98700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa47c15d540 0x7fa47c0af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58990/0 (socket says 172.21.15.3:58990) 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa481a98700 1 -- 172.21.15.3:0/1592931899 learned_addr learned my addr 172.21.15.3:0/1592931899 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa480a96700 1 --2- 172.21.15.3:0/1592931899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa47bfff700 1 -- 172.21.15.3:0/1592931899 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404714798 0 0) 0x7fa47c15a290 con 0x7fa47c15d540 2023-10-06T04:59:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa47bfff700 1 -- 172.21.15.3:0/1592931899 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa464003570 con 0x7fa47c15d540 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa480a96700 1 -- 172.21.15.3:0/1592931899 >> v1:172.21.15.153:6789/0 conn(0x7fa47c150690 legacy=0x7fa47c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa480a96700 1 -- 172.21.15.3:0/1592931899 >> v1:172.21.15.3:6789/0 conn(0x7fa47c15d540 legacy=0x7fa47c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa480a96700 1 -- 172.21.15.3:0/1592931899 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c15dcb0 con 0x7fa47c0cba50 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.539+0000 7fa480a96700 1 --2- 172.21.15.3:0/1592931899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c150a60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa470009910 tx=0x7fa4700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c022be59d17627f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa47bfff700 1 -- 172.21.15.3:0/1592931899 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000b680 con 0x7fa47c0cba50 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa47bfff700 1 -- 172.21.15.3:0/1592931899 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47000b7e0 con 0x7fa47c0cba50 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa47bfff700 1 -- 172.21.15.3:0/1592931899 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000bba0 con 0x7fa47c0cba50 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa4834fb700 1 -- 172.21.15.3:0/1592931899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 msgr2=0x7fa47c150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1592931899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c150a60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa470009910 tx=0x7fa4700091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa4834fb700 1 -- 172.21.15.3:0/1592931899 shutdown_connections 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1592931899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c150a60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa4834fb700 1 -- 172.21.15.3:0/1592931899 >> 172.21.15.3:0/1592931899 conn(0x7fa47c070310 msgr2=0x7fa47c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:20.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa4834fb700 1 -- 172.21.15.3:0/1592931899 shutdown_connections 2023-10-06T04:59:20.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.540+0000 7fa4834fb700 1 -- 172.21.15.3:0/1592931899 wait complete. 2023-10-06T04:59:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.541+0000 7fa4834fb700 1 Processor -- start 2023-10-06T04:59:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.541+0000 7fa4834fb700 1 -- start start 2023-10-06T04:59:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa4834fb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa4834fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa47c150690 0x7fa47c16de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa4834fb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa47c15d540 0x7fa47c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa4834fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c150380 con 0x7fa47c150690 2023-10-06T04:59:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa4834fb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0c9fd0 con 0x7fa47c15d540 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa4834fb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c14fe10 con 0x7fa47c0cba50 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa480a96700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa47c150690 0x7fa47c16de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa480a96700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa47c150690 0x7fa47c16de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37316/0 (socket says 172.21.15.3:37316) 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa480a96700 1 -- 172.21.15.3:0/1319676 learned_addr learned my addr 172.21.15.3:0/1319676 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa481a98700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa47c15d540 0x7fa47c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.542+0000 7fa481297700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa480a96700 1 -- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 msgr2=0x7fa47c153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa480a96700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa480a96700 1 -- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa47c15d540 msgr2=0x7fa47c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa480a96700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa47c15d540 0x7fa47c14f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa480a96700 1 -- 172.21.15.3:0/1319676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470009420 con 0x7fa47c150690 2023-10-06T04:59:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa480a96700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa47c150690 0x7fa47c16de50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0c71f0 tx=0x7fa470017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa4797fa700 1 -- 172.21.15.3:0/1319676 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000bb60 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa4797fa700 1 -- 172.21.15.3:0/1319676 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa470000c90 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c1538c0 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa4797fa700 1 -- 172.21.15.3:0/1319676 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47001a660 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.543+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c150de0 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.544+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c04c380 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.544+0000 7fa4797fa700 1 -- 172.21.15.3:0/1319676 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa47001aa10 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.545+0000 7fa4797fa700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa468066f30 0x7fa4680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.545+0000 7fa4797fa700 1 -- 172.21.15.3:0/1319676 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa470086200 con 0x7fa47c150690 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.545+0000 7fa481297700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa468066f30 0x7fa4680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.546+0000 7fa481297700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa468066f30 0x7fa4680693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa474003a80 tx=0x7fa474007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.549+0000 7fa4797fa700 1 -- 172.21.15.3:0/1319676 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa470053310 con 0x7fa47c150690 2023-10-06T04:59:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.744+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7fa47c0ce4b0 con 0x7fa47c150690 2023-10-06T04:59:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.748+0000 7fa4797fa700 1 -- 172.21.15.3:0/1319676 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v36)=0 v36) v1 ==== 119+0+0 (secure 0 0 0) 0x7fa470056930 con 0x7fa47c150690 2023-10-06T04:59:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa468066f30 msgr2=0x7fa4680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa468066f30 0x7fa4680693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa474003a80 tx=0x7fa474007ec0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa47c150690 msgr2=0x7fa47c16de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa47c150690 0x7fa47c16de50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0c71f0 tx=0x7fa470017000 comp rx=0 tx=0).stop 2023-10-06T04:59:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 shutdown_connections 2023-10-06T04:59:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa468066f30 0x7fa4680693e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa47c15d540 0x7fa47c14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa47c0cba50 0x7fa47c153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 --2- 172.21.15.3:0/1319676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa47c150690 0x7fa47c16de50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 >> 172.21.15.3:0/1319676 conn(0x7fa47c070310 msgr2=0x7fa47c0c6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 shutdown_connections 2023-10-06T04:59:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.750+0000 7fa4834fb700 1 -- 172.21.15.3:0/1319676 wait complete. 2023-10-06T04:59:20.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2023-10-06T04:59:20.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.884+0000 7f7b9eff1700 1 Processor -- start 2023-10-06T04:59:20.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.884+0000 7f7b9eff1700 1 -- start start 2023-10-06T04:59:20.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.884+0000 7f7b9eff1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b98150690 0x7f7b98150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9eff1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9815a290 con 0x7f7b9815d540 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9eff1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980d0020 con 0x7f7b980cba50 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9eff1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b98159f40 con 0x7f7b98150690 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9cd8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b98150690 0x7f7b98150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9cd8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b98150690 0x7f7b98150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37326/0 (socket says 172.21.15.3:37326) 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9cd8d700 1 -- 172.21.15.3:0/3408044534 learned_addr learned my addr 172.21.15.3:0/3408044534 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9d58e700 1 --1- 172.21.15.3:0/3408044534 >> v1:172.21.15.3:6789/0 conn(0x7f7b9815d540 0x7f7b980af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59006/0 (socket says 172.21.15.3:59006) 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9cd8d700 1 -- 172.21.15.3:0/3408044534 >> v1:172.21.15.153:6789/0 conn(0x7f7b980cba50 legacy=0x7f7b9815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9cd8d700 1 -- 172.21.15.3:0/3408044534 >> v1:172.21.15.3:6789/0 conn(0x7f7b9815d540 legacy=0x7f7b980af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9cd8d700 1 -- 172.21.15.3:0/3408044534 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9815dcb0 con 0x7f7b98150690 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b9cd8d700 1 --2- 172.21.15.3:0/3408044534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b98150a60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7b880099b0 tx=0x7f7b88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187746979b9a490a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b977fe700 1 -- 172.21.15.3:0/3408044534 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800b820 con 0x7f7b98150690 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b977fe700 1 -- 172.21.15.3:0/3408044534 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8800b980 con 0x7f7b98150690 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.885+0000 7f7b977fe700 1 -- 172.21.15.3:0/3408044534 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800bd20 con 0x7f7b98150690 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.886+0000 7f7b9eff1700 1 -- 172.21.15.3:0/3408044534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 msgr2=0x7f7b98150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.886+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/3408044534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b98150a60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7b880099b0 tx=0x7f7b88009250 comp rx=0 tx=0).stop 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.886+0000 7f7b9eff1700 1 -- 172.21.15.3:0/3408044534 shutdown_connections 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.886+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/3408044534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b98150a60 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.886+0000 7f7b9eff1700 1 -- 172.21.15.3:0/3408044534 >> 172.21.15.3:0/3408044534 conn(0x7f7b98070310 msgr2=0x7f7b98070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.886+0000 7f7b9eff1700 1 -- 172.21.15.3:0/3408044534 shutdown_connections 2023-10-06T04:59:20.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.886+0000 7f7b9eff1700 1 -- 172.21.15.3:0/3408044534 wait complete. 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.887+0000 7f7b9eff1700 1 Processor -- start 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.887+0000 7f7b9eff1700 1 -- start start 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9eff1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b980cba50 0x7f7b980c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9eff1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b9814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9eff1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b9815d540 0x7f7b980c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9eff1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980c8d80 con 0x7f7b98150690 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9eff1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980b36f0 con 0x7f7b9815d540 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9eff1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b98151300 con 0x7f7b980cba50 2023-10-06T04:59:20.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b9814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b9814efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37336/0 (socket says 172.21.15.3:37336) 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 -- 172.21.15.3:0/1791027229 learned_addr learned my addr 172.21.15.3:0/1791027229 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9d58e700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b9815d540 0x7f7b980c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9cd8d700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b980cba50 0x7f7b980c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 -- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b980cba50 msgr2=0x7f7b980c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b980cba50 0x7f7b980c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 -- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b9815d540 msgr2=0x7f7b980c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b9815d540 0x7f7b980c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 -- 172.21.15.3:0/1791027229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880094f0 con 0x7f7b98150690 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b9d58e700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b9815d540 0x7f7b980c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:20.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.888+0000 7f7b97fff700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b9814efc0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00b600 tx=0x7f7b8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.889+0000 7f7b94ff9700 1 -- 172.21.15.3:0/1791027229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00f510 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.889+0000 7f7b94ff9700 1 -- 172.21.15.3:0/1791027229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00f670 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.889+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980c87c0 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.889+0000 7f7b94ff9700 1 -- 172.21.15.3:0/1791027229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00fa30 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.889+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b980b3380 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.889+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b980b31e0 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.890+0000 7f7b94ff9700 1 -- 172.21.15.3:0/1791027229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b8c01a400 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.890+0000 7f7b94ff9700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.890+0000 7f7b94ff9700 1 -- 172.21.15.3:0/1791027229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7b8c0841f0 con 0x7f7b98150690 2023-10-06T04:59:20.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.890+0000 7f7b9cd8d700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:20.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.891+0000 7f7b9cd8d700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b84066f30 0x7f7b840693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7b880099b0 tx=0x7f7b88016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:20.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:20.895+0000 7f7b94ff9700 1 -- 172.21.15.3:0/1791027229 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b980b31e0 con 0x7f7b98150690 2023-10-06T04:59:21.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.089+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f7b980b31e0 con 0x7f7b98150690 2023-10-06T04:59:21.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.093+0000 7f7b94ff9700 1 -- 172.21.15.3:0/1791027229 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v37)=0 v37) v1 ==== 128+0+0 (secure 0 0 0) 0x7f7b980b31e0 con 0x7f7b98150690 2023-10-06T04:59:21.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b84066f30 msgr2=0x7f7b840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b84066f30 0x7f7b840693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7b880099b0 tx=0x7f7b88016000 comp rx=0 tx=0).stop 2023-10-06T04:59:21.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 msgr2=0x7f7b9814efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b9814efc0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00b600 tx=0x7f7b8c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 shutdown_connections 2023-10-06T04:59:21.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b9815d540 0x7f7b980c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b980cba50 0x7f7b980c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 --2- 172.21.15.3:0/1791027229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b98150690 0x7f7b9814efc0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 >> 172.21.15.3:0/1791027229 conn(0x7f7b98070310 msgr2=0x7f7b980b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:21.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 shutdown_connections 2023-10-06T04:59:21.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.095+0000 7f7b9eff1700 1 -- 172.21.15.3:0/1791027229 wait complete. 2023-10-06T04:59:21.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-10-06T04:59:21.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.228+0000 7f298fabd700 1 Processor -- start 2023-10-06T04:59:21.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.228+0000 7f298fabd700 1 -- start start 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298fabd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f29880cfb80 0x7f29880af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298fabd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29880afad0 0x7f29880ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298fabd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298815d510 con 0x7f29880cef10 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298fabd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298815a480 con 0x7f29880afad0 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298fabd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298815a760 con 0x7f29880cfb80 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298d058700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29880afad0 0x7f29880ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298e05a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f29880cef10 0x7f298815da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59018/0 (socket says 172.21.15.3:59018) 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298e05a700 1 -- 172.21.15.3:0/3163477488 learned_addr learned my addr 172.21.15.3:0/3163477488 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298d859700 1 --2- 172.21.15.3:0/3163477488 >> v2:172.21.15.153:3300/0 conn(0x7f29880cfb80 0x7f29880af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298c857700 1 -- 172.21.15.3:0/3163477488 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954210427 0 0) 0x7f298815d510 con 0x7f29880cef10 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298c857700 1 -- 172.21.15.3:0/3163477488 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2974003570 con 0x7f29880cef10 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298d058700 1 -- 172.21.15.3:0/3163477488 >> v2:172.21.15.153:3300/0 conn(0x7f29880cfb80 msgr2=0x7f29880af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298d058700 1 --2- 172.21.15.3:0/3163477488 >> v2:172.21.15.153:3300/0 conn(0x7f29880cfb80 0x7f29880af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298d058700 1 -- 172.21.15.3:0/3163477488 >> v1:172.21.15.3:6789/0 conn(0x7f29880cef10 legacy=0x7f298815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298d058700 1 -- 172.21.15.3:0/3163477488 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298815aa40 con 0x7f29880afad0 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.229+0000 7f298d058700 1 --2- 172.21.15.3:0/3163477488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880afad0 0x7f29880ce9d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f29880ce1a0 tx=0x7f29780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5943e9b391b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298c857700 1 -- 172.21.15.3:0/3163477488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978011630 con 0x7f29880afad0 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298c857700 1 -- 172.21.15.3:0/3163477488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29780117b0 con 0x7f29880afad0 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298c857700 1 -- 172.21.15.3:0/3163477488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978011b30 con 0x7f29880afad0 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 -- 172.21.15.3:0/3163477488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880afad0 msgr2=0x7f29880ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 --2- 172.21.15.3:0/3163477488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880afad0 0x7f29880ce9d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f29880ce1a0 tx=0x7f29780091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 -- 172.21.15.3:0/3163477488 shutdown_connections 2023-10-06T04:59:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 --2- 172.21.15.3:0/3163477488 >> v2:172.21.15.153:3300/0 conn(0x7f29880cfb80 0x7f29880af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 --2- 172.21.15.3:0/3163477488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880afad0 0x7f29880ce9d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 -- 172.21.15.3:0/3163477488 >> 172.21.15.3:0/3163477488 conn(0x7f2988070310 msgr2=0x7f2988070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 -- 172.21.15.3:0/3163477488 shutdown_connections 2023-10-06T04:59:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.230+0000 7f298fabd700 1 -- 172.21.15.3:0/3163477488 wait complete. 2023-10-06T04:59:21.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.231+0000 7f298fabd700 1 Processor -- start 2023-10-06T04:59:21.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.231+0000 7f298fabd700 1 -- start start 2023-10-06T04:59:21.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298fabd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29880afad0 0x7f29880c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298fabd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880cef10 0x7f29880c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298fabd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29880cfb80 0x7f298816df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298fabd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880c88a0 con 0x7f29880cef10 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298fabd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880c7d20 con 0x7f29880afad0 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298fabd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29880c8000 con 0x7f29880cfb80 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880cef10 0x7f29880c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880cef10 0x7f29880c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37350/0 (socket says 172.21.15.3:37350) 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 -- 172.21.15.3:0/569761376 learned_addr learned my addr 172.21.15.3:0/569761376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d859700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29880afad0 0x7f29880c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298e05a700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29880cfb80 0x7f298816df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 -- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29880cfb80 msgr2=0x7f298816df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29880cfb80 0x7f298816df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 -- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29880afad0 msgr2=0x7f29880c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29880afad0 0x7f29880c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298d058700 1 -- 172.21.15.3:0/569761376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978009420 con 0x7f29880cef10 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.232+0000 7f298e05a700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29880cfb80 0x7f298816df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.233+0000 7f298d058700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880cef10 0x7f29880c74f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2978013000 tx=0x7f2978017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.233+0000 7f297dffb700 1 -- 172.21.15.3:0/569761376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978011b30 con 0x7f29880cef10 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.233+0000 7f297dffb700 1 -- 172.21.15.3:0/569761376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2978000c90 con 0x7f29880cef10 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.233+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29880c82e0 con 0x7f29880cef10 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.233+0000 7f297dffb700 1 -- 172.21.15.3:0/569761376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297801a660 con 0x7f29880cef10 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.233+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2988155580 con 0x7f29880cef10 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.234+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298804c380 con 0x7f29880cef10 2023-10-06T04:59:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.235+0000 7f297dffb700 1 -- 172.21.15.3:0/569761376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f297801a9d0 con 0x7f29880cef10 2023-10-06T04:59:21.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.235+0000 7f297dffb700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.235+0000 7f297dffb700 1 -- 172.21.15.3:0/569761376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2978086aa0 con 0x7f29880cef10 2023-10-06T04:59:21.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.236+0000 7f298d859700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.236+0000 7f298d859700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2988156560 tx=0x7f29840091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.239+0000 7f297dffb700 1 -- 172.21.15.3:0/569761376 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2978053c30 con 0x7f29880cef10 2023-10-06T04:59:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.433+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f29880ca860 con 0x7f29880cef10 2023-10-06T04:59:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.433+0000 7f297dffb700 1 -- 172.21.15.3:0/569761376 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v37) v1 ==== 106+0+4 (secure 0 0 0) 0x7f29780340b0 con 0x7f29880cef10 2023-10-06T04:59:21.436 INFO:teuthology.orchestra.run.smithi003.stdout:100 2023-10-06T04:59:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.435+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 msgr2=0x7f296c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2988156560 tx=0x7f29840091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880cef10 msgr2=0x7f29880c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880cef10 0x7f29880c74f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2978013000 tx=0x7f2978017000 comp rx=0 tx=0).stop 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 shutdown_connections 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f296c066f30 0x7f296c0693e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29880afad0 0x7f29880c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29880cfb80 0x7f298816df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 --2- 172.21.15.3:0/569761376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29880cef10 0x7f29880c74f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 >> 172.21.15.3:0/569761376 conn(0x7f2988070310 msgr2=0x7f2988156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 shutdown_connections 2023-10-06T04:59:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.436+0000 7f298fabd700 1 -- 172.21.15.3:0/569761376 wait complete. 2023-10-06T04:59:21.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-10-06T04:59:21.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.570+0000 7ffa5f2d4700 1 Processor -- start 2023-10-06T04:59:21.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.570+0000 7ffa5f2d4700 1 -- start start 2023-10-06T04:59:21.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5f2d4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffa580cba50 0x7ffa58150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5f2d4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5815a290 con 0x7ffa58150690 2023-10-06T04:59:21.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5f2d4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580d0020 con 0x7ffa5815d540 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5f2d4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa58159f40 con 0x7ffa580cba50 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5d070700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffa58150690 0x7ffa5815a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59032/0 (socket says 172.21.15.3:59032) 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5d070700 1 -- 172.21.15.3:0/1774378476 learned_addr learned my addr 172.21.15.3:0/1774378476 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5c86f700 1 --2- 172.21.15.3:0/1774378476 >> v2:172.21.15.153:3300/0 conn(0x7ffa580cba50 0x7ffa58150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407728051 0 0) 0x7ffa5815a290 con 0x7ffa58150690 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa40003570 con 0x7ffa58150690 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174267943 0 0) 0x7ffa580d0020 con 0x7ffa5815d540 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5815a290 con 0x7ffa5815d540 2023-10-06T04:59:21.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5c86f700 1 -- 172.21.15.3:0/1774378476 >> v1:172.21.15.153:6789/0 conn(0x7ffa5815d540 legacy=0x7ffa580af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5c86f700 1 -- 172.21.15.3:0/1774378476 >> v1:172.21.15.3:6789/0 conn(0x7ffa58150690 legacy=0x7ffa5815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827369245 0 0) 0x7ffa40003570 con 0x7ffa58150690 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.571+0000 7ffa5c86f700 1 -- 172.21.15.3:0/1774378476 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5815dcb0 con 0x7ffa580cba50 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5c86f700 1 --2- 172.21.15.3:0/1774378476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580cba50 0x7ffa58150a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0098e0 tx=0x7ffa4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b9d5e4682c0d2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50001470 con 0x7ffa580cba50 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c011420 con 0x7ffa580cba50 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa57fff700 1 -- 172.21.15.3:0/1774378476 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0117c0 con 0x7ffa580cba50 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1774378476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580cba50 msgr2=0x7ffa58150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1774378476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580cba50 0x7ffa58150a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0098e0 tx=0x7ffa4c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1774378476 shutdown_connections 2023-10-06T04:59:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1774378476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580cba50 0x7ffa58150a60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1774378476 >> 172.21.15.3:0/1774378476 conn(0x7ffa58070310 msgr2=0x7ffa58070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1774378476 shutdown_connections 2023-10-06T04:59:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.572+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1774378476 wait complete. 2023-10-06T04:59:21.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.573+0000 7ffa5f2d4700 1 Processor -- start 2023-10-06T04:59:21.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.573+0000 7ffa5f2d4700 1 -- start start 2023-10-06T04:59:21.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5f2d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa580cba50 0x7ffa58153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5f2d4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa58150690 0x7ffa580b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5f2d4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa5815d540 0x7ffa580b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5f2d4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58150a60 con 0x7ffa580cba50 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5f2d4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58150fe0 con 0x7ffa5815d540 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5f2d4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580ca1e0 con 0x7ffa58150690 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5d070700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa580cba50 0x7ffa58153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5d070700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa580cba50 0x7ffa58153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37354/0 (socket says 172.21.15.3:37354) 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.574+0000 7ffa5d070700 1 -- 172.21.15.3:0/1165442362 learned_addr learned my addr 172.21.15.3:0/1165442362 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5d070700 1 -- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa58150690 msgr2=0x7ffa580b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5d070700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa58150690 0x7ffa580b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5d070700 1 -- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa5815d540 msgr2=0x7ffa580b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5d871700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa5815d540 0x7ffa580b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5d070700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa5815d540 0x7ffa580b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5d070700 1 -- 172.21.15.3:0/1165442362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c009420 con 0x7ffa580cba50 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5d070700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa580cba50 0x7ffa58153600 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000cb90 tx=0x7ffa50001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa557fa700 1 -- 172.21.15.3:0/1165442362 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50007360 con 0x7ffa580cba50 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa557fa700 1 -- 172.21.15.3:0/1165442362 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa500074e0 con 0x7ffa580cba50 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa580ca4c0 con 0x7ffa580cba50 2023-10-06T04:59:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa557fa700 1 -- 172.21.15.3:0/1165442362 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50004830 con 0x7ffa580cba50 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.575+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa580b5d20 con 0x7ffa580cba50 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.576+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5804c380 con 0x7ffa580cba50 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.576+0000 7ffa557fa700 1 -- 172.21.15.3:0/1165442362 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa50004ba0 con 0x7ffa580cba50 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.577+0000 7ffa557fa700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa44066f30 0x7ffa440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.577+0000 7ffa557fa700 1 -- 172.21.15.3:0/1165442362 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa50083e80 con 0x7ffa580cba50 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.577+0000 7ffa5c86f700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa44066f30 0x7ffa440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.578+0000 7ffa5c86f700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa44066f30 0x7ffa440693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ffa580b6170 tx=0x7ffa4c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.581+0000 7ffa557fa700 1 -- 172.21.15.3:0/1165442362 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa50051010 con 0x7ffa580cba50 2023-10-06T04:59:21.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.774+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7ffa5815df90 con 0x7ffa580cba50 2023-10-06T04:59:21.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.778+0000 7ffa557fa700 1 -- 172.21.15.3:0/1165442362 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v38)=0 v38) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffa500180c0 con 0x7ffa580cba50 2023-10-06T04:59:21.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa44066f30 msgr2=0x7ffa440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa44066f30 0x7ffa440693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ffa580b6170 tx=0x7ffa4c016000 comp rx=0 tx=0).stop 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa580cba50 msgr2=0x7ffa58153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa580cba50 0x7ffa58153600 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000cb90 tx=0x7ffa50001b50 comp rx=0 tx=0).stop 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 shutdown_connections 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa44066f30 0x7ffa440693e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa5815d540 0x7ffa580b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa580cba50 0x7ffa58153600 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 --2- 172.21.15.3:0/1165442362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa58150690 0x7ffa580b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 >> 172.21.15.3:0/1165442362 conn(0x7ffa58070310 msgr2=0x7ffa580af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 shutdown_connections 2023-10-06T04:59:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.780+0000 7ffa5f2d4700 1 -- 172.21.15.3:0/1165442362 wait complete. 2023-10-06T04:59:21.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-10-06T04:59:21.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.912+0000 7f2530e89700 1 Processor -- start 2023-10-06T04:59:21.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.912+0000 7f2530e89700 1 -- start start 2023-10-06T04:59:21.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.913+0000 7f2530e89700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 0x7f252c0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.913+0000 7f2530e89700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f252c0afad0 0x7f252c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.913+0000 7f2530e89700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c15d510 con 0x7f252c0cef10 2023-10-06T04:59:21.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.913+0000 7f2530e89700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252c15a480 con 0x7f252c0afad0 2023-10-06T04:59:21.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.913+0000 7f2530e89700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c15a760 con 0x7f252c0cfb80 2023-10-06T04:59:21.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.913+0000 7f252ad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f252c0cef10 0x7f252c15da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59038/0 (socket says 172.21.15.3:59038) 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.913+0000 7f252ad9d700 1 -- 172.21.15.3:0/3665847630 learned_addr learned my addr 172.21.15.3:0/3665847630 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f2529d9b700 1 --2- 172.21.15.3:0/3665847630 >> v2:172.21.15.153:3300/0 conn(0x7f252c0afad0 0x7f252c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252a59c700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 0x7f252c0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493712463 0 0) 0x7f252c15d510 con 0x7f252c0cef10 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2518003570 con 0x7f252c0cef10 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 52646966 0 0) 0x7f2518003570 con 0x7f252c0cef10 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 msgr2=0x7f252c0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 0x7f252c0af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 >> v2:172.21.15.153:3300/0 conn(0x7f252c0afad0 msgr2=0x7f252c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 --2- 172.21.15.3:0/3665847630 >> v2:172.21.15.153:3300/0 conn(0x7f252c0afad0 0x7f252c0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c15aa40 con 0x7f252c0cef10 2023-10-06T04:59:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f25240017f0 con 0x7f252c0cef10 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 >> v1:172.21.15.3:6789/0 conn(0x7f252c0cef10 legacy=0x7f252c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2518003a60 0x7f2518003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2518004810 0x7f2518004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 0x7f25180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c15a480 con 0x7f2518003a60 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c15a760 con 0x7f2518004810 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251800ba60 con 0x7f252c0cfb80 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252a59c700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2518003a60 0x7f2518003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f2529d9b700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2518004810 0x7f2518004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252ad9d700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 0x7f25180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252a59c700 1 -- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 msgr2=0x7f25180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:21.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252a59c700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f252c0cfb80 0x7f25180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252a59c700 1 -- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2518004810 msgr2=0x7f2518004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252a59c700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2518004810 0x7f2518004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f252a59c700 1 -- 172.21.15.3:0/3665847630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518003570 con 0x7f2518003a60 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.914+0000 7f2529d9b700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2518004810 0x7f2518004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f252a59c700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2518003a60 0x7f2518003e70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f251c009c30 tx=0x7f251c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c3c7c0d1d2e872 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c00e710 con 0x7f2518003a60 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c00e870 con 0x7f2518003a60 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f252959a700 1 -- 172.21.15.3:0/3665847630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c00ec30 con 0x7f2518003a60 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f2530e89700 1 -- 172.21.15.3:0/3665847630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2518003a60 msgr2=0x7f2518003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f2530e89700 1 --2- 172.21.15.3:0/3665847630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2518003a60 0x7f2518003e70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f251c009c30 tx=0x7f251c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f252ad9d700 1 -- 172.21.15.3:0/3665847630 reap_dead start 2023-10-06T04:59:21.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f2530e89700 1 -- 172.21.15.3:0/3665847630 shutdown_connections 2023-10-06T04:59:21.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f2530e89700 1 -- 172.21.15.3:0/3665847630 >> 172.21.15.3:0/3665847630 conn(0x7f252c070310 msgr2=0x7f252c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:21.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f2530e89700 1 -- 172.21.15.3:0/3665847630 shutdown_connections 2023-10-06T04:59:21.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.915+0000 7f2530e89700 1 -- 172.21.15.3:0/3665847630 wait complete. 2023-10-06T04:59:21.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.916+0000 7f2530e89700 1 Processor -- start 2023-10-06T04:59:21.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.916+0000 7f2530e89700 1 -- start start 2023-10-06T04:59:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f2530e89700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2518003a60 0x7f252c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f2530e89700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f252c0b0fe0 0x7f252c1697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f2530e89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252c169ce0 0x7f252c16a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f2530e89700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0b31e0 con 0x7f252c169ce0 2023-10-06T04:59:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f2530e89700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c16a6a0 con 0x7f252c0b0fe0 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f2530e89700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0b1740 con 0x7f2518003a60 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f252ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252c169ce0 0x7f252c16a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f252ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252c169ce0 0x7f252c16a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37374/0 (socket says 172.21.15.3:37374) 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f252ad9d700 1 -- 172.21.15.3:0/2780586951 learned_addr learned my addr 172.21.15.3:0/2780586951 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f252a59c700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2518003a60 0x7f252c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.917+0000 7f2529d9b700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f252c0b0fe0 0x7f252c1697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f252ad9d700 1 -- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2518003a60 msgr2=0x7f252c0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f252ad9d700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2518003a60 0x7f252c0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f252ad9d700 1 -- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f252c0b0fe0 msgr2=0x7f252c1697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f252ad9d700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f252c0b0fe0 0x7f252c1697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f252ad9d700 1 -- 172.21.15.3:0/2780586951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0b1a20 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f252ad9d700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252c169ce0 0x7f252c16a160 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f252c0c7480 tx=0x7f2524001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f25227fc700 1 -- 172.21.15.3:0/2780586951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524017070 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f25227fc700 1 -- 172.21.15.3:0/2780586951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252400eb10 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c0b1d00 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f25227fc700 1 -- 172.21.15.3:0/2780586951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252400eed0 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.918+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c0b1450 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.919+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252c0b3670 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.919+0000 7f25227fc700 1 -- 172.21.15.3:0/2780586951 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2524013630 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.920+0000 7f25227fc700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f250c066f30 0x7f250c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.920+0000 7f25227fc700 1 -- 172.21.15.3:0/2780586951 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2524084670 con 0x7f252c169ce0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.920+0000 7f252a59c700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f250c066f30 0x7f250c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.920+0000 7f252a59c700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f250c066f30 0x7f250c0693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f251c0091e0 tx=0x7f251c01a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:21.924+0000 7f25227fc700 1 -- 172.21.15.3:0/2780586951 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f252c0b3670 con 0x7f252c169ce0 2023-10-06T04:59:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.119+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7f252c0b3670 con 0x7f252c169ce0 2023-10-06T04:59:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.120+0000 7f25227fc700 1 -- 172.21.15.3:0/2780586951 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v38) v1 ==== 106+0+8 (secure 0 0 0) 0x7f252c0b3670 con 0x7f252c169ce0 2023-10-06T04:59:22.123 INFO:teuthology.orchestra.run.smithi003.stdout:1048576 2023-10-06T04:59:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f250c066f30 msgr2=0x7f250c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f250c066f30 0x7f250c0693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f251c0091e0 tx=0x7f251c01a000 comp rx=0 tx=0).stop 2023-10-06T04:59:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252c169ce0 msgr2=0x7f252c16a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252c169ce0 0x7f252c16a160 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f252c0c7480 tx=0x7f2524001860 comp rx=0 tx=0).stop 2023-10-06T04:59:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 shutdown_connections 2023-10-06T04:59:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f250c066f30 0x7f250c0693e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252c169ce0 0x7f252c16a160 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2518003a60 0x7f252c0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 --2- 172.21.15.3:0/2780586951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f252c0b0fe0 0x7f252c1697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 >> 172.21.15.3:0/2780586951 conn(0x7f252c070310 msgr2=0x7f252c06d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:22.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 shutdown_connections 2023-10-06T04:59:22.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.122+0000 7f2530e89700 1 -- 172.21.15.3:0/2780586951 wait complete. 2023-10-06T04:59:22.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-10-06T04:59:22.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.255+0000 7fd6fc146700 1 Processor -- start 2023-10-06T04:59:22.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.255+0000 7fd6fc146700 1 -- start start 2023-10-06T04:59:22.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fc146700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd6f4068190 0x7fd6f415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:22.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fc146700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f415cd40 0x7fd6f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fc146700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f415d180 con 0x7fd6f4067dc0 2023-10-06T04:59:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fc146700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f40691c0 con 0x7fd6f4068190 2023-10-06T04:59:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fc146700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40694a0 con 0x7fd6f415cd40 2023-10-06T04:59:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6f9ee2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd6f4067dc0 0x7fd6f40b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51628/0 (socket says 172.21.15.3:51628) 2023-10-06T04:59:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6f9ee2700 1 -- 172.21.15.3:0/2046423829 learned_addr learned my addr 172.21.15.3:0/2046423829 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6f96e1700 1 --2- 172.21.15.3:0/2046423829 >> v2:172.21.15.153:3300/0 conn(0x7fd6f4068190 0x7fd6f415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fa6e3700 1 --2- 172.21.15.3:0/2046423829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f415cd40 0x7fd6f406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6f8ee0700 1 -- 172.21.15.3:0/2046423829 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742305302 0 0) 0x7fd6f415d180 con 0x7fd6f4067dc0 2023-10-06T04:59:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6f8ee0700 1 -- 172.21.15.3:0/2046423829 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6dc003570 con 0x7fd6f4067dc0 2023-10-06T04:59:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fa6e3700 1 -- 172.21.15.3:0/2046423829 >> v2:172.21.15.153:3300/0 conn(0x7fd6f4068190 msgr2=0x7fd6f415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fa6e3700 1 --2- 172.21.15.3:0/2046423829 >> v2:172.21.15.153:3300/0 conn(0x7fd6f4068190 0x7fd6f415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fa6e3700 1 -- 172.21.15.3:0/2046423829 >> v1:172.21.15.3:6789/0 conn(0x7fd6f4067dc0 legacy=0x7fd6f40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6fa6e3700 1 -- 172.21.15.3:0/2046423829 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4069780 con 0x7fd6f415cd40 2023-10-06T04:59:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.256+0000 7fd6f96e1700 1 --2- 172.21.15.3:0/2046423829 >> v2:172.21.15.153:3300/0 conn(0x7fd6f4068190 0x7fd6f415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fa6e3700 1 --2- 172.21.15.3:0/2046423829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f415cd40 0x7fd6f406d480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e60 tx=0x7fd6e4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ab1657dde3d030 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6f8ee0700 1 -- 172.21.15.3:0/2046423829 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400e070 con 0x7fd6f415cd40 2023-10-06T04:59:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6f8ee0700 1 -- 172.21.15.3:0/2046423829 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e400bab0 con 0x7fd6f415cd40 2023-10-06T04:59:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6f8ee0700 1 -- 172.21.15.3:0/2046423829 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400be70 con 0x7fd6f415cd40 2023-10-06T04:59:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 -- 172.21.15.3:0/2046423829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f415cd40 msgr2=0x7fd6f406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 --2- 172.21.15.3:0/2046423829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f415cd40 0x7fd6f406d480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e60 tx=0x7fd6e4009780 comp rx=0 tx=0).stop 2023-10-06T04:59:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 -- 172.21.15.3:0/2046423829 shutdown_connections 2023-10-06T04:59:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 --2- 172.21.15.3:0/2046423829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f415cd40 0x7fd6f406d480 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 --2- 172.21.15.3:0/2046423829 >> v2:172.21.15.153:3300/0 conn(0x7fd6f4068190 0x7fd6f415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 -- 172.21.15.3:0/2046423829 >> 172.21.15.3:0/2046423829 conn(0x7fd6f40ce270 msgr2=0x7fd6f40ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 -- 172.21.15.3:0/2046423829 shutdown_connections 2023-10-06T04:59:22.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.257+0000 7fd6fc146700 1 -- 172.21.15.3:0/2046423829 wait complete. 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.258+0000 7fd6fc146700 1 Processor -- start 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6fc146700 1 -- start start 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6fc146700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f4067dc0 0x7fd6f40b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6fc146700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f4068190 0x7fd6f40aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6fc146700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6f415cd40 0x7fd6f4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6fc146700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40af450 con 0x7fd6f4068190 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6fc146700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40af730 con 0x7fd6f415cd40 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6fc146700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40afa10 con 0x7fd6f4067dc0 2023-10-06T04:59:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6f96e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f4068190 0x7fd6f40aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6f96e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f4068190 0x7fd6f40aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37344/0 (socket says 172.21.15.3:37344) 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.259+0000 7fd6f96e1700 1 -- 172.21.15.3:0/173224774 learned_addr learned my addr 172.21.15.3:0/173224774 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6fa6e3700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6f415cd40 0x7fd6f4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f9ee2700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f4067dc0 0x7fd6f40b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f96e1700 1 -- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f4067dc0 msgr2=0x7fd6f40b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f96e1700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f4067dc0 0x7fd6f40b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f96e1700 1 -- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6f415cd40 msgr2=0x7fd6f4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f96e1700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6f415cd40 0x7fd6f4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f96e1700 1 -- 172.21.15.3:0/173224774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009420 con 0x7fd6f4068190 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6fa6e3700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6f415cd40 0x7fd6f4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f9ee2700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f4067dc0 0x7fd6f40b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f96e1700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f4068190 0x7fd6f40aed70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800ec80 tx=0x7fd6e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f1ffb700 1 -- 172.21.15.3:0/173224774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800c9d0 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f1ffb700 1 -- 172.21.15.3:0/173224774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800cb30 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e4009a20 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6f1ffb700 1 -- 172.21.15.3:0/173224774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800ceb0 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.260+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f40affd0 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.261+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f404c380 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.262+0000 7fd6f1ffb700 1 -- 172.21.15.3:0/173224774 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6e8013680 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.262+0000 7fd6f1ffb700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f30 0x7fd6e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.262+0000 7fd6f1ffb700 1 -- 172.21.15.3:0/173224774 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd6e8084bf0 con 0x7fd6f4068190 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.262+0000 7fd6f9ee2700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f30 0x7fd6e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.263+0000 7fd6f9ee2700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f30 0x7fd6e00693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec001360 tx=0x7fd6ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.266+0000 7fd6f1ffb700 1 -- 172.21.15.3:0/173224774 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6e8051d00 con 0x7fd6f4068190 2023-10-06T04:59:22.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.461+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7fd6f4069a60 con 0x7fd6f4068190 2023-10-06T04:59:22.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.461+0000 7fd6f1ffb700 1 -- 172.21.15.3:0/173224774 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v38) v1 ==== 112+0+10 (secure 0 0 0) 0x7fd6e80170b0 con 0x7fd6f4068190 2023-10-06T04:59:22.466 INFO:teuthology.orchestra.run.smithi003.stdout:60.000000 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f30 msgr2=0x7fd6e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f30 0x7fd6e00693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec001360 tx=0x7fd6ec009040 comp rx=0 tx=0).stop 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f4068190 msgr2=0x7fd6f40aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f4068190 0x7fd6f40aed70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800ec80 tx=0x7fd6e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 shutdown_connections 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd6e0066f30 0x7fd6e00693e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6f415cd40 0x7fd6f4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6f4067dc0 0x7fd6f40b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 --2- 172.21.15.3:0/173224774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f4068190 0x7fd6f40aed70 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 >> 172.21.15.3:0/173224774 conn(0x7fd6f40ce270 msgr2=0x7fd6f41527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 shutdown_connections 2023-10-06T04:59:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:22.463+0000 7fd6fc146700 1 -- 172.21.15.3:0/173224774 wait complete. 2023-10-06T04:59:22.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:59:22.478 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T04:59:22.478 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T04:59:22.478 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:59:22.478 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T04:59:22.478 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.478 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:59:22.479 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:59:22.480 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T04:59:22.480 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T04:59:22.480 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T04:59:22.480 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:59:22.480 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.480 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T04:59:22.480 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:59:22.563 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.120433.asok 2023-10-06T04:59:22.745 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.120433.asok 2023-10-06T04:59:22.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:59:22.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.120433.asok mds_sessions 2023-10-06T04:59:22.900 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:59:22.900 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7248, 2023-10-06T04:59:22.901 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T04:59:22.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T04:59:22.901 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T04:59:22.902 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 7248 2023-10-06T04:59:22.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:22.903 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T04:59:22.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:59:22.905 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:59:22.905 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1770037939 2023-10-06T04:59:22.906 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:22.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.7248 v1:192.168.0.1:0/1770037939", 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/1770037939", 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-06T04:59:22.907 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6835", 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1968748272 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6837", 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1968748272 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:59:22.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 0, 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2023-10-06T05:00:18.658006+0000", 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2023-10-06T04:59:18.658006+0000", 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 1, 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 59 2023-10-06T04:59:22.909 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> import sys 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> import time 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> n = 800 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> if not os.path.exists(abs_path): 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(abs_path) 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.911 DEBUG:teuthology.orchestra.run.smithi003:> handles = [] 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(0, n): 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> fname = "file_"+str(i) 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> path = os.path.join(abs_path, fname) 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> handles.append(open(path, '"'"'w'"'"')) 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2023-10-06T04:59:22.912 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T04:59:22.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.036+0000 7f4427552700 1 Processor -- start 2023-10-06T04:59:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.037+0000 7f4427552700 1 -- start start 2023-10-06T04:59:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4427552700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155750 0x7f4420159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4427552700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f442006c620 0x7f44200b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4427552700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442015a520 con 0x7f4420155b20 2023-10-06T04:59:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4427552700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442006d710 con 0x7f442006c620 2023-10-06T04:59:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4427552700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442006cb00 con 0x7f4420155750 2023-10-06T04:59:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4424aed700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f442006c620 0x7f44200b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f44252ee700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155750 0x7f4420159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4425aef700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4420155b20 0x7f442006a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:57838/0 (socket says 172.21.15.3:57838) 2023-10-06T04:59:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4425aef700 1 -- 172.21.15.3:0/1584477064 learned_addr learned my addr 172.21.15.3:0/1584477064 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4417fff700 1 -- 172.21.15.3:0/1584477064 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349008410 0 0) 0x7f442015a520 con 0x7f4420155b20 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4417fff700 1 -- 172.21.15.3:0/1584477064 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440c003570 con 0x7f4420155b20 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4424aed700 1 -- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155750 msgr2=0x7f4420159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4424aed700 1 --2- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155750 0x7f4420159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4424aed700 1 -- 172.21.15.3:0/1584477064 >> v1:172.21.15.153:6789/0 conn(0x7f4420155b20 legacy=0x7f442006a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f4424aed700 1 -- 172.21.15.3:0/1584477064 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442006cde0 con 0x7f442006c620 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.038+0000 7f44252ee700 1 --2- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155750 0x7f4420159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4424aed700 1 --2- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f442006c620 0x7f44200b1800 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f44100098e0 tx=0x7f44100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c3410b52b9bcead server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4417fff700 1 -- 172.21.15.3:0/1584477064 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410011630 con 0x7f442006c620 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4417fff700 1 -- 172.21.15.3:0/1584477064 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410011790 con 0x7f442006c620 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4417fff700 1 -- 172.21.15.3:0/1584477064 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410011b30 con 0x7f442006c620 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 -- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f442006c620 msgr2=0x7f44200b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 --2- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f442006c620 0x7f44200b1800 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f44100098e0 tx=0x7f44100091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 -- 172.21.15.3:0/1584477064 shutdown_connections 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 --2- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155750 0x7f4420159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 --2- 172.21.15.3:0/1584477064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f442006c620 0x7f44200b1800 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 -- 172.21.15.3:0/1584477064 >> 172.21.15.3:0/1584477064 conn(0x7f4420066fb0 msgr2=0x7f44200673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 -- 172.21.15.3:0/1584477064 shutdown_connections 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.039+0000 7f4427552700 1 -- 172.21.15.3:0/1584477064 wait complete. 2023-10-06T04:59:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.041+0000 7f4427552700 1 Processor -- start 2023-10-06T04:59:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.041+0000 7f4427552700 1 -- start start 2023-10-06T04:59:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4427552700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f442006c620 0x7f442014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4427552700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4420155750 0x7f44200c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4427552700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155b20 0x7f44200c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4427552700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200c3a50 con 0x7f442006c620 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4427552700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4420156f70 con 0x7f4420155750 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4427552700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4420157250 con 0x7f4420155b20 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f442006c620 0x7f442014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f442006c620 0x7f442014c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37348/0 (socket says 172.21.15.3:37348) 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 -- 172.21.15.3:0/1977249078 learned_addr learned my addr 172.21.15.3:0/1977249078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4424aed700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4420155750 0x7f44200c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4425aef700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155b20 0x7f44200c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 -- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155b20 msgr2=0x7f44200c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155b20 0x7f44200c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 -- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4420155750 msgr2=0x7f44200c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4420155750 0x7f44200c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 -- 172.21.15.3:0/1977249078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c009420 con 0x7f442006c620 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44252ee700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f442006c620 0x7f442014c820 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f50 tx=0x7f441c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44157fa700 1 -- 172.21.15.3:0/1977249078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c00c8a0 con 0x7f442006c620 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44157fa700 1 -- 172.21.15.3:0/1977249078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c00ca00 con 0x7f442006c620 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4410009420 con 0x7f442006c620 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.042+0000 7f44157fa700 1 -- 172.21.15.3:0/1977249078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c00cd80 con 0x7f442006c620 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.043+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4420157810 con 0x7f442006c620 2023-10-06T04:59:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.044+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420071680 con 0x7f442006c620 2023-10-06T04:59:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.044+0000 7f44157fa700 1 -- 172.21.15.3:0/1977249078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f441c011660 con 0x7f442006c620 2023-10-06T04:59:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.044+0000 7f44157fa700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4404066f30 0x7f44040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.045+0000 7f44157fa700 1 -- 172.21.15.3:0/1977249078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f441c086650 con 0x7f442006c620 2023-10-06T04:59:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.045+0000 7f4424aed700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4404066f30 0x7f44040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.045+0000 7f4424aed700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4404066f30 0x7f44040693e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f442006ca30 tx=0x7f4410017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.049+0000 7f44157fa700 1 -- 172.21.15.3:0/1977249078 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f441c053760 con 0x7f442006c620 2023-10-06T04:59:23.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.312+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f442006c400 con 0x7f442006c620 2023-10-06T04:59:23.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.312+0000 7f44157fa700 1 -- 172.21.15.3:0/1977249078 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f441c056d80 con 0x7f442006c620 2023-10-06T04:59:23.315 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:23.315 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:09.043192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:23.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4404066f30 msgr2=0x7f44040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4404066f30 0x7f44040693e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f442006ca30 tx=0x7f4410017000 comp rx=0 tx=0).stop 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f442006c620 msgr2=0x7f442014c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f442006c620 0x7f442014c820 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f50 tx=0x7f441c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 shutdown_connections 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4404066f30 0x7f44040693e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4420155b20 0x7f44200c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f442006c620 0x7f442014c820 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 --2- 172.21.15.3:0/1977249078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4420155750 0x7f44200c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 >> 172.21.15.3:0/1977249078 conn(0x7f4420066fb0 msgr2=0x7f442014fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 shutdown_connections 2023-10-06T04:59:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:23.315+0000 7f4427552700 1 -- 172.21.15.3:0/1977249078 wait complete. 2023-10-06T04:59:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 59 2023-10-06T04:59:23.330 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T04:59:23.468 INFO:teuthology.orchestra.run.smithi153.stdout:[{"id":6802,"entity":{"name":{"type":"client","num":6802},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2791282291}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.00492983,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9885909508603237,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6802 192.168.0.1:0/2791282291","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"101273","root":"/"}},{"id":7248,"entity":{"name":{"type":"client","num":7248},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1770037939}},"state":"open","num_leases":1,"num_caps":346,"request_load_avg":346,"uptime":5.7988983999999997,"requests_in_flight":344,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":693.61246549856389,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7248 192.168.0.1:0/1770037939","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"120433","root":"/"}}] 2023-10-06T04:59:23.469 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "101273", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2791282291, "type": "any" }, "name": { "num": 6802, "type": "client" } }, "id": 6802, "inst": "client.6802 192.168.0.1:0/2791282291", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9885909508603237 }, "state": "open", "uptime": 4.00492983 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "120433", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1770037939, "type": "any" }, "name": { "num": 7248, "type": "client" } }, "id": 7248, "inst": "client.7248 192.168.0.1:0/1770037939", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 346, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 346, "requests_in_flight": 344, "session_cache_liveness": { "halflife": 300, "value": 693.6124654985639 }, "state": "open", "uptime": 5.7988984 } ] 2023-10-06T04:59:23.469 DEBUG:tasks.ceph_test_case:wait_until_equal: 346 != 802, waiting (timeout=600)... 2023-10-06T04:59:25.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:25.442 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:28.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.601+0000 7faedeb96700 1 Processor -- start 2023-10-06T04:59:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.602+0000 7faedeb96700 1 -- start start 2023-10-06T04:59:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.602+0000 7faedeb96700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 0x7faed80cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.602+0000 7faedeb96700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faed81596d0 0x7faed80cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.602+0000 7faedeb96700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed80cf130 con 0x7faed80cb780 2023-10-06T04:59:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.602+0000 7faedeb96700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed80ce520 con 0x7faed81596d0 2023-10-06T04:59:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.602+0000 7faedeb96700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80ce800 con 0x7faed80cb3b0 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedd393700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faed80cb780 0x7faed8158fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51644/0 (socket says 172.21.15.3:51644) 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedd393700 1 -- 172.21.15.3:0/2103921214 learned_addr learned my addr 172.21.15.3:0/2103921214 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faeddb94700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 0x7faed80cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faede395700 1 --2- 172.21.15.3:0/2103921214 >> v2:172.21.15.153:3300/0 conn(0x7faed81596d0 0x7faed80cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993173910 0 0) 0x7faed80cf130 con 0x7faed80cb780 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec0003570 con 0x7faed80cb780 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2701650718 0 0) 0x7faec0003570 con 0x7faed80cb780 2023-10-06T04:59:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 msgr2=0x7faed80cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 0x7faed80cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 >> v2:172.21.15.153:3300/0 conn(0x7faed81596d0 msgr2=0x7faed80cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 --2- 172.21.15.3:0/2103921214 >> v2:172.21.15.153:3300/0 conn(0x7faed81596d0 0x7faed80cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80ceae0 con 0x7faed80cb780 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7faecc001470 con 0x7faed80cb780 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faede395700 1 --2- 172.21.15.3:0/2103921214 >> v2:172.21.15.153:3300/0 conn(0x7faed81596d0 0x7faed80cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faeddb94700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 0x7faed80cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 >> v1:172.21.15.3:6789/0 conn(0x7faed80cb780 legacy=0x7faed8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 0x7faec0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faec0003fa0 0x7faec0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faec0004d50 0x7faec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed80ce520 con 0x7faec0004d50 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed80ce800 con 0x7faec0003fa0 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.603+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec000ba60 con 0x7faed80cb3b0 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faede395700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faec0004d50 0x7faec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faeddb94700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 0x7faec0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faedd393700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faec0003fa0 0x7faec0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faede395700 1 -- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 msgr2=0x7faec0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faede395700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed80cb3b0 0x7faec0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faede395700 1 -- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faec0003fa0 msgr2=0x7faec0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faede395700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faec0003fa0 0x7faec0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faede395700 1 -- 172.21.15.3:0/2103921214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0009420 con 0x7faec0004d50 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faede395700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faec0004d50 0x7faec00051c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7faec800e3c0 tx=0x7faec800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb9563b98b6c43c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800ec70 con 0x7faec0004d50 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800f040 con 0x7faec0004d50 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.604+0000 7faedcb92700 1 -- 172.21.15.3:0/2103921214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8014480 con 0x7faec0004d50 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.605+0000 7faedeb96700 1 -- 172.21.15.3:0/2103921214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faec0004d50 msgr2=0x7faec00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.605+0000 7faedeb96700 1 --2- 172.21.15.3:0/2103921214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faec0004d50 0x7faec00051c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7faec800e3c0 tx=0x7faec800b040 comp rx=0 tx=0).stop 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.605+0000 7faede395700 1 -- 172.21.15.3:0/2103921214 reap_dead start 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.605+0000 7faedeb96700 1 -- 172.21.15.3:0/2103921214 shutdown_connections 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.605+0000 7faedeb96700 1 -- 172.21.15.3:0/2103921214 >> 172.21.15.3:0/2103921214 conn(0x7faed8070310 msgr2=0x7faed8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:28.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.605+0000 7faedeb96700 1 -- 172.21.15.3:0/2103921214 shutdown_connections 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.605+0000 7faedeb96700 1 -- 172.21.15.3:0/2103921214 wait complete. 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.606+0000 7faedeb96700 1 Processor -- start 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.606+0000 7faedeb96700 1 -- start start 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faedeb96700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed8153790 0x7faed8157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faedeb96700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8153b60 0x7faed8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faedeb96700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed815e320 0x7faed815e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faedeb96700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8156940 con 0x7faed815e320 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faedeb96700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed815f000 con 0x7faed8153b60 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faedeb96700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed815f2e0 con 0x7faed8153790 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed815e320 0x7faed815e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed815e320 0x7faed815e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37366/0 (socket says 172.21.15.3:37366) 2023-10-06T04:59:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 -- 172.21.15.3:0/2671047719 learned_addr learned my addr 172.21.15.3:0/2671047719 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:28.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faedd393700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8153b60 0x7faed8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faeddb94700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed8153790 0x7faed8157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 -- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed8153790 msgr2=0x7faed8157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:28.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed8153790 0x7faed8157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 -- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8153b60 msgr2=0x7faed8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:28.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8153b60 0x7faed8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 -- 172.21.15.3:0/2671047719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed815f5c0 con 0x7faed815e320 2023-10-06T04:59:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.607+0000 7faede395700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed815e320 0x7faed815e7c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7faec8002230 tx=0x7faec801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.608+0000 7faed5ffb700 1 -- 172.21.15.3:0/2671047719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8014480 con 0x7faed815e320 2023-10-06T04:59:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.608+0000 7faed5ffb700 1 -- 172.21.15.3:0/2671047719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec801d630 con 0x7faed815e320 2023-10-06T04:59:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.608+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed815f8a0 con 0x7faed815e320 2023-10-06T04:59:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.608+0000 7faed5ffb700 1 -- 172.21.15.3:0/2671047719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec801d9f0 con 0x7faed815e320 2023-10-06T04:59:28.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.608+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed815ed00 con 0x7faed815e320 2023-10-06T04:59:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.609+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed80b3050 con 0x7faed815e320 2023-10-06T04:59:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.609+0000 7faed5ffb700 1 -- 172.21.15.3:0/2671047719 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faec8003040 con 0x7faed815e320 2023-10-06T04:59:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.610+0000 7faed5ffb700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faec4066f30 0x7faec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.610+0000 7faed5ffb700 1 -- 172.21.15.3:0/2671047719 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faec80856d0 con 0x7faed815e320 2023-10-06T04:59:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.610+0000 7faeddb94700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faec4066f30 0x7faec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:28.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.610+0000 7faeddb94700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faec4066f30 0x7faec40693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faed0006f00 tx=0x7faed00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:28.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.614+0000 7faed5ffb700 1 -- 172.21.15.3:0/2671047719 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faed80b3050 con 0x7faed815e320 2023-10-06T04:59:28.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.854+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faed80b3050 con 0x7faed815e320 2023-10-06T04:59:28.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.854+0000 7faed5ffb700 1 -- 172.21.15.3:0/2671047719 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4596 (secure 0 0 0) 0x7faed80b3050 con 0x7faed815e320 2023-10-06T04:59:28.857 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:28.857 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:26.788485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":8,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:28.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faec4066f30 msgr2=0x7faec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:28.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faec4066f30 0x7faec40693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faed0006f00 tx=0x7faed00091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed815e320 msgr2=0x7faed815e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:28.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed815e320 0x7faed815e7c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7faec8002230 tx=0x7faec801a000 comp rx=0 tx=0).stop 2023-10-06T04:59:28.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 shutdown_connections 2023-10-06T04:59:28.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faec4066f30 0x7faec40693e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed815e320 0x7faed815e7c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faed8153790 0x7faed8157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 --2- 172.21.15.3:0/2671047719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8153b60 0x7faed8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:28.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 >> 172.21.15.3:0/2671047719 conn(0x7faed8070310 msgr2=0x7faed80b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:28.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 shutdown_connections 2023-10-06T04:59:28.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:28.856+0000 7faedeb96700 1 -- 172.21.15.3:0/2671047719 wait complete. 2023-10-06T04:59:28.860 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 60 2023-10-06T04:59:28.871 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T04:59:29.018 INFO:teuthology.orchestra.run.smithi153.stdout:[{"id":6802,"entity":{"name":{"type":"client","num":6802},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2791282291}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":9.5508326599999993,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9505333647398309,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6802 192.168.0.1:0/2791282291","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"101273","root":"/"}},{"id":7248,"entity":{"name":{"type":"client","num":7248},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1770037939}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":981,"uptime":11.345801213,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1933.0876225208128,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":625.85303927254824,"halflife":1.5},"recall_caps_throttle2o":{"value":141.56991696145286,"halflife":0.5},"session_cache_liveness":{"value":1586.0219371289888,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":506,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7248 192.168.0.1:0/1770037939","completed_requests":[{"tid":506,"created_ino":"0x100000003e9"},{"tid":507,"created_ino":"0x100000003ea"},{"tid":508,"created_ino":"0x100000003eb"},{"tid":509,"created_ino":"0x100000003ec"},{"tid":510,"created_ino":"0x100000003ed"},{"tid":511,"created_ino":"0x100000003ee"},{"tid":512,"created_ino":"0x100000003ef"},{"tid":513,"created_ino":"0x100000003f0"},{"tid":514,"created_ino":"0x100000003f1"},{"tid":515,"created_ino":"0x100000003f2"},{"tid":516,"created_ino":"0x100000003f3"},{"tid":517,"created_ino":"0x100000003f4"},{"tid":518,"created_ino":"0x100000003f5"},{"tid":519,"created_ino":"0x100000003f6"},{"tid":520,"created_ino":"0x100000003f7"},{"tid":521,"created_ino":"0x100000003f8"},{"tid":522,"created_ino":"0x100000003f9"},{"tid":523,"created_ino":"0x100000003fa"},{"tid":524,"created_ino":"0x100000003fb"},{"tid":525,"created_ino":"0x100000003fc"},{"tid":526,"created_ino":"0x100000003fd"},{"tid":527,"created_ino":"0x100000003fe"},{"tid":528,"created_ino":"0x100000003ff"},{"tid":529,"created_ino":"0x10000000400"},{"tid":530,"created_ino":"0x10000000401"},{"tid":531,"created_ino":"0x10000000402"},{"tid":532,"created_ino":"0x10000000403"},{"tid":533,"created_ino":"0x10000000404"},{"tid":534,"created_ino":"0x10000000405"},{"tid":535,"created_ino":"0x10000000406"},{"tid":536,"created_ino":"0x10000000407"},{"tid":537,"created_ino":"0x10000000408"},{"tid":538,"created_ino":"0x10000000409"},{"tid":539,"created_ino":"0x1000000040a"},{"tid":540,"created_ino":"0x1000000040b"},{"tid":541,"created_ino":"0x1000000040c"},{"tid":542,"created_ino":"0x1000000040d"},{"tid":543,"created_ino":"0x1000000040e"},{"tid":544,"created_ino":"0x1000000040f"},{"tid":545,"created_ino":"0x10000000410"},{"tid":546,"created_ino":"0x10000000411"},{"tid":547,"created_ino":"0x10000000412"},{"tid":548,"created_ino":"0x10000000413"},{"tid":549,"created_ino":"0x10000000414"},{"tid":550,"created_ino":"0x10000000415"},{"tid":551,"created_ino":"0x10000000416"},{"tid":552,"created_ino":"0x10000000417"},{"tid":553,"created_ino":"0x10000000418"},{"tid":554,"created_ino":"0x10000000419"},{"tid":555,"created_ino":"0x1000000041a"},{"tid":556,"created_ino":"0x1000000041b"},{"tid":557,"created_ino":"0x1000000041c"},{"tid":558,"created_ino":"0x1000000041d"},{"tid":559,"created_ino":"0x1000000041e"},{"tid":560,"created_ino":"0x1000000041f"},{"tid":561,"created_ino":"0x10000000420"},{"tid":562,"created_ino":"0x10000000421"},{"tid":563,"created_ino":"0x10000000422"},{"tid":564,"created_ino":"0x10000000423"},{"tid":565,"created_ino":"0x10000000424"},{"tid":566,"created_ino":"0x10000000425"},{"tid":567,"created_ino":"0x10000000426"},{"tid":568,"created_ino":"0x10000000427"},{"tid":569,"created_ino":"0x10000000428"},{"tid":570,"created_ino":"0x10000000429"},{"tid":571,"created_ino":"0x1000000042a"},{"tid":572,"created_ino":"0x1000000042b"},{"tid":573,"created_ino":"0x1000000042c"},{"tid":574,"created_ino":"0x1000000042d"},{"tid":575,"created_ino":"0x1000000042e"},{"tid":576,"created_ino":"0x1000000042f"},{"tid":577,"created_ino":"0x10000000430"},{"tid":578,"created_ino":"0x10000000431"},{"tid":579,"created_ino":"0x10000000432"},{"tid":580,"created_ino":"0x10000000433"},{"tid":581,"created_ino":"0x10000000434"},{"tid":582,"created_ino":"0x10000000435"},{"tid":583,"created_ino":"0x10000000436"},{"tid":584,"created_ino":"0x10000000437"},{"tid":585,"created_ino":"0x10000000438"},{"tid":586,"created_ino":"0x10000000439"},{"tid":587,"created_ino":"0x1000000043a"},{"tid":588,"created_ino":"0x1000000043b"},{"tid":589,"created_ino":"0x1000000043c"},{"tid":590,"created_ino":"0x1000000043d"},{"tid":591,"created_ino":"0x1000000043e"},{"tid":592,"created_ino":"0x1000000043f"},{"tid":593,"created_ino":"0x10000000440"},{"tid":594,"created_ino":"0x10000000441"},{"tid":595,"created_ino":"0x10000000442"},{"tid":596,"created_ino":"0x10000000443"},{"tid":597,"created_ino":"0x10000000444"},{"tid":598,"created_ino":"0x10000000445"},{"tid":599,"created_ino":"0x10000000446"},{"tid":600,"created_ino":"0x10000000447"},{"tid":601,"created_ino":"0x10000000448"},{"tid":602,"created_ino":"0x10000000449"},{"tid":603,"created_ino":"0x1000000044a"},{"tid":604,"created_ino":"0x1000000044b"},{"tid":605,"created_ino":"0x1000000044c"},{"tid":606,"created_ino":"0x1000000044d"},{"tid":607,"created_ino":"0x1000000044e"},{"tid":608,"created_ino":"0x1000000044f"},{"tid":609,"created_ino":"0x10000000450"},{"tid":610,"created_ino":"0x10000000451"},{"tid":611,"created_ino":"0x10000000452"},{"tid":612,"created_ino":"0x10000000453"},{"tid":613,"created_ino":"0x10000000454"},{"tid":614,"created_ino":"0x10000000455"},{"tid":615,"created_ino":"0x10000000456"},{"tid":616,"created_ino":"0x10000000457"},{"tid":617,"created_ino":"0x10000000458"},{"tid":618,"created_ino":"0x10000000459"},{"tid":619,"created_ino":"0x1000000045a"},{"tid":620,"created_ino":"0x1000000045b"},{"tid":621,"created_ino":"0x1000000045c"},{"tid":622,"created_ino":"0x1000000045d"},{"tid":623,"created_ino":"0x1000000045e"},{"tid":624,"created_ino":"0x1000000045f"},{"tid":625,"created_ino":"0x10000000460"},{"tid":626,"created_ino":"0x10000000461"},{"tid":627,"created_ino":"0x10000000462"},{"tid":628,"created_ino":"0x10000000463"},{"tid":629,"created_ino":"0x10000000464"},{"tid":630,"created_ino":"0x10000000465"},{"tid":631,"created_ino":"0x10000000466"},{"tid":632,"created_ino":"0x10000000467"},{"tid":633,"created_ino":"0x10000000468"},{"tid":634,"created_ino":"0x10000000469"},{"tid":635,"created_ino":"0x1000000046a"},{"tid":636,"created_ino":"0x1000000046b"},{"tid":637,"created_ino":"0x1000000046c"},{"tid":638,"created_ino":"0x1000000046d"},{"tid":639,"created_ino":"0x1000000046e"},{"tid":640,"created_ino":"0x1000000046f"},{"tid":641,"created_ino":"0x10000000470"},{"tid":642,"created_ino":"0x10000000471"},{"tid":643,"created_ino":"0x10000000472"},{"tid":644,"created_ino":"0x10000000473"},{"tid":645,"created_ino":"0x10000000474"},{"tid":646,"created_ino":"0x10000000475"},{"tid":647,"created_ino":"0x10000000476"},{"tid":648,"created_ino":"0x10000000477"},{"tid":649,"created_ino":"0x10000000478"},{"tid":650,"created_ino":"0x10000000479"},{"tid":651,"created_ino":"0x1000000047a"},{"tid":652,"created_ino":"0x1000000047b"},{"tid":653,"created_ino":"0x1000000047c"},{"tid":654,"created_ino":"0x1000000047d"},{"tid":655,"created_ino":"0x1000000047e"},{"tid":656,"created_ino":"0x1000000047f"},{"tid":657,"created_ino":"0x10000000480"},{"tid":658,"created_ino":"0x10000000481"},{"tid":659,"created_ino":"0x10000000482"},{"tid":660,"created_ino":"0x10000000483"},{"tid":661,"created_ino":"0x10000000484"},{"tid":662,"created_ino":"0x10000000485"},{"tid":663,"created_ino":"0x10000000486"},{"tid":664,"created_ino":"0x10000000487"},{"tid":665,"created_ino":"0x10000000488"},{"tid":666,"created_ino":"0x10000000489"},{"tid":667,"created_ino":"0x1000000048a"},{"tid":668,"created_ino":"0x1000000048b"},{"tid":669,"created_ino":"0x1000000048c"},{"tid":670,"created_ino":"0x1000000048d"},{"tid":671,"created_ino":"0x1000000048e"},{"tid":672,"created_ino":"0x1000000048f"},{"tid":673,"created_ino":"0x10000000490"},{"tid":674,"created_ino":"0x10000000491"},{"tid":675,"created_ino":"0x10000000492"},{"tid":676,"created_ino":"0x10000000493"},{"tid":677,"created_ino":"0x10000000494"},{"tid":678,"created_ino":"0x10000000495"},{"tid":679,"created_ino":"0x10000000496"},{"tid":680,"created_ino":"0x10000000497"},{"tid":681,"created_ino":"0x10000000498"},{"tid":682,"created_ino":"0x10000000499"},{"tid":683,"created_ino":"0x1000000049a"},{"tid":684,"created_ino":"0x1000000049b"},{"tid":685,"created_ino":"0x1000000049c"},{"tid":688,"created_ino":"0x1000000049d"},{"tid":692,"created_ino":"0x1000000049e"},{"tid":695,"created_ino":"0x1000000049f"},{"tid":698,"created_ino":"0x100000004a0"},{"tid":701,"created_ino":"0x100000004a1"},{"tid":704,"created_ino":"0x100000004a2"},{"tid":707,"created_ino":"0x100000004a3"},{"tid":710,"created_ino":"0x100000004a4"},{"tid":713,"created_ino":"0x100000004a5"},{"tid":716,"created_ino":"0x100000004a6"},{"tid":719,"created_ino":"0x100000004a7"},{"tid":722,"created_ino":"0x100000004a8"},{"tid":725,"created_ino":"0x100000004a9"},{"tid":728,"created_ino":"0x100000004aa"},{"tid":731,"created_ino":"0x100000004ab"},{"tid":734,"created_ino":"0x100000004ac"},{"tid":737,"created_ino":"0x100000004ad"},{"tid":740,"created_ino":"0x100000004ae"},{"tid":743,"created_ino":"0x100000004af"},{"tid":746,"created_ino":"0x100000004b0"},{"tid":749,"created_ino":"0x100000004b1"},{"tid":752,"created_ino":"0x100000004b2"},{"tid":755,"created_ino":"0x100000004b3"},{"tid":758,"created_ino":"0x100000004b4"},{"tid":761,"created_ino":"0x100000004b5"},{"tid":764,"created_ino":"0x100000004b6"},{"tid":767,"created_ino":"0x100000004b7"},{"tid":770,"created_ino":"0x100000004b8"},{"tid":773,"created_ino":"0x100000004b9"},{"tid":776,"created_ino":"0x100000004ba"},{"tid":779,"created_ino":"0x100000004bb"},{"tid":782,"created_ino":"0x100000004bc"},{"tid":785,"created_ino":"0x100000004bd"},{"tid":788,"created_ino":"0x100000004be"},{"tid":791,"created_ino":"0x100000004bf"},{"tid":794,"created_ino":"0x100000004c0"},{"tid":797,"created_ino":"0x100000004c1"},{"tid":800,"created_ino":"0x100000004c2"},{"tid":803,"created_ino":"0x100000004c3"},{"tid":806,"created_ino":"0x100000004c4"},{"tid":809,"created_ino":"0x100000004c5"},{"tid":812,"created_ino":"0x100000004c6"},{"tid":815,"created_ino":"0x100000004c7"},{"tid":818,"created_ino":"0x100000004c8"},{"tid":821,"created_ino":"0x100000004c9"},{"tid":824,"created_ino":"0x100000004ca"},{"tid":827,"created_ino":"0x100000004cb"},{"tid":830,"created_ino":"0x100000004cc"},{"tid":833,"created_ino":"0x100000004cd"},{"tid":836,"created_ino":"0x100000004ce"},{"tid":839,"created_ino":"0x100000004cf"},{"tid":842,"created_ino":"0x100000004d0"},{"tid":845,"created_ino":"0x100000004d1"},{"tid":848,"created_ino":"0x100000004d2"},{"tid":851,"created_ino":"0x100000004d3"},{"tid":854,"created_ino":"0x100000004d4"},{"tid":857,"created_ino":"0x100000004d5"},{"tid":860,"created_ino":"0x100000004d6"},{"tid":863,"created_ino":"0x100000004d7"},{"tid":866,"created_ino":"0x100000004d8"},{"tid":869,"created_ino":"0x100000004d9"},{"tid":872,"created_ino":"0x100000004da"},{"tid":875,"created_ino":"0x100000004db"},{"tid":878,"created_ino":"0x100000004dc"},{"tid":881,"created_ino":"0x100000004dd"},{"tid":884,"created_ino":"0x100000004de"},{"tid":887,"created_ino":"0x100000004df"},{"tid":890,"created_ino":"0x100000004e0"},{"tid":893,"created_ino":"0x100000004e1"},{"tid":896,"created_ino":"0x100000004e2"},{"tid":899,"created_ino":"0x100000004e3"},{"tid":902,"created_ino":"0x100000004e4"},{"tid":905,"created_ino":"0x100000004e5"},{"tid":908,"created_ino":"0x100000004e6"},{"tid":911,"created_ino":"0x100000004e7"},{"tid":914,"created_ino":"0x100000004e8"},{"tid":917,"created_ino":"0x100000004e9"},{"tid":920,"created_ino":"0x100000004ea"},{"tid":923,"created_ino":"0x100000004eb"},{"tid":926,"created_ino":"0x100000004ec"},{"tid":929,"created_ino":"0x100000004ed"},{"tid":932,"created_ino":"0x100000004ee"},{"tid":935,"created_ino":"0x100000004ef"},{"tid":938,"created_ino":"0x100000004f0"},{"tid":941,"created_ino":"0x100000004f1"},{"tid":944,"created_ino":"0x100000004f2"},{"tid":947,"created_ino":"0x100000004f3"},{"tid":950,"created_ino":"0x100000004f4"},{"tid":953,"created_ino":"0x100000004f5"},{"tid":956,"created_ino":"0x100000004f6"},{"tid":959,"created_ino":"0x100000004f7"},{"tid":962,"created_ino":"0x100000004f8"},{"tid":965,"created_ino":"0x100000004f9"},{"tid":968,"created_ino":"0x100000004fa"},{"tid":971,"created_ino":"0x100000004fb"},{"tid":974,"created_ino":"0x100000004fc"},{"tid":977,"created_ino":"0x100000004fd"},{"tid":980,"created_ino":"0x100000004fe"},{"tid":983,"created_ino":"0x100000004ff"},{"tid":986,"created_ino":"0x10000000500"},{"tid":989,"created_ino":"0x10000000501"},{"tid":992,"created_ino":"0x10000000502"},{"tid":995,"created_ino":"0x10000000503"},{"tid":998,"created_ino":"0x10000000504"},{"tid":1001,"created_ino":"0x10000000505"},{"tid":1004,"created_ino":"0x10000000506"},{"tid":1007,"created_ino":"0x10000000507"},{"tid":1010,"created_ino":"0x10000000508"},{"tid":1013,"created_ino":"0x10000000509"},{"tid":1016,"created_ino":"0x1000000050a"},{"tid":1019,"created_ino":"0x1000000050b"},{"tid":1022,"created_ino":"0x1000000050c"},{"tid":1025,"created_ino":"0x1000000050d"},{"tid":1028,"created_ino":"0x1000000050e"},{"tid":1031,"created_ino":"0x1000000050f"},{"tid":1034,"created_ino":"0x10000000510"},{"tid":1037,"created_ino":"0x10000000511"},{"tid":1040,"created_ino":"0x10000000512"},{"tid":1043,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"120433","root":"/"}}] 2023-10-06T04:59:29.021 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "101273", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2791282291, "type": "any" }, "name": { "num": 6802, "type": "client" } }, "id": 6802, "inst": "client.6802 192.168.0.1:0/2791282291", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.950533364739831 }, "state": "open", "uptime": 9.55083266 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "120433", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 506 }, { "created_ino": "0x100000003ea", "tid": 507 }, { "created_ino": "0x100000003eb", "tid": 508 }, { "created_ino": "0x100000003ec", "tid": 509 }, { "created_ino": "0x100000003ed", "tid": 510 }, { "created_ino": "0x100000003ee", "tid": 511 }, { "created_ino": "0x100000003ef", "tid": 512 }, { "created_ino": "0x100000003f0", "tid": 513 }, { "created_ino": "0x100000003f1", "tid": 514 }, { "created_ino": "0x100000003f2", "tid": 515 }, { "created_ino": "0x100000003f3", "tid": 516 }, { "created_ino": "0x100000003f4", "tid": 517 }, { "created_ino": "0x100000003f5", "tid": 518 }, { "created_ino": "0x100000003f6", "tid": 519 }, { "created_ino": "0x100000003f7", "tid": 520 }, { "created_ino": "0x100000003f8", "tid": 521 }, { "created_ino": "0x100000003f9", "tid": 522 }, { "created_ino": "0x100000003fa", "tid": 523 }, { "created_ino": "0x100000003fb", "tid": 524 }, { "created_ino": "0x100000003fc", "tid": 525 }, { "created_ino": "0x100000003fd", "tid": 526 }, { "created_ino": "0x100000003fe", "tid": 527 }, { "created_ino": "0x100000003ff", "tid": 528 }, { "created_ino": "0x10000000400", "tid": 529 }, { "created_ino": "0x10000000401", "tid": 530 }, { "created_ino": "0x10000000402", "tid": 531 }, { "created_ino": "0x10000000403", "tid": 532 }, { "created_ino": "0x10000000404", "tid": 533 }, { "created_ino": "0x10000000405", "tid": 534 }, { "created_ino": "0x10000000406", "tid": 535 }, { "created_ino": "0x10000000407", "tid": 536 }, { "created_ino": "0x10000000408", "tid": 537 }, { "created_ino": "0x10000000409", "tid": 538 }, { "created_ino": "0x1000000040a", "tid": 539 }, { "created_ino": "0x1000000040b", "tid": 540 }, { "created_ino": "0x1000000040c", "tid": 541 }, { "created_ino": "0x1000000040d", "tid": 542 }, { "created_ino": "0x1000000040e", "tid": 543 }, { "created_ino": "0x1000000040f", "tid": 544 }, { "created_ino": "0x10000000410", "tid": 545 }, { "created_ino": "0x10000000411", "tid": 546 }, { "created_ino": "0x10000000412", "tid": 547 }, { "created_ino": "0x10000000413", "tid": 548 }, { "created_ino": "0x10000000414", "tid": 549 }, { "created_ino": "0x10000000415", "tid": 550 }, { "created_ino": "0x10000000416", "tid": 551 }, { "created_ino": "0x10000000417", "tid": 552 }, { "created_ino": "0x10000000418", "tid": 553 }, { "created_ino": "0x10000000419", "tid": 554 }, { "created_ino": "0x1000000041a", "tid": 555 }, { "created_ino": "0x1000000041b", "tid": 556 }, { "created_ino": "0x1000000041c", "tid": 557 }, { "created_ino": "0x1000000041d", "tid": 558 }, { "created_ino": "0x1000000041e", "tid": 559 }, { "created_ino": "0x1000000041f", "tid": 560 }, { "created_ino": "0x10000000420", "tid": 561 }, { "created_ino": "0x10000000421", "tid": 562 }, { "created_ino": "0x10000000422", "tid": 563 }, { "created_ino": "0x10000000423", "tid": 564 }, { "created_ino": "0x10000000424", "tid": 565 }, { "created_ino": "0x10000000425", "tid": 566 }, { "created_ino": "0x10000000426", "tid": 567 }, { "created_ino": "0x10000000427", "tid": 568 }, { "created_ino": "0x10000000428", "tid": 569 }, { "created_ino": "0x10000000429", "tid": 570 }, { "created_ino": "0x1000000042a", "tid": 571 }, { "created_ino": "0x1000000042b", "tid": 572 }, { "created_ino": "0x1000000042c", "tid": 573 }, { "created_ino": "0x1000000042d", "tid": 574 }, { "created_ino": "0x1000000042e", "tid": 575 }, { "created_ino": "0x1000000042f", "tid": 576 }, { "created_ino": "0x10000000430", "tid": 577 }, { "created_ino": "0x10000000431", "tid": 578 }, { "created_ino": "0x10000000432", "tid": 579 }, { "created_ino": "0x10000000433", "tid": 580 }, { "created_ino": "0x10000000434", "tid": 581 }, { "created_ino": "0x10000000435", "tid": 582 }, { "created_ino": "0x10000000436", "tid": 583 }, { "created_ino": "0x10000000437", "tid": 584 }, { "created_ino": "0x10000000438", "tid": 585 }, { "created_ino": "0x10000000439", "tid": 586 }, { "created_ino": "0x1000000043a", "tid": 587 }, { "created_ino": "0x1000000043b", "tid": 588 }, { "created_ino": "0x1000000043c", "tid": 589 }, { "created_ino": "0x1000000043d", "tid": 590 }, { "created_ino": "0x1000000043e", "tid": 591 }, { "created_ino": "0x1000000043f", "tid": 592 }, { "created_ino": "0x10000000440", "tid": 593 }, { "created_ino": "0x10000000441", "tid": 594 }, { "created_ino": "0x10000000442", "tid": 595 }, { "created_ino": "0x10000000443", "tid": 596 }, { "created_ino": "0x10000000444", "tid": 597 }, { "created_ino": "0x10000000445", "tid": 598 }, { "created_ino": "0x10000000446", "tid": 599 }, { "created_ino": "0x10000000447", "tid": 600 }, { "created_ino": "0x10000000448", "tid": 601 }, { "created_ino": "0x10000000449", "tid": 602 }, { "created_ino": "0x1000000044a", "tid": 603 }, { "created_ino": "0x1000000044b", "tid": 604 }, { "created_ino": "0x1000000044c", "tid": 605 }, { "created_ino": "0x1000000044d", "tid": 606 }, { "created_ino": "0x1000000044e", "tid": 607 }, { "created_ino": "0x1000000044f", "tid": 608 }, { "created_ino": "0x10000000450", "tid": 609 }, { "created_ino": "0x10000000451", "tid": 610 }, { "created_ino": "0x10000000452", "tid": 611 }, { "created_ino": "0x10000000453", "tid": 612 }, { "created_ino": "0x10000000454", "tid": 613 }, { "created_ino": "0x10000000455", "tid": 614 }, { "created_ino": "0x10000000456", "tid": 615 }, { "created_ino": "0x10000000457", "tid": 616 }, { "created_ino": "0x10000000458", "tid": 617 }, { "created_ino": "0x10000000459", "tid": 618 }, { "created_ino": "0x1000000045a", "tid": 619 }, { "created_ino": "0x1000000045b", "tid": 620 }, { "created_ino": "0x1000000045c", "tid": 621 }, { "created_ino": "0x1000000045d", "tid": 622 }, { "created_ino": "0x1000000045e", "tid": 623 }, { "created_ino": "0x1000000045f", "tid": 624 }, { "created_ino": "0x10000000460", "tid": 625 }, { "created_ino": "0x10000000461", "tid": 626 }, { "created_ino": "0x10000000462", "tid": 627 }, { "created_ino": "0x10000000463", "tid": 628 }, { "created_ino": "0x10000000464", "tid": 629 }, { "created_ino": "0x10000000465", "tid": 630 }, { "created_ino": "0x10000000466", "tid": 631 }, { "created_ino": "0x10000000467", "tid": 632 }, { "created_ino": "0x10000000468", "tid": 633 }, { "created_ino": "0x10000000469", "tid": 634 }, { "created_ino": "0x1000000046a", "tid": 635 }, { "created_ino": "0x1000000046b", "tid": 636 }, { "created_ino": "0x1000000046c", "tid": 637 }, { "created_ino": "0x1000000046d", "tid": 638 }, { "created_ino": "0x1000000046e", "tid": 639 }, { "created_ino": "0x1000000046f", "tid": 640 }, { "created_ino": "0x10000000470", "tid": 641 }, { "created_ino": "0x10000000471", "tid": 642 }, { "created_ino": "0x10000000472", "tid": 643 }, { "created_ino": "0x10000000473", "tid": 644 }, { "created_ino": "0x10000000474", "tid": 645 }, { "created_ino": "0x10000000475", "tid": 646 }, { "created_ino": "0x10000000476", "tid": 647 }, { "created_ino": "0x10000000477", "tid": 648 }, { "created_ino": "0x10000000478", "tid": 649 }, { "created_ino": "0x10000000479", "tid": 650 }, { "created_ino": "0x1000000047a", "tid": 651 }, { "created_ino": "0x1000000047b", "tid": 652 }, { "created_ino": "0x1000000047c", "tid": 653 }, { "created_ino": "0x1000000047d", "tid": 654 }, { "created_ino": "0x1000000047e", "tid": 655 }, { "created_ino": "0x1000000047f", "tid": 656 }, { "created_ino": "0x10000000480", "tid": 657 }, { "created_ino": "0x10000000481", "tid": 658 }, { "created_ino": "0x10000000482", "tid": 659 }, { "created_ino": "0x10000000483", "tid": 660 }, { "created_ino": "0x10000000484", "tid": 661 }, { "created_ino": "0x10000000485", "tid": 662 }, { "created_ino": "0x10000000486", "tid": 663 }, { "created_ino": "0x10000000487", "tid": 664 }, { "created_ino": "0x10000000488", "tid": 665 }, { "created_ino": "0x10000000489", "tid": 666 }, { "created_ino": "0x1000000048a", "tid": 667 }, { "created_ino": "0x1000000048b", "tid": 668 }, { "created_ino": "0x1000000048c", "tid": 669 }, { "created_ino": "0x1000000048d", "tid": 670 }, { "created_ino": "0x1000000048e", "tid": 671 }, { "created_ino": "0x1000000048f", "tid": 672 }, { "created_ino": "0x10000000490", "tid": 673 }, { "created_ino": "0x10000000491", "tid": 674 }, { "created_ino": "0x10000000492", "tid": 675 }, { "created_ino": "0x10000000493", "tid": 676 }, { "created_ino": "0x10000000494", "tid": 677 }, { "created_ino": "0x10000000495", "tid": 678 }, { "created_ino": "0x10000000496", "tid": 679 }, { "created_ino": "0x10000000497", "tid": 680 }, { "created_ino": "0x10000000498", "tid": 681 }, { "created_ino": "0x10000000499", "tid": 682 }, { "created_ino": "0x1000000049a", "tid": 683 }, { "created_ino": "0x1000000049b", "tid": 684 }, { "created_ino": "0x1000000049c", "tid": 685 }, { "created_ino": "0x1000000049d", "tid": 688 }, { "created_ino": "0x1000000049e", "tid": 692 }, { "created_ino": "0x1000000049f", "tid": 695 }, { "created_ino": "0x100000004a0", "tid": 698 }, { "created_ino": "0x100000004a1", "tid": 701 }, { "created_ino": "0x100000004a2", "tid": 704 }, { "created_ino": "0x100000004a3", "tid": 707 }, { "created_ino": "0x100000004a4", "tid": 710 }, { "created_ino": "0x100000004a5", "tid": 713 }, { "created_ino": "0x100000004a6", "tid": 716 }, { "created_ino": "0x100000004a7", "tid": 719 }, { "created_ino": "0x100000004a8", "tid": 722 }, { "created_ino": "0x100000004a9", "tid": 725 }, { "created_ino": "0x100000004aa", "tid": 728 }, { "created_ino": "0x100000004ab", "tid": 731 }, { "created_ino": "0x100000004ac", "tid": 734 }, { "created_ino": "0x100000004ad", "tid": 737 }, { "created_ino": "0x100000004ae", "tid": 740 }, { "created_ino": "0x100000004af", "tid": 743 }, { "created_ino": "0x100000004b0", "tid": 746 }, { "created_ino": "0x100000004b1", "tid": 749 }, { "created_ino": "0x100000004b2", "tid": 752 }, { "created_ino": "0x100000004b3", "tid": 755 }, { "created_ino": "0x100000004b4", "tid": 758 }, { "created_ino": "0x100000004b5", "tid": 761 }, { "created_ino": "0x100000004b6", "tid": 764 }, { "created_ino": "0x100000004b7", "tid": 767 }, { "created_ino": "0x100000004b8", "tid": 770 }, { "created_ino": "0x100000004b9", "tid": 773 }, { "created_ino": "0x100000004ba", "tid": 776 }, { "created_ino": "0x100000004bb", "tid": 779 }, { "created_ino": "0x100000004bc", "tid": 782 }, { "created_ino": "0x100000004bd", "tid": 785 }, { "created_ino": "0x100000004be", "tid": 788 }, { "created_ino": "0x100000004bf", "tid": 791 }, { "created_ino": "0x100000004c0", "tid": 794 }, { "created_ino": "0x100000004c1", "tid": 797 }, { "created_ino": "0x100000004c2", "tid": 800 }, { "created_ino": "0x100000004c3", "tid": 803 }, { "created_ino": "0x100000004c4", "tid": 806 }, { "created_ino": "0x100000004c5", "tid": 809 }, { "created_ino": "0x100000004c6", "tid": 812 }, { "created_ino": "0x100000004c7", "tid": 815 }, { "created_ino": "0x100000004c8", "tid": 818 }, { "created_ino": "0x100000004c9", "tid": 821 }, { "created_ino": "0x100000004ca", "tid": 824 }, { "created_ino": "0x100000004cb", "tid": 827 }, { "created_ino": "0x100000004cc", "tid": 830 }, { "created_ino": "0x100000004cd", "tid": 833 }, { "created_ino": "0x100000004ce", "tid": 836 }, { "created_ino": "0x100000004cf", "tid": 839 }, { "created_ino": "0x100000004d0", "tid": 842 }, { "created_ino": "0x100000004d1", "tid": 845 }, { "created_ino": "0x100000004d2", "tid": 848 }, { "created_ino": "0x100000004d3", "tid": 851 }, { "created_ino": "0x100000004d4", "tid": 854 }, { "created_ino": "0x100000004d5", "tid": 857 }, { "created_ino": "0x100000004d6", "tid": 860 }, { "created_ino": "0x100000004d7", "tid": 863 }, { "created_ino": "0x100000004d8", "tid": 866 }, { "created_ino": "0x100000004d9", "tid": 869 }, { "created_ino": "0x100000004da", "tid": 872 }, { "created_ino": "0x100000004db", "tid": 875 }, { "created_ino": "0x100000004dc", "tid": 878 }, { "created_ino": "0x100000004dd", "tid": 881 }, { "created_ino": "0x100000004de", "tid": 884 }, { "created_ino": "0x100000004df", "tid": 887 }, { "created_ino": "0x100000004e0", "tid": 890 }, { "created_ino": "0x100000004e1", "tid": 893 }, { "created_ino": "0x100000004e2", "tid": 896 }, { "created_ino": "0x100000004e3", "tid": 899 }, { "created_ino": "0x100000004e4", "tid": 902 }, { "created_ino": "0x100000004e5", "tid": 905 }, { "created_ino": "0x100000004e6", "tid": 908 }, { "created_ino": "0x100000004e7", "tid": 911 }, { "created_ino": "0x100000004e8", "tid": 914 }, { "created_ino": "0x100000004e9", "tid": 917 }, { "created_ino": "0x100000004ea", "tid": 920 }, { "created_ino": "0x100000004eb", "tid": 923 }, { "created_ino": "0x100000004ec", "tid": 926 }, { "created_ino": "0x100000004ed", "tid": 929 }, { "created_ino": "0x100000004ee", "tid": 932 }, { "created_ino": "0x100000004ef", "tid": 935 }, { "created_ino": "0x100000004f0", "tid": 938 }, { "created_ino": "0x100000004f1", "tid": 941 }, { "created_ino": "0x100000004f2", "tid": 944 }, { "created_ino": "0x100000004f3", "tid": 947 }, { "created_ino": "0x100000004f4", "tid": 950 }, { "created_ino": "0x100000004f5", "tid": 953 }, { "created_ino": "0x100000004f6", "tid": 956 }, { "created_ino": "0x100000004f7", "tid": 959 }, { "created_ino": "0x100000004f8", "tid": 962 }, { "created_ino": "0x100000004f9", "tid": 965 }, { "created_ino": "0x100000004fa", "tid": 968 }, { "created_ino": "0x100000004fb", "tid": 971 }, { "created_ino": "0x100000004fc", "tid": 974 }, { "created_ino": "0x100000004fd", "tid": 977 }, { "created_ino": "0x100000004fe", "tid": 980 }, { "created_ino": "0x100000004ff", "tid": 983 }, { "created_ino": "0x10000000500", "tid": 986 }, { "created_ino": "0x10000000501", "tid": 989 }, { "created_ino": "0x10000000502", "tid": 992 }, { "created_ino": "0x10000000503", "tid": 995 }, { "created_ino": "0x10000000504", "tid": 998 }, { "created_ino": "0x10000000505", "tid": 1001 }, { "created_ino": "0x10000000506", "tid": 1004 }, { "created_ino": "0x10000000507", "tid": 1007 }, { "created_ino": "0x10000000508", "tid": 1010 }, { "created_ino": "0x10000000509", "tid": 1013 }, { "created_ino": "0x1000000050a", "tid": 1016 }, { "created_ino": "0x1000000050b", "tid": 1019 }, { "created_ino": "0x1000000050c", "tid": 1022 }, { "created_ino": "0x1000000050d", "tid": 1025 }, { "created_ino": "0x1000000050e", "tid": 1028 }, { "created_ino": "0x1000000050f", "tid": 1031 }, { "created_ino": "0x10000000510", "tid": 1034 }, { "created_ino": "0x10000000511", "tid": 1037 }, { "created_ino": "0x10000000512", "tid": 1040 }, { "created_ino": "0x10000000513", "tid": 1043 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1770037939, "type": "any" }, "name": { "num": 7248, "type": "client" } }, "id": 7248, "inst": "client.7248 192.168.0.1:0/1770037939", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 506, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 1933.0876225208128 }, "recall_caps_throttle": { "halflife": 1.5, "value": 625.8530392725482 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 141.56991696145286 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 981, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1586.0219371289888 }, "state": "open", "uptime": 11.345801213 } ] 2023-10-06T04:59:29.024 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-10-06T04:59:29.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:59:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.141+0000 7f9cfb59e700 1 Processor -- start 2023-10-06T04:59:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.141+0000 7f9cfb59e700 1 -- start start 2023-10-06T04:59:29.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 0x7f9cfc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfb59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9cfc15d660 0x7f9cfc0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfb59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cfc0cb240 con 0x7f9cfc1592c0 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfb59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cfc15e0c0 con 0x7f9cfc15d660 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0cb7e0 con 0x7f9cfc15a6a0 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfa59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9cfc1592c0 0x7f9cfc0ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51658/0 (socket says 172.21.15.3:51658) 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfa59c700 1 -- 172.21.15.3:0/3701662994 learned_addr learned my addr 172.21.15.3:0/3701662994 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cfad9d700 1 --2- 172.21.15.3:0/3701662994 >> v2:172.21.15.153:3300/0 conn(0x7f9cfc15d660 0x7f9cfc0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.142+0000 7f9cf9d9b700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 0x7f9cfc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660924109 0 0) 0x7f9cfc0cb240 con 0x7f9cfc1592c0 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce8003570 con 0x7f9cfc1592c0 2023-10-06T04:59:29.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2535888290 0 0) 0x7f9ce8003570 con 0x7f9cfc1592c0 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 msgr2=0x7f9cfc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 0x7f9cfc15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 >> v2:172.21.15.153:3300/0 conn(0x7f9cfc15d660 msgr2=0x7f9cfc0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 --2- 172.21.15.3:0/3701662994 >> v2:172.21.15.153:3300/0 conn(0x7f9cfc15d660 0x7f9cfc0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc0ce900 con 0x7f9cfc1592c0 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f9cec001470 con 0x7f9cfc1592c0 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 >> v1:172.21.15.3:6789/0 conn(0x7f9cfc1592c0 legacy=0x7f9cfc0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 0x7f9ce80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc15e0c0 con 0x7f9ce8004810 2023-10-06T04:59:29.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0cb7e0 con 0x7f9ce8003a60 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce800ba60 con 0x7f9cfc15a6a0 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cf9d9b700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfa59c700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfad9d700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 0x7f9ce80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfa59c700 1 -- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 msgr2=0x7f9ce80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfa59c700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 0x7f9ce80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfa59c700 1 -- 172.21.15.3:0/3701662994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8004810 msgr2=0x7f9ce8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfa59c700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfa59c700 1 -- 172.21.15.3:0/3701662994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce8003570 con 0x7f9ce8003a60 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.143+0000 7f9cfad9d700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc15a6a0 0x7f9ce80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfa59c700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00c3f0 tx=0x7f9cec001e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3769d9672f14c76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00cc80 con 0x7f9ce8003a60 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec00ce00 con 0x7f9ce8003a60 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cf959a700 1 -- 172.21.15.3:0/3701662994 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec005b70 con 0x7f9ce8003a60 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfb59e700 1 -- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce8003a60 msgr2=0x7f9ce8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfb59e700 1 --2- 172.21.15.3:0/3701662994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00c3f0 tx=0x7f9cec001e90 comp rx=0 tx=0).stop 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfad9d700 1 -- 172.21.15.3:0/3701662994 reap_dead start 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfb59e700 1 -- 172.21.15.3:0/3701662994 shutdown_connections 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfb59e700 1 -- 172.21.15.3:0/3701662994 >> 172.21.15.3:0/3701662994 conn(0x7f9cfc089d90 msgr2=0x7f9cfc0996b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfb59e700 1 -- 172.21.15.3:0/3701662994 shutdown_connections 2023-10-06T04:59:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.144+0000 7f9cfb59e700 1 -- 172.21.15.3:0/3701662994 wait complete. 2023-10-06T04:59:29.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 Processor -- start 2023-10-06T04:59:29.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 -- start start 2023-10-06T04:59:29.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8003a60 0x7f9cfc16a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc153a80 0x7f9cfc0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cfc0c9620 0x7f9cfc0c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0ca000 con 0x7f9ce8003a60 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc15ea00 con 0x7f9cfc0c9620 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0b34e0 con 0x7f9cfc153a80 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8003a60 0x7f9cfc16a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8003a60 0x7f9cfc16a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37380/0 (socket says 172.21.15.3:37380) 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.146+0000 7f9cfa59c700 1 -- 172.21.15.3:0/2039173374 learned_addr learned my addr 172.21.15.3:0/2039173374 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cf9d9b700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc153a80 0x7f9cfc0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfa59c700 1 -- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc153a80 msgr2=0x7f9cfc0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfa59c700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc153a80 0x7f9cfc0b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfa59c700 1 -- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cfc0c9620 msgr2=0x7f9cfc0c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfa59c700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cfc0c9620 0x7f9cfc0c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfa59c700 1 -- 172.21.15.3:0/2039173374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc15e3b0 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cf9d9b700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc153a80 0x7f9cfc0b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfa59c700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8003a60 0x7f9cfc16a9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9cec001ed0 tx=0x7f9cec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cf27fc700 1 -- 172.21.15.3:0/2039173374 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00cc80 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cf27fc700 1 -- 172.21.15.3:0/2039173374 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0072e0 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc15e690 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cf27fc700 1 -- 172.21.15.3:0/2039173374 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0076c0 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.147+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc153ef0 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.148+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc04c380 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.148+0000 7f9cf27fc700 1 -- 172.21.15.3:0/2039173374 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9cec019030 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.149+0000 7f9cf27fc700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cdc066f30 0x7f9cdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.149+0000 7f9cf27fc700 1 -- 172.21.15.3:0/2039173374 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9cec085e90 con 0x7f9ce8003a60 2023-10-06T04:59:29.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.149+0000 7f9cf9d9b700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cdc066f30 0x7f9cdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:29.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.149+0000 7f9cf9d9b700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cdc066f30 0x7f9cdc0693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40093f0 tx=0x7f9ce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:29.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.153+0000 7f9cf27fc700 1 -- 172.21.15.3:0/2039173374 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cec052fa0 con 0x7f9ce8003a60 2023-10-06T04:59:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.391+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9cfc0cdc20 con 0x7f9ce8003a60 2023-10-06T04:59:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.391+0000 7f9cf27fc700 1 -- 172.21.15.3:0/2039173374 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f9cec0565c0 con 0x7f9ce8003a60 2023-10-06T04:59:29.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:29.394 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cdc066f30 msgr2=0x7f9cdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cdc066f30 0x7f9cdc0693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40093f0 tx=0x7f9ce4008040 comp rx=0 tx=0).stop 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8003a60 msgr2=0x7f9cfc16a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8003a60 0x7f9cfc16a9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9cec001ed0 tx=0x7f9cec015000 comp rx=0 tx=0).stop 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 shutdown_connections 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cdc066f30 0x7f9cdc0693e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cfc0c9620 0x7f9cfc0c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce8003a60 0x7f9cfc16a9c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 --2- 172.21.15.3:0/2039173374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cfc153a80 0x7f9cfc0b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 >> 172.21.15.3:0/2039173374 conn(0x7f9cfc089d90 msgr2=0x7f9cfc0c6110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:29.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 shutdown_connections 2023-10-06T04:59:29.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:29.393+0000 7f9cfb59e700 1 -- 172.21.15.3:0/2039173374 wait complete. 2023-10-06T04:59:29.407 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T04:59:29.408 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T04:59:34.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:59:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.539+0000 7f51a32a1700 1 Processor -- start 2023-10-06T04:59:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a32a1700 1 -- start start 2023-10-06T04:59:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a32a1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f519c0b27b0 0x7f519c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a32a1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f519c15d240 0x7f519c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a32a1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c0cd150 0x7f519c0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a32a1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c15aba0 con 0x7f519c15d240 2023-10-06T04:59:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a32a1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c150b30 con 0x7f519c0b27b0 2023-10-06T04:59:34.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a32a1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0ce560 con 0x7f519c0cd150 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.540+0000 7f51a083c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f519c15d240 0x7f519c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f519c15d240 0x7f519c0ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34810/0 (socket says 172.21.15.3:34810) 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 -- 172.21.15.3:0/2740575967 learned_addr learned my addr 172.21.15.3:0/2740575967 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a183e700 1 --2- 172.21.15.3:0/2740575967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c0cd150 0x7f519c0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a103d700 1 --2- 172.21.15.3:0/2740575967 >> v2:172.21.15.153:3300/0 conn(0x7f519c0b27b0 0x7f519c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 -- 172.21.15.3:0/2740575967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c0cd150 msgr2=0x7f519c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 --2- 172.21.15.3:0/2740575967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c0cd150 0x7f519c0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 -- 172.21.15.3:0/2740575967 >> v2:172.21.15.153:3300/0 conn(0x7f519c0b27b0 msgr2=0x7f519c15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 --2- 172.21.15.3:0/2740575967 >> v2:172.21.15.153:3300/0 conn(0x7f519c0b27b0 0x7f519c15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 -- 172.21.15.3:0/2740575967 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0cd5d0 con 0x7f519c15d240 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a183e700 1 --2- 172.21.15.3:0/2740575967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c0cd150 0x7f519c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f51a083c700 1 --2- 172.21.15.3:0/2740575967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c15d240 0x7f519c0ccc10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f519c0b2f40 tx=0x7f5190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98725be0161496aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f519bfff700 1 -- 172.21.15.3:0/2740575967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000b860 con 0x7f519c15d240 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f519bfff700 1 -- 172.21.15.3:0/2740575967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519000b9c0 con 0x7f519c15d240 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.541+0000 7f519bfff700 1 -- 172.21.15.3:0/2740575967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000bd60 con 0x7f519c15d240 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 -- 172.21.15.3:0/2740575967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c15d240 msgr2=0x7f519c0ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 --2- 172.21.15.3:0/2740575967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c15d240 0x7f519c0ccc10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f519c0b2f40 tx=0x7f5190009250 comp rx=0 tx=0).stop 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 -- 172.21.15.3:0/2740575967 shutdown_connections 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 --2- 172.21.15.3:0/2740575967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c0cd150 0x7f519c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 --2- 172.21.15.3:0/2740575967 >> v2:172.21.15.153:3300/0 conn(0x7f519c0b27b0 0x7f519c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 --2- 172.21.15.3:0/2740575967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c15d240 0x7f519c0ccc10 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 -- 172.21.15.3:0/2740575967 >> 172.21.15.3:0/2740575967 conn(0x7f519c070310 msgr2=0x7f519c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 -- 172.21.15.3:0/2740575967 shutdown_connections 2023-10-06T04:59:34.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.542+0000 7f51a32a1700 1 -- 172.21.15.3:0/2740575967 wait complete. 2023-10-06T04:59:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.543+0000 7f51a32a1700 1 Processor -- start 2023-10-06T04:59:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.543+0000 7f51a32a1700 1 -- start start 2023-10-06T04:59:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a32a1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f519c0b27b0 0x7f519c0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a32a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c0cd150 0x7f519c0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a32a1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c15d240 0x7f519c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a32a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c150570 con 0x7f519c0cd150 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a32a1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c153820 con 0x7f519c0b27b0 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a32a1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c8d20 con 0x7f519c15d240 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c0cd150 0x7f519c0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c0cd150 0x7f519c0c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34814/0 (socket says 172.21.15.3:34814) 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 -- 172.21.15.3:0/1417706038 learned_addr learned my addr 172.21.15.3:0/1417706038 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a183e700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c15d240 0x7f519c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a103d700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f519c0b27b0 0x7f519c0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 -- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c15d240 msgr2=0x7f519c1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c15d240 0x7f519c1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 -- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f519c0b27b0 msgr2=0x7f519c0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f519c0b27b0 0x7f519c0c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.544+0000 7f51a083c700 1 -- 172.21.15.3:0/1417706038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900094f0 con 0x7f519c0cd150 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.545+0000 7f51a083c700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c0cd150 0x7f519c0c7a70 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5190006fd0 tx=0x7f5190017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.545+0000 7f51997fa700 1 -- 172.21.15.3:0/1417706038 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190011e00 con 0x7f519c0cd150 2023-10-06T04:59:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.545+0000 7f51997fa700 1 -- 172.21.15.3:0/1417706038 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519001a450 con 0x7f519c0cd150 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.545+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c0c8110 con 0x7f519c0cd150 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.545+0000 7f51997fa700 1 -- 172.21.15.3:0/1417706038 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519001a810 con 0x7f519c0cd150 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.545+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c0c86d0 con 0x7f519c0cd150 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.546+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c071680 con 0x7f519c0cd150 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.546+0000 7f51997fa700 1 -- 172.21.15.3:0/1417706038 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f519001ab80 con 0x7f519c0cd150 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.547+0000 7f51997fa700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5188066f30 0x7f51880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.547+0000 7f51997fa700 1 -- 172.21.15.3:0/1417706038 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51900860e0 con 0x7f519c0cd150 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.547+0000 7f51a103d700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5188066f30 0x7f51880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.547+0000 7f51a103d700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5188066f30 0x7f51880693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f518c0075e0 tx=0x7f518c009830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.551+0000 7f51997fa700 1 -- 172.21.15.3:0/1417706038 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5190053270 con 0x7f519c0cd150 2023-10-06T04:59:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.791+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f519c0c8e60 con 0x7f519c0cd150 2023-10-06T04:59:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.791+0000 7f51997fa700 1 -- 172.21.15.3:0/1417706038 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f5190056890 con 0x7f519c0cd150 2023-10-06T04:59:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:34.793 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:59:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5188066f30 msgr2=0x7f51880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5188066f30 0x7f51880693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f518c0075e0 tx=0x7f518c009830 comp rx=0 tx=0).stop 2023-10-06T04:59:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c0cd150 msgr2=0x7f519c0c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c0cd150 0x7f519c0c7a70 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5190006fd0 tx=0x7f5190017000 comp rx=0 tx=0).stop 2023-10-06T04:59:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 shutdown_connections 2023-10-06T04:59:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5188066f30 0x7f51880693e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f519c0b27b0 0x7f519c0c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f519c15d240 0x7f519c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 --2- 172.21.15.3:0/1417706038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519c0cd150 0x7f519c0c7a70 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 >> 172.21.15.3:0/1417706038 conn(0x7f519c070310 msgr2=0x7f519c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 shutdown_connections 2023-10-06T04:59:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.793+0000 7f51a32a1700 1 -- 172.21.15.3:0/1417706038 wait complete. 2023-10-06T04:59:34.807 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T04:59:34.807 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-10-06T04:59:34.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:59:34.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.923+0000 7f99e0edd700 1 Processor -- start 2023-10-06T04:59:34.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.923+0000 7f99e0edd700 1 -- start start 2023-10-06T04:59:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99e0edd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc0b27b0 0x7f99dc15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99e0edd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f99dc15d240 0x7f99dc0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99e0edd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99dc0cd150 0x7f99dc0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99e0edd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc15aba0 con 0x7f99dc15d240 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99e0edd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc150b30 con 0x7f99dc0cd150 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99e0edd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0ce560 con 0x7f99dc0b27b0 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99d9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f99dc15d240 0x7f99dc0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.924+0000 7f99d9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f99dc15d240 0x7f99dc0ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34816/0 (socket says 172.21.15.3:34816) 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d9d9b700 1 -- 172.21.15.3:0/1939704731 learned_addr learned my addr 172.21.15.3:0/1939704731 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99da59c700 1 --2- 172.21.15.3:0/1939704731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc0b27b0 0x7f99dc15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99dad9d700 1 --2- 172.21.15.3:0/1939704731 >> v2:172.21.15.153:3300/0 conn(0x7f99dc0cd150 0x7f99dc0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d9d9b700 1 -- 172.21.15.3:0/1939704731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc0b27b0 msgr2=0x7f99dc15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d9d9b700 1 --2- 172.21.15.3:0/1939704731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc0b27b0 0x7f99dc15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d9d9b700 1 -- 172.21.15.3:0/1939704731 >> v2:172.21.15.153:3300/0 conn(0x7f99dc0cd150 msgr2=0x7f99dc0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d9d9b700 1 --2- 172.21.15.3:0/1939704731 >> v2:172.21.15.153:3300/0 conn(0x7f99dc0cd150 0x7f99dc0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d9d9b700 1 -- 172.21.15.3:0/1939704731 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0cd5f0 con 0x7f99dc15d240 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99dad9d700 1 --2- 172.21.15.3:0/1939704731 >> v2:172.21.15.153:3300/0 conn(0x7f99dc0cd150 0x7f99dc0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d9d9b700 1 --2- 172.21.15.3:0/1939704731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc15d240 0x7f99dc0ccc10 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f99c40099b0 tx=0x7f99c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccb1dbf82f36d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d959a700 1 -- 172.21.15.3:0/1939704731 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400b820 con 0x7f99dc15d240 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d959a700 1 -- 172.21.15.3:0/1939704731 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c400b980 con 0x7f99dc15d240 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.925+0000 7f99d959a700 1 -- 172.21.15.3:0/1939704731 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400bd20 con 0x7f99dc15d240 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 -- 172.21.15.3:0/1939704731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc15d240 msgr2=0x7f99dc0ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 --2- 172.21.15.3:0/1939704731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc15d240 0x7f99dc0ccc10 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f99c40099b0 tx=0x7f99c4009250 comp rx=0 tx=0).stop 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 -- 172.21.15.3:0/1939704731 shutdown_connections 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 --2- 172.21.15.3:0/1939704731 >> v2:172.21.15.153:3300/0 conn(0x7f99dc0cd150 0x7f99dc0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 --2- 172.21.15.3:0/1939704731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc0b27b0 0x7f99dc15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 --2- 172.21.15.3:0/1939704731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc15d240 0x7f99dc0ccc10 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 -- 172.21.15.3:0/1939704731 >> 172.21.15.3:0/1939704731 conn(0x7f99dc070310 msgr2=0x7f99dc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:34.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 -- 172.21.15.3:0/1939704731 shutdown_connections 2023-10-06T04:59:34.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.926+0000 7f99e0edd700 1 -- 172.21.15.3:0/1939704731 wait complete. 2023-10-06T04:59:34.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.927+0000 7f99e0edd700 1 Processor -- start 2023-10-06T04:59:34.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.927+0000 7f99e0edd700 1 -- start start 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc0b27b0 0x7f99dc14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc0cd150 0x7f99dc0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc15d240 0x7f99dc0d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c81e0 con 0x7f99dc0b27b0 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c75d0 con 0x7f99dc0cd150 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c78b0 con 0x7f99dc15d240 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc0b27b0 0x7f99dc14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc0b27b0 0x7f99dc14f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34828/0 (socket says 172.21.15.3:34828) 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 -- 172.21.15.3:0/4072499648 learned_addr learned my addr 172.21.15.3:0/4072499648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99dad9d700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc15d240 0x7f99dc0d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99d9d9b700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc0cd150 0x7f99dc0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 -- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc15d240 msgr2=0x7f99dc0d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc15d240 0x7f99dc0d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 -- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc0cd150 msgr2=0x7f99dc0d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc0cd150 0x7f99dc0d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 -- 172.21.15.3:0/4072499648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c40094f0 con 0x7f99dc0b27b0 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99d9d9b700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc0cd150 0x7f99dc0d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99da59c700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc0b27b0 0x7f99dc14f750 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d860 tx=0x7f99cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99d27fc700 1 -- 172.21.15.3:0/4072499648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc009610 con 0x7f99dc0b27b0 2023-10-06T04:59:34.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99d27fc700 1 -- 172.21.15.3:0/4072499648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc009770 con 0x7f99dc0b27b0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99d27fc700 1 -- 172.21.15.3:0/4072499648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc009b30 con 0x7f99dc0b27b0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc0c7b90 con 0x7f99dc0b27b0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.928+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc14fc90 con 0x7f99dc0b27b0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.929+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc0c84c0 con 0x7f99dc0b27b0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.930+0000 7f99d27fc700 1 -- 172.21.15.3:0/4072499648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99cc01e460 con 0x7f99dc0b27b0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.930+0000 7f99d27fc700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f99bc066f30 0x7f99bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.930+0000 7f99d27fc700 1 -- 172.21.15.3:0/4072499648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f99cc084d00 con 0x7f99dc0b27b0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.930+0000 7f99d9d9b700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f99bc066f30 0x7f99bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:34.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.931+0000 7f99d9d9b700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f99bc066f30 0x7f99bc0693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f99c40099b0 tx=0x7f99c4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:34.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:34.934+0000 7f99d27fc700 1 -- 172.21.15.3:0/4072499648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99cc051e10 con 0x7f99dc0b27b0 2023-10-06T04:59:35.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.177+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f99dc14fe80 con 0x7f99dc0b27b0 2023-10-06T04:59:35.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.177+0000 7f99d27fc700 1 -- 172.21.15.3:0/4072499648 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f99cc055430 con 0x7f99dc0b27b0 2023-10-06T04:59:35.179 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:35.179 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-10-06T04:59:35.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f99bc066f30 msgr2=0x7f99bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f99bc066f30 0x7f99bc0693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f99c40099b0 tx=0x7f99c4017000 comp rx=0 tx=0).stop 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc0b27b0 msgr2=0x7f99dc14f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc0b27b0 0x7f99dc14f750 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d860 tx=0x7f99cc008040 comp rx=0 tx=0).stop 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 shutdown_connections 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f99bc066f30 0x7f99bc0693e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99dc15d240 0x7f99dc0d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99dc0b27b0 0x7f99dc14f750 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 --2- 172.21.15.3:0/4072499648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc0cd150 0x7f99dc0d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 >> 172.21.15.3:0/4072499648 conn(0x7f99dc070310 msgr2=0x7f99dc0c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 shutdown_connections 2023-10-06T04:59:35.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.180+0000 7f99e0edd700 1 -- 172.21.15.3:0/4072499648 wait complete. 2023-10-06T04:59:35.193 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T04:59:35.193 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-10-06T04:59:35.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:35.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.377+0000 7fd35b7e3700 1 Processor -- start 2023-10-06T04:59:35.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.377+0000 7fd35b7e3700 1 -- start start 2023-10-06T04:59:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd35b7e3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd3540b1b10 0x7fd3540cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd35b7e3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157ad0 0x7fd354067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd35b7e3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3540b2e50 con 0x7fd354157700 2023-10-06T04:59:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd35b7e3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd354158290 con 0x7fd3540b1b10 2023-10-06T04:59:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd35b7e3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540b1fc0 con 0x7fd354157ad0 2023-10-06T04:59:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd35957f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd354157700 0x7fd3540b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57904/0 (socket says 172.21.15.3:57904) 2023-10-06T04:59:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd35957f700 1 -- 172.21.15.3:0/3108679312 learned_addr learned my addr 172.21.15.3:0/3108679312 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd358d7e700 1 --2- 172.21.15.3:0/3108679312 >> v2:172.21.15.153:3300/0 conn(0x7fd3540b1b10 0x7fd3540cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.378+0000 7fd359d80700 1 --2- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157ad0 0x7fd354067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd353fff700 1 -- 172.21.15.3:0/3108679312 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767830534 0 0) 0x7fd3540b2e50 con 0x7fd354157700 2023-10-06T04:59:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd353fff700 1 -- 172.21.15.3:0/3108679312 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd33c003570 con 0x7fd354157700 2023-10-06T04:59:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd358d7e700 1 -- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157ad0 msgr2=0x7fd354067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd358d7e700 1 --2- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157ad0 0x7fd354067b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd358d7e700 1 -- 172.21.15.3:0/3108679312 >> v1:172.21.15.3:6789/0 conn(0x7fd354157700 legacy=0x7fd3540b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd358d7e700 1 -- 172.21.15.3:0/3108679312 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3540b22a0 con 0x7fd3540b1b10 2023-10-06T04:59:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd359d80700 1 --2- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157ad0 0x7fd354067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd358d7e700 1 --2- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3540b1b10 0x7fd3540cc270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd3480098e0 tx=0x7fd3480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c7b8ed375f0c2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd353fff700 1 -- 172.21.15.3:0/3108679312 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348011630 con 0x7fd3540b1b10 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd353fff700 1 -- 172.21.15.3:0/3108679312 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348011790 con 0x7fd3540b1b10 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.379+0000 7fd353fff700 1 -- 172.21.15.3:0/3108679312 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348011b10 con 0x7fd3540b1b10 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 -- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3540b1b10 msgr2=0x7fd3540cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3540b1b10 0x7fd3540cc270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd3480098e0 tx=0x7fd3480091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 -- 172.21.15.3:0/3108679312 shutdown_connections 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157ad0 0x7fd354067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/3108679312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3540b1b10 0x7fd3540cc270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 -- 172.21.15.3:0/3108679312 >> 172.21.15.3:0/3108679312 conn(0x7fd3540ce2c0 msgr2=0x7fd3540ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 -- 172.21.15.3:0/3108679312 shutdown_connections 2023-10-06T04:59:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.380+0000 7fd35b7e3700 1 -- 172.21.15.3:0/3108679312 wait complete. 2023-10-06T04:59:35.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.381+0000 7fd35b7e3700 1 Processor -- start 2023-10-06T04:59:35.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.381+0000 7fd35b7e3700 1 -- start start 2023-10-06T04:59:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35b7e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3540b1b10 0x7fd354145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35b7e3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157700 0x7fd3540af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35b7e3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd354157ad0 0x7fd3540cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35b7e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540afbd0 con 0x7fd3540b1b10 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35b7e3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540afeb0 con 0x7fd354157ad0 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35b7e3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540b0190 con 0x7fd354157700 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3540b1b10 0x7fd354145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3540b1b10 0x7fd354145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34838/0 (socket says 172.21.15.3:34838) 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 -- 172.21.15.3:0/2265829472 learned_addr learned my addr 172.21.15.3:0/2265829472 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd358d7e700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157700 0x7fd3540af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd359d80700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd354157ad0 0x7fd3540cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 -- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157700 msgr2=0x7fd3540af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157700 0x7fd3540af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 -- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd354157ad0 msgr2=0x7fd3540cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd354157ad0 0x7fd3540cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.382+0000 7fd35957f700 1 -- 172.21.15.3:0/2265829472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009420 con 0x7fd3540b1b10 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.383+0000 7fd35957f700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3540b1b10 0x7fd354145d20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd34400cd00 tx=0x7fd344001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.383+0000 7fd3517fa700 1 -- 172.21.15.3:0/2265829472 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400ab10 con 0x7fd3540b1b10 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.383+0000 7fd3517fa700 1 -- 172.21.15.3:0/2265829472 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34400ac70 con 0x7fd3540b1b10 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.383+0000 7fd3517fa700 1 -- 172.21.15.3:0/2265829472 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3440070f0 con 0x7fd3540b1b10 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.383+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd348009420 con 0x7fd3540b1b10 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.383+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3540b0750 con 0x7fd3540b1b10 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.384+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3540644c0 con 0x7fd3540b1b10 2023-10-06T04:59:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.384+0000 7fd3517fa700 1 -- 172.21.15.3:0/2265829472 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3440074a0 con 0x7fd3540b1b10 2023-10-06T04:59:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.384+0000 7fd3517fa700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.385+0000 7fd3517fa700 1 -- 172.21.15.3:0/2265829472 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd344084b60 con 0x7fd3540b1b10 2023-10-06T04:59:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.385+0000 7fd358d7e700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.385+0000 7fd358d7e700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd340066f30 0x7fd3400693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd3540adc50 tx=0x7fd348017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.389+0000 7fd3517fa700 1 -- 172.21.15.3:0/2265829472 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd344051cf0 con 0x7fd3540b1b10 2023-10-06T04:59:35.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.627+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd354071680 con 0x7fd3540b1b10 2023-10-06T04:59:35.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.628+0000 7fd3517fa700 1 -- 172.21.15.3:0/2265829472 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fd344055310 con 0x7fd3540b1b10 2023-10-06T04:59:35.629 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:35.629 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:30.805835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:35.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.629+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd340066f30 msgr2=0x7fd3400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.629+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd340066f30 0x7fd3400693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd3540adc50 tx=0x7fd348017000 comp rx=0 tx=0).stop 2023-10-06T04:59:35.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.629+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3540b1b10 msgr2=0x7fd354145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:35.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.629+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3540b1b10 0x7fd354145d20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd34400cd00 tx=0x7fd344001b00 comp rx=0 tx=0).stop 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.629+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 shutdown_connections 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.630+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.630+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd354157ad0 0x7fd3540cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.630+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3540b1b10 0x7fd354145d20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.630+0000 7fd35b7e3700 1 --2- 172.21.15.3:0/2265829472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd354157700 0x7fd3540af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.630+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 >> 172.21.15.3:0/2265829472 conn(0x7fd3540ce2c0 msgr2=0x7fd3540b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.630+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 shutdown_connections 2023-10-06T04:59:35.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:35.630+0000 7fd35b7e3700 1 -- 172.21.15.3:0/2265829472 wait complete. 2023-10-06T04:59:35.633 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 61 2023-10-06T04:59:35.643 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T04:59:35.789 INFO:teuthology.orchestra.run.smithi153.stdout:[{"id":6802,"entity":{"name":{"type":"client","num":6802},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2791282291}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":16.324713978999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.904720804099528,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6802 192.168.0.1:0/2791282291","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"101273","root":"/"}},{"id":7248,"entity":{"name":{"type":"client","num":7248},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1770037939}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":907,"uptime":18.119682530999999,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":4470.2845530158038,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":769.72021252061631,"halflife":1.5},"recall_caps_throttle2o":{"value":195.12739853152806,"halflife":0.5},"session_cache_liveness":{"value":1561.3923663697092,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":506,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7248 192.168.0.1:0/1770037939","completed_requests":[{"tid":506,"created_ino":"0x100000003e9"},{"tid":507,"created_ino":"0x100000003ea"},{"tid":508,"created_ino":"0x100000003eb"},{"tid":509,"created_ino":"0x100000003ec"},{"tid":510,"created_ino":"0x100000003ed"},{"tid":511,"created_ino":"0x100000003ee"},{"tid":512,"created_ino":"0x100000003ef"},{"tid":513,"created_ino":"0x100000003f0"},{"tid":514,"created_ino":"0x100000003f1"},{"tid":515,"created_ino":"0x100000003f2"},{"tid":516,"created_ino":"0x100000003f3"},{"tid":517,"created_ino":"0x100000003f4"},{"tid":518,"created_ino":"0x100000003f5"},{"tid":519,"created_ino":"0x100000003f6"},{"tid":520,"created_ino":"0x100000003f7"},{"tid":521,"created_ino":"0x100000003f8"},{"tid":522,"created_ino":"0x100000003f9"},{"tid":523,"created_ino":"0x100000003fa"},{"tid":524,"created_ino":"0x100000003fb"},{"tid":525,"created_ino":"0x100000003fc"},{"tid":526,"created_ino":"0x100000003fd"},{"tid":527,"created_ino":"0x100000003fe"},{"tid":528,"created_ino":"0x100000003ff"},{"tid":529,"created_ino":"0x10000000400"},{"tid":530,"created_ino":"0x10000000401"},{"tid":531,"created_ino":"0x10000000402"},{"tid":532,"created_ino":"0x10000000403"},{"tid":533,"created_ino":"0x10000000404"},{"tid":534,"created_ino":"0x10000000405"},{"tid":535,"created_ino":"0x10000000406"},{"tid":536,"created_ino":"0x10000000407"},{"tid":537,"created_ino":"0x10000000408"},{"tid":538,"created_ino":"0x10000000409"},{"tid":539,"created_ino":"0x1000000040a"},{"tid":540,"created_ino":"0x1000000040b"},{"tid":541,"created_ino":"0x1000000040c"},{"tid":542,"created_ino":"0x1000000040d"},{"tid":543,"created_ino":"0x1000000040e"},{"tid":544,"created_ino":"0x1000000040f"},{"tid":545,"created_ino":"0x10000000410"},{"tid":546,"created_ino":"0x10000000411"},{"tid":547,"created_ino":"0x10000000412"},{"tid":548,"created_ino":"0x10000000413"},{"tid":549,"created_ino":"0x10000000414"},{"tid":550,"created_ino":"0x10000000415"},{"tid":551,"created_ino":"0x10000000416"},{"tid":552,"created_ino":"0x10000000417"},{"tid":553,"created_ino":"0x10000000418"},{"tid":554,"created_ino":"0x10000000419"},{"tid":555,"created_ino":"0x1000000041a"},{"tid":556,"created_ino":"0x1000000041b"},{"tid":557,"created_ino":"0x1000000041c"},{"tid":558,"created_ino":"0x1000000041d"},{"tid":559,"created_ino":"0x1000000041e"},{"tid":560,"created_ino":"0x1000000041f"},{"tid":561,"created_ino":"0x10000000420"},{"tid":562,"created_ino":"0x10000000421"},{"tid":563,"created_ino":"0x10000000422"},{"tid":564,"created_ino":"0x10000000423"},{"tid":565,"created_ino":"0x10000000424"},{"tid":566,"created_ino":"0x10000000425"},{"tid":567,"created_ino":"0x10000000426"},{"tid":568,"created_ino":"0x10000000427"},{"tid":569,"created_ino":"0x10000000428"},{"tid":570,"created_ino":"0x10000000429"},{"tid":571,"created_ino":"0x1000000042a"},{"tid":572,"created_ino":"0x1000000042b"},{"tid":573,"created_ino":"0x1000000042c"},{"tid":574,"created_ino":"0x1000000042d"},{"tid":575,"created_ino":"0x1000000042e"},{"tid":576,"created_ino":"0x1000000042f"},{"tid":577,"created_ino":"0x10000000430"},{"tid":578,"created_ino":"0x10000000431"},{"tid":579,"created_ino":"0x10000000432"},{"tid":580,"created_ino":"0x10000000433"},{"tid":581,"created_ino":"0x10000000434"},{"tid":582,"created_ino":"0x10000000435"},{"tid":583,"created_ino":"0x10000000436"},{"tid":584,"created_ino":"0x10000000437"},{"tid":585,"created_ino":"0x10000000438"},{"tid":586,"created_ino":"0x10000000439"},{"tid":587,"created_ino":"0x1000000043a"},{"tid":588,"created_ino":"0x1000000043b"},{"tid":589,"created_ino":"0x1000000043c"},{"tid":590,"created_ino":"0x1000000043d"},{"tid":591,"created_ino":"0x1000000043e"},{"tid":592,"created_ino":"0x1000000043f"},{"tid":593,"created_ino":"0x10000000440"},{"tid":594,"created_ino":"0x10000000441"},{"tid":595,"created_ino":"0x10000000442"},{"tid":596,"created_ino":"0x10000000443"},{"tid":597,"created_ino":"0x10000000444"},{"tid":598,"created_ino":"0x10000000445"},{"tid":599,"created_ino":"0x10000000446"},{"tid":600,"created_ino":"0x10000000447"},{"tid":601,"created_ino":"0x10000000448"},{"tid":602,"created_ino":"0x10000000449"},{"tid":603,"created_ino":"0x1000000044a"},{"tid":604,"created_ino":"0x1000000044b"},{"tid":605,"created_ino":"0x1000000044c"},{"tid":606,"created_ino":"0x1000000044d"},{"tid":607,"created_ino":"0x1000000044e"},{"tid":608,"created_ino":"0x1000000044f"},{"tid":609,"created_ino":"0x10000000450"},{"tid":610,"created_ino":"0x10000000451"},{"tid":611,"created_ino":"0x10000000452"},{"tid":612,"created_ino":"0x10000000453"},{"tid":613,"created_ino":"0x10000000454"},{"tid":614,"created_ino":"0x10000000455"},{"tid":615,"created_ino":"0x10000000456"},{"tid":616,"created_ino":"0x10000000457"},{"tid":617,"created_ino":"0x10000000458"},{"tid":618,"created_ino":"0x10000000459"},{"tid":619,"created_ino":"0x1000000045a"},{"tid":620,"created_ino":"0x1000000045b"},{"tid":621,"created_ino":"0x1000000045c"},{"tid":622,"created_ino":"0x1000000045d"},{"tid":623,"created_ino":"0x1000000045e"},{"tid":624,"created_ino":"0x1000000045f"},{"tid":625,"created_ino":"0x10000000460"},{"tid":626,"created_ino":"0x10000000461"},{"tid":627,"created_ino":"0x10000000462"},{"tid":628,"created_ino":"0x10000000463"},{"tid":629,"created_ino":"0x10000000464"},{"tid":630,"created_ino":"0x10000000465"},{"tid":631,"created_ino":"0x10000000466"},{"tid":632,"created_ino":"0x10000000467"},{"tid":633,"created_ino":"0x10000000468"},{"tid":634,"created_ino":"0x10000000469"},{"tid":635,"created_ino":"0x1000000046a"},{"tid":636,"created_ino":"0x1000000046b"},{"tid":637,"created_ino":"0x1000000046c"},{"tid":638,"created_ino":"0x1000000046d"},{"tid":639,"created_ino":"0x1000000046e"},{"tid":640,"created_ino":"0x1000000046f"},{"tid":641,"created_ino":"0x10000000470"},{"tid":642,"created_ino":"0x10000000471"},{"tid":643,"created_ino":"0x10000000472"},{"tid":644,"created_ino":"0x10000000473"},{"tid":645,"created_ino":"0x10000000474"},{"tid":646,"created_ino":"0x10000000475"},{"tid":647,"created_ino":"0x10000000476"},{"tid":648,"created_ino":"0x10000000477"},{"tid":649,"created_ino":"0x10000000478"},{"tid":650,"created_ino":"0x10000000479"},{"tid":651,"created_ino":"0x1000000047a"},{"tid":652,"created_ino":"0x1000000047b"},{"tid":653,"created_ino":"0x1000000047c"},{"tid":654,"created_ino":"0x1000000047d"},{"tid":655,"created_ino":"0x1000000047e"},{"tid":656,"created_ino":"0x1000000047f"},{"tid":657,"created_ino":"0x10000000480"},{"tid":658,"created_ino":"0x10000000481"},{"tid":659,"created_ino":"0x10000000482"},{"tid":660,"created_ino":"0x10000000483"},{"tid":661,"created_ino":"0x10000000484"},{"tid":662,"created_ino":"0x10000000485"},{"tid":663,"created_ino":"0x10000000486"},{"tid":664,"created_ino":"0x10000000487"},{"tid":665,"created_ino":"0x10000000488"},{"tid":666,"created_ino":"0x10000000489"},{"tid":667,"created_ino":"0x1000000048a"},{"tid":668,"created_ino":"0x1000000048b"},{"tid":669,"created_ino":"0x1000000048c"},{"tid":670,"created_ino":"0x1000000048d"},{"tid":671,"created_ino":"0x1000000048e"},{"tid":672,"created_ino":"0x1000000048f"},{"tid":673,"created_ino":"0x10000000490"},{"tid":674,"created_ino":"0x10000000491"},{"tid":675,"created_ino":"0x10000000492"},{"tid":676,"created_ino":"0x10000000493"},{"tid":677,"created_ino":"0x10000000494"},{"tid":678,"created_ino":"0x10000000495"},{"tid":679,"created_ino":"0x10000000496"},{"tid":680,"created_ino":"0x10000000497"},{"tid":681,"created_ino":"0x10000000498"},{"tid":682,"created_ino":"0x10000000499"},{"tid":683,"created_ino":"0x1000000049a"},{"tid":684,"created_ino":"0x1000000049b"},{"tid":685,"created_ino":"0x1000000049c"},{"tid":688,"created_ino":"0x1000000049d"},{"tid":692,"created_ino":"0x1000000049e"},{"tid":695,"created_ino":"0x1000000049f"},{"tid":698,"created_ino":"0x100000004a0"},{"tid":701,"created_ino":"0x100000004a1"},{"tid":704,"created_ino":"0x100000004a2"},{"tid":707,"created_ino":"0x100000004a3"},{"tid":710,"created_ino":"0x100000004a4"},{"tid":713,"created_ino":"0x100000004a5"},{"tid":716,"created_ino":"0x100000004a6"},{"tid":719,"created_ino":"0x100000004a7"},{"tid":722,"created_ino":"0x100000004a8"},{"tid":725,"created_ino":"0x100000004a9"},{"tid":728,"created_ino":"0x100000004aa"},{"tid":731,"created_ino":"0x100000004ab"},{"tid":734,"created_ino":"0x100000004ac"},{"tid":737,"created_ino":"0x100000004ad"},{"tid":740,"created_ino":"0x100000004ae"},{"tid":743,"created_ino":"0x100000004af"},{"tid":746,"created_ino":"0x100000004b0"},{"tid":749,"created_ino":"0x100000004b1"},{"tid":752,"created_ino":"0x100000004b2"},{"tid":755,"created_ino":"0x100000004b3"},{"tid":758,"created_ino":"0x100000004b4"},{"tid":761,"created_ino":"0x100000004b5"},{"tid":764,"created_ino":"0x100000004b6"},{"tid":767,"created_ino":"0x100000004b7"},{"tid":770,"created_ino":"0x100000004b8"},{"tid":773,"created_ino":"0x100000004b9"},{"tid":776,"created_ino":"0x100000004ba"},{"tid":779,"created_ino":"0x100000004bb"},{"tid":782,"created_ino":"0x100000004bc"},{"tid":785,"created_ino":"0x100000004bd"},{"tid":788,"created_ino":"0x100000004be"},{"tid":791,"created_ino":"0x100000004bf"},{"tid":794,"created_ino":"0x100000004c0"},{"tid":797,"created_ino":"0x100000004c1"},{"tid":800,"created_ino":"0x100000004c2"},{"tid":803,"created_ino":"0x100000004c3"},{"tid":806,"created_ino":"0x100000004c4"},{"tid":809,"created_ino":"0x100000004c5"},{"tid":812,"created_ino":"0x100000004c6"},{"tid":815,"created_ino":"0x100000004c7"},{"tid":818,"created_ino":"0x100000004c8"},{"tid":821,"created_ino":"0x100000004c9"},{"tid":824,"created_ino":"0x100000004ca"},{"tid":827,"created_ino":"0x100000004cb"},{"tid":830,"created_ino":"0x100000004cc"},{"tid":833,"created_ino":"0x100000004cd"},{"tid":836,"created_ino":"0x100000004ce"},{"tid":839,"created_ino":"0x100000004cf"},{"tid":842,"created_ino":"0x100000004d0"},{"tid":845,"created_ino":"0x100000004d1"},{"tid":848,"created_ino":"0x100000004d2"},{"tid":851,"created_ino":"0x100000004d3"},{"tid":854,"created_ino":"0x100000004d4"},{"tid":857,"created_ino":"0x100000004d5"},{"tid":860,"created_ino":"0x100000004d6"},{"tid":863,"created_ino":"0x100000004d7"},{"tid":866,"created_ino":"0x100000004d8"},{"tid":869,"created_ino":"0x100000004d9"},{"tid":872,"created_ino":"0x100000004da"},{"tid":875,"created_ino":"0x100000004db"},{"tid":878,"created_ino":"0x100000004dc"},{"tid":881,"created_ino":"0x100000004dd"},{"tid":884,"created_ino":"0x100000004de"},{"tid":887,"created_ino":"0x100000004df"},{"tid":890,"created_ino":"0x100000004e0"},{"tid":893,"created_ino":"0x100000004e1"},{"tid":896,"created_ino":"0x100000004e2"},{"tid":899,"created_ino":"0x100000004e3"},{"tid":902,"created_ino":"0x100000004e4"},{"tid":905,"created_ino":"0x100000004e5"},{"tid":908,"created_ino":"0x100000004e6"},{"tid":911,"created_ino":"0x100000004e7"},{"tid":914,"created_ino":"0x100000004e8"},{"tid":917,"created_ino":"0x100000004e9"},{"tid":920,"created_ino":"0x100000004ea"},{"tid":923,"created_ino":"0x100000004eb"},{"tid":926,"created_ino":"0x100000004ec"},{"tid":929,"created_ino":"0x100000004ed"},{"tid":932,"created_ino":"0x100000004ee"},{"tid":935,"created_ino":"0x100000004ef"},{"tid":938,"created_ino":"0x100000004f0"},{"tid":941,"created_ino":"0x100000004f1"},{"tid":944,"created_ino":"0x100000004f2"},{"tid":947,"created_ino":"0x100000004f3"},{"tid":950,"created_ino":"0x100000004f4"},{"tid":953,"created_ino":"0x100000004f5"},{"tid":956,"created_ino":"0x100000004f6"},{"tid":959,"created_ino":"0x100000004f7"},{"tid":962,"created_ino":"0x100000004f8"},{"tid":965,"created_ino":"0x100000004f9"},{"tid":968,"created_ino":"0x100000004fa"},{"tid":971,"created_ino":"0x100000004fb"},{"tid":974,"created_ino":"0x100000004fc"},{"tid":977,"created_ino":"0x100000004fd"},{"tid":980,"created_ino":"0x100000004fe"},{"tid":983,"created_ino":"0x100000004ff"},{"tid":986,"created_ino":"0x10000000500"},{"tid":989,"created_ino":"0x10000000501"},{"tid":992,"created_ino":"0x10000000502"},{"tid":995,"created_ino":"0x10000000503"},{"tid":998,"created_ino":"0x10000000504"},{"tid":1001,"created_ino":"0x10000000505"},{"tid":1004,"created_ino":"0x10000000506"},{"tid":1007,"created_ino":"0x10000000507"},{"tid":1010,"created_ino":"0x10000000508"},{"tid":1013,"created_ino":"0x10000000509"},{"tid":1016,"created_ino":"0x1000000050a"},{"tid":1019,"created_ino":"0x1000000050b"},{"tid":1022,"created_ino":"0x1000000050c"},{"tid":1025,"created_ino":"0x1000000050d"},{"tid":1028,"created_ino":"0x1000000050e"},{"tid":1031,"created_ino":"0x1000000050f"},{"tid":1034,"created_ino":"0x10000000510"},{"tid":1037,"created_ino":"0x10000000511"},{"tid":1040,"created_ino":"0x10000000512"},{"tid":1043,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"120433","root":"/"}}] 2023-10-06T04:59:35.794 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "101273", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2791282291, "type": "any" }, "name": { "num": 6802, "type": "client" } }, "id": 6802, "inst": "client.6802 192.168.0.1:0/2791282291", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.904720804099528 }, "state": "open", "uptime": 16.324713979 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "120433", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 506 }, { "created_ino": "0x100000003ea", "tid": 507 }, { "created_ino": "0x100000003eb", "tid": 508 }, { "created_ino": "0x100000003ec", "tid": 509 }, { "created_ino": "0x100000003ed", "tid": 510 }, { "created_ino": "0x100000003ee", "tid": 511 }, { "created_ino": "0x100000003ef", "tid": 512 }, { "created_ino": "0x100000003f0", "tid": 513 }, { "created_ino": "0x100000003f1", "tid": 514 }, { "created_ino": "0x100000003f2", "tid": 515 }, { "created_ino": "0x100000003f3", "tid": 516 }, { "created_ino": "0x100000003f4", "tid": 517 }, { "created_ino": "0x100000003f5", "tid": 518 }, { "created_ino": "0x100000003f6", "tid": 519 }, { "created_ino": "0x100000003f7", "tid": 520 }, { "created_ino": "0x100000003f8", "tid": 521 }, { "created_ino": "0x100000003f9", "tid": 522 }, { "created_ino": "0x100000003fa", "tid": 523 }, { "created_ino": "0x100000003fb", "tid": 524 }, { "created_ino": "0x100000003fc", "tid": 525 }, { "created_ino": "0x100000003fd", "tid": 526 }, { "created_ino": "0x100000003fe", "tid": 527 }, { "created_ino": "0x100000003ff", "tid": 528 }, { "created_ino": "0x10000000400", "tid": 529 }, { "created_ino": "0x10000000401", "tid": 530 }, { "created_ino": "0x10000000402", "tid": 531 }, { "created_ino": "0x10000000403", "tid": 532 }, { "created_ino": "0x10000000404", "tid": 533 }, { "created_ino": "0x10000000405", "tid": 534 }, { "created_ino": "0x10000000406", "tid": 535 }, { "created_ino": "0x10000000407", "tid": 536 }, { "created_ino": "0x10000000408", "tid": 537 }, { "created_ino": "0x10000000409", "tid": 538 }, { "created_ino": "0x1000000040a", "tid": 539 }, { "created_ino": "0x1000000040b", "tid": 540 }, { "created_ino": "0x1000000040c", "tid": 541 }, { "created_ino": "0x1000000040d", "tid": 542 }, { "created_ino": "0x1000000040e", "tid": 543 }, { "created_ino": "0x1000000040f", "tid": 544 }, { "created_ino": "0x10000000410", "tid": 545 }, { "created_ino": "0x10000000411", "tid": 546 }, { "created_ino": "0x10000000412", "tid": 547 }, { "created_ino": "0x10000000413", "tid": 548 }, { "created_ino": "0x10000000414", "tid": 549 }, { "created_ino": "0x10000000415", "tid": 550 }, { "created_ino": "0x10000000416", "tid": 551 }, { "created_ino": "0x10000000417", "tid": 552 }, { "created_ino": "0x10000000418", "tid": 553 }, { "created_ino": "0x10000000419", "tid": 554 }, { "created_ino": "0x1000000041a", "tid": 555 }, { "created_ino": "0x1000000041b", "tid": 556 }, { "created_ino": "0x1000000041c", "tid": 557 }, { "created_ino": "0x1000000041d", "tid": 558 }, { "created_ino": "0x1000000041e", "tid": 559 }, { "created_ino": "0x1000000041f", "tid": 560 }, { "created_ino": "0x10000000420", "tid": 561 }, { "created_ino": "0x10000000421", "tid": 562 }, { "created_ino": "0x10000000422", "tid": 563 }, { "created_ino": "0x10000000423", "tid": 564 }, { "created_ino": "0x10000000424", "tid": 565 }, { "created_ino": "0x10000000425", "tid": 566 }, { "created_ino": "0x10000000426", "tid": 567 }, { "created_ino": "0x10000000427", "tid": 568 }, { "created_ino": "0x10000000428", "tid": 569 }, { "created_ino": "0x10000000429", "tid": 570 }, { "created_ino": "0x1000000042a", "tid": 571 }, { "created_ino": "0x1000000042b", "tid": 572 }, { "created_ino": "0x1000000042c", "tid": 573 }, { "created_ino": "0x1000000042d", "tid": 574 }, { "created_ino": "0x1000000042e", "tid": 575 }, { "created_ino": "0x1000000042f", "tid": 576 }, { "created_ino": "0x10000000430", "tid": 577 }, { "created_ino": "0x10000000431", "tid": 578 }, { "created_ino": "0x10000000432", "tid": 579 }, { "created_ino": "0x10000000433", "tid": 580 }, { "created_ino": "0x10000000434", "tid": 581 }, { "created_ino": "0x10000000435", "tid": 582 }, { "created_ino": "0x10000000436", "tid": 583 }, { "created_ino": "0x10000000437", "tid": 584 }, { "created_ino": "0x10000000438", "tid": 585 }, { "created_ino": "0x10000000439", "tid": 586 }, { "created_ino": "0x1000000043a", "tid": 587 }, { "created_ino": "0x1000000043b", "tid": 588 }, { "created_ino": "0x1000000043c", "tid": 589 }, { "created_ino": "0x1000000043d", "tid": 590 }, { "created_ino": "0x1000000043e", "tid": 591 }, { "created_ino": "0x1000000043f", "tid": 592 }, { "created_ino": "0x10000000440", "tid": 593 }, { "created_ino": "0x10000000441", "tid": 594 }, { "created_ino": "0x10000000442", "tid": 595 }, { "created_ino": "0x10000000443", "tid": 596 }, { "created_ino": "0x10000000444", "tid": 597 }, { "created_ino": "0x10000000445", "tid": 598 }, { "created_ino": "0x10000000446", "tid": 599 }, { "created_ino": "0x10000000447", "tid": 600 }, { "created_ino": "0x10000000448", "tid": 601 }, { "created_ino": "0x10000000449", "tid": 602 }, { "created_ino": "0x1000000044a", "tid": 603 }, { "created_ino": "0x1000000044b", "tid": 604 }, { "created_ino": "0x1000000044c", "tid": 605 }, { "created_ino": "0x1000000044d", "tid": 606 }, { "created_ino": "0x1000000044e", "tid": 607 }, { "created_ino": "0x1000000044f", "tid": 608 }, { "created_ino": "0x10000000450", "tid": 609 }, { "created_ino": "0x10000000451", "tid": 610 }, { "created_ino": "0x10000000452", "tid": 611 }, { "created_ino": "0x10000000453", "tid": 612 }, { "created_ino": "0x10000000454", "tid": 613 }, { "created_ino": "0x10000000455", "tid": 614 }, { "created_ino": "0x10000000456", "tid": 615 }, { "created_ino": "0x10000000457", "tid": 616 }, { "created_ino": "0x10000000458", "tid": 617 }, { "created_ino": "0x10000000459", "tid": 618 }, { "created_ino": "0x1000000045a", "tid": 619 }, { "created_ino": "0x1000000045b", "tid": 620 }, { "created_ino": "0x1000000045c", "tid": 621 }, { "created_ino": "0x1000000045d", "tid": 622 }, { "created_ino": "0x1000000045e", "tid": 623 }, { "created_ino": "0x1000000045f", "tid": 624 }, { "created_ino": "0x10000000460", "tid": 625 }, { "created_ino": "0x10000000461", "tid": 626 }, { "created_ino": "0x10000000462", "tid": 627 }, { "created_ino": "0x10000000463", "tid": 628 }, { "created_ino": "0x10000000464", "tid": 629 }, { "created_ino": "0x10000000465", "tid": 630 }, { "created_ino": "0x10000000466", "tid": 631 }, { "created_ino": "0x10000000467", "tid": 632 }, { "created_ino": "0x10000000468", "tid": 633 }, { "created_ino": "0x10000000469", "tid": 634 }, { "created_ino": "0x1000000046a", "tid": 635 }, { "created_ino": "0x1000000046b", "tid": 636 }, { "created_ino": "0x1000000046c", "tid": 637 }, { "created_ino": "0x1000000046d", "tid": 638 }, { "created_ino": "0x1000000046e", "tid": 639 }, { "created_ino": "0x1000000046f", "tid": 640 }, { "created_ino": "0x10000000470", "tid": 641 }, { "created_ino": "0x10000000471", "tid": 642 }, { "created_ino": "0x10000000472", "tid": 643 }, { "created_ino": "0x10000000473", "tid": 644 }, { "created_ino": "0x10000000474", "tid": 645 }, { "created_ino": "0x10000000475", "tid": 646 }, { "created_ino": "0x10000000476", "tid": 647 }, { "created_ino": "0x10000000477", "tid": 648 }, { "created_ino": "0x10000000478", "tid": 649 }, { "created_ino": "0x10000000479", "tid": 650 }, { "created_ino": "0x1000000047a", "tid": 651 }, { "created_ino": "0x1000000047b", "tid": 652 }, { "created_ino": "0x1000000047c", "tid": 653 }, { "created_ino": "0x1000000047d", "tid": 654 }, { "created_ino": "0x1000000047e", "tid": 655 }, { "created_ino": "0x1000000047f", "tid": 656 }, { "created_ino": "0x10000000480", "tid": 657 }, { "created_ino": "0x10000000481", "tid": 658 }, { "created_ino": "0x10000000482", "tid": 659 }, { "created_ino": "0x10000000483", "tid": 660 }, { "created_ino": "0x10000000484", "tid": 661 }, { "created_ino": "0x10000000485", "tid": 662 }, { "created_ino": "0x10000000486", "tid": 663 }, { "created_ino": "0x10000000487", "tid": 664 }, { "created_ino": "0x10000000488", "tid": 665 }, { "created_ino": "0x10000000489", "tid": 666 }, { "created_ino": "0x1000000048a", "tid": 667 }, { "created_ino": "0x1000000048b", "tid": 668 }, { "created_ino": "0x1000000048c", "tid": 669 }, { "created_ino": "0x1000000048d", "tid": 670 }, { "created_ino": "0x1000000048e", "tid": 671 }, { "created_ino": "0x1000000048f", "tid": 672 }, { "created_ino": "0x10000000490", "tid": 673 }, { "created_ino": "0x10000000491", "tid": 674 }, { "created_ino": "0x10000000492", "tid": 675 }, { "created_ino": "0x10000000493", "tid": 676 }, { "created_ino": "0x10000000494", "tid": 677 }, { "created_ino": "0x10000000495", "tid": 678 }, { "created_ino": "0x10000000496", "tid": 679 }, { "created_ino": "0x10000000497", "tid": 680 }, { "created_ino": "0x10000000498", "tid": 681 }, { "created_ino": "0x10000000499", "tid": 682 }, { "created_ino": "0x1000000049a", "tid": 683 }, { "created_ino": "0x1000000049b", "tid": 684 }, { "created_ino": "0x1000000049c", "tid": 685 }, { "created_ino": "0x1000000049d", "tid": 688 }, { "created_ino": "0x1000000049e", "tid": 692 }, { "created_ino": "0x1000000049f", "tid": 695 }, { "created_ino": "0x100000004a0", "tid": 698 }, { "created_ino": "0x100000004a1", "tid": 701 }, { "created_ino": "0x100000004a2", "tid": 704 }, { "created_ino": "0x100000004a3", "tid": 707 }, { "created_ino": "0x100000004a4", "tid": 710 }, { "created_ino": "0x100000004a5", "tid": 713 }, { "created_ino": "0x100000004a6", "tid": 716 }, { "created_ino": "0x100000004a7", "tid": 719 }, { "created_ino": "0x100000004a8", "tid": 722 }, { "created_ino": "0x100000004a9", "tid": 725 }, { "created_ino": "0x100000004aa", "tid": 728 }, { "created_ino": "0x100000004ab", "tid": 731 }, { "created_ino": "0x100000004ac", "tid": 734 }, { "created_ino": "0x100000004ad", "tid": 737 }, { "created_ino": "0x100000004ae", "tid": 740 }, { "created_ino": "0x100000004af", "tid": 743 }, { "created_ino": "0x100000004b0", "tid": 746 }, { "created_ino": "0x100000004b1", "tid": 749 }, { "created_ino": "0x100000004b2", "tid": 752 }, { "created_ino": "0x100000004b3", "tid": 755 }, { "created_ino": "0x100000004b4", "tid": 758 }, { "created_ino": "0x100000004b5", "tid": 761 }, { "created_ino": "0x100000004b6", "tid": 764 }, { "created_ino": "0x100000004b7", "tid": 767 }, { "created_ino": "0x100000004b8", "tid": 770 }, { "created_ino": "0x100000004b9", "tid": 773 }, { "created_ino": "0x100000004ba", "tid": 776 }, { "created_ino": "0x100000004bb", "tid": 779 }, { "created_ino": "0x100000004bc", "tid": 782 }, { "created_ino": "0x100000004bd", "tid": 785 }, { "created_ino": "0x100000004be", "tid": 788 }, { "created_ino": "0x100000004bf", "tid": 791 }, { "created_ino": "0x100000004c0", "tid": 794 }, { "created_ino": "0x100000004c1", "tid": 797 }, { "created_ino": "0x100000004c2", "tid": 800 }, { "created_ino": "0x100000004c3", "tid": 803 }, { "created_ino": "0x100000004c4", "tid": 806 }, { "created_ino": "0x100000004c5", "tid": 809 }, { "created_ino": "0x100000004c6", "tid": 812 }, { "created_ino": "0x100000004c7", "tid": 815 }, { "created_ino": "0x100000004c8", "tid": 818 }, { "created_ino": "0x100000004c9", "tid": 821 }, { "created_ino": "0x100000004ca", "tid": 824 }, { "created_ino": "0x100000004cb", "tid": 827 }, { "created_ino": "0x100000004cc", "tid": 830 }, { "created_ino": "0x100000004cd", "tid": 833 }, { "created_ino": "0x100000004ce", "tid": 836 }, { "created_ino": "0x100000004cf", "tid": 839 }, { "created_ino": "0x100000004d0", "tid": 842 }, { "created_ino": "0x100000004d1", "tid": 845 }, { "created_ino": "0x100000004d2", "tid": 848 }, { "created_ino": "0x100000004d3", "tid": 851 }, { "created_ino": "0x100000004d4", "tid": 854 }, { "created_ino": "0x100000004d5", "tid": 857 }, { "created_ino": "0x100000004d6", "tid": 860 }, { "created_ino": "0x100000004d7", "tid": 863 }, { "created_ino": "0x100000004d8", "tid": 866 }, { "created_ino": "0x100000004d9", "tid": 869 }, { "created_ino": "0x100000004da", "tid": 872 }, { "created_ino": "0x100000004db", "tid": 875 }, { "created_ino": "0x100000004dc", "tid": 878 }, { "created_ino": "0x100000004dd", "tid": 881 }, { "created_ino": "0x100000004de", "tid": 884 }, { "created_ino": "0x100000004df", "tid": 887 }, { "created_ino": "0x100000004e0", "tid": 890 }, { "created_ino": "0x100000004e1", "tid": 893 }, { "created_ino": "0x100000004e2", "tid": 896 }, { "created_ino": "0x100000004e3", "tid": 899 }, { "created_ino": "0x100000004e4", "tid": 902 }, { "created_ino": "0x100000004e5", "tid": 905 }, { "created_ino": "0x100000004e6", "tid": 908 }, { "created_ino": "0x100000004e7", "tid": 911 }, { "created_ino": "0x100000004e8", "tid": 914 }, { "created_ino": "0x100000004e9", "tid": 917 }, { "created_ino": "0x100000004ea", "tid": 920 }, { "created_ino": "0x100000004eb", "tid": 923 }, { "created_ino": "0x100000004ec", "tid": 926 }, { "created_ino": "0x100000004ed", "tid": 929 }, { "created_ino": "0x100000004ee", "tid": 932 }, { "created_ino": "0x100000004ef", "tid": 935 }, { "created_ino": "0x100000004f0", "tid": 938 }, { "created_ino": "0x100000004f1", "tid": 941 }, { "created_ino": "0x100000004f2", "tid": 944 }, { "created_ino": "0x100000004f3", "tid": 947 }, { "created_ino": "0x100000004f4", "tid": 950 }, { "created_ino": "0x100000004f5", "tid": 953 }, { "created_ino": "0x100000004f6", "tid": 956 }, { "created_ino": "0x100000004f7", "tid": 959 }, { "created_ino": "0x100000004f8", "tid": 962 }, { "created_ino": "0x100000004f9", "tid": 965 }, { "created_ino": "0x100000004fa", "tid": 968 }, { "created_ino": "0x100000004fb", "tid": 971 }, { "created_ino": "0x100000004fc", "tid": 974 }, { "created_ino": "0x100000004fd", "tid": 977 }, { "created_ino": "0x100000004fe", "tid": 980 }, { "created_ino": "0x100000004ff", "tid": 983 }, { "created_ino": "0x10000000500", "tid": 986 }, { "created_ino": "0x10000000501", "tid": 989 }, { "created_ino": "0x10000000502", "tid": 992 }, { "created_ino": "0x10000000503", "tid": 995 }, { "created_ino": "0x10000000504", "tid": 998 }, { "created_ino": "0x10000000505", "tid": 1001 }, { "created_ino": "0x10000000506", "tid": 1004 }, { "created_ino": "0x10000000507", "tid": 1007 }, { "created_ino": "0x10000000508", "tid": 1010 }, { "created_ino": "0x10000000509", "tid": 1013 }, { "created_ino": "0x1000000050a", "tid": 1016 }, { "created_ino": "0x1000000050b", "tid": 1019 }, { "created_ino": "0x1000000050c", "tid": 1022 }, { "created_ino": "0x1000000050d", "tid": 1025 }, { "created_ino": "0x1000000050e", "tid": 1028 }, { "created_ino": "0x1000000050f", "tid": 1031 }, { "created_ino": "0x10000000510", "tid": 1034 }, { "created_ino": "0x10000000511", "tid": 1037 }, { "created_ino": "0x10000000512", "tid": 1040 }, { "created_ino": "0x10000000513", "tid": 1043 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1770037939, "type": "any" }, "name": { "num": 7248, "type": "client" } }, "id": 7248, "inst": "client.7248 192.168.0.1:0/1770037939", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 506, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 4470.284553015804 }, "recall_caps_throttle": { "halflife": 1.5, "value": 769.7202125206163 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 195.12739853152806 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 907, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1561.3923663697092 }, "state": "open", "uptime": 18.119682531 } ] 2023-10-06T04:59:35.797 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T04:59:35.797 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:59:35.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:59:35.838 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:59:35.882 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:59:35.882 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T04:59:35.882 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T04:59:35.883 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:59:35.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:59:35.921 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[120433]: fuse finished with error 0 and tester_r 0 2023-10-06T04:59:35.923 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:59:35.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:35.924 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:59:36.104 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T04:59:36.105 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:59:36.105 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:59:36.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[101273]: fuse finished with error 0 and tester_r 0 2023-10-06T04:59:36.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:59:36.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:36.139 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:59:36.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T04:59:36.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:36.306 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:59:36.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:59:36.322 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T04:59:36.323 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T04:59:36.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:59:36.323 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:59:36.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:59:36.361 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T04:59:36.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.480+0000 7f2e54f2a700 1 Processor -- start 2023-10-06T04:59:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.480+0000 7f2e54f2a700 1 -- start start 2023-10-06T04:59:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.481+0000 7f2e54f2a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2e501550f0 0x7f2e5006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.481+0000 7f2e54f2a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e500adcb0 con 0x7f2e50157de0 2023-10-06T04:59:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.481+0000 7f2e54f2a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e500ad980 con 0x7f2e50157960 2023-10-06T04:59:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.481+0000 7f2e54f2a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e501582d0 con 0x7f2e501550f0 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.481+0000 7f2e4ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2e50157de0 0x7f2e500ae140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57912/0 (socket says 172.21.15.3:57912) 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.481+0000 7f2e4ed9d700 1 -- 172.21.15.3:0/917654173 learned_addr learned my addr 172.21.15.3:0/917654173 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.481+0000 7f2e4e59c700 1 --2- 172.21.15.3:0/917654173 >> v2:172.21.15.153:3300/0 conn(0x7f2e501550f0 0x7f2e5006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4d59a700 1 -- 172.21.15.3:0/917654173 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431840484 0 0) 0x7f2e500adcb0 con 0x7f2e50157de0 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4d59a700 1 -- 172.21.15.3:0/917654173 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e38003570 con 0x7f2e50157de0 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4d59a700 1 -- 172.21.15.3:0/917654173 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976731945 0 0) 0x7f2e500ad980 con 0x7f2e50157960 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4d59a700 1 -- 172.21.15.3:0/917654173 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e500adcb0 con 0x7f2e50157960 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4e59c700 1 -- 172.21.15.3:0/917654173 >> v1:172.21.15.153:6789/0 conn(0x7f2e50157960 legacy=0x7f2e50158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4e59c700 1 -- 172.21.15.3:0/917654173 >> v1:172.21.15.3:6789/0 conn(0x7f2e50157de0 legacy=0x7f2e500ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4e59c700 1 -- 172.21.15.3:0/917654173 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e501585b0 con 0x7f2e501550f0 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4e59c700 1 --2- 172.21.15.3:0/917654173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e5006eff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e480098e0 tx=0x7f2e480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e686507d5e06f9ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4d59a700 1 -- 172.21.15.3:0/917654173 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48011630 con 0x7f2e501550f0 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4d59a700 1 -- 172.21.15.3:0/917654173 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e48011790 con 0x7f2e501550f0 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.482+0000 7f2e4d59a700 1 -- 172.21.15.3:0/917654173 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48011b30 con 0x7f2e501550f0 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.483+0000 7f2e54f2a700 1 -- 172.21.15.3:0/917654173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 msgr2=0x7f2e5006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.483+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/917654173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e5006eff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e480098e0 tx=0x7f2e480091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.483+0000 7f2e54f2a700 1 -- 172.21.15.3:0/917654173 shutdown_connections 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.483+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/917654173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e5006eff0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.483+0000 7f2e54f2a700 1 -- 172.21.15.3:0/917654173 >> 172.21.15.3:0/917654173 conn(0x7f2e5006c370 msgr2=0x7f2e5006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:36.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.483+0000 7f2e54f2a700 1 -- 172.21.15.3:0/917654173 shutdown_connections 2023-10-06T04:59:36.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.483+0000 7f2e54f2a700 1 -- 172.21.15.3:0/917654173 wait complete. 2023-10-06T04:59:36.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.484+0000 7f2e54f2a700 1 Processor -- start 2023-10-06T04:59:36.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.484+0000 7f2e54f2a700 1 -- start start 2023-10-06T04:59:36.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e50146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e50157960 0x7f2e50150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e50157de0 0x7f2e5015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e501596f0 con 0x7f2e50157960 2023-10-06T04:59:36.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e50146aa0 con 0x7f2e501550f0 2023-10-06T04:59:36.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e50158ae0 con 0x7f2e50157de0 2023-10-06T04:59:36.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e50157960 0x7f2e50150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e50157960 0x7f2e50150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34844/0 (socket says 172.21.15.3:34844) 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 -- 172.21.15.3:0/2642290886 learned_addr learned my addr 172.21.15.3:0/2642290886 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4e59c700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e50146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4ed9d700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e50157de0 0x7f2e5015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 -- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e50157de0 msgr2=0x7f2e5015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e50157de0 0x7f2e5015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 -- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 msgr2=0x7f2e50146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e50146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 -- 172.21.15.3:0/2642290886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48009420 con 0x7f2e50157960 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4e59c700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e50146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e4dd9b700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e50157960 0x7f2e50150440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400c790 tx=0x7f2e44007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e3e7fc700 1 -- 172.21.15.3:0/2642290886 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44010510 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e3e7fc700 1 -- 172.21.15.3:0/2642290886 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e44010670 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e50158dc0 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e3e7fc700 1 -- 172.21.15.3:0/2642290886 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44010a10 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.485+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e50159380 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.486+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e50159860 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.487+0000 7f2e3e7fc700 1 -- 172.21.15.3:0/2642290886 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e4401b400 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.487+0000 7f2e3e7fc700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.487+0000 7f2e3e7fc700 1 -- 172.21.15.3:0/2642290886 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2e44084460 con 0x7f2e50157960 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.487+0000 7f2e4e59c700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.488+0000 7f2e4e59c700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2e480093f0 tx=0x7f2e48017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:36.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.491+0000 7f2e3e7fc700 1 -- 172.21.15.3:0/2642290886 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e44051570 con 0x7f2e50157960 2023-10-06T04:59:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.729+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e50071680 con 0x7f2e50157960 2023-10-06T04:59:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.730+0000 7f2e3e7fc700 1 -- 172.21.15.3:0/2642290886 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f2e44054b90 con 0x7f2e50157960 2023-10-06T04:59:36.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:36.731 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:30.805835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 msgr2=0x7f2e300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2e480093f0 tx=0x7f2e48017000 comp rx=0 tx=0).stop 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e50157960 msgr2=0x7f2e50150440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e50157960 0x7f2e50150440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400c790 tx=0x7f2e44007e10 comp rx=0 tx=0).stop 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 shutdown_connections 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e501550f0 0x7f2e50146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e50157de0 0x7f2e5015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.731+0000 7f2e54f2a700 1 --2- 172.21.15.3:0/2642290886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e50157960 0x7f2e50150440 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.732+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 >> 172.21.15.3:0/2642290886 conn(0x7f2e5006c370 msgr2=0x7f2e5006a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.732+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 shutdown_connections 2023-10-06T04:59:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.732+0000 7f2e54f2a700 1 -- 172.21.15.3:0/2642290886 wait complete. 2023-10-06T04:59:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 61 2023-10-06T04:59:36.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:36.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.863+0000 7fa7fb10b700 1 Processor -- start 2023-10-06T04:59:36.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.863+0000 7fa7fb10b700 1 -- start start 2023-10-06T04:59:36.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fb10b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7f41593d0 0x7fa7f40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fb10b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa7f41597a0 0x7fa7f40c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fb10b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fb10b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40cc910 con 0x7fa7f41593d0 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fb10b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40cbd40 con 0x7fa7f41597a0 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fb10b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40cbfe0 con 0x7fa7f40ccdb0 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7f41593d0 0x7fa7f40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7f41593d0 0x7fa7f40c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34860/0 (socket says 172.21.15.3:34860) 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 -- 172.21.15.3:0/1983107502 learned_addr learned my addr 172.21.15.3:0/1983107502 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa90a700 1 --2- 172.21.15.3:0/1983107502 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7f9908700 1 --2- 172.21.15.3:0/1983107502 >> v2:172.21.15.153:3300/0 conn(0x7fa7f41597a0 0x7fa7f40c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 -- 172.21.15.3:0/1983107502 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 msgr2=0x7fa7f40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 --2- 172.21.15.3:0/1983107502 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f40cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 -- 172.21.15.3:0/1983107502 >> v2:172.21.15.153:3300/0 conn(0x7fa7f41597a0 msgr2=0x7fa7f40c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:36.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 --2- 172.21.15.3:0/1983107502 >> v2:172.21.15.153:3300/0 conn(0x7fa7f41597a0 0x7fa7f40c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa109700 1 -- 172.21.15.3:0/1983107502 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40cc2c0 con 0x7fa7f41593d0 2023-10-06T04:59:36.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.864+0000 7fa7fa90a700 1 --2- 172.21.15.3:0/1983107502 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:36.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fa109700 1 --2- 172.21.15.3:0/1983107502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c5bd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4089860 tx=0x7fa7ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ea6864e730a710 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:36.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7f9107700 1 -- 172.21.15.3:0/1983107502 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec009e70 con 0x7fa7f41593d0 2023-10-06T04:59:36.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7f9107700 1 -- 172.21.15.3:0/1983107502 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec00b860 con 0x7fa7f41593d0 2023-10-06T04:59:36.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7f9107700 1 -- 172.21.15.3:0/1983107502 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00bc00 con 0x7fa7f41593d0 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 -- 172.21.15.3:0/1983107502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 msgr2=0x7fa7f40c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/1983107502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c5bd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4089860 tx=0x7fa7ec009250 comp rx=0 tx=0).stop 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 -- 172.21.15.3:0/1983107502 shutdown_connections 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/1983107502 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/1983107502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c5bd0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/1983107502 >> v2:172.21.15.153:3300/0 conn(0x7fa7f41597a0 0x7fa7f40c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 -- 172.21.15.3:0/1983107502 >> 172.21.15.3:0/1983107502 conn(0x7fa7f4089df0 msgr2=0x7fa7f4099710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:36.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 -- 172.21.15.3:0/1983107502 shutdown_connections 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.865+0000 7fa7fb10b700 1 -- 172.21.15.3:0/1983107502 wait complete. 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 Processor -- start 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 -- start start 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f4153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f41597a0 0x7fa7f40c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40c7bb0 con 0x7fa7f41593d0 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40c7e90 con 0x7fa7f41597a0 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7fb10b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40c8170 con 0x7fa7f40ccdb0 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7f9908700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7f9908700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34868/0 (socket says 172.21.15.3:34868) 2023-10-06T04:59:36.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.867+0000 7fa7f9908700 1 -- 172.21.15.3:0/3569155767 learned_addr learned my addr 172.21.15.3:0/3569155767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7fa109700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f4153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7fa90a700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f41597a0 0x7fa7f40c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f9908700 1 -- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 msgr2=0x7fa7f4153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f9908700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f4153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f9908700 1 -- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f41597a0 msgr2=0x7fa7f40c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f9908700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f41597a0 0x7fa7f40c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f9908700 1 -- 172.21.15.3:0/3569155767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0094f0 con 0x7fa7f41593d0 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f9908700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c86d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ec80 tx=0x7fa7e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f27fc700 1 -- 172.21.15.3:0/3569155767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800c850 con 0x7fa7f41593d0 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f27fc700 1 -- 172.21.15.3:0/3569155767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e800c9b0 con 0x7fa7f41593d0 2023-10-06T04:59:36.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f40c8450 con 0x7fa7f41593d0 2023-10-06T04:59:36.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7f27fc700 1 -- 172.21.15.3:0/3569155767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800cd50 con 0x7fa7f41593d0 2023-10-06T04:59:36.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.868+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f40c78c0 con 0x7fa7f41593d0 2023-10-06T04:59:36.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.869+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f409ca90 con 0x7fa7f41593d0 2023-10-06T04:59:36.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.870+0000 7fa7f27fc700 1 -- 172.21.15.3:0/3569155767 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7e8013680 con 0x7fa7f41593d0 2023-10-06T04:59:36.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.870+0000 7fa7f27fc700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa7e0066f30 0x7fa7e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:36.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.870+0000 7fa7f27fc700 1 -- 172.21.15.3:0/3569155767 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa7e80848c0 con 0x7fa7f41593d0 2023-10-06T04:59:36.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.870+0000 7fa7fa109700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa7e0066f30 0x7fa7e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:36.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.871+0000 7fa7fa109700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa7e0066f30 0x7fa7e00693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4089860 tx=0x7fa7ec017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:36.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:36.874+0000 7fa7f27fc700 1 -- 172.21.15.3:0/3569155767 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7f409ca90 con 0x7fa7f41593d0 2023-10-06T04:59:37.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.072+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7f409ca90 con 0x7fa7f41593d0 2023-10-06T04:59:37.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.073+0000 7fa7f27fc700 1 -- 172.21.15.3:0/3569155767 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16821 (secure 0 0 0) 0x7fa7f409ca90 con 0x7fa7f41593d0 2023-10-06T04:59:37.075 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:37.075 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:07.019162+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:03.167920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:04.391498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa7e0066f30 msgr2=0x7fa7e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa7e0066f30 0x7fa7e00693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4089860 tx=0x7fa7ec017000 comp rx=0 tx=0).stop 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 msgr2=0x7fa7f40c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c86d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ec80 tx=0x7fa7e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 shutdown_connections 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa7e0066f30 0x7fa7e00693e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f41597a0 0x7fa7f40c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7f40ccdb0 0x7fa7f4153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 --2- 172.21.15.3:0/3569155767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7f41593d0 0x7fa7f40c86d0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 >> 172.21.15.3:0/3569155767 conn(0x7fa7f4089df0 msgr2=0x7fa7f4158410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 shutdown_connections 2023-10-06T04:59:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.075+0000 7fa7fb10b700 1 -- 172.21.15.3:0/3569155767 wait complete. 2023-10-06T04:59:37.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:59:03.167920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:59:04.391498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:59:37.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:59:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.210+0000 7f44f789c700 1 Processor -- start 2023-10-06T04:59:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.210+0000 7f44f789c700 1 -- start start 2023-10-06T04:59:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f789c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0150690 0x7f44f0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f789c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f00cba50 0x7f44f015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f789c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f015da80 con 0x7f44f015a720 2023-10-06T04:59:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f789c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f015a290 con 0x7f44f00cba50 2023-10-06T04:59:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f789c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f00d0020 con 0x7f44f0150690 2023-10-06T04:59:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f00cba50 0x7f44f015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f00cba50 0x7f44f015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34880/0 (socket says 172.21.15.3:34880) 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 -- 172.21.15.3:0/2983925263 learned_addr learned my addr 172.21.15.3:0/2983925263 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f5638700 1 --2- 172.21.15.3:0/2983925263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0150690 0x7f44f0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 -- 172.21.15.3:0/2983925263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0150690 msgr2=0x7f44f0150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 --2- 172.21.15.3:0/2983925263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0150690 0x7f44f0150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 -- 172.21.15.3:0/2983925263 >> v1:172.21.15.153:6789/0 conn(0x7f44f015a720 legacy=0x7f44f00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 -- 172.21.15.3:0/2983925263 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f0159f40 con 0x7f44f00cba50 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.211+0000 7f44f4e37700 1 --2- 172.21.15.3:0/2983925263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f015d540 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f44e40099b0 tx=0x7f44e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa72c79a0165acb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44effff700 1 -- 172.21.15.3:0/2983925263 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4009ea0 con 0x7f44f00cba50 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44effff700 1 -- 172.21.15.3:0/2983925263 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e400b860 con 0x7f44f00cba50 2023-10-06T04:59:37.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44effff700 1 -- 172.21.15.3:0/2983925263 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400bc00 con 0x7f44f00cba50 2023-10-06T04:59:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 -- 172.21.15.3:0/2983925263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 msgr2=0x7f44f015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 --2- 172.21.15.3:0/2983925263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f015d540 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f44e40099b0 tx=0x7f44e4009250 comp rx=0 tx=0).stop 2023-10-06T04:59:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 -- 172.21.15.3:0/2983925263 shutdown_connections 2023-10-06T04:59:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 --2- 172.21.15.3:0/2983925263 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f0150690 0x7f44f0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 --2- 172.21.15.3:0/2983925263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f015d540 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 -- 172.21.15.3:0/2983925263 >> 172.21.15.3:0/2983925263 conn(0x7f44f0070310 msgr2=0x7f44f0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 -- 172.21.15.3:0/2983925263 shutdown_connections 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.212+0000 7f44f789c700 1 -- 172.21.15.3:0/2983925263 wait complete. 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.213+0000 7f44f789c700 1 Processor -- start 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.213+0000 7f44f789c700 1 -- start start 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f789c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f00b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f789c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f0150690 0x7f44f014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f789c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f015a720 0x7f44f0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f789c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f015c9e0 con 0x7f44f00cba50 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f789c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f00b2240 con 0x7f44f0150690 2023-10-06T04:59:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f789c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f00b40f0 con 0x7f44f015a720 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f5638700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f00b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f5638700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f00b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34894/0 (socket says 172.21.15.3:34894) 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f5638700 1 -- 172.21.15.3:0/952099152 learned_addr learned my addr 172.21.15.3:0/952099152 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f4e37700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f0150690 0x7f44f014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.214+0000 7f44f5e39700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f015a720 0x7f44f0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f5638700 1 -- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f015a720 msgr2=0x7f44f0150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f5638700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f015a720 0x7f44f0150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f5638700 1 -- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f0150690 msgr2=0x7f44f014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f5638700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f0150690 0x7f44f014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f5638700 1 -- 172.21.15.3:0/952099152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40094f0 con 0x7f44f00cba50 2023-10-06T04:59:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f5e39700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f015a720 0x7f44f0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f5638700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f00b1d00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f44e800ec80 tx=0x7f44e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44ed7fa700 1 -- 172.21.15.3:0/952099152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800c850 con 0x7f44f00cba50 2023-10-06T04:59:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44ed7fa700 1 -- 172.21.15.3:0/952099152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e800c9b0 con 0x7f44f00cba50 2023-10-06T04:59:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f015c1d0 con 0x7f44f00cba50 2023-10-06T04:59:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44ed7fa700 1 -- 172.21.15.3:0/952099152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800cd50 con 0x7f44f00cba50 2023-10-06T04:59:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.215+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f00b4300 con 0x7f44f00cba50 2023-10-06T04:59:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.216+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f004c380 con 0x7f44f00cba50 2023-10-06T04:59:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.216+0000 7f44ed7fa700 1 -- 172.21.15.3:0/952099152 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44e8013680 con 0x7f44f00cba50 2023-10-06T04:59:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.217+0000 7f44ed7fa700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f60 0x7f44dc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.217+0000 7f44ed7fa700 1 -- 172.21.15.3:0/952099152 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44e8084930 con 0x7f44f00cba50 2023-10-06T04:59:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.217+0000 7f44f4e37700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f60 0x7f44dc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.217+0000 7f44f4e37700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f60 0x7f44dc069410 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44e40099b0 tx=0x7f44e4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.221+0000 7f44ed7fa700 1 -- 172.21.15.3:0/952099152 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44e8051a40 con 0x7f44f00cba50 2023-10-06T04:59:37.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.414+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44f015dcb0 con 0x7f44f00cba50 2023-10-06T04:59:37.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.414+0000 7f44ed7fa700 1 -- 172.21.15.3:0/952099152 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7f44e8055060 con 0x7f44f00cba50 2023-10-06T04:59:37.416 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:59:37.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f60 msgr2=0x7f44dc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f60 0x7f44dc069410 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44e40099b0 tx=0x7f44e4017000 comp rx=0 tx=0).stop 2023-10-06T04:59:37.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 msgr2=0x7f44f00b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f00b1d00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f44e800ec80 tx=0x7f44e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 shutdown_connections 2023-10-06T04:59:37.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f44dc066f60 0x7f44dc069410 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44f015a720 0x7f44f0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f00cba50 0x7f44f00b1d00 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 --2- 172.21.15.3:0/952099152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f0150690 0x7f44f014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 >> 172.21.15.3:0/952099152 conn(0x7f44f0070310 msgr2=0x7f44f014fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:37.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 shutdown_connections 2023-10-06T04:59:37.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.416+0000 7f44f789c700 1 -- 172.21.15.3:0/952099152 wait complete. 2023-10-06T04:59:37.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:59:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.550+0000 7f774b59e700 1 Processor -- start 2023-10-06T04:59:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.550+0000 7f774b59e700 1 -- start start 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774b59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f774c1591e0 0x7f774c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774b59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f774c0aece0 0x7f774c0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774b59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774c06ca80 con 0x7f774c1595b0 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774b59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774c0af120 con 0x7f774c0aece0 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774b59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774c15d560 con 0x7f774c1591e0 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f774c0aece0 0x7f774c0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f7749d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f774c1595b0 0x7f774c06c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57918/0 (socket says 172.21.15.3:57918) 2023-10-06T04:59:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f7749d9b700 1 -- 172.21.15.3:0/4246732201 learned_addr learned my addr 172.21.15.3:0/4246732201 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774a59c700 1 --2- 172.21.15.3:0/4246732201 >> v2:172.21.15.153:3300/0 conn(0x7f774c1591e0 0x7f774c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774ad9d700 1 -- 172.21.15.3:0/4246732201 >> v2:172.21.15.153:3300/0 conn(0x7f774c1591e0 msgr2=0x7f774c15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774ad9d700 1 --2- 172.21.15.3:0/4246732201 >> v2:172.21.15.153:3300/0 conn(0x7f774c1591e0 0x7f774c15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774ad9d700 1 -- 172.21.15.3:0/4246732201 >> v1:172.21.15.3:6789/0 conn(0x7f774c1595b0 legacy=0x7f774c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774959a700 1 -- 172.21.15.3:0/4246732201 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481500214 0 0) 0x7f774c06ca80 con 0x7f774c1595b0 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774ad9d700 1 -- 172.21.15.3:0/4246732201 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c15d840 con 0x7f774c0aece0 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.551+0000 7f774a59c700 1 --2- 172.21.15.3:0/4246732201 >> v2:172.21.15.153:3300/0 conn(0x7f774c1591e0 0x7f774c15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774ad9d700 1 --2- 172.21.15.3:0/4246732201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c0aece0 0x7f774c0b1660 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f774c069f20 tx=0x7f773c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fc6736f9cac854 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774959a700 1 -- 172.21.15.3:0/4246732201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c00b6b0 con 0x7f774c0aece0 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774959a700 1 -- 172.21.15.3:0/4246732201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c00b810 con 0x7f774c0aece0 2023-10-06T04:59:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774959a700 1 -- 172.21.15.3:0/4246732201 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c00bbb0 con 0x7f774c0aece0 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 -- 172.21.15.3:0/4246732201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c0aece0 msgr2=0x7f774c0b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 --2- 172.21.15.3:0/4246732201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c0aece0 0x7f774c0b1660 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f774c069f20 tx=0x7f773c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 -- 172.21.15.3:0/4246732201 shutdown_connections 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 --2- 172.21.15.3:0/4246732201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c0aece0 0x7f774c0b1660 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 --2- 172.21.15.3:0/4246732201 >> v2:172.21.15.153:3300/0 conn(0x7f774c1591e0 0x7f774c15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 -- 172.21.15.3:0/4246732201 >> 172.21.15.3:0/4246732201 conn(0x7f774c066fb0 msgr2=0x7f774c0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 -- 172.21.15.3:0/4246732201 shutdown_connections 2023-10-06T04:59:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.552+0000 7f774b59e700 1 -- 172.21.15.3:0/4246732201 wait complete. 2023-10-06T04:59:37.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.553+0000 7f774b59e700 1 Processor -- start 2023-10-06T04:59:37.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774b59e700 1 -- start start 2023-10-06T04:59:37.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774b59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f774c0aece0 0x7f774c15c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c1591e0 0x7f774c16a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774b59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f774c1595b0 0x7f774c1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774b59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c155d70 con 0x7f774c1591e0 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774b59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c154ea0 con 0x7f774c0aece0 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774b59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774c16a760 con 0x7f774c1595b0 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f7749d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c1591e0 0x7f774c16a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f7749d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c1591e0 0x7f774c16a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34912/0 (socket says 172.21.15.3:34912) 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f7749d9b700 1 -- 172.21.15.3:0/3652803252 learned_addr learned my addr 172.21.15.3:0/3652803252 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.554+0000 7f774a59c700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f774c0aece0 0x7f774c15c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f774ad9d700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f774c1595b0 0x7f774c1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f7749d9b700 1 -- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f774c1595b0 msgr2=0x7f774c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f7749d9b700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f774c1595b0 0x7f774c1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f7749d9b700 1 -- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f774c0aece0 msgr2=0x7f774c15c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f7749d9b700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f774c0aece0 0x7f774c15c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f7749d9b700 1 -- 172.21.15.3:0/3652803252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c009420 con 0x7f774c1591e0 2023-10-06T04:59:37.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f774a59c700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f774c0aece0 0x7f774c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:37.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f7749d9b700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c1591e0 0x7f774c16a220 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f773400c810 tx=0x7f7734007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f77427fc700 1 -- 172.21.15.3:0/3652803252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734005c00 con 0x7f774c1591e0 2023-10-06T04:59:37.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f77427fc700 1 -- 172.21.15.3:0/3652803252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7734005d60 con 0x7f774c1591e0 2023-10-06T04:59:37.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c1550f0 con 0x7f774c1591e0 2023-10-06T04:59:37.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f77427fc700 1 -- 172.21.15.3:0/3652803252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734007320 con 0x7f774c1591e0 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.555+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c1556b0 con 0x7f774c1591e0 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.556+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774c155eb0 con 0x7f774c1591e0 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.556+0000 7f77427fc700 1 -- 172.21.15.3:0/3652803252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f773401b400 con 0x7f774c1591e0 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.557+0000 7f77427fc700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f772c066f30 0x7f772c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.557+0000 7f77427fc700 1 -- 172.21.15.3:0/3652803252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7734084170 con 0x7f774c1591e0 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.557+0000 7f774a59c700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f772c066f30 0x7f772c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.557+0000 7f774a59c700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f772c066f30 0x7f772c0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f774c151890 tx=0x7f7744009870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.561+0000 7f77427fc700 1 -- 172.21.15.3:0/3652803252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f774c155eb0 con 0x7f774c1591e0 2023-10-06T04:59:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.753+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f774c155eb0 con 0x7f774c1591e0 2023-10-06T04:59:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.754+0000 7f77427fc700 1 -- 172.21.15.3:0/3652803252 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f774c155eb0 con 0x7f774c1591e0 2023-10-06T04:59:37.755 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:59:37.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f772c066f30 msgr2=0x7f772c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f772c066f30 0x7f772c0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f774c151890 tx=0x7f7744009870 comp rx=0 tx=0).stop 2023-10-06T04:59:37.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c1591e0 msgr2=0x7f774c16a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c1591e0 0x7f774c16a220 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f773400c810 tx=0x7f7734007db0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 shutdown_connections 2023-10-06T04:59:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f772c066f30 0x7f772c0693e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f774c0aece0 0x7f774c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f774c1595b0 0x7f774c1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 --2- 172.21.15.3:0/3652803252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f774c1591e0 0x7f774c16a220 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 >> 172.21.15.3:0/3652803252 conn(0x7f774c066fb0 msgr2=0x7f774c0b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 shutdown_connections 2023-10-06T04:59:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.756+0000 7f774b59e700 1 -- 172.21.15.3:0/3652803252 wait complete. 2023-10-06T04:59:37.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:59:37.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.893+0000 7f34d8842700 1 Processor -- start 2023-10-06T04:59:37.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.893+0000 7f34d8842700 1 -- start start 2023-10-06T04:59:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.893+0000 7f34d8842700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34d40af8a0 0x7f34d4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d8842700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d415a290 con 0x7f34d4150690 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d8842700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d40d0020 con 0x7f34d40cba50 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d8842700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d4159f40 con 0x7f34d40af8a0 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34d40af8a0 0x7f34d4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f34d4150690 0x7f34d415a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57932/0 (socket says 172.21.15.3:57932) 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d259c700 1 -- 172.21.15.3:0/119899168 learned_addr learned my addr 172.21.15.3:0/119899168 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d2d9d700 1 -- 172.21.15.3:0/119899168 >> v1:172.21.15.153:6789/0 conn(0x7f34d40cba50 legacy=0x7f34d40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d2d9d700 1 -- 172.21.15.3:0/119899168 >> v1:172.21.15.3:6789/0 conn(0x7f34d4150690 legacy=0x7f34d415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d159a700 1 -- 172.21.15.3:0/119899168 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916747862 0 0) 0x7f34d415a290 con 0x7f34d4150690 2023-10-06T04:59:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d2d9d700 1 -- 172.21.15.3:0/119899168 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d415dcb0 con 0x7f34d40af8a0 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d2d9d700 1 --2- 172.21.15.3:0/119899168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d4150a60 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009e60 tx=0x7f34c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6ef37ab4adc421 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d159a700 1 -- 172.21.15.3:0/119899168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c400e070 con 0x7f34d40af8a0 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d159a700 1 -- 172.21.15.3:0/119899168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c400bbf0 con 0x7f34d40af8a0 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.894+0000 7f34d159a700 1 -- 172.21.15.3:0/119899168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4012430 con 0x7f34d40af8a0 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.895+0000 7f34d8842700 1 -- 172.21.15.3:0/119899168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 msgr2=0x7f34d4150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.895+0000 7f34d8842700 1 --2- 172.21.15.3:0/119899168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d4150a60 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009e60 tx=0x7f34c4009780 comp rx=0 tx=0).stop 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.895+0000 7f34d8842700 1 -- 172.21.15.3:0/119899168 shutdown_connections 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.895+0000 7f34d8842700 1 --2- 172.21.15.3:0/119899168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d4150a60 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.895+0000 7f34d8842700 1 -- 172.21.15.3:0/119899168 >> 172.21.15.3:0/119899168 conn(0x7f34d4070310 msgr2=0x7f34d4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.895+0000 7f34d8842700 1 -- 172.21.15.3:0/119899168 shutdown_connections 2023-10-06T04:59:37.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.895+0000 7f34d8842700 1 -- 172.21.15.3:0/119899168 wait complete. 2023-10-06T04:59:37.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.896+0000 7f34d8842700 1 Processor -- start 2023-10-06T04:59:37.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.896+0000 7f34d8842700 1 -- start start 2023-10-06T04:59:37.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d8842700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d40c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d8842700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34d40cba50 0x7f34d415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d8842700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34d4150690 0x7f34d415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d8842700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40b43d0 con 0x7f34d40af8a0 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d8842700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40ca1f0 con 0x7f34d40cba50 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d8842700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40b3aa0 con 0x7f34d4150690 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d40c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d40c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34930/0 (socket says 172.21.15.3:34930) 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 -- 172.21.15.3:0/1384455287 learned_addr learned my addr 172.21.15.3:0/1384455287 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d2d9d700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34d4150690 0x7f34d415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d1d9b700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34d40cba50 0x7f34d415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 -- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34d4150690 msgr2=0x7f34d415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34d4150690 0x7f34d415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 -- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34d40cba50 msgr2=0x7f34d415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34d40cba50 0x7f34d415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 -- 172.21.15.3:0/1384455287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4009a20 con 0x7f34d40af8a0 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d259c700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d40c98a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00c5b0 tx=0x7f34cc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34ca7fc700 1 -- 172.21.15.3:0/1384455287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc005b30 con 0x7f34d40af8a0 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34ca7fc700 1 -- 172.21.15.3:0/1384455287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc005c90 con 0x7f34d40af8a0 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d40b3d80 con 0x7f34d40af8a0 2023-10-06T04:59:37.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.897+0000 7f34ca7fc700 1 -- 172.21.15.3:0/1384455287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc007320 con 0x7f34d40af8a0 2023-10-06T04:59:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.898+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d40c9de0 con 0x7f34d40af8a0 2023-10-06T04:59:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.898+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d404c380 con 0x7f34d40af8a0 2023-10-06T04:59:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.899+0000 7f34ca7fc700 1 -- 172.21.15.3:0/1384455287 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34cc01b400 con 0x7f34d40af8a0 2023-10-06T04:59:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.899+0000 7f34ca7fc700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34b4066f30 0x7f34b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.899+0000 7f34ca7fc700 1 -- 172.21.15.3:0/1384455287 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f34cc084110 con 0x7f34d40af8a0 2023-10-06T04:59:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.899+0000 7f34d1d9b700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34b4066f30 0x7f34b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:37.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.900+0000 7f34d1d9b700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34b4066f30 0x7f34b40693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f34d40c6c80 tx=0x7f34bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:37.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:37.904+0000 7f34ca7fc700 1 -- 172.21.15.3:0/1384455287 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34cc0512a0 con 0x7f34d40af8a0 2023-10-06T04:59:38.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.096+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34d415df90 con 0x7f34d40af8a0 2023-10-06T04:59:38.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.097+0000 7f34ca7fc700 1 -- 172.21.15.3:0/1384455287 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34cc0548c0 con 0x7f34d40af8a0 2023-10-06T04:59:38.098 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34b4066f30 msgr2=0x7f34b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34b4066f30 0x7f34b40693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f34d40c6c80 tx=0x7f34bc009040 comp rx=0 tx=0).stop 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 msgr2=0x7f34d40c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d40c98a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00c5b0 tx=0x7f34cc001070 comp rx=0 tx=0).stop 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 shutdown_connections 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f34b4066f30 0x7f34b40693e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34d4150690 0x7f34d415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d40af8a0 0x7f34d40c98a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 --2- 172.21.15.3:0/1384455287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34d40cba50 0x7f34d415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 >> 172.21.15.3:0/1384455287 conn(0x7f34d4070310 msgr2=0x7f34d40cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 shutdown_connections 2023-10-06T04:59:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.099+0000 7f34d8842700 1 -- 172.21.15.3:0/1384455287 wait complete. 2023-10-06T04:59:38.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:59:38.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.232+0000 7f53ac3fd700 1 Processor -- start 2023-10-06T04:59:38.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.232+0000 7f53ac3fd700 1 -- start start 2023-10-06T04:59:38.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.233+0000 7f53ac3fd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f53a4150690 0x7f53a4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.233+0000 7f53ac3fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53a415d540 0x7f53a40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.233+0000 7f53ac3fd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a415a290 con 0x7f53a40cba50 2023-10-06T04:59:38.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.233+0000 7f53ac3fd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a40d0020 con 0x7f53a415d540 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.233+0000 7f53ac3fd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a4159f40 con 0x7f53a4150690 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.233+0000 7f53aa99a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53a415d540 0x7f53a40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.233+0000 7f53aa99a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53a415d540 0x7f53a40af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34936/0 (socket says 172.21.15.3:34936) 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53aa99a700 1 -- 172.21.15.3:0/170451035 learned_addr learned my addr 172.21.15.3:0/170451035 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53aa199700 1 --2- 172.21.15.3:0/170451035 >> v2:172.21.15.153:3300/0 conn(0x7f53a4150690 0x7f53a4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53aa99a700 1 -- 172.21.15.3:0/170451035 >> v2:172.21.15.153:3300/0 conn(0x7f53a4150690 msgr2=0x7f53a4150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53aa99a700 1 --2- 172.21.15.3:0/170451035 >> v2:172.21.15.153:3300/0 conn(0x7f53a4150690 0x7f53a4150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53aa99a700 1 -- 172.21.15.3:0/170451035 >> v1:172.21.15.153:6789/0 conn(0x7f53a40cba50 legacy=0x7f53a415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53aa99a700 1 -- 172.21.15.3:0/170451035 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a415dcb0 con 0x7f53a415d540 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53aa99a700 1 --2- 172.21.15.3:0/170451035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40af190 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f53a40cdf80 tx=0x7f5394009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817bbd89bfc4769b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53a9197700 1 -- 172.21.15.3:0/170451035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400e070 con 0x7f53a415d540 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53a9197700 1 -- 172.21.15.3:0/170451035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400ba20 con 0x7f53a415d540 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.234+0000 7f53a9197700 1 -- 172.21.15.3:0/170451035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53940092e0 con 0x7f53a415d540 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 -- 172.21.15.3:0/170451035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 msgr2=0x7f53a40af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/170451035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40af190 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f53a40cdf80 tx=0x7f5394009840 comp rx=0 tx=0).stop 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 -- 172.21.15.3:0/170451035 shutdown_connections 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/170451035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40af190 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/170451035 >> v2:172.21.15.153:3300/0 conn(0x7f53a4150690 0x7f53a4150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 -- 172.21.15.3:0/170451035 >> 172.21.15.3:0/170451035 conn(0x7f53a4070310 msgr2=0x7f53a4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 -- 172.21.15.3:0/170451035 shutdown_connections 2023-10-06T04:59:38.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.235+0000 7f53ac3fd700 1 -- 172.21.15.3:0/170451035 wait complete. 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.236+0000 7f53ac3fd700 1 Processor -- start 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.236+0000 7f53ac3fd700 1 -- start start 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53ac3fd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a40cba50 0x7f53a40b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53ac3fd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a4150690 0x7f53a414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53ac3fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53ac3fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4153a90 con 0x7f53a415d540 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53ac3fd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4152e10 con 0x7f53a40cba50 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53ac3fd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40b2d40 con 0x7f53a4150690 2023-10-06T04:59:38.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34940/0 (socket says 172.21.15.3:34940) 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 -- 172.21.15.3:0/3572550937 learned_addr learned my addr 172.21.15.3:0/3572550937 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa199700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a40cba50 0x7f53a40b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53a9998700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a4150690 0x7f53a414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 -- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a4150690 msgr2=0x7f53a414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a4150690 0x7f53a414fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 -- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a40cba50 msgr2=0x7f53a40b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a40cba50 0x7f53a40b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 -- 172.21.15.3:0/3572550937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5394009ae0 con 0x7f53a415d540 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa199700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a40cba50 0x7f53a40b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53aa99a700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40b4a20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f53a40c71f0 tx=0x7f5394018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53a27fc700 1 -- 172.21.15.3:0/3572550937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53940089b0 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53a27fc700 1 -- 172.21.15.3:0/3572550937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5394008b10 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.237+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a41524e0 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.238+0000 7f53a27fc700 1 -- 172.21.15.3:0/3572550937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394025460 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.238+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a4152aa0 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.238+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a404c380 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.239+0000 7f53a27fc700 1 -- 172.21.15.3:0/3572550937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5394003040 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.239+0000 7f53a27fc700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5390066f30 0x7f53900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.240+0000 7f53a27fc700 1 -- 172.21.15.3:0/3572550937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5394086160 con 0x7f53a415d540 2023-10-06T04:59:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.240+0000 7f53aa199700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5390066f30 0x7f53900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.240+0000 7f53aa199700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5390066f30 0x7f53900693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f53a40ca110 tx=0x7f539c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.244+0000 7f53a27fc700 1 -- 172.21.15.3:0/3572550937 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5394053320 con 0x7f53a415d540 2023-10-06T04:59:38.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.436+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53a4152f50 con 0x7f53a415d540 2023-10-06T04:59:38.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.436+0000 7f53a27fc700 1 -- 172.21.15.3:0/3572550937 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5394056940 con 0x7f53a415d540 2023-10-06T04:59:38.438 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T04:59:38.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.438+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5390066f30 msgr2=0x7f53900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.438+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5390066f30 0x7f53900693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f53a40ca110 tx=0x7f539c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 msgr2=0x7f53a40b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40b4a20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f53a40c71f0 tx=0x7f5394018000 comp rx=0 tx=0).stop 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 shutdown_connections 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5390066f30 0x7f53900693e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a40cba50 0x7f53a40b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a415d540 0x7f53a40b4a20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 --2- 172.21.15.3:0/3572550937 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a4150690 0x7f53a414fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 >> 172.21.15.3:0/3572550937 conn(0x7f53a4070310 msgr2=0x7f53a40c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 shutdown_connections 2023-10-06T04:59:38.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.439+0000 7f53ac3fd700 1 -- 172.21.15.3:0/3572550937 wait complete. 2023-10-06T04:59:38.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.571+0000 7f42cb1e1700 1 Processor -- start 2023-10-06T04:59:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.571+0000 7f42cb1e1700 1 -- start start 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.571+0000 7f42cb1e1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f42c406a2d0 0x7f42c40aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.571+0000 7f42cb1e1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c4155cf0 0x7f42c4156170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.571+0000 7f42cb1e1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c40af2b0 con 0x7f42c406a6a0 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.571+0000 7f42cb1e1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c406c410 con 0x7f42c406a2d0 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.571+0000 7f42cb1e1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c406c6f0 con 0x7f42c4155cf0 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f42c406a2d0 0x7f42c40aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c977e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c4155cf0 0x7f42c4156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f42c406a2d0 0x7f42c40aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:59168/0 (socket says 172.21.15.3:59168) 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c977e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c4155cf0 0x7f42c4156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:53938/0 (socket says 172.21.15.3:53938) 2023-10-06T04:59:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 -- 172.21.15.3:0/3657460999 learned_addr learned my addr 172.21.15.3:0/3657460999 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c3fff700 1 --1- 172.21.15.3:0/3657460999 >> v1:172.21.15.153:6789/0 conn(0x7f42c406a6a0 0x7f42c415a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:54438/0 (socket says 172.21.15.3:54438) 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 -- 172.21.15.3:0/3657460999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c4155cf0 msgr2=0x7f42c4156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 --2- 172.21.15.3:0/3657460999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c4155cf0 0x7f42c4156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 -- 172.21.15.3:0/3657460999 >> v1:172.21.15.153:6789/0 conn(0x7f42c406a6a0 legacy=0x7f42c415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 -- 172.21.15.3:0/3657460999 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c406c9d0 con 0x7f42c406a2d0 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c37fe700 1 -- 172.21.15.3:0/3657460999 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703583663 0 0) 0x7f42c40af2b0 con 0x7f42c406a6a0 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c8f7d700 1 --2- 172.21.15.3:0/3657460999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a2d0 0x7f42c40aecd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009f50 tx=0x7f42bc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832d8efc3b589af1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c37fe700 1 -- 172.21.15.3:0/3657460999 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00e070 con 0x7f42c406a2d0 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c37fe700 1 -- 172.21.15.3:0/3657460999 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc00ba50 con 0x7f42c406a2d0 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.572+0000 7f42c37fe700 1 -- 172.21.15.3:0/3657460999 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0092e0 con 0x7f42c406a2d0 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 -- 172.21.15.3:0/3657460999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a2d0 msgr2=0x7f42c40aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/3657460999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a2d0 0x7f42c40aecd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009f50 tx=0x7f42bc009840 comp rx=0 tx=0).stop 2023-10-06T04:59:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 -- 172.21.15.3:0/3657460999 shutdown_connections 2023-10-06T04:59:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/3657460999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c4155cf0 0x7f42c4156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/3657460999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a2d0 0x7f42c40aecd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 -- 172.21.15.3:0/3657460999 >> 172.21.15.3:0/3657460999 conn(0x7f42c4066fb0 msgr2=0x7f42c40673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 -- 172.21.15.3:0/3657460999 shutdown_connections 2023-10-06T04:59:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.573+0000 7f42cb1e1700 1 -- 172.21.15.3:0/3657460999 wait complete. 2023-10-06T04:59:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.574+0000 7f42cb1e1700 1 Processor -- start 2023-10-06T04:59:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.574+0000 7f42cb1e1700 1 -- start start 2023-10-06T04:59:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42cb1e1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c406a2d0 0x7f42c4167870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42cb1e1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a6a0 0x7f42c4169590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42cb1e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4155cf0 0x7f42c41670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42cb1e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4169ad0 con 0x7f42c4155cf0 2023-10-06T04:59:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42cb1e1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40c26a0 con 0x7f42c406a6a0 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42cb1e1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4167610 con 0x7f42c406a2d0 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42c977e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4155cf0 0x7f42c41670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42c977e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4155cf0 0x7f42c41670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34946/0 (socket says 172.21.15.3:34946) 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42c977e700 1 -- 172.21.15.3:0/67712674 learned_addr learned my addr 172.21.15.3:0/67712674 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42c3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a6a0 0x7f42c4169590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.575+0000 7f42c8f7d700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c406a2d0 0x7f42c4167870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c977e700 1 -- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c406a2d0 msgr2=0x7f42c4167870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c977e700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c406a2d0 0x7f42c4167870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c977e700 1 -- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a6a0 msgr2=0x7f42c4169590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c977e700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a6a0 0x7f42c4169590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c977e700 1 -- 172.21.15.3:0/67712674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc009ae0 con 0x7f42c4155cf0 2023-10-06T04:59:38.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c8f7d700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c406a2d0 0x7f42c4167870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:38.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c3fff700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a6a0 0x7f42c4169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:38.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c977e700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4155cf0 0x7f42c41670d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f42b4010ae0 tx=0x7f42b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c0ff9700 1 -- 172.21.15.3:0/67712674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b400ea60 con 0x7f42c4155cf0 2023-10-06T04:59:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c0ff9700 1 -- 172.21.15.3:0/67712674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b400ebc0 con 0x7f42c4155cf0 2023-10-06T04:59:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c40c17b0 con 0x7f42c4155cf0 2023-10-06T04:59:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42c0ff9700 1 -- 172.21.15.3:0/67712674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4011450 con 0x7f42c4155cf0 2023-10-06T04:59:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.576+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c40c1d70 con 0x7f42c4155cf0 2023-10-06T04:59:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.577+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c40c2850 con 0x7f42c4155cf0 2023-10-06T04:59:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.577+0000 7f42c0ff9700 1 -- 172.21.15.3:0/67712674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42b4011880 con 0x7f42c4155cf0 2023-10-06T04:59:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.578+0000 7f42c0ff9700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f42b0066f30 0x7f42b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.578+0000 7f42c0ff9700 1 -- 172.21.15.3:0/67712674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f42b4086780 con 0x7f42c4155cf0 2023-10-06T04:59:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.578+0000 7f42c8f7d700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f42b0066f30 0x7f42b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.579+0000 7f42c8f7d700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f42b0066f30 0x7f42b00693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009880 tx=0x7f42bc018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.582+0000 7f42c0ff9700 1 -- 172.21.15.3:0/67712674 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42c40c2850 con 0x7f42c4155cf0 2023-10-06T04:59:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.818+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42c40c2810 con 0x7f42c4155cf0 2023-10-06T04:59:38.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.819+0000 7f42c0ff9700 1 -- 172.21.15.3:0/67712674 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f42b40537f0 con 0x7f42c4155cf0 2023-10-06T04:59:38.821 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:38.821 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:30.805835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:38.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f42b0066f30 msgr2=0x7f42b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f42b0066f30 0x7f42b00693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009880 tx=0x7f42bc018000 comp rx=0 tx=0).stop 2023-10-06T04:59:38.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4155cf0 msgr2=0x7f42c41670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4155cf0 0x7f42c41670d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f42b4010ae0 tx=0x7f42b40091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 shutdown_connections 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f42b0066f30 0x7f42b00693e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4155cf0 0x7f42c41670d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42c406a2d0 0x7f42c4167870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 --2- 172.21.15.3:0/67712674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42c406a6a0 0x7f42c4169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 >> 172.21.15.3:0/67712674 conn(0x7f42c4066fb0 msgr2=0x7f42c4152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 shutdown_connections 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.821+0000 7f42cb1e1700 1 -- 172.21.15.3:0/67712674 wait complete. 2023-10-06T04:59:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 61 2023-10-06T04:59:38.835 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:38.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.953+0000 7f355edcf700 1 Processor -- start 2023-10-06T04:59:38.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.953+0000 7f355edcf700 1 -- start start 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.953+0000 7f355edcf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558155980 0x7f355806aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f355edcf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3558156a30 0x7f355806e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f355edcf700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3558069790 con 0x7f3558156ea0 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f355edcf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35580ae9e0 con 0x7f3558156a30 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f355edcf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c91a0 con 0x7f3558155980 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3558156a30 0x7f355806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3558156a30 0x7f355806e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34950/0 (socket says 172.21.15.3:34950) 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 -- 172.21.15.3:0/855900177 learned_addr learned my addr 172.21.15.3:0/855900177 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:38.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f355cb6b700 1 --2- 172.21.15.3:0/855900177 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558155980 0x7f355806aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 -- 172.21.15.3:0/855900177 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558155980 msgr2=0x7f355806aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 --2- 172.21.15.3:0/855900177 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558155980 0x7f355806aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 -- 172.21.15.3:0/855900177 >> v1:172.21.15.153:6789/0 conn(0x7f3558156ea0 legacy=0x7f35580c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 -- 172.21.15.3:0/855900177 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580699e0 con 0x7f3558156a30 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f355cb6b700 1 --2- 172.21.15.3:0/855900177 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558155980 0x7f355806aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f3557fff700 1 --2- 172.21.15.3:0/855900177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558156a30 0x7f355806e530 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f354800d1d0 tx=0x7f354800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3614294672f3bca6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f35577fe700 1 -- 172.21.15.3:0/855900177 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548010070 con 0x7f3558156a30 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f35577fe700 1 -- 172.21.15.3:0/855900177 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354800de70 con 0x7f3558156a30 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.954+0000 7f35577fe700 1 -- 172.21.15.3:0/855900177 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354800b300 con 0x7f3558156a30 2023-10-06T04:59:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 -- 172.21.15.3:0/855900177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558156a30 msgr2=0x7f355806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 --2- 172.21.15.3:0/855900177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558156a30 0x7f355806e530 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f354800d1d0 tx=0x7f354800ba50 comp rx=0 tx=0).stop 2023-10-06T04:59:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 -- 172.21.15.3:0/855900177 shutdown_connections 2023-10-06T04:59:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 --2- 172.21.15.3:0/855900177 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558155980 0x7f355806aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 --2- 172.21.15.3:0/855900177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558156a30 0x7f355806e530 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 -- 172.21.15.3:0/855900177 >> 172.21.15.3:0/855900177 conn(0x7f355806c370 msgr2=0x7f355806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 -- 172.21.15.3:0/855900177 shutdown_connections 2023-10-06T04:59:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.955+0000 7f355edcf700 1 -- 172.21.15.3:0/855900177 wait complete. 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.956+0000 7f355edcf700 1 Processor -- start 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.956+0000 7f355edcf700 1 -- start start 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355edcf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558155980 0x7f35580c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355edcf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558156a30 0x7f35580c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355edcf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3558156ea0 0x7f35580c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355edcf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c2c50 con 0x7f3558155980 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355edcf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35580ca4f0 con 0x7f3558156ea0 2023-10-06T04:59:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355edcf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3558165d90 con 0x7f3558156a30 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558155980 0x7f35580c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558155980 0x7f35580c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34954/0 (socket says 172.21.15.3:34954) 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 -- 172.21.15.3:0/1020568064 learned_addr learned my addr 172.21.15.3:0/1020568064 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f3557fff700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558156a30 0x7f35580c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355d36c700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3558156ea0 0x7f35580c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 -- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558156a30 msgr2=0x7f35580c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558156a30 0x7f35580c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 -- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3558156ea0 msgr2=0x7f35580c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3558156ea0 0x7f35580c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355cb6b700 1 -- 172.21.15.3:0/1020568064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354800bcf0 con 0x7f3558155980 2023-10-06T04:59:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.957+0000 7f355d36c700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3558156ea0 0x7f35580c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.958+0000 7f355cb6b700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558155980 0x7f35580c5ac0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f355000c860 tx=0x7f3550008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.958+0000 7f3554ff9700 1 -- 172.21.15.3:0/1020568064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550011600 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.958+0000 7f3554ff9700 1 -- 172.21.15.3:0/1020568064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3550011760 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.958+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35580ca740 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.958+0000 7f3554ff9700 1 -- 172.21.15.3:0/1020568064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550011b20 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.958+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35580cad00 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.959+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3558071680 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.959+0000 7f3554ff9700 1 -- 172.21.15.3:0/1020568064 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35500193b0 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.960+0000 7f3554ff9700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.960+0000 7f3554ff9700 1 -- 172.21.15.3:0/1020568064 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3550084670 con 0x7f3558155980 2023-10-06T04:59:38.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.960+0000 7f3557fff700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:38.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.960+0000 7f3557fff700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3538066f30 0x7f35380693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f354800d120 tx=0x7f3548019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:38.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:38.964+0000 7f3554ff9700 1 -- 172.21.15.3:0/1020568064 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3550051780 con 0x7f3558155980 2023-10-06T04:59:39.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.157+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3558069f90 con 0x7f3558155980 2023-10-06T04:59:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.158+0000 7f3554ff9700 1 -- 172.21.15.3:0/1020568064 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f3550054da0 con 0x7f3558155980 2023-10-06T04:59:39.160 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:39.160 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:07.019162+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:03.167920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:04.391498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3538066f30 msgr2=0x7f35380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3538066f30 0x7f35380693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f354800d120 tx=0x7f3548019000 comp rx=0 tx=0).stop 2023-10-06T04:59:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558155980 msgr2=0x7f35580c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558155980 0x7f35580c5ac0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f355000c860 tx=0x7f3550008040 comp rx=0 tx=0).stop 2023-10-06T04:59:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 shutdown_connections 2023-10-06T04:59:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3558156ea0 0x7f35580c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3558155980 0x7f35580c5ac0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 --2- 172.21.15.3:0/1020568064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3558156a30 0x7f35580c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 >> 172.21.15.3:0/1020568064 conn(0x7f355806c370 msgr2=0x7f35580c6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 shutdown_connections 2023-10-06T04:59:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.160+0000 7f355edcf700 1 -- 172.21.15.3:0/1020568064 wait complete. 2023-10-06T04:59:39.174 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:59:39.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:39.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.292+0000 7fa18b59e700 1 Processor -- start 2023-10-06T04:59:39.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.292+0000 7fa18b59e700 1 -- start start 2023-10-06T04:59:39.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18b59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa18c0cb780 0x7fa18c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18b59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c159500 0x7fa18c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18b59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c0cf130 con 0x7fa18c0cb3b0 2023-10-06T04:59:39.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18b59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c0ce520 con 0x7fa18c0cb780 2023-10-06T04:59:39.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18b59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0ce800 con 0x7fa18c159500 2023-10-06T04:59:39.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa189d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa18c0cb780 0x7fa18c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa18c0cb3b0 0x7fa18c0cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57946/0 (socket says 172.21.15.3:57946) 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18a59c700 1 -- 172.21.15.3:0/2118879871 learned_addr learned my addr 172.21.15.3:0/2118879871 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/2118879871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c159500 0x7fa18c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa189d9b700 1 -- 172.21.15.3:0/2118879871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c159500 msgr2=0x7fa18c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa189d9b700 1 --2- 172.21.15.3:0/2118879871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c159500 0x7fa18c0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa189d9b700 1 -- 172.21.15.3:0/2118879871 >> v1:172.21.15.3:6789/0 conn(0x7fa18c0cb3b0 legacy=0x7fa18c0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.293+0000 7fa189d9b700 1 -- 172.21.15.3:0/2118879871 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0ceae0 con 0x7fa18c0cb780 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa189d9b700 1 --2- 172.21.15.3:0/2118879871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c0cb780 0x7fa18c158fc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa1740098e0 tx=0x7fa1740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1bdfa2af801f16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18959a700 1 -- 172.21.15.3:0/2118879871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174009e80 con 0x7fa18c0cb780 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18959a700 1 -- 172.21.15.3:0/2118879871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17400b860 con 0x7fa18c0cb780 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18959a700 1 -- 172.21.15.3:0/2118879871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400bbe0 con 0x7fa18c0cb780 2023-10-06T04:59:39.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 -- 172.21.15.3:0/2118879871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c0cb780 msgr2=0x7fa18c158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 --2- 172.21.15.3:0/2118879871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c0cb780 0x7fa18c158fc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa1740098e0 tx=0x7fa1740091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 -- 172.21.15.3:0/2118879871 shutdown_connections 2023-10-06T04:59:39.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 --2- 172.21.15.3:0/2118879871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c159500 0x7fa18c0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 --2- 172.21.15.3:0/2118879871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c0cb780 0x7fa18c158fc0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 -- 172.21.15.3:0/2118879871 >> 172.21.15.3:0/2118879871 conn(0x7fa18c070310 msgr2=0x7fa18c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:39.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 -- 172.21.15.3:0/2118879871 shutdown_connections 2023-10-06T04:59:39.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.294+0000 7fa18b59e700 1 -- 172.21.15.3:0/2118879871 wait complete. 2023-10-06T04:59:39.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 Processor -- start 2023-10-06T04:59:39.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 -- start start 2023-10-06T04:59:39.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa18c0cb3b0 0x7fa18c14cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c0cb780 0x7fa18c14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c159500 0x7fa18c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0b5ee0 con 0x7fa18c159500 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c14d480 con 0x7fa18c0cb3b0 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18b59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c14b7b0 con 0x7fa18c0cb780 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c159500 0x7fa18c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c159500 0x7fa18c0b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34966/0 (socket says 172.21.15.3:34966) 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.296+0000 7fa18ad9d700 1 -- 172.21.15.3:0/232604260 learned_addr learned my addr 172.21.15.3:0/232604260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa189d9b700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c0cb780 0x7fa18c14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18ad9d700 1 -- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c0cb780 msgr2=0x7fa18c14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c0cb780 0x7fa18c14c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18ad9d700 1 -- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa18c0cb3b0 msgr2=0x7fa18c14cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa18c0cb3b0 0x7fa18c14cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18ad9d700 1 -- 172.21.15.3:0/232604260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009420 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c159500 0x7fa18c0b59a0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00cdb0 tx=0x7fa17c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa1827fc700 1 -- 172.21.15.3:0/232604260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00aa80 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa1827fc700 1 -- 172.21.15.3:0/232604260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c00abe0 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c14ba90 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa1827fc700 1 -- 172.21.15.3:0/232604260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c017430 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.297+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c14c050 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.298+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c14c270 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.298+0000 7fa1827fc700 1 -- 172.21.15.3:0/232604260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa17c003020 con 0x7fa18c159500 2023-10-06T04:59:39.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.299+0000 7fa1827fc700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa16c066f30 0x7fa16c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.299+0000 7fa1827fc700 1 -- 172.21.15.3:0/232604260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa17c013070 con 0x7fa18c159500 2023-10-06T04:59:39.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.299+0000 7fa18a59c700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa16c066f30 0x7fa16c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.300+0000 7fa18a59c700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa16c066f30 0x7fa16c0693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa1840016a0 tx=0x7fa184007b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:39.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.303+0000 7fa1827fc700 1 -- 172.21.15.3:0/232604260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa17c0518f0 con 0x7fa18c159500 2023-10-06T04:59:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.539+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa18c04c380 con 0x7fa18c159500 2023-10-06T04:59:39.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.540+0000 7fa1827fc700 1 -- 172.21.15.3:0/232604260 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fa17c054f10 con 0x7fa18c159500 2023-10-06T04:59:39.542 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:39.542 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:30.805835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa16c066f30 msgr2=0x7fa16c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa16c066f30 0x7fa16c0693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa1840016a0 tx=0x7fa184007b30 comp rx=0 tx=0).stop 2023-10-06T04:59:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c159500 msgr2=0x7fa18c0b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c159500 0x7fa18c0b59a0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00cdb0 tx=0x7fa17c0077d0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 shutdown_connections 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa16c066f30 0x7fa16c0693e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa18c0cb3b0 0x7fa18c14cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c159500 0x7fa18c0b59a0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 --2- 172.21.15.3:0/232604260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa18c0cb780 0x7fa18c14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 >> 172.21.15.3:0/232604260 conn(0x7fa18c070310 msgr2=0x7fa18c149960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 shutdown_connections 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.542+0000 7fa18b59e700 1 -- 172.21.15.3:0/232604260 wait complete. 2023-10-06T04:59:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 61 2023-10-06T04:59:39.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:39.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.673+0000 7f52e83b4700 1 Processor -- start 2023-10-06T04:59:39.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.673+0000 7f52e83b4700 1 -- start start 2023-10-06T04:59:39.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e83b4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52e00b2530 0x7f52e00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e83b4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e015e0d0 con 0x7f52e0159080 2023-10-06T04:59:39.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e83b4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e00cbd70 con 0x7f52e00b2900 2023-10-06T04:59:39.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e83b4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e00b0e00 con 0x7f52e00b2530 2023-10-06T04:59:39.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e6951700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f52e0159080 0x7f52e01594c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57950/0 (socket says 172.21.15.3:57950) 2023-10-06T04:59:39.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e6951700 1 -- 172.21.15.3:0/161089296 learned_addr learned my addr 172.21.15.3:0/161089296 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:39.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e6150700 1 --2- 172.21.15.3:0/161089296 >> v2:172.21.15.153:3300/0 conn(0x7f52e00b2530 0x7f52e00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483141315 0 0) 0x7f52d0001490 con 0x7f52e0159080 2023-10-06T04:59:39.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.674+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c8003570 con 0x7f52e0159080 2023-10-06T04:59:39.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351165807 0 0) 0x7f52c8003570 con 0x7f52e0159080 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 >> v2:172.21.15.153:3300/0 conn(0x7f52e00b2530 msgr2=0x7f52e00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 --2- 172.21.15.3:0/161089296 >> v2:172.21.15.153:3300/0 conn(0x7f52e00b2530 0x7f52e00b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 >> v1:172.21.15.153:6789/0 conn(0x7f52e00b2900 legacy=0x7f52e015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e00cb160 con 0x7f52e0159080 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f52d0001980 con 0x7f52e0159080 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 >> v1:172.21.15.3:6789/0 conn(0x7f52e0159080 legacy=0x7f52e01594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c8003a60 0x7f52c8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c80051c0 0x7f52c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00b0e00 con 0x7f52c8004810 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c800c0d0 con 0x7f52c8003a60 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c800c290 con 0x7f52c80051c0 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e594f700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e6150700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c8003a60 0x7f52c8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e6951700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c80051c0 0x7f52c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e594f700 1 -- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c80051c0 msgr2=0x7f52c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e594f700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52c80051c0 0x7f52c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e594f700 1 -- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c8003a60 msgr2=0x7f52c8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e594f700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52c8003a60 0x7f52c8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e594f700 1 -- 172.21.15.3:0/161089296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c8003570 con 0x7f52c8004810 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e6951700 1 -- 172.21.15.3:0/161089296 reap_dead start 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.675+0000 7f52e594f700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52c8004c80 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f52d40018b0 tx=0x7f52d4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bf30ede15167df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4012420 con 0x7f52c8004810 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d40125a0 con 0x7f52c8004810 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e514e700 1 -- 172.21.15.3:0/161089296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4012960 con 0x7f52c8004810 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e83b4700 1 -- 172.21.15.3:0/161089296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 msgr2=0x7f52c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e83b4700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52c8004c80 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f52d40018b0 tx=0x7f52d4007d30 comp rx=0 tx=0).stop 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e83b4700 1 -- 172.21.15.3:0/161089296 shutdown_connections 2023-10-06T04:59:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e83b4700 1 --2- 172.21.15.3:0/161089296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52c8004c80 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e83b4700 1 -- 172.21.15.3:0/161089296 >> 172.21.15.3:0/161089296 conn(0x7f52e0070310 msgr2=0x7f52e0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e83b4700 1 -- 172.21.15.3:0/161089296 shutdown_connections 2023-10-06T04:59:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.676+0000 7f52e83b4700 1 -- 172.21.15.3:0/161089296 wait complete. 2023-10-06T04:59:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.677+0000 7f52e83b4700 1 Processor -- start 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.677+0000 7f52e83b4700 1 -- start start 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e83b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52e00c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e83b4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52e015b800 0x7f52e00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e83b4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52e01556d0 0x7f52e0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e83b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e016d470 con 0x7f52c8004810 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e83b4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e016dff0 con 0x7f52e01556d0 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e83b4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0156080 con 0x7f52e015b800 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e6150700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52e00c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e6150700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52e00c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34984/0 (socket says 172.21.15.3:34984) 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e6150700 1 -- 172.21.15.3:0/4069568343 learned_addr learned my addr 172.21.15.3:0/4069568343 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e594f700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52e015b800 0x7f52e00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e6951700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52e01556d0 0x7f52e0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e6150700 1 -- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52e015b800 msgr2=0x7f52e00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.678+0000 7f52e6150700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52e015b800 0x7f52e00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52e6150700 1 -- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52e01556d0 msgr2=0x7f52e0155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52e6150700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52e01556d0 0x7f52e0155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52e6150700 1 -- 172.21.15.3:0/4069568343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e016d6c0 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52e6951700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52e01556d0 0x7f52e0155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52e6150700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52e00c6b00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f52d8011d40 tx=0x7f52d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52de7fc700 1 -- 172.21.15.3:0/4069568343 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d800fac0 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52de7fc700 1 -- 172.21.15.3:0/4069568343 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d800fc20 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52de7fc700 1 -- 172.21.15.3:0/4069568343 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8014440 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e016d9a0 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.679+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e00c7040 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.680+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e0071680 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.680+0000 7f52de7fc700 1 -- 172.21.15.3:0/4069568343 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52d8014a80 con 0x7f52c8004810 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.681+0000 7f52de7fc700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52cc066f30 0x7f52cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.681+0000 7f52de7fc700 1 -- 172.21.15.3:0/4069568343 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f52d8018030 con 0x7f52c8004810 2023-10-06T04:59:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.681+0000 7f52e594f700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52cc066f30 0x7f52cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.682+0000 7f52e594f700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52cc066f30 0x7f52cc0693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f52e016d3d0 tx=0x7f52d4015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.685+0000 7f52de7fc700 1 -- 172.21.15.3:0/4069568343 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52d8052520 con 0x7f52c8004810 2023-10-06T04:59:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.923+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52e016e2d0 con 0x7f52c8004810 2023-10-06T04:59:39.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.923+0000 7f52de7fc700 1 -- 172.21.15.3:0/4069568343 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f52d805b030 con 0x7f52c8004810 2023-10-06T04:59:39.925 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:39.925 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:05.413962+0000","modified":"2023-10-06T04:59:30.805835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"d","rank":0,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/1968748272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1968748272},{"type":"v1","addr":"172.21.15.153:6837","nonce":1968748272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T04:59:39.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52cc066f30 msgr2=0x7f52cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52cc066f30 0x7f52cc0693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f52e016d3d0 tx=0x7f52d4015000 comp rx=0 tx=0).stop 2023-10-06T04:59:39.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 msgr2=0x7f52e00c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52e00c6b00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f52d8011d40 tx=0x7f52d80091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 shutdown_connections 2023-10-06T04:59:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f52cc066f30 0x7f52cc0693e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52e01556d0 0x7f52e0155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c8004810 0x7f52e00c6b00 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 --2- 172.21.15.3:0/4069568343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f52e015b800 0x7f52e00b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.925+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 >> 172.21.15.3:0/4069568343 conn(0x7f52e0070310 msgr2=0x7f52e0156380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.926+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 shutdown_connections 2023-10-06T04:59:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:39.926+0000 7f52e83b4700 1 -- 172.21.15.3:0/4069568343 wait complete. 2023-10-06T04:59:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 61 2023-10-06T04:59:39.939 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.057+0000 7ffbf2a07700 1 Processor -- start 2023-10-06T04:59:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.057+0000 7ffbf2a07700 1 -- start start 2023-10-06T04:59:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf2a07700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffbec06ec60 0x7ffbec06d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf2a07700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec0a9860 con 0x7ffbec159260 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf2a07700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec06c6e0 con 0x7ffbec06e7f0 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf2a07700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbec0c6de0 con 0x7ffbec06ec60 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf1a05700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffbec159260 0x7ffbec0cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57960/0 (socket says 172.21.15.3:57960) 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf1a05700 1 -- 172.21.15.3:0/774362691 learned_addr learned my addr 172.21.15.3:0/774362691 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf2206700 1 --2- 172.21.15.3:0/774362691 >> v2:172.21.15.153:3300/0 conn(0x7ffbec06ec60 0x7ffbec06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759984049 0 0) 0x7ffbec0a9860 con 0x7ffbec159260 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.058+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd4003570 con 0x7ffbec159260 2023-10-06T04:59:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698068766 0 0) 0x7ffbec06c6e0 con 0x7ffbec06e7f0 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbec0a9860 con 0x7ffbec06e7f0 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619957228 0 0) 0x7ffbd4003570 con 0x7ffbec159260 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf2206700 1 -- 172.21.15.3:0/774362691 >> v1:172.21.15.153:6789/0 conn(0x7ffbec06e7f0 legacy=0x7ffbec06cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf2206700 1 -- 172.21.15.3:0/774362691 >> v1:172.21.15.3:6789/0 conn(0x7ffbec159260 legacy=0x7ffbec0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf2206700 1 -- 172.21.15.3:0/774362691 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec0c7080 con 0x7ffbec06ec60 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf2206700 1 --2- 172.21.15.3:0/774362691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 0x7ffbec06d280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40098e0 tx=0x7ffbe40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c07b84e0d8c958b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0017f0 con 0x7ffbec06ec60 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe4011420 con 0x7ffbec06ec60 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.059+0000 7ffbf0a03700 1 -- 172.21.15.3:0/774362691 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe40117a0 con 0x7ffbec06ec60 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.060+0000 7ffbf2a07700 1 -- 172.21.15.3:0/774362691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 msgr2=0x7ffbec06d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.060+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/774362691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 0x7ffbec06d280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40098e0 tx=0x7ffbe40091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.060+0000 7ffbf2a07700 1 -- 172.21.15.3:0/774362691 shutdown_connections 2023-10-06T04:59:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.060+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/774362691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 0x7ffbec06d280 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.060+0000 7ffbf2a07700 1 -- 172.21.15.3:0/774362691 >> 172.21.15.3:0/774362691 conn(0x7ffbec067a80 msgr2=0x7ffbec0cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.060+0000 7ffbf2a07700 1 -- 172.21.15.3:0/774362691 shutdown_connections 2023-10-06T04:59:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.060+0000 7ffbf2a07700 1 -- 172.21.15.3:0/774362691 wait complete. 2023-10-06T04:59:40.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.061+0000 7ffbf2a07700 1 Processor -- start 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.061+0000 7ffbf2a07700 1 -- start start 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf2a07700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbec06e7f0 0x7ffbec1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf2a07700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 0x7ffbec0c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf2a07700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbec159260 0x7ffbec0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf2a07700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0c0150 con 0x7ffbec06e7f0 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf2a07700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0c0f70 con 0x7ffbec06ec60 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf2a07700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0c0360 con 0x7ffbec159260 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbec06e7f0 0x7ffbec1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbec06e7f0 0x7ffbec1524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35000/0 (socket says 172.21.15.3:35000) 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 -- 172.21.15.3:0/252667159 learned_addr learned my addr 172.21.15.3:0/252667159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 -- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbec159260 msgr2=0x7ffbec0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbec159260 0x7ffbec0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 -- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 msgr2=0x7ffbec0c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 0x7ffbec0c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 -- 172.21.15.3:0/252667159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe4009420 con 0x7ffbec06e7f0 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf1a05700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbec06e7f0 0x7ffbec1524f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00cbe0 tx=0x7ffbdc007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbe9ffb700 1 -- 172.21.15.3:0/252667159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc004dc0 con 0x7ffbec06e7f0 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbe9ffb700 1 -- 172.21.15.3:0/252667159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc00a6f0 con 0x7ffbec06e7f0 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec0c0640 con 0x7ffbec06e7f0 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.062+0000 7ffbe9ffb700 1 -- 172.21.15.3:0/252667159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00aab0 con 0x7ffbec06e7f0 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.063+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec0c0c00 con 0x7ffbec06e7f0 2023-10-06T04:59:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.063+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec0c1250 con 0x7ffbec06e7f0 2023-10-06T04:59:40.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.064+0000 7ffbe9ffb700 1 -- 172.21.15.3:0/252667159 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbdc007150 con 0x7ffbec06e7f0 2023-10-06T04:59:40.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.064+0000 7ffbe9ffb700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffbd8066f30 0x7ffbd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.064+0000 7ffbe9ffb700 1 -- 172.21.15.3:0/252667159 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffbdc084220 con 0x7ffbec06e7f0 2023-10-06T04:59:40.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.064+0000 7ffbf1204700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffbd8066f30 0x7ffbd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.065+0000 7ffbf1204700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffbd8066f30 0x7ffbd80693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0aee80 tx=0x7ffbe00079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.068+0000 7ffbe9ffb700 1 -- 172.21.15.3:0/252667159 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbdc054e40 con 0x7ffbec06e7f0 2023-10-06T04:59:40.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.261+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbec04c380 con 0x7ffbec06e7f0 2023-10-06T04:59:40.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.262+0000 7ffbe9ffb700 1 -- 172.21.15.3:0/252667159 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16821 (secure 0 0 0) 0x7ffbdc0549d0 con 0x7ffbec06e7f0 2023-10-06T04:59:40.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:40.264 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:07.019162+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:03.167920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:04.391498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:40.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffbd8066f30 msgr2=0x7ffbd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffbd8066f30 0x7ffbd80693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0aee80 tx=0x7ffbe00079d0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbec06e7f0 msgr2=0x7ffbec1524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbec06e7f0 0x7ffbec1524f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00cbe0 tx=0x7ffbdc007d20 comp rx=0 tx=0).stop 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 shutdown_connections 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffbd8066f30 0x7ffbd80693e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbec159260 0x7ffbec0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbec06e7f0 0x7ffbec1524f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 --2- 172.21.15.3:0/252667159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec06ec60 0x7ffbec0c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 >> 172.21.15.3:0/252667159 conn(0x7ffbec067a80 msgr2=0x7ffbec06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 shutdown_connections 2023-10-06T04:59:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.264+0000 7ffbf2a07700 1 -- 172.21.15.3:0/252667159 wait complete. 2023-10-06T04:59:40.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:59:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.397+0000 7fb92f755700 1 Processor -- start 2023-10-06T04:59:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.397+0000 7fb92f755700 1 -- start start 2023-10-06T04:59:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92f755700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb928159500 0x7fb9280cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92f755700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9281598d0 0x7fb9280cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92f755700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280cc5c0 con 0x7fb9280b2220 2023-10-06T04:59:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92f755700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280cb7b0 con 0x7fb928159500 2023-10-06T04:59:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92f755700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280cda60 con 0x7fb9281598d0 2023-10-06T04:59:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92dcf2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb9280b2220 0x7fb9280b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57962/0 (socket says 172.21.15.3:57962) 2023-10-06T04:59:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92dcf2700 1 -- 172.21.15.3:0/1307987913 learned_addr learned my addr 172.21.15.3:0/1307987913 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92d4f1700 1 --2- 172.21.15.3:0/1307987913 >> v2:172.21.15.153:3300/0 conn(0x7fb928159500 0x7fb9280cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92ccf0700 1 --2- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9281598d0 0x7fb9280cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb927fff700 1 -- 172.21.15.3:0/1307987913 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116036356 0 0) 0x7fb9280cc5c0 con 0x7fb9280b2220 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb927fff700 1 -- 172.21.15.3:0/1307987913 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb910003570 con 0x7fb9280b2220 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92d4f1700 1 -- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9281598d0 msgr2=0x7fb9280cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92d4f1700 1 --2- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9281598d0 0x7fb9280cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92d4f1700 1 -- 172.21.15.3:0/1307987913 >> v1:172.21.15.3:6789/0 conn(0x7fb9280b2220 legacy=0x7fb9280b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92d4f1700 1 -- 172.21.15.3:0/1307987913 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280cb9b0 con 0x7fb928159500 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.398+0000 7fb92ccf0700 1 --2- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9281598d0 0x7fb9280cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92d4f1700 1 --2- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb928159500 0x7fb9280cad30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb9200098e0 tx=0x7fb9200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a60d78496474d79e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb927fff700 1 -- 172.21.15.3:0/1307987913 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920011630 con 0x7fb928159500 2023-10-06T04:59:40.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb927fff700 1 -- 172.21.15.3:0/1307987913 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920011790 con 0x7fb928159500 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb927fff700 1 -- 172.21.15.3:0/1307987913 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920011b30 con 0x7fb928159500 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 -- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb928159500 msgr2=0x7fb9280cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 --2- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb928159500 0x7fb9280cad30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb9200098e0 tx=0x7fb9200091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 -- 172.21.15.3:0/1307987913 shutdown_connections 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 --2- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb928159500 0x7fb9280cad30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 --2- 172.21.15.3:0/1307987913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9281598d0 0x7fb9280cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 -- 172.21.15.3:0/1307987913 >> 172.21.15.3:0/1307987913 conn(0x7fb928070310 msgr2=0x7fb928070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 -- 172.21.15.3:0/1307987913 shutdown_connections 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.399+0000 7fb92f755700 1 -- 172.21.15.3:0/1307987913 wait complete. 2023-10-06T04:59:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.400+0000 7fb92f755700 1 Processor -- start 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.401+0000 7fb92f755700 1 -- start start 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.401+0000 7fb92f755700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9280b2220 0x7fb92816d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.401+0000 7fb92f755700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb928159500 0x7fb9280b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.401+0000 7fb92f755700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9281598d0 0x7fb9280c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.401+0000 7fb92f755700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280c3f30 con 0x7fb928159500 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.401+0000 7fb92f755700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280b5b60 con 0x7fb9281598d0 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.401+0000 7fb92f755700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92816d6d0 con 0x7fb9280b2220 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb928159500 0x7fb9280b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb928159500 0x7fb9280b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35006/0 (socket says 172.21.15.3:35006) 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 -- 172.21.15.3:0/3543018105 learned_addr learned my addr 172.21.15.3:0/3543018105 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:40.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92d4f1700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9280b2220 0x7fb92816d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92dcf2700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9281598d0 0x7fb9280c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 -- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9280b2220 msgr2=0x7fb92816d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9280b2220 0x7fb92816d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 -- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9281598d0 msgr2=0x7fb9280c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9281598d0 0x7fb9280c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 -- 172.21.15.3:0/3543018105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c009420 con 0x7fb928159500 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92dcf2700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9281598d0 0x7fb9280c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92ccf0700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb928159500 0x7fb9280b5620 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00ec80 tx=0x7fb91c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb9257fa700 1 -- 172.21.15.3:0/3543018105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00c850 con 0x7fb928159500 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb9257fa700 1 -- 172.21.15.3:0/3543018105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c00c9b0 con 0x7fb928159500 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb920009420 con 0x7fb928159500 2023-10-06T04:59:40.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb9257fa700 1 -- 172.21.15.3:0/3543018105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00cd30 con 0x7fb928159500 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.402+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92816dc90 con 0x7fb928159500 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.403+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92816e030 con 0x7fb928159500 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.404+0000 7fb9257fa700 1 -- 172.21.15.3:0/3543018105 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb91c013680 con 0x7fb928159500 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.404+0000 7fb9257fa700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb914066f30 0x7fb9140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.404+0000 7fb9257fa700 1 -- 172.21.15.3:0/3543018105 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb91c084c70 con 0x7fb928159500 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.404+0000 7fb92d4f1700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb914066f30 0x7fb9140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.405+0000 7fb92d4f1700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb914066f30 0x7fb9140693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb9200091e0 tx=0x7fb920017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.409+0000 7fb9257fa700 1 -- 172.21.15.3:0/3543018105 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb91c051d80 con 0x7fb928159500 2023-10-06T04:59:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.646+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb928071680 con 0x7fb928159500 2023-10-06T04:59:40.648 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T04:59:40.647+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:59:40.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.651+0000 7fb9257fa700 1 -- 172.21.15.3:0/3543018105 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb91c017090 con 0x7fb928159500 2023-10-06T04:59:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb914066f30 msgr2=0x7fb9140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb914066f30 0x7fb9140693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb9200091e0 tx=0x7fb920017000 comp rx=0 tx=0).stop 2023-10-06T04:59:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb928159500 msgr2=0x7fb9280b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb928159500 0x7fb9280b5620 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00ec80 tx=0x7fb91c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 shutdown_connections 2023-10-06T04:59:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb914066f30 0x7fb9140693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9281598d0 0x7fb9280c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb9280b2220 0x7fb92816d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 --2- 172.21.15.3:0/3543018105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb928159500 0x7fb9280b5620 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 >> 172.21.15.3:0/3543018105 conn(0x7fb928070310 msgr2=0x7fb9280ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 shutdown_connections 2023-10-06T04:59:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.654+0000 7fb92f755700 1 -- 172.21.15.3:0/3543018105 wait complete. 2023-10-06T04:59:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:59:40.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:59:40.690 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:59:40.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.786+0000 7fe325957700 1 Processor -- start 2023-10-06T04:59:40.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.787+0000 7fe325957700 1 -- start start 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.787+0000 7fe325957700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe320140e70 0x7fe3200ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.787+0000 7fe325957700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe3200b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.787+0000 7fe325957700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe320069f10 con 0x7fe3200b14f0 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.787+0000 7fe325957700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3200cca70 con 0x7fe320140e70 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.787+0000 7fe325957700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3200cc140 con 0x7fe3200cd440 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe320140e70 0x7fe3200ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31e7fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe3200b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe3200b14f0 0x7fe320067ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:54470/0 (socket says 172.21.15.3:54470) 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe320140e70 0x7fe3200ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:59272/0 (socket says 172.21.15.3:59272) 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 -- 172.21.15.3:0/437742933 learned_addr learned my addr 172.21.15.3:0/437742933 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 -- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 msgr2=0x7fe3200b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 --2- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe3200b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 -- 172.21.15.3:0/437742933 >> v1:172.21.15.153:6789/0 conn(0x7fe3200b14f0 legacy=0x7fe320067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 -- 172.21.15.3:0/437742933 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3200cc420 con 0x7fe320140e70 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31e7fc700 1 --2- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe3200b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31effd700 1 --2- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe320140e70 0x7fe3200ccf00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe310009f20 tx=0x7fe310009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a921928c11b4bb25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.788+0000 7fe31dffb700 1 -- 172.21.15.3:0/437742933 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000e070 con 0x7fe320140e70 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe31dffb700 1 -- 172.21.15.3:0/437742933 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31000b9d0 con 0x7fe320140e70 2023-10-06T04:59:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe31dffb700 1 -- 172.21.15.3:0/437742933 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100092e0 con 0x7fe320140e70 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 -- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe320140e70 msgr2=0x7fe3200ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 --2- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe320140e70 0x7fe3200ccf00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe310009f20 tx=0x7fe310009840 comp rx=0 tx=0).stop 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 -- 172.21.15.3:0/437742933 shutdown_connections 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 --2- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe320140e70 0x7fe3200ccf00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 --2- 172.21.15.3:0/437742933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe3200b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 -- 172.21.15.3:0/437742933 >> 172.21.15.3:0/437742933 conn(0x7fe3200ce210 msgr2=0x7fe3200ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 -- 172.21.15.3:0/437742933 shutdown_connections 2023-10-06T04:59:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.789+0000 7fe325957700 1 -- 172.21.15.3:0/437742933 wait complete. 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.790+0000 7fe325957700 1 Processor -- start 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.790+0000 7fe325957700 1 -- start start 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe325957700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3200b14f0 0x7fe3200c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe325957700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe32006e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe325957700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe320140e70 0x7fe32013e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe325957700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32006e3e0 con 0x7fe320140e70 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe325957700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3200c8ad0 con 0x7fe3200b14f0 2023-10-06T04:59:40.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe325957700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32006edb0 con 0x7fe3200cd440 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe31f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe320140e70 0x7fe32013e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe31e7fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe32006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe31effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3200b14f0 0x7fe3200c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe31e7fc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe32006e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:54060/0 (socket says 172.21.15.3:54060) 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe31f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe320140e70 0x7fe32013e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35014/0 (socket says 172.21.15.3:35014) 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.791+0000 7fe31e7fc700 1 -- 172.21.15.3:0/2421536813 learned_addr learned my addr 172.21.15.3:0/2421536813 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31f7fe700 1 -- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 msgr2=0x7fe32006e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31f7fe700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe32006e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31f7fe700 1 -- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3200b14f0 msgr2=0x7fe3200c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31f7fe700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3200b14f0 0x7fe3200c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:40.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31f7fe700 1 -- 172.21.15.3:0/2421536813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009420 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31e7fc700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe32006e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31effd700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3200b14f0 0x7fe3200c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe31f7fe700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe320140e70 0x7fe32013e050 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c490 tx=0x7fe318007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe316ffd700 1 -- 172.21.15.3:0/2421536813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318004830 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe316ffd700 1 -- 172.21.15.3:0/2421536813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe318004990 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe310009ae0 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe316ffd700 1 -- 172.21.15.3:0/2421536813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318004d50 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.792+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32006d7d0 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.793+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3200b0fb0 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.793+0000 7fe316ffd700 1 -- 172.21.15.3:0/2421536813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3180074f0 con 0x7fe320140e70 2023-10-06T04:59:40.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.794+0000 7fe316ffd700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe300066f30 0x7fe3000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:40.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.794+0000 7fe316ffd700 1 -- 172.21.15.3:0/2421536813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fe318083fa0 con 0x7fe320140e70 2023-10-06T04:59:40.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.794+0000 7fe31effd700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe300066f30 0x7fe3000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:40.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.795+0000 7fe31effd700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe300066f30 0x7fe3000693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe310009880 tx=0x7fe310018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:40.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:40.798+0000 7fe316ffd700 1 -- 172.21.15.3:0/2421536813 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe318051060 con 0x7fe320140e70 2023-10-06T04:59:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.037+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe32004c380 con 0x7fe320140e70 2023-10-06T04:59:41.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.652+0000 7fe316ffd700 1 -- 172.21.15.3:0/2421536813 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe318054680 con 0x7fe320140e70 2023-10-06T04:59:41.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe300066f30 msgr2=0x7fe3000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe300066f30 0x7fe3000693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe310009880 tx=0x7fe310018000 comp rx=0 tx=0).stop 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe320140e70 msgr2=0x7fe32013e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe320140e70 0x7fe32013e050 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c490 tx=0x7fe318007e90 comp rx=0 tx=0).stop 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 shutdown_connections 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe300066f30 0x7fe3000693e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3200b14f0 0x7fe3200c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe320140e70 0x7fe32013e050 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 --2- 172.21.15.3:0/2421536813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3200cd440 0x7fe32006e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 >> 172.21.15.3:0/2421536813 conn(0x7fe3200ce210 msgr2=0x7fe3200b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 shutdown_connections 2023-10-06T04:59:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.655+0000 7fe325957700 1 -- 172.21.15.3:0/2421536813 wait complete. 2023-10-06T04:59:41.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:59:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.800+0000 7eff41c99700 1 Processor -- start 2023-10-06T04:59:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.800+0000 7eff41c99700 1 -- start start 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.800+0000 7eff41c99700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7eff3c140a70 0x7eff3c0c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.800+0000 7eff41c99700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c0c68b0 con 0x7eff3c0ccaf0 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.800+0000 7eff41c99700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c0c6b20 con 0x7eff3c0c7e20 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.800+0000 7eff41c99700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c0c6e00 con 0x7eff3c140a70 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7eff3c0ccaf0 0x7eff3c0aee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57976/0 (socket says 172.21.15.3:57976) 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3bfff700 1 -- 172.21.15.3:0/3346033410 learned_addr learned my addr 172.21.15.3:0/3346033410 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3b7fe700 1 --2- 172.21.15.3:0/3346033410 >> v2:172.21.15.153:3300/0 conn(0x7eff3c140a70 0x7eff3c0c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723030061 0 0) 0x7eff3c0c68b0 con 0x7eff3c0ccaf0 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff28003570 con 0x7eff3c0ccaf0 2023-10-06T04:59:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114094165 0 0) 0x7eff3c0c6b20 con 0x7eff3c0c7e20 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff3c0c68b0 con 0x7eff3c0c7e20 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3193516125 0 0) 0x7eff28003570 con 0x7eff3c0ccaf0 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3b7fe700 1 -- 172.21.15.3:0/3346033410 >> v1:172.21.15.153:6789/0 conn(0x7eff3c0c7e20 legacy=0x7eff3c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3b7fe700 1 -- 172.21.15.3:0/3346033410 >> v1:172.21.15.3:6789/0 conn(0x7eff3c0ccaf0 legacy=0x7eff3c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3b7fe700 1 -- 172.21.15.3:0/3346033410 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c0c70e0 con 0x7eff3c140a70 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.801+0000 7eff3b7fe700 1 --2- 172.21.15.3:0/3346033410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 0x7eff3c0c78e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0098e0 tx=0x7eff2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a137885e07b188 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff340017f0 con 0x7eff3c140a70 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c011420 con 0x7eff3c140a70 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff3a7fc700 1 -- 172.21.15.3:0/3346033410 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0117c0 con 0x7eff3c140a70 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff41c99700 1 -- 172.21.15.3:0/3346033410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 msgr2=0x7eff3c0c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff41c99700 1 --2- 172.21.15.3:0/3346033410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 0x7eff3c0c78e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0098e0 tx=0x7eff2c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff41c99700 1 -- 172.21.15.3:0/3346033410 shutdown_connections 2023-10-06T04:59:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff41c99700 1 --2- 172.21.15.3:0/3346033410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 0x7eff3c0c78e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff41c99700 1 -- 172.21.15.3:0/3346033410 >> 172.21.15.3:0/3346033410 conn(0x7eff3c070310 msgr2=0x7eff3c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff41c99700 1 -- 172.21.15.3:0/3346033410 shutdown_connections 2023-10-06T04:59:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.802+0000 7eff41c99700 1 -- 172.21.15.3:0/3346033410 wait complete. 2023-10-06T04:59:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.803+0000 7eff41c99700 1 Processor -- start 2023-10-06T04:59:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.803+0000 7eff41c99700 1 -- start start 2023-10-06T04:59:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff41c99700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c0c7e20 0x7eff3c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff41c99700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff3c0ccaf0 0x7eff3c0ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff41c99700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 0x7eff3c0adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff41c99700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0c5850 con 0x7eff3c0c7e20 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff41c99700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0c4c40 con 0x7eff3c140a70 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff41c99700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0c4f20 con 0x7eff3c0ccaf0 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c0c7e20 0x7eff3c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c0c7e20 0x7eff3c1569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35024/0 (socket says 172.21.15.3:35024) 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff3b7fe700 1 -- 172.21.15.3:0/606325890 learned_addr learned my addr 172.21.15.3:0/606325890 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff3bfff700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 0x7eff3c0adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.804+0000 7eff3affd700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff3c0ccaf0 0x7eff3c0ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff3b7fe700 1 -- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff3c0ccaf0 msgr2=0x7eff3c0ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff3b7fe700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff3c0ccaf0 0x7eff3c0ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff3b7fe700 1 -- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 msgr2=0x7eff3c0adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff3b7fe700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 0x7eff3c0adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff3b7fe700 1 -- 172.21.15.3:0/606325890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c009420 con 0x7eff3c0c7e20 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff3b7fe700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c0c7e20 0x7eff3c1569e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7eff3c067a20 tx=0x7eff2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff337fe700 1 -- 172.21.15.3:0/606325890 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c01cb30 con 0x7eff3c0c7e20 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff337fe700 1 -- 172.21.15.3:0/606325890 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c01cc90 con 0x7eff3c0c7e20 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c0c5200 con 0x7eff3c0c7e20 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff337fe700 1 -- 172.21.15.3:0/606325890 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c01a6a0 con 0x7eff3c0c7e20 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.805+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c156f20 con 0x7eff3c0c7e20 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.806+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c071680 con 0x7eff3c0c7e20 2023-10-06T04:59:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.806+0000 7eff337fe700 1 -- 172.21.15.3:0/606325890 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff2c01aa50 con 0x7eff3c0c7e20 2023-10-06T04:59:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.807+0000 7eff337fe700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff1c066f30 0x7eff1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.807+0000 7eff337fe700 1 -- 172.21.15.3:0/606325890 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6144+0+0 (secure 0 0 0) 0x7eff2c085ff0 con 0x7eff3c0c7e20 2023-10-06T04:59:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.807+0000 7eff3affd700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff1c066f30 0x7eff1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.807+0000 7eff3affd700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff1c066f30 0x7eff1c0693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7eff3c16ac10 tx=0x7eff24007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:41.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:41.811+0000 7eff337fe700 1 -- 172.21.15.3:0/606325890 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff2c0530b0 con 0x7eff3c0c7e20 2023-10-06T04:59:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.005+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7eff3c0c5b30 con 0x7eff3c0c7e20 2023-10-06T04:59:42.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.849+0000 7eff337fe700 1 -- 172.21.15.3:0/606325890 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7eff2c0566d0 con 0x7eff3c0c7e20 2023-10-06T04:59:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff1c066f30 msgr2=0x7eff1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff1c066f30 0x7eff1c0693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7eff3c16ac10 tx=0x7eff24007d50 comp rx=0 tx=0).stop 2023-10-06T04:59:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c0c7e20 msgr2=0x7eff3c1569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c0c7e20 0x7eff3c1569e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7eff3c067a20 tx=0x7eff2c016000 comp rx=0 tx=0).stop 2023-10-06T04:59:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 shutdown_connections 2023-10-06T04:59:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7eff1c066f30 0x7eff1c0693e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c140a70 0x7eff3c0adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:42.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c0c7e20 0x7eff3c1569e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:42.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 --2- 172.21.15.3:0/606325890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff3c0ccaf0 0x7eff3c0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:42.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 >> 172.21.15.3:0/606325890 conn(0x7eff3c070310 msgr2=0x7eff3c0c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:42.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.853+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 shutdown_connections 2023-10-06T04:59:42.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.854+0000 7eff41c99700 1 -- 172.21.15.3:0/606325890 wait complete. 2023-10-06T04:59:42.857 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:59:42.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:59:42.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.996+0000 7f1d5b071700 1 Processor -- start 2023-10-06T04:59:42.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.996+0000 7f1d5b071700 1 -- start start 2023-10-06T04:59:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d5b071700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1d5415c740 0x7f1d540cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d5b071700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 0x7f1d540ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d5b071700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d540ce7b0 con 0x7f1d5415cb10 2023-10-06T04:59:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d5b071700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5415cf70 con 0x7f1d5415c740 2023-10-06T04:59:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d5b071700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540cea90 con 0x7f1d540afe60 2023-10-06T04:59:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d53fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1d5415cb10 0x7f1d5415dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39932/0 (socket says 172.21.15.3:39932) 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d53fff700 1 -- 172.21.15.3:0/758268765 learned_addr learned my addr 172.21.15.3:0/758268765 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617234660 0 0) 0x7f1d540ce7b0 con 0x7f1d5415cb10 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/758268765 >> v2:172.21.15.153:3300/0 conn(0x7f1d5415c740 0x7f1d540cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d5960e700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 0x7f1d540ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d40003570 con 0x7f1d5415cb10 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3434222477 0 0) 0x7f1d40003570 con 0x7f1d5415cb10 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 msgr2=0x7f1d540ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 0x7f1d540ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 >> v2:172.21.15.153:3300/0 conn(0x7f1d5415c740 msgr2=0x7f1d540cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 --2- 172.21.15.3:0/758268765 >> v2:172.21.15.153:3300/0 conn(0x7f1d5415c740 0x7f1d540cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540ced70 con 0x7f1d5415cb10 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f1d440017f0 con 0x7f1d5415cb10 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.997+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 >> v1:172.21.15.3:6789/0 conn(0x7f1d5415cb10 legacy=0x7f1d5415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40004810 0x7f1d40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 0x7f1d400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5415cf70 con 0x7f1d40003a60 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540cea90 con 0x7f1d40004810 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4000ba60 con 0x7f1d540afe60 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d53fff700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40004810 0x7f1d40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d5960e700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 0x7f1d400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d58e0d700 1 -- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 msgr2=0x7f1d400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540afe60 0x7f1d400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d58e0d700 1 -- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40004810 msgr2=0x7f1d40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40004810 0x7f1d40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d58e0d700 1 -- 172.21.15.3:0/758268765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d40003570 con 0x7f1d40003a60 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d40003e70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009c30 tx=0x7f1d4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d55c4cbde80ad4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00e710 con 0x7f1d40003a60 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c00e870 con 0x7f1d40003a60 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.998+0000 7f1d537fe700 1 -- 172.21.15.3:0/758268765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00ec30 con 0x7f1d40003a60 2023-10-06T04:59:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.999+0000 7f1d5b071700 1 -- 172.21.15.3:0/758268765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 msgr2=0x7f1d40003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.999+0000 7f1d5b071700 1 --2- 172.21.15.3:0/758268765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d40003e70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009c30 tx=0x7f1d4c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.999+0000 7f1d5960e700 1 -- 172.21.15.3:0/758268765 reap_dead start 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.999+0000 7f1d5b071700 1 -- 172.21.15.3:0/758268765 shutdown_connections 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.999+0000 7f1d5b071700 1 -- 172.21.15.3:0/758268765 >> 172.21.15.3:0/758268765 conn(0x7f1d5406f960 msgr2=0x7f1d54070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.999+0000 7f1d5b071700 1 -- 172.21.15.3:0/758268765 shutdown_connections 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:42.999+0000 7f1d5b071700 1 -- 172.21.15.3:0/758268765 wait complete. 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.000+0000 7f1d5b071700 1 Processor -- start 2023-10-06T04:59:43.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.000+0000 7f1d5b071700 1 -- start start 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5b071700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d540c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5b071700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540cd0b0 0x7f1d540cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5b071700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d540c9460 0x7f1d540c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5b071700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d541515b0 con 0x7f1d40003a60 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5b071700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540cda60 con 0x7f1d540c9460 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5b071700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5415abe0 con 0x7f1d540cd0b0 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d540c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d540c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42024/0 (socket says 172.21.15.3:42024) 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 -- 172.21.15.3:0/3576388662 learned_addr learned my addr 172.21.15.3:0/3576388662 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5960e700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d540c9460 0x7f1d540c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d53fff700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540cd0b0 0x7f1d540cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 -- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540cd0b0 msgr2=0x7f1d540cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540cd0b0 0x7f1d540cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 -- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d540c9460 msgr2=0x7f1d540c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d540c9460 0x7f1d540c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 -- 172.21.15.3:0/3576388662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540ca240 con 0x7f1d40003a60 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d53fff700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540cd0b0 0x7f1d540cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d5960e700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d540c9460 0x7f1d540c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.001+0000 7f1d58e0d700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d540c74e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1d40004700 tx=0x7f1d4c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.002+0000 7f1d50ff9700 1 -- 172.21.15.3:0/3576388662 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c022070 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.002+0000 7f1d50ff9700 1 -- 172.21.15.3:0/3576388662 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c00e9e0 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.002+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d540b5440 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.002+0000 7f1d50ff9700 1 -- 172.21.15.3:0/3576388662 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c01c450 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.002+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540b5a00 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.003+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54071680 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.003+0000 7f1d50ff9700 1 -- 172.21.15.3:0/3576388662 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d4c01c800 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.003+0000 7f1d50ff9700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1d34066f30 0x7f1d340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.004+0000 7f1d50ff9700 1 -- 172.21.15.3:0/3576388662 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1d4c085c80 con 0x7f1d40003a60 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.004+0000 7f1d53fff700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1d34066f30 0x7f1d340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:43.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.004+0000 7f1d53fff700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1d34066f30 0x7f1d340693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1d440020a0 tx=0x7f1d44006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:43.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.008+0000 7f1d50ff9700 1 -- 172.21.15.3:0/3576388662 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d4c052f70 con 0x7f1d40003a60 2023-10-06T04:59:43.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.199+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1d540cf050 con 0x7f1d40003a60 2023-10-06T04:59:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.855+0000 7f1d50ff9700 1 -- 172.21.15.3:0/3576388662 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1d4c02b0d0 con 0x7f1d40003a60 2023-10-06T04:59:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.861+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1d34066f30 msgr2=0x7f1d340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.861+0000 7f1d5b071700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1d34066f30 0x7f1d340693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1d440020a0 tx=0x7f1d44006040 comp rx=0 tx=0).stop 2023-10-06T04:59:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.861+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 msgr2=0x7f1d540c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.861+0000 7f1d5b071700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d540c74e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1d40004700 tx=0x7f1d4c01a000 comp rx=0 tx=0).stop 2023-10-06T04:59:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 shutdown_connections 2023-10-06T04:59:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1d34066f30 0x7f1d340693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d540c9460 0x7f1d540c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40003a60 0x7f1d540c74e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 --2- 172.21.15.3:0/3576388662 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d540cd0b0 0x7f1d540cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 >> 172.21.15.3:0/3576388662 conn(0x7f1d5406f960 msgr2=0x7f1d5406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 shutdown_connections 2023-10-06T04:59:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:43.862+0000 7f1d5b071700 1 -- 172.21.15.3:0/3576388662 wait complete. 2023-10-06T04:59:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T04:59:43.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-10-06T04:59:44.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.005+0000 7f92ad856700 1 Processor -- start 2023-10-06T04:59:44.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.005+0000 7f92ad856700 1 -- start start 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92ad856700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92a81575d0 0x7f92a806d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92ad856700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a806dd80 0x7f92a8158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92ad856700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a81509a0 con 0x7f92a80b5d20 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92ad856700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a806e1e0 con 0x7f92a81575d0 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92ad856700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a815af00 con 0x7f92a806dd80 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92a67fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92a81575d0 0x7f92a806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92a6ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f92a80b5d20 0x7f92a815a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39942/0 (socket says 172.21.15.3:39942) 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92a6ffd700 1 -- 172.21.15.3:0/3607461805 learned_addr learned my addr 172.21.15.3:0/3607461805 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.006+0000 7f92a77fe700 1 --2- 172.21.15.3:0/3607461805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a806dd80 0x7f92a8158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a67fc700 1 -- 172.21.15.3:0/3607461805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a806dd80 msgr2=0x7f92a8158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a67fc700 1 --2- 172.21.15.3:0/3607461805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a806dd80 0x7f92a8158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a67fc700 1 -- 172.21.15.3:0/3607461805 >> v1:172.21.15.3:6789/0 conn(0x7f92a80b5d20 legacy=0x7f92a815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a67fc700 1 -- 172.21.15.3:0/3607461805 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a815b1e0 con 0x7f92a81575d0 2023-10-06T04:59:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a5ffb700 1 -- 172.21.15.3:0/3607461805 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011450556 0 0) 0x7f92a81509a0 con 0x7f92a80b5d20 2023-10-06T04:59:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a67fc700 1 --2- 172.21.15.3:0/3607461805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a806d840 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f92900098e0 tx=0x7f92900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ef65c9624d582d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a5ffb700 1 -- 172.21.15.3:0/3607461805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000b640 con 0x7f92a81575d0 2023-10-06T04:59:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a5ffb700 1 -- 172.21.15.3:0/3607461805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929000b7a0 con 0x7f92a81575d0 2023-10-06T04:59:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.007+0000 7f92a5ffb700 1 -- 172.21.15.3:0/3607461805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000bb40 con 0x7f92a81575d0 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 -- 172.21.15.3:0/3607461805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 msgr2=0x7f92a806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 --2- 172.21.15.3:0/3607461805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a806d840 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f92900098e0 tx=0x7f92900091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 -- 172.21.15.3:0/3607461805 shutdown_connections 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 --2- 172.21.15.3:0/3607461805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a806dd80 0x7f92a8158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 --2- 172.21.15.3:0/3607461805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a806d840 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 -- 172.21.15.3:0/3607461805 >> 172.21.15.3:0/3607461805 conn(0x7f92a806ecf0 msgr2=0x7f92a806f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 -- 172.21.15.3:0/3607461805 shutdown_connections 2023-10-06T04:59:44.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.008+0000 7f92ad856700 1 -- 172.21.15.3:0/3607461805 wait complete. 2023-10-06T04:59:44.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.009+0000 7f92ad856700 1 Processor -- start 2023-10-06T04:59:44.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.009+0000 7f92ad856700 1 -- start start 2023-10-06T04:59:44.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92ad856700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a806dd80 0x7f92a80afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92ad856700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b5d20 0x7f92a8147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92ad856700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92ad856700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8151e70 con 0x7f92a81575d0 2023-10-06T04:59:44.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92ad856700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80b2b00 con 0x7f92a806dd80 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92ad856700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80b01a0 con 0x7f92a80b5d20 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a815a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42034/0 (socket says 172.21.15.3:42034) 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 -- 172.21.15.3:0/347124819 learned_addr learned my addr 172.21.15.3:0/347124819 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a6ffd700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a806dd80 0x7f92a80afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a67fc700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b5d20 0x7f92a8147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 -- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b5d20 msgr2=0x7f92a8147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b5d20 0x7f92a8147c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 -- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a806dd80 msgr2=0x7f92a80afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a806dd80 0x7f92a80afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 -- 172.21.15.3:0/347124819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9290009420 con 0x7f92a81575d0 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92a77fe700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a815a230 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9298009e90 tx=0x7f929800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f929effd700 1 -- 172.21.15.3:0/347124819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800cc20 con 0x7f92a81575d0 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f929effd700 1 -- 172.21.15.3:0/347124819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929800f040 con 0x7f92a81575d0 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a80b2540 con 0x7f92a81575d0 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.010+0000 7f929effd700 1 -- 172.21.15.3:0/347124819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298014480 con 0x7f92a81575d0 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.011+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a81520a0 con 0x7f92a81575d0 2023-10-06T04:59:44.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.012+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a804c380 con 0x7f92a81575d0 2023-10-06T04:59:44.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.012+0000 7f929effd700 1 -- 172.21.15.3:0/347124819 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9298014830 con 0x7f92a81575d0 2023-10-06T04:59:44.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.012+0000 7f929effd700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.012+0000 7f929effd700 1 -- 172.21.15.3:0/347124819 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9298084db0 con 0x7f92a81575d0 2023-10-06T04:59:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.013+0000 7f92a6ffd700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.013+0000 7f92a6ffd700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f92a0007fd0 tx=0x7f92a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.017+0000 7f929effd700 1 -- 172.21.15.3:0/347124819 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9298055c50 con 0x7f92a81575d0 2023-10-06T04:59:44.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.215+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f92a815b4c0 con 0x7f92a81575d0 2023-10-06T04:59:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.218+0000 7f929effd700 1 -- 172.21.15.3:0/347124819 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v39) v1 ==== 105+0+0 (secure 0 0 0) 0x7f92980557e0 con 0x7f92a81575d0 2023-10-06T04:59:44.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 msgr2=0x7f92880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f92a0007fd0 tx=0x7f92a0009040 comp rx=0 tx=0).stop 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 msgr2=0x7f92a815a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a815a230 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9298009e90 tx=0x7f929800b040 comp rx=0 tx=0).stop 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 shutdown_connections 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a806dd80 0x7f92a80afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.220+0000 7f92ad856700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92a81575d0 0x7f92a815a230 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.221+0000 7f92ad856700 1 --2- 172.21.15.3:0/347124819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92a80b5d20 0x7f92a8147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.221+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 >> 172.21.15.3:0/347124819 conn(0x7f92a806ecf0 msgr2=0x7f92a8157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:44.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.221+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 shutdown_connections 2023-10-06T04:59:44.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.221+0000 7f92ad856700 1 -- 172.21.15.3:0/347124819 wait complete. 2023-10-06T04:59:44.234 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:59:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.353+0000 7fb3210f6700 1 Processor -- start 2023-10-06T04:59:44.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.353+0000 7fb3210f6700 1 -- start start 2023-10-06T04:59:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb3210f6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb31c0cfb80 0x7fb31c0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb3210f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb31c0afad0 0x7fb31c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb3210f6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0cef10 0x7fb31c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb3210f6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31c15d510 con 0x7fb31c0afad0 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb3210f6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31c15a480 con 0x7fb31c0cfb80 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb3210f6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c15a760 con 0x7fb31c0cef10 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb31a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb31c0afad0 0x7fb31c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb31a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb31c0afad0 0x7fb31c0ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42036/0 (socket says 172.21.15.3:42036) 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb31a59c700 1 -- 172.21.15.3:0/2514302581 learned_addr learned my addr 172.21.15.3:0/2514302581 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb31b59e700 1 --2- 172.21.15.3:0/2514302581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0cef10 0x7fb31c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb31ad9d700 1 --2- 172.21.15.3:0/2514302581 >> v2:172.21.15.153:3300/0 conn(0x7fb31c0cfb80 0x7fb31c0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb31a59c700 1 -- 172.21.15.3:0/2514302581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0cef10 msgr2=0x7fb31c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.354+0000 7fb31a59c700 1 --2- 172.21.15.3:0/2514302581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0cef10 0x7fb31c15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb31a59c700 1 -- 172.21.15.3:0/2514302581 >> v2:172.21.15.153:3300/0 conn(0x7fb31c0cfb80 msgr2=0x7fb31c0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb31a59c700 1 --2- 172.21.15.3:0/2514302581 >> v2:172.21.15.153:3300/0 conn(0x7fb31c0cfb80 0x7fb31c0af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb31a59c700 1 -- 172.21.15.3:0/2514302581 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c15aa40 con 0x7fb31c0afad0 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb31ad9d700 1 --2- 172.21.15.3:0/2514302581 >> v2:172.21.15.153:3300/0 conn(0x7fb31c0cfb80 0x7fb31c0af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb31b59e700 1 --2- 172.21.15.3:0/2514302581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0cef10 0x7fb31c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb31a59c700 1 --2- 172.21.15.3:0/2514302581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0afad0 0x7fb31c0ce9d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0ce1a0 tx=0x7fb304009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3787aa6ef96ecd58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb319d9b700 1 -- 172.21.15.3:0/2514302581 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400b860 con 0x7fb31c0afad0 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb319d9b700 1 -- 172.21.15.3:0/2514302581 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30400b9c0 con 0x7fb31c0afad0 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb319d9b700 1 -- 172.21.15.3:0/2514302581 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400bd60 con 0x7fb31c0afad0 2023-10-06T04:59:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 -- 172.21.15.3:0/2514302581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0afad0 msgr2=0x7fb31c0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 --2- 172.21.15.3:0/2514302581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0afad0 0x7fb31c0ce9d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0ce1a0 tx=0x7fb304009250 comp rx=0 tx=0).stop 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 -- 172.21.15.3:0/2514302581 shutdown_connections 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 --2- 172.21.15.3:0/2514302581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0cef10 0x7fb31c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 --2- 172.21.15.3:0/2514302581 >> v2:172.21.15.153:3300/0 conn(0x7fb31c0cfb80 0x7fb31c0af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 --2- 172.21.15.3:0/2514302581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0afad0 0x7fb31c0ce9d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 -- 172.21.15.3:0/2514302581 >> 172.21.15.3:0/2514302581 conn(0x7fb31c070310 msgr2=0x7fb31c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 -- 172.21.15.3:0/2514302581 shutdown_connections 2023-10-06T04:59:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.355+0000 7fb3210f6700 1 -- 172.21.15.3:0/2514302581 wait complete. 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 Processor -- start 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 -- start start 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0afad0 0x7fb31c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c0cef10 0x7fb31c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0cfb80 0x7fb31c0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0b31e0 con 0x7fb31c0cfb80 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0b34c0 con 0x7fb31c0cef10 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb3210f6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0b37a0 con 0x7fb31c0afad0 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.357+0000 7fb31b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0cfb80 0x7fb31c0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0cfb80 0x7fb31c0b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42050/0 (socket says 172.21.15.3:42050) 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 -- 172.21.15.3:0/1026988533 learned_addr learned my addr 172.21.15.3:0/1026988533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31ad9d700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0afad0 0x7fb31c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31a59c700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c0cef10 0x7fb31c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 -- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0afad0 msgr2=0x7fb31c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0afad0 0x7fb31c0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 -- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c0cef10 msgr2=0x7fb31c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c0cef10 0x7fb31c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 -- 172.21.15.3:0/1026988533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3040094f0 con 0x7fb31c0cfb80 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb31b59e700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0cfb80 0x7fb31c0b1270 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00e870 tx=0x7fb30c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb312ffd700 1 -- 172.21.15.3:0/1026988533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00c430 con 0x7fb31c0cfb80 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb312ffd700 1 -- 172.21.15.3:0/1026988533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00c590 con 0x7fb31c0cfb80 2023-10-06T04:59:44.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb312ffd700 1 -- 172.21.15.3:0/1026988533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00c910 con 0x7fb31c0cfb80 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31c0b3a80 con 0x7fb31c0cfb80 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.358+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c0b4040 con 0x7fb31c0cfb80 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.359+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31c04c380 con 0x7fb31c0cfb80 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.360+0000 7fb312ffd700 1 -- 172.21.15.3:0/1026988533 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb30c00cd40 con 0x7fb31c0cfb80 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.360+0000 7fb312ffd700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb2fc066f30 0x7fb2fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.360+0000 7fb312ffd700 1 -- 172.21.15.3:0/1026988533 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb30c0840f0 con 0x7fb31c0cfb80 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.360+0000 7fb31ad9d700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb2fc066f30 0x7fb2fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.361+0000 7fb31ad9d700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb2fc066f30 0x7fb2fc0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb314007580 tx=0x7fb3140097d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.364+0000 7fb312ffd700 1 -- 172.21.15.3:0/1026988533 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb30c051500 con 0x7fb31c0cfb80 2023-10-06T04:59:44.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.558+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb31c0ce470 con 0x7fb31c0cfb80 2023-10-06T04:59:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.564+0000 7fb312ffd700 1 -- 172.21.15.3:0/1026988533 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb30c054b20 con 0x7fb31c0cfb80 2023-10-06T04:59:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb2fc066f30 msgr2=0x7fb2fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb2fc066f30 0x7fb2fc0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb314007580 tx=0x7fb3140097d0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0cfb80 msgr2=0x7fb31c0b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0cfb80 0x7fb31c0b1270 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00e870 tx=0x7fb30c008040 comp rx=0 tx=0).stop 2023-10-06T04:59:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 shutdown_connections 2023-10-06T04:59:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb2fc066f30 0x7fb2fc0693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb31c0cfb80 0x7fb31c0b1270 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb31c0afad0 0x7fb31c0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 --2- 172.21.15.3:0/1026988533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c0cef10 0x7fb31c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 >> 172.21.15.3:0/1026988533 conn(0x7fb31c070310 msgr2=0x7fb31c0c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 shutdown_connections 2023-10-06T04:59:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.566+0000 7fb3210f6700 1 -- 172.21.15.3:0/1026988533 wait complete. 2023-10-06T04:59:44.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-10-06T04:59:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.702+0000 7f582359d700 1 Processor -- start 2023-10-06T04:59:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.702+0000 7f582359d700 1 -- start start 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.703+0000 7f582359d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.703+0000 7f582359d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f581c0b1b00 0x7f581c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.703+0000 7f582359d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c0691c0 con 0x7f581c068190 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.703+0000 7f582359d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0694a0 con 0x7f581c0b1b00 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.703+0000 7f582359d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c069780 con 0x7f581c067dc0 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.703+0000 7f5821b3a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f581c068190 0x7f581c15d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39948/0 (socket says 172.21.15.3:39948) 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.703+0000 7f5821b3a700 1 -- 172.21.15.3:0/684196610 learned_addr learned my addr 172.21.15.3:0/684196610 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5820b38700 1 --2- 172.21.15.3:0/684196610 >> v2:172.21.15.153:3300/0 conn(0x7f581c0b1b00 0x7f581c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5821339700 1 --2- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f581bfff700 1 -- 172.21.15.3:0/684196610 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301769961 0 0) 0x7f581c0691c0 con 0x7f581c068190 2023-10-06T04:59:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f581bfff700 1 -- 172.21.15.3:0/684196610 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5804003570 con 0x7f581c068190 2023-10-06T04:59:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5820b38700 1 -- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 msgr2=0x7f581c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5820b38700 1 --2- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5820b38700 1 -- 172.21.15.3:0/684196610 >> v1:172.21.15.3:6789/0 conn(0x7f581c068190 legacy=0x7f581c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f581bfff700 1 -- 172.21.15.3:0/684196610 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1232676098 0 0) 0x7f5804003570 con 0x7f581c068190 2023-10-06T04:59:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5820b38700 1 -- 172.21.15.3:0/684196610 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c069a60 con 0x7f581c0b1b00 2023-10-06T04:59:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5821339700 1 --2- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f5820b38700 1 --2- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 0x7f581c15c800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f58100098e0 tx=0x7f58100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8465effe6fa647 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f581bfff700 1 -- 172.21.15.3:0/684196610 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814001470 con 0x7f581c0b1b00 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f581bfff700 1 -- 172.21.15.3:0/684196610 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810011420 con 0x7f581c0b1b00 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.704+0000 7f581bfff700 1 -- 172.21.15.3:0/684196610 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100117c0 con 0x7f581c0b1b00 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 -- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 msgr2=0x7f581c15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 --2- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 0x7f581c15c800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f58100098e0 tx=0x7f58100091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 -- 172.21.15.3:0/684196610 shutdown_connections 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 --2- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 --2- 172.21.15.3:0/684196610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 0x7f581c15c800 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 -- 172.21.15.3:0/684196610 >> 172.21.15.3:0/684196610 conn(0x7f581c0ce270 msgr2=0x7f581c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 -- 172.21.15.3:0/684196610 shutdown_connections 2023-10-06T04:59:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.705+0000 7f582359d700 1 -- 172.21.15.3:0/684196610 wait complete. 2023-10-06T04:59:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.706+0000 7f582359d700 1 Processor -- start 2023-10-06T04:59:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.706+0000 7f582359d700 1 -- start start 2023-10-06T04:59:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f582359d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c16adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f582359d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581c068190 0x7f581c151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f582359d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 0x7f581c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f582359d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c1517b0 con 0x7f581c068190 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f582359d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c13eab0 con 0x7f581c0b1b00 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f582359d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c16a120 con 0x7f581c067dc0 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f5820b38700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581c068190 0x7f581c151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f5820b38700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581c068190 0x7f581c151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42052/0 (socket says 172.21.15.3:42052) 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f5820b38700 1 -- 172.21.15.3:0/2455328860 learned_addr learned my addr 172.21.15.3:0/2455328860 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f5821339700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c16adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.707+0000 7f5821b3a700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 0x7f581c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f5820b38700 1 -- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 msgr2=0x7f581c16adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f5820b38700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c16adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f5820b38700 1 -- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 msgr2=0x7f581c06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f5820b38700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 0x7f581c06e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f5820b38700 1 -- 172.21.15.3:0/2455328860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009a20 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f5820b38700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581c068190 0x7f581c151270 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f58100030c0 tx=0x7f5810016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f58197fa700 1 -- 172.21.15.3:0/2455328860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581001cc90 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f58197fa700 1 -- 172.21.15.3:0/2455328860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581001cdf0 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5810009420 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f58197fa700 1 -- 172.21.15.3:0/2455328860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581001a6a0 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.708+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c16a6e0 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.709+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c13ecb0 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.709+0000 7f58197fa700 1 -- 172.21.15.3:0/2455328860 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f581001aad0 con 0x7f581c068190 2023-10-06T04:59:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.710+0000 7f58197fa700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5808066f30 0x7f58080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.710+0000 7f58197fa700 1 -- 172.21.15.3:0/2455328860 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f5810085c90 con 0x7f581c068190 2023-10-06T04:59:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.710+0000 7f5821339700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5808066f30 0x7f58080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.710+0000 7f5821339700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5808066f30 0x7f58080693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f580c0099f0 tx=0x7f580c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.714+0000 7f58197fa700 1 -- 172.21.15.3:0/2455328860 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f581c13ecb0 con 0x7f581c068190 2023-10-06T04:59:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.908+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f581c04c380 con 0x7f581c068190 2023-10-06T04:59:44.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.912+0000 7f58197fa700 1 -- 172.21.15.3:0/2455328860 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v41) v1 ==== 107+0+0 (secure 0 0 0) 0x7f58100530e0 con 0x7f581c068190 2023-10-06T04:59:44.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5808066f30 msgr2=0x7f58080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5808066f30 0x7f58080693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f580c0099f0 tx=0x7f580c00b040 comp rx=0 tx=0).stop 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581c068190 msgr2=0x7f581c151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581c068190 0x7f581c151270 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f58100030c0 tx=0x7f5810016000 comp rx=0 tx=0).stop 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 shutdown_connections 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5808066f30 0x7f58080693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f581c0b1b00 0x7f581c06e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f581c067dc0 0x7f581c16adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 --2- 172.21.15.3:0/2455328860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581c068190 0x7f581c151270 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 >> 172.21.15.3:0/2455328860 conn(0x7f581c0ce270 msgr2=0x7f581c0b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:44.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 shutdown_connections 2023-10-06T04:59:44.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:44.914+0000 7f582359d700 1 -- 172.21.15.3:0/2455328860 wait complete. 2023-10-06T04:59:44.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-10-06T04:59:45.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.047+0000 7f9023b6a700 1 Processor -- start 2023-10-06T04:59:45.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.047+0000 7f9023b6a700 1 -- start start 2023-10-06T04:59:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9023b6a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f901c0af8a0 0x7f901c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9023b6a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c15a290 con 0x7f901c150690 2023-10-06T04:59:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9023b6a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0d0020 con 0x7f901c0cba50 2023-10-06T04:59:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9023b6a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c159f40 con 0x7f901c0af8a0 2023-10-06T04:59:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9022107700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f901c0af8a0 0x7f901c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9021906700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f901c150690 0x7f901c15a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39956/0 (socket says 172.21.15.3:39956) 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9021906700 1 -- 172.21.15.3:0/2998497830 learned_addr learned my addr 172.21.15.3:0/2998497830 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9022107700 1 -- 172.21.15.3:0/2998497830 >> v1:172.21.15.153:6789/0 conn(0x7f901c0cba50 legacy=0x7f901c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9022107700 1 -- 172.21.15.3:0/2998497830 >> v1:172.21.15.3:6789/0 conn(0x7f901c150690 legacy=0x7f901c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9020904700 1 -- 172.21.15.3:0/2998497830 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414070042 0 0) 0x7f901c15a290 con 0x7f901c150690 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.048+0000 7f9022107700 1 -- 172.21.15.3:0/2998497830 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c15dcb0 con 0x7f901c0af8a0 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9022107700 1 --2- 172.21.15.3:0/2998497830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150a60 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f900c009e60 tx=0x7f900c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d955dc27c4164f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9020904700 1 -- 172.21.15.3:0/2998497830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00e070 con 0x7f901c0af8a0 2023-10-06T04:59:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9020904700 1 -- 172.21.15.3:0/2998497830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c00bbf0 con 0x7f901c0af8a0 2023-10-06T04:59:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9020904700 1 -- 172.21.15.3:0/2998497830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c012430 con 0x7f901c0af8a0 2023-10-06T04:59:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9023b6a700 1 -- 172.21.15.3:0/2998497830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 msgr2=0x7f901c150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9023b6a700 1 --2- 172.21.15.3:0/2998497830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150a60 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f900c009e60 tx=0x7f900c009780 comp rx=0 tx=0).stop 2023-10-06T04:59:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9023b6a700 1 -- 172.21.15.3:0/2998497830 shutdown_connections 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9023b6a700 1 --2- 172.21.15.3:0/2998497830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150a60 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9023b6a700 1 -- 172.21.15.3:0/2998497830 >> 172.21.15.3:0/2998497830 conn(0x7f901c070310 msgr2=0x7f901c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9023b6a700 1 -- 172.21.15.3:0/2998497830 shutdown_connections 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.049+0000 7f9023b6a700 1 -- 172.21.15.3:0/2998497830 wait complete. 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.050+0000 7f9023b6a700 1 Processor -- start 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.050+0000 7f9023b6a700 1 -- start start 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9023b6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9023b6a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f901c0cba50 0x7f901c0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9023b6a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c150690 0x7f901c0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9023b6a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c15bc00 con 0x7f901c0af8a0 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9023b6a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c15b8d0 con 0x7f901c150690 2023-10-06T04:59:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9023b6a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b3fd0 con 0x7f901c0cba50 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9021906700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9021906700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42070/0 (socket says 172.21.15.3:42070) 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.051+0000 7f9021906700 1 -- 172.21.15.3:0/3601722024 learned_addr learned my addr 172.21.15.3:0/3601722024 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9022107700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c150690 0x7f901c0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9021105700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f901c0cba50 0x7f901c0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9021906700 1 -- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f901c0cba50 msgr2=0x7f901c0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9021906700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f901c0cba50 0x7f901c0b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9021906700 1 -- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c150690 msgr2=0x7f901c0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9021906700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c150690 0x7f901c0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9021906700 1 -- 172.21.15.3:0/3601722024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c009a20 con 0x7f901c0af8a0 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9022107700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c150690 0x7f901c0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9021906700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150100 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f901400c5b0 tx=0x7f9014001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9019ffb700 1 -- 172.21.15.3:0/3601722024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014005b30 con 0x7f901c0af8a0 2023-10-06T04:59:45.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9019ffb700 1 -- 172.21.15.3:0/3601722024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9014005c90 con 0x7f901c0af8a0 2023-10-06T04:59:45.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c0b3a10 con 0x7f901c0af8a0 2023-10-06T04:59:45.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9019ffb700 1 -- 172.21.15.3:0/3601722024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014007300 con 0x7f901c0af8a0 2023-10-06T04:59:45.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.052+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c0b3720 con 0x7f901c0af8a0 2023-10-06T04:59:45.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.053+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c04c380 con 0x7f901c0af8a0 2023-10-06T04:59:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.053+0000 7f9019ffb700 1 -- 172.21.15.3:0/3601722024 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f901401b3b0 con 0x7f901c0af8a0 2023-10-06T04:59:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.054+0000 7f9019ffb700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9008066f30 0x7f90080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.054+0000 7f9019ffb700 1 -- 172.21.15.3:0/3601722024 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9014083a10 con 0x7f901c0af8a0 2023-10-06T04:59:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.054+0000 7f9021105700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9008066f30 0x7f90080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.055+0000 7f9021105700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9008066f30 0x7f90080693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f901c16d1f0 tx=0x7f9010008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.058+0000 7f9019ffb700 1 -- 172.21.15.3:0/3601722024 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9014050d80 con 0x7f901c0af8a0 2023-10-06T04:59:45.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.251+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f901c15df90 con 0x7f901c0af8a0 2023-10-06T04:59:45.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.255+0000 7f9019ffb700 1 -- 172.21.15.3:0/3601722024 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v42) v1 ==== 102+0+0 (secure 0 0 0) 0x7f90140170a0 con 0x7f901c0af8a0 2023-10-06T04:59:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9008066f30 msgr2=0x7f90080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9008066f30 0x7f90080693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f901c16d1f0 tx=0x7f9010008040 comp rx=0 tx=0).stop 2023-10-06T04:59:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 msgr2=0x7f901c150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150100 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f901400c5b0 tx=0x7f9014001070 comp rx=0 tx=0).stop 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 shutdown_connections 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9008066f30 0x7f90080693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c150690 0x7f901c0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c0af8a0 0x7f901c150100 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 --2- 172.21.15.3:0/3601722024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f901c0cba50 0x7f901c0b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 >> 172.21.15.3:0/3601722024 conn(0x7f901c070310 msgr2=0x7f901c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 shutdown_connections 2023-10-06T04:59:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.257+0000 7f9023b6a700 1 -- 172.21.15.3:0/3601722024 wait complete. 2023-10-06T04:59:45.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-10-06T04:59:45.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.390+0000 7f4df7968700 1 Processor -- start 2023-10-06T04:59:45.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.390+0000 7f4df7968700 1 -- start start 2023-10-06T04:59:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.391+0000 7f4df7968700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4df006e5c0 0x7f4df0158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.391+0000 7f4df7968700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df006eb00 con 0x7f4df006e120 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.391+0000 7f4df7968700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df006ede0 con 0x7f4df00698f0 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.391+0000 7f4df7968700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df006f0c0 con 0x7f4df006e5c0 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.391+0000 7f4df5f05700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4df006e5c0 0x7f4df0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.391+0000 7f4df4f03700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4df006e120 0x7f4df0157b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39960/0 (socket says 172.21.15.3:39960) 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.391+0000 7f4df4f03700 1 -- 172.21.15.3:0/3170180535 learned_addr learned my addr 172.21.15.3:0/3170180535 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4df5f05700 1 -- 172.21.15.3:0/3170180535 >> v1:172.21.15.153:6789/0 conn(0x7f4df00698f0 legacy=0x7f4df0069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4df5f05700 1 -- 172.21.15.3:0/3170180535 >> v1:172.21.15.3:6789/0 conn(0x7f4df006e120 legacy=0x7f4df0157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4df5f05700 1 -- 172.21.15.3:0/3170180535 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df006f3a0 con 0x7f4df006e5c0 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4deffff700 1 -- 172.21.15.3:0/3170180535 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013434770 0 0) 0x7f4df006eb00 con 0x7f4df006e120 2023-10-06T04:59:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4df5f05700 1 --2- 172.21.15.3:0/3170180535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e5c0 0x7f4df0158210 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009e60 tx=0x7f4de0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6bb446686290bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4deffff700 1 -- 172.21.15.3:0/3170180535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000e070 con 0x7f4df006e5c0 2023-10-06T04:59:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4deffff700 1 -- 172.21.15.3:0/3170180535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de000bbf0 con 0x7f4df006e5c0 2023-10-06T04:59:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.392+0000 7f4deffff700 1 -- 172.21.15.3:0/3170180535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0012430 con 0x7f4df006e5c0 2023-10-06T04:59:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.393+0000 7f4df7968700 1 -- 172.21.15.3:0/3170180535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e5c0 msgr2=0x7f4df0158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.393+0000 7f4df7968700 1 --2- 172.21.15.3:0/3170180535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e5c0 0x7f4df0158210 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009e60 tx=0x7f4de0009780 comp rx=0 tx=0).stop 2023-10-06T04:59:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.393+0000 7f4df7968700 1 -- 172.21.15.3:0/3170180535 shutdown_connections 2023-10-06T04:59:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.393+0000 7f4df7968700 1 --2- 172.21.15.3:0/3170180535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e5c0 0x7f4df0158210 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.393+0000 7f4df7968700 1 -- 172.21.15.3:0/3170180535 >> 172.21.15.3:0/3170180535 conn(0x7f4df006ce40 msgr2=0x7f4df00cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.393+0000 7f4df7968700 1 -- 172.21.15.3:0/3170180535 shutdown_connections 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.393+0000 7f4df7968700 1 -- 172.21.15.3:0/3170180535 wait complete. 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.394+0000 7f4df7968700 1 Processor -- start 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.394+0000 7f4df7968700 1 -- start start 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df00698f0 0x7f4df015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e120 0x7f4df00b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4df006e5c0 0x7f4df0154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df016ace0 con 0x7f4df006e120 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0156590 con 0x7f4df00698f0 2023-10-06T04:59:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0156870 con 0x7f4df006e5c0 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e120 0x7f4df00b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e120 0x7f4df00b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42102/0 (socket says 172.21.15.3:42102) 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 -- 172.21.15.3:0/2691740342 learned_addr learned my addr 172.21.15.3:0/2691740342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df5704700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df00698f0 0x7f4df015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 -- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4df006e5c0 msgr2=0x7f4df0154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4df006e5c0 0x7f4df0154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 -- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df00698f0 msgr2=0x7f4df015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df00698f0 0x7f4df015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 -- 172.21.15.3:0/2691740342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0009a20 con 0x7f4df006e120 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df4f03700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e120 0x7f4df00b1010 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c7e0 tx=0x7f4de4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4ded7fa700 1 -- 172.21.15.3:0/2691740342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4005b30 con 0x7f4df006e120 2023-10-06T04:59:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4ded7fa700 1 -- 172.21.15.3:0/2691740342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de4005c90 con 0x7f4df006e120 2023-10-06T04:59:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df0156b50 con 0x7f4df006e120 2023-10-06T04:59:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4ded7fa700 1 -- 172.21.15.3:0/2691740342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4007300 con 0x7f4df006e120 2023-10-06T04:59:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.395+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df015aef0 con 0x7f4df006e120 2023-10-06T04:59:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.396+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df016ae50 con 0x7f4df006e120 2023-10-06T04:59:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.397+0000 7f4ded7fa700 1 -- 172.21.15.3:0/2691740342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4de401b400 con 0x7f4df006e120 2023-10-06T04:59:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.397+0000 7f4ded7fa700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4ddc066f30 0x7f4ddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.397+0000 7f4ded7fa700 1 -- 172.21.15.3:0/2691740342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4de4083ea0 con 0x7f4df006e120 2023-10-06T04:59:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.397+0000 7f4df5704700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4ddc066f30 0x7f4ddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.398+0000 7f4df5704700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4ddc066f30 0x7f4ddc0693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4de80027e0 tx=0x7f4de8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.401+0000 7f4ded7fa700 1 -- 172.21.15.3:0/2691740342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4df016ae50 con 0x7f4df006e120 2023-10-06T04:59:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.595+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f4df016ae20 con 0x7f4df006e120 2023-10-06T04:59:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.598+0000 7f4ded7fa700 1 -- 172.21.15.3:0/2691740342 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v43) v1 ==== 111+0+0 (secure 0 0 0) 0x7f4de4051330 con 0x7f4df006e120 2023-10-06T04:59:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4ddc066f30 msgr2=0x7f4ddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4ddc066f30 0x7f4ddc0693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4de80027e0 tx=0x7f4de8009040 comp rx=0 tx=0).stop 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e120 msgr2=0x7f4df00b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e120 0x7f4df00b1010 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c7e0 tx=0x7f4de4007df0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 shutdown_connections 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4ddc066f30 0x7f4ddc0693e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df00698f0 0x7f4df015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4df006e5c0 0x7f4df0154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 --2- 172.21.15.3:0/2691740342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4df006e120 0x7f4df00b1010 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 >> 172.21.15.3:0/2691740342 conn(0x7f4df006ce40 msgr2=0x7f4df0150e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 shutdown_connections 2023-10-06T04:59:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.601+0000 7f4df7968700 1 -- 172.21.15.3:0/2691740342 wait complete. 2023-10-06T04:59:45.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2023-10-06T04:59:45.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.732+0000 7f75b8dfd700 1 Processor -- start 2023-10-06T04:59:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.732+0000 7f75b8dfd700 1 -- start start 2023-10-06T04:59:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b8dfd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f75b40b3d40 0x7f75b4164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b8dfd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b415aeb0 con 0x7f75b4157400 2023-10-06T04:59:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b8dfd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b415a7a0 con 0x7f75b4157030 2023-10-06T04:59:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b8dfd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b415aa80 con 0x7f75b40b3d40 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75abfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f75b4157400 0x7f75b40b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39970/0 (socket says 172.21.15.3:39970) 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75abfff700 1 -- 172.21.15.3:0/666010422 learned_addr learned my addr 172.21.15.3:0/666010422 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b2d9d700 1 --2- 172.21.15.3:0/666010422 >> v2:172.21.15.153:3300/0 conn(0x7f75b40b3d40 0x7f75b4164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889302998 0 0) 0x7f75b415aeb0 con 0x7f75b4157400 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a0003570 con 0x7f75b4157400 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100777526 0 0) 0x7f75a0003570 con 0x7f75b4157400 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 >> v2:172.21.15.153:3300/0 conn(0x7f75b40b3d40 msgr2=0x7f75b4164320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 --2- 172.21.15.3:0/666010422 >> v2:172.21.15.153:3300/0 conn(0x7f75b40b3d40 0x7f75b4164320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 >> v1:172.21.15.153:6789/0 conn(0x7f75b4157030 legacy=0x7f75b4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b406dba0 con 0x7f75b4157400 2023-10-06T04:59:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f759c001470 con 0x7f75b4157400 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 >> v1:172.21.15.3:6789/0 conn(0x7f75b4157400 legacy=0x7f75b40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a0003a60 0x7f75a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.733+0000 7f75b1d9b700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75a00051c0 0x7f75a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b415aa80 con 0x7f75a0004810 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a000c0d0 con 0x7f75a0003a60 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a000c290 con 0x7f75a00051c0 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75abfff700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b259c700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a0003a60 0x7f75a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b2d9d700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75a00051c0 0x7f75a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75abfff700 1 -- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75a00051c0 msgr2=0x7f75a000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75abfff700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75a00051c0 0x7f75a000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75abfff700 1 -- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a0003a60 msgr2=0x7f75a0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75abfff700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a0003a60 0x7f75a0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75abfff700 1 -- 172.21.15.3:0/666010422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a0003570 con 0x7f75a0004810 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b2d9d700 1 -- 172.21.15.3:0/666010422 reap_dead start 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75abfff700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75a0004c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f759c00c390 tx=0x7f75b40b2be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ab9863d0974efe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00cc60 con 0x7f75a0004810 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c00cdc0 con 0x7f75a0004810 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.734+0000 7f75b1d9b700 1 -- 172.21.15.3:0/666010422 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c005b70 con 0x7f75a0004810 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.735+0000 7f75b8dfd700 1 -- 172.21.15.3:0/666010422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 msgr2=0x7f75a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.735+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75a0004c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f759c00c390 tx=0x7f75b40b2be0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.735+0000 7f75b8dfd700 1 -- 172.21.15.3:0/666010422 shutdown_connections 2023-10-06T04:59:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.735+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/666010422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75a0004c80 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.735+0000 7f75b8dfd700 1 -- 172.21.15.3:0/666010422 >> 172.21.15.3:0/666010422 conn(0x7f75b40ce270 msgr2=0x7f75b40ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.735+0000 7f75b8dfd700 1 -- 172.21.15.3:0/666010422 shutdown_connections 2023-10-06T04:59:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.735+0000 7f75b8dfd700 1 -- 172.21.15.3:0/666010422 wait complete. 2023-10-06T04:59:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.736+0000 7f75b8dfd700 1 Processor -- start 2023-10-06T04:59:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.736+0000 7f75b8dfd700 1 -- start start 2023-10-06T04:59:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75b4153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b415e300 0x7f75b415e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75b415c030 0x7f75b415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b415ecb0 con 0x7f75a0004810 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b415d970 con 0x7f75b415e300 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b415cd60 con 0x7f75b415c030 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75b4153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75b4153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42118/0 (socket says 172.21.15.3:42118) 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 -- 172.21.15.3:0/3747261738 learned_addr learned my addr 172.21.15.3:0/3747261738 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75abfff700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b415e300 0x7f75b415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b2d9d700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75b415c030 0x7f75b415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 -- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75b415c030 msgr2=0x7f75b415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75b415c030 0x7f75b415c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 -- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b415e300 msgr2=0x7f75b415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b415e300 0x7f75b415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 -- 172.21.15.3:0/3747261738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b415d040 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75abfff700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b415e300 0x7f75b415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b259c700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75b4153e10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f75a4002950 tx=0x7f75a4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75aa7fc700 1 -- 172.21.15.3:0/3747261738 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400f850 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75aa7fc700 1 -- 172.21.15.3:0/3747261738 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a400f9b0 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b415d320 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75aa7fc700 1 -- 172.21.15.3:0/3747261738 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400fd50 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.737+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b4154350 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.738+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b40644c0 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.739+0000 7f75aa7fc700 1 -- 172.21.15.3:0/3747261738 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75a4013600 con 0x7f75a0004810 2023-10-06T04:59:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.739+0000 7f75aa7fc700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7594066f30 0x7f75940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.739+0000 7f75aa7fc700 1 -- 172.21.15.3:0/3747261738 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f75a4084850 con 0x7f75a0004810 2023-10-06T04:59:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.739+0000 7f75abfff700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7594066f30 0x7f75940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.740+0000 7f75abfff700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7594066f30 0x7f75940693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f75b4155b00 tx=0x7f759c015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:45.743+0000 7f75aa7fc700 1 -- 172.21.15.3:0/3747261738 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75a4051c60 con 0x7f75a0004810 2023-10-06T04:59:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.014+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) v1 -- 0x7f75b4071680 con 0x7f75a0004810 2023-10-06T04:59:46.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.857+0000 7f75aa7fc700 1 -- 172.21.15.3:0/3747261738 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v373) v1 ==== 143+0+0 (secure 0 0 0) 0x7f75a4055280 con 0x7f75a0004810 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7594066f30 msgr2=0x7f75940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7594066f30 0x7f75940693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f75b4155b00 tx=0x7f759c015000 comp rx=0 tx=0).stop 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 msgr2=0x7f75b4153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75b4153e10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f75a4002950 tx=0x7f75a4007d30 comp rx=0 tx=0).stop 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 shutdown_connections 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7594066f30 0x7f75940693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f75b415c030 0x7f75b415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75a0004810 0x7f75b4153e10 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 --2- 172.21.15.3:0/3747261738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b415e300 0x7f75b415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:46.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 >> 172.21.15.3:0/3747261738 conn(0x7f75b40ce270 msgr2=0x7f75b41578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:46.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 shutdown_connections 2023-10-06T04:59:46.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:46.860+0000 7f75b8dfd700 1 -- 172.21.15.3:0/3747261738 wait complete. 2023-10-06T04:59:46.877 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-10-06T04:59:46.877 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T04:59:46.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2023-10-06T04:59:47.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.002+0000 7f2ab095e700 1 Processor -- start 2023-10-06T04:59:47.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.002+0000 7f2ab095e700 1 -- start start 2023-10-06T04:59:47.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.002+0000 7f2ab095e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2aac0b2910 0x7f2aac1642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:47.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2ab095e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac0b0ce0 con 0x7f2aac0cd950 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2ab095e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac0cb820 con 0x7f2aac15cb50 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2ab095e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac0cccc0 con 0x7f2aac0b2910 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aaad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2aac0b2910 0x7f2aac1642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aa9d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2aac0cd950 0x7f2aac0b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39972/0 (socket says 172.21.15.3:39972) 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aa9d9b700 1 -- 172.21.15.3:0/2965551488 learned_addr learned my addr 172.21.15.3:0/2965551488 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aaad9d700 1 -- 172.21.15.3:0/2965551488 >> v1:172.21.15.153:6789/0 conn(0x7f2aac15cb50 legacy=0x7f2aac0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aaad9d700 1 -- 172.21.15.3:0/2965551488 >> v1:172.21.15.3:6789/0 conn(0x7f2aac0cd950 legacy=0x7f2aac0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aaad9d700 1 -- 172.21.15.3:0/2965551488 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0b0760 con 0x7f2aac0b2910 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aa959a700 1 -- 172.21.15.3:0/2965551488 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470369225 0 0) 0x7f2aac0b0ce0 con 0x7f2aac0cd950 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aaad9d700 1 --2- 172.21.15.3:0/2965551488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac0b2910 0x7f2aac1642a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0098e0 tx=0x7f2a9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0b694ab22e9192 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aa959a700 1 -- 172.21.15.3:0/2965551488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c009e80 con 0x7f2aac0b2910 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aa959a700 1 -- 172.21.15.3:0/2965551488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c00b860 con 0x7f2aac0b2910 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.003+0000 7f2aa959a700 1 -- 172.21.15.3:0/2965551488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00bbe0 con 0x7f2aac0b2910 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.004+0000 7f2ab095e700 1 -- 172.21.15.3:0/2965551488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac0b2910 msgr2=0x7f2aac1642a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.004+0000 7f2ab095e700 1 --2- 172.21.15.3:0/2965551488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac0b2910 0x7f2aac1642a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0098e0 tx=0x7f2a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.004+0000 7f2ab095e700 1 -- 172.21.15.3:0/2965551488 shutdown_connections 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.004+0000 7f2ab095e700 1 --2- 172.21.15.3:0/2965551488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac0b2910 0x7f2aac1642a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.004+0000 7f2ab095e700 1 -- 172.21.15.3:0/2965551488 >> 172.21.15.3:0/2965551488 conn(0x7f2aac070310 msgr2=0x7f2aac070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.004+0000 7f2ab095e700 1 -- 172.21.15.3:0/2965551488 shutdown_connections 2023-10-06T04:59:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.004+0000 7f2ab095e700 1 -- 172.21.15.3:0/2965551488 wait complete. 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.005+0000 7f2ab095e700 1 Processor -- start 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.005+0000 7f2ab095e700 1 -- start start 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2ab095e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aac0b2910 0x7f2aac150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2ab095e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aac0cd950 0x7f2aac0c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2ab095e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac15cb50 0x7f2aac0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2ab095e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c5030 con 0x7f2aac15cb50 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2ab095e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0ce1b0 con 0x7f2aac0b2910 2023-10-06T04:59:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2ab095e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac1510d0 con 0x7f2aac0cd950 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac15cb50 0x7f2aac0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac15cb50 0x7f2aac0d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42130/0 (socket says 172.21.15.3:42130) 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 -- 172.21.15.3:0/865800567 learned_addr learned my addr 172.21.15.3:0/865800567 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaa59c700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aac0b2910 0x7f2aac150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aa9d9b700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aac0cd950 0x7f2aac0c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 -- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aac0cd950 msgr2=0x7f2aac0c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aac0cd950 0x7f2aac0c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 -- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aac0b2910 msgr2=0x7f2aac150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aac0b2910 0x7f2aac150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 -- 172.21.15.3:0/865800567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c009420 con 0x7f2aac15cb50 2023-10-06T04:59:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aa9d9b700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aac0cd950 0x7f2aac0c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aaad9d700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac15cb50 0x7f2aac0d07b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2aac064a20 tx=0x7f2a9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.006+0000 7f2aa27fc700 1 -- 172.21.15.3:0/865800567 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c01f070 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.007+0000 7f2aa27fc700 1 -- 172.21.15.3:0/865800567 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c011420 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.007+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac0c4790 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.007+0000 7f2aa27fc700 1 -- 172.21.15.3:0/865800567 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0117e0 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.007+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac0c4d50 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.007+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac0cdf80 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.008+0000 7f2aa27fc700 1 -- 172.21.15.3:0/865800567 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a9c011b50 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.008+0000 7f2aa27fc700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a8c066f30 0x7f2a8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.008+0000 7f2aa27fc700 1 -- 172.21.15.3:0/865800567 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2a9c085a60 con 0x7f2aac15cb50 2023-10-06T04:59:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.008+0000 7f2aaa59c700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a8c066f30 0x7f2a8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.009+0000 7f2aaa59c700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a8c066f30 0x7f2a8c0693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400c580 tx=0x7f2aa4001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.013+0000 7f2aa27fc700 1 -- 172.21.15.3:0/865800567 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2aac0cdf80 con 0x7f2aac15cb50 2023-10-06T04:59:47.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.284+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) v1 -- 0x7f2aac0cdf80 con 0x7f2aac15cb50 2023-10-06T04:59:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.862+0000 7f2aa27fc700 1 -- 172.21.15.3:0/865800567 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v374) v1 ==== 154+0+0 (secure 0 0 0) 0x7f2aac0cdf80 con 0x7f2aac15cb50 2023-10-06T04:59:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a8c066f30 msgr2=0x7f2a8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a8c066f30 0x7f2a8c0693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400c580 tx=0x7f2aa4001a80 comp rx=0 tx=0).stop 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac15cb50 msgr2=0x7f2aac0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac15cb50 0x7f2aac0d07b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2aac064a20 tx=0x7f2a9c017000 comp rx=0 tx=0).stop 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 shutdown_connections 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a8c066f30 0x7f2a8c0693e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aac0b2910 0x7f2aac150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac15cb50 0x7f2aac0d07b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 --2- 172.21.15.3:0/865800567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aac0cd950 0x7f2aac0c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.864+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 >> 172.21.15.3:0/865800567 conn(0x7f2aac070310 msgr2=0x7f2aac15ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.865+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 shutdown_connections 2023-10-06T04:59:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:47.865+0000 7f2ab095e700 1 -- 172.21.15.3:0/865800567 wait complete. 2023-10-06T04:59:47.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:59:48.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.007+0000 7f6aded44700 1 Processor -- start 2023-10-06T04:59:48.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.007+0000 7f6aded44700 1 -- start start 2023-10-06T04:59:48.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6aded44700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 0x7f6ad80c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6aded44700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad80cb190 con 0x7f6ad80af290 2023-10-06T04:59:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6aded44700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad80c65a0 con 0x7f6ad80af660 2023-10-06T04:59:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6aded44700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8159950 con 0x7f6ad80c5c20 2023-10-06T04:59:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6addd42700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ad80af290 0x7f6ad80ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39986/0 (socket says 172.21.15.3:39986) 2023-10-06T04:59:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6addd42700 1 -- 172.21.15.3:0/2505642092 learned_addr learned my addr 172.21.15.3:0/2505642092 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6ade543700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 0x7f6ad80c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.008+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351852247 0 0) 0x7f6ad80cb190 con 0x7f6ad80af290 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac0003570 con 0x7f6ad80af290 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490796210 0 0) 0x7f6ac0003570 con 0x7f6ad80af290 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 msgr2=0x7f6ad80c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 0x7f6ad80c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 >> v1:172.21.15.153:6789/0 conn(0x7f6ad80af660 legacy=0x7f6ad80ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80cd6a0 con 0x7f6ad80af290 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6ad00015f0 con 0x7f6ad80af290 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 >> v1:172.21.15.3:6789/0 conn(0x7f6ad80af290 legacy=0x7f6ad80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ac0003a90 0x7f6ac0003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 0x7f6ac0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8159950 con 0x7f6ac0004810 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac000bb90 con 0x7f6ac0003a90 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac000bd50 con 0x7f6ad80c5c20 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6add541700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6addd42700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ac0003a90 0x7f6ac0003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6ade543700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 0x7f6ac0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6add541700 1 -- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 msgr2=0x7f6ac0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6add541700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 0x7f6ac0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6add541700 1 -- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ac0003a90 msgr2=0x7f6ac0003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6add541700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ac0003a90 0x7f6ac0003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6add541700 1 -- 172.21.15.3:0/2505642092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac0003570 con 0x7f6ac0004810 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6ade543700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad80c5c20 0x7f6ac0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.009+0000 7f6add541700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80afae0 tx=0x7f6acc007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d0451ad592e003 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc012420 con 0x7f6ac0004810 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc012580 con 0x7f6ac0004810 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6adcd40700 1 -- 172.21.15.3:0/2505642092 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc012940 con 0x7f6ac0004810 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6aded44700 1 -- 172.21.15.3:0/2505642092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ac0004810 msgr2=0x7f6ac0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6aded44700 1 --2- 172.21.15.3:0/2505642092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80afae0 tx=0x7f6acc007d80 comp rx=0 tx=0).stop 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6ade543700 1 -- 172.21.15.3:0/2505642092 reap_dead start 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6aded44700 1 -- 172.21.15.3:0/2505642092 shutdown_connections 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6aded44700 1 -- 172.21.15.3:0/2505642092 >> 172.21.15.3:0/2505642092 conn(0x7f6ad8089dd0 msgr2=0x7f6ad80996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6aded44700 1 -- 172.21.15.3:0/2505642092 shutdown_connections 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.010+0000 7f6aded44700 1 -- 172.21.15.3:0/2505642092 wait complete. 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.011+0000 7f6aded44700 1 Processor -- start 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.011+0000 7f6aded44700 1 -- start start 2023-10-06T04:59:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6aded44700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ac0004810 0x7f6ad8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6aded44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad816a450 0x7f6ad80b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6aded44700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad80b4560 0x7f6ad80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6aded44700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80b1d70 con 0x7f6ad816a450 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6aded44700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80b0290 con 0x7f6ad80b4560 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6aded44700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8155660 con 0x7f6ac0004810 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad816a450 0x7f6ad80b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad816a450 0x7f6ad80b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42148/0 (socket says 172.21.15.3:42148) 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 -- 172.21.15.3:0/4218261601 learned_addr learned my addr 172.21.15.3:0/4218261601 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6ade543700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad80b4560 0x7f6ad80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6addd42700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ac0004810 0x7f6ad8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 -- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ac0004810 msgr2=0x7f6ad8155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ac0004810 0x7f6ad8155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 -- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad80b4560 msgr2=0x7f6ad80ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad80b4560 0x7f6ad80ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.012+0000 7f6add541700 1 -- 172.21.15.3:0/4218261601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80b1a90 con 0x7f6ad816a450 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.013+0000 7f6add541700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad816a450 0x7f6ad80b4020 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000bb60 tx=0x7f6acc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.013+0000 7f6ad5ffb700 1 -- 172.21.15.3:0/4218261601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc012cb0 con 0x7f6ad816a450 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.013+0000 7f6ad5ffb700 1 -- 172.21.15.3:0/4218261601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc012e10 con 0x7f6ad816a450 2023-10-06T04:59:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.013+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad80afc40 con 0x7f6ad816a450 2023-10-06T04:59:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.013+0000 7f6ad5ffb700 1 -- 172.21.15.3:0/4218261601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc012940 con 0x7f6ad816a450 2023-10-06T04:59:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.013+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad80cef00 con 0x7f6ad816a450 2023-10-06T04:59:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.014+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad804c380 con 0x7f6ad816a450 2023-10-06T04:59:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.015+0000 7f6ad5ffb700 1 -- 172.21.15.3:0/4218261601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6acc006a80 con 0x7f6ad816a450 2023-10-06T04:59:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.015+0000 7f6ad5ffb700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ac4066f30 0x7f6ac40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.015+0000 7f6ad5ffb700 1 -- 172.21.15.3:0/4218261601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f6acc086130 con 0x7f6ad816a450 2023-10-06T04:59:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.016+0000 7f6addd42700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ac4066f30 0x7f6ac40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.016+0000 7f6addd42700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ac4066f30 0x7f6ac40693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80ca850 tx=0x7f6ad0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.019+0000 7f6ad5ffb700 1 -- 172.21.15.3:0/4218261601 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6acc0535e0 con 0x7f6ad816a450 2023-10-06T04:59:48.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.212+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6ad8159260 con 0x7f6ad816a450 2023-10-06T04:59:48.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.215+0000 7f6ad5ffb700 1 -- 172.21.15.3:0/4218261601 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6acc0240c0 con 0x7f6ad816a450 2023-10-06T04:59:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ac4066f30 msgr2=0x7f6ac40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ac4066f30 0x7f6ac40693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80ca850 tx=0x7f6ad0009040 comp rx=0 tx=0).stop 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad816a450 msgr2=0x7f6ad80b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad816a450 0x7f6ad80b4020 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000bb60 tx=0x7f6acc015000 comp rx=0 tx=0).stop 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 shutdown_connections 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ac4066f30 0x7f6ac40693e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad80b4560 0x7f6ad80ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ac0004810 0x7f6ad8155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 --2- 172.21.15.3:0/4218261601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad816a450 0x7f6ad80b4020 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 >> 172.21.15.3:0/4218261601 conn(0x7f6ad8089dd0 msgr2=0x7f6ad806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 shutdown_connections 2023-10-06T04:59:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.218+0000 7f6aded44700 1 -- 172.21.15.3:0/4218261601 wait complete. 2023-10-06T04:59:48.231 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:59:48.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:59:48.270 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:59:48.317 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:59:48.318 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:59:48.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:48.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.436+0000 7f9868e74700 1 Processor -- start 2023-10-06T04:59:48.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.436+0000 7f9868e74700 1 -- start start 2023-10-06T04:59:48.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f9868e74700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 0x7f9864158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f9868e74700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640cf130 con 0x7f9864159500 2023-10-06T04:59:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f9868e74700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640ce520 con 0x7f98640cb3b0 2023-10-06T04:59:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f9868e74700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98640ce800 con 0x7f98640cb780 2023-10-06T04:59:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f9863fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9864159500 0x7f98640cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39994/0 (socket says 172.21.15.3:39994) 2023-10-06T04:59:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f9863fff700 1 -- 172.21.15.3:0/2442952989 learned_addr learned my addr 172.21.15.3:0/2442952989 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f9862ffd700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 0x7f9864158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186609150 0 0) 0x7f98640cf130 con 0x7f9864159500 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.437+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f984c003570 con 0x7f9864159500 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 384414242 0 0) 0x7f984c003570 con 0x7f9864159500 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 msgr2=0x7f9864158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 0x7f9864158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 >> v1:172.21.15.153:6789/0 conn(0x7f98640cb3b0 legacy=0x7f98640cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98640ceae0 con 0x7f9864159500 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f985c0017f0 con 0x7f9864159500 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 >> v1:172.21.15.3:6789/0 conn(0x7f9864159500 legacy=0x7f98640cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f984c003a90 0x7f984c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f984c004810 0x7f984c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 0x7f984c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640ce800 con 0x7f984c003a90 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c00bb10 con 0x7f984c004810 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c00bcd0 con 0x7f98640cb780 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98637fe700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f984c003a90 0x7f984c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f9863fff700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 0x7f984c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f9862ffd700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f984c004810 0x7f984c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98637fe700 1 -- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 msgr2=0x7f984c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98637fe700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 0x7f984c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98637fe700 1 -- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f984c004810 msgr2=0x7f984c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98637fe700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f984c004810 0x7f984c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98637fe700 1 -- 172.21.15.3:0/2442952989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c003570 con 0x7f984c003a90 2023-10-06T04:59:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f9863fff700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98640cb780 0x7f984c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f9862ffd700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f984c004810 0x7f984c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.438+0000 7f98637fe700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f984c003a90 0x7f984c003e60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f98540019a0 tx=0x7f9854007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675cb440510a8cc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400c5d0 con 0x7f984c003a90 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985400c730 con 0x7f984c003a90 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f98627fc700 1 -- 172.21.15.3:0/2442952989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400cad0 con 0x7f984c003a90 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f9868e74700 1 -- 172.21.15.3:0/2442952989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f984c003a90 msgr2=0x7f984c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f9868e74700 1 --2- 172.21.15.3:0/2442952989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f984c003a90 0x7f984c003e60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f98540019a0 tx=0x7f9854007d10 comp rx=0 tx=0).stop 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f9863fff700 1 -- 172.21.15.3:0/2442952989 reap_dead start 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f9868e74700 1 -- 172.21.15.3:0/2442952989 shutdown_connections 2023-10-06T04:59:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f9868e74700 1 -- 172.21.15.3:0/2442952989 >> 172.21.15.3:0/2442952989 conn(0x7f9864070310 msgr2=0x7f9864070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:48.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f9868e74700 1 -- 172.21.15.3:0/2442952989 shutdown_connections 2023-10-06T04:59:48.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.439+0000 7f9868e74700 1 -- 172.21.15.3:0/2442952989 wait complete. 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.440+0000 7f9868e74700 1 Processor -- start 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9868e74700 1 -- start start 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9868e74700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f984c003a90 0x7f98640b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9868e74700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98641574b0 0x7f98640c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9868e74700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98640c7b60 0x7f98640c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9868e74700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640c8540 con 0x7f98640c7b60 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9868e74700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9864155f40 con 0x7f98641574b0 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9868e74700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9864155980 con 0x7f984c003a90 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9863fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98640c7b60 0x7f98640c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9863fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98640c7b60 0x7f98640c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42166/0 (socket says 172.21.15.3:42166) 2023-10-06T04:59:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9863fff700 1 -- 172.21.15.3:0/201538121 learned_addr learned my addr 172.21.15.3:0/201538121 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f9862ffd700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98641574b0 0x7f98640c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.441+0000 7f98637fe700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f984c003a90 0x7f98640b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9863fff700 1 -- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f984c003a90 msgr2=0x7f98640b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9863fff700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f984c003a90 0x7f98640b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9863fff700 1 -- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98641574b0 msgr2=0x7f98640c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9863fff700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98641574b0 0x7f98640c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9863fff700 1 -- 172.21.15.3:0/201538121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864155c60 con 0x7f98640c7b60 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9863fff700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98640c7b60 0x7f98640c8000 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f98640cf0a0 tx=0x7f985c002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f984b7fe700 1 -- 172.21.15.3:0/201538121 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c007b60 con 0x7f98640c7b60 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f984b7fe700 1 -- 172.21.15.3:0/201538121 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00eab0 con 0x7f98640c7b60 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9864156cb0 con 0x7f98640c7b60 2023-10-06T04:59:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f984b7fe700 1 -- 172.21.15.3:0/201538121 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00ee70 con 0x7f98640c7b60 2023-10-06T04:59:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.442+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9864151060 con 0x7f98640c7b60 2023-10-06T04:59:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.443+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98641571d0 con 0x7f98640c7b60 2023-10-06T04:59:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.443+0000 7f984b7fe700 1 -- 172.21.15.3:0/201538121 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f985c013630 con 0x7f98640c7b60 2023-10-06T04:59:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.444+0000 7f984b7fe700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9850066f30 0x7f98500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.444+0000 7f984b7fe700 1 -- 172.21.15.3:0/201538121 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f985c0847a0 con 0x7f98640c7b60 2023-10-06T04:59:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.444+0000 7f98637fe700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9850066f30 0x7f98500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.444+0000 7f98637fe700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9850066f30 0x7f98500693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9854000fd0 tx=0x7f9854015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.448+0000 7f984b7fe700 1 -- 172.21.15.3:0/201538121 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98641571d0 con 0x7f98640c7b60 2023-10-06T04:59:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.686+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98641571d0 con 0x7f98640c7b60 2023-10-06T04:59:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.687+0000 7f984b7fe700 1 -- 172.21.15.3:0/201538121 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f98641571d0 con 0x7f98640c7b60 2023-10-06T04:59:48.688 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:48.689 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":63,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2116670403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2116670403},{"type":"v1","addr":"172.21.15.153:6836","nonce":2116670403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/787495346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":787495346},{"type":"v1","addr":"172.21.15.153:6837","nonce":787495346}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2512441313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2512441313},{"type":"v1","addr":"172.21.15.3:6834","nonce":2512441313}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7062,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/251829181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":251829181},{"type":"v1","addr":"172.21.15.3:6835","nonce":251829181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[]} 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9850066f30 msgr2=0x7f98500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9850066f30 0x7f98500693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9854000fd0 tx=0x7f9854015000 comp rx=0 tx=0).stop 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98640c7b60 msgr2=0x7f98640c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98640c7b60 0x7f98640c8000 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f98640cf0a0 tx=0x7f985c002160 comp rx=0 tx=0).stop 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 shutdown_connections 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9850066f30 0x7f98500693e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98640c7b60 0x7f98640c8000 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f984c003a90 0x7f98640b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 --2- 172.21.15.3:0/201538121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98641574b0 0x7f98640c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 >> 172.21.15.3:0/201538121 conn(0x7f9864070310 msgr2=0x7f98640b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 shutdown_connections 2023-10-06T04:59:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.689+0000 7f9868e74700 1 -- 172.21.15.3:0/201538121 wait complete. 2023-10-06T04:59:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 63 2023-10-06T04:59:48.703 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:59:48.703 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:59:48.703 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:59:48.704 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:59:48.704 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:59:48.704 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:59:48.704 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:59:48.704 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:59:48.704 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:59:48.705 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:59:48.705 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:59:48.705 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:59:48.705 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:59:48.705 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:59:48.705 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:59:48.705 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:59:48.712 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:59:48.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:59:48.713 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:59:48.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:59:48.715 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:59:48.716 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:59:48.716 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:59:48.716 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:59:48.754 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:59:48.755 DEBUG:teuthology.parallel:result is None 2023-10-06T04:59:48.755 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:59:48.755 DEBUG:teuthology.parallel:result is None 2023-10-06T04:59:48.758 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:59:48.759 DEBUG:teuthology.parallel:result is None 2023-10-06T04:59:48.759 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:59:48.759 DEBUG:teuthology.parallel:result is None 2023-10-06T04:59:48.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:59:48.831 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T04:59:48.831 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T04:59:48.839 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T04:59:48.839 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T04:59:48.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.890+0000 7ff6d8452700 1 Processor -- start 2023-10-06T04:59:48.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.890+0000 7ff6d8452700 1 -- start start 2023-10-06T04:59:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d8452700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6d00b3a40 0x7ff6d00ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d8452700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6d00b3e10 0x7ff6d00680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d8452700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d00685f0 con 0x7ff6d01406b0 2023-10-06T04:59:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d8452700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d00cd380 con 0x7ff6d00b3a40 2023-10-06T04:59:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d8452700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d00cd660 con 0x7ff6d00b3e10 2023-10-06T04:59:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d61ee700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6d00b3a40 0x7ff6d00ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d69ef700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff6d01406b0 0x7ff6d0140b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40026/0 (socket says 172.21.15.3:40026) 2023-10-06T04:59:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d69ef700 1 -- 172.21.15.3:0/2324204250 learned_addr learned my addr 172.21.15.3:0/2324204250 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d59ed700 1 --2- 172.21.15.3:0/2324204250 >> v2:172.21.15.153:3300/0 conn(0x7ff6d00b3e10 0x7ff6d00680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d61ee700 1 -- 172.21.15.3:0/2324204250 >> v2:172.21.15.153:3300/0 conn(0x7ff6d00b3e10 msgr2=0x7ff6d00680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d61ee700 1 --2- 172.21.15.3:0/2324204250 >> v2:172.21.15.153:3300/0 conn(0x7ff6d00b3e10 0x7ff6d00680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d61ee700 1 -- 172.21.15.3:0/2324204250 >> v1:172.21.15.3:6789/0 conn(0x7ff6d01406b0 legacy=0x7ff6d0140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d61ee700 1 -- 172.21.15.3:0/2324204250 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d00cd940 con 0x7ff6d00b3a40 2023-10-06T04:59:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d51ec700 1 -- 172.21.15.3:0/2324204250 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322736449 0 0) 0x7ff6d00685f0 con 0x7ff6d01406b0 2023-10-06T04:59:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d59ed700 1 --2- 172.21.15.3:0/2324204250 >> v2:172.21.15.153:3300/0 conn(0x7ff6d00b3e10 0x7ff6d00680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d61ee700 1 --2- 172.21.15.3:0/2324204250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d00b3a40 0x7ff6d00ccd30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00cd270 tx=0x7ff6cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2976eec3d8c2c4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d51ec700 1 -- 172.21.15.3:0/2324204250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00b6b0 con 0x7ff6d00b3a40 2023-10-06T04:59:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d51ec700 1 -- 172.21.15.3:0/2324204250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc00b810 con 0x7ff6d00b3a40 2023-10-06T04:59:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.891+0000 7ff6d51ec700 1 -- 172.21.15.3:0/2324204250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00bbb0 con 0x7ff6d00b3a40 2023-10-06T04:59:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 -- 172.21.15.3:0/2324204250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d00b3a40 msgr2=0x7ff6d00ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 --2- 172.21.15.3:0/2324204250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d00b3a40 0x7ff6d00ccd30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00cd270 tx=0x7ff6cc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 -- 172.21.15.3:0/2324204250 shutdown_connections 2023-10-06T04:59:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 --2- 172.21.15.3:0/2324204250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d00b3a40 0x7ff6d00ccd30 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 --2- 172.21.15.3:0/2324204250 >> v2:172.21.15.153:3300/0 conn(0x7ff6d00b3e10 0x7ff6d00680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 -- 172.21.15.3:0/2324204250 >> 172.21.15.3:0/2324204250 conn(0x7ff6d00ce270 msgr2=0x7ff6d00ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 -- 172.21.15.3:0/2324204250 shutdown_connections 2023-10-06T04:59:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.892+0000 7ff6d8452700 1 -- 172.21.15.3:0/2324204250 wait complete. 2023-10-06T04:59:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.893+0000 7ff6d8452700 1 Processor -- start 2023-10-06T04:59:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.893+0000 7ff6d8452700 1 -- start start 2023-10-06T04:59:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d8452700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6d00b3a40 0x7ff6d0156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d8452700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6d00b3e10 0x7ff6d006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d8452700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d01406b0 0x7ff6d00c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d8452700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00a63a0 con 0x7ff6d01406b0 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d8452700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d006de50 con 0x7ff6d00b3e10 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d8452700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d006e9d0 con 0x7ff6d00b3a40 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d01406b0 0x7ff6d00c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d01406b0 0x7ff6d00c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42236/0 (socket says 172.21.15.3:42236) 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 -- 172.21.15.3:0/1266674003 learned_addr learned my addr 172.21.15.3:0/1266674003 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d59ed700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6d00b3e10 0x7ff6d006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d61ee700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6d00b3a40 0x7ff6d0156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 -- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6d00b3a40 msgr2=0x7ff6d0156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6d00b3a40 0x7ff6d0156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 -- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6d00b3e10 msgr2=0x7ff6d006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6d00b3e10 0x7ff6d006ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.894+0000 7ff6d69ef700 1 -- 172.21.15.3:0/1266674003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc009420 con 0x7ff6d01406b0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.895+0000 7ff6d69ef700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d01406b0 0x7ff6d00c96b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800cbc0 tx=0x7ff6c8001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.895+0000 7ff6c67fc700 1 -- 172.21.15.3:0/1266674003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800a7a0 con 0x7ff6d01406b0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.895+0000 7ff6c67fc700 1 -- 172.21.15.3:0/1266674003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c800a920 con 0x7ff6d01406b0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.895+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d006e0a0 con 0x7ff6d01406b0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.895+0000 7ff6c67fc700 1 -- 172.21.15.3:0/1266674003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800ace0 con 0x7ff6d01406b0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.895+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d006e660 con 0x7ff6d01406b0 2023-10-06T04:59:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.896+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d00644c0 con 0x7ff6d01406b0 2023-10-06T04:59:48.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.896+0000 7ff6c67fc700 1 -- 172.21.15.3:0/1266674003 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6c8007300 con 0x7ff6d01406b0 2023-10-06T04:59:48.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.897+0000 7ff6c67fc700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff6b4066f30 0x7ff6b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:48.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.897+0000 7ff6c67fc700 1 -- 172.21.15.3:0/1266674003 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ff6c80845e0 con 0x7ff6d01406b0 2023-10-06T04:59:48.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.897+0000 7ff6d61ee700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff6b4066f30 0x7ff6b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:48.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.897+0000 7ff6d61ee700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff6b4066f30 0x7ff6b40693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0093f0 tx=0x7ff6cc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:48.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:48.901+0000 7ff6c67fc700 1 -- 172.21.15.3:0/1266674003 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6c8051a70 con 0x7ff6d01406b0 2023-10-06T04:59:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.108+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff6d0071680 con 0x7ff6d01406b0 2023-10-06T04:59:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.884+0000 7ff6c67fc700 1 -- 172.21.15.3:0/1266674003 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff6c8055090 con 0x7ff6d01406b0 2023-10-06T04:59:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff6b4066f30 msgr2=0x7ff6b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff6b4066f30 0x7ff6b40693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0093f0 tx=0x7ff6cc017000 comp rx=0 tx=0).stop 2023-10-06T04:59:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d01406b0 msgr2=0x7ff6d00c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d01406b0 0x7ff6d00c96b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800cbc0 tx=0x7ff6c8001940 comp rx=0 tx=0).stop 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 shutdown_connections 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff6b4066f30 0x7ff6b40693e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6d01406b0 0x7ff6d00c96b0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6d00b3a40 0x7ff6d0156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 --2- 172.21.15.3:0/1266674003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6d00b3e10 0x7ff6d006ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.887+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 >> 172.21.15.3:0/1266674003 conn(0x7ff6d00ce270 msgr2=0x7ff6d013fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.888+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 shutdown_connections 2023-10-06T04:59:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:49.888+0000 7ff6d8452700 1 -- 172.21.15.3:0/1266674003 wait complete. 2023-10-06T04:59:49.891 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T04:59:49.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:59:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.029+0000 7f64656d1700 1 Processor -- start 2023-10-06T04:59:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.029+0000 7f64656d1700 1 -- start start 2023-10-06T04:59:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f64656d1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6460157030 0x7f6460159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f64656d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f646015a220 0x7f64600b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f64656d1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646015aeb0 con 0x7f6460157400 2023-10-06T04:59:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f64656d1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646015a7a0 con 0x7f646015a220 2023-10-06T04:59:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f64656d1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646015aa80 con 0x7f6460157030 2023-10-06T04:59:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f646015a220 0x7f64600b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6460157400 0x7f6460069340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40032/0 (socket says 172.21.15.3:40032) 2023-10-06T04:59:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645f7fe700 1 -- 172.21.15.3:0/2590428859 learned_addr learned my addr 172.21.15.3:0/2590428859 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645effd700 1 --2- 172.21.15.3:0/2590428859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6460157030 0x7f6460159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645e7fc700 1 -- 172.21.15.3:0/2590428859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6460157030 msgr2=0x7f6460159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645e7fc700 1 --2- 172.21.15.3:0/2590428859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6460157030 0x7f6460159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645e7fc700 1 -- 172.21.15.3:0/2590428859 >> v1:172.21.15.3:6789/0 conn(0x7f6460157400 legacy=0x7f6460069340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645e7fc700 1 -- 172.21.15.3:0/2590428859 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646006dba0 con 0x7f646015a220 2023-10-06T04:59:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645dffb700 1 -- 172.21.15.3:0/2590428859 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339636980 0 0) 0x7f646015aeb0 con 0x7f6460157400 2023-10-06T04:59:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645effd700 1 --2- 172.21.15.3:0/2590428859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6460157030 0x7f6460159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.030+0000 7f645e7fc700 1 --2- 172.21.15.3:0/2590428859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646015a220 0x7f64600b3630 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f64540098e0 tx=0x7f64540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e088fd9501604486 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f645dffb700 1 -- 172.21.15.3:0/2590428859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454009e80 con 0x7f646015a220 2023-10-06T04:59:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f645dffb700 1 -- 172.21.15.3:0/2590428859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645400b860 con 0x7f646015a220 2023-10-06T04:59:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f645dffb700 1 -- 172.21.15.3:0/2590428859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400bbe0 con 0x7f646015a220 2023-10-06T04:59:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 -- 172.21.15.3:0/2590428859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646015a220 msgr2=0x7f64600b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 --2- 172.21.15.3:0/2590428859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646015a220 0x7f64600b3630 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f64540098e0 tx=0x7f64540091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 -- 172.21.15.3:0/2590428859 shutdown_connections 2023-10-06T04:59:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 --2- 172.21.15.3:0/2590428859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6460157030 0x7f6460159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 --2- 172.21.15.3:0/2590428859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646015a220 0x7f64600b3630 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 -- 172.21.15.3:0/2590428859 >> 172.21.15.3:0/2590428859 conn(0x7f64600ce270 msgr2=0x7f64600ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 -- 172.21.15.3:0/2590428859 shutdown_connections 2023-10-06T04:59:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.031+0000 7f64656d1700 1 -- 172.21.15.3:0/2590428859 wait complete. 2023-10-06T04:59:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.032+0000 7f64656d1700 1 Processor -- start 2023-10-06T04:59:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.032+0000 7f64656d1700 1 -- start start 2023-10-06T04:59:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f64656d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6460157030 0x7f64600b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f64656d1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6460157400 0x7f646014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f64656d1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f646015a220 0x7f64600cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f64656d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64600af4e0 con 0x7f6460157030 2023-10-06T04:59:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f64656d1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64600ae5f0 con 0x7f6460157400 2023-10-06T04:59:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f64656d1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64600ae8d0 con 0x7f646015a220 2023-10-06T04:59:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f645effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6460157030 0x7f64600b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f645effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6460157030 0x7f64600b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42258/0 (socket says 172.21.15.3:42258) 2023-10-06T04:59:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f645effd700 1 -- 172.21.15.3:0/250818905 learned_addr learned my addr 172.21.15.3:0/250818905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f645e7fc700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6460157400 0x7f646014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.033+0000 7f645f7fe700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f646015a220 0x7f64600cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f645effd700 1 -- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f646015a220 msgr2=0x7f64600cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f645effd700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f646015a220 0x7f64600cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f645effd700 1 -- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6460157400 msgr2=0x7f646014eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f645effd700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6460157400 0x7f646014eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f645effd700 1 -- 172.21.15.3:0/250818905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454009420 con 0x7f6460157030 2023-10-06T04:59:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f645e7fc700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6460157400 0x7f646014eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f645effd700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6460157030 0x7f64600b0630 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f645000c830 tx=0x7f6450008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f6446ffd700 1 -- 172.21.15.3:0/250818905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011600 con 0x7f6460157030 2023-10-06T04:59:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f6446ffd700 1 -- 172.21.15.3:0/250818905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6450011760 con 0x7f6460157030 2023-10-06T04:59:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f6446ffd700 1 -- 172.21.15.3:0/250818905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011b20 con 0x7f6460157030 2023-10-06T04:59:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64600aebb0 con 0x7f6460157030 2023-10-06T04:59:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.034+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64600af170 con 0x7f6460157030 2023-10-06T04:59:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.035+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64600644c0 con 0x7f6460157030 2023-10-06T04:59:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.035+0000 7f6446ffd700 1 -- 172.21.15.3:0/250818905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f645000a3e0 con 0x7f6460157030 2023-10-06T04:59:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.036+0000 7f6446ffd700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f644c066f30 0x7f644c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.036+0000 7f6446ffd700 1 -- 172.21.15.3:0/250818905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6450084850 con 0x7f6460157030 2023-10-06T04:59:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.036+0000 7f645e7fc700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f644c066f30 0x7f644c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.037+0000 7f645e7fc700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f644c066f30 0x7f644c0693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f64540091e0 tx=0x7f6454017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.040+0000 7f6446ffd700 1 -- 172.21.15.3:0/250818905 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6450051cb0 con 0x7f6460157030 2023-10-06T04:59:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.312+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6460071680 con 0x7f6460157030 2023-10-06T04:59:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.312+0000 7f6446ffd700 1 -- 172.21.15.3:0/250818905 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6450015090 con 0x7f6460157030 2023-10-06T04:59:50.314 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:50.314 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T04:59:50.314 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T04:59:50.314 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.314 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T04:59:50.314 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T04:59:50.315 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:50.317 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.318 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T04:59:50.319 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.320 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T04:59:50.322 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T04:59:50.323 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.324 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T04:59:50.325 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T04:59:50.327 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T04:59:50.329 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T04:59:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f644c066f30 msgr2=0x7f644c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f644c066f30 0x7f644c0693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f64540091e0 tx=0x7f6454017000 comp rx=0 tx=0).stop 2023-10-06T04:59:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6460157030 msgr2=0x7f64600b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6460157030 0x7f64600b0630 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f645000c830 tx=0x7f6450008040 comp rx=0 tx=0).stop 2023-10-06T04:59:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 shutdown_connections 2023-10-06T04:59:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f644c066f30 0x7f644c0693e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f646015a220 0x7f64600cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6460157030 0x7f64600b0630 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 --2- 172.21.15.3:0/250818905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6460157400 0x7f646014eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 >> 172.21.15.3:0/250818905 conn(0x7f64600ce270 msgr2=0x7f6460152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.314+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 shutdown_connections 2023-10-06T04:59:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.315+0000 7f64656d1700 1 -- 172.21.15.3:0/250818905 wait complete. 2023-10-06T04:59:50.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.451+0000 7fac347b8700 1 Processor -- start 2023-10-06T04:59:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.452+0000 7fac347b8700 1 -- start start 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.452+0000 7fac347b8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac2c15a4b0 0x7fac2c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.452+0000 7fac347b8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c15d9f0 con 0x7fac2c153bd0 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.452+0000 7fac347b8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c0b0aa0 con 0x7fac2c153fa0 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.452+0000 7fac347b8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c15a980 con 0x7fac2c15a4b0 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31d53700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac2c15a4b0 0x7fac2c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac32554700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fac2c153bd0 0x7fac2c159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40042/0 (socket says 172.21.15.3:40042) 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac32554700 1 -- 172.21.15.3:0/1344684210 learned_addr learned my addr 172.21.15.3:0/1344684210 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31d53700 1 -- 172.21.15.3:0/1344684210 >> v1:172.21.15.153:6789/0 conn(0x7fac2c153fa0 legacy=0x7fac2c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31d53700 1 -- 172.21.15.3:0/1344684210 >> v1:172.21.15.3:6789/0 conn(0x7fac2c153bd0 legacy=0x7fac2c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31552700 1 -- 172.21.15.3:0/1344684210 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119935784 0 0) 0x7fac2c15d9f0 con 0x7fac2c153bd0 2023-10-06T04:59:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31d53700 1 -- 172.21.15.3:0/1344684210 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c15ac60 con 0x7fac2c15a4b0 2023-10-06T04:59:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31d53700 1 --2- 172.21.15.3:0/1344684210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c15a4b0 0x7fac2c15de80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fac20009910 tx=0x7fac200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2c48d26e138225 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31552700 1 -- 172.21.15.3:0/1344684210 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000b7e0 con 0x7fac2c15a4b0 2023-10-06T04:59:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31552700 1 -- 172.21.15.3:0/1344684210 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000b940 con 0x7fac2c15a4b0 2023-10-06T04:59:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.453+0000 7fac31552700 1 -- 172.21.15.3:0/1344684210 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000bce0 con 0x7fac2c15a4b0 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.454+0000 7fac347b8700 1 -- 172.21.15.3:0/1344684210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c15a4b0 msgr2=0x7fac2c15de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.454+0000 7fac347b8700 1 --2- 172.21.15.3:0/1344684210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c15a4b0 0x7fac2c15de80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fac20009910 tx=0x7fac200091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.454+0000 7fac347b8700 1 -- 172.21.15.3:0/1344684210 shutdown_connections 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.454+0000 7fac347b8700 1 --2- 172.21.15.3:0/1344684210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c15a4b0 0x7fac2c15de80 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.454+0000 7fac347b8700 1 -- 172.21.15.3:0/1344684210 >> 172.21.15.3:0/1344684210 conn(0x7fac2c0cdc30 msgr2=0x7fac2c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.454+0000 7fac347b8700 1 -- 172.21.15.3:0/1344684210 shutdown_connections 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.454+0000 7fac347b8700 1 -- 172.21.15.3:0/1344684210 wait complete. 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.455+0000 7fac347b8700 1 Processor -- start 2023-10-06T04:59:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.455+0000 7fac347b8700 1 -- start start 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac347b8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c153bd0 0x7fac2c143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac347b8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac2c153fa0 0x7fac2c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac347b8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac2c15a4b0 0x7fac2c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac347b8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c144630 con 0x7fac2c153bd0 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac347b8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c16d920 con 0x7fac2c153fa0 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac347b8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c16dc00 con 0x7fac2c15a4b0 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c153bd0 0x7fac2c143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c153bd0 0x7fac2c143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42282/0 (socket says 172.21.15.3:42282) 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 -- 172.21.15.3:0/2607027514 learned_addr learned my addr 172.21.15.3:0/2607027514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac31d53700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac2c153fa0 0x7fac2c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32d55700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac2c15a4b0 0x7fac2c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 -- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac2c15a4b0 msgr2=0x7fac2c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac2c15a4b0 0x7fac2c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 -- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac2c153fa0 msgr2=0x7fac2c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac2c153fa0 0x7fac2c155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 -- 172.21.15.3:0/2607027514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20009420 con 0x7fac2c153bd0 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32d55700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac2c15a4b0 0x7fac2c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.456+0000 7fac32554700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c153bd0 0x7fac2c143c60 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c5b0 tx=0x7fac24001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.457+0000 7fac2a7fc700 1 -- 172.21.15.3:0/2607027514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24005b30 con 0x7fac2c153bd0 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.457+0000 7fac2a7fc700 1 -- 172.21.15.3:0/2607027514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac24005c90 con 0x7fac2c153bd0 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.457+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c16dee0 con 0x7fac2c153bd0 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.457+0000 7fac2a7fc700 1 -- 172.21.15.3:0/2607027514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24007300 con 0x7fac2c153bd0 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.457+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c1441a0 con 0x7fac2c153bd0 2023-10-06T04:59:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.458+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c071680 con 0x7fac2c153bd0 2023-10-06T04:59:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.458+0000 7fac2a7fc700 1 -- 172.21.15.3:0/2607027514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac2401b3b0 con 0x7fac2c153bd0 2023-10-06T04:59:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.458+0000 7fac2a7fc700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac18066f30 0x7fac180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.458+0000 7fac2a7fc700 1 -- 172.21.15.3:0/2607027514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fac24083870 con 0x7fac2c153bd0 2023-10-06T04:59:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.459+0000 7fac31d53700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac18066f30 0x7fac180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.459+0000 7fac31d53700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac18066f30 0x7fac180693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fac2c15a8b0 tx=0x7fac20016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.463+0000 7fac2a7fc700 1 -- 172.21.15.3:0/2607027514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac24050cd0 con 0x7fac2c153bd0 2023-10-06T04:59:50.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.658+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac2c154450 con 0x7fac2c153bd0 2023-10-06T04:59:50.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.659+0000 7fac2a7fc700 1 -- 172.21.15.3:0/2607027514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+13604 (secure 0 0 0) 0x7fac24017020 con 0x7fac2c153bd0 2023-10-06T04:59:50.660 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:50.660 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":104,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:49.881467+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:50.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac18066f30 msgr2=0x7fac180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac18066f30 0x7fac180693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fac2c15a8b0 tx=0x7fac20016000 comp rx=0 tx=0).stop 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c153bd0 msgr2=0x7fac2c143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c153bd0 0x7fac2c143c60 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c5b0 tx=0x7fac24001070 comp rx=0 tx=0).stop 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 shutdown_connections 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac18066f30 0x7fac180693e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac2c15a4b0 0x7fac2c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac2c153bd0 0x7fac2c143c60 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 --2- 172.21.15.3:0/2607027514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac2c153fa0 0x7fac2c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 >> 172.21.15.3:0/2607027514 conn(0x7fac2c0cdc30 msgr2=0x7fac2c1599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 shutdown_connections 2023-10-06T04:59:50.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.661+0000 7fac347b8700 1 -- 172.21.15.3:0/2607027514 wait complete. 2023-10-06T04:59:50.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:59:50.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:59:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.795+0000 7f2d5666a700 1 Processor -- start 2023-10-06T04:59:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.795+0000 7f2d5666a700 1 -- start start 2023-10-06T04:59:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d5666a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2d500cc140 0x7f2d50159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d5666a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d500cc510 0x7f2d500aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d5666a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d5015a330 0x7f2d500af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d5666a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d500ce420 con 0x7f2d500cc510 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d5666a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d500af9e0 con 0x7f2d500cc140 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d5666a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015a880 con 0x7f2d5015a330 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d4f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d500cc510 0x7f2d500aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d4f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d500cc510 0x7f2d500aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42294/0 (socket says 172.21.15.3:42294) 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.796+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/335286326 learned_addr learned my addr 172.21.15.3:0/335286326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d54c07700 1 --2- 172.21.15.3:0/335286326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d5015a330 0x7f2d500af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4ffff700 1 --2- 172.21.15.3:0/335286326 >> v2:172.21.15.153:3300/0 conn(0x7f2d500cc140 0x7f2d50159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/335286326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d5015a330 msgr2=0x7f2d500af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4f7fe700 1 --2- 172.21.15.3:0/335286326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d5015a330 0x7f2d500af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/335286326 >> v2:172.21.15.153:3300/0 conn(0x7f2d500cc140 msgr2=0x7f2d50159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4f7fe700 1 --2- 172.21.15.3:0/335286326 >> v2:172.21.15.153:3300/0 conn(0x7f2d500cc140 0x7f2d50159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/335286326 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5015ab60 con 0x7f2d500cc510 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d54c07700 1 --2- 172.21.15.3:0/335286326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d5015a330 0x7f2d500af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4f7fe700 1 --2- 172.21.15.3:0/335286326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d500aef60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2d500894e0 tx=0x7f2d44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7888052171f36410 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4effd700 1 -- 172.21.15.3:0/335286326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400b860 con 0x7f2d500cc510 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4effd700 1 -- 172.21.15.3:0/335286326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4400b9c0 con 0x7f2d500cc510 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d4effd700 1 -- 172.21.15.3:0/335286326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400bd60 con 0x7f2d500cc510 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d5666a700 1 -- 172.21.15.3:0/335286326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 msgr2=0x7f2d500aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.797+0000 7f2d5666a700 1 --2- 172.21.15.3:0/335286326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d500aef60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2d500894e0 tx=0x7f2d44009250 comp rx=0 tx=0).stop 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.798+0000 7f2d5666a700 1 -- 172.21.15.3:0/335286326 shutdown_connections 2023-10-06T04:59:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.798+0000 7f2d5666a700 1 --2- 172.21.15.3:0/335286326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d5015a330 0x7f2d500af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.798+0000 7f2d5666a700 1 --2- 172.21.15.3:0/335286326 >> v2:172.21.15.153:3300/0 conn(0x7f2d500cc140 0x7f2d50159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.798+0000 7f2d5666a700 1 --2- 172.21.15.3:0/335286326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d500aef60 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.798+0000 7f2d5666a700 1 -- 172.21.15.3:0/335286326 >> 172.21.15.3:0/335286326 conn(0x7f2d500d03e0 msgr2=0x7f2d500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.798+0000 7f2d5666a700 1 -- 172.21.15.3:0/335286326 shutdown_connections 2023-10-06T04:59:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.798+0000 7f2d5666a700 1 -- 172.21.15.3:0/335286326 wait complete. 2023-10-06T04:59:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.799+0000 7f2d5666a700 1 Processor -- start 2023-10-06T04:59:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.799+0000 7f2d5666a700 1 -- start start 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.799+0000 7f2d5666a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d500cc140 0x7f2d50153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d5666a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d5015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d5666a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d5015a330 0x7f2d5015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d5666a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50169f80 con 0x7f2d500cc510 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d5666a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500c8af0 con 0x7f2d5015a330 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d5666a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500c66a0 con 0x7f2d500cc140 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d5015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d5015ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42300/0 (socket says 172.21.15.3:42300) 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/1825770908 learned_addr learned my addr 172.21.15.3:0/1825770908 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4ffff700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d500cc140 0x7f2d50153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d54c07700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d5015a330 0x7f2d5015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d500cc140 msgr2=0x7f2d50153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d500cc140 0x7f2d50153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d5015a330 msgr2=0x7f2d5015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d5015a330 0x7f2d5015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 -- 172.21.15.3:0/1825770908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d440094f0 con 0x7f2d500cc510 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d4f7fe700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d5015ada0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2d5006ce30 tx=0x7f2d44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d37fff700 1 -- 172.21.15.3:0/1825770908 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44024440 con 0x7f2d500cc510 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d37fff700 1 -- 172.21.15.3:0/1825770908 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d440245a0 con 0x7f2d500cc510 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d500c6980 con 0x7f2d500cc510 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d37fff700 1 -- 172.21.15.3:0/1825770908 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44024960 con 0x7f2d500cc510 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.800+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d500c6f40 con 0x7f2d500cc510 2023-10-06T04:59:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.801+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d500c88c0 con 0x7f2d500cc510 2023-10-06T04:59:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.802+0000 7f2d37fff700 1 -- 172.21.15.3:0/1825770908 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d4401a420 con 0x7f2d500cc510 2023-10-06T04:59:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.802+0000 7f2d37fff700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2d3c066f30 0x7f2d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.802+0000 7f2d37fff700 1 -- 172.21.15.3:0/1825770908 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2d44086160 con 0x7f2d500cc510 2023-10-06T04:59:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.803+0000 7f2d4ffff700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2d3c066f30 0x7f2d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.803+0000 7f2d4ffff700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2d3c066f30 0x7f2d3c0693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2d40006fd0 tx=0x7f2d40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:50.807+0000 7f2d37fff700 1 -- 172.21.15.3:0/1825770908 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d440535c0 con 0x7f2d500cc510 2023-10-06T04:59:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.003+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2d5004c380 con 0x7f2d500cc510 2023-10-06T04:59:51.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.004+0000 7f2d37fff700 1 -- 172.21.15.3:0/1825770908 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v104) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2d44056be0 con 0x7f2d500cc510 2023-10-06T04:59:51.006 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2d3c066f30 msgr2=0x7f2d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2d3c066f30 0x7f2d3c0693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2d40006fd0 tx=0x7f2d40008040 comp rx=0 tx=0).stop 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 msgr2=0x7f2d5015ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d5015ada0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2d5006ce30 tx=0x7f2d44017000 comp rx=0 tx=0).stop 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 shutdown_connections 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2d3c066f30 0x7f2d3c0693e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d5015a330 0x7f2d5015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d500cc140 0x7f2d50153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 --2- 172.21.15.3:0/1825770908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d500cc510 0x7f2d5015ada0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 >> 172.21.15.3:0/1825770908 conn(0x7f2d500d03e0 msgr2=0x7f2d50159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:51.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 shutdown_connections 2023-10-06T04:59:51.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.006+0000 7f2d5666a700 1 -- 172.21.15.3:0/1825770908 wait complete. 2023-10-06T04:59:51.019 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:59:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.140+0000 7fd140883700 1 Processor -- start 2023-10-06T04:59:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.140+0000 7fd140883700 1 -- start start 2023-10-06T04:59:51.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd140883700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd13c0cfb80 0x7fd13c0af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd140883700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c159eb0 0x7fd13c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd140883700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13c15d510 con 0x7fd13c0afad0 2023-10-06T04:59:51.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd140883700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13c15a480 con 0x7fd13c0cfb80 2023-10-06T04:59:51.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd140883700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c15a760 con 0x7fd13c159eb0 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd13a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd13c0cfb80 0x7fd13c0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd13a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd13c0cfb80 0x7fd13c0af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42310/0 (socket says 172.21.15.3:42310) 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.141+0000 7fd13a59c700 1 -- 172.21.15.3:0/88188537 learned_addr learned my addr 172.21.15.3:0/88188537 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13a59c700 1 -- 172.21.15.3:0/88188537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c159eb0 msgr2=0x7fd13c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13ad9d700 1 --2- 172.21.15.3:0/88188537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c159eb0 0x7fd13c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13a59c700 1 --2- 172.21.15.3:0/88188537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c159eb0 0x7fd13c15da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13a59c700 1 -- 172.21.15.3:0/88188537 >> v1:172.21.15.153:6789/0 conn(0x7fd13c0afad0 legacy=0x7fd13c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13a59c700 1 -- 172.21.15.3:0/88188537 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c15aa40 con 0x7fd13c0cfb80 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13a59c700 1 --2- 172.21.15.3:0/88188537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c0af590 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd1340099b0 tx=0x7fd134009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f8ee4f07e9a2c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13959a700 1 -- 172.21.15.3:0/88188537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13400b820 con 0x7fd13c0cfb80 2023-10-06T04:59:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13959a700 1 -- 172.21.15.3:0/88188537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13400b980 con 0x7fd13c0cfb80 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd13959a700 1 -- 172.21.15.3:0/88188537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13400bd20 con 0x7fd13c0cfb80 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd140883700 1 -- 172.21.15.3:0/88188537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 msgr2=0x7fd13c0af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.142+0000 7fd140883700 1 --2- 172.21.15.3:0/88188537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c0af590 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd1340099b0 tx=0x7fd134009250 comp rx=0 tx=0).stop 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.143+0000 7fd140883700 1 -- 172.21.15.3:0/88188537 shutdown_connections 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.143+0000 7fd140883700 1 --2- 172.21.15.3:0/88188537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c159eb0 0x7fd13c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.143+0000 7fd140883700 1 --2- 172.21.15.3:0/88188537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c0af590 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.143+0000 7fd140883700 1 -- 172.21.15.3:0/88188537 >> 172.21.15.3:0/88188537 conn(0x7fd13c070310 msgr2=0x7fd13c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.143+0000 7fd140883700 1 -- 172.21.15.3:0/88188537 shutdown_connections 2023-10-06T04:59:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.143+0000 7fd140883700 1 -- 172.21.15.3:0/88188537 wait complete. 2023-10-06T04:59:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 Processor -- start 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 -- start start 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c0afad0 0x7fd13c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd13c159eb0 0x7fd13c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b31e0 con 0x7fd13c0cfb80 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b34c0 con 0x7fd13c159eb0 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd140883700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b37a0 con 0x7fd13c0afad0 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.144+0000 7fd139d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42316/0 (socket says 172.21.15.3:42316) 2023-10-06T04:59:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 -- 172.21.15.3:0/195258716 learned_addr learned my addr 172.21.15.3:0/195258716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd13ad9d700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd13c159eb0 0x7fd13c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd13a59c700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c0afad0 0x7fd13c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 -- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c0afad0 msgr2=0x7fd13c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c0afad0 0x7fd13c153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 -- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd13c159eb0 msgr2=0x7fd13c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd13c159eb0 0x7fd13c158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 -- 172.21.15.3:0/195258716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340094f0 con 0x7fd13c0cfb80 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd139d9b700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c158a50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c590 tx=0x7fd124007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd1327fc700 1 -- 172.21.15.3:0/195258716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124011590 con 0x7fd13c0cfb80 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd1327fc700 1 -- 172.21.15.3:0/195258716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124011730 con 0x7fd13c0cfb80 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c0b3a80 con 0x7fd13c0cfb80 2023-10-06T04:59:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd1327fc700 1 -- 172.21.15.3:0/195258716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124011af0 con 0x7fd13c0cfb80 2023-10-06T04:59:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.145+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c154120 con 0x7fd13c0cfb80 2023-10-06T04:59:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.146+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c04c380 con 0x7fd13c0cfb80 2023-10-06T04:59:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.147+0000 7fd1327fc700 1 -- 172.21.15.3:0/195258716 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd12401c460 con 0x7fd13c0cfb80 2023-10-06T04:59:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.147+0000 7fd1327fc700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd11c066f30 0x7fd11c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.147+0000 7fd1327fc700 1 -- 172.21.15.3:0/195258716 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd1240840f0 con 0x7fd13c0cfb80 2023-10-06T04:59:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.147+0000 7fd13a59c700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd11c066f30 0x7fd11c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.148+0000 7fd13a59c700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd11c066f30 0x7fd11c0693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd1340099b0 tx=0x7fd134016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.151+0000 7fd1327fc700 1 -- 172.21.15.3:0/195258716 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd124051550 con 0x7fd13c0cfb80 2023-10-06T04:59:51.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.343+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd13c0ce470 con 0x7fd13c0cfb80 2023-10-06T04:59:51.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.343+0000 7fd1327fc700 1 -- 172.21.15.3:0/195258716 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd124054b70 con 0x7fd13c0cfb80 2023-10-06T04:59:51.345 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T04:59:51.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd11c066f30 msgr2=0x7fd11c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd11c066f30 0x7fd11c0693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd1340099b0 tx=0x7fd134016000 comp rx=0 tx=0).stop 2023-10-06T04:59:51.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 msgr2=0x7fd13c158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c158a50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c590 tx=0x7fd124007f30 comp rx=0 tx=0).stop 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 shutdown_connections 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd11c066f30 0x7fd11c0693e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd13c159eb0 0x7fd13c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd13c0afad0 0x7fd13c153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 --2- 172.21.15.3:0/195258716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd13c0cfb80 0x7fd13c158a50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 >> 172.21.15.3:0/195258716 conn(0x7fd13c070310 msgr2=0x7fd13c0c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 shutdown_connections 2023-10-06T04:59:51.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.345+0000 7fd140883700 1 -- 172.21.15.3:0/195258716 wait complete. 2023-10-06T04:59:51.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:59:51.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.478+0000 7f7d0d80c700 1 Processor -- start 2023-10-06T04:59:51.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.478+0000 7f7d0d80c700 1 -- start start 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d0d80c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 0x7f7d080d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d0d80c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d080d06a0 0x7f7d080b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d0d80c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d080cf6c0 con 0x7f7d0815c8f0 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d0d80c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d080cfb50 con 0x7f7d080d06a0 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d0d80c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080d0b20 con 0x7f7d0815ccc0 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d06ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7d0815c8f0 0x7f7d0815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40054/0 (socket says 172.21.15.3:40054) 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d06ffd700 1 -- 172.21.15.3:0/580374060 learned_addr learned my addr 172.21.15.3:0/580374060 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d077fe700 1 --2- 172.21.15.3:0/580374060 >> v2:172.21.15.153:3300/0 conn(0x7f7d080d06a0 0x7f7d080b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.479+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250420919 0 0) 0x7f7d080cf6c0 con 0x7f7d0815c8f0 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d067fc700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 0x7f7d080d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf4003570 con 0x7f7d0815c8f0 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3419239705 0 0) 0x7f7cf4003570 con 0x7f7d0815c8f0 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 msgr2=0x7f7d080d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 0x7f7d080d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 >> v2:172.21.15.153:3300/0 conn(0x7f7d080d06a0 msgr2=0x7f7d080b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 --2- 172.21.15.3:0/580374060 >> v2:172.21.15.153:3300/0 conn(0x7f7d080d06a0 0x7f7d080b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d080ceab0 con 0x7f7d0815c8f0 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f7d000017f0 con 0x7f7d0815c8f0 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 >> v1:172.21.15.3:6789/0 conn(0x7f7d0815c8f0 legacy=0x7f7d0815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf4003a60 0x7f7cf4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 0x7f7cf4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cf4004d50 0x7f7cf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080cfb50 con 0x7f7cf4003a60 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080d0b20 con 0x7f7cf4004d50 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf400ba60 con 0x7f7d0815ccc0 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d06ffd700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf4003a60 0x7f7cf4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d067fc700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 0x7f7cf4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d077fe700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cf4004d50 0x7f7cf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d06ffd700 1 -- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 msgr2=0x7f7cf4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d06ffd700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815ccc0 0x7f7cf4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d06ffd700 1 -- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cf4004d50 msgr2=0x7f7cf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d06ffd700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cf4004d50 0x7f7cf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d06ffd700 1 -- 172.21.15.3:0/580374060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4003570 con 0x7f7cf4003a60 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.480+0000 7f7d077fe700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cf4004d50 0x7f7cf40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d06ffd700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf4003a60 0x7f7cf4003e70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a970 tx=0x7f7d080d0c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a782e23de59dd8f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000070d0 con 0x7f7cf4003a60 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00007230 con 0x7f7cf4003a60 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d05ffb700 1 -- 172.21.15.3:0/580374060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000075f0 con 0x7f7cf4003a60 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d0d80c700 1 -- 172.21.15.3:0/580374060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf4003a60 msgr2=0x7f7cf4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d0d80c700 1 --2- 172.21.15.3:0/580374060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf4003a60 0x7f7cf4003e70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a970 tx=0x7f7d080d0c90 comp rx=0 tx=0).stop 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d077fe700 1 -- 172.21.15.3:0/580374060 reap_dead start 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d0d80c700 1 -- 172.21.15.3:0/580374060 shutdown_connections 2023-10-06T04:59:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d0d80c700 1 -- 172.21.15.3:0/580374060 >> 172.21.15.3:0/580374060 conn(0x7f7d08070310 msgr2=0x7f7d08070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d0d80c700 1 -- 172.21.15.3:0/580374060 shutdown_connections 2023-10-06T04:59:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.481+0000 7f7d0d80c700 1 -- 172.21.15.3:0/580374060 wait complete. 2023-10-06T04:59:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.482+0000 7f7d0d80c700 1 Processor -- start 2023-10-06T04:59:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.482+0000 7f7d0d80c700 1 -- start start 2023-10-06T04:59:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d0d80c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cf4003a60 0x7f7d080c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d0d80c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d080c5220 0x7f7d080c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d0d80c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d081566b0 0x7f7d08156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d0d80c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08169d00 con 0x7f7d080c5220 2023-10-06T04:59:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d0d80c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0815d480 con 0x7f7d081566b0 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d0d80c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0815d180 con 0x7f7cf4003a60 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d080c5220 0x7f7d080c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d080c5220 0x7f7d080c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42332/0 (socket says 172.21.15.3:42332) 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d067fc700 1 -- 172.21.15.3:0/190543871 learned_addr learned my addr 172.21.15.3:0/190543871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.483+0000 7f7d06ffd700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cf4003a60 0x7f7d080c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d077fe700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d081566b0 0x7f7d08156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d067fc700 1 -- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cf4003a60 msgr2=0x7f7d080c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d067fc700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cf4003a60 0x7f7d080c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d067fc700 1 -- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d081566b0 msgr2=0x7f7d08156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d067fc700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d081566b0 0x7f7d08156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d067fc700 1 -- 172.21.15.3:0/190543871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d080c5c80 con 0x7f7d080c5220 2023-10-06T04:59:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d077fe700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d081566b0 0x7f7d08156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d067fc700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d080c5220 0x7f7d080c5690 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0011b70 tx=0x7f7cf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7cfeffd700 1 -- 172.21.15.3:0/190543871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000fa70 con 0x7f7d080c5220 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7cfeffd700 1 -- 172.21.15.3:0/190543871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf000fbd0 con 0x7f7d080c5220 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7cfeffd700 1 -- 172.21.15.3:0/190543871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0014440 con 0x7f7d080c5220 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d080c5f60 con 0x7f7d080c5220 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.484+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d08157590 con 0x7f7d080c5220 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.485+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0815d630 con 0x7f7d080c5220 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.485+0000 7f7cfeffd700 1 -- 172.21.15.3:0/190543871 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cf0014870 con 0x7f7d080c5220 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.486+0000 7f7cfeffd700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ce8066f30 0x7f7ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.486+0000 7f7cfeffd700 1 -- 172.21.15.3:0/190543871 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7cf0018030 con 0x7f7d080c5220 2023-10-06T04:59:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.486+0000 7f7d06ffd700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ce8066f30 0x7f7ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.487+0000 7f7d06ffd700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ce8066f30 0x7f7ce80693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000afd0 tx=0x7f7d00016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.490+0000 7f7cfeffd700 1 -- 172.21.15.3:0/190543871 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d0815d630 con 0x7f7d080c5220 2023-10-06T04:59:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.688+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7d0815d630 con 0x7f7d080c5220 2023-10-06T04:59:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.688+0000 7f7cfeffd700 1 -- 172.21.15.3:0/190543871 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7d0815d630 con 0x7f7d080c5220 2023-10-06T04:59:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:51.690 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T04:59:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T04:59:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T04:59:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T04:59:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T04:59:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T04:59:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T04:59:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T04:59:51.691 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T04:59:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ce8066f30 msgr2=0x7f7ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ce8066f30 0x7f7ce80693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000afd0 tx=0x7f7d00016000 comp rx=0 tx=0).stop 2023-10-06T04:59:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d080c5220 msgr2=0x7f7d080c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d080c5220 0x7f7d080c5690 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0011b70 tx=0x7f7cf00091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 shutdown_connections 2023-10-06T04:59:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7ce8066f30 0x7f7ce80693e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d081566b0 0x7f7d08156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7cf4003a60 0x7f7d080c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 --2- 172.21.15.3:0/190543871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d080c5220 0x7f7d080c5690 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 >> 172.21.15.3:0/190543871 conn(0x7f7d08070310 msgr2=0x7f7d081605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 shutdown_connections 2023-10-06T04:59:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.690+0000 7f7d0d80c700 1 -- 172.21.15.3:0/190543871 wait complete. 2023-10-06T04:59:51.704 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:59:51.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:59:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.825+0000 7f8becfd0700 1 Processor -- start 2023-10-06T04:59:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.825+0000 7f8becfd0700 1 -- start start 2023-10-06T04:59:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.826+0000 7f8becfd0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8be815c740 0x7f8be80cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.826+0000 7f8becfd0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80ce7b0 con 0x7f8be815cb10 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.826+0000 7f8becfd0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80cea90 con 0x7f8be80afe60 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.826+0000 7f8becfd0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be815cf70 con 0x7f8be815c740 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8be815cb10 0x7f8be815dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40056/0 (socket says 172.21.15.3:40056) 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be5d9b700 1 -- 172.21.15.3:0/3137387108 learned_addr learned my addr 172.21.15.3:0/3137387108 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be659c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8be815c740 0x7f8be80cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be559a700 1 -- 172.21.15.3:0/3137387108 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737938709 0 0) 0x7f8be80ce7b0 con 0x7f8be815cb10 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be559a700 1 -- 172.21.15.3:0/3137387108 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bd4003570 con 0x7f8be815cb10 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be659c700 1 -- 172.21.15.3:0/3137387108 >> v1:172.21.15.153:6789/0 conn(0x7f8be80afe60 legacy=0x7f8be80ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be659c700 1 -- 172.21.15.3:0/3137387108 >> v1:172.21.15.3:6789/0 conn(0x7f8be815cb10 legacy=0x7f8be815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be559a700 1 -- 172.21.15.3:0/3137387108 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282393775 0 0) 0x7f8be80cea90 con 0x7f8be80afe60 2023-10-06T04:59:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be659c700 1 -- 172.21.15.3:0/3137387108 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be80ced70 con 0x7f8be815c740 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be659c700 1 --2- 172.21.15.3:0/3137387108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80cb3a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8be00098e0 tx=0x7f8be00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d951e37939165d07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be559a700 1 -- 172.21.15.3:0/3137387108 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be000b6d0 con 0x7f8be815c740 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be559a700 1 -- 172.21.15.3:0/3137387108 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be000b830 con 0x7f8be815c740 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.827+0000 7f8be559a700 1 -- 172.21.15.3:0/3137387108 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be000bbd0 con 0x7f8be815c740 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.828+0000 7f8becfd0700 1 -- 172.21.15.3:0/3137387108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 msgr2=0x7f8be80cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.828+0000 7f8becfd0700 1 --2- 172.21.15.3:0/3137387108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80cb3a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8be00098e0 tx=0x7f8be00091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.828+0000 7f8becfd0700 1 -- 172.21.15.3:0/3137387108 shutdown_connections 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.828+0000 7f8becfd0700 1 --2- 172.21.15.3:0/3137387108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80cb3a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.828+0000 7f8becfd0700 1 -- 172.21.15.3:0/3137387108 >> 172.21.15.3:0/3137387108 conn(0x7f8be806f960 msgr2=0x7f8be8070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.828+0000 7f8becfd0700 1 -- 172.21.15.3:0/3137387108 shutdown_connections 2023-10-06T04:59:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.828+0000 7f8becfd0700 1 -- 172.21.15.3:0/3137387108 wait complete. 2023-10-06T04:59:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.829+0000 7f8becfd0700 1 Processor -- start 2023-10-06T04:59:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.829+0000 7f8becfd0700 1 -- start start 2023-10-06T04:59:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8becfd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be80afe60 0x7f8be80b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8becfd0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8becfd0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8be815cb10 0x7f8be80c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8becfd0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be80c88d0 con 0x7f8be80afe60 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8becfd0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8157740 con 0x7f8be815c740 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8becfd0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8156e10 con 0x7f8be815cb10 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be80afe60 0x7f8be80b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be80afe60 0x7f8be80b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42336/0 (socket says 172.21.15.3:42336) 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 -- 172.21.15.3:0/2174131778 learned_addr learned my addr 172.21.15.3:0/2174131778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be5d9b700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be6d9d700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8be815cb10 0x7f8be80c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 -- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8be815cb10 msgr2=0x7f8be80c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8be815cb10 0x7f8be80c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 -- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 msgr2=0x7f8be80c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 -- 172.21.15.3:0/2174131778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0009420 con 0x7f8be80afe60 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be5d9b700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be6d9d700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8be815cb10 0x7f8be80c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.830+0000 7f8be659c700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be80afe60 0x7f8be80b4210 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8be00093f0 tx=0x7f8be0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.831+0000 7f8bde7fc700 1 -- 172.21.15.3:0/2174131778 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be001f070 con 0x7f8be80afe60 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.831+0000 7f8bde7fc700 1 -- 172.21.15.3:0/2174131778 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be000bbb0 con 0x7f8be80afe60 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.831+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be81570f0 con 0x7f8be80afe60 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.831+0000 7f8bde7fc700 1 -- 172.21.15.3:0/2174131778 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0011670 con 0x7f8be80afe60 2023-10-06T04:59:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.831+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be80b4750 con 0x7f8be80afe60 2023-10-06T04:59:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.831+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be80b4940 con 0x7f8be80afe60 2023-10-06T04:59:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.832+0000 7f8bde7fc700 1 -- 172.21.15.3:0/2174131778 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8be0011a60 con 0x7f8be80afe60 2023-10-06T04:59:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.832+0000 7f8bde7fc700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bc8066f30 0x7f8bc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.833+0000 7f8bde7fc700 1 -- 172.21.15.3:0/2174131778 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8be0085590 con 0x7f8be80afe60 2023-10-06T04:59:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.833+0000 7f8be5d9b700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bc8066f30 0x7f8bc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.833+0000 7f8be5d9b700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bc8066f30 0x7f8bc80693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8be816ce80 tx=0x7f8bd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:51.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:51.837+0000 7f8bde7fc700 1 -- 172.21.15.3:0/2174131778 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8be00529f0 con 0x7f8be80afe60 2023-10-06T04:59:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.033+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8be8071680 con 0x7f8be80afe60 2023-10-06T04:59:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.896+0000 7f8bde7fc700 1 -- 172.21.15.3:0/2174131778 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v105) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8be0056010 con 0x7f8be80afe60 2023-10-06T04:59:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.902+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bc8066f30 msgr2=0x7f8bc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.902+0000 7f8becfd0700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bc8066f30 0x7f8bc80693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8be816ce80 tx=0x7f8bd0009040 comp rx=0 tx=0).stop 2023-10-06T04:59:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.902+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be80afe60 msgr2=0x7f8be80b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.902+0000 7f8becfd0700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be80afe60 0x7f8be80b4210 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8be00093f0 tx=0x7f8be0017000 comp rx=0 tx=0).stop 2023-10-06T04:59:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 shutdown_connections 2023-10-06T04:59:52.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8bc8066f30 0x7f8bc80693e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:52.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8be815cb10 0x7f8be80c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:52.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be80afe60 0x7f8be80b4210 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:52.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 --2- 172.21.15.3:0/2174131778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8be815c740 0x7f8be80c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:52.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 >> 172.21.15.3:0/2174131778 conn(0x7f8be806f960 msgr2=0x7f8be815a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:52.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 shutdown_connections 2023-10-06T04:59:52.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:52.903+0000 7f8becfd0700 1 -- 172.21.15.3:0/2174131778 wait complete. 2023-10-06T04:59:52.908 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T04:59:52.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:59:53.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.051+0000 7fc41acdd700 1 Processor -- start 2023-10-06T04:59:53.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.051+0000 7fc41acdd700 1 -- start start 2023-10-06T04:59:53.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc41acdd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4140cba50 0x7fc414150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc41acdd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc41415d540 0x7fc4140af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc41acdd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41415a290 con 0x7fc414150690 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc41acdd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140d0020 con 0x7fc4140cba50 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc41acdd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc414159f40 con 0x7fc41415d540 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc418a79700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc414150690 0x7fc41415a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49964/0 (socket says 172.21.15.3:49964) 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc418a79700 1 -- 172.21.15.3:0/3580762458 learned_addr learned my addr 172.21.15.3:0/3580762458 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.052+0000 7fc413fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4140cba50 0x7fc414150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc41927a700 1 --2- 172.21.15.3:0/3580762458 >> v2:172.21.15.153:3300/0 conn(0x7fc41415d540 0x7fc4140af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc4137fe700 1 -- 172.21.15.3:0/3580762458 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947471975 0 0) 0x7fc41415a290 con 0x7fc414150690 2023-10-06T04:59:53.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc4137fe700 1 -- 172.21.15.3:0/3580762458 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc400003570 con 0x7fc414150690 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc413fff700 1 -- 172.21.15.3:0/3580762458 >> v2:172.21.15.153:3300/0 conn(0x7fc41415d540 msgr2=0x7fc4140af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc413fff700 1 --2- 172.21.15.3:0/3580762458 >> v2:172.21.15.153:3300/0 conn(0x7fc41415d540 0x7fc4140af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc413fff700 1 -- 172.21.15.3:0/3580762458 >> v1:172.21.15.3:6789/0 conn(0x7fc414150690 legacy=0x7fc41415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc413fff700 1 -- 172.21.15.3:0/3580762458 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41415dcb0 con 0x7fc4140cba50 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc413fff700 1 --2- 172.21.15.3:0/3580762458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4140cba50 0x7fc414150a60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc41415a200 tx=0x7fc404009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e6500e7d8e58ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc4137fe700 1 -- 172.21.15.3:0/3580762458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400e070 con 0x7fc4140cba50 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc4137fe700 1 -- 172.21.15.3:0/3580762458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40400bab0 con 0x7fc4140cba50 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.053+0000 7fc4137fe700 1 -- 172.21.15.3:0/3580762458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400be70 con 0x7fc4140cba50 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 -- 172.21.15.3:0/3580762458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4140cba50 msgr2=0x7fc414150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 --2- 172.21.15.3:0/3580762458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4140cba50 0x7fc414150a60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc41415a200 tx=0x7fc404009780 comp rx=0 tx=0).stop 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 -- 172.21.15.3:0/3580762458 shutdown_connections 2023-10-06T04:59:53.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 --2- 172.21.15.3:0/3580762458 >> v2:172.21.15.153:3300/0 conn(0x7fc41415d540 0x7fc4140af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 --2- 172.21.15.3:0/3580762458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4140cba50 0x7fc414150a60 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 -- 172.21.15.3:0/3580762458 >> 172.21.15.3:0/3580762458 conn(0x7fc414070310 msgr2=0x7fc414070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:53.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 -- 172.21.15.3:0/3580762458 shutdown_connections 2023-10-06T04:59:53.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.054+0000 7fc41acdd700 1 -- 172.21.15.3:0/3580762458 wait complete. 2023-10-06T04:59:53.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.055+0000 7fc41acdd700 1 Processor -- start 2023-10-06T04:59:53.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.055+0000 7fc41acdd700 1 -- start start 2023-10-06T04:59:53.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41acdd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4140cba50 0x7fc41414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41acdd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc414150690 0x7fc4140c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41acdd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41415d540 0x7fc414151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41acdd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c7e20 con 0x7fc414150690 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41acdd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41414f9d0 con 0x7fc4140cba50 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41acdd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c9a60 con 0x7fc41415d540 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc414150690 0x7fc4140c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc414150690 0x7fc4140c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47846/0 (socket says 172.21.15.3:47846) 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 -- 172.21.15.3:0/1628969107 learned_addr learned my addr 172.21.15.3:0/1628969107 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41927a700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41415d540 0x7fc414151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 -- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41415d540 msgr2=0x7fc414151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:53.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41415d540 0x7fc414151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 -- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4140cba50 msgr2=0x7fc41414f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc418a79700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4140cba50 0x7fc41414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4140cba50 0x7fc41414f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 -- 172.21.15.3:0/1628969107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc404009a20 con 0x7fc414150690 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc41927a700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41415d540 0x7fc414151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc418a79700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4140cba50 0x7fc41414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc413fff700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc414150690 0x7fc4140c9160 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc4140c71f0 tx=0x7fc404019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.056+0000 7fc410ff9700 1 -- 172.21.15.3:0/1628969107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404012cc0 con 0x7fc414150690 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.057+0000 7fc410ff9700 1 -- 172.21.15.3:0/1628969107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404012e20 con 0x7fc414150690 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.057+0000 7fc410ff9700 1 -- 172.21.15.3:0/1628969107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4040276c0 con 0x7fc414150690 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.057+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140c74f0 con 0x7fc414150690 2023-10-06T04:59:53.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.057+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4140c7ab0 con 0x7fc414150690 2023-10-06T04:59:53.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.058+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4140c7fd0 con 0x7fc414150690 2023-10-06T04:59:53.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.058+0000 7fc410ff9700 1 -- 172.21.15.3:0/1628969107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc404024070 con 0x7fc414150690 2023-10-06T04:59:53.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.058+0000 7fc410ff9700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f4066f30 0x7fc3f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:53.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.058+0000 7fc410ff9700 1 -- 172.21.15.3:0/1628969107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fc404086a70 con 0x7fc414150690 2023-10-06T04:59:53.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.059+0000 7fc418a79700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f4066f30 0x7fc3f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:53.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.059+0000 7fc418a79700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f4066f30 0x7fc3f40693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0058d0 tx=0x7fc40c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:53.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.063+0000 7fc410ff9700 1 -- 172.21.15.3:0/1628969107 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4140c7fd0 con 0x7fc414150690 2023-10-06T04:59:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.255+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc41404c380 con 0x7fc414150690 2023-10-06T04:59:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.903+0000 7fc410ff9700 1 -- 172.21.15.3:0/1628969107 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v106) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc40402c0d0 con 0x7fc414150690 2023-10-06T04:59:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f4066f30 msgr2=0x7fc3f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f4066f30 0x7fc3f40693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0058d0 tx=0x7fc40c009040 comp rx=0 tx=0).stop 2023-10-06T04:59:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc414150690 msgr2=0x7fc4140c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc414150690 0x7fc4140c9160 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc4140c71f0 tx=0x7fc404019000 comp rx=0 tx=0).stop 2023-10-06T04:59:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 shutdown_connections 2023-10-06T04:59:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f4066f30 0x7fc3f40693e0 secure :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0058d0 tx=0x7fc40c009040 comp rx=0 tx=0).stop 2023-10-06T04:59:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4140cba50 0x7fc41414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41415d540 0x7fc414151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 --2- 172.21.15.3:0/1628969107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc414150690 0x7fc4140c9160 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.916+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 >> 172.21.15.3:0/1628969107 conn(0x7fc414070310 msgr2=0x7fc41415c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.917+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 shutdown_connections 2023-10-06T04:59:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:53.918+0000 7fc41acdd700 1 -- 172.21.15.3:0/1628969107 wait complete. 2023-10-06T04:59:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T04:59:53.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:59:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.059+0000 7f6bc3246700 1 Processor -- start 2023-10-06T04:59:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.059+0000 7f6bc3246700 1 -- start start 2023-10-06T04:59:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc3246700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 0x7f6bbc0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc3246700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc0cb190 con 0x7f6bbc0af660 2023-10-06T04:59:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc3246700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc0c65a0 con 0x7f6bbc0af290 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc3246700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc159950 con 0x7f6bbc0c5c20 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc1a43700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6bbc0af660 0x7f6bbc0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49974/0 (socket says 172.21.15.3:49974) 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc1a43700 1 -- 172.21.15.3:0/368525635 learned_addr learned my addr 172.21.15.3:0/368525635 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc2a45700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 0x7f6bbc0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633281901 0 0) 0x7f6bbc0cb190 con 0x7f6bbc0af660 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.060+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba4003570 con 0x7f6bbc0af660 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 975782048 0 0) 0x7f6ba4003570 con 0x7f6bbc0af660 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 msgr2=0x7f6bbc0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 0x7f6bbc0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 >> v1:172.21.15.153:6789/0 conn(0x7f6bbc0af290 legacy=0x7f6bbc0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0cd6a0 con 0x7f6bbc0af660 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6bb00015f0 con 0x7f6bbc0af660 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 >> v1:172.21.15.3:6789/0 conn(0x7f6bbc0af660 legacy=0x7f6bbc0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ba4003a90 0x7f6ba4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 0x7f6ba4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba4004d50 0x7f6ba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc159950 con 0x7f6ba4003a90 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba400bb10 con 0x7f6ba4004d50 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba400bcd0 con 0x7f6bbc0c5c20 2023-10-06T04:59:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2244700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ba4003a90 0x7f6ba4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc1a43700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 0x7f6ba4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2a45700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba4004d50 0x7f6ba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2244700 1 -- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 msgr2=0x7f6ba4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2244700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bbc0c5c20 0x7f6ba4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2244700 1 -- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba4004d50 msgr2=0x7f6ba40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2244700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba4004d50 0x7f6ba40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2244700 1 -- 172.21.15.3:0/368525635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba4003570 con 0x7f6ba4003a90 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.061+0000 7f6bc2244700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ba4003a90 0x7f6ba4003e80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4007fd0 tx=0x7f6bb4007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7218af049cb3a214 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4001330 con 0x7f6ba4003a90 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb400c5d0 con 0x7f6ba4003a90 2023-10-06T04:59:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc1242700 1 -- 172.21.15.3:0/368525635 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400c970 con 0x7f6ba4003a90 2023-10-06T04:59:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc3246700 1 -- 172.21.15.3:0/368525635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ba4003a90 msgr2=0x7f6ba4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc3246700 1 --2- 172.21.15.3:0/368525635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ba4003a90 0x7f6ba4003e80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4007fd0 tx=0x7f6bb4007d60 comp rx=0 tx=0).stop 2023-10-06T04:59:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc2a45700 1 -- 172.21.15.3:0/368525635 reap_dead start 2023-10-06T04:59:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc3246700 1 -- 172.21.15.3:0/368525635 shutdown_connections 2023-10-06T04:59:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc3246700 1 -- 172.21.15.3:0/368525635 >> 172.21.15.3:0/368525635 conn(0x7f6bbc089dd0 msgr2=0x7f6bbc0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc3246700 1 -- 172.21.15.3:0/368525635 shutdown_connections 2023-10-06T04:59:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.062+0000 7f6bc3246700 1 -- 172.21.15.3:0/368525635 wait complete. 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.063+0000 7f6bc3246700 1 Processor -- start 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.063+0000 7f6bc3246700 1 -- start start 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc3246700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba4003a90 0x7f6bbc0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc3246700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bbc141d50 0x7f6bbc0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc3246700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bbc0b4560 0x7f6bbc0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc3246700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc15a190 con 0x7f6bbc0b4560 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc3246700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0b0d90 con 0x7f6bbc141d50 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc3246700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc159eb0 con 0x7f6ba4003a90 2023-10-06T04:59:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc2a45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bbc0b4560 0x7f6bbc0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc2244700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba4003a90 0x7f6bbc0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc2a45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bbc0b4560 0x7f6bbc0b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47860/0 (socket says 172.21.15.3:47860) 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc2a45700 1 -- 172.21.15.3:0/3565023805 learned_addr learned my addr 172.21.15.3:0/3565023805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc1a43700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bbc141d50 0x7f6bbc0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc2a45700 1 -- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba4003a90 msgr2=0x7f6bbc0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.064+0000 7f6bc2a45700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba4003a90 0x7f6bbc0b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bc2a45700 1 -- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bbc141d50 msgr2=0x7f6bbc0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bc2a45700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bbc141d50 0x7f6bbc0b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bc2a45700 1 -- 172.21.15.3:0/3565023805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0b0740 con 0x7f6bbc0b4560 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bc2244700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba4003a90 0x7f6bbc0b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bc2a45700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bbc0b4560 0x7f6bbc0b49d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc064850 tx=0x7f6bac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:54.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bba7fc700 1 -- 172.21.15.3:0/3565023805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00e040 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bba7fc700 1 -- 172.21.15.3:0/3565023805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac00f040 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc0b0a20 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bba7fc700 1 -- 172.21.15.3:0/3565023805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00fd70 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.065+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc16cef0 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.066+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc04c380 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.067+0000 7f6bba7fc700 1 -- 172.21.15.3:0/3565023805 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6bac015640 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.067+0000 7f6bba7fc700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ba8066f30 0x7f6ba80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.067+0000 7f6bba7fc700 1 -- 172.21.15.3:0/3565023805 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f6bac084cf0 con 0x7f6bbc0b4560 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.067+0000 7f6bc2244700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ba8066f30 0x7f6ba80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.068+0000 7f6bc2244700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ba8066f30 0x7f6ba80693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4001060 tx=0x7f6bb4015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.072+0000 7f6bba7fc700 1 -- 172.21.15.3:0/3565023805 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bac089070 con 0x7f6bbc0b4560 2023-10-06T04:59:54.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.312+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6bbc0cd3c0 con 0x7f6bbc0b4560 2023-10-06T04:59:54.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.906+0000 7f6bba7fc700 1 -- 172.21.15.3:0/3565023805 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v65) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6bac051e60 con 0x7f6bbc0b4560 2023-10-06T04:59:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ba8066f30 msgr2=0x7f6ba80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ba8066f30 0x7f6ba80693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4001060 tx=0x7f6bb4015000 comp rx=0 tx=0).stop 2023-10-06T04:59:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bbc0b4560 msgr2=0x7f6bbc0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bbc0b4560 0x7f6bbc0b49d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc064850 tx=0x7f6bac00b040 comp rx=0 tx=0).stop 2023-10-06T04:59:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 shutdown_connections 2023-10-06T04:59:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ba8066f30 0x7f6ba80693e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bbc0b4560 0x7f6bbc0b49d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba4003a90 0x7f6bbc0b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 --2- 172.21.15.3:0/3565023805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bbc141d50 0x7f6bbc0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.911+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 >> 172.21.15.3:0/3565023805 conn(0x7f6bbc089dd0 msgr2=0x7f6bbc0c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.912+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 shutdown_connections 2023-10-06T04:59:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:54.912+0000 7f6bc3246700 1 -- 172.21.15.3:0/3565023805 wait complete. 2023-10-06T04:59:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 17 and data pool 18 2023-10-06T04:59:54.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:55.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.049+0000 7fa9f892e700 1 Processor -- start 2023-10-06T04:59:55.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.049+0000 7fa9f892e700 1 -- start start 2023-10-06T04:59:55.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f892e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9f4157400 0x7fa9f40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f892e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa9f40b3b70 0x7fa9f406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f892e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f415aeb0 con 0x7fa9f4157030 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f892e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f415a7a0 con 0x7fa9f4157400 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f892e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f415aa80 con 0x7fa9f40b3b70 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9f4157400 0x7fa9f40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9f4157400 0x7fa9f40b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47876/0 (socket says 172.21.15.3:47876) 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f1d9b700 1 -- 172.21.15.3:0/2317504289 learned_addr learned my addr 172.21.15.3:0/2317504289 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.050+0000 7fa9f2d9d700 1 --2- 172.21.15.3:0/2317504289 >> v2:172.21.15.153:3300/0 conn(0x7fa9f40b3b70 0x7fa9f406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f1d9b700 1 -- 172.21.15.3:0/2317504289 >> v2:172.21.15.153:3300/0 conn(0x7fa9f40b3b70 msgr2=0x7fa9f406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:55.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f1d9b700 1 --2- 172.21.15.3:0/2317504289 >> v2:172.21.15.153:3300/0 conn(0x7fa9f40b3b70 0x7fa9f406e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f1d9b700 1 -- 172.21.15.3:0/2317504289 >> v1:172.21.15.153:6789/0 conn(0x7fa9f4157030 legacy=0x7fa9f4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f1d9b700 1 -- 172.21.15.3:0/2317504289 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f406dba0 con 0x7fa9f4157400 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f1d9b700 1 --2- 172.21.15.3:0/2317504289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f40b3630 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40687c0 tx=0x7fa9dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42121d2dde2c0205 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f159a700 1 -- 172.21.15.3:0/2317504289 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00b820 con 0x7fa9f4157400 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f159a700 1 -- 172.21.15.3:0/2317504289 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc00b980 con 0x7fa9f4157400 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f159a700 1 -- 172.21.15.3:0/2317504289 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00bd20 con 0x7fa9f4157400 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f892e700 1 -- 172.21.15.3:0/2317504289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 msgr2=0x7fa9f40b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.051+0000 7fa9f892e700 1 --2- 172.21.15.3:0/2317504289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f40b3630 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40687c0 tx=0x7fa9dc009250 comp rx=0 tx=0).stop 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.052+0000 7fa9f892e700 1 -- 172.21.15.3:0/2317504289 shutdown_connections 2023-10-06T04:59:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.052+0000 7fa9f892e700 1 --2- 172.21.15.3:0/2317504289 >> v2:172.21.15.153:3300/0 conn(0x7fa9f40b3b70 0x7fa9f406e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.052+0000 7fa9f892e700 1 --2- 172.21.15.3:0/2317504289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f40b3630 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.052+0000 7fa9f892e700 1 -- 172.21.15.3:0/2317504289 >> 172.21.15.3:0/2317504289 conn(0x7fa9f40ce270 msgr2=0x7fa9f40ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:55.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.052+0000 7fa9f892e700 1 -- 172.21.15.3:0/2317504289 shutdown_connections 2023-10-06T04:59:55.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.052+0000 7fa9f892e700 1 -- 172.21.15.3:0/2317504289 wait complete. 2023-10-06T04:59:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 Processor -- start 2023-10-06T04:59:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 -- start start 2023-10-06T04:59:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f40b3b70 0x7fa9f40c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9f4157030 0x7fa9f40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f4148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40c7ae0 con 0x7fa9f4157400 2023-10-06T04:59:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40c7dc0 con 0x7fa9f40b3b70 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.053+0000 7fa9f892e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40c80a0 con 0x7fa9f4157030 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f4148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f4148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47878/0 (socket says 172.21.15.3:47878) 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 -- 172.21.15.3:0/3150370870 learned_addr learned my addr 172.21.15.3:0/3150370870 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f1d9b700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9f4157030 0x7fa9f40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f259c700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f40b3b70 0x7fa9f40c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 -- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9f4157030 msgr2=0x7fa9f40c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9f4157030 0x7fa9f40c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 -- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f40b3b70 msgr2=0x7fa9f40c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f40b3b70 0x7fa9f40c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 -- 172.21.15.3:0/3150370870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0094f0 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f2d9d700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f4148db0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400d890 tx=0x7fa9e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9ea7fc700 1 -- 172.21.15.3:0/3150370870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40125f0 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9ea7fc700 1 -- 172.21.15.3:0/3150370870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e4012770 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f40c8380 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9ea7fc700 1 -- 172.21.15.3:0/3150370870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4012b30 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.054+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f41492f0 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.055+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f41494e0 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.055+0000 7fa9ea7fc700 1 -- 172.21.15.3:0/3150370870 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9e401e460 con 0x7fa9f4157400 2023-10-06T04:59:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.056+0000 7fa9ea7fc700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa9d4066f30 0x7fa9d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.056+0000 7fa9ea7fc700 1 -- 172.21.15.3:0/3150370870 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa9e4084df0 con 0x7fa9f4157400 2023-10-06T04:59:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.056+0000 7fa9f259c700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa9d4066f30 0x7fa9d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.057+0000 7fa9f259c700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa9d4066f30 0x7fa9d40693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec007fd0 tx=0x7fa9ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.060+0000 7fa9ea7fc700 1 -- 172.21.15.3:0/3150370870 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9f41494e0 con 0x7fa9f4157400 2023-10-06T04:59:55.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.256+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9f41494e0 con 0x7fa9f4157400 2023-10-06T04:59:55.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.257+0000 7fa9ea7fc700 1 -- 172.21.15.3:0/3150370870 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16845 (secure 0 0 0) 0x7fa9f41494e0 con 0x7fa9f4157400 2023-10-06T04:59:55.259 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:55.259 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":107,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:54.313234+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:52.063522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:53.285974+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa9d4066f30 msgr2=0x7fa9d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa9d4066f30 0x7fa9d40693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec007fd0 tx=0x7fa9ec009040 comp rx=0 tx=0).stop 2023-10-06T04:59:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 msgr2=0x7fa9f4148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f4148db0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400d890 tx=0x7fa9e4008040 comp rx=0 tx=0).stop 2023-10-06T04:59:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 shutdown_connections 2023-10-06T04:59:55.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa9d4066f30 0x7fa9d40693e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f40b3b70 0x7fa9f40c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f4157400 0x7fa9f4148db0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 --2- 172.21.15.3:0/3150370870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9f4157030 0x7fa9f40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 >> 172.21.15.3:0/3150370870 conn(0x7fa9f40ce270 msgr2=0x7fa9f4156110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:55.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 shutdown_connections 2023-10-06T04:59:55.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.259+0000 7fa9f892e700 1 -- 172.21.15.3:0/3150370870 wait complete. 2023-10-06T04:59:55.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.390+0000 7f41838ea700 1 Processor -- start 2023-10-06T04:59:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.391+0000 7f41838ea700 1 -- start start 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.391+0000 7f41838ea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f417c15c8f0 0x7f417c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.391+0000 7f41838ea700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c0cf6c0 con 0x7f417c0d0870 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.391+0000 7f41838ea700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c0cfb50 con 0x7f417c15ccc0 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.391+0000 7f41838ea700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c15e050 con 0x7f417c15c8f0 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.391+0000 7f4181686700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f417c15c8f0 0x7f417c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f4181e87700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f417c0d0870 0x7f417c0b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49980/0 (socket says 172.21.15.3:49980) 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f4181e87700 1 -- 172.21.15.3:0/100788493 learned_addr learned my addr 172.21.15.3:0/100788493 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f4181686700 1 -- 172.21.15.3:0/100788493 >> v1:172.21.15.153:6789/0 conn(0x7f417c15ccc0 legacy=0x7f417c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f4181686700 1 -- 172.21.15.3:0/100788493 >> v1:172.21.15.3:6789/0 conn(0x7f417c0d0870 legacy=0x7f417c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f4181686700 1 -- 172.21.15.3:0/100788493 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0ceab0 con 0x7f417c15c8f0 2023-10-06T04:59:55.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f417bfff700 1 -- 172.21.15.3:0/100788493 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511454000 0 0) 0x7f417c0cf6c0 con 0x7f417c0d0870 2023-10-06T04:59:55.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f4181686700 1 --2- 172.21.15.3:0/100788493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15c8f0 0x7f417c15db10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f416c009e90 tx=0x7f416c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1194730a0c02ef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f417bfff700 1 -- 172.21.15.3:0/100788493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00e070 con 0x7f417c15c8f0 2023-10-06T04:59:55.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f417bfff700 1 -- 172.21.15.3:0/100788493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c00bbb0 con 0x7f417c15c8f0 2023-10-06T04:59:55.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.392+0000 7f417bfff700 1 -- 172.21.15.3:0/100788493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c012430 con 0x7f417c15c8f0 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.393+0000 7f41838ea700 1 -- 172.21.15.3:0/100788493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15c8f0 msgr2=0x7f417c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.393+0000 7f41838ea700 1 --2- 172.21.15.3:0/100788493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15c8f0 0x7f417c15db10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f416c009e90 tx=0x7f416c009780 comp rx=0 tx=0).stop 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.393+0000 7f41838ea700 1 -- 172.21.15.3:0/100788493 shutdown_connections 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.393+0000 7f41838ea700 1 --2- 172.21.15.3:0/100788493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15c8f0 0x7f417c15db10 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.393+0000 7f41838ea700 1 -- 172.21.15.3:0/100788493 >> 172.21.15.3:0/100788493 conn(0x7f417c070310 msgr2=0x7f417c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.393+0000 7f41838ea700 1 -- 172.21.15.3:0/100788493 shutdown_connections 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.393+0000 7f41838ea700 1 -- 172.21.15.3:0/100788493 wait complete. 2023-10-06T04:59:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.394+0000 7f41838ea700 1 Processor -- start 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.394+0000 7f41838ea700 1 -- start start 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f41838ea700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f417c0d0870 0x7f417c155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f41838ea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f417c15c8f0 0x7f417c0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f41838ea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15ccc0 0x7f417c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f41838ea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0c68d0 con 0x7f417c15ccc0 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f41838ea700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c155860 con 0x7f417c15c8f0 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f41838ea700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0c5cc0 con 0x7f417c0d0870 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15ccc0 0x7f417c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15ccc0 0x7f417c15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47906/0 (socket says 172.21.15.3:47906) 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 -- 172.21.15.3:0/2610494465 learned_addr learned my addr 172.21.15.3:0/2610494465 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181686700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f417c0d0870 0x7f417c155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4180e85700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f417c15c8f0 0x7f417c0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 -- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f417c0d0870 msgr2=0x7f417c155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f417c0d0870 0x7f417c155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 -- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f417c15c8f0 msgr2=0x7f417c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f417c15c8f0 0x7f417c0c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 -- 172.21.15.3:0/2610494465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009a20 con 0x7f417c15ccc0 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4180e85700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f417c15c8f0 0x7f417c0c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.395+0000 7f4181e87700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15ccc0 0x7f417c15b6b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f417400c560 tx=0x7f4174007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.396+0000 7f41797fa700 1 -- 172.21.15.3:0/2610494465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174005b30 con 0x7f417c15ccc0 2023-10-06T04:59:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.396+0000 7f41797fa700 1 -- 172.21.15.3:0/2610494465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4174005cb0 con 0x7f417c15ccc0 2023-10-06T04:59:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.396+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c0c5fa0 con 0x7f417c15ccc0 2023-10-06T04:59:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.396+0000 7f41797fa700 1 -- 172.21.15.3:0/2610494465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174007320 con 0x7f417c15ccc0 2023-10-06T04:59:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.396+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c0c6560 con 0x7f417c15ccc0 2023-10-06T04:59:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.397+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c0cccb0 con 0x7f417c15ccc0 2023-10-06T04:59:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.397+0000 7f41797fa700 1 -- 172.21.15.3:0/2610494465 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f417401b3b0 con 0x7f417c15ccc0 2023-10-06T04:59:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.398+0000 7f41797fa700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4168066f30 0x7f41680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.398+0000 7f41797fa700 1 -- 172.21.15.3:0/2610494465 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4174083c50 con 0x7f417c15ccc0 2023-10-06T04:59:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.398+0000 7f4181686700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4168066f30 0x7f41680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.399+0000 7f4181686700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4168066f30 0x7f41680693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f416c0099f0 tx=0x7f416c018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.402+0000 7f41797fa700 1 -- 172.21.15.3:0/2610494465 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4174050c30 con 0x7f417c15ccc0 2023-10-06T04:59:55.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:55.499 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:55.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.600+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f417c0c6a10 con 0x7f417c15ccc0 2023-10-06T04:59:55.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.600+0000 7f41797fa700 1 -- 172.21.15.3:0/2610494465 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16832 (secure 0 0 0) 0x7f417c0c6a10 con 0x7f417c15ccc0 2023-10-06T04:59:55.602 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:55.602 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":108,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:55.572851+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:52.063522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:53.285974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.602+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4168066f30 msgr2=0x7f41680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.602+0000 7f41838ea700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4168066f30 0x7f41680693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f416c0099f0 tx=0x7f416c018000 comp rx=0 tx=0).stop 2023-10-06T04:59:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15ccc0 msgr2=0x7f417c15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15ccc0 0x7f417c15b6b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f417400c560 tx=0x7f4174007cf0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 shutdown_connections 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4168066f30 0x7f41680693e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f417c15ccc0 0x7f417c15b6b0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f417c0d0870 0x7f417c155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 --2- 172.21.15.3:0/2610494465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f417c15c8f0 0x7f417c0c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 >> 172.21.15.3:0/2610494465 conn(0x7f417c070310 msgr2=0x7f417c0b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 shutdown_connections 2023-10-06T04:59:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.603+0000 7f41838ea700 1 -- 172.21.15.3:0/2610494465 wait complete. 2023-10-06T04:59:55.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:59:55.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.738+0000 7ffa8a224700 1 Processor -- start 2023-10-06T04:59:55.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.738+0000 7ffa8a224700 1 -- start start 2023-10-06T04:59:55.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa8a224700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffa840af8a0 0x7ffa84150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa8a224700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8415a290 con 0x7ffa840cba50 2023-10-06T04:59:55.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa8a224700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa840d0020 con 0x7ffa84150690 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa8a224700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa84159f40 con 0x7ffa840af8a0 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa82ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffa840cba50 0x7ffa840af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49982/0 (socket says 172.21.15.3:49982) 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa82ffd700 1 -- 172.21.15.3:0/1875911124 learned_addr learned my addr 172.21.15.3:0/1875911124 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa83fff700 1 --2- 172.21.15.3:0/1875911124 >> v2:172.21.15.153:3300/0 conn(0x7ffa840af8a0 0x7ffa84150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965342105 0 0) 0x7ffa8415a290 con 0x7ffa840cba50 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa6c003570 con 0x7ffa840cba50 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1373679362 0 0) 0x7ffa6c003570 con 0x7ffa840cba50 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 >> v2:172.21.15.153:3300/0 conn(0x7ffa840af8a0 msgr2=0x7ffa84150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 --2- 172.21.15.3:0/1875911124 >> v2:172.21.15.153:3300/0 conn(0x7ffa840af8a0 0x7ffa84150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 >> v1:172.21.15.153:6789/0 conn(0x7ffa84150690 legacy=0x7ffa8415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8415dcb0 con 0x7ffa840cba50 2023-10-06T04:59:55.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7ffa74001470 con 0x7ffa840cba50 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.739+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 >> v1:172.21.15.3:6789/0 conn(0x7ffa840cba50 legacy=0x7ffa840af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c003a60 0x7ffa6c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c004810 0x7ffa6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa6c0051c0 0x7ffa6c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa84159f40 con 0x7ffa6c003a60 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c00c160 con 0x7ffa6c0051c0 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c00c320 con 0x7ffa6c004810 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa837fe700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c003a60 0x7ffa6c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa82ffd700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c004810 0x7ffa6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa83fff700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa6c0051c0 0x7ffa6c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa837fe700 1 -- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c004810 msgr2=0x7ffa6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa837fe700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c004810 0x7ffa6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa837fe700 1 -- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa6c0051c0 msgr2=0x7ffa6c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa837fe700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa6c0051c0 0x7ffa6c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa837fe700 1 -- 172.21.15.3:0/1875911124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c003570 con 0x7ffa6c003a60 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa83fff700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa6c0051c0 0x7ffa6c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa83fff700 1 -- 172.21.15.3:0/1875911124 reap_dead start 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa837fe700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c003a60 0x7ffa6c003e70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ffa780018b0 tx=0x7ffa78007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d045b1dfdee89e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78012420 con 0x7ffa6c003a60 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa780125a0 con 0x7ffa6c003a60 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.740+0000 7ffa827fc700 1 -- 172.21.15.3:0/1875911124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78012960 con 0x7ffa6c003a60 2023-10-06T04:59:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.741+0000 7ffa8a224700 1 -- 172.21.15.3:0/1875911124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c003a60 msgr2=0x7ffa6c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.741+0000 7ffa8a224700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c003a60 0x7ffa6c003e70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ffa780018b0 tx=0x7ffa78007d30 comp rx=0 tx=0).stop 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.741+0000 7ffa8a224700 1 -- 172.21.15.3:0/1875911124 shutdown_connections 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.741+0000 7ffa8a224700 1 --2- 172.21.15.3:0/1875911124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c003a60 0x7ffa6c003e70 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.741+0000 7ffa8a224700 1 -- 172.21.15.3:0/1875911124 >> 172.21.15.3:0/1875911124 conn(0x7ffa84070310 msgr2=0x7ffa84070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.741+0000 7ffa8a224700 1 -- 172.21.15.3:0/1875911124 shutdown_connections 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.741+0000 7ffa8a224700 1 -- 172.21.15.3:0/1875911124 wait complete. 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.742+0000 7ffa8a224700 1 Processor -- start 2023-10-06T04:59:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.742+0000 7ffa8a224700 1 -- start start 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa8a224700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c003a60 0x7ffa8415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa8a224700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8415c430 0x7ffa840b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa8a224700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa840b5600 0x7ffa84151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa8a224700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8415b460 con 0x7ffa8415c430 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa8a224700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa840b32f0 con 0x7ffa840b5600 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa8a224700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa840b35d0 con 0x7ffa6c003a60 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8415c430 0x7ffa840b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8415c430 0x7ffa840b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47936/0 (socket says 172.21.15.3:47936) 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 -- 172.21.15.3:0/2359446168 learned_addr learned my addr 172.21.15.3:0/2359446168 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa837fe700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c003a60 0x7ffa8415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa83fff700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa840b5600 0x7ffa84151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 -- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c003a60 msgr2=0x7ffa8415bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c003a60 0x7ffa8415bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 -- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa840b5600 msgr2=0x7ffa84151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa840b5600 0x7ffa84151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 -- 172.21.15.3:0/2359446168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840b38b0 con 0x7ffa8415c430 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa83fff700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa840b5600 0x7ffa84151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.743+0000 7ffa82ffd700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8415c430 0x7ffa840b50c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ffa74001ce0 tx=0x7ffa74001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.744+0000 7ffa6b7fe700 1 -- 172.21.15.3:0/2359446168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400f6c0 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.744+0000 7ffa6b7fe700 1 -- 172.21.15.3:0/2359446168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7400f840 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.744+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa840b3b90 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.744+0000 7ffa6b7fe700 1 -- 172.21.15.3:0/2359446168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400fc00 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.744+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa841516d0 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.745+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8404c380 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.745+0000 7ffa6b7fe700 1 -- 172.21.15.3:0/2359446168 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa7401d670 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.745+0000 7ffa6b7fe700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa70066f30 0x7ffa700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.745+0000 7ffa6b7fe700 1 -- 172.21.15.3:0/2359446168 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa74084a50 con 0x7ffa8415c430 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.745+0000 7ffa837fe700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa70066f30 0x7ffa700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:55.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.746+0000 7ffa837fe700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa70066f30 0x7ffa700693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ffa78001010 tx=0x7ffa78015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:55.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.749+0000 7ffa6b7fe700 1 -- 172.21.15.3:0/2359446168 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa74051b60 con 0x7ffa8415c430 2023-10-06T04:59:55.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:55.990+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffa8415df90 con 0x7ffa8415c430 2023-10-06T04:59:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.924+0000 7ffa6b7fe700 1 -- 172.21.15.3:0/2359446168 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffa74055180 con 0x7ffa8415c430 2023-10-06T04:59:56.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa70066f30 msgr2=0x7ffa700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa70066f30 0x7ffa700693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ffa78001010 tx=0x7ffa78015000 comp rx=0 tx=0).stop 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8415c430 msgr2=0x7ffa840b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8415c430 0x7ffa840b50c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ffa74001ce0 tx=0x7ffa74001e50 comp rx=0 tx=0).stop 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 shutdown_connections 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ffa70066f30 0x7ffa700693e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa840b5600 0x7ffa84151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa6c003a60 0x7ffa8415bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 --2- 172.21.15.3:0/2359446168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8415c430 0x7ffa840b50c0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.927+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 >> 172.21.15.3:0/2359446168 conn(0x7ffa84070310 msgr2=0x7ffa840706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.928+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 shutdown_connections 2023-10-06T04:59:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:56.928+0000 7ffa8a224700 1 -- 172.21.15.3:0/2359446168 wait complete. 2023-10-06T04:59:56.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:59:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.072+0000 7fe67a01d700 1 Processor -- start 2023-10-06T04:59:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.072+0000 7fe67a01d700 1 -- start start 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe67a01d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe674067dc0 0x7fe674154c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe67a01d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6740b39a0 0x7fe674155150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe67a01d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67415bcc0 con 0x7fe674068190 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe67a01d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6740b3e20 con 0x7fe6740b39a0 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe67a01d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6740b4000 con 0x7fe674067dc0 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe673fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6740b39a0 0x7fe674155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe672ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe674068190 0x7fe6740b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49996/0 (socket says 172.21.15.3:49996) 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe672ffd700 1 -- 172.21.15.3:0/271966969 learned_addr learned my addr 172.21.15.3:0/271966969 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe6737fe700 1 --2- 172.21.15.3:0/271966969 >> v2:172.21.15.153:3300/0 conn(0x7fe674067dc0 0x7fe674154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:57.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe673fff700 1 -- 172.21.15.3:0/271966969 >> v2:172.21.15.153:3300/0 conn(0x7fe674067dc0 msgr2=0x7fe674154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe673fff700 1 --2- 172.21.15.3:0/271966969 >> v2:172.21.15.153:3300/0 conn(0x7fe674067dc0 0x7fe674154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe673fff700 1 -- 172.21.15.3:0/271966969 >> v1:172.21.15.3:6789/0 conn(0x7fe674068190 legacy=0x7fe6740b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe673fff700 1 -- 172.21.15.3:0/271966969 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67415b390 con 0x7fe6740b39a0 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe6727fc700 1 -- 172.21.15.3:0/271966969 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317881678 0 0) 0x7fe67415bcc0 con 0x7fe674068190 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe673fff700 1 --2- 172.21.15.3:0/271966969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe674155150 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe67406af90 tx=0x7fe66c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d9f071a50a4ab7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.073+0000 7fe6727fc700 1 -- 172.21.15.3:0/271966969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00e070 con 0x7fe6740b39a0 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe6727fc700 1 -- 172.21.15.3:0/271966969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c00bc30 con 0x7fe6740b39a0 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe6727fc700 1 -- 172.21.15.3:0/271966969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c012430 con 0x7fe6740b39a0 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 -- 172.21.15.3:0/271966969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 msgr2=0x7fe674155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 --2- 172.21.15.3:0/271966969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe674155150 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe67406af90 tx=0x7fe66c009780 comp rx=0 tx=0).stop 2023-10-06T04:59:57.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 -- 172.21.15.3:0/271966969 shutdown_connections 2023-10-06T04:59:57.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 --2- 172.21.15.3:0/271966969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe674155150 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 --2- 172.21.15.3:0/271966969 >> v2:172.21.15.153:3300/0 conn(0x7fe674067dc0 0x7fe674154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 -- 172.21.15.3:0/271966969 >> 172.21.15.3:0/271966969 conn(0x7fe6740ce270 msgr2=0x7fe6740ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:57.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 -- 172.21.15.3:0/271966969 shutdown_connections 2023-10-06T04:59:57.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.074+0000 7fe67a01d700 1 -- 172.21.15.3:0/271966969 wait complete. 2023-10-06T04:59:57.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.075+0000 7fe67a01d700 1 Processor -- start 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.075+0000 7fe67a01d700 1 -- start start 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe67a01d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674067dc0 0x7fe67414df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe67a01d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe674068190 0x7fe674159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe67a01d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe6740aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe67a01d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674152a70 con 0x7fe6740b39a0 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe67a01d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674151e60 con 0x7fe674067dc0 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe67a01d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe674152140 con 0x7fe674068190 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe673fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe6740aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe673fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe6740aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47946/0 (socket says 172.21.15.3:47946) 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe673fff700 1 -- 172.21.15.3:0/1463721407 learned_addr learned my addr 172.21.15.3:0/1463721407 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe6737fe700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674067dc0 0x7fe67414df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe672ffd700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe674068190 0x7fe674159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe673fff700 1 -- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe674068190 msgr2=0x7fe674159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe673fff700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe674068190 0x7fe674159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe673fff700 1 -- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674067dc0 msgr2=0x7fe67414df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.076+0000 7fe673fff700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674067dc0 0x7fe67414df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.077+0000 7fe673fff700 1 -- 172.21.15.3:0/1463721407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c009a20 con 0x7fe6740b39a0 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.077+0000 7fe673fff700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe6740aeb00 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe67406b700 tx=0x7fe66c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.077+0000 7fe66b7fe700 1 -- 172.21.15.3:0/1463721407 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c027470 con 0x7fe6740b39a0 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.077+0000 7fe66b7fe700 1 -- 172.21.15.3:0/1463721407 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c0275d0 con 0x7fe6740b39a0 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.077+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe674152420 con 0x7fe6740b39a0 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.077+0000 7fe66b7fe700 1 -- 172.21.15.3:0/1463721407 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c027990 con 0x7fe6740b39a0 2023-10-06T04:59:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.077+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67414e480 con 0x7fe6740b39a0 2023-10-06T04:59:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.078+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe674152c20 con 0x7fe6740b39a0 2023-10-06T04:59:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.078+0000 7fe66b7fe700 1 -- 172.21.15.3:0/1463721407 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe66c024070 con 0x7fe6740b39a0 2023-10-06T04:59:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.079+0000 7fe66b7fe700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe654066f60 0x7fe654069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.079+0000 7fe66b7fe700 1 -- 172.21.15.3:0/1463721407 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe66c086c50 con 0x7fe6740b39a0 2023-10-06T04:59:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.079+0000 7fe6737fe700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe654066f60 0x7fe654069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:57.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.080+0000 7fe6737fe700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe654066f60 0x7fe654069410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006aa0 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:57.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.083+0000 7fe66b7fe700 1 -- 172.21.15.3:0/1463721407 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe674152c20 con 0x7fe6740b39a0 2023-10-06T04:59:57.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.324+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe67404c380 con 0x7fe6740b39a0 2023-10-06T04:59:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.929+0000 7fe66b7fe700 1 -- 172.21.15.3:0/1463721407 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe66c08c020 con 0x7fe6740b39a0 2023-10-06T04:59:57.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.931+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe654066f60 msgr2=0x7fe654069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:57.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.931+0000 7fe67a01d700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe654066f60 0x7fe654069410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006aa0 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2023-10-06T04:59:57.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.931+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 msgr2=0x7fe6740aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.931+0000 7fe67a01d700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe6740aeb00 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe67406b700 tx=0x7fe66c019000 comp rx=0 tx=0).stop 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 shutdown_connections 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe654066f60 0x7fe654069410 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674067dc0 0x7fe67414df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6740b39a0 0x7fe6740aeb00 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 --2- 172.21.15.3:0/1463721407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe674068190 0x7fe674159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 >> 172.21.15.3:0/1463721407 conn(0x7fe6740ce270 msgr2=0x7fe67414ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 shutdown_connections 2023-10-06T04:59:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:57.932+0000 7fe67a01d700 1 -- 172.21.15.3:0/1463721407 wait complete. 2023-10-06T04:59:57.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:59:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.067+0000 7f975d93c700 1 Processor -- start 2023-10-06T04:59:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.067+0000 7f975d93c700 1 -- start start 2023-10-06T04:59:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.068+0000 7f975d93c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9758159500 0x7f97580cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.068+0000 7f975d93c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580cc5c0 con 0x7f97580b2930 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.068+0000 7f975d93c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580cb9b0 con 0x7f97581598d0 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.068+0000 7f975d93c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97580cbc90 con 0x7f9758159500 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.068+0000 7f97577fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f97580b2930 0x7f97580cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49998/0 (socket says 172.21.15.3:49998) 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.068+0000 7f97577fe700 1 -- 172.21.15.3:0/4212370916 learned_addr learned my addr 172.21.15.3:0/4212370916 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.068+0000 7f9756ffd700 1 --2- 172.21.15.3:0/4212370916 >> v2:172.21.15.153:3300/0 conn(0x7f9758159500 0x7f97580cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317983111 0 0) 0x7f97580cc5c0 con 0x7f97580b2930 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9744003570 con 0x7f97580b2930 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782171586 0 0) 0x7f97580cb9b0 con 0x7f97581598d0 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97580cc5c0 con 0x7f97581598d0 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2715881926 0 0) 0x7f9744003570 con 0x7f97580b2930 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9756ffd700 1 -- 172.21.15.3:0/4212370916 >> v1:172.21.15.153:6789/0 conn(0x7f97581598d0 legacy=0x7f97580b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9756ffd700 1 -- 172.21.15.3:0/4212370916 >> v1:172.21.15.3:6789/0 conn(0x7f97580b2930 legacy=0x7f97580cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9756ffd700 1 -- 172.21.15.3:0/4212370916 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580cbf70 con 0x7f9758159500 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9756ffd700 1 --2- 172.21.15.3:0/4212370916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758159500 0x7f97580cad30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f97500098e0 tx=0x7f97500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c134489f9a6d987 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97480017f0 con 0x7f9758159500 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9750011420 con 0x7f9758159500 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.069+0000 7f9755ffb700 1 -- 172.21.15.3:0/4212370916 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97500117c0 con 0x7f9758159500 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.070+0000 7f975d93c700 1 -- 172.21.15.3:0/4212370916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758159500 msgr2=0x7f97580cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.070+0000 7f975d93c700 1 --2- 172.21.15.3:0/4212370916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758159500 0x7f97580cad30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f97500098e0 tx=0x7f97500091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.070+0000 7f975d93c700 1 -- 172.21.15.3:0/4212370916 shutdown_connections 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.070+0000 7f975d93c700 1 --2- 172.21.15.3:0/4212370916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758159500 0x7f97580cad30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.070+0000 7f975d93c700 1 -- 172.21.15.3:0/4212370916 >> 172.21.15.3:0/4212370916 conn(0x7f9758070310 msgr2=0x7f9758070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.070+0000 7f975d93c700 1 -- 172.21.15.3:0/4212370916 shutdown_connections 2023-10-06T04:59:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.070+0000 7f975d93c700 1 -- 172.21.15.3:0/4212370916 wait complete. 2023-10-06T04:59:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.071+0000 7f975d93c700 1 Processor -- start 2023-10-06T04:59:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.071+0000 7f975d93c700 1 -- start start 2023-10-06T04:59:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f975d93c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97580b2930 0x7f97580b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f975d93c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9758159500 0x7f975815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f975d93c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97581598d0 0x7f975815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f975d93c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975815ae80 con 0x7f97581598d0 2023-10-06T04:59:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f975d93c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758154820 con 0x7f97580b2930 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f975d93c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97581550c0 con 0x7f9758159500 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97581598d0 0x7f975815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97581598d0 0x7f975815a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47950/0 (socket says 172.21.15.3:47950) 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 -- 172.21.15.3:0/26304794 learned_addr learned my addr 172.21.15.3:0/26304794 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f9756ffd700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97580b2930 0x7f97580b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97567fc700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9758159500 0x7f975815a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 -- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9758159500 msgr2=0x7f975815a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9758159500 0x7f975815a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 -- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97580b2930 msgr2=0x7f97580b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97580b2930 0x7f97580b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 -- 172.21.15.3:0/26304794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750009420 con 0x7f97581598d0 2023-10-06T04:59:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.072+0000 7f97577fe700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97581598d0 0x7f975815a940 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f974800cbd0 tx=0x7f9748007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.073+0000 7f974effd700 1 -- 172.21.15.3:0/26304794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748004dc0 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.073+0000 7f974effd700 1 -- 172.21.15.3:0/26304794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974800a6b0 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.073+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97580b3720 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.073+0000 7f974effd700 1 -- 172.21.15.3:0/26304794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974800aa70 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.073+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9758154a70 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.074+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9758155230 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.074+0000 7f974effd700 1 -- 172.21.15.3:0/26304794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9748007150 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.075+0000 7f974effd700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9738066f30 0x7f97380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.075+0000 7f974effd700 1 -- 172.21.15.3:0/26304794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9748084050 con 0x7f97581598d0 2023-10-06T04:59:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.075+0000 7f9756ffd700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9738066f30 0x7f97380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.075+0000 7f9756ffd700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9738066f30 0x7f97380693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f97500093f0 tx=0x7f9750016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.079+0000 7f974effd700 1 -- 172.21.15.3:0/26304794 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97480511e0 con 0x7f97581598d0 2023-10-06T04:59:58.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.319+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9758071680 con 0x7f97581598d0 2023-10-06T04:59:58.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.319+0000 7f974effd700 1 -- 172.21.15.3:0/26304794 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f974800ae60 con 0x7f97581598d0 2023-10-06T04:59:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:58.321 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T04:59:57.924199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T04:59:58.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.321+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9738066f30 msgr2=0x7f97380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.321+0000 7f975d93c700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9738066f30 0x7f97380693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f97500093f0 tx=0x7f9750016000 comp rx=0 tx=0).stop 2023-10-06T04:59:58.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.321+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97581598d0 msgr2=0x7f975815a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.321+0000 7f975d93c700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97581598d0 0x7f975815a940 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f974800cbd0 tx=0x7f9748007d20 comp rx=0 tx=0).stop 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 shutdown_connections 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9738066f30 0x7f97380693e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97580b2930 0x7f97580b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97581598d0 0x7f975815a940 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 --2- 172.21.15.3:0/26304794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9758159500 0x7f975815a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 >> 172.21.15.3:0/26304794 conn(0x7f9758070310 msgr2=0x7f9758157520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 shutdown_connections 2023-10-06T04:59:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.322+0000 7f975d93c700 1 -- 172.21.15.3:0/26304794 wait complete. 2023-10-06T04:59:58.325 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 69 2023-10-06T04:59:58.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:59:58.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.453+0000 7f0c7fb50700 1 Processor -- start 2023-10-06T04:59:58.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.453+0000 7f0c7fb50700 1 -- start start 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7fb50700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c7815c8f0 0x7f0c7815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7fb50700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0c7815ccc0 0x7f0c780d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7fb50700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c780cf6c0 con 0x7f0c780d06a0 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7fb50700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780cfb50 con 0x7f0c7815c8f0 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7fb50700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780d0b20 con 0x7f0c7815ccc0 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7d8ec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c7815c8f0 0x7f0c7815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7e0ed700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0c780d06a0 0x7f0c7806d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50012/0 (socket says 172.21.15.3:50012) 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7e0ed700 1 -- 172.21.15.3:0/2783765423 learned_addr learned my addr 172.21.15.3:0/2783765423 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7d0eb700 1 --2- 172.21.15.3:0/2783765423 >> v2:172.21.15.153:3300/0 conn(0x7f0c7815ccc0 0x7f0c780d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7d8ec700 1 -- 172.21.15.3:0/2783765423 >> v2:172.21.15.153:3300/0 conn(0x7f0c7815ccc0 msgr2=0x7f0c780d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7d8ec700 1 --2- 172.21.15.3:0/2783765423 >> v2:172.21.15.153:3300/0 conn(0x7f0c7815ccc0 0x7f0c780d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7d8ec700 1 -- 172.21.15.3:0/2783765423 >> v1:172.21.15.3:6789/0 conn(0x7f0c780d06a0 legacy=0x7f0c7806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7d8ec700 1 -- 172.21.15.3:0/2783765423 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7815e050 con 0x7f0c7815c8f0 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7d8ec700 1 --2- 172.21.15.3:0/2783765423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c7815db10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0c700098e0 tx=0x7f0c700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbec7b039587e77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7c8ea700 1 -- 172.21.15.3:0/2783765423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70009e80 con 0x7f0c7815c8f0 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7c8ea700 1 -- 172.21.15.3:0/2783765423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7000b820 con 0x7f0c7815c8f0 2023-10-06T04:59:58.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.454+0000 7f0c7c8ea700 1 -- 172.21.15.3:0/2783765423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7000bba0 con 0x7f0c7815c8f0 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2783765423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 msgr2=0x7f0c7815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2783765423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c7815db10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0c700098e0 tx=0x7f0c700091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2783765423 shutdown_connections 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2783765423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c7815db10 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2783765423 >> v2:172.21.15.153:3300/0 conn(0x7f0c7815ccc0 0x7f0c780d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2783765423 >> 172.21.15.3:0/2783765423 conn(0x7f0c78070310 msgr2=0x7f0c78070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2783765423 shutdown_connections 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.455+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2783765423 wait complete. 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.456+0000 7f0c7fb50700 1 Processor -- start 2023-10-06T04:59:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.456+0000 7f0c7fb50700 1 -- start start 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7fb50700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c780d06a0 0x7f0c7814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7fb50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c78153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7fb50700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c7815ccc0 0x7f0c78151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7fb50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c78152510 con 0x7f0c7815c8f0 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7fb50700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c781537e0 con 0x7f0c780d06a0 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7fb50700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c78152eb0 con 0x7f0c7815ccc0 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c78153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c78153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47958/0 (socket says 172.21.15.3:47958) 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 -- 172.21.15.3:0/2302254317 learned_addr learned my addr 172.21.15.3:0/2302254317 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7e0ed700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c7815ccc0 0x7f0c78151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d8ec700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c780d06a0 0x7f0c7814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 -- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c7815ccc0 msgr2=0x7f0c78151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c7815ccc0 0x7f0c78151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 -- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c780d06a0 msgr2=0x7f0c7814ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c780d06a0 0x7f0c7814ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 -- 172.21.15.3:0/2302254317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70009420 con 0x7f0c7815c8f0 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.457+0000 7f0c7d0eb700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c78153c70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c850 tx=0x7f0c6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.458+0000 7f0c75ffb700 1 -- 172.21.15.3:0/2302254317 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c011600 con 0x7f0c7815c8f0 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.458+0000 7f0c75ffb700 1 -- 172.21.15.3:0/2302254317 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c011780 con 0x7f0c7815c8f0 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.458+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c78153190 con 0x7f0c7815c8f0 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.458+0000 7f0c75ffb700 1 -- 172.21.15.3:0/2302254317 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c011b40 con 0x7f0c7815c8f0 2023-10-06T04:59:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.458+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c78152010 con 0x7f0c7815c8f0 2023-10-06T04:59:58.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.459+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c781541b0 con 0x7f0c7815c8f0 2023-10-06T04:59:58.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.459+0000 7f0c75ffb700 1 -- 172.21.15.3:0/2302254317 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c6c0193f0 con 0x7f0c7815c8f0 2023-10-06T04:59:58.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.460+0000 7f0c75ffb700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c64066f30 0x7f0c640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.460+0000 7f0c75ffb700 1 -- 172.21.15.3:0/2302254317 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c6c084a70 con 0x7f0c7815c8f0 2023-10-06T04:59:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.460+0000 7f0c7d8ec700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c64066f30 0x7f0c640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.460+0000 7f0c7d8ec700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c64066f30 0x7f0c640693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0c700093f0 tx=0x7f0c70017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.464+0000 7f0c75ffb700 1 -- 172.21.15.3:0/2302254317 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c6c051b80 con 0x7f0c7815c8f0 2023-10-06T04:59:58.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.659+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c78153950 con 0x7f0c7815c8f0 2023-10-06T04:59:58.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.660+0000 7f0c75ffb700 1 -- 172.21.15.3:0/2302254317 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16832 (secure 0 0 0) 0x7f0c6c06be60 con 0x7f0c7815c8f0 2023-10-06T04:59:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:58.662 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":108,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:55.572851+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:52.063522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:53.285974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:59:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c64066f30 msgr2=0x7f0c640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c64066f30 0x7f0c640693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0c700093f0 tx=0x7f0c70017000 comp rx=0 tx=0).stop 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 msgr2=0x7f0c78153c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c78153c70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c850 tx=0x7f0c6c008040 comp rx=0 tx=0).stop 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 shutdown_connections 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c64066f30 0x7f0c640693e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c780d06a0 0x7f0c7814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c7815ccc0 0x7f0c78151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 --2- 172.21.15.3:0/2302254317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c7815c8f0 0x7f0c78153c70 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 >> 172.21.15.3:0/2302254317 conn(0x7f0c78070310 msgr2=0x7f0c780cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 shutdown_connections 2023-10-06T04:59:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.662+0000 7f0c7fb50700 1 -- 172.21.15.3:0/2302254317 wait complete. 2023-10-06T04:59:58.676 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:59:58.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:59:58.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.793+0000 7f302d160700 1 Processor -- start 2023-10-06T04:59:58.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.793+0000 7f302d160700 1 -- start start 2023-10-06T04:59:58.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302d160700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3028157400 0x7f30280b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302d160700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f302806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302d160700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302815aeb0 con 0x7f3028157030 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302d160700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302815a7a0 con 0x7f3028157400 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302d160700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302815aa80 con 0x7f30280b3b70 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3028157400 0x7f30280b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f3026d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3028157030 0x7f3028159ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50020/0 (socket says 172.21.15.3:50020) 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f3026d9d700 1 -- 172.21.15.3:0/1067724063 learned_addr learned my addr 172.21.15.3:0/1067724063 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302659c700 1 -- 172.21.15.3:0/1067724063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 msgr2=0x7f302806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302759e700 1 --2- 172.21.15.3:0/1067724063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f302806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f3025d9b700 1 -- 172.21.15.3:0/1067724063 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650787233 0 0) 0x7f302815aeb0 con 0x7f3028157030 2023-10-06T04:59:58.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302659c700 1 --2- 172.21.15.3:0/1067724063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f302806e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302659c700 1 -- 172.21.15.3:0/1067724063 >> v1:172.21.15.3:6789/0 conn(0x7f3028157030 legacy=0x7f3028159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302659c700 1 -- 172.21.15.3:0/1067724063 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302806dba0 con 0x7f3028157400 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302759e700 1 --2- 172.21.15.3:0/1067724063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f302806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f302659c700 1 --2- 172.21.15.3:0/1067724063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f30280b3630 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f30100098e0 tx=0x7f30100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507a645ae4bcec74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.794+0000 7f3025d9b700 1 -- 172.21.15.3:0/1067724063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010009e80 con 0x7f3028157400 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f3025d9b700 1 -- 172.21.15.3:0/1067724063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301000b860 con 0x7f3028157400 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f3025d9b700 1 -- 172.21.15.3:0/1067724063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301000bbe0 con 0x7f3028157400 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 -- 172.21.15.3:0/1067724063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 msgr2=0x7f30280b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 --2- 172.21.15.3:0/1067724063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f30280b3630 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f30100098e0 tx=0x7f30100091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 -- 172.21.15.3:0/1067724063 shutdown_connections 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 --2- 172.21.15.3:0/1067724063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f302806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 --2- 172.21.15.3:0/1067724063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f30280b3630 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 -- 172.21.15.3:0/1067724063 >> 172.21.15.3:0/1067724063 conn(0x7f30280ce270 msgr2=0x7f30280ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:58.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 -- 172.21.15.3:0/1067724063 shutdown_connections 2023-10-06T04:59:58.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.795+0000 7f302d160700 1 -- 172.21.15.3:0/1067724063 wait complete. 2023-10-06T04:59:58.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.796+0000 7f302d160700 1 Processor -- start 2023-10-06T04:59:58.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.796+0000 7f302d160700 1 -- start start 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302d160700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f30280cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302d160700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3028157030 0x7f30280adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302d160700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f3028151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302d160700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30280aedd0 con 0x7f3028157400 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302d160700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30280ae170 con 0x7f3028157030 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302d160700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30280ae410 con 0x7f30280b3b70 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f3028151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f3028151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47982/0 (socket says 172.21.15.3:47982) 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 -- 172.21.15.3:0/2849380986 learned_addr learned my addr 172.21.15.3:0/2849380986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f3026d9d700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f30280cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302659c700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3028157030 0x7f30280adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 -- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 msgr2=0x7f30280cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f30280cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 -- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3028157030 msgr2=0x7f30280adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3028157030 0x7f30280adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f302759e700 1 -- 172.21.15.3:0/2849380986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010009420 con 0x7f3028157400 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.797+0000 7f3026d9d700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f30280cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.798+0000 7f302759e700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f3028151b30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f302000c880 tx=0x7f3020008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.798+0000 7f301effd700 1 -- 172.21.15.3:0/2849380986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020011600 con 0x7f3028157400 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.798+0000 7f301effd700 1 -- 172.21.15.3:0/2849380986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020011760 con 0x7f3028157400 2023-10-06T04:59:58.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.798+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30280ae6f0 con 0x7f3028157400 2023-10-06T04:59:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.798+0000 7f301effd700 1 -- 172.21.15.3:0/2849380986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020011b20 con 0x7f3028157400 2023-10-06T04:59:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.798+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30280af020 con 0x7f3028157400 2023-10-06T04:59:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.799+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30280644c0 con 0x7f3028157400 2023-10-06T04:59:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.799+0000 7f301effd700 1 -- 172.21.15.3:0/2849380986 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30200193b0 con 0x7f3028157400 2023-10-06T04:59:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.799+0000 7f301effd700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3008066f30 0x7f30080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.800+0000 7f301effd700 1 -- 172.21.15.3:0/2849380986 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3020084710 con 0x7f3028157400 2023-10-06T04:59:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.800+0000 7f3026d9d700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3008066f30 0x7f30080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.800+0000 7f3026d9d700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3008066f30 0x7f30080693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3018001560 tx=0x7f3018009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.804+0000 7f301effd700 1 -- 172.21.15.3:0/2849380986 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30200552b0 con 0x7f3028157400 2023-10-06T04:59:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:58.994+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30280af210 con 0x7f3008066f30 2023-10-06T04:59:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.004+0000 7f301effd700 1 -- 172.21.15.3:0/2849380986 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312125 (secure 0 0 0) 0x7f30280af210 con 0x7f3008066f30 2023-10-06T04:59:59.009 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3008066f30 msgr2=0x7f30080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3008066f30 0x7f30080693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3018001560 tx=0x7f3018009040 comp rx=0 tx=0).stop 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 msgr2=0x7f3028151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f3028151b30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f302000c880 tx=0x7f3020008040 comp rx=0 tx=0).stop 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 shutdown_connections 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3008066f30 0x7f30080693e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3028157400 0x7f3028151b30 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30280b3b70 0x7f30280cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 --2- 172.21.15.3:0/2849380986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3028157030 0x7f30280adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 >> 172.21.15.3:0/2849380986 conn(0x7f30280ce270 msgr2=0x7f3028069090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 shutdown_connections 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.007+0000 7f302d160700 1 -- 172.21.15.3:0/2849380986 wait complete. 2023-10-06T04:59:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:59:59.021 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-10-06T04:59:57.537052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2447248,"kb_used_data":11600,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752527472,"statfs":{"total":773094113280,"available":770588131328,"internally_reserved":0,"allocated":11878400,"data_stored":5995121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.641641"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584608+0000","last_change":"2023-10-06T04:59:54.584608+0000","last_active":"2023-10-06T04:59:54.584608+0000","last_peered":"2023-10-06T04:59:54.584608+0000","last_clean":"2023-10-06T04:59:54.584608+0000","last_became_active":"2023-10-06T04:59:54.584357+0000","last_became_peered":"2023-10-06T04:59:54.584357+0000","last_unstale":"2023-10-06T04:59:54.584608+0000","last_undegraded":"2023-10-06T04:59:54.584608+0000","last_fullsized":"2023-10-06T04:59:54.584608+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:54.440341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595020+0000","last_change":"2023-10-06T04:59:54.595020+0000","last_active":"2023-10-06T04:59:54.595020+0000","last_peered":"2023-10-06T04:59:54.595020+0000","last_clean":"2023-10-06T04:59:54.595020+0000","last_became_active":"2023-10-06T04:59:54.594703+0000","last_became_peered":"2023-10-06T04:59:54.594703+0000","last_unstale":"2023-10-06T04:59:54.595020+0000","last_undegraded":"2023-10-06T04:59:54.595020+0000","last_fullsized":"2023-10-06T04:59:54.595020+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:05.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.600180+0000","last_change":"2023-10-06T04:59:54.600180+0000","last_active":"2023-10-06T04:59:54.600180+0000","last_peered":"2023-10-06T04:59:54.600180+0000","last_clean":"2023-10-06T04:59:54.600180+0000","last_became_active":"2023-10-06T04:59:54.599965+0000","last_became_peered":"2023-10-06T04:59:54.599965+0000","last_unstale":"2023-10-06T04:59:54.600180+0000","last_undegraded":"2023-10-06T04:59:54.600180+0000","last_fullsized":"2023-10-06T04:59:54.600180+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:47.262846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592814+0000","last_change":"2023-10-06T04:59:54.592814+0000","last_active":"2023-10-06T04:59:54.592814+0000","last_peered":"2023-10-06T04:59:54.592814+0000","last_clean":"2023-10-06T04:59:54.592814+0000","last_became_active":"2023-10-06T04:59:54.592550+0000","last_became_peered":"2023-10-06T04:59:54.592550+0000","last_unstale":"2023-10-06T04:59:54.592814+0000","last_undegraded":"2023-10-06T04:59:54.592814+0000","last_fullsized":"2023-10-06T04:59:54.592814+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:16.473260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589887+0000","last_change":"2023-10-06T04:59:54.589887+0000","last_active":"2023-10-06T04:59:54.589887+0000","last_peered":"2023-10-06T04:59:54.589887+0000","last_clean":"2023-10-06T04:59:54.589887+0000","last_became_active":"2023-10-06T04:59:54.589616+0000","last_became_peered":"2023-10-06T04:59:54.589616+0000","last_unstale":"2023-10-06T04:59:54.589887+0000","last_undegraded":"2023-10-06T04:59:54.589887+0000","last_fullsized":"2023-10-06T04:59:54.589887+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:47.017432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591514+0000","last_change":"2023-10-06T04:59:54.591514+0000","last_active":"2023-10-06T04:59:54.591514+0000","last_peered":"2023-10-06T04:59:54.591514+0000","last_clean":"2023-10-06T04:59:54.591514+0000","last_became_active":"2023-10-06T04:59:54.591266+0000","last_became_peered":"2023-10-06T04:59:54.591266+0000","last_unstale":"2023-10-06T04:59:54.591514+0000","last_undegraded":"2023-10-06T04:59:54.591514+0000","last_fullsized":"2023-10-06T04:59:54.591514+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:13.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595349+0000","last_change":"2023-10-06T04:59:54.595349+0000","last_active":"2023-10-06T04:59:54.595349+0000","last_peered":"2023-10-06T04:59:54.595349+0000","last_clean":"2023-10-06T04:59:54.595349+0000","last_became_active":"2023-10-06T04:59:54.595078+0000","last_became_peered":"2023-10-06T04:59:54.595078+0000","last_unstale":"2023-10-06T04:59:54.595349+0000","last_undegraded":"2023-10-06T04:59:54.595349+0000","last_fullsized":"2023-10-06T04:59:54.595349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:31.006317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594510+0000","last_change":"2023-10-06T04:59:54.594510+0000","last_active":"2023-10-06T04:59:54.594510+0000","last_peered":"2023-10-06T04:59:54.594510+0000","last_clean":"2023-10-06T04:59:54.594510+0000","last_became_active":"2023-10-06T04:59:54.593986+0000","last_became_peered":"2023-10-06T04:59:54.593986+0000","last_unstale":"2023-10-06T04:59:54.594510+0000","last_undegraded":"2023-10-06T04:59:54.594510+0000","last_fullsized":"2023-10-06T04:59:54.594510+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:12.261663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593027+0000","last_change":"2023-10-06T04:59:54.593027+0000","last_active":"2023-10-06T04:59:54.593027+0000","last_peered":"2023-10-06T04:59:54.593027+0000","last_clean":"2023-10-06T04:59:54.593027+0000","last_became_active":"2023-10-06T04:59:54.592801+0000","last_became_peered":"2023-10-06T04:59:54.592801+0000","last_unstale":"2023-10-06T04:59:54.593027+0000","last_undegraded":"2023-10-06T04:59:54.593027+0000","last_fullsized":"2023-10-06T04:59:54.593027+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:19.500134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.580589+0000","last_change":"2023-10-06T04:59:54.580589+0000","last_active":"2023-10-06T04:59:54.580589+0000","last_peered":"2023-10-06T04:59:54.580589+0000","last_clean":"2023-10-06T04:59:54.580589+0000","last_became_active":"2023-10-06T04:59:54.580262+0000","last_became_peered":"2023-10-06T04:59:54.580262+0000","last_unstale":"2023-10-06T04:59:54.580589+0000","last_undegraded":"2023-10-06T04:59:54.580589+0000","last_fullsized":"2023-10-06T04:59:54.580589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:54.440341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596118+0000","last_change":"2023-10-06T04:59:54.596118+0000","last_active":"2023-10-06T04:59:54.596118+0000","last_peered":"2023-10-06T04:59:54.596118+0000","last_clean":"2023-10-06T04:59:54.596118+0000","last_became_active":"2023-10-06T04:59:54.595339+0000","last_became_peered":"2023-10-06T04:59:54.595339+0000","last_unstale":"2023-10-06T04:59:54.596118+0000","last_undegraded":"2023-10-06T04:59:54.596118+0000","last_fullsized":"2023-10-06T04:59:54.596118+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.343032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.586247+0000","last_change":"2023-10-06T04:59:54.586247+0000","last_active":"2023-10-06T04:59:54.586247+0000","last_peered":"2023-10-06T04:59:54.586247+0000","last_clean":"2023-10-06T04:59:54.586247+0000","last_became_active":"2023-10-06T04:59:54.585235+0000","last_became_peered":"2023-10-06T04:59:54.585235+0000","last_unstale":"2023-10-06T04:59:54.586247+0000","last_undegraded":"2023-10-06T04:59:54.586247+0000","last_fullsized":"2023-10-06T04:59:54.586247+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:06.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591643+0000","last_change":"2023-10-06T04:59:54.591643+0000","last_active":"2023-10-06T04:59:54.591643+0000","last_peered":"2023-10-06T04:59:54.591643+0000","last_clean":"2023-10-06T04:59:54.591643+0000","last_became_active":"2023-10-06T04:59:54.591384+0000","last_became_peered":"2023-10-06T04:59:54.591384+0000","last_unstale":"2023-10-06T04:59:54.591643+0000","last_undegraded":"2023-10-06T04:59:54.591643+0000","last_fullsized":"2023-10-06T04:59:54.591643+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:35.758881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595274+0000","last_change":"2023-10-06T04:59:54.595274+0000","last_active":"2023-10-06T04:59:54.595274+0000","last_peered":"2023-10-06T04:59:54.595274+0000","last_clean":"2023-10-06T04:59:54.595274+0000","last_became_active":"2023-10-06T04:59:54.594020+0000","last_became_peered":"2023-10-06T04:59:54.594020+0000","last_unstale":"2023-10-06T04:59:54.595274+0000","last_undegraded":"2023-10-06T04:59:54.595274+0000","last_fullsized":"2023-10-06T04:59:54.595274+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591584+0000","last_change":"2023-10-06T04:59:54.591584+0000","last_active":"2023-10-06T04:59:54.591584+0000","last_peered":"2023-10-06T04:59:54.591584+0000","last_clean":"2023-10-06T04:59:54.591584+0000","last_became_active":"2023-10-06T04:59:54.591304+0000","last_became_peered":"2023-10-06T04:59:54.591304+0000","last_unstale":"2023-10-06T04:59:54.591584+0000","last_undegraded":"2023-10-06T04:59:54.591584+0000","last_fullsized":"2023-10-06T04:59:54.591584+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:42.017800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.586426+0000","last_change":"2023-10-06T04:59:54.586426+0000","last_active":"2023-10-06T04:59:54.586426+0000","last_peered":"2023-10-06T04:59:54.586426+0000","last_clean":"2023-10-06T04:59:54.586426+0000","last_became_active":"2023-10-06T04:59:54.585782+0000","last_became_peered":"2023-10-06T04:59:54.585782+0000","last_unstale":"2023-10-06T04:59:54.586426+0000","last_undegraded":"2023-10-06T04:59:54.586426+0000","last_fullsized":"2023-10-06T04:59:54.586426+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:09.639856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593880+0000","last_change":"2023-10-06T04:59:54.593880+0000","last_active":"2023-10-06T04:59:54.593880+0000","last_peered":"2023-10-06T04:59:54.593880+0000","last_clean":"2023-10-06T04:59:54.593880+0000","last_became_active":"2023-10-06T04:59:54.593600+0000","last_became_peered":"2023-10-06T04:59:54.593600+0000","last_unstale":"2023-10-06T04:59:54.593880+0000","last_undegraded":"2023-10-06T04:59:54.593880+0000","last_fullsized":"2023-10-06T04:59:54.593880+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:22.397690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592678+0000","last_change":"2023-10-06T04:59:54.592678+0000","last_active":"2023-10-06T04:59:54.592678+0000","last_peered":"2023-10-06T04:59:54.592678+0000","last_clean":"2023-10-06T04:59:54.592678+0000","last_became_active":"2023-10-06T04:59:54.592280+0000","last_became_peered":"2023-10-06T04:59:54.592280+0000","last_unstale":"2023-10-06T04:59:54.592678+0000","last_undegraded":"2023-10-06T04:59:54.592678+0000","last_fullsized":"2023-10-06T04:59:54.592678+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:47.710437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596701+0000","last_change":"2023-10-06T04:59:54.596701+0000","last_active":"2023-10-06T04:59:54.596701+0000","last_peered":"2023-10-06T04:59:54.596701+0000","last_clean":"2023-10-06T04:59:54.596701+0000","last_became_active":"2023-10-06T04:59:54.596330+0000","last_became_peered":"2023-10-06T04:59:54.596330+0000","last_unstale":"2023-10-06T04:59:54.596701+0000","last_undegraded":"2023-10-06T04:59:54.596701+0000","last_fullsized":"2023-10-06T04:59:54.596701+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:28.571600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584320+0000","last_change":"2023-10-06T04:59:54.584320+0000","last_active":"2023-10-06T04:59:54.584320+0000","last_peered":"2023-10-06T04:59:54.584320+0000","last_clean":"2023-10-06T04:59:54.584320+0000","last_became_active":"2023-10-06T04:59:54.584045+0000","last_became_peered":"2023-10-06T04:59:54.584045+0000","last_unstale":"2023-10-06T04:59:54.584320+0000","last_undegraded":"2023-10-06T04:59:54.584320+0000","last_fullsized":"2023-10-06T04:59:54.584320+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:17.752152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594295+0000","last_change":"2023-10-06T04:59:54.594295+0000","last_active":"2023-10-06T04:59:54.594295+0000","last_peered":"2023-10-06T04:59:54.594295+0000","last_clean":"2023-10-06T04:59:54.594295+0000","last_became_active":"2023-10-06T04:59:54.594020+0000","last_became_peered":"2023-10-06T04:59:54.594020+0000","last_unstale":"2023-10-06T04:59:54.594295+0000","last_undegraded":"2023-10-06T04:59:54.594295+0000","last_fullsized":"2023-10-06T04:59:54.594295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:56.363467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592083+0000","last_change":"2023-10-06T04:59:54.592083+0000","last_active":"2023-10-06T04:59:54.592083+0000","last_peered":"2023-10-06T04:59:54.592083+0000","last_clean":"2023-10-06T04:59:54.592083+0000","last_became_active":"2023-10-06T04:59:54.591804+0000","last_became_peered":"2023-10-06T04:59:54.591804+0000","last_unstale":"2023-10-06T04:59:54.592083+0000","last_undegraded":"2023-10-06T04:59:54.592083+0000","last_fullsized":"2023-10-06T04:59:54.592083+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:12.340942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593962+0000","last_change":"2023-10-06T04:59:54.593962+0000","last_active":"2023-10-06T04:59:54.593962+0000","last_peered":"2023-10-06T04:59:54.593962+0000","last_clean":"2023-10-06T04:59:54.593962+0000","last_became_active":"2023-10-06T04:59:54.593661+0000","last_became_peered":"2023-10-06T04:59:54.593661+0000","last_unstale":"2023-10-06T04:59:54.593962+0000","last_undegraded":"2023-10-06T04:59:54.593962+0000","last_fullsized":"2023-10-06T04:59:54.593962+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:07.744381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594314+0000","last_change":"2023-10-06T04:59:54.594314+0000","last_active":"2023-10-06T04:59:54.594314+0000","last_peered":"2023-10-06T04:59:54.594314+0000","last_clean":"2023-10-06T04:59:54.594314+0000","last_became_active":"2023-10-06T04:59:54.594044+0000","last_became_peered":"2023-10-06T04:59:54.594044+0000","last_unstale":"2023-10-06T04:59:54.594314+0000","last_undegraded":"2023-10-06T04:59:54.594314+0000","last_fullsized":"2023-10-06T04:59:54.594314+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:55.261863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596484+0000","last_change":"2023-10-06T04:59:54.596484+0000","last_active":"2023-10-06T04:59:54.596484+0000","last_peered":"2023-10-06T04:59:54.596484+0000","last_clean":"2023-10-06T04:59:54.596484+0000","last_became_active":"2023-10-06T04:59:54.595647+0000","last_became_peered":"2023-10-06T04:59:54.595647+0000","last_unstale":"2023-10-06T04:59:54.596484+0000","last_undegraded":"2023-10-06T04:59:54.596484+0000","last_fullsized":"2023-10-06T04:59:54.596484+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:41.823804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.582583+0000","last_change":"2023-10-06T04:59:54.582583+0000","last_active":"2023-10-06T04:59:54.582583+0000","last_peered":"2023-10-06T04:59:54.582583+0000","last_clean":"2023-10-06T04:59:54.582583+0000","last_became_active":"2023-10-06T04:59:54.582362+0000","last_became_peered":"2023-10-06T04:59:54.582362+0000","last_unstale":"2023-10-06T04:59:54.582583+0000","last_undegraded":"2023-10-06T04:59:54.582583+0000","last_fullsized":"2023-10-06T04:59:54.582583+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:51.997295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.597696+0000","last_change":"2023-10-06T04:59:54.597696+0000","last_active":"2023-10-06T04:59:54.597696+0000","last_peered":"2023-10-06T04:59:54.597696+0000","last_clean":"2023-10-06T04:59:54.597696+0000","last_became_active":"2023-10-06T04:59:54.596957+0000","last_became_peered":"2023-10-06T04:59:54.596957+0000","last_unstale":"2023-10-06T04:59:54.597696+0000","last_undegraded":"2023-10-06T04:59:54.597696+0000","last_fullsized":"2023-10-06T04:59:54.597696+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:54.220363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592465+0000","last_change":"2023-10-06T04:59:54.592465+0000","last_active":"2023-10-06T04:59:54.592465+0000","last_peered":"2023-10-06T04:59:54.592465+0000","last_clean":"2023-10-06T04:59:54.592465+0000","last_became_active":"2023-10-06T04:59:54.592173+0000","last_became_peered":"2023-10-06T04:59:54.592173+0000","last_unstale":"2023-10-06T04:59:54.592465+0000","last_undegraded":"2023-10-06T04:59:54.592465+0000","last_fullsized":"2023-10-06T04:59:54.592465+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:30.969822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590575+0000","last_change":"2023-10-06T04:59:54.590575+0000","last_active":"2023-10-06T04:59:54.590575+0000","last_peered":"2023-10-06T04:59:54.590575+0000","last_clean":"2023-10-06T04:59:54.590575+0000","last_became_active":"2023-10-06T04:59:54.590317+0000","last_became_peered":"2023-10-06T04:59:54.590317+0000","last_unstale":"2023-10-06T04:59:54.590575+0000","last_undegraded":"2023-10-06T04:59:54.590575+0000","last_fullsized":"2023-10-06T04:59:54.590575+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:45.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590402+0000","last_change":"2023-10-06T04:59:54.590402+0000","last_active":"2023-10-06T04:59:54.590402+0000","last_peered":"2023-10-06T04:59:54.590402+0000","last_clean":"2023-10-06T04:59:54.590402+0000","last_became_active":"2023-10-06T04:59:54.589995+0000","last_became_peered":"2023-10-06T04:59:54.589995+0000","last_unstale":"2023-10-06T04:59:54.590402+0000","last_undegraded":"2023-10-06T04:59:54.590402+0000","last_fullsized":"2023-10-06T04:59:54.590402+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:39.522866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591166+0000","last_change":"2023-10-06T04:59:54.591166+0000","last_active":"2023-10-06T04:59:54.591166+0000","last_peered":"2023-10-06T04:59:54.591166+0000","last_clean":"2023-10-06T04:59:54.591166+0000","last_became_active":"2023-10-06T04:59:54.590833+0000","last_became_peered":"2023-10-06T04:59:54.590833+0000","last_unstale":"2023-10-06T04:59:54.591166+0000","last_undegraded":"2023-10-06T04:59:54.591166+0000","last_fullsized":"2023-10-06T04:59:54.591166+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:35.567647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594295+0000","last_change":"2023-10-06T04:59:54.594295+0000","last_active":"2023-10-06T04:59:54.594295+0000","last_peered":"2023-10-06T04:59:54.594295+0000","last_clean":"2023-10-06T04:59:54.594295+0000","last_became_active":"2023-10-06T04:59:54.594000+0000","last_became_peered":"2023-10-06T04:59:54.594000+0000","last_unstale":"2023-10-06T04:59:54.594295+0000","last_undegraded":"2023-10-06T04:59:54.594295+0000","last_fullsized":"2023-10-06T04:59:54.594295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:22.397690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584446+0000","last_change":"2023-10-06T04:59:54.584446+0000","last_active":"2023-10-06T04:59:54.584446+0000","last_peered":"2023-10-06T04:59:54.584446+0000","last_clean":"2023-10-06T04:59:54.584446+0000","last_became_active":"2023-10-06T04:59:54.584141+0000","last_became_peered":"2023-10-06T04:59:54.584141+0000","last_unstale":"2023-10-06T04:59:54.584446+0000","last_undegraded":"2023-10-06T04:59:54.584446+0000","last_fullsized":"2023-10-06T04:59:54.584446+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:26.633928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"107'4","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.916911+0000","last_change":"2023-10-06T04:59:53.919668+0000","last_active":"2023-10-06T04:59:54.916911+0000","last_peered":"2023-10-06T04:59:54.916911+0000","last_clean":"2023-10-06T04:59:54.916911+0000","last_became_active":"2023-10-06T04:59:53.919254+0000","last_became_peered":"2023-10-06T04:59:53.919254+0000","last_unstale":"2023-10-06T04:59:54.916911+0000","last_undegraded":"2023-10-06T04:59:54.916911+0000","last_fullsized":"2023-10-06T04:59:54.916911+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:06.327449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590344+0000","last_change":"2023-10-06T04:59:54.590344+0000","last_active":"2023-10-06T04:59:54.590344+0000","last_peered":"2023-10-06T04:59:54.590344+0000","last_clean":"2023-10-06T04:59:54.590344+0000","last_became_active":"2023-10-06T04:59:54.590055+0000","last_became_peered":"2023-10-06T04:59:54.590055+0000","last_unstale":"2023-10-06T04:59:54.590344+0000","last_undegraded":"2023-10-06T04:59:54.590344+0000","last_fullsized":"2023-10-06T04:59:54.590344+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:06.670106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.575356+0000","last_change":"2023-10-06T04:59:53.912231+0000","last_active":"2023-10-06T04:59:54.575356+0000","last_peered":"2023-10-06T04:59:54.575356+0000","last_clean":"2023-10-06T04:59:54.575356+0000","last_became_active":"2023-10-06T04:59:53.912073+0000","last_became_peered":"2023-10-06T04:59:53.912073+0000","last_unstale":"2023-10-06T04:59:54.575356+0000","last_undegraded":"2023-10-06T04:59:54.575356+0000","last_fullsized":"2023-10-06T04:59:54.575356+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:46:00.861292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591018+0000","last_change":"2023-10-06T04:59:54.591018+0000","last_active":"2023-10-06T04:59:54.591018+0000","last_peered":"2023-10-06T04:59:54.591018+0000","last_clean":"2023-10-06T04:59:54.591018+0000","last_became_active":"2023-10-06T04:59:54.590710+0000","last_became_peered":"2023-10-06T04:59:54.590710+0000","last_unstale":"2023-10-06T04:59:54.591018+0000","last_undegraded":"2023-10-06T04:59:54.591018+0000","last_fullsized":"2023-10-06T04:59:54.591018+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:58.864838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.914099+0000","last_change":"2023-10-06T04:59:53.920720+0000","last_active":"2023-10-06T04:59:54.914099+0000","last_peered":"2023-10-06T04:59:54.914099+0000","last_clean":"2023-10-06T04:59:54.914099+0000","last_became_active":"2023-10-06T04:59:53.920411+0000","last_became_peered":"2023-10-06T04:59:53.920411+0000","last_unstale":"2023-10-06T04:59:54.914099+0000","last_undegraded":"2023-10-06T04:59:54.914099+0000","last_fullsized":"2023-10-06T04:59:54.914099+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:41.012060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594908+0000","last_change":"2023-10-06T04:59:54.594908+0000","last_active":"2023-10-06T04:59:54.594908+0000","last_peered":"2023-10-06T04:59:54.594908+0000","last_clean":"2023-10-06T04:59:54.594908+0000","last_became_active":"2023-10-06T04:59:54.594711+0000","last_became_peered":"2023-10-06T04:59:54.594711+0000","last_unstale":"2023-10-06T04:59:54.594908+0000","last_undegraded":"2023-10-06T04:59:54.594908+0000","last_fullsized":"2023-10-06T04:59:54.594908+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:28.347251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.914439+0000","last_change":"2023-10-06T04:59:53.911212+0000","last_active":"2023-10-06T04:59:54.914439+0000","last_peered":"2023-10-06T04:59:54.914439+0000","last_clean":"2023-10-06T04:59:54.914439+0000","last_became_active":"2023-10-06T04:59:53.910952+0000","last_became_peered":"2023-10-06T04:59:53.910952+0000","last_unstale":"2023-10-06T04:59:54.914439+0000","last_undegraded":"2023-10-06T04:59:54.914439+0000","last_fullsized":"2023-10-06T04:59:54.914439+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:52.293461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589691+0000","last_change":"2023-10-06T04:59:54.589691+0000","last_active":"2023-10-06T04:59:54.589691+0000","last_peered":"2023-10-06T04:59:54.589691+0000","last_clean":"2023-10-06T04:59:54.589691+0000","last_became_active":"2023-10-06T04:59:54.589389+0000","last_became_peered":"2023-10-06T04:59:54.589389+0000","last_unstale":"2023-10-06T04:59:54.589691+0000","last_undegraded":"2023-10-06T04:59:54.589691+0000","last_fullsized":"2023-10-06T04:59:54.589691+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:55.190779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.579055+0000","last_change":"2023-10-06T04:59:53.912202+0000","last_active":"2023-10-06T04:59:54.579055+0000","last_peered":"2023-10-06T04:59:54.579055+0000","last_clean":"2023-10-06T04:59:54.579055+0000","last_became_active":"2023-10-06T04:59:53.911897+0000","last_became_peered":"2023-10-06T04:59:53.911897+0000","last_unstale":"2023-10-06T04:59:54.579055+0000","last_undegraded":"2023-10-06T04:59:54.579055+0000","last_fullsized":"2023-10-06T04:59:54.579055+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:40.818064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595739+0000","last_change":"2023-10-06T04:59:54.595739+0000","last_active":"2023-10-06T04:59:54.595739+0000","last_peered":"2023-10-06T04:59:54.595739+0000","last_clean":"2023-10-06T04:59:54.595739+0000","last_became_active":"2023-10-06T04:59:54.595372+0000","last_became_peered":"2023-10-06T04:59:54.595372+0000","last_unstale":"2023-10-06T04:59:54.595739+0000","last_undegraded":"2023-10-06T04:59:54.595739+0000","last_fullsized":"2023-10-06T04:59:54.595739+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:40.788038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.579054+0000","last_change":"2023-10-06T04:59:53.918089+0000","last_active":"2023-10-06T04:59:54.579054+0000","last_peered":"2023-10-06T04:59:54.579054+0000","last_clean":"2023-10-06T04:59:54.579054+0000","last_became_active":"2023-10-06T04:59:53.917829+0000","last_became_peered":"2023-10-06T04:59:53.917829+0000","last_unstale":"2023-10-06T04:59:54.579054+0000","last_undegraded":"2023-10-06T04:59:54.579054+0000","last_fullsized":"2023-10-06T04:59:54.579054+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:38.517125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590230+0000","last_change":"2023-10-06T04:59:54.590230+0000","last_active":"2023-10-06T04:59:54.590230+0000","last_peered":"2023-10-06T04:59:54.590230+0000","last_clean":"2023-10-06T04:59:54.590230+0000","last_became_active":"2023-10-06T04:59:54.589685+0000","last_became_peered":"2023-10-06T04:59:54.589685+0000","last_unstale":"2023-10-06T04:59:54.590230+0000","last_undegraded":"2023-10-06T04:59:54.590230+0000","last_fullsized":"2023-10-06T04:59:54.590230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:17.642637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.915490+0000","last_change":"2023-10-06T04:59:53.914581+0000","last_active":"2023-10-06T04:59:53.915490+0000","last_peered":"2023-10-06T04:59:53.915490+0000","last_clean":"2023-10-06T04:59:53.915490+0000","last_became_active":"2023-10-06T04:59:53.914289+0000","last_became_peered":"2023-10-06T04:59:53.914289+0000","last_unstale":"2023-10-06T04:59:53.915490+0000","last_undegraded":"2023-10-06T04:59:53.915490+0000","last_fullsized":"2023-10-06T04:59:53.915490+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:05.644438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589831+0000","last_change":"2023-10-06T04:59:54.589831+0000","last_active":"2023-10-06T04:59:54.589831+0000","last_peered":"2023-10-06T04:59:54.589831+0000","last_clean":"2023-10-06T04:59:54.589831+0000","last_became_active":"2023-10-06T04:59:54.589567+0000","last_became_peered":"2023-10-06T04:59:54.589567+0000","last_unstale":"2023-10-06T04:59:54.589831+0000","last_undegraded":"2023-10-06T04:59:54.589831+0000","last_fullsized":"2023-10-06T04:59:54.589831+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:22.237368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916540+0000","last_change":"2023-10-06T04:59:53.913902+0000","last_active":"2023-10-06T04:59:53.916540+0000","last_peered":"2023-10-06T04:59:53.916540+0000","last_clean":"2023-10-06T04:59:53.916540+0000","last_became_active":"2023-10-06T04:59:53.913365+0000","last_became_peered":"2023-10-06T04:59:53.913365+0000","last_unstale":"2023-10-06T04:59:53.916540+0000","last_undegraded":"2023-10-06T04:59:53.916540+0000","last_fullsized":"2023-10-06T04:59:53.916540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:41.287419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591869+0000","last_change":"2023-10-06T04:59:54.591869+0000","last_active":"2023-10-06T04:59:54.591869+0000","last_peered":"2023-10-06T04:59:54.591869+0000","last_clean":"2023-10-06T04:59:54.591869+0000","last_became_active":"2023-10-06T04:59:54.591501+0000","last_became_peered":"2023-10-06T04:59:54.591501+0000","last_unstale":"2023-10-06T04:59:54.591869+0000","last_undegraded":"2023-10-06T04:59:54.591869+0000","last_fullsized":"2023-10-06T04:59:54.591869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:26.633928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.575162+0000","last_change":"2023-10-06T04:59:53.916172+0000","last_active":"2023-10-06T04:59:54.575162+0000","last_peered":"2023-10-06T04:59:54.575162+0000","last_clean":"2023-10-06T04:59:54.575162+0000","last_became_active":"2023-10-06T04:59:53.915905+0000","last_became_peered":"2023-10-06T04:59:53.915905+0000","last_unstale":"2023-10-06T04:59:54.575162+0000","last_undegraded":"2023-10-06T04:59:54.575162+0000","last_fullsized":"2023-10-06T04:59:54.575162+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:59.997399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590453+0000","last_change":"2023-10-06T04:59:54.590453+0000","last_active":"2023-10-06T04:59:54.590453+0000","last_peered":"2023-10-06T04:59:54.590453+0000","last_clean":"2023-10-06T04:59:54.590453+0000","last_became_active":"2023-10-06T04:59:54.590158+0000","last_became_peered":"2023-10-06T04:59:54.590158+0000","last_unstale":"2023-10-06T04:59:54.590453+0000","last_undegraded":"2023-10-06T04:59:54.590453+0000","last_fullsized":"2023-10-06T04:59:54.590453+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:39.975563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.914147+0000","last_change":"2023-10-06T04:59:53.917104+0000","last_active":"2023-10-06T04:59:54.914147+0000","last_peered":"2023-10-06T04:59:54.914147+0000","last_clean":"2023-10-06T04:59:54.914147+0000","last_became_active":"2023-10-06T04:59:53.916838+0000","last_became_peered":"2023-10-06T04:59:53.916838+0000","last_unstale":"2023-10-06T04:59:54.914147+0000","last_undegraded":"2023-10-06T04:59:54.914147+0000","last_fullsized":"2023-10-06T04:59:54.914147+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:51.480986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593706+0000","last_change":"2023-10-06T04:59:54.593706+0000","last_active":"2023-10-06T04:59:54.593706+0000","last_peered":"2023-10-06T04:59:54.593706+0000","last_clean":"2023-10-06T04:59:54.593706+0000","last_became_active":"2023-10-06T04:59:54.593256+0000","last_became_peered":"2023-10-06T04:59:54.593256+0000","last_unstale":"2023-10-06T04:59:54.593706+0000","last_undegraded":"2023-10-06T04:59:54.593706+0000","last_fullsized":"2023-10-06T04:59:54.593706+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:05.401334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584725+0000","last_change":"2023-10-06T04:59:53.927934+0000","last_active":"2023-10-06T04:59:54.584725+0000","last_peered":"2023-10-06T04:59:54.584725+0000","last_clean":"2023-10-06T04:59:54.584725+0000","last_became_active":"2023-10-06T04:59:53.927607+0000","last_became_peered":"2023-10-06T04:59:53.927607+0000","last_unstale":"2023-10-06T04:59:54.584725+0000","last_undegraded":"2023-10-06T04:59:54.584725+0000","last_fullsized":"2023-10-06T04:59:54.584725+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:57.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.598566+0000","last_change":"2023-10-06T04:59:54.598566+0000","last_active":"2023-10-06T04:59:54.598566+0000","last_peered":"2023-10-06T04:59:54.598566+0000","last_clean":"2023-10-06T04:59:54.598566+0000","last_became_active":"2023-10-06T04:59:54.598306+0000","last_became_peered":"2023-10-06T04:59:54.598306+0000","last_unstale":"2023-10-06T04:59:54.598566+0000","last_undegraded":"2023-10-06T04:59:54.598566+0000","last_fullsized":"2023-10-06T04:59:54.598566+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:54.440341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927502+0000","last_change":"2023-10-06T04:59:53.922491+0000","last_active":"2023-10-06T04:59:53.927502+0000","last_peered":"2023-10-06T04:59:53.927502+0000","last_clean":"2023-10-06T04:59:53.927502+0000","last_became_active":"2023-10-06T04:59:53.922280+0000","last_became_peered":"2023-10-06T04:59:53.922280+0000","last_unstale":"2023-10-06T04:59:53.927502+0000","last_undegraded":"2023-10-06T04:59:53.927502+0000","last_fullsized":"2023-10-06T04:59:53.927502+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:25.628188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590396+0000","last_change":"2023-10-06T04:59:54.590396+0000","last_active":"2023-10-06T04:59:54.590396+0000","last_peered":"2023-10-06T04:59:54.590396+0000","last_clean":"2023-10-06T04:59:54.590396+0000","last_became_active":"2023-10-06T04:59:54.590152+0000","last_became_peered":"2023-10-06T04:59:54.590152+0000","last_unstale":"2023-10-06T04:59:54.590396+0000","last_undegraded":"2023-10-06T04:59:54.590396+0000","last_fullsized":"2023-10-06T04:59:54.590396+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.557538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.917012+0000","last_change":"2023-10-06T04:59:53.920186+0000","last_active":"2023-10-06T04:59:54.917012+0000","last_peered":"2023-10-06T04:59:54.917012+0000","last_clean":"2023-10-06T04:59:54.917012+0000","last_became_active":"2023-10-06T04:59:53.919875+0000","last_became_peered":"2023-10-06T04:59:53.919875+0000","last_unstale":"2023-10-06T04:59:54.917012+0000","last_undegraded":"2023-10-06T04:59:54.917012+0000","last_fullsized":"2023-10-06T04:59:54.917012+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:12.945202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595872+0000","last_change":"2023-10-06T04:59:54.595872+0000","last_active":"2023-10-06T04:59:54.595872+0000","last_peered":"2023-10-06T04:59:54.595872+0000","last_clean":"2023-10-06T04:59:54.595872+0000","last_became_active":"2023-10-06T04:59:54.595314+0000","last_became_peered":"2023-10-06T04:59:54.595314+0000","last_unstale":"2023-10-06T04:59:54.595872+0000","last_undegraded":"2023-10-06T04:59:54.595872+0000","last_fullsized":"2023-10-06T04:59:54.595872+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:13.950943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927405+0000","last_change":"2023-10-06T04:59:53.920921+0000","last_active":"2023-10-06T04:59:53.927405+0000","last_peered":"2023-10-06T04:59:53.927405+0000","last_clean":"2023-10-06T04:59:53.927405+0000","last_became_active":"2023-10-06T04:59:53.920587+0000","last_became_peered":"2023-10-06T04:59:53.920587+0000","last_unstale":"2023-10-06T04:59:53.927405+0000","last_undegraded":"2023-10-06T04:59:53.927405+0000","last_fullsized":"2023-10-06T04:59:53.927405+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:53.434601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595940+0000","last_change":"2023-10-06T04:59:54.595940+0000","last_active":"2023-10-06T04:59:54.595940+0000","last_peered":"2023-10-06T04:59:54.595940+0000","last_clean":"2023-10-06T04:59:54.595940+0000","last_became_active":"2023-10-06T04:59:54.594649+0000","last_became_peered":"2023-10-06T04:59:54.594649+0000","last_unstale":"2023-10-06T04:59:54.595940+0000","last_undegraded":"2023-10-06T04:59:54.595940+0000","last_fullsized":"2023-10-06T04:59:54.595940+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:59.860364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.920198+0000","last_change":"2023-10-06T04:59:53.916874+0000","last_active":"2023-10-06T04:59:53.920198+0000","last_peered":"2023-10-06T04:59:53.920198+0000","last_clean":"2023-10-06T04:59:53.920198+0000","last_became_active":"2023-10-06T04:59:53.916418+0000","last_became_peered":"2023-10-06T04:59:53.916418+0000","last_unstale":"2023-10-06T04:59:53.920198+0000","last_undegraded":"2023-10-06T04:59:53.920198+0000","last_fullsized":"2023-10-06T04:59:53.920198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:46.011691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590884+0000","last_change":"2023-10-06T04:59:54.590884+0000","last_active":"2023-10-06T04:59:54.590884+0000","last_peered":"2023-10-06T04:59:54.590884+0000","last_clean":"2023-10-06T04:59:54.590884+0000","last_became_active":"2023-10-06T04:59:54.590526+0000","last_became_peered":"2023-10-06T04:59:54.590526+0000","last_unstale":"2023-10-06T04:59:54.590884+0000","last_undegraded":"2023-10-06T04:59:54.590884+0000","last_fullsized":"2023-10-06T04:59:54.590884+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:55.190779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927422+0000","last_change":"2023-10-06T04:59:53.920946+0000","last_active":"2023-10-06T04:59:53.927422+0000","last_peered":"2023-10-06T04:59:53.927422+0000","last_clean":"2023-10-06T04:59:53.927422+0000","last_became_active":"2023-10-06T04:59:53.920354+0000","last_became_peered":"2023-10-06T04:59:53.920354+0000","last_unstale":"2023-10-06T04:59:53.927422+0000","last_undegraded":"2023-10-06T04:59:53.927422+0000","last_fullsized":"2023-10-06T04:59:53.927422+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:11.335202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.585504+0000","last_change":"2023-10-06T04:59:54.585504+0000","last_active":"2023-10-06T04:59:54.585504+0000","last_peered":"2023-10-06T04:59:54.585504+0000","last_clean":"2023-10-06T04:59:54.585504+0000","last_became_active":"2023-10-06T04:59:54.584677+0000","last_became_peered":"2023-10-06T04:59:54.584677+0000","last_unstale":"2023-10-06T04:59:54.585504+0000","last_undegraded":"2023-10-06T04:59:54.585504+0000","last_fullsized":"2023-10-06T04:59:54.585504+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:32.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.916783+0000","last_change":"2023-10-06T04:59:53.914923+0000","last_active":"2023-10-06T04:59:54.916783+0000","last_peered":"2023-10-06T04:59:54.916783+0000","last_clean":"2023-10-06T04:59:54.916783+0000","last_became_active":"2023-10-06T04:59:53.914619+0000","last_became_peered":"2023-10-06T04:59:53.914619+0000","last_unstale":"2023-10-06T04:59:54.916783+0000","last_undegraded":"2023-10-06T04:59:54.916783+0000","last_fullsized":"2023-10-06T04:59:54.916783+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:16.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.597666+0000","last_change":"2023-10-06T04:59:54.597666+0000","last_active":"2023-10-06T04:59:54.597666+0000","last_peered":"2023-10-06T04:59:54.597666+0000","last_clean":"2023-10-06T04:59:54.597666+0000","last_became_active":"2023-10-06T04:59:54.597410+0000","last_became_peered":"2023-10-06T04:59:54.597410+0000","last_unstale":"2023-10-06T04:59:54.597666+0000","last_undegraded":"2023-10-06T04:59:54.597666+0000","last_fullsized":"2023-10-06T04:59:54.597666+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:55.261863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.919655+0000","last_change":"2023-10-06T04:59:53.918336+0000","last_active":"2023-10-06T04:59:53.919655+0000","last_peered":"2023-10-06T04:59:53.919655+0000","last_clean":"2023-10-06T04:59:53.919655+0000","last_became_active":"2023-10-06T04:59:53.918037+0000","last_became_peered":"2023-10-06T04:59:53.918037+0000","last_unstale":"2023-10-06T04:59:53.919655+0000","last_undegraded":"2023-10-06T04:59:53.919655+0000","last_fullsized":"2023-10-06T04:59:53.919655+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:33.195194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596385+0000","last_change":"2023-10-06T04:59:54.596385+0000","last_active":"2023-10-06T04:59:54.596385+0000","last_peered":"2023-10-06T04:59:54.596385+0000","last_clean":"2023-10-06T04:59:54.596385+0000","last_became_active":"2023-10-06T04:59:54.595661+0000","last_became_peered":"2023-10-06T04:59:54.595661+0000","last_unstale":"2023-10-06T04:59:54.596385+0000","last_undegraded":"2023-10-06T04:59:54.596385+0000","last_fullsized":"2023-10-06T04:59:54.596385+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:59.772459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.917540+0000","last_change":"2023-10-06T04:59:53.916228+0000","last_active":"2023-10-06T04:59:53.917540+0000","last_peered":"2023-10-06T04:59:53.917540+0000","last_clean":"2023-10-06T04:59:53.917540+0000","last_became_active":"2023-10-06T04:59:53.915923+0000","last_became_peered":"2023-10-06T04:59:53.915923+0000","last_unstale":"2023-10-06T04:59:53.917540+0000","last_undegraded":"2023-10-06T04:59:53.917540+0000","last_fullsized":"2023-10-06T04:59:53.917540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:12.321308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593203+0000","last_change":"2023-10-06T04:59:54.593203+0000","last_active":"2023-10-06T04:59:54.593203+0000","last_peered":"2023-10-06T04:59:54.593203+0000","last_clean":"2023-10-06T04:59:54.593203+0000","last_became_active":"2023-10-06T04:59:54.592905+0000","last_became_peered":"2023-10-06T04:59:54.592905+0000","last_unstale":"2023-10-06T04:59:54.593203+0000","last_undegraded":"2023-10-06T04:59:54.593203+0000","last_fullsized":"2023-10-06T04:59:54.593203+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:29.949833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.921433+0000","last_change":"2023-10-06T04:59:53.918098+0000","last_active":"2023-10-06T04:59:53.921433+0000","last_peered":"2023-10-06T04:59:53.921433+0000","last_clean":"2023-10-06T04:59:53.921433+0000","last_became_active":"2023-10-06T04:59:53.917474+0000","last_became_peered":"2023-10-06T04:59:53.917474+0000","last_unstale":"2023-10-06T04:59:53.921433+0000","last_undegraded":"2023-10-06T04:59:53.921433+0000","last_fullsized":"2023-10-06T04:59:53.921433+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:11.255923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584615+0000","last_change":"2023-10-06T04:59:54.584615+0000","last_active":"2023-10-06T04:59:54.584615+0000","last_peered":"2023-10-06T04:59:54.584615+0000","last_clean":"2023-10-06T04:59:54.584615+0000","last_became_active":"2023-10-06T04:59:54.584327+0000","last_became_peered":"2023-10-06T04:59:54.584327+0000","last_unstale":"2023-10-06T04:59:54.584615+0000","last_undegraded":"2023-10-06T04:59:54.584615+0000","last_fullsized":"2023-10-06T04:59:54.584615+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:33.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.922779+0000","last_change":"2023-10-06T04:59:53.919498+0000","last_active":"2023-10-06T04:59:53.922779+0000","last_peered":"2023-10-06T04:59:53.922779+0000","last_clean":"2023-10-06T04:59:53.922779+0000","last_became_active":"2023-10-06T04:59:53.919198+0000","last_became_peered":"2023-10-06T04:59:53.919198+0000","last_unstale":"2023-10-06T04:59:53.922779+0000","last_undegraded":"2023-10-06T04:59:53.922779+0000","last_fullsized":"2023-10-06T04:59:53.922779+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:34.753141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589522+0000","last_change":"2023-10-06T04:59:54.589522+0000","last_active":"2023-10-06T04:59:54.589522+0000","last_peered":"2023-10-06T04:59:54.589522+0000","last_clean":"2023-10-06T04:59:54.589522+0000","last_became_active":"2023-10-06T04:59:54.589251+0000","last_became_peered":"2023-10-06T04:59:54.589251+0000","last_unstale":"2023-10-06T04:59:54.589522+0000","last_undegraded":"2023-10-06T04:59:54.589522+0000","last_fullsized":"2023-10-06T04:59:54.589522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:42.293160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916879+0000","last_change":"2023-10-06T04:59:53.914977+0000","last_active":"2023-10-06T04:59:53.916879+0000","last_peered":"2023-10-06T04:59:53.916879+0000","last_clean":"2023-10-06T04:59:53.916879+0000","last_became_active":"2023-10-06T04:59:53.914697+0000","last_became_peered":"2023-10-06T04:59:53.914697+0000","last_unstale":"2023-10-06T04:59:53.916879+0000","last_undegraded":"2023-10-06T04:59:53.916879+0000","last_fullsized":"2023-10-06T04:59:53.916879+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:01.612930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.588138+0000","last_change":"2023-10-06T04:59:54.588138+0000","last_active":"2023-10-06T04:59:54.588138+0000","last_peered":"2023-10-06T04:59:54.588138+0000","last_clean":"2023-10-06T04:59:54.588138+0000","last_became_active":"2023-10-06T04:59:54.587863+0000","last_became_peered":"2023-10-06T04:59:54.587863+0000","last_unstale":"2023-10-06T04:59:54.588138+0000","last_undegraded":"2023-10-06T04:59:54.588138+0000","last_fullsized":"2023-10-06T04:59:54.588138+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:51.858326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.921853+0000","last_change":"2023-10-06T04:59:53.919329+0000","last_active":"2023-10-06T04:59:53.921853+0000","last_peered":"2023-10-06T04:59:53.921853+0000","last_clean":"2023-10-06T04:59:53.921853+0000","last_became_active":"2023-10-06T04:59:53.918715+0000","last_became_peered":"2023-10-06T04:59:53.918715+0000","last_unstale":"2023-10-06T04:59:53.921853+0000","last_undegraded":"2023-10-06T04:59:53.921853+0000","last_fullsized":"2023-10-06T04:59:53.921853+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:58.309413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594155+0000","last_change":"2023-10-06T04:59:54.594155+0000","last_active":"2023-10-06T04:59:54.594155+0000","last_peered":"2023-10-06T04:59:54.594155+0000","last_clean":"2023-10-06T04:59:54.594155+0000","last_became_active":"2023-10-06T04:59:54.593750+0000","last_became_peered":"2023-10-06T04:59:54.593750+0000","last_unstale":"2023-10-06T04:59:54.594155+0000","last_undegraded":"2023-10-06T04:59:54.594155+0000","last_fullsized":"2023-10-06T04:59:54.594155+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:58.864838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927312+0000","last_change":"2023-10-06T04:59:53.920372+0000","last_active":"2023-10-06T04:59:53.927312+0000","last_peered":"2023-10-06T04:59:53.927312+0000","last_clean":"2023-10-06T04:59:53.927312+0000","last_became_active":"2023-10-06T04:59:53.919988+0000","last_became_peered":"2023-10-06T04:59:53.919988+0000","last_unstale":"2023-10-06T04:59:53.927312+0000","last_undegraded":"2023-10-06T04:59:53.927312+0000","last_fullsized":"2023-10-06T04:59:53.927312+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:28.338164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594099+0000","last_change":"2023-10-06T04:59:54.594099+0000","last_active":"2023-10-06T04:59:54.594099+0000","last_peered":"2023-10-06T04:59:54.594099+0000","last_clean":"2023-10-06T04:59:54.594099+0000","last_became_active":"2023-10-06T04:59:54.593802+0000","last_became_peered":"2023-10-06T04:59:54.593802+0000","last_unstale":"2023-10-06T04:59:54.594099+0000","last_undegraded":"2023-10-06T04:59:54.594099+0000","last_fullsized":"2023-10-06T04:59:54.594099+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:59.315153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"107'3","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.918804+0000","last_change":"2023-10-06T04:59:53.920677+0000","last_active":"2023-10-06T04:59:54.918804+0000","last_peered":"2023-10-06T04:59:54.918804+0000","last_clean":"2023-10-06T04:59:54.918804+0000","last_became_active":"2023-10-06T04:59:53.920381+0000","last_became_peered":"2023-10-06T04:59:53.920381+0000","last_unstale":"2023-10-06T04:59:54.918804+0000","last_undegraded":"2023-10-06T04:59:54.918804+0000","last_fullsized":"2023-10-06T04:59:54.918804+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:53.701865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.909660+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:59:53.909660+0000","last_peered":"2023-10-06T04:59:53.909660+0000","last_clean":"2023-10-06T04:59:53.909660+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:59:53.909660+0000","last_undegraded":"2023-10-06T04:59:53.909660+0000","last_fullsized":"2023-10-06T04:59:53.909660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592739+0000","last_change":"2023-10-06T04:59:54.592739+0000","last_active":"2023-10-06T04:59:54.592739+0000","last_peered":"2023-10-06T04:59:54.592739+0000","last_clean":"2023-10-06T04:59:54.592739+0000","last_became_active":"2023-10-06T04:59:54.592223+0000","last_became_peered":"2023-10-06T04:59:54.592223+0000","last_unstale":"2023-10-06T04:59:54.592739+0000","last_undegraded":"2023-10-06T04:59:54.592739+0000","last_fullsized":"2023-10-06T04:59:54.592739+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:22.237368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"107'6","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.919014+0000","last_change":"2023-10-06T04:59:53.914473+0000","last_active":"2023-10-06T04:59:54.919014+0000","last_peered":"2023-10-06T04:59:54.919014+0000","last_clean":"2023-10-06T04:59:54.919014+0000","last_became_active":"2023-10-06T04:59:53.914101+0000","last_became_peered":"2023-10-06T04:59:53.914101+0000","last_unstale":"2023-10-06T04:59:54.919014+0000","last_undegraded":"2023-10-06T04:59:54.919014+0000","last_fullsized":"2023-10-06T04:59:54.919014+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:17.623242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.902685+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:59:53.902685+0000","last_peered":"2023-10-06T04:59:53.902685+0000","last_clean":"2023-10-06T04:59:53.902685+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:59:53.902685+0000","last_undegraded":"2023-10-06T04:59:53.902685+0000","last_fullsized":"2023-10-06T04:59:53.902685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593318+0000","last_change":"2023-10-06T04:59:54.593318+0000","last_active":"2023-10-06T04:59:54.593318+0000","last_peered":"2023-10-06T04:59:54.593318+0000","last_clean":"2023-10-06T04:59:54.593318+0000","last_became_active":"2023-10-06T04:59:54.592986+0000","last_became_peered":"2023-10-06T04:59:54.592986+0000","last_unstale":"2023-10-06T04:59:54.593318+0000","last_undegraded":"2023-10-06T04:59:54.593318+0000","last_fullsized":"2023-10-06T04:59:54.593318+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.526790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.915807+0000","last_change":"2023-10-06T04:59:53.916261+0000","last_active":"2023-10-06T04:59:54.915807+0000","last_peered":"2023-10-06T04:59:54.915807+0000","last_clean":"2023-10-06T04:59:54.915807+0000","last_became_active":"2023-10-06T04:59:53.915963+0000","last_became_peered":"2023-10-06T04:59:53.915963+0000","last_unstale":"2023-10-06T04:59:54.915807+0000","last_undegraded":"2023-10-06T04:59:54.915807+0000","last_fullsized":"2023-10-06T04:59:54.915807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":86,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.901709+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:59:53.901709+0000","last_peered":"2023-10-06T04:59:53.901709+0000","last_clean":"2023-10-06T04:59:53.901709+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:59:53.901709+0000","last_undegraded":"2023-10-06T04:59:53.901709+0000","last_fullsized":"2023-10-06T04:59:53.901709+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":233,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916693+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:59:53.916693+0000","last_peered":"2023-10-06T04:59:53.916693+0000","last_clean":"2023-10-06T04:59:53.916693+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:59:53.916693+0000","last_undegraded":"2023-10-06T04:59:53.916693+0000","last_fullsized":"2023-10-06T04:59:53.916693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594581+0000","last_change":"2023-10-06T04:59:54.594581+0000","last_active":"2023-10-06T04:59:54.594581+0000","last_peered":"2023-10-06T04:59:54.594581+0000","last_clean":"2023-10-06T04:59:54.594581+0000","last_became_active":"2023-10-06T04:59:54.594292+0000","last_became_peered":"2023-10-06T04:59:54.594292+0000","last_unstale":"2023-10-06T04:59:54.594581+0000","last_undegraded":"2023-10-06T04:59:54.594581+0000","last_fullsized":"2023-10-06T04:59:54.594581+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:35.567647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.918408+0000","last_change":"2023-10-06T04:59:53.914899+0000","last_active":"2023-10-06T04:59:53.918408+0000","last_peered":"2023-10-06T04:59:53.918408+0000","last_clean":"2023-10-06T04:59:53.918408+0000","last_became_active":"2023-10-06T04:59:53.914638+0000","last_became_peered":"2023-10-06T04:59:53.914638+0000","last_unstale":"2023-10-06T04:59:53.918408+0000","last_undegraded":"2023-10-06T04:59:53.918408+0000","last_fullsized":"2023-10-06T04:59:53.918408+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:41.337292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.917142+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:59:53.917142+0000","last_peered":"2023-10-06T04:59:53.917142+0000","last_clean":"2023-10-06T04:59:53.917142+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:59:53.917142+0000","last_undegraded":"2023-10-06T04:59:53.917142+0000","last_fullsized":"2023-10-06T04:59:53.917142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596669+0000","last_change":"2023-10-06T04:59:54.596669+0000","last_active":"2023-10-06T04:59:54.596669+0000","last_peered":"2023-10-06T04:59:54.596669+0000","last_clean":"2023-10-06T04:59:54.596669+0000","last_became_active":"2023-10-06T04:59:54.596119+0000","last_became_peered":"2023-10-06T04:59:54.596119+0000","last_unstale":"2023-10-06T04:59:54.596669+0000","last_undegraded":"2023-10-06T04:59:54.596669+0000","last_fullsized":"2023-10-06T04:59:54.596669+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:36.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"107'8","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.917227+0000","last_change":"2023-10-06T04:59:53.916243+0000","last_active":"2023-10-06T04:59:54.917227+0000","last_peered":"2023-10-06T04:59:54.917227+0000","last_clean":"2023-10-06T04:59:54.917227+0000","last_became_active":"2023-10-06T04:59:53.915976+0000","last_became_peered":"2023-10-06T04:59:53.915976+0000","last_unstale":"2023-10-06T04:59:54.917227+0000","last_undegraded":"2023-10-06T04:59:54.917227+0000","last_fullsized":"2023-10-06T04:59:54.917227+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:58.766719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916782+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:59:53.916782+0000","last_peered":"2023-10-06T04:59:53.916782+0000","last_clean":"2023-10-06T04:59:53.916782+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:59:53.916782+0000","last_undegraded":"2023-10-06T04:59:53.916782+0000","last_fullsized":"2023-10-06T04:59:53.916782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.581157+0000","last_change":"2023-10-06T04:59:54.581157+0000","last_active":"2023-10-06T04:59:54.581157+0000","last_peered":"2023-10-06T04:59:54.581157+0000","last_clean":"2023-10-06T04:59:54.581157+0000","last_became_active":"2023-10-06T04:59:54.580882+0000","last_became_peered":"2023-10-06T04:59:54.580882+0000","last_unstale":"2023-10-06T04:59:54.581157+0000","last_undegraded":"2023-10-06T04:59:54.581157+0000","last_fullsized":"2023-10-06T04:59:54.581157+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:12.640313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.920180+0000","last_change":"2023-10-06T04:59:53.916708+0000","last_active":"2023-10-06T04:59:53.920180+0000","last_peered":"2023-10-06T04:59:53.920180+0000","last_clean":"2023-10-06T04:59:53.920180+0000","last_became_active":"2023-10-06T04:59:53.916041+0000","last_became_peered":"2023-10-06T04:59:53.916041+0000","last_unstale":"2023-10-06T04:59:53.920180+0000","last_undegraded":"2023-10-06T04:59:53.920180+0000","last_fullsized":"2023-10-06T04:59:53.920180+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:39.186347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.906613+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:59:53.906613+0000","last_peered":"2023-10-06T04:59:53.906613+0000","last_clean":"2023-10-06T04:59:53.906613+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:59:53.906613+0000","last_undegraded":"2023-10-06T04:59:53.906613+0000","last_fullsized":"2023-10-06T04:59:53.906613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594645+0000","last_change":"2023-10-06T04:59:54.594645+0000","last_active":"2023-10-06T04:59:54.594645+0000","last_peered":"2023-10-06T04:59:54.594645+0000","last_clean":"2023-10-06T04:59:54.594645+0000","last_became_active":"2023-10-06T04:59:54.594286+0000","last_became_peered":"2023-10-06T04:59:54.594286+0000","last_unstale":"2023-10-06T04:59:54.594645+0000","last_undegraded":"2023-10-06T04:59:54.594645+0000","last_fullsized":"2023-10-06T04:59:54.594645+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:35.758881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927240+0000","last_change":"2023-10-06T04:59:53.920334+0000","last_active":"2023-10-06T04:59:53.927240+0000","last_peered":"2023-10-06T04:59:53.927240+0000","last_clean":"2023-10-06T04:59:53.927240+0000","last_became_active":"2023-10-06T04:59:53.920021+0000","last_became_peered":"2023-10-06T04:59:53.920021+0000","last_unstale":"2023-10-06T04:59:53.927240+0000","last_undegraded":"2023-10-06T04:59:53.927240+0000","last_fullsized":"2023-10-06T04:59:53.927240+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:29.175360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.909907+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:59:53.909907+0000","last_peered":"2023-10-06T04:59:53.909907+0000","last_clean":"2023-10-06T04:59:53.909907+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:59:53.909907+0000","last_undegraded":"2023-10-06T04:59:53.909907+0000","last_fullsized":"2023-10-06T04:59:53.909907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596684+0000","last_change":"2023-10-06T04:59:54.596684+0000","last_active":"2023-10-06T04:59:54.596684+0000","last_peered":"2023-10-06T04:59:54.596684+0000","last_clean":"2023-10-06T04:59:54.596684+0000","last_became_active":"2023-10-06T04:59:54.596397+0000","last_became_peered":"2023-10-06T04:59:54.596397+0000","last_unstale":"2023-10-06T04:59:54.596684+0000","last_undegraded":"2023-10-06T04:59:54.596684+0000","last_fullsized":"2023-10-06T04:59:54.596684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:54.707605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.922992+0000","last_change":"2023-10-06T04:59:53.918963+0000","last_active":"2023-10-06T04:59:53.922992+0000","last_peered":"2023-10-06T04:59:53.922992+0000","last_clean":"2023-10-06T04:59:53.922992+0000","last_became_active":"2023-10-06T04:59:53.918643+0000","last_became_peered":"2023-10-06T04:59:53.918643+0000","last_unstale":"2023-10-06T04:59:53.922992+0000","last_undegraded":"2023-10-06T04:59:53.922992+0000","last_fullsized":"2023-10-06T04:59:53.922992+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:52.729068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.910367+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:59:53.910367+0000","last_peered":"2023-10-06T04:59:53.910367+0000","last_clean":"2023-10-06T04:59:53.910367+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:59:53.910367+0000","last_undegraded":"2023-10-06T04:59:53.910367+0000","last_fullsized":"2023-10-06T04:59:53.910367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307372,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064468,"statfs":{"total":96636764160,"available":96322015232,"internally_reserved":0,"allocated":1937408,"data_stored":1191684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":0.71699999999999997},"min":{"1min":0.35499999999999998,"5min":0.28999999999999998,"15min":0.22800000000000001},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.753},"min":{"1min":0.442,"5min":0.38,"15min":0.311},"max":{"1min":0.79800000000000004,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.69299999999999995,"5min":0.747,"15min":1.073},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75800000000000001,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73799999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.82599999999999996,"5min":0.90000000000000002,"15min":1.111},"last":0.89300000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.58299999999999996},"min":{"1min":0.36799999999999999,"5min":0.36699999999999999,"15min":0.33200000000000002},"max":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78000000000000003,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.85799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45900000000000002,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.73299999999999998,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.53900000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81399999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.63},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.61899999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.214},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.58799999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1335296,"data_stored":602475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59699999999999998,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58999999999999997,"15min":0.54400000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.86399999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.54100000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.83099999999999996,"5min":1.234,"15min":1.234},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84599999999999997,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.58399999999999996},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.755},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.60399999999999998,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.76500000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.78200000000000003,"5min":3.669,"15min":3.669},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.88,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83199999999999996,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.379,"5min":0.375,"15min":0.35599999999999998},"max":{"1min":0.85899999999999999,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.80500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307360,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064480,"statfs":{"total":96636764160,"available":96322027520,"internally_reserved":0,"allocated":1925120,"data_stored":1190958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.246,"5min":0.216,"15min":0.216},"max":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38300000000000001,"15min":0.38100000000000001},"min":{"1min":0.248,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32000000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39600000000000002,"15min":0.40200000000000002},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57299999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.245}]},{"osd":3,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42099999999999999,"15min":0.42599999999999999},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.70999999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.877,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63,"15min":0.63100000000000001},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.33700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.872,"5min":0.998,"15min":0.998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.65400000000000003},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84099999999999997,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.50600000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0900000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65700000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.66700000000000004},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.026},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67900000000000005,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.57199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306784,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065056,"statfs":{"total":96636764160,"available":96322617344,"internally_reserved":0,"allocated":1335296,"data_stored":601943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.24099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.38300000000000001,"15min":0.36199999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.40000000000000002},"min":{"1min":0.27400000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56499999999999995,"5min":0.628,"15min":0.628},"last":0.433},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42199999999999999,"15min":0.38500000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.55900000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39800000000000002,"15min":0.371},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53000000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45500000000000002,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63400000000000001,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79200000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":1.022,"15min":1.022},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81000000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84699999999999998,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86499999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67700000000000005,"15min":0.65400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78600000000000003,"5min":0.995,"15min":0.995},"last":0.73799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302684,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069156,"statfs":{"total":96636764160,"available":96326815744,"internally_reserved":0,"allocated":1331200,"data_stored":601921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.252,"5min":0.19500000000000001,"15min":0.186},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":1.048},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42299999999999999,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.56399999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.30399999999999999,"5min":0.23100000000000001,"15min":0.216},"max":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.253},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.36399999999999999},"min":{"1min":0.27800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.623,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.376}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.373,"15min":0.35099999999999998},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44700000000000001,"15min":0.45200000000000001},"min":{"1min":0.313,"5min":0.29599999999999999,"15min":0.27400000000000002},"max":{"1min":0.59899999999999998,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.35699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.45300000000000001,"5min":0.35799999999999998,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.88600000000000001,"15min":1.246},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78100000000000003,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.38400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83799999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40300000000000002},"max":{"1min":0.752,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.81899999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64000000000000001,"15min":0.627},"min":{"1min":0.36899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80300000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.65700000000000003,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.79100000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306796,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065044,"statfs":{"total":96636764160,"available":96322605056,"internally_reserved":0,"allocated":1347584,"data_stored":603491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38800000000000001,"15min":0.40100000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.47899999999999998,"5min":2.008,"15min":2.008},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.41299999999999998,"15min":0.54700000000000004},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.26200000000000001},"max":{"1min":0.57299999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.372}]},{"osd":1,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.372,"15min":0.35899999999999999},"min":{"1min":0.23000000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.67000000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.45500000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.57099999999999995,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42299999999999999,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":1.379,"15min":5.069},"last":0.307},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.45500000000000002,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.628,"5min":2.246,"15min":2.246},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.73699999999999999,"15min":0.77800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.37},"max":{"1min":0.81899999999999995,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.72999999999999998,"15min":0.73099999999999998},"min":{"1min":0.373,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.41199999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.30599999999999999},"max":{"1min":0.84299999999999997,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.75700000000000001},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81499999999999995,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65400000000000003,"15min":0.748},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.5},{"interface":"front","average":{"1min":0.623,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.77600000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.87,"5min":1.964,"15min":6.367},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.73299999999999998},"min":{"1min":0.48699999999999999,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":1.3,"15min":5},"last":0.439}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306780,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065060,"statfs":{"total":96636764160,"available":96322621440,"internally_reserved":0,"allocated":1331200,"data_stored":601921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.313,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.54700000000000004,"15min":0.75700000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.67300000000000004,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.78500000000000003},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65400000000000003,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57899999999999996,"15min":0.79300000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.308},"max":{"1min":0.72399999999999998,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.68300000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.65300000000000002},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.69899999999999995,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.54300000000000004,"15min":0.66700000000000004},"min":{"1min":0.38600000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60399999999999998,"5min":0.80400000000000005,"15min":5.415},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.65700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.70399999999999996,"5min":0.83199999999999996,"15min":4.202},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61499999999999999,"15min":0.71299999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.751,"5min":1.044,"15min":4.2130000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.497},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60199999999999998,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.745,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51400000000000001,"15min":0.54200000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.66100000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54000000000000004,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.68200000000000005,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.47899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.66700000000000004,"5min":0.86199999999999999,"15min":0.872},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59199999999999997,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.313},"max":{"1min":0.75800000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.501}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306784,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065056,"statfs":{"total":96636764160,"available":96322617344,"internally_reserved":0,"allocated":1335296,"data_stored":600728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52800000000000002,"15min":0.66100000000000003},"min":{"1min":0.36799999999999999,"5min":0.26300000000000001,"15min":0.20499999999999999},"max":{"1min":0.73299999999999998,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.61799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.69199999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":6.7949999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.65700000000000003},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.73899999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85099999999999998,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.64500000000000002},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.26200000000000001},"max":{"1min":0.71199999999999997,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.90200000000000002},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.30199999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.89900000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":9.7539999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.61399999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.56200000000000006},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.74399999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.54700000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67500000000000004,"5min":0.746,"15min":0.746},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.80100000000000005},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59099999999999997,"15min":0.63100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76200000000000001,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:59:59.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:59:59.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.179+0000 7f0c5e9a2700 1 Processor -- start 2023-10-06T04:59:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.179+0000 7f0c5e9a2700 1 -- start start 2023-10-06T04:59:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c5e9a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c5815c8f0 0x7f0c5815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c5e9a2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0c5815ccc0 0x7f0c580d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c5e9a2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c580cf6c0 con 0x7f0c580d06a0 2023-10-06T04:59:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c5e9a2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c580cfb50 con 0x7f0c5815c8f0 2023-10-06T04:59:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c5e9a2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c580d0b20 con 0x7f0c5815ccc0 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c57fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c5815c8f0 0x7f0c5815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c5cf3f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0c580d06a0 0x7f0c5806d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50030/0 (socket says 172.21.15.3:50030) 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c5cf3f700 1 -- 172.21.15.3:0/834332933 learned_addr learned my addr 172.21.15.3:0/834332933 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c577fe700 1 --2- 172.21.15.3:0/834332933 >> v2:172.21.15.153:3300/0 conn(0x7f0c5815ccc0 0x7f0c580d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c56ffd700 1 -- 172.21.15.3:0/834332933 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273715791 0 0) 0x7f0c580cf6c0 con 0x7f0c580d06a0 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c56ffd700 1 -- 172.21.15.3:0/834332933 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c44003570 con 0x7f0c580d06a0 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c57fff700 1 -- 172.21.15.3:0/834332933 >> v2:172.21.15.153:3300/0 conn(0x7f0c5815ccc0 msgr2=0x7f0c580d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c57fff700 1 --2- 172.21.15.3:0/834332933 >> v2:172.21.15.153:3300/0 conn(0x7f0c5815ccc0 0x7f0c580d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c57fff700 1 -- 172.21.15.3:0/834332933 >> v1:172.21.15.3:6789/0 conn(0x7f0c580d06a0 legacy=0x7f0c5806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c57fff700 1 -- 172.21.15.3:0/834332933 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5815e050 con 0x7f0c5815c8f0 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.180+0000 7f0c577fe700 1 --2- 172.21.15.3:0/834332933 >> v2:172.21.15.153:3300/0 conn(0x7f0c5815ccc0 0x7f0c580d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c57fff700 1 --2- 172.21.15.3:0/834332933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c5815c8f0 0x7f0c5815db10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0c480098e0 tx=0x7f0c480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7537c6b86cd19b50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c56ffd700 1 -- 172.21.15.3:0/834332933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48011630 con 0x7f0c5815c8f0 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c56ffd700 1 -- 172.21.15.3:0/834332933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c48011790 con 0x7f0c5815c8f0 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c56ffd700 1 -- 172.21.15.3:0/834332933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48011b30 con 0x7f0c5815c8f0 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/834332933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c5815c8f0 msgr2=0x7f0c5815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/834332933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c5815c8f0 0x7f0c5815db10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0c480098e0 tx=0x7f0c480091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/834332933 shutdown_connections 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/834332933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c5815c8f0 0x7f0c5815db10 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/834332933 >> v2:172.21.15.153:3300/0 conn(0x7f0c5815ccc0 0x7f0c580d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/834332933 >> 172.21.15.3:0/834332933 conn(0x7f0c58070310 msgr2=0x7f0c58070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/834332933 shutdown_connections 2023-10-06T04:59:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.181+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/834332933 wait complete. 2023-10-06T04:59:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.182+0000 7f0c5e9a2700 1 Processor -- start 2023-10-06T04:59:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c5e9a2700 1 -- start start 2023-10-06T04:59:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c5e9a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c580d06a0 0x7f0c58143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c5e9a2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5815c8f0 0x7f0c580c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c5e9a2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5815ccc0 0x7f0c58155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c5e9a2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580cad90 con 0x7f0c580d06a0 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c5e9a2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580ca180 con 0x7f0c5815ccc0 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c5e9a2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580ca460 con 0x7f0c5815c8f0 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c57fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c580d06a0 0x7f0c58143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c57fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c580d06a0 0x7f0c58143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47990/0 (socket says 172.21.15.3:47990) 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.183+0000 7f0c57fff700 1 -- 172.21.15.3:0/2872605452 learned_addr learned my addr 172.21.15.3:0/2872605452 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c57fff700 1 -- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5815c8f0 msgr2=0x7f0c580c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c57fff700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5815c8f0 0x7f0c580c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c57fff700 1 -- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5815ccc0 msgr2=0x7f0c58155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c57fff700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5815ccc0 0x7f0c58155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c57fff700 1 -- 172.21.15.3:0/2872605452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48009420 con 0x7f0c580d06a0 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c57fff700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c580d06a0 0x7f0c58143860 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0c480091e0 tx=0x7f0c48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c4ffff700 1 -- 172.21.15.3:0/2872605452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4801bce0 con 0x7f0c580d06a0 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c4ffff700 1 -- 172.21.15.3:0/2872605452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4801be60 con 0x7f0c580d06a0 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c4ffff700 1 -- 172.21.15.3:0/2872605452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48025670 con 0x7f0c580d06a0 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c580ca740 con 0x7f0c580d06a0 2023-10-06T04:59:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.184+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c58143da0 con 0x7f0c580d06a0 2023-10-06T04:59:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.185+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c58071680 con 0x7f0c580d06a0 2023-10-06T04:59:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.185+0000 7f0c4ffff700 1 -- 172.21.15.3:0/2872605452 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c48023070 con 0x7f0c580d06a0 2023-10-06T04:59:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.186+0000 7f0c4ffff700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c38066f30 0x7f0c380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.186+0000 7f0c4ffff700 1 -- 172.21.15.3:0/2872605452 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c48085c90 con 0x7f0c580d06a0 2023-10-06T04:59:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.186+0000 7f0c577fe700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c38066f30 0x7f0c380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.187+0000 7f0c577fe700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c38066f30 0x7f0c380693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006fd0 tx=0x7f0c40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:59.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.190+0000 7f0c4ffff700 1 -- 172.21.15.3:0/2872605452 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c48052da0 con 0x7f0c580d06a0 2023-10-06T04:59:59.221 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-10-06T04:59:59.220+0000 7f73cb68a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:59:59.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.382+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c580caed0 con 0x7f0c38066f30 2023-10-06T04:59:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.391+0000 7f0c4ffff700 1 -- 172.21.15.3:0/2872605452 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312125 (secure 0 0 0) 0x7f0c40007e50 con 0x7f0c38066f30 2023-10-06T04:59:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T04:59:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.394+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c38066f30 msgr2=0x7f0c380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.394+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c38066f30 0x7f0c380693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006fd0 tx=0x7f0c40008040 comp rx=0 tx=0).stop 2023-10-06T04:59:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.394+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c580d06a0 msgr2=0x7f0c58143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.394+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c580d06a0 0x7f0c58143860 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0c480091e0 tx=0x7f0c48017000 comp rx=0 tx=0).stop 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.394+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 shutdown_connections 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.395+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c38066f30 0x7f0c380693e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.395+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5815ccc0 0x7f0c58155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.395+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c580d06a0 0x7f0c58143860 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.395+0000 7f0c5e9a2700 1 --2- 172.21.15.3:0/2872605452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5815c8f0 0x7f0c580c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.395+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 >> 172.21.15.3:0/2872605452 conn(0x7f0c58070310 msgr2=0x7f0c580b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.395+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 shutdown_connections 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.395+0000 7f0c5e9a2700 1 -- 172.21.15.3:0/2872605452 wait complete. 2023-10-06T04:59:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T04:59:59.408 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-10-06T04:59:57.537052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2447248,"kb_used_data":11600,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752527472,"statfs":{"total":773094113280,"available":770588131328,"internally_reserved":0,"allocated":11878400,"data_stored":5995121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.641641"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584608+0000","last_change":"2023-10-06T04:59:54.584608+0000","last_active":"2023-10-06T04:59:54.584608+0000","last_peered":"2023-10-06T04:59:54.584608+0000","last_clean":"2023-10-06T04:59:54.584608+0000","last_became_active":"2023-10-06T04:59:54.584357+0000","last_became_peered":"2023-10-06T04:59:54.584357+0000","last_unstale":"2023-10-06T04:59:54.584608+0000","last_undegraded":"2023-10-06T04:59:54.584608+0000","last_fullsized":"2023-10-06T04:59:54.584608+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:54.440341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595020+0000","last_change":"2023-10-06T04:59:54.595020+0000","last_active":"2023-10-06T04:59:54.595020+0000","last_peered":"2023-10-06T04:59:54.595020+0000","last_clean":"2023-10-06T04:59:54.595020+0000","last_became_active":"2023-10-06T04:59:54.594703+0000","last_became_peered":"2023-10-06T04:59:54.594703+0000","last_unstale":"2023-10-06T04:59:54.595020+0000","last_undegraded":"2023-10-06T04:59:54.595020+0000","last_fullsized":"2023-10-06T04:59:54.595020+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:05.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.600180+0000","last_change":"2023-10-06T04:59:54.600180+0000","last_active":"2023-10-06T04:59:54.600180+0000","last_peered":"2023-10-06T04:59:54.600180+0000","last_clean":"2023-10-06T04:59:54.600180+0000","last_became_active":"2023-10-06T04:59:54.599965+0000","last_became_peered":"2023-10-06T04:59:54.599965+0000","last_unstale":"2023-10-06T04:59:54.600180+0000","last_undegraded":"2023-10-06T04:59:54.600180+0000","last_fullsized":"2023-10-06T04:59:54.600180+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:47.262846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592814+0000","last_change":"2023-10-06T04:59:54.592814+0000","last_active":"2023-10-06T04:59:54.592814+0000","last_peered":"2023-10-06T04:59:54.592814+0000","last_clean":"2023-10-06T04:59:54.592814+0000","last_became_active":"2023-10-06T04:59:54.592550+0000","last_became_peered":"2023-10-06T04:59:54.592550+0000","last_unstale":"2023-10-06T04:59:54.592814+0000","last_undegraded":"2023-10-06T04:59:54.592814+0000","last_fullsized":"2023-10-06T04:59:54.592814+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:16.473260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589887+0000","last_change":"2023-10-06T04:59:54.589887+0000","last_active":"2023-10-06T04:59:54.589887+0000","last_peered":"2023-10-06T04:59:54.589887+0000","last_clean":"2023-10-06T04:59:54.589887+0000","last_became_active":"2023-10-06T04:59:54.589616+0000","last_became_peered":"2023-10-06T04:59:54.589616+0000","last_unstale":"2023-10-06T04:59:54.589887+0000","last_undegraded":"2023-10-06T04:59:54.589887+0000","last_fullsized":"2023-10-06T04:59:54.589887+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:47.017432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591514+0000","last_change":"2023-10-06T04:59:54.591514+0000","last_active":"2023-10-06T04:59:54.591514+0000","last_peered":"2023-10-06T04:59:54.591514+0000","last_clean":"2023-10-06T04:59:54.591514+0000","last_became_active":"2023-10-06T04:59:54.591266+0000","last_became_peered":"2023-10-06T04:59:54.591266+0000","last_unstale":"2023-10-06T04:59:54.591514+0000","last_undegraded":"2023-10-06T04:59:54.591514+0000","last_fullsized":"2023-10-06T04:59:54.591514+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:13.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595349+0000","last_change":"2023-10-06T04:59:54.595349+0000","last_active":"2023-10-06T04:59:54.595349+0000","last_peered":"2023-10-06T04:59:54.595349+0000","last_clean":"2023-10-06T04:59:54.595349+0000","last_became_active":"2023-10-06T04:59:54.595078+0000","last_became_peered":"2023-10-06T04:59:54.595078+0000","last_unstale":"2023-10-06T04:59:54.595349+0000","last_undegraded":"2023-10-06T04:59:54.595349+0000","last_fullsized":"2023-10-06T04:59:54.595349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:31.006317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594510+0000","last_change":"2023-10-06T04:59:54.594510+0000","last_active":"2023-10-06T04:59:54.594510+0000","last_peered":"2023-10-06T04:59:54.594510+0000","last_clean":"2023-10-06T04:59:54.594510+0000","last_became_active":"2023-10-06T04:59:54.593986+0000","last_became_peered":"2023-10-06T04:59:54.593986+0000","last_unstale":"2023-10-06T04:59:54.594510+0000","last_undegraded":"2023-10-06T04:59:54.594510+0000","last_fullsized":"2023-10-06T04:59:54.594510+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:12.261663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593027+0000","last_change":"2023-10-06T04:59:54.593027+0000","last_active":"2023-10-06T04:59:54.593027+0000","last_peered":"2023-10-06T04:59:54.593027+0000","last_clean":"2023-10-06T04:59:54.593027+0000","last_became_active":"2023-10-06T04:59:54.592801+0000","last_became_peered":"2023-10-06T04:59:54.592801+0000","last_unstale":"2023-10-06T04:59:54.593027+0000","last_undegraded":"2023-10-06T04:59:54.593027+0000","last_fullsized":"2023-10-06T04:59:54.593027+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:19.500134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.580589+0000","last_change":"2023-10-06T04:59:54.580589+0000","last_active":"2023-10-06T04:59:54.580589+0000","last_peered":"2023-10-06T04:59:54.580589+0000","last_clean":"2023-10-06T04:59:54.580589+0000","last_became_active":"2023-10-06T04:59:54.580262+0000","last_became_peered":"2023-10-06T04:59:54.580262+0000","last_unstale":"2023-10-06T04:59:54.580589+0000","last_undegraded":"2023-10-06T04:59:54.580589+0000","last_fullsized":"2023-10-06T04:59:54.580589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:54.440341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596118+0000","last_change":"2023-10-06T04:59:54.596118+0000","last_active":"2023-10-06T04:59:54.596118+0000","last_peered":"2023-10-06T04:59:54.596118+0000","last_clean":"2023-10-06T04:59:54.596118+0000","last_became_active":"2023-10-06T04:59:54.595339+0000","last_became_peered":"2023-10-06T04:59:54.595339+0000","last_unstale":"2023-10-06T04:59:54.596118+0000","last_undegraded":"2023-10-06T04:59:54.596118+0000","last_fullsized":"2023-10-06T04:59:54.596118+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:42.343032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.586247+0000","last_change":"2023-10-06T04:59:54.586247+0000","last_active":"2023-10-06T04:59:54.586247+0000","last_peered":"2023-10-06T04:59:54.586247+0000","last_clean":"2023-10-06T04:59:54.586247+0000","last_became_active":"2023-10-06T04:59:54.585235+0000","last_became_peered":"2023-10-06T04:59:54.585235+0000","last_unstale":"2023-10-06T04:59:54.586247+0000","last_undegraded":"2023-10-06T04:59:54.586247+0000","last_fullsized":"2023-10-06T04:59:54.586247+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:06.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591643+0000","last_change":"2023-10-06T04:59:54.591643+0000","last_active":"2023-10-06T04:59:54.591643+0000","last_peered":"2023-10-06T04:59:54.591643+0000","last_clean":"2023-10-06T04:59:54.591643+0000","last_became_active":"2023-10-06T04:59:54.591384+0000","last_became_peered":"2023-10-06T04:59:54.591384+0000","last_unstale":"2023-10-06T04:59:54.591643+0000","last_undegraded":"2023-10-06T04:59:54.591643+0000","last_fullsized":"2023-10-06T04:59:54.591643+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:35.758881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595274+0000","last_change":"2023-10-06T04:59:54.595274+0000","last_active":"2023-10-06T04:59:54.595274+0000","last_peered":"2023-10-06T04:59:54.595274+0000","last_clean":"2023-10-06T04:59:54.595274+0000","last_became_active":"2023-10-06T04:59:54.594020+0000","last_became_peered":"2023-10-06T04:59:54.594020+0000","last_unstale":"2023-10-06T04:59:54.595274+0000","last_undegraded":"2023-10-06T04:59:54.595274+0000","last_fullsized":"2023-10-06T04:59:54.595274+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591584+0000","last_change":"2023-10-06T04:59:54.591584+0000","last_active":"2023-10-06T04:59:54.591584+0000","last_peered":"2023-10-06T04:59:54.591584+0000","last_clean":"2023-10-06T04:59:54.591584+0000","last_became_active":"2023-10-06T04:59:54.591304+0000","last_became_peered":"2023-10-06T04:59:54.591304+0000","last_unstale":"2023-10-06T04:59:54.591584+0000","last_undegraded":"2023-10-06T04:59:54.591584+0000","last_fullsized":"2023-10-06T04:59:54.591584+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:42.017800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.586426+0000","last_change":"2023-10-06T04:59:54.586426+0000","last_active":"2023-10-06T04:59:54.586426+0000","last_peered":"2023-10-06T04:59:54.586426+0000","last_clean":"2023-10-06T04:59:54.586426+0000","last_became_active":"2023-10-06T04:59:54.585782+0000","last_became_peered":"2023-10-06T04:59:54.585782+0000","last_unstale":"2023-10-06T04:59:54.586426+0000","last_undegraded":"2023-10-06T04:59:54.586426+0000","last_fullsized":"2023-10-06T04:59:54.586426+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:09.639856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593880+0000","last_change":"2023-10-06T04:59:54.593880+0000","last_active":"2023-10-06T04:59:54.593880+0000","last_peered":"2023-10-06T04:59:54.593880+0000","last_clean":"2023-10-06T04:59:54.593880+0000","last_became_active":"2023-10-06T04:59:54.593600+0000","last_became_peered":"2023-10-06T04:59:54.593600+0000","last_unstale":"2023-10-06T04:59:54.593880+0000","last_undegraded":"2023-10-06T04:59:54.593880+0000","last_fullsized":"2023-10-06T04:59:54.593880+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:22.397690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592678+0000","last_change":"2023-10-06T04:59:54.592678+0000","last_active":"2023-10-06T04:59:54.592678+0000","last_peered":"2023-10-06T04:59:54.592678+0000","last_clean":"2023-10-06T04:59:54.592678+0000","last_became_active":"2023-10-06T04:59:54.592280+0000","last_became_peered":"2023-10-06T04:59:54.592280+0000","last_unstale":"2023-10-06T04:59:54.592678+0000","last_undegraded":"2023-10-06T04:59:54.592678+0000","last_fullsized":"2023-10-06T04:59:54.592678+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:47.710437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596701+0000","last_change":"2023-10-06T04:59:54.596701+0000","last_active":"2023-10-06T04:59:54.596701+0000","last_peered":"2023-10-06T04:59:54.596701+0000","last_clean":"2023-10-06T04:59:54.596701+0000","last_became_active":"2023-10-06T04:59:54.596330+0000","last_became_peered":"2023-10-06T04:59:54.596330+0000","last_unstale":"2023-10-06T04:59:54.596701+0000","last_undegraded":"2023-10-06T04:59:54.596701+0000","last_fullsized":"2023-10-06T04:59:54.596701+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:28.571600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584320+0000","last_change":"2023-10-06T04:59:54.584320+0000","last_active":"2023-10-06T04:59:54.584320+0000","last_peered":"2023-10-06T04:59:54.584320+0000","last_clean":"2023-10-06T04:59:54.584320+0000","last_became_active":"2023-10-06T04:59:54.584045+0000","last_became_peered":"2023-10-06T04:59:54.584045+0000","last_unstale":"2023-10-06T04:59:54.584320+0000","last_undegraded":"2023-10-06T04:59:54.584320+0000","last_fullsized":"2023-10-06T04:59:54.584320+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:17.752152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594295+0000","last_change":"2023-10-06T04:59:54.594295+0000","last_active":"2023-10-06T04:59:54.594295+0000","last_peered":"2023-10-06T04:59:54.594295+0000","last_clean":"2023-10-06T04:59:54.594295+0000","last_became_active":"2023-10-06T04:59:54.594020+0000","last_became_peered":"2023-10-06T04:59:54.594020+0000","last_unstale":"2023-10-06T04:59:54.594295+0000","last_undegraded":"2023-10-06T04:59:54.594295+0000","last_fullsized":"2023-10-06T04:59:54.594295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:56.363467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592083+0000","last_change":"2023-10-06T04:59:54.592083+0000","last_active":"2023-10-06T04:59:54.592083+0000","last_peered":"2023-10-06T04:59:54.592083+0000","last_clean":"2023-10-06T04:59:54.592083+0000","last_became_active":"2023-10-06T04:59:54.591804+0000","last_became_peered":"2023-10-06T04:59:54.591804+0000","last_unstale":"2023-10-06T04:59:54.592083+0000","last_undegraded":"2023-10-06T04:59:54.592083+0000","last_fullsized":"2023-10-06T04:59:54.592083+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:12.340942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593962+0000","last_change":"2023-10-06T04:59:54.593962+0000","last_active":"2023-10-06T04:59:54.593962+0000","last_peered":"2023-10-06T04:59:54.593962+0000","last_clean":"2023-10-06T04:59:54.593962+0000","last_became_active":"2023-10-06T04:59:54.593661+0000","last_became_peered":"2023-10-06T04:59:54.593661+0000","last_unstale":"2023-10-06T04:59:54.593962+0000","last_undegraded":"2023-10-06T04:59:54.593962+0000","last_fullsized":"2023-10-06T04:59:54.593962+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:07.744381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594314+0000","last_change":"2023-10-06T04:59:54.594314+0000","last_active":"2023-10-06T04:59:54.594314+0000","last_peered":"2023-10-06T04:59:54.594314+0000","last_clean":"2023-10-06T04:59:54.594314+0000","last_became_active":"2023-10-06T04:59:54.594044+0000","last_became_peered":"2023-10-06T04:59:54.594044+0000","last_unstale":"2023-10-06T04:59:54.594314+0000","last_undegraded":"2023-10-06T04:59:54.594314+0000","last_fullsized":"2023-10-06T04:59:54.594314+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:55.261863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596484+0000","last_change":"2023-10-06T04:59:54.596484+0000","last_active":"2023-10-06T04:59:54.596484+0000","last_peered":"2023-10-06T04:59:54.596484+0000","last_clean":"2023-10-06T04:59:54.596484+0000","last_became_active":"2023-10-06T04:59:54.595647+0000","last_became_peered":"2023-10-06T04:59:54.595647+0000","last_unstale":"2023-10-06T04:59:54.596484+0000","last_undegraded":"2023-10-06T04:59:54.596484+0000","last_fullsized":"2023-10-06T04:59:54.596484+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:41.823804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.582583+0000","last_change":"2023-10-06T04:59:54.582583+0000","last_active":"2023-10-06T04:59:54.582583+0000","last_peered":"2023-10-06T04:59:54.582583+0000","last_clean":"2023-10-06T04:59:54.582583+0000","last_became_active":"2023-10-06T04:59:54.582362+0000","last_became_peered":"2023-10-06T04:59:54.582362+0000","last_unstale":"2023-10-06T04:59:54.582583+0000","last_undegraded":"2023-10-06T04:59:54.582583+0000","last_fullsized":"2023-10-06T04:59:54.582583+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:51.997295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.597696+0000","last_change":"2023-10-06T04:59:54.597696+0000","last_active":"2023-10-06T04:59:54.597696+0000","last_peered":"2023-10-06T04:59:54.597696+0000","last_clean":"2023-10-06T04:59:54.597696+0000","last_became_active":"2023-10-06T04:59:54.596957+0000","last_became_peered":"2023-10-06T04:59:54.596957+0000","last_unstale":"2023-10-06T04:59:54.597696+0000","last_undegraded":"2023-10-06T04:59:54.597696+0000","last_fullsized":"2023-10-06T04:59:54.597696+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:54.220363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592465+0000","last_change":"2023-10-06T04:59:54.592465+0000","last_active":"2023-10-06T04:59:54.592465+0000","last_peered":"2023-10-06T04:59:54.592465+0000","last_clean":"2023-10-06T04:59:54.592465+0000","last_became_active":"2023-10-06T04:59:54.592173+0000","last_became_peered":"2023-10-06T04:59:54.592173+0000","last_unstale":"2023-10-06T04:59:54.592465+0000","last_undegraded":"2023-10-06T04:59:54.592465+0000","last_fullsized":"2023-10-06T04:59:54.592465+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:30.969822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590575+0000","last_change":"2023-10-06T04:59:54.590575+0000","last_active":"2023-10-06T04:59:54.590575+0000","last_peered":"2023-10-06T04:59:54.590575+0000","last_clean":"2023-10-06T04:59:54.590575+0000","last_became_active":"2023-10-06T04:59:54.590317+0000","last_became_peered":"2023-10-06T04:59:54.590317+0000","last_unstale":"2023-10-06T04:59:54.590575+0000","last_undegraded":"2023-10-06T04:59:54.590575+0000","last_fullsized":"2023-10-06T04:59:54.590575+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:45.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590402+0000","last_change":"2023-10-06T04:59:54.590402+0000","last_active":"2023-10-06T04:59:54.590402+0000","last_peered":"2023-10-06T04:59:54.590402+0000","last_clean":"2023-10-06T04:59:54.590402+0000","last_became_active":"2023-10-06T04:59:54.589995+0000","last_became_peered":"2023-10-06T04:59:54.589995+0000","last_unstale":"2023-10-06T04:59:54.590402+0000","last_undegraded":"2023-10-06T04:59:54.590402+0000","last_fullsized":"2023-10-06T04:59:54.590402+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:39.522866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591166+0000","last_change":"2023-10-06T04:59:54.591166+0000","last_active":"2023-10-06T04:59:54.591166+0000","last_peered":"2023-10-06T04:59:54.591166+0000","last_clean":"2023-10-06T04:59:54.591166+0000","last_became_active":"2023-10-06T04:59:54.590833+0000","last_became_peered":"2023-10-06T04:59:54.590833+0000","last_unstale":"2023-10-06T04:59:54.591166+0000","last_undegraded":"2023-10-06T04:59:54.591166+0000","last_fullsized":"2023-10-06T04:59:54.591166+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:35.567647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594295+0000","last_change":"2023-10-06T04:59:54.594295+0000","last_active":"2023-10-06T04:59:54.594295+0000","last_peered":"2023-10-06T04:59:54.594295+0000","last_clean":"2023-10-06T04:59:54.594295+0000","last_became_active":"2023-10-06T04:59:54.594000+0000","last_became_peered":"2023-10-06T04:59:54.594000+0000","last_unstale":"2023-10-06T04:59:54.594295+0000","last_undegraded":"2023-10-06T04:59:54.594295+0000","last_fullsized":"2023-10-06T04:59:54.594295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:22.397690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584446+0000","last_change":"2023-10-06T04:59:54.584446+0000","last_active":"2023-10-06T04:59:54.584446+0000","last_peered":"2023-10-06T04:59:54.584446+0000","last_clean":"2023-10-06T04:59:54.584446+0000","last_became_active":"2023-10-06T04:59:54.584141+0000","last_became_peered":"2023-10-06T04:59:54.584141+0000","last_unstale":"2023-10-06T04:59:54.584446+0000","last_undegraded":"2023-10-06T04:59:54.584446+0000","last_fullsized":"2023-10-06T04:59:54.584446+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:26.633928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"107'4","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.916911+0000","last_change":"2023-10-06T04:59:53.919668+0000","last_active":"2023-10-06T04:59:54.916911+0000","last_peered":"2023-10-06T04:59:54.916911+0000","last_clean":"2023-10-06T04:59:54.916911+0000","last_became_active":"2023-10-06T04:59:53.919254+0000","last_became_peered":"2023-10-06T04:59:53.919254+0000","last_unstale":"2023-10-06T04:59:54.916911+0000","last_undegraded":"2023-10-06T04:59:54.916911+0000","last_fullsized":"2023-10-06T04:59:54.916911+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:06.327449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590344+0000","last_change":"2023-10-06T04:59:54.590344+0000","last_active":"2023-10-06T04:59:54.590344+0000","last_peered":"2023-10-06T04:59:54.590344+0000","last_clean":"2023-10-06T04:59:54.590344+0000","last_became_active":"2023-10-06T04:59:54.590055+0000","last_became_peered":"2023-10-06T04:59:54.590055+0000","last_unstale":"2023-10-06T04:59:54.590344+0000","last_undegraded":"2023-10-06T04:59:54.590344+0000","last_fullsized":"2023-10-06T04:59:54.590344+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:06.670106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.575356+0000","last_change":"2023-10-06T04:59:53.912231+0000","last_active":"2023-10-06T04:59:54.575356+0000","last_peered":"2023-10-06T04:59:54.575356+0000","last_clean":"2023-10-06T04:59:54.575356+0000","last_became_active":"2023-10-06T04:59:53.912073+0000","last_became_peered":"2023-10-06T04:59:53.912073+0000","last_unstale":"2023-10-06T04:59:54.575356+0000","last_undegraded":"2023-10-06T04:59:54.575356+0000","last_fullsized":"2023-10-06T04:59:54.575356+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:46:00.861292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591018+0000","last_change":"2023-10-06T04:59:54.591018+0000","last_active":"2023-10-06T04:59:54.591018+0000","last_peered":"2023-10-06T04:59:54.591018+0000","last_clean":"2023-10-06T04:59:54.591018+0000","last_became_active":"2023-10-06T04:59:54.590710+0000","last_became_peered":"2023-10-06T04:59:54.590710+0000","last_unstale":"2023-10-06T04:59:54.591018+0000","last_undegraded":"2023-10-06T04:59:54.591018+0000","last_fullsized":"2023-10-06T04:59:54.591018+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:58.864838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.914099+0000","last_change":"2023-10-06T04:59:53.920720+0000","last_active":"2023-10-06T04:59:54.914099+0000","last_peered":"2023-10-06T04:59:54.914099+0000","last_clean":"2023-10-06T04:59:54.914099+0000","last_became_active":"2023-10-06T04:59:53.920411+0000","last_became_peered":"2023-10-06T04:59:53.920411+0000","last_unstale":"2023-10-06T04:59:54.914099+0000","last_undegraded":"2023-10-06T04:59:54.914099+0000","last_fullsized":"2023-10-06T04:59:54.914099+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:41.012060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594908+0000","last_change":"2023-10-06T04:59:54.594908+0000","last_active":"2023-10-06T04:59:54.594908+0000","last_peered":"2023-10-06T04:59:54.594908+0000","last_clean":"2023-10-06T04:59:54.594908+0000","last_became_active":"2023-10-06T04:59:54.594711+0000","last_became_peered":"2023-10-06T04:59:54.594711+0000","last_unstale":"2023-10-06T04:59:54.594908+0000","last_undegraded":"2023-10-06T04:59:54.594908+0000","last_fullsized":"2023-10-06T04:59:54.594908+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:28.347251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.914439+0000","last_change":"2023-10-06T04:59:53.911212+0000","last_active":"2023-10-06T04:59:54.914439+0000","last_peered":"2023-10-06T04:59:54.914439+0000","last_clean":"2023-10-06T04:59:54.914439+0000","last_became_active":"2023-10-06T04:59:53.910952+0000","last_became_peered":"2023-10-06T04:59:53.910952+0000","last_unstale":"2023-10-06T04:59:54.914439+0000","last_undegraded":"2023-10-06T04:59:54.914439+0000","last_fullsized":"2023-10-06T04:59:54.914439+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:52.293461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589691+0000","last_change":"2023-10-06T04:59:54.589691+0000","last_active":"2023-10-06T04:59:54.589691+0000","last_peered":"2023-10-06T04:59:54.589691+0000","last_clean":"2023-10-06T04:59:54.589691+0000","last_became_active":"2023-10-06T04:59:54.589389+0000","last_became_peered":"2023-10-06T04:59:54.589389+0000","last_unstale":"2023-10-06T04:59:54.589691+0000","last_undegraded":"2023-10-06T04:59:54.589691+0000","last_fullsized":"2023-10-06T04:59:54.589691+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:55.190779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.579055+0000","last_change":"2023-10-06T04:59:53.912202+0000","last_active":"2023-10-06T04:59:54.579055+0000","last_peered":"2023-10-06T04:59:54.579055+0000","last_clean":"2023-10-06T04:59:54.579055+0000","last_became_active":"2023-10-06T04:59:53.911897+0000","last_became_peered":"2023-10-06T04:59:53.911897+0000","last_unstale":"2023-10-06T04:59:54.579055+0000","last_undegraded":"2023-10-06T04:59:54.579055+0000","last_fullsized":"2023-10-06T04:59:54.579055+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:40.818064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595739+0000","last_change":"2023-10-06T04:59:54.595739+0000","last_active":"2023-10-06T04:59:54.595739+0000","last_peered":"2023-10-06T04:59:54.595739+0000","last_clean":"2023-10-06T04:59:54.595739+0000","last_became_active":"2023-10-06T04:59:54.595372+0000","last_became_peered":"2023-10-06T04:59:54.595372+0000","last_unstale":"2023-10-06T04:59:54.595739+0000","last_undegraded":"2023-10-06T04:59:54.595739+0000","last_fullsized":"2023-10-06T04:59:54.595739+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:40.788038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.579054+0000","last_change":"2023-10-06T04:59:53.918089+0000","last_active":"2023-10-06T04:59:54.579054+0000","last_peered":"2023-10-06T04:59:54.579054+0000","last_clean":"2023-10-06T04:59:54.579054+0000","last_became_active":"2023-10-06T04:59:53.917829+0000","last_became_peered":"2023-10-06T04:59:53.917829+0000","last_unstale":"2023-10-06T04:59:54.579054+0000","last_undegraded":"2023-10-06T04:59:54.579054+0000","last_fullsized":"2023-10-06T04:59:54.579054+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:38.517125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590230+0000","last_change":"2023-10-06T04:59:54.590230+0000","last_active":"2023-10-06T04:59:54.590230+0000","last_peered":"2023-10-06T04:59:54.590230+0000","last_clean":"2023-10-06T04:59:54.590230+0000","last_became_active":"2023-10-06T04:59:54.589685+0000","last_became_peered":"2023-10-06T04:59:54.589685+0000","last_unstale":"2023-10-06T04:59:54.590230+0000","last_undegraded":"2023-10-06T04:59:54.590230+0000","last_fullsized":"2023-10-06T04:59:54.590230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:17.642637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.915490+0000","last_change":"2023-10-06T04:59:53.914581+0000","last_active":"2023-10-06T04:59:53.915490+0000","last_peered":"2023-10-06T04:59:53.915490+0000","last_clean":"2023-10-06T04:59:53.915490+0000","last_became_active":"2023-10-06T04:59:53.914289+0000","last_became_peered":"2023-10-06T04:59:53.914289+0000","last_unstale":"2023-10-06T04:59:53.915490+0000","last_undegraded":"2023-10-06T04:59:53.915490+0000","last_fullsized":"2023-10-06T04:59:53.915490+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:05.644438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589831+0000","last_change":"2023-10-06T04:59:54.589831+0000","last_active":"2023-10-06T04:59:54.589831+0000","last_peered":"2023-10-06T04:59:54.589831+0000","last_clean":"2023-10-06T04:59:54.589831+0000","last_became_active":"2023-10-06T04:59:54.589567+0000","last_became_peered":"2023-10-06T04:59:54.589567+0000","last_unstale":"2023-10-06T04:59:54.589831+0000","last_undegraded":"2023-10-06T04:59:54.589831+0000","last_fullsized":"2023-10-06T04:59:54.589831+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:22.237368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916540+0000","last_change":"2023-10-06T04:59:53.913902+0000","last_active":"2023-10-06T04:59:53.916540+0000","last_peered":"2023-10-06T04:59:53.916540+0000","last_clean":"2023-10-06T04:59:53.916540+0000","last_became_active":"2023-10-06T04:59:53.913365+0000","last_became_peered":"2023-10-06T04:59:53.913365+0000","last_unstale":"2023-10-06T04:59:53.916540+0000","last_undegraded":"2023-10-06T04:59:53.916540+0000","last_fullsized":"2023-10-06T04:59:53.916540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:41.287419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.591869+0000","last_change":"2023-10-06T04:59:54.591869+0000","last_active":"2023-10-06T04:59:54.591869+0000","last_peered":"2023-10-06T04:59:54.591869+0000","last_clean":"2023-10-06T04:59:54.591869+0000","last_became_active":"2023-10-06T04:59:54.591501+0000","last_became_peered":"2023-10-06T04:59:54.591501+0000","last_unstale":"2023-10-06T04:59:54.591869+0000","last_undegraded":"2023-10-06T04:59:54.591869+0000","last_fullsized":"2023-10-06T04:59:54.591869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:26.633928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.575162+0000","last_change":"2023-10-06T04:59:53.916172+0000","last_active":"2023-10-06T04:59:54.575162+0000","last_peered":"2023-10-06T04:59:54.575162+0000","last_clean":"2023-10-06T04:59:54.575162+0000","last_became_active":"2023-10-06T04:59:53.915905+0000","last_became_peered":"2023-10-06T04:59:53.915905+0000","last_unstale":"2023-10-06T04:59:54.575162+0000","last_undegraded":"2023-10-06T04:59:54.575162+0000","last_fullsized":"2023-10-06T04:59:54.575162+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:59.997399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590453+0000","last_change":"2023-10-06T04:59:54.590453+0000","last_active":"2023-10-06T04:59:54.590453+0000","last_peered":"2023-10-06T04:59:54.590453+0000","last_clean":"2023-10-06T04:59:54.590453+0000","last_became_active":"2023-10-06T04:59:54.590158+0000","last_became_peered":"2023-10-06T04:59:54.590158+0000","last_unstale":"2023-10-06T04:59:54.590453+0000","last_undegraded":"2023-10-06T04:59:54.590453+0000","last_fullsized":"2023-10-06T04:59:54.590453+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:39.975563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.914147+0000","last_change":"2023-10-06T04:59:53.917104+0000","last_active":"2023-10-06T04:59:54.914147+0000","last_peered":"2023-10-06T04:59:54.914147+0000","last_clean":"2023-10-06T04:59:54.914147+0000","last_became_active":"2023-10-06T04:59:53.916838+0000","last_became_peered":"2023-10-06T04:59:53.916838+0000","last_unstale":"2023-10-06T04:59:54.914147+0000","last_undegraded":"2023-10-06T04:59:54.914147+0000","last_fullsized":"2023-10-06T04:59:54.914147+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:51.480986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593706+0000","last_change":"2023-10-06T04:59:54.593706+0000","last_active":"2023-10-06T04:59:54.593706+0000","last_peered":"2023-10-06T04:59:54.593706+0000","last_clean":"2023-10-06T04:59:54.593706+0000","last_became_active":"2023-10-06T04:59:54.593256+0000","last_became_peered":"2023-10-06T04:59:54.593256+0000","last_unstale":"2023-10-06T04:59:54.593706+0000","last_undegraded":"2023-10-06T04:59:54.593706+0000","last_fullsized":"2023-10-06T04:59:54.593706+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:05.401334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"107'1","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584725+0000","last_change":"2023-10-06T04:59:53.927934+0000","last_active":"2023-10-06T04:59:54.584725+0000","last_peered":"2023-10-06T04:59:54.584725+0000","last_clean":"2023-10-06T04:59:54.584725+0000","last_became_active":"2023-10-06T04:59:53.927607+0000","last_became_peered":"2023-10-06T04:59:53.927607+0000","last_unstale":"2023-10-06T04:59:54.584725+0000","last_undegraded":"2023-10-06T04:59:54.584725+0000","last_fullsized":"2023-10-06T04:59:54.584725+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:57.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.598566+0000","last_change":"2023-10-06T04:59:54.598566+0000","last_active":"2023-10-06T04:59:54.598566+0000","last_peered":"2023-10-06T04:59:54.598566+0000","last_clean":"2023-10-06T04:59:54.598566+0000","last_became_active":"2023-10-06T04:59:54.598306+0000","last_became_peered":"2023-10-06T04:59:54.598306+0000","last_unstale":"2023-10-06T04:59:54.598566+0000","last_undegraded":"2023-10-06T04:59:54.598566+0000","last_fullsized":"2023-10-06T04:59:54.598566+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:54.440341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927502+0000","last_change":"2023-10-06T04:59:53.922491+0000","last_active":"2023-10-06T04:59:53.927502+0000","last_peered":"2023-10-06T04:59:53.927502+0000","last_clean":"2023-10-06T04:59:53.927502+0000","last_became_active":"2023-10-06T04:59:53.922280+0000","last_became_peered":"2023-10-06T04:59:53.922280+0000","last_unstale":"2023-10-06T04:59:53.927502+0000","last_undegraded":"2023-10-06T04:59:53.927502+0000","last_fullsized":"2023-10-06T04:59:53.927502+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:25.628188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590396+0000","last_change":"2023-10-06T04:59:54.590396+0000","last_active":"2023-10-06T04:59:54.590396+0000","last_peered":"2023-10-06T04:59:54.590396+0000","last_clean":"2023-10-06T04:59:54.590396+0000","last_became_active":"2023-10-06T04:59:54.590152+0000","last_became_peered":"2023-10-06T04:59:54.590152+0000","last_unstale":"2023-10-06T04:59:54.590396+0000","last_undegraded":"2023-10-06T04:59:54.590396+0000","last_fullsized":"2023-10-06T04:59:54.590396+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.557538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.917012+0000","last_change":"2023-10-06T04:59:53.920186+0000","last_active":"2023-10-06T04:59:54.917012+0000","last_peered":"2023-10-06T04:59:54.917012+0000","last_clean":"2023-10-06T04:59:54.917012+0000","last_became_active":"2023-10-06T04:59:53.919875+0000","last_became_peered":"2023-10-06T04:59:53.919875+0000","last_unstale":"2023-10-06T04:59:54.917012+0000","last_undegraded":"2023-10-06T04:59:54.917012+0000","last_fullsized":"2023-10-06T04:59:54.917012+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:12.945202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595872+0000","last_change":"2023-10-06T04:59:54.595872+0000","last_active":"2023-10-06T04:59:54.595872+0000","last_peered":"2023-10-06T04:59:54.595872+0000","last_clean":"2023-10-06T04:59:54.595872+0000","last_became_active":"2023-10-06T04:59:54.595314+0000","last_became_peered":"2023-10-06T04:59:54.595314+0000","last_unstale":"2023-10-06T04:59:54.595872+0000","last_undegraded":"2023-10-06T04:59:54.595872+0000","last_fullsized":"2023-10-06T04:59:54.595872+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:13.950943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927405+0000","last_change":"2023-10-06T04:59:53.920921+0000","last_active":"2023-10-06T04:59:53.927405+0000","last_peered":"2023-10-06T04:59:53.927405+0000","last_clean":"2023-10-06T04:59:53.927405+0000","last_became_active":"2023-10-06T04:59:53.920587+0000","last_became_peered":"2023-10-06T04:59:53.920587+0000","last_unstale":"2023-10-06T04:59:53.927405+0000","last_undegraded":"2023-10-06T04:59:53.927405+0000","last_fullsized":"2023-10-06T04:59:53.927405+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:53.434601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.595940+0000","last_change":"2023-10-06T04:59:54.595940+0000","last_active":"2023-10-06T04:59:54.595940+0000","last_peered":"2023-10-06T04:59:54.595940+0000","last_clean":"2023-10-06T04:59:54.595940+0000","last_became_active":"2023-10-06T04:59:54.594649+0000","last_became_peered":"2023-10-06T04:59:54.594649+0000","last_unstale":"2023-10-06T04:59:54.595940+0000","last_undegraded":"2023-10-06T04:59:54.595940+0000","last_fullsized":"2023-10-06T04:59:54.595940+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:59.860364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.920198+0000","last_change":"2023-10-06T04:59:53.916874+0000","last_active":"2023-10-06T04:59:53.920198+0000","last_peered":"2023-10-06T04:59:53.920198+0000","last_clean":"2023-10-06T04:59:53.920198+0000","last_became_active":"2023-10-06T04:59:53.916418+0000","last_became_peered":"2023-10-06T04:59:53.916418+0000","last_unstale":"2023-10-06T04:59:53.920198+0000","last_undegraded":"2023-10-06T04:59:53.920198+0000","last_fullsized":"2023-10-06T04:59:53.920198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:46.011691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.590884+0000","last_change":"2023-10-06T04:59:54.590884+0000","last_active":"2023-10-06T04:59:54.590884+0000","last_peered":"2023-10-06T04:59:54.590884+0000","last_clean":"2023-10-06T04:59:54.590884+0000","last_became_active":"2023-10-06T04:59:54.590526+0000","last_became_peered":"2023-10-06T04:59:54.590526+0000","last_unstale":"2023-10-06T04:59:54.590884+0000","last_undegraded":"2023-10-06T04:59:54.590884+0000","last_fullsized":"2023-10-06T04:59:54.590884+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:55.190779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927422+0000","last_change":"2023-10-06T04:59:53.920946+0000","last_active":"2023-10-06T04:59:53.927422+0000","last_peered":"2023-10-06T04:59:53.927422+0000","last_clean":"2023-10-06T04:59:53.927422+0000","last_became_active":"2023-10-06T04:59:53.920354+0000","last_became_peered":"2023-10-06T04:59:53.920354+0000","last_unstale":"2023-10-06T04:59:53.927422+0000","last_undegraded":"2023-10-06T04:59:53.927422+0000","last_fullsized":"2023-10-06T04:59:53.927422+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:11.335202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.585504+0000","last_change":"2023-10-06T04:59:54.585504+0000","last_active":"2023-10-06T04:59:54.585504+0000","last_peered":"2023-10-06T04:59:54.585504+0000","last_clean":"2023-10-06T04:59:54.585504+0000","last_became_active":"2023-10-06T04:59:54.584677+0000","last_became_peered":"2023-10-06T04:59:54.584677+0000","last_unstale":"2023-10-06T04:59:54.585504+0000","last_undegraded":"2023-10-06T04:59:54.585504+0000","last_fullsized":"2023-10-06T04:59:54.585504+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:32.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.916783+0000","last_change":"2023-10-06T04:59:53.914923+0000","last_active":"2023-10-06T04:59:54.916783+0000","last_peered":"2023-10-06T04:59:54.916783+0000","last_clean":"2023-10-06T04:59:54.916783+0000","last_became_active":"2023-10-06T04:59:53.914619+0000","last_became_peered":"2023-10-06T04:59:53.914619+0000","last_unstale":"2023-10-06T04:59:54.916783+0000","last_undegraded":"2023-10-06T04:59:54.916783+0000","last_fullsized":"2023-10-06T04:59:54.916783+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:16.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.597666+0000","last_change":"2023-10-06T04:59:54.597666+0000","last_active":"2023-10-06T04:59:54.597666+0000","last_peered":"2023-10-06T04:59:54.597666+0000","last_clean":"2023-10-06T04:59:54.597666+0000","last_became_active":"2023-10-06T04:59:54.597410+0000","last_became_peered":"2023-10-06T04:59:54.597410+0000","last_unstale":"2023-10-06T04:59:54.597666+0000","last_undegraded":"2023-10-06T04:59:54.597666+0000","last_fullsized":"2023-10-06T04:59:54.597666+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:55.261863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.919655+0000","last_change":"2023-10-06T04:59:53.918336+0000","last_active":"2023-10-06T04:59:53.919655+0000","last_peered":"2023-10-06T04:59:53.919655+0000","last_clean":"2023-10-06T04:59:53.919655+0000","last_became_active":"2023-10-06T04:59:53.918037+0000","last_became_peered":"2023-10-06T04:59:53.918037+0000","last_unstale":"2023-10-06T04:59:53.919655+0000","last_undegraded":"2023-10-06T04:59:53.919655+0000","last_fullsized":"2023-10-06T04:59:53.919655+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:33.195194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596385+0000","last_change":"2023-10-06T04:59:54.596385+0000","last_active":"2023-10-06T04:59:54.596385+0000","last_peered":"2023-10-06T04:59:54.596385+0000","last_clean":"2023-10-06T04:59:54.596385+0000","last_became_active":"2023-10-06T04:59:54.595661+0000","last_became_peered":"2023-10-06T04:59:54.595661+0000","last_unstale":"2023-10-06T04:59:54.596385+0000","last_undegraded":"2023-10-06T04:59:54.596385+0000","last_fullsized":"2023-10-06T04:59:54.596385+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:59.772459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.917540+0000","last_change":"2023-10-06T04:59:53.916228+0000","last_active":"2023-10-06T04:59:53.917540+0000","last_peered":"2023-10-06T04:59:53.917540+0000","last_clean":"2023-10-06T04:59:53.917540+0000","last_became_active":"2023-10-06T04:59:53.915923+0000","last_became_peered":"2023-10-06T04:59:53.915923+0000","last_unstale":"2023-10-06T04:59:53.917540+0000","last_undegraded":"2023-10-06T04:59:53.917540+0000","last_fullsized":"2023-10-06T04:59:53.917540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:12.321308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593203+0000","last_change":"2023-10-06T04:59:54.593203+0000","last_active":"2023-10-06T04:59:54.593203+0000","last_peered":"2023-10-06T04:59:54.593203+0000","last_clean":"2023-10-06T04:59:54.593203+0000","last_became_active":"2023-10-06T04:59:54.592905+0000","last_became_peered":"2023-10-06T04:59:54.592905+0000","last_unstale":"2023-10-06T04:59:54.593203+0000","last_undegraded":"2023-10-06T04:59:54.593203+0000","last_fullsized":"2023-10-06T04:59:54.593203+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:29.949833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.921433+0000","last_change":"2023-10-06T04:59:53.918098+0000","last_active":"2023-10-06T04:59:53.921433+0000","last_peered":"2023-10-06T04:59:53.921433+0000","last_clean":"2023-10-06T04:59:53.921433+0000","last_became_active":"2023-10-06T04:59:53.917474+0000","last_became_peered":"2023-10-06T04:59:53.917474+0000","last_unstale":"2023-10-06T04:59:53.921433+0000","last_undegraded":"2023-10-06T04:59:53.921433+0000","last_fullsized":"2023-10-06T04:59:53.921433+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:11.255923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.584615+0000","last_change":"2023-10-06T04:59:54.584615+0000","last_active":"2023-10-06T04:59:54.584615+0000","last_peered":"2023-10-06T04:59:54.584615+0000","last_clean":"2023-10-06T04:59:54.584615+0000","last_became_active":"2023-10-06T04:59:54.584327+0000","last_became_peered":"2023-10-06T04:59:54.584327+0000","last_unstale":"2023-10-06T04:59:54.584615+0000","last_undegraded":"2023-10-06T04:59:54.584615+0000","last_fullsized":"2023-10-06T04:59:54.584615+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:33.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.922779+0000","last_change":"2023-10-06T04:59:53.919498+0000","last_active":"2023-10-06T04:59:53.922779+0000","last_peered":"2023-10-06T04:59:53.922779+0000","last_clean":"2023-10-06T04:59:53.922779+0000","last_became_active":"2023-10-06T04:59:53.919198+0000","last_became_peered":"2023-10-06T04:59:53.919198+0000","last_unstale":"2023-10-06T04:59:53.922779+0000","last_undegraded":"2023-10-06T04:59:53.922779+0000","last_fullsized":"2023-10-06T04:59:53.922779+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:34.753141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.589522+0000","last_change":"2023-10-06T04:59:54.589522+0000","last_active":"2023-10-06T04:59:54.589522+0000","last_peered":"2023-10-06T04:59:54.589522+0000","last_clean":"2023-10-06T04:59:54.589522+0000","last_became_active":"2023-10-06T04:59:54.589251+0000","last_became_peered":"2023-10-06T04:59:54.589251+0000","last_unstale":"2023-10-06T04:59:54.589522+0000","last_undegraded":"2023-10-06T04:59:54.589522+0000","last_fullsized":"2023-10-06T04:59:54.589522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:42.293160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916879+0000","last_change":"2023-10-06T04:59:53.914977+0000","last_active":"2023-10-06T04:59:53.916879+0000","last_peered":"2023-10-06T04:59:53.916879+0000","last_clean":"2023-10-06T04:59:53.916879+0000","last_became_active":"2023-10-06T04:59:53.914697+0000","last_became_peered":"2023-10-06T04:59:53.914697+0000","last_unstale":"2023-10-06T04:59:53.916879+0000","last_undegraded":"2023-10-06T04:59:53.916879+0000","last_fullsized":"2023-10-06T04:59:53.916879+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:01.612930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.588138+0000","last_change":"2023-10-06T04:59:54.588138+0000","last_active":"2023-10-06T04:59:54.588138+0000","last_peered":"2023-10-06T04:59:54.588138+0000","last_clean":"2023-10-06T04:59:54.588138+0000","last_became_active":"2023-10-06T04:59:54.587863+0000","last_became_peered":"2023-10-06T04:59:54.587863+0000","last_unstale":"2023-10-06T04:59:54.588138+0000","last_undegraded":"2023-10-06T04:59:54.588138+0000","last_fullsized":"2023-10-06T04:59:54.588138+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:51.858326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.921853+0000","last_change":"2023-10-06T04:59:53.919329+0000","last_active":"2023-10-06T04:59:53.921853+0000","last_peered":"2023-10-06T04:59:53.921853+0000","last_clean":"2023-10-06T04:59:53.921853+0000","last_became_active":"2023-10-06T04:59:53.918715+0000","last_became_peered":"2023-10-06T04:59:53.918715+0000","last_unstale":"2023-10-06T04:59:53.921853+0000","last_undegraded":"2023-10-06T04:59:53.921853+0000","last_fullsized":"2023-10-06T04:59:53.921853+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:58.309413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594155+0000","last_change":"2023-10-06T04:59:54.594155+0000","last_active":"2023-10-06T04:59:54.594155+0000","last_peered":"2023-10-06T04:59:54.594155+0000","last_clean":"2023-10-06T04:59:54.594155+0000","last_became_active":"2023-10-06T04:59:54.593750+0000","last_became_peered":"2023-10-06T04:59:54.593750+0000","last_unstale":"2023-10-06T04:59:54.594155+0000","last_undegraded":"2023-10-06T04:59:54.594155+0000","last_fullsized":"2023-10-06T04:59:54.594155+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:58.864838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927312+0000","last_change":"2023-10-06T04:59:53.920372+0000","last_active":"2023-10-06T04:59:53.927312+0000","last_peered":"2023-10-06T04:59:53.927312+0000","last_clean":"2023-10-06T04:59:53.927312+0000","last_became_active":"2023-10-06T04:59:53.919988+0000","last_became_peered":"2023-10-06T04:59:53.919988+0000","last_unstale":"2023-10-06T04:59:53.927312+0000","last_undegraded":"2023-10-06T04:59:53.927312+0000","last_fullsized":"2023-10-06T04:59:53.927312+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:28.338164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594099+0000","last_change":"2023-10-06T04:59:54.594099+0000","last_active":"2023-10-06T04:59:54.594099+0000","last_peered":"2023-10-06T04:59:54.594099+0000","last_clean":"2023-10-06T04:59:54.594099+0000","last_became_active":"2023-10-06T04:59:54.593802+0000","last_became_peered":"2023-10-06T04:59:54.593802+0000","last_unstale":"2023-10-06T04:59:54.594099+0000","last_undegraded":"2023-10-06T04:59:54.594099+0000","last_fullsized":"2023-10-06T04:59:54.594099+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:59.315153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"107'3","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.918804+0000","last_change":"2023-10-06T04:59:53.920677+0000","last_active":"2023-10-06T04:59:54.918804+0000","last_peered":"2023-10-06T04:59:54.918804+0000","last_clean":"2023-10-06T04:59:54.918804+0000","last_became_active":"2023-10-06T04:59:53.920381+0000","last_became_peered":"2023-10-06T04:59:53.920381+0000","last_unstale":"2023-10-06T04:59:54.918804+0000","last_undegraded":"2023-10-06T04:59:54.918804+0000","last_fullsized":"2023-10-06T04:59:54.918804+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:53.701865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.909660+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T04:59:53.909660+0000","last_peered":"2023-10-06T04:59:53.909660+0000","last_clean":"2023-10-06T04:59:53.909660+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T04:59:53.909660+0000","last_undegraded":"2023-10-06T04:59:53.909660+0000","last_fullsized":"2023-10-06T04:59:53.909660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.592739+0000","last_change":"2023-10-06T04:59:54.592739+0000","last_active":"2023-10-06T04:59:54.592739+0000","last_peered":"2023-10-06T04:59:54.592739+0000","last_clean":"2023-10-06T04:59:54.592739+0000","last_became_active":"2023-10-06T04:59:54.592223+0000","last_became_peered":"2023-10-06T04:59:54.592223+0000","last_unstale":"2023-10-06T04:59:54.592739+0000","last_undegraded":"2023-10-06T04:59:54.592739+0000","last_fullsized":"2023-10-06T04:59:54.592739+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:22.237368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"107'6","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.919014+0000","last_change":"2023-10-06T04:59:53.914473+0000","last_active":"2023-10-06T04:59:54.919014+0000","last_peered":"2023-10-06T04:59:54.919014+0000","last_clean":"2023-10-06T04:59:54.919014+0000","last_became_active":"2023-10-06T04:59:53.914101+0000","last_became_peered":"2023-10-06T04:59:53.914101+0000","last_unstale":"2023-10-06T04:59:54.919014+0000","last_undegraded":"2023-10-06T04:59:54.919014+0000","last_fullsized":"2023-10-06T04:59:54.919014+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:17.623242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.902685+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T04:59:53.902685+0000","last_peered":"2023-10-06T04:59:53.902685+0000","last_clean":"2023-10-06T04:59:53.902685+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T04:59:53.902685+0000","last_undegraded":"2023-10-06T04:59:53.902685+0000","last_fullsized":"2023-10-06T04:59:53.902685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.593318+0000","last_change":"2023-10-06T04:59:54.593318+0000","last_active":"2023-10-06T04:59:54.593318+0000","last_peered":"2023-10-06T04:59:54.593318+0000","last_clean":"2023-10-06T04:59:54.593318+0000","last_became_active":"2023-10-06T04:59:54.592986+0000","last_became_peered":"2023-10-06T04:59:54.592986+0000","last_unstale":"2023-10-06T04:59:54.593318+0000","last_undegraded":"2023-10-06T04:59:54.593318+0000","last_fullsized":"2023-10-06T04:59:54.593318+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.526790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"107'2","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.915807+0000","last_change":"2023-10-06T04:59:53.916261+0000","last_active":"2023-10-06T04:59:54.915807+0000","last_peered":"2023-10-06T04:59:54.915807+0000","last_clean":"2023-10-06T04:59:54.915807+0000","last_became_active":"2023-10-06T04:59:53.915963+0000","last_became_peered":"2023-10-06T04:59:53.915963+0000","last_unstale":"2023-10-06T04:59:54.915807+0000","last_undegraded":"2023-10-06T04:59:54.915807+0000","last_fullsized":"2023-10-06T04:59:54.915807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:39.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":86,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.901709+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T04:59:53.901709+0000","last_peered":"2023-10-06T04:59:53.901709+0000","last_clean":"2023-10-06T04:59:53.901709+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T04:59:53.901709+0000","last_undegraded":"2023-10-06T04:59:53.901709+0000","last_fullsized":"2023-10-06T04:59:53.901709+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":233,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916693+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T04:59:53.916693+0000","last_peered":"2023-10-06T04:59:53.916693+0000","last_clean":"2023-10-06T04:59:53.916693+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T04:59:53.916693+0000","last_undegraded":"2023-10-06T04:59:53.916693+0000","last_fullsized":"2023-10-06T04:59:53.916693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594581+0000","last_change":"2023-10-06T04:59:54.594581+0000","last_active":"2023-10-06T04:59:54.594581+0000","last_peered":"2023-10-06T04:59:54.594581+0000","last_clean":"2023-10-06T04:59:54.594581+0000","last_became_active":"2023-10-06T04:59:54.594292+0000","last_became_peered":"2023-10-06T04:59:54.594292+0000","last_unstale":"2023-10-06T04:59:54.594581+0000","last_undegraded":"2023-10-06T04:59:54.594581+0000","last_fullsized":"2023-10-06T04:59:54.594581+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:35.567647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.918408+0000","last_change":"2023-10-06T04:59:53.914899+0000","last_active":"2023-10-06T04:59:53.918408+0000","last_peered":"2023-10-06T04:59:53.918408+0000","last_clean":"2023-10-06T04:59:53.918408+0000","last_became_active":"2023-10-06T04:59:53.914638+0000","last_became_peered":"2023-10-06T04:59:53.914638+0000","last_unstale":"2023-10-06T04:59:53.918408+0000","last_undegraded":"2023-10-06T04:59:53.918408+0000","last_fullsized":"2023-10-06T04:59:53.918408+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:41.337292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.917142+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T04:59:53.917142+0000","last_peered":"2023-10-06T04:59:53.917142+0000","last_clean":"2023-10-06T04:59:53.917142+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T04:59:53.917142+0000","last_undegraded":"2023-10-06T04:59:53.917142+0000","last_fullsized":"2023-10-06T04:59:53.917142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596669+0000","last_change":"2023-10-06T04:59:54.596669+0000","last_active":"2023-10-06T04:59:54.596669+0000","last_peered":"2023-10-06T04:59:54.596669+0000","last_clean":"2023-10-06T04:59:54.596669+0000","last_became_active":"2023-10-06T04:59:54.596119+0000","last_became_peered":"2023-10-06T04:59:54.596119+0000","last_unstale":"2023-10-06T04:59:54.596669+0000","last_undegraded":"2023-10-06T04:59:54.596669+0000","last_fullsized":"2023-10-06T04:59:54.596669+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:36.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"107'8","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.917227+0000","last_change":"2023-10-06T04:59:53.916243+0000","last_active":"2023-10-06T04:59:54.917227+0000","last_peered":"2023-10-06T04:59:54.917227+0000","last_clean":"2023-10-06T04:59:54.917227+0000","last_became_active":"2023-10-06T04:59:53.915976+0000","last_became_peered":"2023-10-06T04:59:53.915976+0000","last_unstale":"2023-10-06T04:59:54.917227+0000","last_undegraded":"2023-10-06T04:59:54.917227+0000","last_fullsized":"2023-10-06T04:59:54.917227+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:58.766719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.916782+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T04:59:53.916782+0000","last_peered":"2023-10-06T04:59:53.916782+0000","last_clean":"2023-10-06T04:59:53.916782+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T04:59:53.916782+0000","last_undegraded":"2023-10-06T04:59:53.916782+0000","last_fullsized":"2023-10-06T04:59:53.916782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.581157+0000","last_change":"2023-10-06T04:59:54.581157+0000","last_active":"2023-10-06T04:59:54.581157+0000","last_peered":"2023-10-06T04:59:54.581157+0000","last_clean":"2023-10-06T04:59:54.581157+0000","last_became_active":"2023-10-06T04:59:54.580882+0000","last_became_peered":"2023-10-06T04:59:54.580882+0000","last_unstale":"2023-10-06T04:59:54.581157+0000","last_undegraded":"2023-10-06T04:59:54.581157+0000","last_fullsized":"2023-10-06T04:59:54.581157+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:12.640313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.920180+0000","last_change":"2023-10-06T04:59:53.916708+0000","last_active":"2023-10-06T04:59:53.920180+0000","last_peered":"2023-10-06T04:59:53.920180+0000","last_clean":"2023-10-06T04:59:53.920180+0000","last_became_active":"2023-10-06T04:59:53.916041+0000","last_became_peered":"2023-10-06T04:59:53.916041+0000","last_unstale":"2023-10-06T04:59:53.920180+0000","last_undegraded":"2023-10-06T04:59:53.920180+0000","last_fullsized":"2023-10-06T04:59:53.920180+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:39.186347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.906613+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T04:59:53.906613+0000","last_peered":"2023-10-06T04:59:53.906613+0000","last_clean":"2023-10-06T04:59:53.906613+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T04:59:53.906613+0000","last_undegraded":"2023-10-06T04:59:53.906613+0000","last_fullsized":"2023-10-06T04:59:53.906613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.594645+0000","last_change":"2023-10-06T04:59:54.594645+0000","last_active":"2023-10-06T04:59:54.594645+0000","last_peered":"2023-10-06T04:59:54.594645+0000","last_clean":"2023-10-06T04:59:54.594645+0000","last_became_active":"2023-10-06T04:59:54.594286+0000","last_became_peered":"2023-10-06T04:59:54.594286+0000","last_unstale":"2023-10-06T04:59:54.594645+0000","last_undegraded":"2023-10-06T04:59:54.594645+0000","last_fullsized":"2023-10-06T04:59:54.594645+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:35.758881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.927240+0000","last_change":"2023-10-06T04:59:53.920334+0000","last_active":"2023-10-06T04:59:53.927240+0000","last_peered":"2023-10-06T04:59:53.927240+0000","last_clean":"2023-10-06T04:59:53.927240+0000","last_became_active":"2023-10-06T04:59:53.920021+0000","last_became_peered":"2023-10-06T04:59:53.920021+0000","last_unstale":"2023-10-06T04:59:53.927240+0000","last_undegraded":"2023-10-06T04:59:53.927240+0000","last_fullsized":"2023-10-06T04:59:53.927240+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:29.175360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.909907+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T04:59:53.909907+0000","last_peered":"2023-10-06T04:59:53.909907+0000","last_clean":"2023-10-06T04:59:53.909907+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T04:59:53.909907+0000","last_undegraded":"2023-10-06T04:59:53.909907+0000","last_fullsized":"2023-10-06T04:59:53.909907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T04:59:54.596684+0000","last_change":"2023-10-06T04:59:54.596684+0000","last_active":"2023-10-06T04:59:54.596684+0000","last_peered":"2023-10-06T04:59:54.596684+0000","last_clean":"2023-10-06T04:59:54.596684+0000","last_became_active":"2023-10-06T04:59:54.596397+0000","last_became_peered":"2023-10-06T04:59:54.596397+0000","last_unstale":"2023-10-06T04:59:54.596684+0000","last_undegraded":"2023-10-06T04:59:54.596684+0000","last_fullsized":"2023-10-06T04:59:54.596684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:53.895131+0000","last_clean_scrub_stamp":"2023-10-06T04:59:53.895131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:54.707605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.922992+0000","last_change":"2023-10-06T04:59:53.918963+0000","last_active":"2023-10-06T04:59:53.922992+0000","last_peered":"2023-10-06T04:59:53.922992+0000","last_clean":"2023-10-06T04:59:53.922992+0000","last_became_active":"2023-10-06T04:59:53.918643+0000","last_became_peered":"2023-10-06T04:59:53.918643+0000","last_unstale":"2023-10-06T04:59:53.922992+0000","last_undegraded":"2023-10-06T04:59:53.922992+0000","last_fullsized":"2023-10-06T04:59:53.922992+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:59:52.889390+0000","last_clean_scrub_stamp":"2023-10-06T04:59:52.889390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:52.729068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T04:59:53.910367+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T04:59:53.910367+0000","last_peered":"2023-10-06T04:59:53.910367+0000","last_clean":"2023-10-06T04:59:53.910367+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T04:59:53.910367+0000","last_undegraded":"2023-10-06T04:59:53.910367+0000","last_fullsized":"2023-10-06T04:59:53.910367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307372,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064468,"statfs":{"total":96636764160,"available":96322015232,"internally_reserved":0,"allocated":1937408,"data_stored":1191684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":0.71699999999999997},"min":{"1min":0.35499999999999998,"5min":0.28999999999999998,"15min":0.22800000000000001},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":7.1779999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.753},"min":{"1min":0.442,"5min":0.38,"15min":0.311},"max":{"1min":0.79800000000000004,"5min":0.90500000000000003,"15min":7.1879999999999997},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.69299999999999995,"5min":0.747,"15min":1.073},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75800000000000001,"5min":0.81799999999999995,"15min":1.1379999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73799999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.82599999999999996,"5min":0.90000000000000002,"15min":1.111},"last":0.89300000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.58299999999999996},"min":{"1min":0.36799999999999999,"5min":0.36699999999999999,"15min":0.33200000000000002},"max":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":1.0880000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78000000000000003,"5min":0.84699999999999998,"15min":1.1539999999999999},"last":0.85799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45900000000000002,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.73299999999999998,"5min":0.80000000000000004,"15min":0.85999999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.53900000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81399999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.63},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.61899999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.214},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.58799999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1335296,"data_stored":602475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59699999999999998,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58999999999999997,"15min":0.54400000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.86399999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.54100000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.83099999999999996,"5min":1.234,"15min":1.234},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84599999999999997,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.58399999999999996},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.755},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.60399999999999998,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.76500000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.78200000000000003,"5min":3.669,"15min":3.669},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.88,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83199999999999996,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.379,"5min":0.375,"15min":0.35599999999999998},"max":{"1min":0.85899999999999999,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.80500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307360,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064480,"statfs":{"total":96636764160,"available":96322027520,"internally_reserved":0,"allocated":1925120,"data_stored":1190958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.246,"5min":0.216,"15min":0.216},"max":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38300000000000001,"15min":0.38100000000000001},"min":{"1min":0.248,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32000000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39600000000000002,"15min":0.40200000000000002},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.57299999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.245}]},{"osd":3,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42099999999999999,"15min":0.42599999999999999},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.70999999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.877,"5min":0.90000000000000002,"15min":1.0780000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63,"15min":0.63100000000000001},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.33700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.872,"5min":0.998,"15min":0.998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.65400000000000003},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84099999999999997,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.50600000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0900000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65700000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.66700000000000004},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.40799999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.026},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67900000000000005,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":1.022,"15min":1.0429999999999999},"last":0.57199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306784,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065056,"statfs":{"total":96636764160,"available":96322617344,"internally_reserved":0,"allocated":1335296,"data_stored":601943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.24099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.38300000000000001,"15min":0.36199999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.40000000000000002},"min":{"1min":0.27400000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56499999999999995,"5min":0.628,"15min":0.628},"last":0.433},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42199999999999999,"15min":0.38500000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.55900000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39800000000000002,"15min":0.371},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53000000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45500000000000002,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63400000000000001,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79200000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":1.022,"15min":1.022},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81000000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84699999999999998,"5min":0.93600000000000005,"15min":0.97299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86499999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67700000000000005,"15min":0.65400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78600000000000003,"5min":0.995,"15min":0.995},"last":0.73799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302684,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069156,"statfs":{"total":96636764160,"available":96326815744,"internally_reserved":0,"allocated":1331200,"data_stored":601921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.252,"5min":0.19500000000000001,"15min":0.186},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":1.048},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42299999999999999,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.56399999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.30399999999999999,"5min":0.23100000000000001,"15min":0.216},"max":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.253},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.36399999999999999},"min":{"1min":0.27800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.623,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.376}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.373,"15min":0.35099999999999998},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44700000000000001,"15min":0.45200000000000001},"min":{"1min":0.313,"5min":0.29599999999999999,"15min":0.27400000000000002},"max":{"1min":0.59899999999999998,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.35699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.45300000000000001,"5min":0.35799999999999998,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.88600000000000001,"15min":1.246},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78100000000000003,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.38400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83799999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40300000000000002},"max":{"1min":0.752,"5min":0.89400000000000002,"15min":1.2649999999999999},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.81899999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64000000000000001,"15min":0.627},"min":{"1min":0.36899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80300000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.65700000000000003,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.79100000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306796,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065044,"statfs":{"total":96636764160,"available":96322605056,"internally_reserved":0,"allocated":1347584,"data_stored":603491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38800000000000001,"15min":0.40100000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.47899999999999998,"5min":2.008,"15min":2.008},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.41299999999999998,"15min":0.54700000000000004},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.26200000000000001},"max":{"1min":0.57299999999999995,"5min":2.1480000000000001,"15min":6.4370000000000003},"last":0.372}]},{"osd":1,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.372,"15min":0.35899999999999999},"min":{"1min":0.23000000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.67000000000000004,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.45500000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.57099999999999995,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42299999999999999,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.61799999999999999,"5min":1.379,"15min":5.069},"last":0.307},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.45500000000000002,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.628,"5min":2.246,"15min":2.246},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.73699999999999999,"15min":0.77800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.37},"max":{"1min":0.81899999999999995,"5min":14.646000000000001,"15min":14.646000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.72999999999999998,"15min":0.73099999999999998},"min":{"1min":0.373,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":14.606999999999999,"15min":14.606999999999999},"last":0.41199999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.30599999999999999},"max":{"1min":0.84299999999999997,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.75700000000000001},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81499999999999995,"5min":1.9219999999999999,"15min":6.4009999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65400000000000003,"15min":0.748},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":1.4079999999999999,"15min":5.0309999999999997},"last":0.5},{"interface":"front","average":{"1min":0.623,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.77600000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.87,"5min":1.964,"15min":6.367},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.73299999999999998},"min":{"1min":0.48699999999999999,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":1.3,"15min":5},"last":0.439}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306780,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065060,"statfs":{"total":96636764160,"available":96322621440,"internally_reserved":0,"allocated":1331200,"data_stored":601921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.313,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.54700000000000004,"15min":0.75700000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.67300000000000004,"5min":0.82299999999999995,"15min":5.4569999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.78500000000000003},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65400000000000003,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57899999999999996,"15min":0.79300000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.308},"max":{"1min":0.72399999999999998,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.68300000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.65300000000000002},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.69899999999999995,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.54300000000000004,"15min":0.66700000000000004},"min":{"1min":0.38600000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60399999999999998,"5min":0.80400000000000005,"15min":5.415},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.65700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.70399999999999996,"5min":0.83199999999999996,"15min":4.202},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61499999999999999,"15min":0.71299999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.751,"5min":1.044,"15min":4.2130000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.497},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60199999999999998,"5min":0.81799999999999995,"15min":1.1279999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.745,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51400000000000001,"15min":0.54200000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.66100000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54000000000000004,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.68200000000000005,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.47899999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.66700000000000004,"5min":0.86199999999999999,"15min":0.872},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59199999999999997,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.313},"max":{"1min":0.75800000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.501}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306784,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065056,"statfs":{"total":96636764160,"available":96322617344,"internally_reserved":0,"allocated":1335296,"data_stored":600728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52800000000000002,"15min":0.66100000000000003},"min":{"1min":0.36799999999999999,"5min":0.26300000000000001,"15min":0.20499999999999999},"max":{"1min":0.73299999999999998,"5min":0.79600000000000004,"15min":4.9039999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.61799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.69199999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":6.7949999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.65700000000000003},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":4.8399999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.73899999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85099999999999998,"5min":0.85599999999999998,"15min":4.9199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.64500000000000002},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.26200000000000001},"max":{"1min":0.71199999999999997,"5min":0.83899999999999997,"15min":4.8170000000000002},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.90200000000000002},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.30199999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.89900000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":9.7539999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.61399999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.56200000000000006},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.74399999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.54700000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67500000000000004,"5min":0.746,"15min":0.746},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.80100000000000005},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59099999999999997,"15min":0.63100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76200000000000001,"5min":0.79200000000000004,"15min":0.92900000000000005},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:59:59.461 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:59:59.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:59:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.587+0000 7f61debd4700 1 Processor -- start 2023-10-06T04:59:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.587+0000 7f61debd4700 1 -- start start 2023-10-06T04:59:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61debd4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61d8150690 0x7f61d8150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61debd4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61d815d540 0x7f61d80af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61debd4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d815a290 con 0x7f61d80cba50 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61debd4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80d0020 con 0x7f61d8150690 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61debd4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d8159f40 con 0x7f61d815d540 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61dc970700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61d8150690 0x7f61d8150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61d7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f61d80cba50 0x7f61d815a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50046/0 (socket says 172.21.15.3:50046) 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61d7fff700 1 -- 172.21.15.3:0/1873141514 learned_addr learned my addr 172.21.15.3:0/1873141514 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.588+0000 7f61dc970700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61d8150690 0x7f61d8150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48004/0 (socket says 172.21.15.3:48004) 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61dd171700 1 --2- 172.21.15.3:0/1873141514 >> v2:172.21.15.153:3300/0 conn(0x7f61d815d540 0x7f61d80af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61dc970700 1 -- 172.21.15.3:0/1873141514 >> v2:172.21.15.153:3300/0 conn(0x7f61d815d540 msgr2=0x7f61d80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:59:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61dc970700 1 --2- 172.21.15.3:0/1873141514 >> v2:172.21.15.153:3300/0 conn(0x7f61d815d540 0x7f61d80af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61dc970700 1 -- 172.21.15.3:0/1873141514 >> v1:172.21.15.3:6789/0 conn(0x7f61d80cba50 legacy=0x7f61d815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61d77fe700 1 -- 172.21.15.3:0/1873141514 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251039793 0 0) 0x7f61d815a290 con 0x7f61d80cba50 2023-10-06T04:59:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61dc970700 1 -- 172.21.15.3:0/1873141514 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d815dcb0 con 0x7f61d8150690 2023-10-06T04:59:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61dd171700 1 --2- 172.21.15.3:0/1873141514 >> v2:172.21.15.153:3300/0 conn(0x7f61d815d540 0x7f61d80af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:59:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61dc970700 1 --2- 172.21.15.3:0/1873141514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d8150690 0x7f61d8150a60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f61d815a200 tx=0x7f61d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bded6452e0f84900 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61d77fe700 1 -- 172.21.15.3:0/1873141514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000b6b0 con 0x7f61d8150690 2023-10-06T04:59:59.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61d77fe700 1 -- 172.21.15.3:0/1873141514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d000b810 con 0x7f61d8150690 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61d77fe700 1 -- 172.21.15.3:0/1873141514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000bbb0 con 0x7f61d8150690 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61debd4700 1 -- 172.21.15.3:0/1873141514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d8150690 msgr2=0x7f61d8150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61debd4700 1 --2- 172.21.15.3:0/1873141514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d8150690 0x7f61d8150a60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f61d815a200 tx=0x7f61d00091a0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61debd4700 1 -- 172.21.15.3:0/1873141514 shutdown_connections 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61debd4700 1 --2- 172.21.15.3:0/1873141514 >> v2:172.21.15.153:3300/0 conn(0x7f61d815d540 0x7f61d80af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.589+0000 7f61debd4700 1 --2- 172.21.15.3:0/1873141514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d8150690 0x7f61d8150a60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.590+0000 7f61debd4700 1 -- 172.21.15.3:0/1873141514 >> 172.21.15.3:0/1873141514 conn(0x7f61d8070310 msgr2=0x7f61d8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.590+0000 7f61debd4700 1 -- 172.21.15.3:0/1873141514 shutdown_connections 2023-10-06T04:59:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.590+0000 7f61debd4700 1 -- 172.21.15.3:0/1873141514 wait complete. 2023-10-06T04:59:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.591+0000 7f61debd4700 1 Processor -- start 2023-10-06T04:59:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.591+0000 7f61debd4700 1 -- start start 2023-10-06T04:59:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d80cba50 0x7f61d8151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d8150690 0x7f61d814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61d815d540 0x7f61d816a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d8150040 con 0x7f61d80cba50 2023-10-06T04:59:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d816ab00 con 0x7f61d815d540 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80b5ec0 con 0x7f61d8150690 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d80cba50 0x7f61d8151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d80cba50 0x7f61d8151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48006/0 (socket says 172.21.15.3:48006) 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 -- 172.21.15.3:0/3536495664 learned_addr learned my addr 172.21.15.3:0/3536495664 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 -- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d8150690 msgr2=0x7f61d814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d8150690 0x7f61d814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 -- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61d815d540 msgr2=0x7f61d816a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61d815d540 0x7f61d816a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 -- 172.21.15.3:0/3536495664 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d0009420 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61dc970700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d80cba50 0x7f61d8151c20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f61d80c71f0 tx=0x7f61d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61d4ff9700 1 -- 172.21.15.3:0/3536495664 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011420 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61d4ff9700 1 -- 172.21.15.3:0/3536495664 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0011580 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d80b4680 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61d4ff9700 1 -- 172.21.15.3:0/3536495664 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011920 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.592+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d80b3d50 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.593+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d804c380 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.594+0000 7f61d4ff9700 1 -- 172.21.15.3:0/3536495664 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61d0011cd0 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.594+0000 7f61d4ff9700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61b8066f30 0x7f61b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.594+0000 7f61d4ff9700 1 -- 172.21.15.3:0/3536495664 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f61d0086240 con 0x7f61d80cba50 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.595+0000 7f61d7fff700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61b8066f30 0x7f61b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:59:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.595+0000 7f61d7fff700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61b8066f30 0x7f61b80693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f61d8159560 tx=0x7f61c80016c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:59:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.599+0000 7f61d4ff9700 1 -- 172.21.15.3:0/3536495664 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61d00533d0 con 0x7f61d80cba50 2023-10-06T04:59:59.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.868+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f61d815df90 con 0x7f61d80cba50 2023-10-06T04:59:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.872+0000 7f61d4ff9700 1 -- 172.21.15.3:0/3536495664 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f61d00569f0 con 0x7f61d80cba50 2023-10-06T04:59:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61b8066f30 msgr2=0x7f61b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61b8066f30 0x7f61b80693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f61d8159560 tx=0x7f61c80016c0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d80cba50 msgr2=0x7f61d8151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:59:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d80cba50 0x7f61d8151c20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f61d80c71f0 tx=0x7f61d0017000 comp rx=0 tx=0).stop 2023-10-06T04:59:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 shutdown_connections 2023-10-06T04:59:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f61b8066f30 0x7f61b80693e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61d815d540 0x7f61d816a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61d80cba50 0x7f61d8151c20 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 --2- 172.21.15.3:0/3536495664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d8150690 0x7f61d814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:59:59.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 >> 172.21.15.3:0/3536495664 conn(0x7f61d8070310 msgr2=0x7f61d8157b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:59:59.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 shutdown_connections 2023-10-06T04:59:59.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T04:59:59.874+0000 7f61debd4700 1 -- 172.21.15.3:0/3536495664 wait complete. 2023-10-06T04:59:59.877 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T04:59:59.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:00.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.005+0000 7f066d013700 1 Processor -- start 2023-10-06T05:00:00.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.005+0000 7f066d013700 1 -- start start 2023-10-06T05:00:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066d013700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 0x7f06680cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066d013700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f066806d480 0x7f066815a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066d013700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066815e4c0 con 0x7f06680cde10 2023-10-06T05:00:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066d013700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066815d8b0 con 0x7f066806d480 2023-10-06T05:00:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066d013700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066815db90 con 0x7f06680b39d0 2023-10-06T05:00:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f06680cde10 0x7f06681571d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50060/0 (socket says 172.21.15.3:50060) 2023-10-06T05:00:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066659c700 1 -- 172.21.15.3:0/456187579 learned_addr learned my addr 172.21.15.3:0/456187579 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f066759e700 1 --2- 172.21.15.3:0/456187579 >> v2:172.21.15.153:3300/0 conn(0x7f066806d480 0x7f066815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f0666d9d700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 0x7f06680cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431048407 0 0) 0x7f066815e4c0 con 0x7f06680cde10 2023-10-06T05:00:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.006+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0654003570 con 0x7f06680cde10 2023-10-06T05:00:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660802082 0 0) 0x7f0654003570 con 0x7f06680cde10 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 msgr2=0x7f06680cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 0x7f06680cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 >> v2:172.21.15.153:3300/0 conn(0x7f066806d480 msgr2=0x7f066815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 --2- 172.21.15.3:0/456187579 >> v2:172.21.15.153:3300/0 conn(0x7f066806d480 0x7f066815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066815de70 con 0x7f06680cde10 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f0650001470 con 0x7f06680cde10 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 >> v1:172.21.15.3:6789/0 conn(0x7f06680cde10 legacy=0x7f06681571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f0654003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654004810 0x7f0654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 0x7f06540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066815d8b0 con 0x7f0654003a60 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066815db90 con 0x7f0654004810 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065400ba60 con 0x7f06680b39d0 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0666d9d700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f0654003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f066659c700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654004810 0x7f0654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f066759e700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 0x7f06540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0666d9d700 1 -- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 msgr2=0x7f06540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0666d9d700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680b39d0 0x7f06540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0666d9d700 1 -- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654004810 msgr2=0x7f0654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0666d9d700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654004810 0x7f0654004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0666d9d700 1 -- 172.21.15.3:0/456187579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654003570 con 0x7f0654003a60 2023-10-06T05:00:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f066659c700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654004810 0x7f0654004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.007+0000 7f0666d9d700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f0654003e70 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0660009c30 tx=0x7f06600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda55f7111a7cd18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000e710 con 0x7f0654003a60 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000e870 con 0x7f0654003a60 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f0665d9b700 1 -- 172.21.15.3:0/456187579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000ec30 con 0x7f0654003a60 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f066d013700 1 -- 172.21.15.3:0/456187579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 msgr2=0x7f0654003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f066d013700 1 --2- 172.21.15.3:0/456187579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f0654003e70 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0660009c30 tx=0x7f06600091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f066759e700 1 -- 172.21.15.3:0/456187579 reap_dead start 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f066d013700 1 -- 172.21.15.3:0/456187579 shutdown_connections 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f066d013700 1 -- 172.21.15.3:0/456187579 >> 172.21.15.3:0/456187579 conn(0x7f06680ce2c0 msgr2=0x7f06680ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f066d013700 1 -- 172.21.15.3:0/456187579 shutdown_connections 2023-10-06T05:00:00.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.008+0000 7f066d013700 1 -- 172.21.15.3:0/456187579 wait complete. 2023-10-06T05:00:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.009+0000 7f066d013700 1 Processor -- start 2023-10-06T05:00:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f066d013700 1 -- start start 2023-10-06T05:00:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f066d013700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f066816ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f066d013700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06680ac9d0 0x7f0668171a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f066d013700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680ace40 0x7f0668171f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f066d013700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0668158b20 con 0x7f0654003a60 2023-10-06T05:00:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f066d013700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0668172550 con 0x7f06680ac9d0 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f066d013700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0668172830 con 0x7f06680ace40 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f0666d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f066816ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f0666d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f066816ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48028/0 (socket says 172.21.15.3:48028) 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.010+0000 7f0666d9d700 1 -- 172.21.15.3:0/3841414437 learned_addr learned my addr 172.21.15.3:0/3841414437 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f066659c700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06680ac9d0 0x7f0668171a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f066759e700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680ace40 0x7f0668171f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f0666d9d700 1 -- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680ace40 msgr2=0x7f0668171f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f0666d9d700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680ace40 0x7f0668171f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f0666d9d700 1 -- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06680ac9d0 msgr2=0x7f0668171a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f0666d9d700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06680ac9d0 0x7f0668171a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f0666d9d700 1 -- 172.21.15.3:0/3841414437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066816e750 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f066759e700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680ace40 0x7f0668171f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f0666d9d700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f066816ef50 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f066000e4b0 tx=0x7f066001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f065effd700 1 -- 172.21.15.3:0/3841414437 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660023070 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f065effd700 1 -- 172.21.15.3:0/3841414437 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000e9e0 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066816ea30 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f065effd700 1 -- 172.21.15.3:0/3841414437 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066001e030 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.011+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0668155820 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.012+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06680644c0 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.012+0000 7f065effd700 1 -- 172.21.15.3:0/3841414437 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0660013470 con 0x7f0654003a60 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.013+0000 7f065effd700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0648066f30 0x7f06480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.013+0000 7f065effd700 1 -- 172.21.15.3:0/3841414437 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0660085cc0 con 0x7f0654003a60 2023-10-06T05:00:00.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.013+0000 7f066659c700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0648066f30 0x7f06480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.014+0000 7f066659c700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0648066f30 0x7f06480693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0650001200 tx=0x7f0650009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.017+0000 7f065effd700 1 -- 172.21.15.3:0/3841414437 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0660052e50 con 0x7f0654003a60 2023-10-06T05:00:00.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.256+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0668071680 con 0x7f0654003a60 2023-10-06T05:00:00.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.257+0000 7f065effd700 1 -- 172.21.15.3:0/3841414437 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f0660017020 con 0x7f0654003a60 2023-10-06T05:00:00.258 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:00.259 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T04:59:57.924199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0648066f30 msgr2=0x7f06480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0648066f30 0x7f06480693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0650001200 tx=0x7f0650009040 comp rx=0 tx=0).stop 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 msgr2=0x7f066816ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f066816ef50 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f066000e4b0 tx=0x7f066001a000 comp rx=0 tx=0).stop 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 shutdown_connections 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0648066f30 0x7f06480693e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06680ace40 0x7f0668171f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0654003a60 0x7f066816ef50 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 --2- 172.21.15.3:0/3841414437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06680ac9d0 0x7f0668171a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 >> 172.21.15.3:0/3841414437 conn(0x7f06680ce2c0 msgr2=0x7f0668150650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 shutdown_connections 2023-10-06T05:00:00.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.259+0000 7f066d013700 1 -- 172.21.15.3:0/3841414437 wait complete. 2023-10-06T05:00:00.262 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 69 2023-10-06T05:00:00.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:59:54.313543+0000', 'modified': '2023-10-06T04:59:57.924199+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7527': {'gid': 7527, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/237919975', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6832', 'nonce': 237919975}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 237919975}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:00:00.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:00:00.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:00:00.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.799+0000 7f32cc4ba700 1 Processor -- start 2023-10-06T05:00:00.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.799+0000 7f32cc4ba700 1 -- start start 2023-10-06T05:00:00.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32cc4ba700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 0x7f32c40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32cc4ba700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f32c40b2740 0x7f32c40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32cc4ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c40b0ce0 con 0x7f32c415cb50 2023-10-06T05:00:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32cc4ba700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c40cb820 con 0x7f32c40b2740 2023-10-06T05:00:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32cc4ba700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40b0760 con 0x7f32c40cd950 2023-10-06T05:00:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32ca256700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f32c415cb50 0x7f32c40b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50076/0 (socket says 172.21.15.3:50076) 2023-10-06T05:00:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32ca256700 1 -- 172.21.15.3:0/2583990914 learned_addr learned my addr 172.21.15.3:0/2583990914 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196086396 0 0) 0x7f32c40b0ce0 con 0x7f32c415cb50 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32c9a55700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 0x7f32c40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32caa57700 1 --2- 172.21.15.3:0/2583990914 >> v2:172.21.15.153:3300/0 conn(0x7f32c40b2740 0x7f32c40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.800+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32ac003570 con 0x7f32c415cb50 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2667331860 0 0) 0x7f32ac003570 con 0x7f32c415cb50 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 msgr2=0x7f32c40b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 0x7f32c40b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 >> v2:172.21.15.153:3300/0 conn(0x7f32c40b2740 msgr2=0x7f32c40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 --2- 172.21.15.3:0/2583990914 >> v2:172.21.15.153:3300/0 conn(0x7f32c40b2740 0x7f32c40cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c40b0a00 con 0x7f32c415cb50 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f32bc0015f0 con 0x7f32c415cb50 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 >> v1:172.21.15.3:6789/0 conn(0x7f32c415cb50 legacy=0x7f32c40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ac003a60 0x7f32ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 0x7f32ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ac004d50 0x7f32ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40cb820 con 0x7f32ac004d50 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40b0760 con 0x7f32ac003a60 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac00ba60 con 0x7f32c40cd950 2023-10-06T05:00:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32caa57700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ac004d50 0x7f32ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32ca256700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ac003a60 0x7f32ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32c9a55700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 0x7f32ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32caa57700 1 -- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 msgr2=0x7f32ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32caa57700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32c40cd950 0x7f32ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32caa57700 1 -- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ac003a60 msgr2=0x7f32ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32caa57700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ac003a60 0x7f32ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32caa57700 1 -- 172.21.15.3:0/2583990914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac003570 con 0x7f32ac004d50 2023-10-06T05:00:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.801+0000 7f32caa57700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ac004d50 0x7f32ac0051c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e210 tx=0x7f32b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd73742c57d6307a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400ea60 con 0x7f32ac004d50 2023-10-06T05:00:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b400f040 con 0x7f32ac004d50 2023-10-06T05:00:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32c9254700 1 -- 172.21.15.3:0/2583990914 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400ee10 con 0x7f32ac004d50 2023-10-06T05:00:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32cc4ba700 1 -- 172.21.15.3:0/2583990914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ac004d50 msgr2=0x7f32ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32cc4ba700 1 --2- 172.21.15.3:0/2583990914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ac004d50 0x7f32ac0051c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e210 tx=0x7f32b400b040 comp rx=0 tx=0).stop 2023-10-06T05:00:00.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32caa57700 1 -- 172.21.15.3:0/2583990914 reap_dead start 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32cc4ba700 1 -- 172.21.15.3:0/2583990914 shutdown_connections 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32cc4ba700 1 -- 172.21.15.3:0/2583990914 >> 172.21.15.3:0/2583990914 conn(0x7f32c4070310 msgr2=0x7f32c4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32cc4ba700 1 -- 172.21.15.3:0/2583990914 shutdown_connections 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.802+0000 7f32cc4ba700 1 -- 172.21.15.3:0/2583990914 wait complete. 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.803+0000 7f32cc4ba700 1 Processor -- start 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.803+0000 7f32cc4ba700 1 -- start start 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32cc4ba700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32ac004d50 0x7f32c40c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32cc4ba700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c40cf370 0x7f32c4168760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32cc4ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32c4168ca0 0x7f32c4070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32cc4ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40c6320 con 0x7f32c4168ca0 2023-10-06T05:00:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32cc4ba700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40c6600 con 0x7f32c40cf370 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32cc4ba700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40c68e0 con 0x7f32ac004d50 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32caa57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32c4168ca0 0x7f32c4070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32caa57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32c4168ca0 0x7f32c4070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48044/0 (socket says 172.21.15.3:48044) 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32caa57700 1 -- 172.21.15.3:0/1692063704 learned_addr learned my addr 172.21.15.3:0/1692063704 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32ca256700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32ac004d50 0x7f32c40c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.804+0000 7f32c9a55700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c40cf370 0x7f32c4168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32caa57700 1 -- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32ac004d50 msgr2=0x7f32c40c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32caa57700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32ac004d50 0x7f32c40c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32caa57700 1 -- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c40cf370 msgr2=0x7f32c4168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32caa57700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c40cf370 0x7f32c4168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32caa57700 1 -- 172.21.15.3:0/1692063704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c40c6bc0 con 0x7f32c4168ca0 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32caa57700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32c4168ca0 0x7f32c4070710 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f32b400afd0 tx=0x7f32b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32c27fc700 1 -- 172.21.15.3:0/1692063704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400eee0 con 0x7f32c4168ca0 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32c27fc700 1 -- 172.21.15.3:0/1692063704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b401d630 con 0x7f32c4168ca0 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c40c6010 con 0x7f32c4168ca0 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32c27fc700 1 -- 172.21.15.3:0/1692063704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b401da30 con 0x7f32c4168ca0 2023-10-06T05:00:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.805+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c4070e10 con 0x7f32c4168ca0 2023-10-06T05:00:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.806+0000 7f32c27fc700 1 -- 172.21.15.3:0/1692063704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32b4003040 con 0x7f32c4168ca0 2023-10-06T05:00:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.807+0000 7f32c27fc700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f32b0066f30 0x7f32b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.807+0000 7f32c27fc700 1 -- 172.21.15.3:0/1692063704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32b4085ac0 con 0x7f32c4168ca0 2023-10-06T05:00:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.807+0000 7f32ca256700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f32b0066f30 0x7f32b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.807+0000 7f32ca256700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f32b0066f30 0x7f32b00693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f32bc007a30 tx=0x7f32bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.809+0000 7f32d57b2b80 1 Processor -- start 2023-10-06T05:00:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.809+0000 7f32d57b2b80 1 -- start start 2023-10-06T05:00:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.809+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e32e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.809+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0deb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0debac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0d6bc80 con 0x55f9c0e32a90 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0d42a70 con 0x55f9c0decbd0 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0d40270 con 0x55f9c0dee9a0 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e32e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e32e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48050/0 (socket says 172.21.15.3:48050) 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 -- 172.21.15.3:0/4034477624 learned_addr learned my addr 172.21.15.3:0/4034477624 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32c9a55700 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0deb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32caa57700 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0debac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 -- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 msgr2=0x55f9c0deb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0deb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 -- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 msgr2=0x55f9c0debac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0debac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 -- 172.21.15.3:0/4034477624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9c0bd6270 con 0x55f9c0e32a90 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32caa57700 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0debac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ca256700 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e32e60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f32bc012ce0 tx=0x7f32bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8800bde7b19a7f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ab7fe700 1 -- 172.21.15.3:0/4034477624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc015d30 con 0x55f9c0e32a90 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ab7fe700 1 -- 172.21.15.3:0/4034477624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc015e90 con 0x55f9c0e32a90 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.810+0000 7f32ab7fe700 1 -- 172.21.15.3:0/4034477624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc021620 con 0x55f9c0e32a90 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4034477624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 msgr2=0x55f9c0e32e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e32e60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f32bc012ce0 tx=0x7f32bc018000 comp rx=0 tx=0).stop 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4034477624 shutdown_connections 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0debac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e32e60 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4034477624 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0deb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4034477624 >> 172.21.15.3:0/4034477624 conn(0x55f9c0ded070 msgr2=0x55f9c0ded440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4034477624 shutdown_connections 2023-10-06T05:00:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.811+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4034477624 wait complete. 2023-10-06T05:00:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.812+0000 7f32d57b2b80 1 -- start start 2023-10-06T05:00:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.812+0000 7f32d57b2b80 1 Processor -- start 2023-10-06T05:00:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e2d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0e32a90 0x55f9c0e13970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0d3c060 con 0x55f9c0dee9a0 2023-10-06T05:00:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0ce7890 con 0x55f9c0decbd0 2023-10-06T05:00:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0d02ea0 con 0x55f9c0e32a90 2023-10-06T05:00:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e2d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 10 client.0 _ll_register_callbacks cb 0x55f9c0bd8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:00:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e2d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48066/0 (socket says 172.21.15.3:48066) 2023-10-06T05:00:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 -- 172.21.15.3:0/1464443982 learned_addr learned my addr 172.21.15.3:0/1464443982 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32ca256700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32caa57700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0e32a90 0x55f9c0e13970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 -- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0e32a90 msgr2=0x55f9c0e13970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0e32a90 0x55f9c0e13970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 -- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 msgr2=0x55f9c0e12220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e12220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 -- 172.21.15.3:0/1464443982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c40b0a00 con 0x55f9c0dee9a0 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32caa57700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0e32a90 0x55f9c0e13970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32c9a55700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e2d8e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f32b8015b70 tx=0x7f32b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781202cbeca4f660 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32a8ff9700 1 -- 172.21.15.3:0/1464443982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b801a070 con 0x55f9c0dee9a0 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32a8ff9700 1 -- 172.21.15.3:0/1464443982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32b8020c40 con 0x55f9c0dee9a0 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9c0de0690 con 0x55f9c0dee9a0 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.813+0000 7f32a8ff9700 1 -- 172.21.15.3:0/1464443982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b8020e00 con 0x55f9c0dee9a0 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 10 client.7701 fetch_fsmap learned FSMap version 69 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55f9c0b78fe0 con 0x55f9c0dee9a0 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32a8ff9700 1 -- 172.21.15.3:0/1464443982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9c0de0690 con 0x55f9c0dee9a0 2023-10-06T05:00:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32a8ff9700 1 -- 172.21.15.3:0/1464443982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 69) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f32b8014a60 con 0x55f9c0dee9a0 2023-10-06T05:00:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 10 client.7701 fetch_fsmap finished waiting for FSMap version 69 2023-10-06T05:00:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 10 client.7701 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:00:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 20 client.7701 populate_metadata read hostname 'smithi003' 2023-10-06T05:00:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0e160a0 0x55f9c0e16470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 4 client.7701 mds_command: new command op to 7527 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:00:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 --> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9c0dbd5c0 con 0x55f9c0e160a0 2023-10-06T05:00:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32ca256700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0e160a0 0x55f9c0e16470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32ca256700 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0e160a0 0x55f9c0e16470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71517f2d75a4c83 server_cookie=6af4f43dc3cbc83b in_seq=0 out_seq=0 2023-10-06T05:00:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.814+0000 7f32a8ff9700 10 client.7701 ms_handle_connect on v2:172.21.15.3:6832/237919975 2023-10-06T05:00:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32a8ff9700 1 -- 172.21.15.3:0/1464443982 <== mds.0 v2:172.21.15.3:6832/237919975 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f9c0dbd5c0 con 0x55f9c0e160a0 2023-10-06T05:00:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32a8ff9700 10 client.7701 handle_command_reply: tid=0 2023-10-06T05:00:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 client.7701 shutdown 2023-10-06T05:00:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 msgr2=0x55f9c0e2d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e2d8e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f32b8015b70 tx=0x7f32b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 shutdown_connections 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0e160a0 0x55f9c0e16470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e12220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0e32a90 0x55f9c0e13970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/1464443982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e2d8e0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32a8ff9700 0 client.7701 ms_handle_reset on v2:172.21.15.3:6832/237919975 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 >> 172.21.15.3:0/1464443982 conn(0x55f9c0ded070 msgr2=0x55f9c0de8910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 shutdown_connections 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 1 -- 172.21.15.3:0/1464443982 wait complete. 2023-10-06T05:00:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.815+0000 7f32d57b2b80 20 client.7701 trim_cache size 0 max 16384 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.832+0000 7f32d57b2b80 1 Processor -- start 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.832+0000 7f32d57b2b80 1 -- start start 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0decbd0 0x55f9c0e917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0e32a90 0x55f9c0e4b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0bbee30 con 0x55f9c0dee9a0 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0dcb030 con 0x55f9c0e32a90 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0e147e0 con 0x55f9c0decbd0 2023-10-06T05:00:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32c9a55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32c9a55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e4b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48080/0 (socket says 172.21.15.3:48080) 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32c9a55700 1 -- 172.21.15.3:0/949572617 learned_addr learned my addr 172.21.15.3:0/949572617 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32ca256700 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0decbd0 0x55f9c0e917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32caa57700 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0e32a90 0x55f9c0e4b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.833+0000 7f32c9a55700 1 -- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0decbd0 msgr2=0x55f9c0e917e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32c9a55700 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0decbd0 0x55f9c0e917e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32c9a55700 1 -- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0e32a90 msgr2=0x55f9c0e4b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32c9a55700 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0e32a90 0x55f9c0e4b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32c9a55700 1 -- 172.21.15.3:0/949572617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9c0d0a780 con 0x55f9c0dee9a0 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32ca256700 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0decbd0 0x55f9c0e917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32c9a55700 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e4b000 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f32b80144c0 tx=0x7f32b801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7aabd2df640c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32abfff700 1 -- 172.21.15.3:0/949572617 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8014ae0 con 0x55f9c0dee9a0 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32abfff700 1 -- 172.21.15.3:0/949572617 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b8000bc0 con 0x55f9c0dee9a0 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32abfff700 1 -- 172.21.15.3:0/949572617 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8029470 con 0x55f9c0dee9a0 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32d57b2b80 1 -- 172.21.15.3:0/949572617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 msgr2=0x55f9c0e4b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e4b000 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f32b80144c0 tx=0x7f32b801d000 comp rx=0 tx=0).stop 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32d57b2b80 1 -- 172.21.15.3:0/949572617 shutdown_connections 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0e32a90 0x55f9c0e4b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0decbd0 0x55f9c0e917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/949572617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0dee9a0 0x55f9c0e4b000 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.834+0000 7f32d57b2b80 1 -- 172.21.15.3:0/949572617 >> 172.21.15.3:0/949572617 conn(0x55f9c0ded070 msgr2=0x55f9c0e31280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.835+0000 7f32d57b2b80 1 -- 172.21.15.3:0/949572617 shutdown_connections 2023-10-06T05:00:00.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.835+0000 7f32d57b2b80 1 -- 172.21.15.3:0/949572617 wait complete. 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.835+0000 7f32d57b2b80 1 -- start start 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.835+0000 7f32d57b2b80 1 Processor -- start 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e8e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0e2be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32d57b2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e16c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0e2c770 con 0x55f9c0e32a90 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0e44810 con 0x55f9c0decbd0 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32d57b2b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9c0e290c0 con 0x55f9c0dee9a0 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32d57b2b80 10 client.0 _ll_register_callbacks cb 0x55f9c0bd8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e16c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e16c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48084/0 (socket says 172.21.15.3:48084) 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 -- 172.21.15.3:0/4199103091 learned_addr learned my addr 172.21.15.3:0/4199103091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32ca256700 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e8e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32c9a55700 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0e2be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 -- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 msgr2=0x55f9c0e2be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0e2be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 -- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 msgr2=0x55f9c0e8e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e8e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.836+0000 7f32caa57700 1 -- 172.21.15.3:0/4199103091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9c0b78fe0 con 0x55f9c0e32a90 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32caa57700 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e16c70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f32b4056650 tx=0x7f32b4083de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2dafe61c3aeb62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T05:00:00.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32a8ff9700 1 -- 172.21.15.3:0/4199103091 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40562b0 con 0x55f9c0e32a90 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32a8ff9700 1 -- 172.21.15.3:0/4199103091 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32b4057050 con 0x55f9c0e32a90 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32bc015e90 con 0x55f9c0e32a90 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32a8ff9700 1 -- 172.21.15.3:0/4199103091 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b401de60 con 0x55f9c0e32a90 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 10 client.7707 fetch_fsmap learned FSMap version 69 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55f9c0cf1470 con 0x55f9c0e32a90 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32a8ff9700 1 -- 172.21.15.3:0/4199103091 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc015e90 con 0x55f9c0e32a90 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32a8ff9700 1 -- 172.21.15.3:0/4199103091 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 69) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f32b4072170 con 0x55f9c0e32a90 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 10 client.7707 fetch_fsmap finished waiting for FSMap version 69 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 10 client.7707 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 20 client.7707 populate_metadata read hostname 'smithi003' 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0deb580 0x55f9c0deb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 4 client.7707 mds_command: new command op to 7527 tid=0[{"prefix": "status"}] 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 --> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f9c0cdee00 con 0x55f9c0deb580 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.837+0000 7f32ca256700 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0deb580 0x55f9c0deb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32ca256700 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0deb580 0x55f9c0deb950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641ea22a1dd80d15 server_cookie=e84f3c0d32a5c32f in_seq=0 out_seq=0 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32a8ff9700 10 client.7707 ms_handle_connect on v2:172.21.15.3:6832/237919975 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32a8ff9700 1 -- 172.21.15.3:0/4199103091 <== mds.0 v2:172.21.15.3:6832/237919975 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f9c0cdee00 con 0x55f9c0deb580 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32a8ff9700 10 client.7707 handle_command_reply: tid=0 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 client.7707 shutdown 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 msgr2=0x55f9c0e16c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e16c70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f32b4056650 tx=0x7f32b4083de0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 shutdown_connections 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:6832/237919975,v1:172.21.15.3:6835/237919975] conn(0x55f9c0deb580 0x55f9c0deb950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9c0decbd0 0x55f9c0e8e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32a8ff9700 0 client.7707 ms_handle_reset on v2:172.21.15.3:6832/237919975 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9c0e32a90 0x55f9c0e16c70 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 --2- 172.21.15.3:0/4199103091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55f9c0dee9a0 0x55f9c0e2be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 >> 172.21.15.3:0/4199103091 conn(0x55f9c0ded070 msgr2=0x55f9c0de7400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 shutdown_connections 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 1 -- 172.21.15.3:0/4199103091 wait complete. 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.838+0000 7f32d57b2b80 20 client.7707 trim_cache size 0 max 16384 2023-10-06T05:00:00.850 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7527, 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 5.9339722579999998, 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 69, 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 108, 2023-10-06T05:00:00.851 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 108, 2023-10-06T05:00:00.852 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 12.006818781 2023-10-06T05:00:00.852 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:00:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f32b0066f30 msgr2=0x7f32b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f32b0066f30 0x7f32b00693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f32bc007a30 tx=0x7f32bc009040 comp rx=0 tx=0).stop 2023-10-06T05:00:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32c4168ca0 msgr2=0x7f32c4070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32c4168ca0 0x7f32c4070710 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f32b400afd0 tx=0x7f32b401a000 comp rx=0 tx=0).stop 2023-10-06T05:00:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 shutdown_connections 2023-10-06T05:00:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f32b0066f30 0x7f32b00693e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32c4168ca0 0x7f32c4070710 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32ac004d50 0x7f32c40c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 --2- 172.21.15.3:0/1692063704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c40cf370 0x7f32c4168760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 >> 172.21.15.3:0/1692063704 conn(0x7f32c4070310 msgr2=0x7f32c40b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 shutdown_connections 2023-10-06T05:00:00.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.839+0000 7f32cc4ba700 1 -- 172.21.15.3:0/1692063704 wait complete. 2023-10-06T05:00:00.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:00:00.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.973+0000 7f3eb1ee5700 1 Processor -- start 2023-10-06T05:00:00.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.973+0000 7f3eb1ee5700 1 -- start start 2023-10-06T05:00:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.973+0000 7f3eb1ee5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3eac15c8f0 0x7f3eac15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.973+0000 7f3eb1ee5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15ccc0 0x7f3eac0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.973+0000 7f3eb1ee5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac0cf6c0 con 0x7f3eac0d06a0 2023-10-06T05:00:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.973+0000 7f3eb1ee5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac0cfb50 con 0x7f3eac15c8f0 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.973+0000 7f3eb1ee5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac15e050 con 0x7f3eac15ccc0 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3eac15c8f0 0x7f3eac15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3eac15c8f0 0x7f3eac15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:56638/0 (socket says 172.21.15.3:56638) 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eaaffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15ccc0 0x7f3eac0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 -- 172.21.15.3:0/4048537351 learned_addr learned my addr 172.21.15.3:0/4048537351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eabfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3eac0d06a0 0x7f3eac06d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:39846/0 (socket says 172.21.15.3:39846) 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 -- 172.21.15.3:0/4048537351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15ccc0 msgr2=0x7f3eac0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 --2- 172.21.15.3:0/4048537351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15ccc0 0x7f3eac0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 -- 172.21.15.3:0/4048537351 >> v1:172.21.15.153:6789/0 conn(0x7f3eac0d06a0 legacy=0x7f3eac06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 -- 172.21.15.3:0/4048537351 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0d0af0 con 0x7f3eac15c8f0 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eab7fe700 1 --2- 172.21.15.3:0/4048537351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac15c8f0 0x7f3eac15db10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0099b0 tx=0x7f3e9c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a43717a1a90d981 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eaa7fc700 1 -- 172.21.15.3:0/4048537351 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c009ea0 con 0x7f3eac15c8f0 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eaa7fc700 1 -- 172.21.15.3:0/4048537351 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00b860 con 0x7f3eac15c8f0 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.974+0000 7f3eaa7fc700 1 -- 172.21.15.3:0/4048537351 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00bc00 con 0x7f3eac15c8f0 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/4048537351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac15c8f0 msgr2=0x7f3eac15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/4048537351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac15c8f0 0x7f3eac15db10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0099b0 tx=0x7f3e9c009250 comp rx=0 tx=0).stop 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/4048537351 shutdown_connections 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/4048537351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac15c8f0 0x7f3eac15db10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/4048537351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15ccc0 0x7f3eac0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/4048537351 >> 172.21.15.3:0/4048537351 conn(0x7f3eac070310 msgr2=0x7f3eac070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/4048537351 shutdown_connections 2023-10-06T05:00:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.975+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/4048537351 wait complete. 2023-10-06T05:00:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.976+0000 7f3eb1ee5700 1 Processor -- start 2023-10-06T05:00:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.976+0000 7f3eb1ee5700 1 -- start start 2023-10-06T05:00:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eb1ee5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac0d06a0 0x7f3eac153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eb1ee5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15c8f0 0x7f3eac0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eb1ee5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eac15ccc0 0x7f3eac16a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eb1ee5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac152740 con 0x7f3eac15ccc0 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eb1ee5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac150590 con 0x7f3eac0d06a0 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eb1ee5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0b4230 con 0x7f3eac15c8f0 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eac15ccc0 0x7f3eac16a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eac15ccc0 0x7f3eac16a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48088/0 (socket says 172.21.15.3:48088) 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 -- 172.21.15.3:0/401367911 learned_addr learned my addr 172.21.15.3:0/401367911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eaaffd700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15c8f0 0x7f3eac0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eab7fe700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac0d06a0 0x7f3eac153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 -- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15c8f0 msgr2=0x7f3eac0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15c8f0 0x7f3eac0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 -- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac0d06a0 msgr2=0x7f3eac153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac0d06a0 0x7f3eac153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eabfff700 1 -- 172.21.15.3:0/401367911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0094f0 con 0x7f3eac15ccc0 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.977+0000 7f3eab7fe700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac0d06a0 0x7f3eac153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.978+0000 7f3eabfff700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eac15ccc0 0x7f3eac16a9e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400cd00 tx=0x7f3ea4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.978+0000 7f3e937fe700 1 -- 172.21.15.3:0/401367911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea400ce60 con 0x7f3eac15ccc0 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.978+0000 7f3e937fe700 1 -- 172.21.15.3:0/401367911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4005df0 con 0x7f3eac15ccc0 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.978+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac0b4510 con 0x7f3eac15ccc0 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.978+0000 7f3e937fe700 1 -- 172.21.15.3:0/401367911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea400ab00 con 0x7f3eac15ccc0 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.978+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac0b4ad0 con 0x7f3eac15ccc0 2023-10-06T05:00:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.979+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac071680 con 0x7f3eac15ccc0 2023-10-06T05:00:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.979+0000 7f3e937fe700 1 -- 172.21.15.3:0/401367911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ea40070f0 con 0x7f3eac15ccc0 2023-10-06T05:00:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.980+0000 7f3e937fe700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3e98066f30 0x7f3e980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.980+0000 7f3e937fe700 1 -- 172.21.15.3:0/401367911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ea4084680 con 0x7f3eac15ccc0 2023-10-06T05:00:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.980+0000 7f3eab7fe700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3e98066f30 0x7f3e980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.980+0000 7f3eab7fe700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3e98066f30 0x7f3e980693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0099b0 tx=0x7f3e9c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:00.984+0000 7f3e937fe700 1 -- 172.21.15.3:0/401367911 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ea4051810 con 0x7f3eac15ccc0 2023-10-06T05:00:01.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.220+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3eac0b35a0 con 0x7f3eac15ccc0 2023-10-06T05:00:01.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.220+0000 7f3e937fe700 1 -- 172.21.15.3:0/401367911 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ea4082e60 con 0x7f3eac15ccc0 2023-10-06T05:00:01.222 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:00:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3e98066f30 msgr2=0x7f3e980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3e98066f30 0x7f3e980693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0099b0 tx=0x7f3e9c017000 comp rx=0 tx=0).stop 2023-10-06T05:00:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eac15ccc0 msgr2=0x7f3eac16a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eac15ccc0 0x7f3eac16a9e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400cd00 tx=0x7f3ea4001b10 comp rx=0 tx=0).stop 2023-10-06T05:00:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 shutdown_connections 2023-10-06T05:00:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3e98066f30 0x7f3e980693e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3eac0d06a0 0x7f3eac153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eac15ccc0 0x7f3eac16a9e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 --2- 172.21.15.3:0/401367911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3eac15c8f0 0x7f3eac0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:01.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.222+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 >> 172.21.15.3:0/401367911 conn(0x7f3eac070310 msgr2=0x7f3eac158400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.223+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 shutdown_connections 2023-10-06T05:00:01.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:01.223+0000 7f3eb1ee5700 1 -- 172.21.15.3:0/401367911 wait complete. 2023-10-06T05:00:01.236 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:00:01.236 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:00:01.236 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T05:00:01.236 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:00:01.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:01.237 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:00:01.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:00:01.237 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:00:01.237 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:00:01.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:01.237 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T05:00:01.253 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:00:01.254 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:00:01.254 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:00:01.254 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:01.254 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T05:00:01.254 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:01.255 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:01.255 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:01.255 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T05:00:01.255 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:00:01.255 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:01.255 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T05:00:01.256 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:00:01.256 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42029sec preferred_lft 42029sec 2023-10-06T05:00:01.256 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T05:00:01.256 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:01.256 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:01.257 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:01.257 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T05:00:01.257 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:01.257 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:01.257 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T05:00:01.257 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:00:01.257 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:01.258 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:00:01.258 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:01.258 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T05:00:01.258 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:01.258 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:01.258 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:00:01.258 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T05:00:01.259 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:01.259 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:01.259 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:00:01.259 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T05:00:01.259 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:01.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:01.259 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:00:01.317 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T05:00:01.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:01.318 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T05:00:01.375 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T05:00:01.376 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:00:01.376 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:00:01.376 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:00:01.376 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:00:01.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:01.377 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T05:00:01.377 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T05:00:01.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:00:01.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:00:01.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:00:01.377 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T05:00:01.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:01.478 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T05:00:01.493 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:00:01.493 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:00:01.493 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:00:01.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:01.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T05:00:01.559 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:00:01.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:01.559 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:01.615 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:00:01.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:01.615 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:01.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:00:01.736 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:01.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:01.801 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:01.802 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:01.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:01.802 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:00:01.857 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:00:01.858 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:00:01.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:00:01.926 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:01.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:01.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T05:00:01.982+0000 7fe8a17a1540 -1 init, newargv = 0x56141abde830 newargc=15 2023-10-06T05:00:01.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[123609]: starting ceph client 2023-10-06T05:00:01.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[123609]: starting fuse 2023-10-06T05:00:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:01.993 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:01.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:01.994 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:00:02.049 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T05:00:02.049 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T05:00:02.050 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T05:00:02.051 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T05:00:02.052 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:00:02.052 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:02.052 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T05:00:02.052 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:00:02.172 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.123609.asok 2023-10-06T05:00:02.354 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.123609.asok 2023-10-06T05:00:02.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:02.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.123609.asok status 2023-10-06T05:00:02.509 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:00:02.509 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T05:00:02.509 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "123609", 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T05:00:02.510 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7719, 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 7719 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:02.511 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3364067658 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3364067658 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.7719 192.168.0.1:0/3364067658", 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3364067658", 2023-10-06T05:00:02.512 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T05:00:02.513 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 69, 2023-10-06T05:00:02.513 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 108, 2023-10-06T05:00:02.513 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:00:02.513 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T05:00:02.513 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T05:00:02.513 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:00:02.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:02.519 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T05:00:02.519 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:02.534 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T05:00:02.535 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:02.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:02.535 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:02.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:00:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.759+0000 7fe2eb7d8700 1 Processor -- start 2023-10-06T05:00:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.759+0000 7fe2eb7d8700 1 -- start start 2023-10-06T05:00:02.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2eb7d8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe2e40cd950 0x7fe2e40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:02.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2eb7d8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40b0ce0 con 0x7fe2e415cb50 2023-10-06T05:00:02.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2eb7d8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40cb820 con 0x7fe2e40b2740 2023-10-06T05:00:02.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2eb7d8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e40b0760 con 0x7fe2e40cd950 2023-10-06T05:00:02.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2e8d73700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe2e40cd950 0x7fe2e40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:02.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2e9574700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe2e415cb50 0x7fe2e40b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33642/0 (socket says 172.21.15.3:33642) 2023-10-06T05:00:02.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2e9574700 1 -- 172.21.15.3:0/4187858587 learned_addr learned my addr 172.21.15.3:0/4187858587 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2e8d73700 1 -- 172.21.15.3:0/4187858587 >> v1:172.21.15.153:6789/0 conn(0x7fe2e40b2740 legacy=0x7fe2e40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.760+0000 7fe2e8d73700 1 -- 172.21.15.3:0/4187858587 >> v1:172.21.15.3:6789/0 conn(0x7fe2e415cb50 legacy=0x7fe2e40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2e3fff700 1 -- 172.21.15.3:0/4187858587 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706346662 0 0) 0x7fe2e40b0ce0 con 0x7fe2e415cb50 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2e8d73700 1 -- 172.21.15.3:0/4187858587 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e40b0a00 con 0x7fe2e40cd950 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2e8d73700 1 --2- 172.21.15.3:0/4187858587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e40b2200 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80098e0 tx=0x7fe2d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a4e2de55c2e0ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2e3fff700 1 -- 172.21.15.3:0/4187858587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8009e80 con 0x7fe2e40cd950 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2e3fff700 1 -- 172.21.15.3:0/4187858587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d800b820 con 0x7fe2e40cd950 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2e3fff700 1 -- 172.21.15.3:0/4187858587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800bba0 con 0x7fe2e40cd950 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/4187858587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 msgr2=0x7fe2e40b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/4187858587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e40b2200 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80098e0 tx=0x7fe2d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/4187858587 shutdown_connections 2023-10-06T05:00:02.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/4187858587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e40b2200 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:02.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/4187858587 >> 172.21.15.3:0/4187858587 conn(0x7fe2e4070310 msgr2=0x7fe2e4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:02.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/4187858587 shutdown_connections 2023-10-06T05:00:02.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.761+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/4187858587 wait complete. 2023-10-06T05:00:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.762+0000 7fe2eb7d8700 1 Processor -- start 2023-10-06T05:00:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2eb7d8700 1 -- start start 2023-10-06T05:00:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2eb7d8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2e40b2740 0x7fe2e40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2eb7d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e4158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2eb7d8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2e415cb50 0x7fe2e40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2eb7d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40ce4c0 con 0x7fe2e40cd950 2023-10-06T05:00:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2eb7d8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4158ff0 con 0x7fe2e415cb50 2023-10-06T05:00:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2eb7d8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40ce6f0 con 0x7fe2e40b2740 2023-10-06T05:00:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.763+0000 7fe2e8d73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e4158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e4158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59698/0 (socket says 172.21.15.3:59698) 2023-10-06T05:00:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 -- 172.21.15.3:0/3491192194 learned_addr learned my addr 172.21.15.3:0/3491192194 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e9574700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2e40b2740 0x7fe2e40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e9d75700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2e415cb50 0x7fe2e40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 -- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2e40b2740 msgr2=0x7fe2e40cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2e40b2740 0x7fe2e40cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 -- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2e415cb50 msgr2=0x7fe2e40c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:02.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2e415cb50 0x7fe2e40c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:02.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 -- 172.21.15.3:0/3491192194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d8009420 con 0x7fe2e40cd950 2023-10-06T05:00:02.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e8d73700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e4158ab0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80030c0 tx=0x7fe2d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:02.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e17fa700 1 -- 172.21.15.3:0/3491192194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800bae0 con 0x7fe2e40cd950 2023-10-06T05:00:02.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e17fa700 1 -- 172.21.15.3:0/3491192194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d8020070 con 0x7fe2e40cd950 2023-10-06T05:00:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e40ce9d0 con 0x7fe2e40cd950 2023-10-06T05:00:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2e17fa700 1 -- 172.21.15.3:0/3491192194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d801b6a0 con 0x7fe2e40cd950 2023-10-06T05:00:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.764+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e40cef90 con 0x7fe2e40cd950 2023-10-06T05:00:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.765+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e40b0d00 con 0x7fe2e40cd950 2023-10-06T05:00:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.765+0000 7fe2e17fa700 1 -- 172.21.15.3:0/3491192194 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2d8022030 con 0x7fe2e40cd950 2023-10-06T05:00:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.766+0000 7fe2e17fa700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2d0066f60 0x7fe2d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.766+0000 7fe2e17fa700 1 -- 172.21.15.3:0/3491192194 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe2d8086670 con 0x7fe2e40cd950 2023-10-06T05:00:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.766+0000 7fe2e9574700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2d0066f60 0x7fe2d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:02.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.767+0000 7fe2e9574700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2d0066f60 0x7fe2d0069410 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc002800 tx=0x7fe2dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:02.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:02.770+0000 7fe2e17fa700 1 -- 172.21.15.3:0/3491192194 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2d8053780 con 0x7fe2e40cd950 2023-10-06T05:00:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.011+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2e40b0d00 con 0x7fe2e40cd950 2023-10-06T05:00:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.011+0000 7fe2e17fa700 1 -- 172.21.15.3:0/3491192194 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2d8056da0 con 0x7fe2e40cd950 2023-10-06T05:00:03.013 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:00:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2d0066f60 msgr2=0x7fe2d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2d0066f60 0x7fe2d0069410 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc002800 tx=0x7fe2dc009040 comp rx=0 tx=0).stop 2023-10-06T05:00:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 msgr2=0x7fe2e4158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e4158ab0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80030c0 tx=0x7fe2d8016000 comp rx=0 tx=0).stop 2023-10-06T05:00:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 shutdown_connections 2023-10-06T05:00:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe2d0066f60 0x7fe2d0069410 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2e415cb50 0x7fe2e40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2e40b2740 0x7fe2e40cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:03.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 --2- 172.21.15.3:0/3491192194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2e40cd950 0x7fe2e4158ab0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:03.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.013+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 >> 172.21.15.3:0/3491192194 conn(0x7fe2e4070310 msgr2=0x7fe2e40b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:03.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.014+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 shutdown_connections 2023-10-06T05:00:03.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:03.014+0000 7fe2eb7d8700 1 -- 172.21.15.3:0/3491192194 wait complete. 2023-10-06T05:00:03.027 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:00:03.027 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:00:03.027 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T05:00:03.027 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:00:03.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:03.027 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:00:03.027 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:00:03.028 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:00:03.028 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:00:03.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:03.028 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T05:00:03.046 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:00:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T05:00:03.047 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42030sec preferred_lft 42030sec 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:03.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:03.050 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:00:03.116 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T05:00:03.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:03.118 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T05:00:03.176 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T05:00:03.176 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:00:03.176 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:00:03.176 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:00:03.177 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:00:03.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:03.177 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T05:00:03.177 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T05:00:03.177 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:00:03.177 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:00:03.177 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:00:03.177 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T05:00:03.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:03.282 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T05:00:03.297 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:00:03.297 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:00:03.297 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:00:03.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:03.297 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T05:00:03.362 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T05:00:03.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:03.362 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:03.416 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:00:03.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:03.417 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:03.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T05:00:03.536 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:03.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:03.602 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:03.602 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:03.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:03.603 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T05:00:03.658 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:00:03.659 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T05:00:03.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T05:00:03.724 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:03.724 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:03.782 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T05:00:03.780+0000 7f7041ed7540 -1 init, newargv = 0x561620a4e8b0 newargc=15 2023-10-06T05:00:03.782 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[101936]: starting ceph client 2023-10-06T05:00:03.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[101936]: starting fuse 2023-10-06T05:00:03.791 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:03.792 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:03.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:03.792 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T05:00:03.848 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T05:00:03.848 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:00:03.848 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:03.848 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T05:00:03.849 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T05:00:03.850 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T05:00:03.851 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T05:00:03.851 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:00:03.851 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:03.851 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T05:00:03.851 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T05:00:03.970 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.101936.asok 2023-10-06T05:00:04.153 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.101936.asok 2023-10-06T05:00:04.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:04.153 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.101936.asok status 2023-10-06T05:00:04.305 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T05:00:04.305 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T05:00:04.305 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:00:04.305 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:00:04.305 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T05:00:04.305 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "101936", 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 6449, 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T05:00:04.306 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 6449 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 1578962507 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T05:00:04.307 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 1578962507 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.6449 192.168.0.1:0/1578962507", 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/1578962507", 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 69, 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 108, 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:00:04.308 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T05:00:04.309 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T05:00:04.309 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T05:00:04.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:04.315 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T05:00:04.315 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:04.331 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T05:00:04.331 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:04.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:04.331 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:04.395 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-10-06T05:00:04.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.530+0000 7f8291ed1700 1 Processor -- start 2023-10-06T05:00:04.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.530+0000 7f8291ed1700 1 -- start start 2023-10-06T05:00:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f8291ed1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f828c068190 0x7f828c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f8291ed1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c15d130 con 0x7f828c067dc0 2023-10-06T05:00:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f8291ed1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0691c0 con 0x7f828c15cd40 2023-10-06T05:00:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f8291ed1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c0694a0 con 0x7f828c068190 2023-10-06T05:00:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f828affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f828c068190 0x7f828c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f828b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f828c067dc0 0x7f828c0b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33656/0 (socket says 172.21.15.3:33656) 2023-10-06T05:00:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f828b7fe700 1 -- 172.21.15.3:0/3647910931 learned_addr learned my addr 172.21.15.3:0/3647910931 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f828affd700 1 -- 172.21.15.3:0/3647910931 >> v1:172.21.15.153:6789/0 conn(0x7f828c15cd40 legacy=0x7f828c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f828affd700 1 -- 172.21.15.3:0/3647910931 >> v1:172.21.15.3:6789/0 conn(0x7f828c067dc0 legacy=0x7f828c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f828affd700 1 -- 172.21.15.3:0/3647910931 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c069780 con 0x7f828c068190 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.531+0000 7f828a7fc700 1 -- 172.21.15.3:0/3647910931 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695382930 0 0) 0x7f828c15d130 con 0x7f828c067dc0 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f828affd700 1 --2- 172.21.15.3:0/3647910931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c15c800 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f82800098e0 tx=0x7f82800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41e5608d6cfea5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f828a7fc700 1 -- 172.21.15.3:0/3647910931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280009e80 con 0x7f828c068190 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f828a7fc700 1 -- 172.21.15.3:0/3647910931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828000b860 con 0x7f828c068190 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f828a7fc700 1 -- 172.21.15.3:0/3647910931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000bbe0 con 0x7f828c068190 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f8291ed1700 1 -- 172.21.15.3:0/3647910931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 msgr2=0x7f828c15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f8291ed1700 1 --2- 172.21.15.3:0/3647910931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c15c800 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f82800098e0 tx=0x7f82800091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f8291ed1700 1 -- 172.21.15.3:0/3647910931 shutdown_connections 2023-10-06T05:00:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f8291ed1700 1 --2- 172.21.15.3:0/3647910931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c15c800 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f8291ed1700 1 -- 172.21.15.3:0/3647910931 >> 172.21.15.3:0/3647910931 conn(0x7f828c0ce270 msgr2=0x7f828c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f8291ed1700 1 -- 172.21.15.3:0/3647910931 shutdown_connections 2023-10-06T05:00:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.532+0000 7f8291ed1700 1 -- 172.21.15.3:0/3647910931 wait complete. 2023-10-06T05:00:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.533+0000 7f8291ed1700 1 Processor -- start 2023-10-06T05:00:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f8291ed1700 1 -- start start 2023-10-06T05:00:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f8291ed1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f828c067dc0 0x7f828c13c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f8291ed1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c169f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f8291ed1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828c15cd40 0x7f828c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f8291ed1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c16a4a0 con 0x7f828c068190 2023-10-06T05:00:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f8291ed1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0c7a90 con 0x7f828c15cd40 2023-10-06T05:00:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f8291ed1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c16af30 con 0x7f828c067dc0 2023-10-06T05:00:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f828affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f828affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c169f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59706/0 (socket says 172.21.15.3:59706) 2023-10-06T05:00:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f828affd700 1 -- 172.21.15.3:0/4087639147 learned_addr learned my addr 172.21.15.3:0/4087639147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f828bfff700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828c15cd40 0x7f828c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.534+0000 7f828b7fe700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f828c067dc0 0x7f828c13c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f828affd700 1 -- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f828c067dc0 msgr2=0x7f828c13c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f828affd700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f828c067dc0 0x7f828c13c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f828affd700 1 -- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828c15cd40 msgr2=0x7f828c0afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f828affd700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828c15cd40 0x7f828c0afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f828affd700 1 -- 172.21.15.3:0/4087639147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009420 con 0x7f828c068190 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f828affd700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c169f60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f828c06b700 tx=0x7f8280017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f82737fe700 1 -- 172.21.15.3:0/4087639147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828001f070 con 0x7f828c068190 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f82737fe700 1 -- 172.21.15.3:0/4087639147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8280000c90 con 0x7f828c068190 2023-10-06T05:00:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c16a690 con 0x7f828c068190 2023-10-06T05:00:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f82737fe700 1 -- 172.21.15.3:0/4087639147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280025420 con 0x7f828c068190 2023-10-06T05:00:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.535+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c16ac50 con 0x7f828c068190 2023-10-06T05:00:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.536+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c0c7c40 con 0x7f828c068190 2023-10-06T05:00:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.536+0000 7f82737fe700 1 -- 172.21.15.3:0/4087639147 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8280023070 con 0x7f828c068190 2023-10-06T05:00:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.537+0000 7f82737fe700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8278066f30 0x7f82780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.537+0000 7f82737fe700 1 -- 172.21.15.3:0/4087639147 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8280085a90 con 0x7f828c068190 2023-10-06T05:00:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.537+0000 7f828b7fe700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8278066f30 0x7f82780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.538+0000 7f828b7fe700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8278066f30 0x7f82780693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f828c15e210 tx=0x7f827c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.542+0000 7f82737fe700 1 -- 172.21.15.3:0/4087639147 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f828c0c7c40 con 0x7f828c068190 2023-10-06T05:00:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.738+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f828c0c7c00 con 0x7f828c068190 2023-10-06T05:00:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.744+0000 7f82737fe700 1 -- 172.21.15.3:0/4087639147 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v45)=0 v45) v1 ==== 121+0+0 (secure 0 0 0) 0x7f82800150c0 con 0x7f828c068190 2023-10-06T05:00:04.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8278066f30 msgr2=0x7f82780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8278066f30 0x7f82780693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f828c15e210 tx=0x7f827c009040 comp rx=0 tx=0).stop 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 msgr2=0x7f828c169f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c169f60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f828c06b700 tx=0x7f8280017000 comp rx=0 tx=0).stop 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 shutdown_connections 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8278066f30 0x7f82780693e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828c15cd40 0x7f828c0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f828c067dc0 0x7f828c13c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 --2- 172.21.15.3:0/4087639147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f828c068190 0x7f828c169f60 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 >> 172.21.15.3:0/4087639147 conn(0x7f828c0ce270 msgr2=0x7f828c15a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 shutdown_connections 2023-10-06T05:00:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.746+0000 7f8291ed1700 1 -- 172.21.15.3:0/4087639147 wait complete. 2023-10-06T05:00:04.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2023-10-06T05:00:04.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c9d57700 1 Processor -- start 2023-10-06T05:00:04.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c9d57700 1 -- start start 2023-10-06T05:00:04.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c9d57700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c9d57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4c40af4a0 0x7fa4c40ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c9d57700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c40ce420 con 0x7fa4c40cc140 2023-10-06T05:00:04.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c9d57700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c415a880 con 0x7fa4c40af4a0 2023-10-06T05:00:04.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c9d57700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c415ab60 con 0x7fa4c40cc510 2023-10-06T05:00:04.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.880+0000 7fa4c3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4c40af4a0 0x7fa4c40ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4c40af4a0 0x7fa4c40ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59718/0 (socket says 172.21.15.3:59718) 2023-10-06T05:00:04.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c3fff700 1 -- 172.21.15.3:0/4154196716 learned_addr learned my addr 172.21.15.3:0/4154196716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:04.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c2ffd700 1 --2- 172.21.15.3:0/4154196716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c3fff700 1 -- 172.21.15.3:0/4154196716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 msgr2=0x7fa4c40aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:04.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c3fff700 1 --2- 172.21.15.3:0/4154196716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c3fff700 1 -- 172.21.15.3:0/4154196716 >> v1:172.21.15.153:6789/0 conn(0x7fa4c40cc140 legacy=0x7fa4c4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c3fff700 1 -- 172.21.15.3:0/4154196716 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40ce0b0 con 0x7fa4c40af4a0 2023-10-06T05:00:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c3fff700 1 --2- 172.21.15.3:0/4154196716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40af4a0 0x7fa4c40ace50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009f20 tx=0x7fa4bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f191f33983c6ba6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c27fc700 1 -- 172.21.15.3:0/4154196716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc00e070 con 0x7fa4c40af4a0 2023-10-06T05:00:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c27fc700 1 -- 172.21.15.3:0/4154196716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc00ba40 con 0x7fa4c40af4a0 2023-10-06T05:00:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.881+0000 7fa4c27fc700 1 -- 172.21.15.3:0/4154196716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0092e0 con 0x7fa4c40af4a0 2023-10-06T05:00:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 -- 172.21.15.3:0/4154196716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40af4a0 msgr2=0x7fa4c40ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/4154196716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40af4a0 0x7fa4c40ace50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009f20 tx=0x7fa4bc009840 comp rx=0 tx=0).stop 2023-10-06T05:00:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 -- 172.21.15.3:0/4154196716 shutdown_connections 2023-10-06T05:00:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/4154196716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40af4a0 0x7fa4c40ace50 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/4154196716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 -- 172.21.15.3:0/4154196716 >> 172.21.15.3:0/4154196716 conn(0x7fa4c40d03e0 msgr2=0x7fa4c40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 -- 172.21.15.3:0/4154196716 shutdown_connections 2023-10-06T05:00:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.882+0000 7fa4c9d57700 1 -- 172.21.15.3:0/4154196716 wait complete. 2023-10-06T05:00:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.883+0000 7fa4c9d57700 1 Processor -- start 2023-10-06T05:00:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.883+0000 7fa4c9d57700 1 -- start start 2023-10-06T05:00:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c9d57700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c40af4a0 0x7fa4c40c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c9d57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40cc140 0x7fa4c4152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c9d57700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c9d57700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c8710 con 0x7fa4c40cc140 2023-10-06T05:00:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c9d57700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40b5b20 con 0x7fa4c40af4a0 2023-10-06T05:00:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c9d57700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40b51f0 con 0x7fa4c40cc510 2023-10-06T05:00:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40cc140 0x7fa4c4152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40cc140 0x7fa4c4152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59726/0 (socket says 172.21.15.3:59726) 2023-10-06T05:00:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 -- 172.21.15.3:0/3743215182 learned_addr learned my addr 172.21.15.3:0/3743215182 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c37fe700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c40af4a0 0x7fa4c40c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c3fff700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 -- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 msgr2=0x7fa4c40c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 -- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c40af4a0 msgr2=0x7fa4c40c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c40af4a0 0x7fa4c40c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 -- 172.21.15.3:0/3743215182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc009ae0 con 0x7fa4c40cc140 2023-10-06T05:00:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.884+0000 7fa4c2ffd700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40cc140 0x7fa4c4152d50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800d890 tx=0x7fa4b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.885+0000 7fa4ab7fe700 1 -- 172.21.15.3:0/3743215182 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b80125f0 con 0x7fa4c40cc140 2023-10-06T05:00:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.885+0000 7fa4ab7fe700 1 -- 172.21.15.3:0/3743215182 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b8012750 con 0x7fa4c40cc140 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.885+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c40b54d0 con 0x7fa4c40cc140 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.885+0000 7fa4ab7fe700 1 -- 172.21.15.3:0/3743215182 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8012b10 con 0x7fa4c40cc140 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.885+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c4153290 con 0x7fa4c40cc140 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.885+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c40c4e10 con 0x7fa4c40cc140 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.886+0000 7fa4ab7fe700 1 -- 172.21.15.3:0/3743215182 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4b801e460 con 0x7fa4c40cc140 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.886+0000 7fa4ab7fe700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4b0066f30 0x7fa4b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.886+0000 7fa4ab7fe700 1 -- 172.21.15.3:0/3743215182 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4b8084b20 con 0x7fa4c40cc140 2023-10-06T05:00:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.887+0000 7fa4c37fe700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4b0066f30 0x7fa4b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.887+0000 7fa4c37fe700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4b0066f30 0x7fa4b00693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4003ac0 tx=0x7fa4b4007f10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:04.891+0000 7fa4ab7fe700 1 -- 172.21.15.3:0/3743215182 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4b8051c30 con 0x7fa4c40cc140 2023-10-06T05:00:05.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.082+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7fa4c404c380 con 0x7fa4c40cc140 2023-10-06T05:00:05.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.087+0000 7fa4ab7fe700 1 -- 172.21.15.3:0/3743215182 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v46)=0 v46) v1 ==== 119+0+0 (secure 0 0 0) 0x7fa4b8055250 con 0x7fa4c40cc140 2023-10-06T05:00:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4b0066f30 msgr2=0x7fa4b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4b0066f30 0x7fa4b00693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4003ac0 tx=0x7fa4b4007f10 comp rx=0 tx=0).stop 2023-10-06T05:00:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40cc140 msgr2=0x7fa4c4152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40cc140 0x7fa4c4152d50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800d890 tx=0x7fa4b8008040 comp rx=0 tx=0).stop 2023-10-06T05:00:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 shutdown_connections 2023-10-06T05:00:05.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa4b0066f30 0x7fa4b00693e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c40af4a0 0x7fa4c40c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4c40cc510 0x7fa4c40c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 --2- 172.21.15.3:0/3743215182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c40cc140 0x7fa4c4152d50 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.089+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 >> 172.21.15.3:0/3743215182 conn(0x7fa4c40d03e0 msgr2=0x7fa4c40ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:05.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.090+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 shutdown_connections 2023-10-06T05:00:05.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.090+0000 7fa4c9d57700 1 -- 172.21.15.3:0/3743215182 wait complete. 2023-10-06T05:00:05.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2023-10-06T05:00:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.223+0000 7fc0b8a1b700 1 Processor -- start 2023-10-06T05:00:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.223+0000 7fc0b8a1b700 1 -- start start 2023-10-06T05:00:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.224+0000 7fc0b8a1b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 0x7fc0b40b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.224+0000 7fc0b8a1b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b4069f10 con 0x7fc0b4067ab0 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.224+0000 7fc0b8a1b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b40cca70 con 0x7fc0b4140e70 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.224+0000 7fc0b8a1b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b40cc140 con 0x7fc0b40b0fb0 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.224+0000 7fc0b2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc0b4067ab0 0x7fc0b40691b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33672/0 (socket says 172.21.15.3:33672) 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.224+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/3788201910 learned_addr learned my addr 172.21.15.3:0/3788201910 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b1d9b700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 0x7fc0b40b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294169449 0 0) 0x7fc0b4069f10 con 0x7fc0b4067ab0 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a0003570 con 0x7fc0b4067ab0 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798504606 0 0) 0x7fc0a0003570 con 0x7fc0b4067ab0 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 msgr2=0x7fc0b40b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 0x7fc0b40b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 >> v1:172.21.15.153:6789/0 conn(0x7fc0b4140e70 legacy=0x7fc0b40ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b40cc420 con 0x7fc0b4067ab0 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc0a40017f0 con 0x7fc0b4067ab0 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 >> v1:172.21.15.3:6789/0 conn(0x7fc0b4067ab0 legacy=0x7fc0b40691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0003a90 0x7fc0a0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 0x7fc0a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0a0004d50 0x7fc0a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b40cc140 con 0x7fc0a0004d50 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a000bb70 con 0x7fc0a0003a90 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a000bd30 con 0x7fc0b40b0fb0 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b2d9d700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0a0004d50 0x7fc0a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b259c700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0003a90 0x7fc0a0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b1d9b700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 0x7fc0a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 msgr2=0x7fc0a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b2d9d700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40b0fb0 0x7fc0a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0003a90 msgr2=0x7fc0a0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b2d9d700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0003a90 0x7fc0a0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.225+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/3788201910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a0003570 con 0x7fc0a0004d50 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b2d9d700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0a0004d50 0x7fc0a00051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400a930 tx=0x7fc0a40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f10670f5f85dfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40070d0 con 0x7fc0a0004d50 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4007230 con 0x7fc0a0004d50 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b159a700 1 -- 172.21.15.3:0/3788201910 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40075d0 con 0x7fc0a0004d50 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/3788201910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0a0004d50 msgr2=0x7fc0a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b8a1b700 1 --2- 172.21.15.3:0/3788201910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0a0004d50 0x7fc0a00051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400a930 tx=0x7fc0a40021d0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/3788201910 reap_dead start 2023-10-06T05:00:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/3788201910 shutdown_connections 2023-10-06T05:00:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/3788201910 >> 172.21.15.3:0/3788201910 conn(0x7fc0b40ce210 msgr2=0x7fc0b40ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/3788201910 shutdown_connections 2023-10-06T05:00:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.226+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/3788201910 wait complete. 2023-10-06T05:00:05.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.227+0000 7fc0b8a1b700 1 Processor -- start 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b8a1b700 1 -- start start 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b8a1b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0004d50 0x7fc0b406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b8a1b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40c5690 0x7fc0b40b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b8a1b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b415a550 0x7fc0b415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b8a1b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b406ddb0 con 0x7fc0b415a550 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b8a1b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b415aee0 con 0x7fc0a0004d50 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b8a1b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b406e030 con 0x7fc0b40c5690 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b415a550 0x7fc0b415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b415a550 0x7fc0b415a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59734/0 (socket says 172.21.15.3:59734) 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.228+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/1408116007 learned_addr learned my addr 172.21.15.3:0/1408116007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b1d9b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40c5690 0x7fc0b40b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b259c700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0004d50 0x7fc0b406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40c5690 msgr2=0x7fc0b40b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b2d9d700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40c5690 0x7fc0b40b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0004d50 msgr2=0x7fc0b406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b2d9d700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0004d50 0x7fc0b406d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b2d9d700 1 -- 172.21.15.3:0/1408116007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b406e310 con 0x7fc0b415a550 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b1d9b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40c5690 0x7fc0b40b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b259c700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0004d50 0x7fc0b406d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b2d9d700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b415a550 0x7fc0b415a9a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400a3f0 tx=0x7fc0a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0aa7fc700 1 -- 172.21.15.3:0/1408116007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40073a0 con 0x7fc0b415a550 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0aa7fc700 1 -- 172.21.15.3:0/1408116007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4007500 con 0x7fc0b415a550 2023-10-06T05:00:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b406e5f0 con 0x7fc0b415a550 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0aa7fc700 1 -- 172.21.15.3:0/1408116007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a401e680 con 0x7fc0b415a550 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.229+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b406ebb0 con 0x7fc0b415a550 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.230+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b406eda0 con 0x7fc0b415a550 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.230+0000 7fc0aa7fc700 1 -- 172.21.15.3:0/1408116007 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0a401ea30 con 0x7fc0b415a550 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.231+0000 7fc0aa7fc700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.231+0000 7fc0aa7fc700 1 -- 172.21.15.3:0/1408116007 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc0a4085ff0 con 0x7fc0b415a550 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.231+0000 7fc0b259c700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.232+0000 7fc0b259c700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc094066f30 0x7fc0940693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac003b40 tx=0x7fc0ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.235+0000 7fc0aa7fc700 1 -- 172.21.15.3:0/1408116007 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0a40530c0 con 0x7fc0b415a550 2023-10-06T05:00:05.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.428+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7fc0b404c380 con 0x7fc0b415a550 2023-10-06T05:00:05.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.434+0000 7fc0aa7fc700 1 -- 172.21.15.3:0/1408116007 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v47)=0 v47) v1 ==== 128+0+0 (secure 0 0 0) 0x7fc0a40566e0 con 0x7fc0b415a550 2023-10-06T05:00:05.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc094066f30 msgr2=0x7fc0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc094066f30 0x7fc0940693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac003b40 tx=0x7fc0ac009040 comp rx=0 tx=0).stop 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b415a550 msgr2=0x7fc0b415a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b415a550 0x7fc0b415a9a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400a3f0 tx=0x7fc0a4016000 comp rx=0 tx=0).stop 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 shutdown_connections 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a0004d50 0x7fc0b406d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b415a550 0x7fc0b415a9a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 --2- 172.21.15.3:0/1408116007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0b40c5690 0x7fc0b40b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 >> 172.21.15.3:0/1408116007 conn(0x7fc0b40ce210 msgr2=0x7fc0b4069020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:05.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 shutdown_connections 2023-10-06T05:00:05.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.436+0000 7fc0b8a1b700 1 -- 172.21.15.3:0/1408116007 wait complete. 2023-10-06T05:00:05.449 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-10-06T05:00:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.569+0000 7fc678cbf700 1 Processor -- start 2023-10-06T05:00:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.569+0000 7fc678cbf700 1 -- start start 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc678cbf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 0x7fc67415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc678cbf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc6740cb3c0 0x7fc67415db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc678cbf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6740cb810 con 0x7fc6740afec0 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc678cbf700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6740cf0f0 con 0x7fc6740cb3c0 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc678cbf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67415e0c0 con 0x7fc67415c760 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc672d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc6740afec0 0x7fc6740cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33680/0 (socket says 172.21.15.3:33680) 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc672d9d700 1 -- 172.21.15.3:0/685136777 learned_addr learned my addr 172.21.15.3:0/685136777 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67259c700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 0x7fc67415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc671d9b700 1 --2- 172.21.15.3:0/685136777 >> v2:172.21.15.153:3300/0 conn(0x7fc6740cb3c0 0x7fc67415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177090203 0 0) 0x7fc66c001110 con 0x7fc6740afec0 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc660003570 con 0x7fc6740afec0 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3680503744 0 0) 0x7fc660003570 con 0x7fc6740afec0 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 msgr2=0x7fc67415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 0x7fc67415cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 >> v2:172.21.15.153:3300/0 conn(0x7fc6740cb3c0 msgr2=0x7fc67415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 --2- 172.21.15.3:0/685136777 >> v2:172.21.15.153:3300/0 conn(0x7fc6740cb3c0 0x7fc67415db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740ce7c0 con 0x7fc6740afec0 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc66c001600 con 0x7fc6740afec0 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 >> v1:172.21.15.3:6789/0 conn(0x7fc6740afec0 legacy=0x7fc6740cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 0x7fc660003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660003fe0 0x7fc660004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc660004d50 0x7fc6600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740cf0f0 con 0x7fc660004d50 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67415e0c0 con 0x7fc660003fe0 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.570+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66000ba90 con 0x7fc67415c760 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc672d9d700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc660004d50 0x7fc6600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc671d9b700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660003fe0 0x7fc660004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc67259c700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 0x7fc660003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc672d9d700 1 -- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 msgr2=0x7fc660003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc672d9d700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67415c760 0x7fc660003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc672d9d700 1 -- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660003fe0 msgr2=0x7fc660004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc672d9d700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660003fe0 0x7fc660004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc672d9d700 1 -- 172.21.15.3:0/685136777 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660003570 con 0x7fc660004d50 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc672d9d700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc660004d50 0x7fc6600051c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c5f0 tx=0x7fc66c007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcfaf4ba4be5944 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00cd90 con 0x7fc660004d50 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c005920 con 0x7fc660004d50 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.571+0000 7fc67159a700 1 -- 172.21.15.3:0/685136777 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c005ce0 con 0x7fc660004d50 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.572+0000 7fc678cbf700 1 -- 172.21.15.3:0/685136777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc660004d50 msgr2=0x7fc6600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.572+0000 7fc678cbf700 1 --2- 172.21.15.3:0/685136777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc660004d50 0x7fc6600051c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c5f0 tx=0x7fc66c007a80 comp rx=0 tx=0).stop 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.572+0000 7fc672d9d700 1 -- 172.21.15.3:0/685136777 reap_dead start 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.572+0000 7fc678cbf700 1 -- 172.21.15.3:0/685136777 shutdown_connections 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.572+0000 7fc678cbf700 1 -- 172.21.15.3:0/685136777 >> 172.21.15.3:0/685136777 conn(0x7fc67406f940 msgr2=0x7fc674070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.572+0000 7fc678cbf700 1 -- 172.21.15.3:0/685136777 shutdown_connections 2023-10-06T05:00:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.572+0000 7fc678cbf700 1 -- 172.21.15.3:0/685136777 wait complete. 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.573+0000 7fc678cbf700 1 Processor -- start 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.573+0000 7fc678cbf700 1 -- start start 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc678cbf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660004d50 0x7fc6740c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc678cbf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6740b3ae0 0x7fc6740c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc678cbf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6740b3f50 0x7fc6740c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc678cbf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6741547d0 con 0x7fc6740b3f50 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc678cbf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc674154210 con 0x7fc660004d50 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc678cbf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6741544f0 con 0x7fc6740b3ae0 2023-10-06T05:00:05.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6740b3f50 0x7fc6740c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6740b3f50 0x7fc6740c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59744/0 (socket says 172.21.15.3:59744) 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 -- 172.21.15.3:0/1297284281 learned_addr learned my addr 172.21.15.3:0/1297284281 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc67259c700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660004d50 0x7fc6740c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc671d9b700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6740b3ae0 0x7fc6740c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 -- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6740b3ae0 msgr2=0x7fc6740c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6740b3ae0 0x7fc6740c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 -- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660004d50 msgr2=0x7fc6740c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660004d50 0x7fc6740c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 -- 172.21.15.3:0/1297284281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740c75a0 con 0x7fc6740b3f50 2023-10-06T05:00:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc672d9d700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6740b3f50 0x7fc6740c87d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc66c011000 tx=0x7fc66c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc66a7fc700 1 -- 172.21.15.3:0/1297284281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c007c90 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc66a7fc700 1 -- 172.21.15.3:0/1297284281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c0072e0 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.574+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6740c7880 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.575+0000 7fc66a7fc700 1 -- 172.21.15.3:0/1297284281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0076a0 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.575+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6740ca260 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.575+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6740c7a70 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.576+0000 7fc66a7fc700 1 -- 172.21.15.3:0/1297284281 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc66c019030 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.576+0000 7fc66a7fc700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc654066f30 0x7fc6540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.576+0000 7fc66a7fc700 1 -- 172.21.15.3:0/1297284281 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc66c085780 con 0x7fc6740b3f50 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.576+0000 7fc67259c700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc654066f30 0x7fc6540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.577+0000 7fc67259c700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc654066f30 0x7fc6540693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc6640094d0 tx=0x7fc66400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.581+0000 7fc66a7fc700 1 -- 172.21.15.3:0/1297284281 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc66c052890 con 0x7fc6740b3f50 2023-10-06T05:00:05.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.774+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7fc6740cb810 con 0x7fc6740b3f50 2023-10-06T05:00:05.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.775+0000 7fc66a7fc700 1 -- 172.21.15.3:0/1297284281 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v47) v1 ==== 106+0+4 (secure 0 0 0) 0x7fc6740cb810 con 0x7fc6740b3f50 2023-10-06T05:00:05.777 INFO:teuthology.orchestra.run.smithi003.stdout:100 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc654066f30 msgr2=0x7fc6540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc654066f30 0x7fc6540693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc6640094d0 tx=0x7fc66400b040 comp rx=0 tx=0).stop 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6740b3f50 msgr2=0x7fc6740c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6740b3f50 0x7fc6740c87d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc66c011000 tx=0x7fc66c015000 comp rx=0 tx=0).stop 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 shutdown_connections 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc654066f30 0x7fc6540693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc660004d50 0x7fc6740c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6740b3f50 0x7fc6740c87d0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 --2- 172.21.15.3:0/1297284281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc6740b3ae0 0x7fc6740c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 >> 172.21.15.3:0/1297284281 conn(0x7fc67406f940 msgr2=0x7fc674150b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 shutdown_connections 2023-10-06T05:00:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.777+0000 7fc678cbf700 1 -- 172.21.15.3:0/1297284281 wait complete. 2023-10-06T05:00:05.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-10-06T05:00:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.911+0000 7f7943250700 1 Processor -- start 2023-10-06T05:00:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.911+0000 7f7943250700 1 -- start start 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f7943250700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f793c143140 0x7f793c067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f7943250700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c069a80 con 0x7f793c142d70 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f7943250700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c068410 con 0x7f793c067ff0 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f7943250700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c0cc880 con 0x7f793c143140 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f7940fec700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f793c142d70 0x7f793c0cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33682/0 (socket says 172.21.15.3:33682) 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f7940fec700 1 -- 172.21.15.3:0/687956192 learned_addr learned my addr 172.21.15.3:0/687956192 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793bfff700 1 --2- 172.21.15.3:0/687956192 >> v2:172.21.15.153:3300/0 conn(0x7f793c143140 0x7f793c067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293116249 0 0) 0x7f793c069a80 con 0x7f793c142d70 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7924003570 con 0x7f793c142d70 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3335702474 0 0) 0x7f7924003570 con 0x7f793c142d70 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 >> v2:172.21.15.153:3300/0 conn(0x7f793c143140 msgr2=0x7f793c067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 --2- 172.21.15.3:0/687956192 >> v2:172.21.15.153:3300/0 conn(0x7f793c143140 0x7f793c067ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 >> v1:172.21.15.153:6789/0 conn(0x7f793c067ff0 legacy=0x7f793c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c069150 con 0x7f793c142d70 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f792c001470 con 0x7f793c142d70 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.912+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 >> v1:172.21.15.3:6789/0 conn(0x7f793c142d70 legacy=0x7f793c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7924003a60 0x7f7924003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7924004810 0x7f7924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79240051c0 0x7f792400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0cc880 con 0x7f79240051c0 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792400c160 con 0x7f7924003a60 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792400c320 con 0x7f7924004810 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79240051c0 0x7f792400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f7940fec700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7924003a60 0x7f7924003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793bfff700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7924004810 0x7f7924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 -- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7924004810 msgr2=0x7f7924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7924004810 0x7f7924004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 -- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7924003a60 msgr2=0x7f7924003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7924003a60 0x7f7924003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 -- 172.21.15.3:0/687956192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924003570 con 0x7f79240051c0 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 -- 172.21.15.3:0/687956192 reap_dead start 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f79417ed700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79240051c0 0x7f792400ba60 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7934001fc0 tx=0x7f7934001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2ff72cef5cdf05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934001950 con 0x7f79240051c0 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793400c8b0 con 0x7f79240051c0 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.913+0000 7f793b7fe700 1 -- 172.21.15.3:0/687956192 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400cc90 con 0x7f79240051c0 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.914+0000 7f7943250700 1 -- 172.21.15.3:0/687956192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79240051c0 msgr2=0x7f792400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.914+0000 7f7943250700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79240051c0 0x7f792400ba60 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7934001fc0 tx=0x7f7934001640 comp rx=0 tx=0).stop 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.914+0000 7f7943250700 1 -- 172.21.15.3:0/687956192 shutdown_connections 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.914+0000 7f7943250700 1 --2- 172.21.15.3:0/687956192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79240051c0 0x7f792400ba60 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.914+0000 7f7943250700 1 -- 172.21.15.3:0/687956192 >> 172.21.15.3:0/687956192 conn(0x7f793c0ce210 msgr2=0x7f793c0ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.914+0000 7f7943250700 1 -- 172.21.15.3:0/687956192 shutdown_connections 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.914+0000 7f7943250700 1 -- 172.21.15.3:0/687956192 wait complete. 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.915+0000 7f7943250700 1 Processor -- start 2023-10-06T05:00:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.915+0000 7f7943250700 1 -- start start 2023-10-06T05:00:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f7943250700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79240051c0 0x7f793c0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f7943250700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f793c0c52c0 0x7f793c0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f7943250700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f793c15c4c0 0x7f793c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f7943250700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0c5730 con 0x7f793c15c4c0 2023-10-06T05:00:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f7943250700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c1412d0 con 0x7f79240051c0 2023-10-06T05:00:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f7943250700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0c5950 con 0x7f793c0c52c0 2023-10-06T05:00:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f793c15c4c0 0x7f793c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f793c15c4c0 0x7f793c15c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59756/0 (socket says 172.21.15.3:59756) 2023-10-06T05:00:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 -- 172.21.15.3:0/1146227197 learned_addr learned my addr 172.21.15.3:0/1146227197 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f7940fec700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79240051c0 0x7f793c0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f793bfff700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f793c0c52c0 0x7f793c0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 -- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f793c0c52c0 msgr2=0x7f793c0c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f793c0c52c0 0x7f793c0c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 -- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79240051c0 msgr2=0x7f793c0c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79240051c0 0x7f793c0c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.916+0000 7f79417ed700 1 -- 172.21.15.3:0/1146227197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c0c5c30 con 0x7f793c15c4c0 2023-10-06T05:00:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.917+0000 7f79417ed700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f793c15c4c0 0x7f793c15c960 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f793400c6a0 tx=0x7f7934015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.917+0000 7f7938ff9700 1 -- 172.21.15.3:0/1146227197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934001950 con 0x7f793c15c4c0 2023-10-06T05:00:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.917+0000 7f7938ff9700 1 -- 172.21.15.3:0/1146227197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934001250 con 0x7f793c15c4c0 2023-10-06T05:00:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.917+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c0c5f10 con 0x7f793c15c4c0 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.917+0000 7f7938ff9700 1 -- 172.21.15.3:0/1146227197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793401e870 con 0x7f793c15c4c0 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.917+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c0c64d0 con 0x7f793c15c4c0 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.918+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c0c66c0 con 0x7f793c15c4c0 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.918+0000 7f7938ff9700 1 -- 172.21.15.3:0/1146227197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f793401ec60 con 0x7f793c15c4c0 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.918+0000 7f7938ff9700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.919+0000 7f7938ff9700 1 -- 172.21.15.3:0/1146227197 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7934086050 con 0x7f793c15c4c0 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.919+0000 7f7940fec700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:05.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.919+0000 7f7940fec700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7928066f30 0x7f79280693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f792c001120 tx=0x7f793c0cef30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:05.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:05.923+0000 7f7938ff9700 1 -- 172.21.15.3:0/1146227197 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7934053160 con 0x7f793c15c4c0 2023-10-06T05:00:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.115+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7f793c04c380 con 0x7f793c15c4c0 2023-10-06T05:00:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.120+0000 7f7938ff9700 1 -- 172.21.15.3:0/1146227197 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v48)=0 v48) v1 ==== 124+0+0 (secure 0 0 0) 0x7f79340170c0 con 0x7f793c15c4c0 2023-10-06T05:00:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7928066f30 msgr2=0x7f79280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7928066f30 0x7f79280693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f792c001120 tx=0x7f793c0cef30 comp rx=0 tx=0).stop 2023-10-06T05:00:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f793c15c4c0 msgr2=0x7f793c15c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f793c15c4c0 0x7f793c15c960 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f793400c6a0 tx=0x7f7934015000 comp rx=0 tx=0).stop 2023-10-06T05:00:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 shutdown_connections 2023-10-06T05:00:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79240051c0 0x7f793c0c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f793c15c4c0 0x7f793c15c960 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 --2- 172.21.15.3:0/1146227197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f793c0c52c0 0x7f793c0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 >> 172.21.15.3:0/1146227197 conn(0x7f793c0ce210 msgr2=0x7f793c141b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 shutdown_connections 2023-10-06T05:00:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.123+0000 7f7943250700 1 -- 172.21.15.3:0/1146227197 wait complete. 2023-10-06T05:00:06.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-10-06T05:00:06.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.256+0000 7fa036e39700 1 Processor -- start 2023-10-06T05:00:06.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.256+0000 7fa036e39700 1 -- start start 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.256+0000 7fa036e39700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa030150690 0x7fa030150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.256+0000 7fa036e39700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa03015d540 0x7fa0300af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.256+0000 7fa036e39700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03015a290 con 0x7fa0300cba50 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.256+0000 7fa036e39700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0300d0020 con 0x7fa030150690 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.256+0000 7fa036e39700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa030159f40 con 0x7fa03015d540 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa034bd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa030150690 0x7fa030150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa02ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa0300cba50 0x7fa03015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33684/0 (socket says 172.21.15.3:33684) 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa02ffff700 1 -- 172.21.15.3:0/2471596090 learned_addr learned my addr 172.21.15.3:0/2471596090 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:06.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa0353d6700 1 --2- 172.21.15.3:0/2471596090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa03015d540 0x7fa0300af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa034bd5700 1 -- 172.21.15.3:0/2471596090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa03015d540 msgr2=0x7fa0300af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa034bd5700 1 --2- 172.21.15.3:0/2471596090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa03015d540 0x7fa0300af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa034bd5700 1 -- 172.21.15.3:0/2471596090 >> v1:172.21.15.3:6789/0 conn(0x7fa0300cba50 legacy=0x7fa03015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa034bd5700 1 -- 172.21.15.3:0/2471596090 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03015dcb0 con 0x7fa030150690 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa02f7fe700 1 -- 172.21.15.3:0/2471596090 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39356824 0 0) 0x7fa03015a290 con 0x7fa0300cba50 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa034bd5700 1 --2- 172.21.15.3:0/2471596090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa030150690 0x7fa030150a60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa028009e60 tx=0x7fa028009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d67f78bd3edec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa02f7fe700 1 -- 172.21.15.3:0/2471596090 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800e070 con 0x7fa030150690 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa02f7fe700 1 -- 172.21.15.3:0/2471596090 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02800bc30 con 0x7fa030150690 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.257+0000 7fa02f7fe700 1 -- 172.21.15.3:0/2471596090 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028012430 con 0x7fa030150690 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 -- 172.21.15.3:0/2471596090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa030150690 msgr2=0x7fa030150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 --2- 172.21.15.3:0/2471596090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa030150690 0x7fa030150a60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa028009e60 tx=0x7fa028009780 comp rx=0 tx=0).stop 2023-10-06T05:00:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 -- 172.21.15.3:0/2471596090 shutdown_connections 2023-10-06T05:00:06.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 --2- 172.21.15.3:0/2471596090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa03015d540 0x7fa0300af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 --2- 172.21.15.3:0/2471596090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa030150690 0x7fa030150a60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 -- 172.21.15.3:0/2471596090 >> 172.21.15.3:0/2471596090 conn(0x7fa030070310 msgr2=0x7fa030070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:06.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 -- 172.21.15.3:0/2471596090 shutdown_connections 2023-10-06T05:00:06.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.258+0000 7fa036e39700 1 -- 172.21.15.3:0/2471596090 wait complete. 2023-10-06T05:00:06.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.259+0000 7fa036e39700 1 Processor -- start 2023-10-06T05:00:06.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.259+0000 7fa036e39700 1 -- start start 2023-10-06T05:00:06.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa036e39700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0300cba50 0x7fa030151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa036e39700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa030150690 0x7fa0300b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa036e39700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03015d540 0x7fa030152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa036e39700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300b1860 con 0x7fa03015d540 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa036e39700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030151750 con 0x7fa0300cba50 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa036e39700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300b1250 con 0x7fa030150690 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03015d540 0x7fa030152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03015d540 0x7fa030152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59760/0 (socket says 172.21.15.3:59760) 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 -- 172.21.15.3:0/3935585667 learned_addr learned my addr 172.21.15.3:0/3935585667 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa034bd5700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0300cba50 0x7fa030151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa02ffff700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa030150690 0x7fa0300b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 -- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa030150690 msgr2=0x7fa0300b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa030150690 0x7fa0300b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 -- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0300cba50 msgr2=0x7fa030151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0300cba50 0x7fa030151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa0353d6700 1 -- 172.21.15.3:0/3935585667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028009a20 con 0x7fa03015d540 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa02ffff700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa030150690 0x7fa0300b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.260+0000 7fa034bd5700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0300cba50 0x7fa030151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.261+0000 7fa0353d6700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03015d540 0x7fa030152ba0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa02000d890 tx=0x7fa020008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.261+0000 7fa02cff9700 1 -- 172.21.15.3:0/3935585667 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0200125f0 con 0x7fa03015d540 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.261+0000 7fa02cff9700 1 -- 172.21.15.3:0/3935585667 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020012770 con 0x7fa03015d540 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.261+0000 7fa02cff9700 1 -- 172.21.15.3:0/3935585667 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020012b30 con 0x7fa03015d540 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.261+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0301535a0 con 0x7fa03015d540 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.261+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300b14f0 con 0x7fa03015d540 2023-10-06T05:00:06.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.262+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0300b1a10 con 0x7fa03015d540 2023-10-06T05:00:06.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.262+0000 7fa02cff9700 1 -- 172.21.15.3:0/3935585667 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa02001e460 con 0x7fa03015d540 2023-10-06T05:00:06.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.263+0000 7fa02cff9700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa01c066f30 0x7fa01c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.263+0000 7fa02cff9700 1 -- 172.21.15.3:0/3935585667 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa020084cc0 con 0x7fa03015d540 2023-10-06T05:00:06.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.263+0000 7fa034bd5700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa01c066f30 0x7fa01c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.264+0000 7fa034bd5700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa01c066f30 0x7fa01c0693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa028009e60 tx=0x7fa028019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.267+0000 7fa02cff9700 1 -- 172.21.15.3:0/3935585667 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0300b1a10 con 0x7fa03015d540 2023-10-06T05:00:06.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.462+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7fa0300b1a10 con 0x7fa03015d540 2023-10-06T05:00:06.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.462+0000 7fa02cff9700 1 -- 172.21.15.3:0/3935585667 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v48) v1 ==== 106+0+8 (secure 0 0 0) 0x7fa0300b1a10 con 0x7fa03015d540 2023-10-06T05:00:06.464 INFO:teuthology.orchestra.run.smithi003.stdout:1048576 2023-10-06T05:00:06.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa01c066f30 msgr2=0x7fa01c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa01c066f30 0x7fa01c0693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa028009e60 tx=0x7fa028019000 comp rx=0 tx=0).stop 2023-10-06T05:00:06.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03015d540 msgr2=0x7fa030152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03015d540 0x7fa030152ba0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa02000d890 tx=0x7fa020008040 comp rx=0 tx=0).stop 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 shutdown_connections 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa01c066f30 0x7fa01c0693e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0300cba50 0x7fa030151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03015d540 0x7fa030152ba0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 --2- 172.21.15.3:0/3935585667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa030150690 0x7fa0300b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 >> 172.21.15.3:0/3935585667 conn(0x7fa030070310 msgr2=0x7fa0300c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 shutdown_connections 2023-10-06T05:00:06.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.464+0000 7fa036e39700 1 -- 172.21.15.3:0/3935585667 wait complete. 2023-10-06T05:00:06.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-10-06T05:00:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.598+0000 7f0875808700 1 Processor -- start 2023-10-06T05:00:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.598+0000 7f0875808700 1 -- start start 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.598+0000 7f0875808700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f08700cba50 0x7f0870150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.598+0000 7f0875808700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087015d540 0x7f08700af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.598+0000 7f0875808700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087015a290 con 0x7f0870150690 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.598+0000 7f0875808700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08700d0020 con 0x7f087015d540 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.598+0000 7f0875808700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0870159f40 con 0x7f08700cba50 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087015d540 0x7f08700af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0870150690 0x7f087015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33686/0 (socket says 172.21.15.3:33686) 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087015d540 0x7f08700af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59770/0 (socket says 172.21.15.3:59770) 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086effd700 1 -- 172.21.15.3:0/1591444184 learned_addr learned my addr 172.21.15.3:0/1591444184 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086f7fe700 1 -- 172.21.15.3:0/1591444184 >> v2:172.21.15.153:3300/0 conn(0x7f08700cba50 msgr2=0x7f0870150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086e7fc700 1 --2- 172.21.15.3:0/1591444184 >> v2:172.21.15.153:3300/0 conn(0x7f08700cba50 0x7f0870150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086f7fe700 1 --2- 172.21.15.3:0/1591444184 >> v2:172.21.15.153:3300/0 conn(0x7f08700cba50 0x7f0870150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086f7fe700 1 -- 172.21.15.3:0/1591444184 >> v1:172.21.15.3:6789/0 conn(0x7f0870150690 legacy=0x7f087015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086dffb700 1 -- 172.21.15.3:0/1591444184 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893749268 0 0) 0x7f087015a290 con 0x7f0870150690 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086f7fe700 1 -- 172.21.15.3:0/1591444184 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087015dcb0 con 0x7f087015d540 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086f7fe700 1 --2- 172.21.15.3:0/1591444184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015d540 0x7f08700af190 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f08700ce1e0 tx=0x7f0860009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128e9f6ed8d037c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086dffb700 1 -- 172.21.15.3:0/1591444184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086000e070 con 0x7f087015d540 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086dffb700 1 -- 172.21.15.3:0/1591444184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086000bc30 con 0x7f087015d540 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.599+0000 7f086dffb700 1 -- 172.21.15.3:0/1591444184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860012430 con 0x7f087015d540 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 -- 172.21.15.3:0/1591444184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015d540 msgr2=0x7f08700af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 --2- 172.21.15.3:0/1591444184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015d540 0x7f08700af190 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f08700ce1e0 tx=0x7f0860009780 comp rx=0 tx=0).stop 2023-10-06T05:00:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 -- 172.21.15.3:0/1591444184 shutdown_connections 2023-10-06T05:00:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 --2- 172.21.15.3:0/1591444184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015d540 0x7f08700af190 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 --2- 172.21.15.3:0/1591444184 >> v2:172.21.15.153:3300/0 conn(0x7f08700cba50 0x7f0870150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 -- 172.21.15.3:0/1591444184 >> 172.21.15.3:0/1591444184 conn(0x7f0870070310 msgr2=0x7f0870070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 -- 172.21.15.3:0/1591444184 shutdown_connections 2023-10-06T05:00:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.600+0000 7f0875808700 1 -- 172.21.15.3:0/1591444184 wait complete. 2023-10-06T05:00:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.601+0000 7f0875808700 1 Processor -- start 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.601+0000 7f0875808700 1 -- start start 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0875808700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08700cba50 0x7f08700b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0875808700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0870150690 0x7f08700d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0875808700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087015d540 0x7f08700d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0875808700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08700b6140 con 0x7f08700cba50 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0875808700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08701504c0 con 0x7f087015d540 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0875808700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0870150b90 con 0x7f0870150690 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08700cba50 0x7f08700b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08700cba50 0x7f08700b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59772/0 (socket says 172.21.15.3:59772) 2023-10-06T05:00:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 -- 172.21.15.3:0/300466847 learned_addr learned my addr 172.21.15.3:0/300466847 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086f7fe700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087015d540 0x7f08700d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086e7fc700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0870150690 0x7f08700d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 -- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0870150690 msgr2=0x7f08700d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0870150690 0x7f08700d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 -- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087015d540 msgr2=0x7f08700d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087015d540 0x7f08700d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 -- 172.21.15.3:0/300466847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860009a20 con 0x7f08700cba50 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f086effd700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08700cba50 0x7f08700b5c00 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f086800c7e0 tx=0x7f0868007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0866ffd700 1 -- 172.21.15.3:0/300466847 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868005b30 con 0x7f08700cba50 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0866ffd700 1 -- 172.21.15.3:0/300466847 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0868005c90 con 0x7f08700cba50 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0870150e70 con 0x7f08700cba50 2023-10-06T05:00:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.602+0000 7f0866ffd700 1 -- 172.21.15.3:0/300466847 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868007300 con 0x7f08700cba50 2023-10-06T05:00:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.603+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0870151430 con 0x7f08700cba50 2023-10-06T05:00:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.603+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087004c380 con 0x7f08700cba50 2023-10-06T05:00:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.604+0000 7f0866ffd700 1 -- 172.21.15.3:0/300466847 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f086801b3b0 con 0x7f08700cba50 2023-10-06T05:00:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.604+0000 7f0866ffd700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0850066f30 0x7f08500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.604+0000 7f0866ffd700 1 -- 172.21.15.3:0/300466847 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0868083b60 con 0x7f08700cba50 2023-10-06T05:00:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.605+0000 7f086e7fc700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0850066f30 0x7f08500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.605+0000 7f086e7fc700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0850066f30 0x7f08500693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0870153b60 tx=0x7f0858008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.609+0000 7f0866ffd700 1 -- 172.21.15.3:0/300466847 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0868050bd0 con 0x7f08700cba50 2023-10-06T05:00:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.802+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f087015df90 con 0x7f08700cba50 2023-10-06T05:00:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.803+0000 7f0866ffd700 1 -- 172.21.15.3:0/300466847 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v48) v1 ==== 112+0+10 (secure 0 0 0) 0x7f08680170b0 con 0x7f08700cba50 2023-10-06T05:00:06.805 INFO:teuthology.orchestra.run.smithi003.stdout:60.000000 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0850066f30 msgr2=0x7f08500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0850066f30 0x7f08500693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0870153b60 tx=0x7f0858008040 comp rx=0 tx=0).stop 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08700cba50 msgr2=0x7f08700b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08700cba50 0x7f08700b5c00 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f086800c7e0 tx=0x7f0868007df0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 shutdown_connections 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0850066f30 0x7f08500693e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087015d540 0x7f08700d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08700cba50 0x7f08700b5c00 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 --2- 172.21.15.3:0/300466847 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0870150690 0x7f08700d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 >> 172.21.15.3:0/300466847 conn(0x7f0870070310 msgr2=0x7f08700b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 shutdown_connections 2023-10-06T05:00:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:06.805+0000 7f0875808700 1 -- 172.21.15.3:0/300466847 wait complete. 2023-10-06T05:00:06.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T05:00:06.819 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T05:00:06.820 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:00:06.903 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.123609.asok 2023-10-06T05:00:07.085 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.123609.asok 2023-10-06T05:00:07.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:07.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.123609.asok mds_sessions 2023-10-06T05:00:07.239 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:00:07.239 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7719, 2023-10-06T05:00:07.239 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T05:00:07.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T05:00:07.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 7719 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3364067658 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.7719 v1:192.168.0.1:0/3364067658", 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/3364067658", 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2023-10-06T05:00:07.240 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6832", 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 237919975 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6835", 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 237919975 2023-10-06T05:00:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 0, 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2023-10-06T05:01:02.984678+0000", 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2023-10-06T05:00:02.984678+0000", 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2023-10-06T05:00:07.242 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 1, 2023-10-06T05:00:07.243 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2023-10-06T05:00:07.243 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:07.243 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-06T05:00:07.243 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 69 2023-10-06T05:00:07.243 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:00:07.249 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:07.249 DEBUG:teuthology.orchestra.run.smithi003:> import sys 2023-10-06T05:00:07.249 DEBUG:teuthology.orchestra.run.smithi003:> import time 2023-10-06T05:00:07.250 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:00:07.250 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:07.250 DEBUG:teuthology.orchestra.run.smithi003:> n = 200 2023-10-06T05:00:07.250 DEBUG:teuthology.orchestra.run.smithi003:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-10-06T05:00:07.250 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:07.250 DEBUG:teuthology.orchestra.run.smithi003:> if not os.path.exists(abs_path): 2023-10-06T05:00:07.250 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(abs_path) 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> handles = [] 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(0, n): 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> fname = "file_"+str(i) 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> path = os.path.join(abs_path, fname) 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> handles.append(open(path, '"'"'w'"'"')) 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2023-10-06T05:00:07.251 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2023-10-06T05:00:07.252 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:00:07.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.375+0000 7f86e08f8700 1 Processor -- start 2023-10-06T05:00:07.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.375+0000 7f86e08f8700 1 -- start start 2023-10-06T05:00:07.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86e08f8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f86dc142de0 0x7f86dc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86e08f8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc067a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86e08f8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86dc067f90 0x7f86dc0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86e08f8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc068410 con 0x7f86dc067f90 2023-10-06T05:00:07.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86e08f8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc15a9b0 con 0x7f86dc142de0 2023-10-06T05:00:07.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86e08f8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1431b0 con 0x7f86dc0b15c0 2023-10-06T05:00:07.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86dc067f90 0x7f86dc0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86dc067f90 0x7f86dc0b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59776/0 (socket says 172.21.15.3:59776) 2023-10-06T05:00:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 -- 172.21.15.3:0/745365184 learned_addr learned my addr 172.21.15.3:0/745365184 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 -- 172.21.15.3:0/745365184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 msgr2=0x7f86dc067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86da59c700 1 --2- 172.21.15.3:0/745365184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc067a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86dad9d700 1 --2- 172.21.15.3:0/745365184 >> v2:172.21.15.153:3300/0 conn(0x7f86dc142de0 0x7f86dc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 --2- 172.21.15.3:0/745365184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc067a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 -- 172.21.15.3:0/745365184 >> v2:172.21.15.153:3300/0 conn(0x7f86dc142de0 msgr2=0x7f86dc06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:07.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 --2- 172.21.15.3:0/745365184 >> v2:172.21.15.153:3300/0 conn(0x7f86dc142de0 0x7f86dc06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86db59e700 1 -- 172.21.15.3:0/745365184 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc159ae0 con 0x7f86dc067f90 2023-10-06T05:00:07.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86da59c700 1 --2- 172.21.15.3:0/745365184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc067a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:07.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.376+0000 7f86dad9d700 1 --2- 172.21.15.3:0/745365184 >> v2:172.21.15.153:3300/0 conn(0x7f86dc142de0 0x7f86dc06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:07.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86db59e700 1 --2- 172.21.15.3:0/745365184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc0b3330 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f86dc06d2b0 tx=0x7f86d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192338924a21ad49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:07.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86d9d9b700 1 -- 172.21.15.3:0/745365184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400e070 con 0x7f86dc067f90 2023-10-06T05:00:07.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86d9d9b700 1 -- 172.21.15.3:0/745365184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400ba50 con 0x7f86dc067f90 2023-10-06T05:00:07.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86d9d9b700 1 -- 172.21.15.3:0/745365184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40092e0 con 0x7f86dc067f90 2023-10-06T05:00:07.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 -- 172.21.15.3:0/745365184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 msgr2=0x7f86dc0b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 --2- 172.21.15.3:0/745365184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc0b3330 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f86dc06d2b0 tx=0x7f86d4009840 comp rx=0 tx=0).stop 2023-10-06T05:00:07.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 -- 172.21.15.3:0/745365184 shutdown_connections 2023-10-06T05:00:07.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 --2- 172.21.15.3:0/745365184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc0b3330 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 --2- 172.21.15.3:0/745365184 >> v2:172.21.15.153:3300/0 conn(0x7f86dc142de0 0x7f86dc06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 --2- 172.21.15.3:0/745365184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc067a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 -- 172.21.15.3:0/745365184 >> 172.21.15.3:0/745365184 conn(0x7f86dc06ce40 msgr2=0x7f86dc0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 -- 172.21.15.3:0/745365184 shutdown_connections 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.377+0000 7f86e08f8700 1 -- 172.21.15.3:0/745365184 wait complete. 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.379+0000 7f86e08f8700 1 Processor -- start 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.379+0000 7f86e08f8700 1 -- start start 2023-10-06T05:00:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86e08f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86e08f8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc0cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86e08f8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86dc142de0 0x7f86dc0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86e08f8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0c7380 con 0x7f86dc067f90 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86e08f8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1696d0 con 0x7f86dc142de0 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86e08f8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1699b0 con 0x7f86dc0b15c0 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc13ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc13ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59784/0 (socket says 172.21.15.3:59784) 2023-10-06T05:00:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 -- 172.21.15.3:0/3120781236 learned_addr learned my addr 172.21.15.3:0/3120781236 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86da59c700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc0cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 -- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 msgr2=0x7f86dc0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86db59e700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86dc142de0 0x7f86dc0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc0cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 -- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86dc142de0 msgr2=0x7f86dc0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86dc142de0 0x7f86dc0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 -- 172.21.15.3:0/3120781236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4009ae0 con 0x7f86dc067f90 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86da59c700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc0cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86dad9d700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc13ed00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c850 tx=0x7f86cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.380+0000 7f86c2ffd700 1 -- 172.21.15.3:0/3120781236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc011600 con 0x7f86dc067f90 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.381+0000 7f86c2ffd700 1 -- 172.21.15.3:0/3120781236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc011760 con 0x7f86dc067f90 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.381+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc169c90 con 0x7f86dc067f90 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.381+0000 7f86c2ffd700 1 -- 172.21.15.3:0/3120781236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc011b00 con 0x7f86dc067f90 2023-10-06T05:00:07.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.381+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc13f240 con 0x7f86dc067f90 2023-10-06T05:00:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.382+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc04c380 con 0x7f86dc067f90 2023-10-06T05:00:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.382+0000 7f86c2ffd700 1 -- 172.21.15.3:0/3120781236 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86cc0073b0 con 0x7f86dc067f90 2023-10-06T05:00:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.383+0000 7f86c2ffd700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f86c8066f30 0x7f86c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.383+0000 7f86c2ffd700 1 -- 172.21.15.3:0/3120781236 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f86cc084690 con 0x7f86dc067f90 2023-10-06T05:00:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.383+0000 7f86da59c700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f86c8066f30 0x7f86c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.383+0000 7f86da59c700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f86c8066f30 0x7f86c80693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f86dc142560 tx=0x7f86d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.387+0000 7f86c2ffd700 1 -- 172.21.15.3:0/3120781236 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86cc055230 con 0x7f86dc067f90 2023-10-06T05:00:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.662+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86dc0b0720 con 0x7f86dc067f90 2023-10-06T05:00:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.662+0000 7f86c2ffd700 1 -- 172.21.15.3:0/3120781236 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f86cc054dc0 con 0x7f86dc067f90 2023-10-06T05:00:07.664 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:07.664 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T04:59:57.924199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:00:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f86c8066f30 msgr2=0x7f86c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f86c8066f30 0x7f86c80693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f86dc142560 tx=0x7f86d0008040 comp rx=0 tx=0).stop 2023-10-06T05:00:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 msgr2=0x7f86dc13ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc13ed00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c850 tx=0x7f86cc008040 comp rx=0 tx=0).stop 2023-10-06T05:00:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 shutdown_connections 2023-10-06T05:00:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f86c8066f30 0x7f86c80693e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86dc142de0 0x7f86dc0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86dc067f90 0x7f86dc13ed00 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 --2- 172.21.15.3:0/3120781236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86dc0b15c0 0x7f86dc0cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 >> 172.21.15.3:0/3120781236 conn(0x7f86dc06ce40 msgr2=0x7f86dc15ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 shutdown_connections 2023-10-06T05:00:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.664+0000 7f86e08f8700 1 -- 172.21.15.3:0/3120781236 wait complete. 2023-10-06T05:00:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 69 2023-10-06T05:00:07.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T05:00:07.817 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":6449,"entity":{"name":{"type":"client","num":6449},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1578962507}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.0219494029999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9885081292988844,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6449 192.168.0.1:0/1578962507","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"101936","root":"/"}},{"id":7719,"entity":{"name":{"type":"client","num":7719},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3364067658}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":203,"uptime":5.8209267230000004,"requests_in_flight":200,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":405.65675081443095,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7719 192.168.0.1:0/3364067658","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"123609","root":"/"}}] 2023-10-06T05:00:07.818 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "101936", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1578962507, "type": "any" }, "name": { "num": 6449, "type": "client" } }, "id": 6449, "inst": "client.6449 192.168.0.1:0/1578962507", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9885081292988844 }, "state": "open", "uptime": 4.021949403 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "123609", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3364067658, "type": "any" }, "name": { "num": 7719, "type": "client" } }, "id": 7719, "inst": "client.7719 192.168.0.1:0/3364067658", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 203, "requests_in_flight": 200, "session_cache_liveness": { "halflife": 300, "value": 405.65675081443095 }, "state": "open", "uptime": 5.820926723 } ] 2023-10-06T05:00:07.818 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-10-06T05:00:07.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:00:07.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.936+0000 7fa3aa0f2700 1 Processor -- start 2023-10-06T05:00:07.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.936+0000 7fa3aa0f2700 1 -- start start 2023-10-06T05:00:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3aa0f2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3a4159560 0x7fa3a40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3aa0f2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a406c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3aa0f2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a406cd30 con 0x7fa3a406c8b0 2023-10-06T05:00:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3aa0f2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a40aec90 con 0x7fa3a4159560 2023-10-06T05:00:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3aa0f2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40b0fa0 con 0x7fa3a40b0b30 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3a4159560 0x7fa3a40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3a4159560 0x7fa3a40b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59794/0 (socket says 172.21.15.3:59794) 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 -- 172.21.15.3:0/237917207 learned_addr learned my addr 172.21.15.3:0/237917207 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a2ffd700 1 --2- 172.21.15.3:0/237917207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 -- 172.21.15.3:0/237917207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 msgr2=0x7fa3a406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 --2- 172.21.15.3:0/237917207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 -- 172.21.15.3:0/237917207 >> v1:172.21.15.153:6789/0 conn(0x7fa3a406c8b0 legacy=0x7fa3a415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 -- 172.21.15.3:0/237917207 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a40ae010 con 0x7fa3a4159560 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.937+0000 7fa3a37fe700 1 --2- 172.21.15.3:0/237917207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a40b05f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa3940099e0 tx=0x7fa394009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e843170f78715c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3a27fc700 1 -- 172.21.15.3:0/237917207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400b7e0 con 0x7fa3a4159560 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3a27fc700 1 -- 172.21.15.3:0/237917207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39400b940 con 0x7fa3a4159560 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3a27fc700 1 -- 172.21.15.3:0/237917207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400bce0 con 0x7fa3a4159560 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/237917207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 msgr2=0x7fa3a40b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/237917207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a40b05f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa3940099e0 tx=0x7fa394009250 comp rx=0 tx=0).stop 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/237917207 shutdown_connections 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/237917207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a40b05f0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/237917207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/237917207 >> 172.21.15.3:0/237917207 conn(0x7fa3a40cd170 msgr2=0x7fa3a40cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/237917207 shutdown_connections 2023-10-06T05:00:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.938+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/237917207 wait complete. 2023-10-06T05:00:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.939+0000 7fa3aa0f2700 1 Processor -- start 2023-10-06T05:00:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.939+0000 7fa3aa0f2700 1 -- start start 2023-10-06T05:00:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3aa0f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3a406c8b0 0x7fa3a416cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3aa0f2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a416a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3aa0f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a416a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3aa0f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40ab4e0 con 0x7fa3a4159560 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3aa0f2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40ab7c0 con 0x7fa3a406c8b0 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3aa0f2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40abaa0 con 0x7fa3a40b0b30 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a416a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a416a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59810/0 (socket says 172.21.15.3:59810) 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3a3fff700 1 -- 172.21.15.3:0/2828977882 learned_addr learned my addr 172.21.15.3:0/2828977882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3a37fe700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3a406c8b0 0x7fa3a416cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.940+0000 7fa3a2ffd700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a416a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3a3fff700 1 -- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 msgr2=0x7fa3a416a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3a3fff700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a416a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3a3fff700 1 -- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3a406c8b0 msgr2=0x7fa3a416cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3a3fff700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3a406c8b0 0x7fa3a416cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3a3fff700 1 -- 172.21.15.3:0/2828977882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940094f0 con 0x7fa3a4159560 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3a3fff700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a416a6a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00bc30 tx=0x7fa39c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa38b7fe700 1 -- 172.21.15.3:0/2828977882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00c840 con 0x7fa3a4159560 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa38b7fe700 1 -- 172.21.15.3:0/2828977882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c00c9e0 con 0x7fa3a4159560 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a40abd80 con 0x7fa3a4159560 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa38b7fe700 1 -- 172.21.15.3:0/2828977882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00cda0 con 0x7fa3a4159560 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.941+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a416d450 con 0x7fa3a4159560 2023-10-06T05:00:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.942+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a4068de0 con 0x7fa3a4159560 2023-10-06T05:00:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.943+0000 7fa38b7fe700 1 -- 172.21.15.3:0/2828977882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa39c014620 con 0x7fa3a4159560 2023-10-06T05:00:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.943+0000 7fa38b7fe700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa390066f30 0x7fa3900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.943+0000 7fa38b7fe700 1 -- 172.21.15.3:0/2828977882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa39c084e30 con 0x7fa3a4159560 2023-10-06T05:00:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.943+0000 7fa3a37fe700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa390066f30 0x7fa3900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.944+0000 7fa3a37fe700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa390066f30 0x7fa3900693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa3940099e0 tx=0x7fa394016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:07.947+0000 7fa38b7fe700 1 -- 172.21.15.3:0/2828977882 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa39c051fc0 con 0x7fa3a4159560 2023-10-06T05:00:08.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.185+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa3a4071680 con 0x7fa3a4159560 2023-10-06T05:00:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.186+0000 7fa38b7fe700 1 -- 172.21.15.3:0/2828977882 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa39c0555e0 con 0x7fa3a4159560 2023-10-06T05:00:08.189 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:08.189 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:00:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa390066f30 msgr2=0x7fa3900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa390066f30 0x7fa3900693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa3940099e0 tx=0x7fa394016000 comp rx=0 tx=0).stop 2023-10-06T05:00:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 msgr2=0x7fa3a416a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a416a6a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00bc30 tx=0x7fa39c009820 comp rx=0 tx=0).stop 2023-10-06T05:00:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 shutdown_connections 2023-10-06T05:00:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa390066f30 0x7fa3900693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3a406c8b0 0x7fa3a416cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a4159560 0x7fa3a416a6a0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 --2- 172.21.15.3:0/2828977882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa3a40b0b30 0x7fa3a416a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 >> 172.21.15.3:0/2828977882 conn(0x7fa3a40cd170 msgr2=0x7fa3a4156f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 shutdown_connections 2023-10-06T05:00:08.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:08.188+0000 7fa3aa0f2700 1 -- 172.21.15.3:0/2828977882 wait complete. 2023-10-06T05:00:08.201 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:00:08.201 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T05:00:13.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:00:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.332+0000 7f0d1e2af700 1 Processor -- start 2023-10-06T05:00:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.332+0000 7f0d1e2af700 1 -- start start 2023-10-06T05:00:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d1e2af700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d1815c7b0 0x7f0d1815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d1e2af700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180cf2a0 con 0x7f0d1815dd00 2023-10-06T05:00:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d1e2af700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180ce970 con 0x7f0d180afed0 2023-10-06T05:00:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d1e2af700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d180cc7d0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d17fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d1815c7b0 0x7f0d1815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d177fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0d1815dd00 0x7f0d180cb390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55068/0 (socket says 172.21.15.3:55068) 2023-10-06T05:00:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d177fe700 1 -- 172.21.15.3:0/3975427725 learned_addr learned my addr 172.21.15.3:0/3975427725 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d17fff700 1 -- 172.21.15.3:0/3975427725 >> v1:172.21.15.153:6789/0 conn(0x7f0d180afed0 legacy=0x7f0d180cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d17fff700 1 -- 172.21.15.3:0/3975427725 >> v1:172.21.15.3:6789/0 conn(0x7f0d1815dd00 legacy=0x7f0d180cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d17fff700 1 -- 172.21.15.3:0/3975427725 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180cec50 con 0x7f0d1815c7b0 2023-10-06T05:00:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d17fff700 1 --2- 172.21.15.3:0/3975427725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d1815cb80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0d000098e0 tx=0x7f0d000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c495434134930dec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.333+0000 7f0d16ffd700 1 -- 172.21.15.3:0/3975427725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00009e80 con 0x7f0d1815c7b0 2023-10-06T05:00:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d16ffd700 1 -- 172.21.15.3:0/3975427725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0000b8b0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d16ffd700 1 -- 172.21.15.3:0/3975427725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000bc30 con 0x7f0d1815c7b0 2023-10-06T05:00:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d1e2af700 1 -- 172.21.15.3:0/3975427725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 msgr2=0x7f0d1815cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/3975427725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d1815cb80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0d000098e0 tx=0x7f0d000091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d1e2af700 1 -- 172.21.15.3:0/3975427725 shutdown_connections 2023-10-06T05:00:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/3975427725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d1815cb80 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d1e2af700 1 -- 172.21.15.3:0/3975427725 >> 172.21.15.3:0/3975427725 conn(0x7f0d18070310 msgr2=0x7f0d18070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:13.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d1e2af700 1 -- 172.21.15.3:0/3975427725 shutdown_connections 2023-10-06T05:00:13.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.334+0000 7f0d1e2af700 1 -- 172.21.15.3:0/3975427725 wait complete. 2023-10-06T05:00:13.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.335+0000 7f0d1e2af700 1 Processor -- start 2023-10-06T05:00:13.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.335+0000 7f0d1e2af700 1 -- start start 2023-10-06T05:00:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d1e2af700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d180afed0 0x7f0d1815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d1e2af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d181699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d1e2af700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1815dd00 0x7f0d180c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d1e2af700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1816d4b0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d1e2af700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1816a3f0 con 0x7f0d1815dd00 2023-10-06T05:00:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d1e2af700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1816a6d0 con 0x7f0d180afed0 2023-10-06T05:00:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d181699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d181699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45998/0 (socket says 172.21.15.3:45998) 2023-10-06T05:00:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d177fe700 1 -- 172.21.15.3:0/2028146017 learned_addr learned my addr 172.21.15.3:0/2028146017 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d1c84c700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1815dd00 0x7f0d180c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.336+0000 7f0d17fff700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d180afed0 0x7f0d1815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d177fe700 1 -- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d180afed0 msgr2=0x7f0d1815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d177fe700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d180afed0 0x7f0d1815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d177fe700 1 -- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1815dd00 msgr2=0x7f0d180c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d177fe700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1815dd00 0x7f0d180c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d177fe700 1 -- 172.21.15.3:0/2028146017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00009420 con 0x7f0d1815c7b0 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d177fe700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d181699a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c560 tx=0x7f0d08001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d0ffff700 1 -- 172.21.15.3:0/2028146017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08005b30 con 0x7f0d1815c7b0 2023-10-06T05:00:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d0ffff700 1 -- 172.21.15.3:0/2028146017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d08005cb0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d0ffff700 1 -- 172.21.15.3:0/2028146017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08007320 con 0x7f0d1815c7b0 2023-10-06T05:00:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1816d140 con 0x7f0d1815c7b0 2023-10-06T05:00:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.337+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d18158f40 con 0x7f0d1815c7b0 2023-10-06T05:00:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.338+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1816cfa0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.338+0000 7f0d0ffff700 1 -- 172.21.15.3:0/2028146017 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d0801b3b0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.339+0000 7f0d0ffff700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0cf8066f30 0x7f0cf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.339+0000 7f0d0ffff700 1 -- 172.21.15.3:0/2028146017 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d08083bb0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.339+0000 7f0d17fff700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0cf8066f30 0x7f0cf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.339+0000 7f0d17fff700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0cf8066f30 0x7f0cf80693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0d000030c0 tx=0x7f0d00016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:13.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.343+0000 7f0d0ffff700 1 -- 172.21.15.3:0/2028146017 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d1816cfa0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.580+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0d1816cfa0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.580+0000 7f0d0ffff700 1 -- 172.21.15.3:0/2028146017 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f0d1816cfa0 con 0x7f0d1815c7b0 2023-10-06T05:00:13.582 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:13.582 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T05:00:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0cf8066f30 msgr2=0x7f0cf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0cf8066f30 0x7f0cf80693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0d000030c0 tx=0x7f0d00016000 comp rx=0 tx=0).stop 2023-10-06T05:00:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 msgr2=0x7f0d181699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d181699a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c560 tx=0x7f0d08001020 comp rx=0 tx=0).stop 2023-10-06T05:00:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 shutdown_connections 2023-10-06T05:00:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0cf8066f30 0x7f0cf80693e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1815dd00 0x7f0d180c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d180afed0 0x7f0d1815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 --2- 172.21.15.3:0/2028146017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d1815c7b0 0x7f0d181699a0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 >> 172.21.15.3:0/2028146017 conn(0x7f0d18070310 msgr2=0x7f0d180b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 shutdown_connections 2023-10-06T05:00:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:13.582+0000 7f0d1e2af700 1 -- 172.21.15.3:0/2028146017 wait complete. 2023-10-06T05:00:13.596 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T05:00:13.596 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T05:00:18.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:00:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.725+0000 7f0c62747700 1 Processor -- start 2023-10-06T05:00:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.725+0000 7f0c62747700 1 -- start start 2023-10-06T05:00:18.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.725+0000 7f0c62747700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 0x7f0c5c15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.725+0000 7f0c62747700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c0cf2a0 con 0x7f0c5c0afed0 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c62747700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c0ce970 con 0x7f0c5c15c7b0 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c62747700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0cc7d0 con 0x7f0c5c15cb80 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c60ce4700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0c5c0afed0 0x7f0c5c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55078/0 (socket says 172.21.15.3:55078) 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c60ce4700 1 -- 172.21.15.3:0/3739547123 learned_addr learned my addr 172.21.15.3:0/3739547123 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5b7fe700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 0x7f0c5c15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777490896 0 0) 0x7f0c5c0cf2a0 con 0x7f0c5c0afed0 2023-10-06T05:00:18.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c44003570 con 0x7f0c5c0afed0 2023-10-06T05:00:18.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853529738 0 0) 0x7f0c5c0ce970 con 0x7f0c5c15c7b0 2023-10-06T05:00:18.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5c0cf2a0 con 0x7f0c5c15c7b0 2023-10-06T05:00:18.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3433231700 0 0) 0x7f0c44003570 con 0x7f0c5c0afed0 2023-10-06T05:00:18.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 msgr2=0x7f0c5c15dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:18.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 0x7f0c5c15dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 >> v1:172.21.15.153:6789/0 conn(0x7f0c5c15c7b0 legacy=0x7f0c5c0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:18.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0cec50 con 0x7f0c5c0afed0 2023-10-06T05:00:18.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f0c540017f0 con 0x7f0c5c0afed0 2023-10-06T05:00:18.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 >> v1:172.21.15.3:6789/0 conn(0x7f0c5c0afed0 legacy=0x7f0c5c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:18.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c44004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 0x7f0c44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c44004d50 0x7f0c440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0cc7d0 con 0x7f0c44003c20 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4400bbf0 con 0x7f0c44004d50 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4400bdb0 con 0x7f0c5c15cb80 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.726+0000 7f0c5bfff700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c44004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5b7fe700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 0x7f0c44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c60ce4700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c44004d50 0x7f0c440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5bfff700 1 -- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 msgr2=0x7f0c44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5bfff700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 0x7f0c44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5bfff700 1 -- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c44004d50 msgr2=0x7f0c440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:18.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5bfff700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c44004d50 0x7f0c440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5bfff700 1 -- 172.21.15.3:0/3739547123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44003570 con 0x7f0c44003c20 2023-10-06T05:00:18.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c60ce4700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c44004d50 0x7f0c440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:18.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5b7fe700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c15cb80 0x7f0c44004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:18.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5bfff700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c44004010 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0019e0 tx=0x7f0c4c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7f260c793d3832 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:18.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00c5d0 con 0x7f0c44003c20 2023-10-06T05:00:18.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c00c730 con 0x7f0c44003c20 2023-10-06T05:00:18.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.727+0000 7f0c5affd700 1 -- 172.21.15.3:0/3739547123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00cad0 con 0x7f0c44003c20 2023-10-06T05:00:18.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.728+0000 7f0c62747700 1 -- 172.21.15.3:0/3739547123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 msgr2=0x7f0c44004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:18.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.728+0000 7f0c62747700 1 --2- 172.21.15.3:0/3739547123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c44004010 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0019e0 tx=0x7f0c4c007d30 comp rx=0 tx=0).stop 2023-10-06T05:00:18.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.728+0000 7f0c60ce4700 1 -- 172.21.15.3:0/3739547123 reap_dead start 2023-10-06T05:00:18.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.728+0000 7f0c62747700 1 -- 172.21.15.3:0/3739547123 shutdown_connections 2023-10-06T05:00:18.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.728+0000 7f0c62747700 1 -- 172.21.15.3:0/3739547123 >> 172.21.15.3:0/3739547123 conn(0x7f0c5c070310 msgr2=0x7f0c5c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:18.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.728+0000 7f0c62747700 1 -- 172.21.15.3:0/3739547123 shutdown_connections 2023-10-06T05:00:18.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.728+0000 7f0c62747700 1 -- 172.21.15.3:0/3739547123 wait complete. 2023-10-06T05:00:18.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.729+0000 7f0c62747700 1 Processor -- start 2023-10-06T05:00:18.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.729+0000 7f0c62747700 1 -- start start 2023-10-06T05:00:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c62747700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c5c0b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c62747700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c16ddf0 0x7f0c5c0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c62747700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5c0ca690 0x7f0c5c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c62747700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0c7d20 con 0x7f0c44003c20 2023-10-06T05:00:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c62747700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0c7550 con 0x7f0c5c0ca690 2023-10-06T05:00:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c62747700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c16d960 con 0x7f0c5c16ddf0 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c5c0b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c5c0b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46020/0 (socket says 172.21.15.3:46020) 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 -- 172.21.15.3:0/1392083501 learned_addr learned my addr 172.21.15.3:0/1392083501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5b7fe700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c16ddf0 0x7f0c5c0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c60ce4700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5c0ca690 0x7f0c5c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 -- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c16ddf0 msgr2=0x7f0c5c0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c16ddf0 0x7f0c5c0ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 -- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5c0ca690 msgr2=0x7f0c5c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5c0ca690 0x7f0c5c0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 -- 172.21.15.3:0/1392083501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0c7760 con 0x7f0c44003c20 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c60ce4700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5c0ca690 0x7f0c5c0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c5bfff700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c5c0b1b20 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c012f00 tx=0x7f0c4c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c43fff700 1 -- 172.21.15.3:0/1392083501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00cab0 con 0x7f0c44003c20 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.730+0000 7f0c43fff700 1 -- 172.21.15.3:0/1392083501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c001360 con 0x7f0c44003c20 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.731+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c0c7a40 con 0x7f0c44003c20 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.731+0000 7f0c43fff700 1 -- 172.21.15.3:0/1392083501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c006a70 con 0x7f0c44003c20 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.731+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c0b6030 con 0x7f0c44003c20 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.732+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c16d450 con 0x7f0c44003c20 2023-10-06T05:00:18.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.732+0000 7f0c43fff700 1 -- 172.21.15.3:0/1392083501 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c4c019070 con 0x7f0c44003c20 2023-10-06T05:00:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.732+0000 7f0c43fff700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.732+0000 7f0c43fff700 1 -- 172.21.15.3:0/1392083501 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c4c085b90 con 0x7f0c44003c20 2023-10-06T05:00:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.732+0000 7f0c5b7fe700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.733+0000 7f0c5b7fe700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c48066f30 0x7f0c480693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0d0660 tx=0x7f0c50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.737+0000 7f0c43fff700 1 -- 172.21.15.3:0/1392083501 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c5c16d450 con 0x7f0c44003c20 2023-10-06T05:00:18.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.978+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0c5c16d450 con 0x7f0c44003c20 2023-10-06T05:00:18.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.978+0000 7f0c43fff700 1 -- 172.21.15.3:0/1392083501 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f0c5c16d450 con 0x7f0c44003c20 2023-10-06T05:00:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:18.980 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-10-06T05:00:18.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c48066f30 msgr2=0x7f0c480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:18.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c48066f30 0x7f0c480693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0d0660 tx=0x7f0c50008040 comp rx=0 tx=0).stop 2023-10-06T05:00:18.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 msgr2=0x7f0c5c0b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:18.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c5c0b1b20 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c012f00 tx=0x7f0c4c015000 comp rx=0 tx=0).stop 2023-10-06T05:00:18.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 shutdown_connections 2023-10-06T05:00:18.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c5c0ca690 0x7f0c5c0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c44003c20 0x7f0c5c0b1b20 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 --2- 172.21.15.3:0/1392083501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c5c16ddf0 0x7f0c5c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:18.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 >> 172.21.15.3:0/1392083501 conn(0x7f0c5c070310 msgr2=0x7f0c5c1572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:18.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 shutdown_connections 2023-10-06T05:00:18.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:18.980+0000 7f0c62747700 1 -- 172.21.15.3:0/1392083501 wait complete. 2023-10-06T05:00:18.993 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-06T05:00:18.994 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-10-06T05:00:18.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:00:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.110+0000 7f6126ebd700 1 Processor -- start 2023-10-06T05:00:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.110+0000 7f6126ebd700 1 -- start start 2023-10-06T05:00:19.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.111+0000 7f6126ebd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61200b27b0 0x7f612015a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.111+0000 7f6126ebd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612015d6d0 con 0x7f612015d240 2023-10-06T05:00:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.111+0000 7f6126ebd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612015d9b0 con 0x7f61200cd320 2023-10-06T05:00:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.111+0000 7f6126ebd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612015aba0 con 0x7f61200b27b0 2023-10-06T05:00:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.111+0000 7f6124c59700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61200b27b0 0x7f612015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.111+0000 7f611ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f612015d240 0x7f61200ccc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55088/0 (socket says 172.21.15.3:55088) 2023-10-06T05:00:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.111+0000 7f611ffff700 1 -- 172.21.15.3:0/3907205107 learned_addr learned my addr 172.21.15.3:0/3907205107 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6124c59700 1 -- 172.21.15.3:0/3907205107 >> v1:172.21.15.153:6789/0 conn(0x7f61200cd320 legacy=0x7f61200aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6124c59700 1 -- 172.21.15.3:0/3907205107 >> v1:172.21.15.3:6789/0 conn(0x7f612015d240 legacy=0x7f61200ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6124c59700 1 -- 172.21.15.3:0/3907205107 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612015dc90 con 0x7f61200b27b0 2023-10-06T05:00:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f611f7fe700 1 -- 172.21.15.3:0/3907205107 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448539969 0 0) 0x7f612015d6d0 con 0x7f612015d240 2023-10-06T05:00:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6124c59700 1 --2- 172.21.15.3:0/3907205107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61200b27b0 0x7f612015a660 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f6118009eb0 tx=0x7f6118009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65733d93cbd3f771 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f611f7fe700 1 -- 172.21.15.3:0/3907205107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800e070 con 0x7f61200b27b0 2023-10-06T05:00:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f611f7fe700 1 -- 172.21.15.3:0/3907205107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611800bc30 con 0x7f61200b27b0 2023-10-06T05:00:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f611f7fe700 1 -- 172.21.15.3:0/3907205107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118012430 con 0x7f61200b27b0 2023-10-06T05:00:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6126ebd700 1 -- 172.21.15.3:0/3907205107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61200b27b0 msgr2=0x7f612015a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3907205107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61200b27b0 0x7f612015a660 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f6118009eb0 tx=0x7f6118009780 comp rx=0 tx=0).stop 2023-10-06T05:00:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6126ebd700 1 -- 172.21.15.3:0/3907205107 shutdown_connections 2023-10-06T05:00:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3907205107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61200b27b0 0x7f612015a660 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6126ebd700 1 -- 172.21.15.3:0/3907205107 >> 172.21.15.3:0/3907205107 conn(0x7f6120070310 msgr2=0x7f6120070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6126ebd700 1 -- 172.21.15.3:0/3907205107 shutdown_connections 2023-10-06T05:00:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.112+0000 7f6126ebd700 1 -- 172.21.15.3:0/3907205107 wait complete. 2023-10-06T05:00:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 Processor -- start 2023-10-06T05:00:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 -- start start 2023-10-06T05:00:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61200b27b0 0x7f6120150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61200cd320 0x7f61200c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f612015d240 0x7f6120143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200b6140 con 0x7f612015d240 2023-10-06T05:00:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200c91b0 con 0x7f61200b27b0 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.114+0000 7f6126ebd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61200c85a0 con 0x7f61200cd320 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f612015d240 0x7f6120143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f612015d240 0x7f6120143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46044/0 (socket says 172.21.15.3:46044) 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 -- 172.21.15.3:0/3199747005 learned_addr learned my addr 172.21.15.3:0/3199747005 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 -- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61200cd320 msgr2=0x7f61200c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61200cd320 0x7f61200c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f6124c59700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61200b27b0 0x7f6120150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 -- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61200b27b0 msgr2=0x7f6120150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61200b27b0 0x7f6120150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 -- 172.21.15.3:0/3199747005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6118009a20 con 0x7f612015d240 2023-10-06T05:00:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f612545a700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f612015d240 0x7f6120143a60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f610800c560 tx=0x7f6108007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f611cff9700 1 -- 172.21.15.3:0/3199747005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108011590 con 0x7f612015d240 2023-10-06T05:00:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f611cff9700 1 -- 172.21.15.3:0/3199747005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61080116f0 con 0x7f612015d240 2023-10-06T05:00:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61200c8880 con 0x7f612015d240 2023-10-06T05:00:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f611cff9700 1 -- 172.21.15.3:0/3199747005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108011ab0 con 0x7f612015d240 2023-10-06T05:00:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.115+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61200c8e40 con 0x7f612015d240 2023-10-06T05:00:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.116+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6120071680 con 0x7f612015d240 2023-10-06T05:00:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.117+0000 7f611cff9700 1 -- 172.21.15.3:0/3199747005 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f610801b3b0 con 0x7f612015d240 2023-10-06T05:00:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.117+0000 7f611cff9700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6100066f30 0x7f61000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.117+0000 7f611cff9700 1 -- 172.21.15.3:0/3199747005 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6108083b70 con 0x7f612015d240 2023-10-06T05:00:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.117+0000 7f6124c59700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6100066f30 0x7f61000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.118+0000 7f6124c59700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6100066f30 0x7f61000693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6118009eb0 tx=0x7f6118017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.121+0000 7f611cff9700 1 -- 172.21.15.3:0/3199747005 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6108050c80 con 0x7f612015d240 2023-10-06T05:00:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.357+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f61200c92f0 con 0x7f612015d240 2023-10-06T05:00:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.357+0000 7f611cff9700 1 -- 172.21.15.3:0/3199747005 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f6108017020 con 0x7f612015d240 2023-10-06T05:00:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:19.359 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-10-06T05:00:19.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6100066f30 msgr2=0x7f61000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6100066f30 0x7f61000693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6118009eb0 tx=0x7f6118017000 comp rx=0 tx=0).stop 2023-10-06T05:00:19.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f612015d240 msgr2=0x7f6120143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f612015d240 0x7f6120143a60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f610800c560 tx=0x7f6108007ef0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 shutdown_connections 2023-10-06T05:00:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6100066f30 0x7f61000693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61200b27b0 0x7f6120150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f612015d240 0x7f6120143a60 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 --2- 172.21.15.3:0/3199747005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61200cd320 0x7f61200c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 >> 172.21.15.3:0/3199747005 conn(0x7f6120070310 msgr2=0x7f612014fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 shutdown_connections 2023-10-06T05:00:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.360+0000 7f6126ebd700 1 -- 172.21.15.3:0/3199747005 wait complete. 2023-10-06T05:00:19.373 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:00:19.373 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-10-06T05:00:19.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:19.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.543+0000 7f98ebfe4700 1 Processor -- start 2023-10-06T05:00:19.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.543+0000 7f98ebfe4700 1 -- start start 2023-10-06T05:00:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98ebfe4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 0x7f98e4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98ebfe4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e415aeb0 con 0x7f98e415a220 2023-10-06T05:00:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98ebfe4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e415a7a0 con 0x7f98e4157400 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98ebfe4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415aa80 con 0x7f98e4157030 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e957f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f98e415a220 0x7f98e40b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55092/0 (socket says 172.21.15.3:55092) 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e957f700 1 -- 172.21.15.3:0/4231816723 learned_addr learned my addr 172.21.15.3:0/4231816723 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e9d80700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 0x7f98e4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127304406 0 0) 0x7f98e415aeb0 con 0x7f98e415a220 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98cc003570 con 0x7f98e415a220 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3010891549 0 0) 0x7f98cc003570 con 0x7f98e415a220 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 msgr2=0x7f98e4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 0x7f98e4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 >> v1:172.21.15.153:6789/0 conn(0x7f98e4157400 legacy=0x7f98e406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e406dba0 con 0x7f98e415a220 2023-10-06T05:00:19.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f98d8001470 con 0x7f98e415a220 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 >> v1:172.21.15.3:6789/0 conn(0x7f98e415a220 legacy=0x7f98e40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 0x7f98cc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98cc004d50 0x7f98cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415aa80 con 0x7f98cc004010 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc00bbf0 con 0x7f98cc004d50 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.544+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc00bdb0 con 0x7f98e4157030 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e957f700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e9d80700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 0x7f98cc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98ea581700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98cc004d50 0x7f98cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e957f700 1 -- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 msgr2=0x7f98cc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:19.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e957f700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e4157030 0x7f98cc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e957f700 1 -- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98cc004d50 msgr2=0x7f98cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e957f700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98cc004d50 0x7f98cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e957f700 1 -- 172.21.15.3:0/4231816723 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc003570 con 0x7f98cc004010 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e957f700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98cc004810 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f98d8001ed0 tx=0x7f98e40b2ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8f257c419360b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800cc30 con 0x7f98cc004010 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d800cd90 con 0x7f98cc004010 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.545+0000 7f98e8d7e700 1 -- 172.21.15.3:0/4231816723 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8005b70 con 0x7f98cc004010 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.546+0000 7f98ebfe4700 1 -- 172.21.15.3:0/4231816723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 msgr2=0x7f98cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.546+0000 7f98ebfe4700 1 --2- 172.21.15.3:0/4231816723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98cc004810 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f98d8001ed0 tx=0x7f98e40b2ec0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.546+0000 7f98ea581700 1 -- 172.21.15.3:0/4231816723 reap_dead start 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.546+0000 7f98ebfe4700 1 -- 172.21.15.3:0/4231816723 shutdown_connections 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.546+0000 7f98ebfe4700 1 -- 172.21.15.3:0/4231816723 >> 172.21.15.3:0/4231816723 conn(0x7f98e40ce270 msgr2=0x7f98e40ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.546+0000 7f98ebfe4700 1 -- 172.21.15.3:0/4231816723 shutdown_connections 2023-10-06T05:00:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.546+0000 7f98ebfe4700 1 -- 172.21.15.3:0/4231816723 wait complete. 2023-10-06T05:00:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.547+0000 7f98ebfe4700 1 Processor -- start 2023-10-06T05:00:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.547+0000 7f98ebfe4700 1 -- start start 2023-10-06T05:00:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ebfe4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98e415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ebfe4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e415b170 0x7f98e4169c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ebfe4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e415b5e0 0x7f98e416a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ebfe4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415d2c0 con 0x7f98cc004010 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ebfe4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415d5a0 con 0x7f98e415b170 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ebfe4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415d880 con 0x7f98e415b5e0 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98e415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98e415cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46064/0 (socket says 172.21.15.3:46064) 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 -- 172.21.15.3:0/875963101 learned_addr learned my addr 172.21.15.3:0/875963101 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e957f700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e415b170 0x7f98e4169c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ea581700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e415b5e0 0x7f98e416a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 -- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e415b5e0 msgr2=0x7f98e416a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e415b5e0 0x7f98e416a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 -- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e415b170 msgr2=0x7f98e4169c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e415b170 0x7f98e4169c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 -- 172.21.15.3:0/875963101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e415db60 con 0x7f98cc004010 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e957f700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e415b170 0x7f98e4169c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ea581700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e415b5e0 0x7f98e416a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e9d80700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98e415cc90 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011d40 tx=0x7f98d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e1ffb700 1 -- 172.21.15.3:0/875963101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400fac0 con 0x7f98cc004010 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e1ffb700 1 -- 172.21.15.3:0/875963101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400fc20 con 0x7f98cc004010 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e415de40 con 0x7f98cc004010 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.548+0000 7f98e1ffb700 1 -- 172.21.15.3:0/875963101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4014440 con 0x7f98cc004010 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.549+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e416a840 con 0x7f98cc004010 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.549+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e40644c0 con 0x7f98cc004010 2023-10-06T05:00:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.550+0000 7f98e1ffb700 1 -- 172.21.15.3:0/875963101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98d4014870 con 0x7f98cc004010 2023-10-06T05:00:19.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.550+0000 7f98e1ffb700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98d0066f30 0x7f98d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:19.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.550+0000 7f98e1ffb700 1 -- 172.21.15.3:0/875963101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98d4018030 con 0x7f98cc004010 2023-10-06T05:00:19.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.550+0000 7f98e957f700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98d0066f30 0x7f98d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:19.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.551+0000 7f98e957f700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98d0066f30 0x7f98d00693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c1a0 tx=0x7f98d8015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:19.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.554+0000 7f98e1ffb700 1 -- 172.21.15.3:0/875963101 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98d4052350 con 0x7f98cc004010 2023-10-06T05:00:19.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.794+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e4071680 con 0x7f98cc004010 2023-10-06T05:00:19.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.794+0000 7f98e1ffb700 1 -- 172.21.15.3:0/875963101 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f98d4055970 con 0x7f98cc004010 2023-10-06T05:00:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:19.796 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T05:00:19.596165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":9,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:00:19.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98d0066f30 msgr2=0x7f98d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98d0066f30 0x7f98d00693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c1a0 tx=0x7f98d8015000 comp rx=0 tx=0).stop 2023-10-06T05:00:19.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 msgr2=0x7f98e415cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98e415cc90 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011d40 tx=0x7f98d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 shutdown_connections 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f98d0066f30 0x7f98d00693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98e415b5e0 0x7f98e416a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98cc004010 0x7f98e415cc90 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 --2- 172.21.15.3:0/875963101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e415b170 0x7f98e4169c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 >> 172.21.15.3:0/875963101 conn(0x7f98e40ce270 msgr2=0x7f98e41578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 shutdown_connections 2023-10-06T05:00:19.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:19.796+0000 7f98ebfe4700 1 -- 172.21.15.3:0/875963101 wait complete. 2023-10-06T05:00:19.800 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 72 2023-10-06T05:00:19.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T05:00:19.952 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":6449,"entity":{"name":{"type":"client","num":6449},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1578962507}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":16.155796754000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9058884235262266,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6449 192.168.0.1:0/1578962507","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"101936","root":"/"}},{"id":7719,"entity":{"name":{"type":"client","num":7719},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3364067658}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":177,"uptime":17.954774073999999,"requests_in_flight":0,"num_completed_requests":200,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1213.7237052720955,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":266.7286197699575,"halflife":1.5},"recall_caps_throttle2o":{"value":129.28141004304669,"halflife":0.5},"session_cache_liveness":{"value":394.44204436327476,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7719 192.168.0.1:0/3364067658","completed_requests":[{"tid":5,"created_ino":"0x10000000001"},{"tid":6,"created_ino":"0x100000001f6"},{"tid":7,"created_ino":"0x100000001f7"},{"tid":8,"created_ino":"0x100000001f8"},{"tid":9,"created_ino":"0x100000001f9"},{"tid":10,"created_ino":"0x100000001fa"},{"tid":11,"created_ino":"0x100000001fb"},{"tid":12,"created_ino":"0x100000001fc"},{"tid":13,"created_ino":"0x100000001fd"},{"tid":14,"created_ino":"0x100000001fe"},{"tid":15,"created_ino":"0x100000001ff"},{"tid":16,"created_ino":"0x10000000200"},{"tid":17,"created_ino":"0x10000000201"},{"tid":18,"created_ino":"0x10000000202"},{"tid":19,"created_ino":"0x10000000203"},{"tid":20,"created_ino":"0x10000000204"},{"tid":21,"created_ino":"0x10000000205"},{"tid":22,"created_ino":"0x10000000206"},{"tid":23,"created_ino":"0x10000000207"},{"tid":24,"created_ino":"0x10000000208"},{"tid":25,"created_ino":"0x10000000209"},{"tid":26,"created_ino":"0x1000000020a"},{"tid":27,"created_ino":"0x1000000020b"},{"tid":28,"created_ino":"0x1000000020c"},{"tid":29,"created_ino":"0x1000000020d"},{"tid":30,"created_ino":"0x1000000020e"},{"tid":31,"created_ino":"0x1000000020f"},{"tid":32,"created_ino":"0x10000000210"},{"tid":33,"created_ino":"0x10000000211"},{"tid":34,"created_ino":"0x10000000212"},{"tid":35,"created_ino":"0x10000000213"},{"tid":36,"created_ino":"0x10000000214"},{"tid":37,"created_ino":"0x10000000215"},{"tid":38,"created_ino":"0x10000000216"},{"tid":39,"created_ino":"0x10000000217"},{"tid":40,"created_ino":"0x10000000218"},{"tid":41,"created_ino":"0x10000000219"},{"tid":42,"created_ino":"0x1000000021a"},{"tid":43,"created_ino":"0x1000000021b"},{"tid":44,"created_ino":"0x1000000021c"},{"tid":45,"created_ino":"0x1000000021d"},{"tid":46,"created_ino":"0x1000000021e"},{"tid":47,"created_ino":"0x1000000021f"},{"tid":48,"created_ino":"0x10000000220"},{"tid":49,"created_ino":"0x10000000221"},{"tid":50,"created_ino":"0x10000000222"},{"tid":51,"created_ino":"0x10000000223"},{"tid":52,"created_ino":"0x10000000224"},{"tid":53,"created_ino":"0x10000000225"},{"tid":54,"created_ino":"0x10000000226"},{"tid":55,"created_ino":"0x10000000227"},{"tid":56,"created_ino":"0x10000000228"},{"tid":57,"created_ino":"0x10000000229"},{"tid":58,"created_ino":"0x1000000022a"},{"tid":59,"created_ino":"0x1000000022b"},{"tid":60,"created_ino":"0x1000000022c"},{"tid":61,"created_ino":"0x1000000022d"},{"tid":62,"created_ino":"0x1000000022e"},{"tid":63,"created_ino":"0x1000000022f"},{"tid":64,"created_ino":"0x10000000230"},{"tid":65,"created_ino":"0x10000000231"},{"tid":66,"created_ino":"0x10000000232"},{"tid":67,"created_ino":"0x10000000233"},{"tid":68,"created_ino":"0x10000000234"},{"tid":69,"created_ino":"0x10000000235"},{"tid":70,"created_ino":"0x10000000236"},{"tid":71,"created_ino":"0x10000000237"},{"tid":72,"created_ino":"0x10000000238"},{"tid":73,"created_ino":"0x10000000239"},{"tid":74,"created_ino":"0x1000000023a"},{"tid":75,"created_ino":"0x1000000023b"},{"tid":76,"created_ino":"0x1000000023c"},{"tid":77,"created_ino":"0x1000000023d"},{"tid":78,"created_ino":"0x1000000023e"},{"tid":79,"created_ino":"0x1000000023f"},{"tid":80,"created_ino":"0x10000000240"},{"tid":81,"created_ino":"0x10000000241"},{"tid":82,"created_ino":"0x10000000242"},{"tid":83,"created_ino":"0x10000000243"},{"tid":84,"created_ino":"0x10000000244"},{"tid":85,"created_ino":"0x10000000245"},{"tid":86,"created_ino":"0x10000000246"},{"tid":87,"created_ino":"0x10000000247"},{"tid":88,"created_ino":"0x10000000248"},{"tid":89,"created_ino":"0x10000000249"},{"tid":90,"created_ino":"0x1000000024a"},{"tid":91,"created_ino":"0x1000000024b"},{"tid":92,"created_ino":"0x1000000024c"},{"tid":93,"created_ino":"0x1000000024d"},{"tid":94,"created_ino":"0x1000000024e"},{"tid":95,"created_ino":"0x1000000024f"},{"tid":96,"created_ino":"0x10000000250"},{"tid":97,"created_ino":"0x10000000251"},{"tid":98,"created_ino":"0x10000000252"},{"tid":99,"created_ino":"0x10000000253"},{"tid":100,"created_ino":"0x10000000254"},{"tid":101,"created_ino":"0x10000000255"},{"tid":102,"created_ino":"0x10000000256"},{"tid":103,"created_ino":"0x10000000257"},{"tid":104,"created_ino":"0x10000000258"},{"tid":105,"created_ino":"0x10000000259"},{"tid":106,"created_ino":"0x1000000025a"},{"tid":107,"created_ino":"0x1000000025b"},{"tid":108,"created_ino":"0x1000000025c"},{"tid":109,"created_ino":"0x1000000025d"},{"tid":110,"created_ino":"0x1000000025e"},{"tid":111,"created_ino":"0x1000000025f"},{"tid":112,"created_ino":"0x10000000260"},{"tid":113,"created_ino":"0x10000000261"},{"tid":114,"created_ino":"0x10000000262"},{"tid":115,"created_ino":"0x10000000263"},{"tid":116,"created_ino":"0x10000000264"},{"tid":117,"created_ino":"0x10000000265"},{"tid":118,"created_ino":"0x10000000266"},{"tid":119,"created_ino":"0x10000000267"},{"tid":120,"created_ino":"0x10000000268"},{"tid":121,"created_ino":"0x10000000269"},{"tid":122,"created_ino":"0x1000000026a"},{"tid":123,"created_ino":"0x1000000026b"},{"tid":124,"created_ino":"0x1000000026c"},{"tid":125,"created_ino":"0x1000000026d"},{"tid":126,"created_ino":"0x1000000026e"},{"tid":127,"created_ino":"0x1000000026f"},{"tid":128,"created_ino":"0x10000000270"},{"tid":129,"created_ino":"0x10000000271"},{"tid":130,"created_ino":"0x10000000272"},{"tid":131,"created_ino":"0x10000000273"},{"tid":132,"created_ino":"0x10000000274"},{"tid":133,"created_ino":"0x10000000275"},{"tid":134,"created_ino":"0x10000000276"},{"tid":135,"created_ino":"0x10000000277"},{"tid":136,"created_ino":"0x10000000278"},{"tid":137,"created_ino":"0x10000000279"},{"tid":138,"created_ino":"0x1000000027a"},{"tid":139,"created_ino":"0x1000000027b"},{"tid":140,"created_ino":"0x1000000027c"},{"tid":141,"created_ino":"0x1000000027d"},{"tid":142,"created_ino":"0x1000000027e"},{"tid":143,"created_ino":"0x1000000027f"},{"tid":144,"created_ino":"0x10000000280"},{"tid":145,"created_ino":"0x10000000281"},{"tid":146,"created_ino":"0x10000000282"},{"tid":147,"created_ino":"0x10000000283"},{"tid":148,"created_ino":"0x10000000284"},{"tid":149,"created_ino":"0x10000000285"},{"tid":150,"created_ino":"0x10000000286"},{"tid":151,"created_ino":"0x10000000287"},{"tid":152,"created_ino":"0x10000000288"},{"tid":153,"created_ino":"0x10000000289"},{"tid":154,"created_ino":"0x1000000028a"},{"tid":155,"created_ino":"0x1000000028b"},{"tid":156,"created_ino":"0x1000000028c"},{"tid":157,"created_ino":"0x1000000028d"},{"tid":158,"created_ino":"0x1000000028e"},{"tid":159,"created_ino":"0x1000000028f"},{"tid":160,"created_ino":"0x10000000290"},{"tid":161,"created_ino":"0x10000000291"},{"tid":162,"created_ino":"0x10000000292"},{"tid":163,"created_ino":"0x10000000293"},{"tid":164,"created_ino":"0x10000000294"},{"tid":165,"created_ino":"0x10000000295"},{"tid":166,"created_ino":"0x10000000296"},{"tid":167,"created_ino":"0x10000000297"},{"tid":168,"created_ino":"0x10000000298"},{"tid":169,"created_ino":"0x10000000299"},{"tid":170,"created_ino":"0x1000000029a"},{"tid":171,"created_ino":"0x1000000029b"},{"tid":172,"created_ino":"0x1000000029c"},{"tid":173,"created_ino":"0x1000000029d"},{"tid":174,"created_ino":"0x1000000029e"},{"tid":175,"created_ino":"0x1000000029f"},{"tid":176,"created_ino":"0x100000002a0"},{"tid":177,"created_ino":"0x100000002a1"},{"tid":178,"created_ino":"0x100000002a2"},{"tid":179,"created_ino":"0x100000002a3"},{"tid":180,"created_ino":"0x100000002a4"},{"tid":181,"created_ino":"0x100000002a5"},{"tid":182,"created_ino":"0x100000002a6"},{"tid":183,"created_ino":"0x100000002a7"},{"tid":184,"created_ino":"0x100000002a8"},{"tid":185,"created_ino":"0x100000002a9"},{"tid":186,"created_ino":"0x100000002aa"},{"tid":187,"created_ino":"0x100000002ab"},{"tid":188,"created_ino":"0x100000002ac"},{"tid":189,"created_ino":"0x100000002ad"},{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"123609","root":"/"}}] 2023-10-06T05:00:19.954 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "101936", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1578962507, "type": "any" }, "name": { "num": 6449, "type": "client" } }, "id": 6449, "inst": "client.6449 192.168.0.1:0/1578962507", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9058884235262266 }, "state": "open", "uptime": 16.155796754 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "123609", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 5 }, { "created_ino": "0x100000001f6", "tid": 6 }, { "created_ino": "0x100000001f7", "tid": 7 }, { "created_ino": "0x100000001f8", "tid": 8 }, { "created_ino": "0x100000001f9", "tid": 9 }, { "created_ino": "0x100000001fa", "tid": 10 }, { "created_ino": "0x100000001fb", "tid": 11 }, { "created_ino": "0x100000001fc", "tid": 12 }, { "created_ino": "0x100000001fd", "tid": 13 }, { "created_ino": "0x100000001fe", "tid": 14 }, { "created_ino": "0x100000001ff", "tid": 15 }, { "created_ino": "0x10000000200", "tid": 16 }, { "created_ino": "0x10000000201", "tid": 17 }, { "created_ino": "0x10000000202", "tid": 18 }, { "created_ino": "0x10000000203", "tid": 19 }, { "created_ino": "0x10000000204", "tid": 20 }, { "created_ino": "0x10000000205", "tid": 21 }, { "created_ino": "0x10000000206", "tid": 22 }, { "created_ino": "0x10000000207", "tid": 23 }, { "created_ino": "0x10000000208", "tid": 24 }, { "created_ino": "0x10000000209", "tid": 25 }, { "created_ino": "0x1000000020a", "tid": 26 }, { "created_ino": "0x1000000020b", "tid": 27 }, { "created_ino": "0x1000000020c", "tid": 28 }, { "created_ino": "0x1000000020d", "tid": 29 }, { "created_ino": "0x1000000020e", "tid": 30 }, { "created_ino": "0x1000000020f", "tid": 31 }, { "created_ino": "0x10000000210", "tid": 32 }, { "created_ino": "0x10000000211", "tid": 33 }, { "created_ino": "0x10000000212", "tid": 34 }, { "created_ino": "0x10000000213", "tid": 35 }, { "created_ino": "0x10000000214", "tid": 36 }, { "created_ino": "0x10000000215", "tid": 37 }, { "created_ino": "0x10000000216", "tid": 38 }, { "created_ino": "0x10000000217", "tid": 39 }, { "created_ino": "0x10000000218", "tid": 40 }, { "created_ino": "0x10000000219", "tid": 41 }, { "created_ino": "0x1000000021a", "tid": 42 }, { "created_ino": "0x1000000021b", "tid": 43 }, { "created_ino": "0x1000000021c", "tid": 44 }, { "created_ino": "0x1000000021d", "tid": 45 }, { "created_ino": "0x1000000021e", "tid": 46 }, { "created_ino": "0x1000000021f", "tid": 47 }, { "created_ino": "0x10000000220", "tid": 48 }, { "created_ino": "0x10000000221", "tid": 49 }, { "created_ino": "0x10000000222", "tid": 50 }, { "created_ino": "0x10000000223", "tid": 51 }, { "created_ino": "0x10000000224", "tid": 52 }, { "created_ino": "0x10000000225", "tid": 53 }, { "created_ino": "0x10000000226", "tid": 54 }, { "created_ino": "0x10000000227", "tid": 55 }, { "created_ino": "0x10000000228", "tid": 56 }, { "created_ino": "0x10000000229", "tid": 57 }, { "created_ino": "0x1000000022a", "tid": 58 }, { "created_ino": "0x1000000022b", "tid": 59 }, { "created_ino": "0x1000000022c", "tid": 60 }, { "created_ino": "0x1000000022d", "tid": 61 }, { "created_ino": "0x1000000022e", "tid": 62 }, { "created_ino": "0x1000000022f", "tid": 63 }, { "created_ino": "0x10000000230", "tid": 64 }, { "created_ino": "0x10000000231", "tid": 65 }, { "created_ino": "0x10000000232", "tid": 66 }, { "created_ino": "0x10000000233", "tid": 67 }, { "created_ino": "0x10000000234", "tid": 68 }, { "created_ino": "0x10000000235", "tid": 69 }, { "created_ino": "0x10000000236", "tid": 70 }, { "created_ino": "0x10000000237", "tid": 71 }, { "created_ino": "0x10000000238", "tid": 72 }, { "created_ino": "0x10000000239", "tid": 73 }, { "created_ino": "0x1000000023a", "tid": 74 }, { "created_ino": "0x1000000023b", "tid": 75 }, { "created_ino": "0x1000000023c", "tid": 76 }, { "created_ino": "0x1000000023d", "tid": 77 }, { "created_ino": "0x1000000023e", "tid": 78 }, { "created_ino": "0x1000000023f", "tid": 79 }, { "created_ino": "0x10000000240", "tid": 80 }, { "created_ino": "0x10000000241", "tid": 81 }, { "created_ino": "0x10000000242", "tid": 82 }, { "created_ino": "0x10000000243", "tid": 83 }, { "created_ino": "0x10000000244", "tid": 84 }, { "created_ino": "0x10000000245", "tid": 85 }, { "created_ino": "0x10000000246", "tid": 86 }, { "created_ino": "0x10000000247", "tid": 87 }, { "created_ino": "0x10000000248", "tid": 88 }, { "created_ino": "0x10000000249", "tid": 89 }, { "created_ino": "0x1000000024a", "tid": 90 }, { "created_ino": "0x1000000024b", "tid": 91 }, { "created_ino": "0x1000000024c", "tid": 92 }, { "created_ino": "0x1000000024d", "tid": 93 }, { "created_ino": "0x1000000024e", "tid": 94 }, { "created_ino": "0x1000000024f", "tid": 95 }, { "created_ino": "0x10000000250", "tid": 96 }, { "created_ino": "0x10000000251", "tid": 97 }, { "created_ino": "0x10000000252", "tid": 98 }, { "created_ino": "0x10000000253", "tid": 99 }, { "created_ino": "0x10000000254", "tid": 100 }, { "created_ino": "0x10000000255", "tid": 101 }, { "created_ino": "0x10000000256", "tid": 102 }, { "created_ino": "0x10000000257", "tid": 103 }, { "created_ino": "0x10000000258", "tid": 104 }, { "created_ino": "0x10000000259", "tid": 105 }, { "created_ino": "0x1000000025a", "tid": 106 }, { "created_ino": "0x1000000025b", "tid": 107 }, { "created_ino": "0x1000000025c", "tid": 108 }, { "created_ino": "0x1000000025d", "tid": 109 }, { "created_ino": "0x1000000025e", "tid": 110 }, { "created_ino": "0x1000000025f", "tid": 111 }, { "created_ino": "0x10000000260", "tid": 112 }, { "created_ino": "0x10000000261", "tid": 113 }, { "created_ino": "0x10000000262", "tid": 114 }, { "created_ino": "0x10000000263", "tid": 115 }, { "created_ino": "0x10000000264", "tid": 116 }, { "created_ino": "0x10000000265", "tid": 117 }, { "created_ino": "0x10000000266", "tid": 118 }, { "created_ino": "0x10000000267", "tid": 119 }, { "created_ino": "0x10000000268", "tid": 120 }, { "created_ino": "0x10000000269", "tid": 121 }, { "created_ino": "0x1000000026a", "tid": 122 }, { "created_ino": "0x1000000026b", "tid": 123 }, { "created_ino": "0x1000000026c", "tid": 124 }, { "created_ino": "0x1000000026d", "tid": 125 }, { "created_ino": "0x1000000026e", "tid": 126 }, { "created_ino": "0x1000000026f", "tid": 127 }, { "created_ino": "0x10000000270", "tid": 128 }, { "created_ino": "0x10000000271", "tid": 129 }, { "created_ino": "0x10000000272", "tid": 130 }, { "created_ino": "0x10000000273", "tid": 131 }, { "created_ino": "0x10000000274", "tid": 132 }, { "created_ino": "0x10000000275", "tid": 133 }, { "created_ino": "0x10000000276", "tid": 134 }, { "created_ino": "0x10000000277", "tid": 135 }, { "created_ino": "0x10000000278", "tid": 136 }, { "created_ino": "0x10000000279", "tid": 137 }, { "created_ino": "0x1000000027a", "tid": 138 }, { "created_ino": "0x1000000027b", "tid": 139 }, { "created_ino": "0x1000000027c", "tid": 140 }, { "created_ino": "0x1000000027d", "tid": 141 }, { "created_ino": "0x1000000027e", "tid": 142 }, { "created_ino": "0x1000000027f", "tid": 143 }, { "created_ino": "0x10000000280", "tid": 144 }, { "created_ino": "0x10000000281", "tid": 145 }, { "created_ino": "0x10000000282", "tid": 146 }, { "created_ino": "0x10000000283", "tid": 147 }, { "created_ino": "0x10000000284", "tid": 148 }, { "created_ino": "0x10000000285", "tid": 149 }, { "created_ino": "0x10000000286", "tid": 150 }, { "created_ino": "0x10000000287", "tid": 151 }, { "created_ino": "0x10000000288", "tid": 152 }, { "created_ino": "0x10000000289", "tid": 153 }, { "created_ino": "0x1000000028a", "tid": 154 }, { "created_ino": "0x1000000028b", "tid": 155 }, { "created_ino": "0x1000000028c", "tid": 156 }, { "created_ino": "0x1000000028d", "tid": 157 }, { "created_ino": "0x1000000028e", "tid": 158 }, { "created_ino": "0x1000000028f", "tid": 159 }, { "created_ino": "0x10000000290", "tid": 160 }, { "created_ino": "0x10000000291", "tid": 161 }, { "created_ino": "0x10000000292", "tid": 162 }, { "created_ino": "0x10000000293", "tid": 163 }, { "created_ino": "0x10000000294", "tid": 164 }, { "created_ino": "0x10000000295", "tid": 165 }, { "created_ino": "0x10000000296", "tid": 166 }, { "created_ino": "0x10000000297", "tid": 167 }, { "created_ino": "0x10000000298", "tid": 168 }, { "created_ino": "0x10000000299", "tid": 169 }, { "created_ino": "0x1000000029a", "tid": 170 }, { "created_ino": "0x1000000029b", "tid": 171 }, { "created_ino": "0x1000000029c", "tid": 172 }, { "created_ino": "0x1000000029d", "tid": 173 }, { "created_ino": "0x1000000029e", "tid": 174 }, { "created_ino": "0x1000000029f", "tid": 175 }, { "created_ino": "0x100000002a0", "tid": 176 }, { "created_ino": "0x100000002a1", "tid": 177 }, { "created_ino": "0x100000002a2", "tid": 178 }, { "created_ino": "0x100000002a3", "tid": 179 }, { "created_ino": "0x100000002a4", "tid": 180 }, { "created_ino": "0x100000002a5", "tid": 181 }, { "created_ino": "0x100000002a6", "tid": 182 }, { "created_ino": "0x100000002a7", "tid": 183 }, { "created_ino": "0x100000002a8", "tid": 184 }, { "created_ino": "0x100000002a9", "tid": 185 }, { "created_ino": "0x100000002aa", "tid": 186 }, { "created_ino": "0x100000002ab", "tid": 187 }, { "created_ino": "0x100000002ac", "tid": 188 }, { "created_ino": "0x100000002ad", "tid": 189 }, { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3364067658, "type": "any" }, "name": { "num": 7719, "type": "client" } }, "id": 7719, "inst": "client.7719 192.168.0.1:0/3364067658", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 200, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 1213.7237052720955 }, "recall_caps_throttle": { "halflife": 1.5, "value": 266.7286197699575 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 129.2814100430467 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 177, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 394.44204436327476 }, "state": "open", "uptime": 17.954774074 } ] 2023-10-06T05:00:19.956 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:00:19.956 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:00:19.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:00:19.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:00:20.042 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:00:20.042 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:00:20.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T05:00:20.042 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:00:20.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:20.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[123609]: fuse finished with error 0 and tester_r 0 2023-10-06T05:00:20.080 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:00:20.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:20.080 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:00:20.246 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T05:00:20.246 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:00:20.246 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:20.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[101936]: fuse finished with error 0 and tester_r 0 2023-10-06T05:00:20.287 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:00:20.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:20.287 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:00:20.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:00:20.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:20.397 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:00:20.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:00:20.412 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:00:20.413 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:00:20.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:20.413 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:00:20.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:00:20.429 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:00:20.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 Processor -- start 2023-10-06T05:00:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 -- start start 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815c8f0 0x7f082815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f082815ccc0 0x7f08280d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08280d06a0 0x7f082806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08280cf6c0 con 0x7f08280d06a0 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08280cfb50 con 0x7f082815ccc0 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.571+0000 7f082ecf1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082815e050 con 0x7f082815c8f0 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08280d06a0 0x7f082806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08280d06a0 0x7f082806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46076/0 (socket says 172.21.15.3:46076) 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 -- 172.21.15.3:0/1075869876 learned_addr learned my addr 172.21.15.3:0/1075869876 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:20.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082ca8d700 1 --2- 172.21.15.3:0/1075869876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815c8f0 0x7f082815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f0827fff700 1 --2- 172.21.15.3:0/1075869876 >> v2:172.21.15.153:3300/0 conn(0x7f082815ccc0 0x7f08280d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 -- 172.21.15.3:0/1075869876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815c8f0 msgr2=0x7f082815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 --2- 172.21.15.3:0/1075869876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815c8f0 0x7f082815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 -- 172.21.15.3:0/1075869876 >> v2:172.21.15.153:3300/0 conn(0x7f082815ccc0 msgr2=0x7f08280d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 --2- 172.21.15.3:0/1075869876 >> v2:172.21.15.153:3300/0 conn(0x7f082815ccc0 0x7f08280d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 -- 172.21.15.3:0/1075869876 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280d0af0 con 0x7f08280d06a0 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f082d28e700 1 --2- 172.21.15.3:0/1075869876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08280d06a0 0x7f082806d950 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f082806bed0 tx=0x7f0820009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f923f269698af51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f08277fe700 1 -- 172.21.15.3:0/1075869876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082000e070 con 0x7f08280d06a0 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f08277fe700 1 -- 172.21.15.3:0/1075869876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082000b970 con 0x7f08280d06a0 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.572+0000 7f08277fe700 1 -- 172.21.15.3:0/1075869876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08200092e0 con 0x7f08280d06a0 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 -- 172.21.15.3:0/1075869876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08280d06a0 msgr2=0x7f082806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 --2- 172.21.15.3:0/1075869876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08280d06a0 0x7f082806d950 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f082806bed0 tx=0x7f0820009840 comp rx=0 tx=0).stop 2023-10-06T05:00:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 -- 172.21.15.3:0/1075869876 shutdown_connections 2023-10-06T05:00:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 --2- 172.21.15.3:0/1075869876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08280d06a0 0x7f082806d950 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 --2- 172.21.15.3:0/1075869876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815c8f0 0x7f082815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 --2- 172.21.15.3:0/1075869876 >> v2:172.21.15.153:3300/0 conn(0x7f082815ccc0 0x7f08280d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 -- 172.21.15.3:0/1075869876 >> 172.21.15.3:0/1075869876 conn(0x7f0828070310 msgr2=0x7f0828070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 -- 172.21.15.3:0/1075869876 shutdown_connections 2023-10-06T05:00:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.573+0000 7f082ecf1700 1 -- 172.21.15.3:0/1075869876 wait complete. 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.574+0000 7f082ecf1700 1 Processor -- start 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.574+0000 7f082ecf1700 1 -- start start 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082ecf1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08280d06a0 0x7f08281561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082ecf1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f082815c8f0 0x7f0828156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082ecf1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815ccc0 0x7f0828169560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082ecf1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08281566f0 con 0x7f082815c8f0 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082ecf1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08280c60d0 con 0x7f08280d06a0 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082ecf1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08280c5220 con 0x7f082815ccc0 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f082815c8f0 0x7f0828156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082ca8d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08280d06a0 0x7f08281561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f082815c8f0 0x7f0828156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46090/0 (socket says 172.21.15.3:46090) 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 -- 172.21.15.3:0/4127726421 learned_addr learned my addr 172.21.15.3:0/4127726421 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f082d28e700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815ccc0 0x7f0828169560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 -- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815ccc0 msgr2=0x7f0828169560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815ccc0 0x7f0828169560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 -- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08280d06a0 msgr2=0x7f08281561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08280d06a0 0x7f08281561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.575+0000 7f0827fff700 1 -- 172.21.15.3:0/4127726421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820009ae0 con 0x7f082815c8f0 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.576+0000 7f0827fff700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f082815c8f0 0x7f0828156870 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f081800ec50 tx=0x7f08180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.576+0000 7f0824ff9700 1 -- 172.21.15.3:0/4127726421 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800c850 con 0x7f082815c8f0 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.576+0000 7f0824ff9700 1 -- 172.21.15.3:0/4127726421 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800c9b0 con 0x7f082815c8f0 2023-10-06T05:00:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.576+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08280c54c0 con 0x7f082815c8f0 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.576+0000 7f0824ff9700 1 -- 172.21.15.3:0/4127726421 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800cd70 con 0x7f082815c8f0 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.576+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08280c5a80 con 0x7f082815c8f0 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.577+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828071680 con 0x7f082815c8f0 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.577+0000 7f0824ff9700 1 -- 172.21.15.3:0/4127726421 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0818013680 con 0x7f082815c8f0 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.577+0000 7f0824ff9700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0808066f30 0x7f08080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.578+0000 7f0824ff9700 1 -- 172.21.15.3:0/4127726421 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0818084880 con 0x7f082815c8f0 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.578+0000 7f082ca8d700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0808066f30 0x7f08080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.578+0000 7f082ca8d700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0808066f30 0x7f08080693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f08280cc250 tx=0x7f08100091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:20.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.582+0000 7f0824ff9700 1 -- 172.21.15.3:0/4127726421 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0818051990 con 0x7f082815c8f0 2023-10-06T05:00:20.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.818+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08280b35a0 con 0x7f082815c8f0 2023-10-06T05:00:20.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.819+0000 7f0824ff9700 1 -- 172.21.15.3:0/4127726421 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f0818054fb0 con 0x7f082815c8f0 2023-10-06T05:00:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:20.821 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T05:00:19.596165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":9,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0808066f30 msgr2=0x7f08080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0808066f30 0x7f08080693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f08280cc250 tx=0x7f08100091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f082815c8f0 msgr2=0x7f0828156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f082815c8f0 0x7f0828156870 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f081800ec50 tx=0x7f08180091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 shutdown_connections 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0808066f30 0x7f08080693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08280d06a0 0x7f08281561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f082815ccc0 0x7f0828169560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 --2- 172.21.15.3:0/4127726421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f082815c8f0 0x7f0828156870 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 >> 172.21.15.3:0/4127726421 conn(0x7f0828070310 msgr2=0x7f082806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 shutdown_connections 2023-10-06T05:00:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.821+0000 7f082ecf1700 1 -- 172.21.15.3:0/4127726421 wait complete. 2023-10-06T05:00:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 72 2023-10-06T05:00:20.835 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:00:20.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.952+0000 7fd7f4865700 1 Processor -- start 2023-10-06T05:00:20.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.952+0000 7fd7f4865700 1 -- start start 2023-10-06T05:00:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7f4865700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd7f0157030 0x7fd7f0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7f4865700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd7f0157400 0x7fd7f006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7f4865700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f015aeb0 con 0x7fd7f015a220 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7f4865700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f015a7a0 con 0x7fd7f0157400 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7f4865700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f015aa80 con 0x7fd7f0157030 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7eed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd7f0157400 0x7fd7f006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7edd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd7f015a220 0x7fd7f00b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55108/0 (socket says 172.21.15.3:55108) 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7edd9b700 1 -- 172.21.15.3:0/1633691061 learned_addr learned my addr 172.21.15.3:0/1633691061 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.953+0000 7fd7eed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd7f0157400 0x7fd7f006e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46096/0 (socket says 172.21.15.3:46096) 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7ee59c700 1 --2- 172.21.15.3:0/1633691061 >> v2:172.21.15.153:3300/0 conn(0x7fd7f0157030 0x7fd7f0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7eed9d700 1 -- 172.21.15.3:0/1633691061 >> v2:172.21.15.153:3300/0 conn(0x7fd7f0157030 msgr2=0x7fd7f0159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7eed9d700 1 --2- 172.21.15.3:0/1633691061 >> v2:172.21.15.153:3300/0 conn(0x7fd7f0157030 0x7fd7f0159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7eed9d700 1 -- 172.21.15.3:0/1633691061 >> v1:172.21.15.3:6789/0 conn(0x7fd7f015a220 legacy=0x7fd7f00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7eed9d700 1 -- 172.21.15.3:0/1633691061 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f006dba0 con 0x7fd7f0157400 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7ed59a700 1 -- 172.21.15.3:0/1633691061 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428374560 0 0) 0x7fd7f015aeb0 con 0x7fd7f015a220 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7eed9d700 1 --2- 172.21.15.3:0/1633691061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f006e3a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd7f006b660 tx=0x7fd7e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a5c8a9039bd840 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7ed59a700 1 -- 172.21.15.3:0/1633691061 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000e070 con 0x7fd7f0157400 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7ed59a700 1 -- 172.21.15.3:0/1633691061 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000bc30 con 0x7fd7f0157400 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7ed59a700 1 -- 172.21.15.3:0/1633691061 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0012430 con 0x7fd7f0157400 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7f4865700 1 -- 172.21.15.3:0/1633691061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 msgr2=0x7fd7f006e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.954+0000 7fd7f4865700 1 --2- 172.21.15.3:0/1633691061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f006e3a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd7f006b660 tx=0x7fd7e0009780 comp rx=0 tx=0).stop 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.955+0000 7fd7f4865700 1 -- 172.21.15.3:0/1633691061 shutdown_connections 2023-10-06T05:00:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.955+0000 7fd7f4865700 1 --2- 172.21.15.3:0/1633691061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f006e3a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.955+0000 7fd7f4865700 1 --2- 172.21.15.3:0/1633691061 >> v2:172.21.15.153:3300/0 conn(0x7fd7f0157030 0x7fd7f0159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.955+0000 7fd7f4865700 1 -- 172.21.15.3:0/1633691061 >> 172.21.15.3:0/1633691061 conn(0x7fd7f00ce270 msgr2=0x7fd7f00ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:20.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.955+0000 7fd7f4865700 1 -- 172.21.15.3:0/1633691061 shutdown_connections 2023-10-06T05:00:20.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.955+0000 7fd7f4865700 1 -- 172.21.15.3:0/1633691061 wait complete. 2023-10-06T05:00:20.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 Processor -- start 2023-10-06T05:00:20.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 -- start start 2023-10-06T05:00:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7f0157030 0x7fd7f00cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f00afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7f015a220 0x7fd7f0153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00cc530 con 0x7fd7f0157400 2023-10-06T05:00:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00cbfd0 con 0x7fd7f015a220 2023-10-06T05:00:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.956+0000 7fd7f4865700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00cc250 con 0x7fd7f0157030 2023-10-06T05:00:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f00afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f00afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46102/0 (socket says 172.21.15.3:46102) 2023-10-06T05:00:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 -- 172.21.15.3:0/386484460 learned_addr learned my addr 172.21.15.3:0/386484460 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7ee59c700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7f0157030 0x7fd7f00cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7eed9d700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7f015a220 0x7fd7f0153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 -- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7f0157030 msgr2=0x7fd7f00cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7f0157030 0x7fd7f00cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 -- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7f015a220 msgr2=0x7fd7f0153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7f015a220 0x7fd7f0153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 -- 172.21.15.3:0/386484460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e0009a20 con 0x7fd7f0157400 2023-10-06T05:00:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7edd9b700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f00afb80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c5b0 tx=0x7fd7d8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7e67fc700 1 -- 172.21.15.3:0/386484460 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8005b30 con 0x7fd7f0157400 2023-10-06T05:00:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7e67fc700 1 -- 172.21.15.3:0/386484460 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d8005c90 con 0x7fd7f0157400 2023-10-06T05:00:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00add40 con 0x7fd7f0157400 2023-10-06T05:00:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7e67fc700 1 -- 172.21.15.3:0/386484460 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8007320 con 0x7fd7f0157400 2023-10-06T05:00:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.957+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f00ae300 con 0x7fd7f0157400 2023-10-06T05:00:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.958+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f00644c0 con 0x7fd7f0157400 2023-10-06T05:00:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.958+0000 7fd7e67fc700 1 -- 172.21.15.3:0/386484460 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7d801b3b0 con 0x7fd7f0157400 2023-10-06T05:00:20.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.959+0000 7fd7e67fc700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7d0066f30 0x7fd7d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:20.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.959+0000 7fd7e67fc700 1 -- 172.21.15.3:0/386484460 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd7d8083b70 con 0x7fd7f0157400 2023-10-06T05:00:20.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.959+0000 7fd7ee59c700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7d0066f30 0x7fd7d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:20.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.960+0000 7fd7ee59c700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7d0066f30 0x7fd7d00693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800ab30 tx=0x7fd7e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:20.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:20.963+0000 7fd7e67fc700 1 -- 172.21.15.3:0/386484460 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7d8050c80 con 0x7fd7f0157400 2023-10-06T05:00:21.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.157+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f00ae520 con 0x7fd7f0157400 2023-10-06T05:00:21.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.158+0000 7fd7e67fc700 1 -- 172.21.15.3:0/386484460 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16832 (secure 0 0 0) 0x7fd7d8017090 con 0x7fd7f0157400 2023-10-06T05:00:21.160 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:21.160 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":108,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:55.572851+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:52.063522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:53.285974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7d0066f30 msgr2=0x7fd7d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7d0066f30 0x7fd7d00693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800ab30 tx=0x7fd7e8008040 comp rx=0 tx=0).stop 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 msgr2=0x7fd7f00afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f00afb80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c5b0 tx=0x7fd7d8001070 comp rx=0 tx=0).stop 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 shutdown_connections 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd7d0066f30 0x7fd7d00693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7f015a220 0x7fd7f0153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7f0157030 0x7fd7f00cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 --2- 172.21.15.3:0/386484460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f0157400 0x7fd7f00afb80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 >> 172.21.15.3:0/386484460 conn(0x7fd7f00ce270 msgr2=0x7fd7f0156250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:21.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 shutdown_connections 2023-10-06T05:00:21.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.160+0000 7fd7f4865700 1 -- 172.21.15.3:0/386484460 wait complete. 2023-10-06T05:00:21.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:59:52.063522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:59:53.285974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:00:21.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:00:21.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.294+0000 7fba41873700 1 Processor -- start 2023-10-06T05:00:21.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.294+0000 7fba41873700 1 -- start start 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.295+0000 7fba41873700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba3c0cc140 0x7fba3c159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.295+0000 7fba41873700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c15a330 0x7fba3c0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.295+0000 7fba41873700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c0ce420 con 0x7fba3c0cc510 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.295+0000 7fba41873700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c15a880 con 0x7fba3c0cc140 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.295+0000 7fba41873700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c15ab60 con 0x7fba3c15a330 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.295+0000 7fba3b7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c15a330 0x7fba3c0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.295+0000 7fba3affd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba3c0cc140 0x7fba3c159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3b7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c15a330 0x7fba3c0ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:43292/0 (socket says 172.21.15.3:43292) 2023-10-06T05:00:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3affd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba3c0cc140 0x7fba3c159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:59944/0 (socket says 172.21.15.3:59944) 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3b7fe700 1 -- 172.21.15.3:0/528114790 learned_addr learned my addr 172.21.15.3:0/528114790 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3a7fc700 1 --1- 172.21.15.3:0/528114790 >> v1:172.21.15.153:6789/0 conn(0x7fba3c0cc510 0x7fba3c0aef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:51170/0 (socket says 172.21.15.3:51170) 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3b7fe700 1 -- 172.21.15.3:0/528114790 >> v2:172.21.15.153:3300/0 conn(0x7fba3c0cc140 msgr2=0x7fba3c159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3b7fe700 1 --2- 172.21.15.3:0/528114790 >> v2:172.21.15.153:3300/0 conn(0x7fba3c0cc140 0x7fba3c159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3b7fe700 1 -- 172.21.15.3:0/528114790 >> v1:172.21.15.153:6789/0 conn(0x7fba3c0cc510 legacy=0x7fba3c0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3b7fe700 1 -- 172.21.15.3:0/528114790 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c0ce0b0 con 0x7fba3c15a330 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3affd700 1 --2- 172.21.15.3:0/528114790 >> v2:172.21.15.153:3300/0 conn(0x7fba3c0cc140 0x7fba3c159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba3b7fe700 1 --2- 172.21.15.3:0/528114790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c15a330 0x7fba3c0ace50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0099b0 tx=0x7fba2c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ab1bb894a92779 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba39ffb700 1 -- 172.21.15.3:0/528114790 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c009ea0 con 0x7fba3c15a330 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba39ffb700 1 -- 172.21.15.3:0/528114790 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c00b860 con 0x7fba3c15a330 2023-10-06T05:00:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.296+0000 7fba39ffb700 1 -- 172.21.15.3:0/528114790 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00bc00 con 0x7fba3c15a330 2023-10-06T05:00:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 -- 172.21.15.3:0/528114790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c15a330 msgr2=0x7fba3c0ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 --2- 172.21.15.3:0/528114790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c15a330 0x7fba3c0ace50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0099b0 tx=0x7fba2c009250 comp rx=0 tx=0).stop 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 -- 172.21.15.3:0/528114790 shutdown_connections 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 --2- 172.21.15.3:0/528114790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c15a330 0x7fba3c0ace50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 --2- 172.21.15.3:0/528114790 >> v2:172.21.15.153:3300/0 conn(0x7fba3c0cc140 0x7fba3c159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 -- 172.21.15.3:0/528114790 >> 172.21.15.3:0/528114790 conn(0x7fba3c0d03e0 msgr2=0x7fba3c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 -- 172.21.15.3:0/528114790 shutdown_connections 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.297+0000 7fba41873700 1 -- 172.21.15.3:0/528114790 wait complete. 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.298+0000 7fba41873700 1 Processor -- start 2023-10-06T05:00:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.298+0000 7fba41873700 1 -- start start 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba41873700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba3c0cc140 0x7fba3c0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba41873700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c0cc510 0x7fba3c0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba41873700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba3c15a330 0x7fba3c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba41873700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b45d0 con 0x7fba3c0cc140 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba41873700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b4fa0 con 0x7fba3c15a330 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba41873700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b39c0 con 0x7fba3c0cc510 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba3c0cc140 0x7fba3c0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba3c0cc140 0x7fba3c0b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46110/0 (socket says 172.21.15.3:46110) 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3affd700 1 -- 172.21.15.3:0/3168478553 learned_addr learned my addr 172.21.15.3:0/3168478553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:21.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3a7fc700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c0cc510 0x7fba3c0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3b7fe700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba3c15a330 0x7fba3c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3affd700 1 -- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c0cc510 msgr2=0x7fba3c0b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3affd700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c0cc510 0x7fba3c0b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3affd700 1 -- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba3c15a330 msgr2=0x7fba3c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.299+0000 7fba3affd700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba3c15a330 0x7fba3c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba3affd700 1 -- 172.21.15.3:0/3168478553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba34009a20 con 0x7fba3c0cc140 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba3b7fe700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba3c15a330 0x7fba3c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba3affd700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba3c0cc140 0x7fba3c0b0b60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c170 tx=0x7fba3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba32ffd700 1 -- 172.21.15.3:0/3168478553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3400cd00 con 0x7fba3c0cc140 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba32ffd700 1 -- 172.21.15.3:0/3168478553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3400f040 con 0x7fba3c0cc140 2023-10-06T05:00:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c0094f0 con 0x7fba3c0cc140 2023-10-06T05:00:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba32ffd700 1 -- 172.21.15.3:0/3168478553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34014690 con 0x7fba3c0cc140 2023-10-06T05:00:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.300+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c0b4000 con 0x7fba3c0cc140 2023-10-06T05:00:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.301+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c0b48b0 con 0x7fba3c0cc140 2023-10-06T05:00:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.301+0000 7fba32ffd700 1 -- 172.21.15.3:0/3168478553 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba34014a00 con 0x7fba3c0cc140 2023-10-06T05:00:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.302+0000 7fba32ffd700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba1c066f30 0x7fba1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.302+0000 7fba32ffd700 1 -- 172.21.15.3:0/3168478553 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba34085640 con 0x7fba3c0cc140 2023-10-06T05:00:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.302+0000 7fba3a7fc700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba1c066f30 0x7fba1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.302+0000 7fba3a7fc700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba1c066f30 0x7fba1c0693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fba3c15b7d0 tx=0x7fba24007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.306+0000 7fba32ffd700 1 -- 172.21.15.3:0/3168478553 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba34056140 con 0x7fba3c0cc140 2023-10-06T05:00:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.498+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba3c04c380 con 0x7fba3c0cc140 2023-10-06T05:00:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.499+0000 7fba32ffd700 1 -- 172.21.15.3:0/3168478553 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba34014cb0 con 0x7fba3c0cc140 2023-10-06T05:00:21.500 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T05:00:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba1c066f30 msgr2=0x7fba1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba1c066f30 0x7fba1c0693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fba3c15b7d0 tx=0x7fba24007eb0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba3c0cc140 msgr2=0x7fba3c0b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba3c0cc140 0x7fba3c0b0b60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c170 tx=0x7fba3400b040 comp rx=0 tx=0).stop 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 shutdown_connections 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba1c066f30 0x7fba1c0693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba3c15a330 0x7fba3c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba3c0cc140 0x7fba3c0b0b60 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 --2- 172.21.15.3:0/3168478553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba3c0cc510 0x7fba3c0b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 >> 172.21.15.3:0/3168478553 conn(0x7fba3c0d03e0 msgr2=0x7fba3c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.500+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 shutdown_connections 2023-10-06T05:00:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.501+0000 7fba41873700 1 -- 172.21.15.3:0/3168478553 wait complete. 2023-10-06T05:00:21.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:00:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.636+0000 7f14ff279700 1 Processor -- start 2023-10-06T05:00:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.636+0000 7f14ff279700 1 -- start start 2023-10-06T05:00:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14ff279700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80cfb80 0x7f14f80af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14ff279700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f14f8159eb0 0x7f14f815da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14ff279700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f815d510 con 0x7f14f80afad0 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14ff279700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f815a480 con 0x7f14f8159eb0 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14ff279700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f815a760 con 0x7f14f80cfb80 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14fc814700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14f80afad0 0x7f14f80ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55118/0 (socket says 172.21.15.3:55118) 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14fc814700 1 -- 172.21.15.3:0/2011868969 learned_addr learned my addr 172.21.15.3:0/2011868969 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14fd015700 1 --2- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80cfb80 0x7f14f80af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.637+0000 7f14fd816700 1 --2- 172.21.15.3:0/2011868969 >> v2:172.21.15.153:3300/0 conn(0x7f14f8159eb0 0x7f14f815da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14f7fff700 1 -- 172.21.15.3:0/2011868969 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846040140 0 0) 0x7f14f815d510 con 0x7f14f80afad0 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14f7fff700 1 -- 172.21.15.3:0/2011868969 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e0003570 con 0x7f14f80afad0 2023-10-06T05:00:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14fd816700 1 -- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80cfb80 msgr2=0x7f14f80af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14fd816700 1 --2- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80cfb80 0x7f14f80af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14fd816700 1 -- 172.21.15.3:0/2011868969 >> v1:172.21.15.3:6789/0 conn(0x7f14f80afad0 legacy=0x7f14f80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14fd816700 1 -- 172.21.15.3:0/2011868969 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f815aa40 con 0x7f14f8159eb0 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14fd015700 1 --2- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80cfb80 0x7f14f80af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14fd816700 1 --2- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f815da10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e60 tx=0x7f14e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9128d21c5953b22e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14f7fff700 1 -- 172.21.15.3:0/2011868969 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e800e070 con 0x7f14f8159eb0 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14f7fff700 1 -- 172.21.15.3:0/2011868969 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e800bab0 con 0x7f14f8159eb0 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.638+0000 7f14f7fff700 1 -- 172.21.15.3:0/2011868969 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e800be70 con 0x7f14f8159eb0 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 -- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 msgr2=0x7f14f815da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 --2- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f815da10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e60 tx=0x7f14e8009780 comp rx=0 tx=0).stop 2023-10-06T05:00:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 -- 172.21.15.3:0/2011868969 shutdown_connections 2023-10-06T05:00:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 --2- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f815da10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 --2- 172.21.15.3:0/2011868969 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80cfb80 0x7f14f80af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 -- 172.21.15.3:0/2011868969 >> 172.21.15.3:0/2011868969 conn(0x7f14f8070310 msgr2=0x7f14f8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 -- 172.21.15.3:0/2011868969 shutdown_connections 2023-10-06T05:00:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.639+0000 7f14ff279700 1 -- 172.21.15.3:0/2011868969 wait complete. 2023-10-06T05:00:21.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.640+0000 7f14ff279700 1 Processor -- start 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.640+0000 7f14ff279700 1 -- start start 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14ff279700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80afad0 0x7f14f816da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14ff279700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f80cfb80 0x7f14f80c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14ff279700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f80b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14ff279700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f80c86a0 con 0x7f14f80cfb80 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14ff279700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f80c7a90 con 0x7f14f8159eb0 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14ff279700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f80c7d70 con 0x7f14f80afad0 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f80cfb80 0x7f14f80c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f80cfb80 0x7f14f80c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46112/0 (socket says 172.21.15.3:46112) 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 -- 172.21.15.3:0/4250688545 learned_addr learned my addr 172.21.15.3:0/4250688545 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fd015700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80afad0 0x7f14f816da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fd816700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f80b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 -- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80afad0 msgr2=0x7f14f816da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80afad0 0x7f14f816da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 -- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 msgr2=0x7f14f80b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f80b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fc814700 1 -- 172.21.15.3:0/4250688545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0009420 con 0x7f14f80cfb80 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.641+0000 7f14fd816700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f80b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.642+0000 7f14fc814700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f80cfb80 0x7f14f80c98e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00c7c0 tx=0x7f14ec007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.642+0000 7f14f57fa700 1 -- 172.21.15.3:0/4250688545 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec005b30 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.642+0000 7f14f57fa700 1 -- 172.21.15.3:0/4250688545 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec005cb0 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.642+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e8009a20 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.642+0000 7f14f57fa700 1 -- 172.21.15.3:0/4250688545 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec007320 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.642+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f80c8330 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.643+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f80c8850 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.643+0000 7f14f57fa700 1 -- 172.21.15.3:0/4250688545 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14ec01b3b0 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.644+0000 7f14f57fa700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14e4066f30 0x7f14e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.644+0000 7f14f57fa700 1 -- 172.21.15.3:0/4250688545 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f14ec083c30 con 0x7f14f80cfb80 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.644+0000 7f14fd015700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14e4066f30 0x7f14e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.644+0000 7f14fd015700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14e4066f30 0x7f14e40693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f14f8070f00 tx=0x7f14f00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.648+0000 7f14f57fa700 1 -- 172.21.15.3:0/4250688545 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14f80c8850 con 0x7f14f80cfb80 2023-10-06T05:00:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.840+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14f804c380 con 0x7f14f80cfb80 2023-10-06T05:00:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.841+0000 7f14f57fa700 1 -- 172.21.15.3:0/4250688545 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14ec017020 con 0x7f14f80cfb80 2023-10-06T05:00:21.843 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T05:00:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14e4066f30 msgr2=0x7f14e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14e4066f30 0x7f14e40693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f14f8070f00 tx=0x7f14f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f80cfb80 msgr2=0x7f14f80c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f80cfb80 0x7f14f80c98e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00c7c0 tx=0x7f14ec007e00 comp rx=0 tx=0).stop 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 shutdown_connections 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14e4066f30 0x7f14e40693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f8159eb0 0x7f14f80b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f80afad0 0x7f14f816da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 --2- 172.21.15.3:0/4250688545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f80cfb80 0x7f14f80c98e0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 >> 172.21.15.3:0/4250688545 conn(0x7f14f8070310 msgr2=0x7f14f8157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 shutdown_connections 2023-10-06T05:00:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.843+0000 7f14ff279700 1 -- 172.21.15.3:0/4250688545 wait complete. 2023-10-06T05:00:21.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:00:21.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.977+0000 7fdbcbe91700 1 Processor -- start 2023-10-06T05:00:21.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.977+0000 7fdbcbe91700 1 -- start start 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbcbe91700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdbc4068190 0x7fdbc415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbcbe91700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc415cd40 0x7fdbc406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbcbe91700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc415d180 con 0x7fdbc4067dc0 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbcbe91700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc40691c0 con 0x7fdbc4068190 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbcbe91700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40694a0 con 0x7fdbc415cd40 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbca42e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc415cd40 0x7fdbc406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbc942c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdbc4068190 0x7fdbc415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbc942c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdbc4068190 0x7fdbc415c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:59960/0 (socket says 172.21.15.3:59960) 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbca42e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc415cd40 0x7fdbc406d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:43320/0 (socket says 172.21.15.3:43320) 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.978+0000 7fdbc942c700 1 -- 172.21.15.3:0/2574778236 learned_addr learned my addr 172.21.15.3:0/2574778236 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc942c700 1 -- 172.21.15.3:0/2574778236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc415cd40 msgr2=0x7fdbc406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc942c700 1 --2- 172.21.15.3:0/2574778236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc415cd40 0x7fdbc406d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc942c700 1 -- 172.21.15.3:0/2574778236 >> v1:172.21.15.153:6789/0 conn(0x7fdbc4067dc0 legacy=0x7fdbc40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc942c700 1 -- 172.21.15.3:0/2574778236 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4069780 con 0x7fdbc4068190 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc8c2b700 1 -- 172.21.15.3:0/2574778236 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768357684 0 0) 0x7fdbc415d180 con 0x7fdbc4067dc0 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc942c700 1 --2- 172.21.15.3:0/2574778236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc4068190 0x7fdbc415c800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80099b0 tx=0x7fdbb8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae2f48a68bef35c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc8c2b700 1 -- 172.21.15.3:0/2574778236 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8009ea0 con 0x7fdbc4068190 2023-10-06T05:00:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc8c2b700 1 -- 172.21.15.3:0/2574778236 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb800b860 con 0x7fdbc4068190 2023-10-06T05:00:21.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.979+0000 7fdbc8c2b700 1 -- 172.21.15.3:0/2574778236 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800bc00 con 0x7fdbc4068190 2023-10-06T05:00:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 -- 172.21.15.3:0/2574778236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc4068190 msgr2=0x7fdbc415c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/2574778236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc4068190 0x7fdbc415c800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80099b0 tx=0x7fdbb8009250 comp rx=0 tx=0).stop 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 -- 172.21.15.3:0/2574778236 shutdown_connections 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/2574778236 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc415cd40 0x7fdbc406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/2574778236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc4068190 0x7fdbc415c800 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 -- 172.21.15.3:0/2574778236 >> 172.21.15.3:0/2574778236 conn(0x7fdbc40ce270 msgr2=0x7fdbc40ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 -- 172.21.15.3:0/2574778236 shutdown_connections 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.980+0000 7fdbcbe91700 1 -- 172.21.15.3:0/2574778236 wait complete. 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.981+0000 7fdbcbe91700 1 Processor -- start 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.981+0000 7fdbcbe91700 1 -- start start 2023-10-06T05:00:21.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbcbe91700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbc4067dc0 0x7fdbc40c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbcbe91700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc4068190 0x7fdbc4155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbcbe91700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc415cd40 0x7fdbc40afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbcbe91700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40c8d40 con 0x7fdbc4067dc0 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbcbe91700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40c8f90 con 0x7fdbc415cd40 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbcbe91700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40c9270 con 0x7fdbc4068190 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbc4067dc0 0x7fdbc40c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbc4067dc0 0x7fdbc40c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46124/0 (socket says 172.21.15.3:46124) 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 -- 172.21.15.3:0/176799449 learned_addr learned my addr 172.21.15.3:0/176799449 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc942c700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc4068190 0x7fdbc4155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbca42e700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc415cd40 0x7fdbc40afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 -- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc4068190 msgr2=0x7fdbc4155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc4068190 0x7fdbc4155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 -- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc415cd40 msgr2=0x7fdbc40afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc415cd40 0x7fdbc40afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 -- 172.21.15.3:0/176799449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb80094f0 con 0x7fdbc4067dc0 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.982+0000 7fdbc9c2d700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbc4067dc0 0x7fdbc40c9d50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cd00 tx=0x7fdbbc001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.983+0000 7fdbc1ffb700 1 -- 172.21.15.3:0/176799449 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00ce60 con 0x7fdbc4067dc0 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.983+0000 7fdbc1ffb700 1 -- 172.21.15.3:0/176799449 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc00aac0 con 0x7fdbc4067dc0 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.983+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc40c9550 con 0x7fdbc4067dc0 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.983+0000 7fdbc1ffb700 1 -- 172.21.15.3:0/176799449 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00ae60 con 0x7fdbc4067dc0 2023-10-06T05:00:21.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.983+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc4155980 con 0x7fdbc4067dc0 2023-10-06T05:00:21.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.984+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc404c380 con 0x7fdbc4067dc0 2023-10-06T05:00:21.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.984+0000 7fdbc1ffb700 1 -- 172.21.15.3:0/176799449 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdbbc012630 con 0x7fdbc4067dc0 2023-10-06T05:00:21.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.984+0000 7fdbc1ffb700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdbb0066f30 0x7fdbb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:21.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.985+0000 7fdbc1ffb700 1 -- 172.21.15.3:0/176799449 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdbbc084db0 con 0x7fdbc4067dc0 2023-10-06T05:00:21.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.985+0000 7fdbc942c700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdbb0066f30 0x7fdbb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:21.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.985+0000 7fdbc942c700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdbb0066f30 0x7fdbb00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80099b0 tx=0x7fdbb8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:21.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:21.989+0000 7fdbc1ffb700 1 -- 172.21.15.3:0/176799449 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbbc089070 con 0x7fdbc4067dc0 2023-10-06T05:00:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.179+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdbc4069a60 con 0x7fdbc4067dc0 2023-10-06T05:00:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.179+0000 7fdbc1ffb700 1 -- 172.21.15.3:0/176799449 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdbbc051e20 con 0x7fdbc4067dc0 2023-10-06T05:00:22.181 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T05:00:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.181+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdbb0066f30 msgr2=0x7fdbb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.181+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdbb0066f30 0x7fdbb00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80099b0 tx=0x7fdbb8017000 comp rx=0 tx=0).stop 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.181+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbc4067dc0 msgr2=0x7fdbc40c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.181+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbc4067dc0 0x7fdbc40c9d50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cd00 tx=0x7fdbbc001b30 comp rx=0 tx=0).stop 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 shutdown_connections 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdbb0066f30 0x7fdbb00693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbc415cd40 0x7fdbc40afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbc4067dc0 0x7fdbc40c9d50 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 --2- 172.21.15.3:0/176799449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdbc4068190 0x7fdbc4155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 >> 172.21.15.3:0/176799449 conn(0x7fdbc40ce270 msgr2=0x7fdbc41562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 shutdown_connections 2023-10-06T05:00:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.182+0000 7fdbcbe91700 1 -- 172.21.15.3:0/176799449 wait complete. 2023-10-06T05:00:22.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:00:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.316+0000 7efc46c76700 1 Processor -- start 2023-10-06T05:00:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.316+0000 7efc46c76700 1 -- start start 2023-10-06T05:00:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.316+0000 7efc46c76700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc40150690 0x7efc40150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.316+0000 7efc46c76700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efc400cba50 0x7efc4015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.316+0000 7efc46c76700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4015da80 con 0x7efc4015a720 2023-10-06T05:00:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.316+0000 7efc46c76700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4015a290 con 0x7efc400cba50 2023-10-06T05:00:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.316+0000 7efc46c76700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc400d0020 con 0x7efc40150690 2023-10-06T05:00:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc45213700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efc4015a720 0x7efc400af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52146/0 (socket says 172.21.15.3:52146) 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc45213700 1 -- 172.21.15.3:0/1751403545 learned_addr learned my addr 172.21.15.3:0/1751403545 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc44a12700 1 --2- 172.21.15.3:0/1751403545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc40150690 0x7efc40150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc3ffff700 1 --2- 172.21.15.3:0/1751403545 >> v2:172.21.15.153:3300/0 conn(0x7efc400cba50 0x7efc4015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc3f7fe700 1 -- 172.21.15.3:0/1751403545 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881204117 0 0) 0x7efc4015da80 con 0x7efc4015a720 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc3f7fe700 1 -- 172.21.15.3:0/1751403545 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc2c003570 con 0x7efc4015a720 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc3f7fe700 1 -- 172.21.15.3:0/1751403545 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2709290046 0 0) 0x7efc2c003570 con 0x7efc4015a720 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc44a12700 1 -- 172.21.15.3:0/1751403545 >> v2:172.21.15.153:3300/0 conn(0x7efc400cba50 msgr2=0x7efc4015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc44a12700 1 --2- 172.21.15.3:0/1751403545 >> v2:172.21.15.153:3300/0 conn(0x7efc400cba50 0x7efc4015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc44a12700 1 -- 172.21.15.3:0/1751403545 >> v1:172.21.15.3:6789/0 conn(0x7efc4015a720 legacy=0x7efc400af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc44a12700 1 -- 172.21.15.3:0/1751403545 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40159f40 con 0x7efc40150690 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.317+0000 7efc44a12700 1 --2- 172.21.15.3:0/1751403545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc40150690 0x7efc40150a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc38009e60 tx=0x7efc38009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46fc02b77a7290f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc3f7fe700 1 -- 172.21.15.3:0/1751403545 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800e070 con 0x7efc40150690 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc3f7fe700 1 -- 172.21.15.3:0/1751403545 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3800b7a0 con 0x7efc40150690 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc3f7fe700 1 -- 172.21.15.3:0/1751403545 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30001470 con 0x7efc40150690 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 -- 172.21.15.3:0/1751403545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc40150690 msgr2=0x7efc40150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 --2- 172.21.15.3:0/1751403545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc40150690 0x7efc40150a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc38009e60 tx=0x7efc38009780 comp rx=0 tx=0).stop 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 -- 172.21.15.3:0/1751403545 shutdown_connections 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 --2- 172.21.15.3:0/1751403545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc40150690 0x7efc40150a60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 --2- 172.21.15.3:0/1751403545 >> v2:172.21.15.153:3300/0 conn(0x7efc400cba50 0x7efc4015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 -- 172.21.15.3:0/1751403545 >> 172.21.15.3:0/1751403545 conn(0x7efc40070310 msgr2=0x7efc40070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 -- 172.21.15.3:0/1751403545 shutdown_connections 2023-10-06T05:00:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.318+0000 7efc46c76700 1 -- 172.21.15.3:0/1751403545 wait complete. 2023-10-06T05:00:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.319+0000 7efc46c76700 1 Processor -- start 2023-10-06T05:00:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.319+0000 7efc46c76700 1 -- start start 2023-10-06T05:00:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc46c76700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc400cba50 0x7efc4015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc46c76700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc40150690 0x7efc4014d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc46c76700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc4015a720 0x7efc400b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc46c76700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4015ab40 con 0x7efc40150690 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc46c76700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4015b9a0 con 0x7efc400cba50 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc46c76700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4015cf00 con 0x7efc4015a720 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc40150690 0x7efc4014d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc40150690 0x7efc4014d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33910/0 (socket says 172.21.15.3:33910) 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc3ffff700 1 -- 172.21.15.3:0/1291904339 learned_addr learned my addr 172.21.15.3:0/1291904339 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc44a12700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc400cba50 0x7efc4015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.320+0000 7efc45213700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc4015a720 0x7efc400b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3ffff700 1 -- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc4015a720 msgr2=0x7efc400b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3ffff700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc4015a720 0x7efc400b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3ffff700 1 -- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc400cba50 msgr2=0x7efc4015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3ffff700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc400cba50 0x7efc4015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3ffff700 1 -- 172.21.15.3:0/1291904339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc38009a20 con 0x7efc40150690 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3ffff700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc40150690 0x7efc4014d9d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efc2800ec80 tx=0x7efc280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3cff9700 1 -- 172.21.15.3:0/1291904339 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800c850 con 0x7efc40150690 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3cff9700 1 -- 172.21.15.3:0/1291904339 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2800c9b0 con 0x7efc40150690 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc3cff9700 1 -- 172.21.15.3:0/1291904339 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800cd50 con 0x7efc40150690 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4015ad90 con 0x7efc40150690 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.321+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4015b350 con 0x7efc40150690 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.322+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4015bb10 con 0x7efc40150690 2023-10-06T05:00:22.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.322+0000 7efc3cff9700 1 -- 172.21.15.3:0/1291904339 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc28013680 con 0x7efc40150690 2023-10-06T05:00:22.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.323+0000 7efc3cff9700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.323+0000 7efc3cff9700 1 -- 172.21.15.3:0/1291904339 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efc280848f0 con 0x7efc40150690 2023-10-06T05:00:22.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.323+0000 7efc44a12700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.323+0000 7efc44a12700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efc20066f30 0x7efc200693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efc380099f0 tx=0x7efc38018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:22.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.327+0000 7efc3cff9700 1 -- 172.21.15.3:0/1291904339 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc28051a00 con 0x7efc40150690 2023-10-06T05:00:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.518+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc4004c380 con 0x7efc40150690 2023-10-06T05:00:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.519+0000 7efc3cff9700 1 -- 172.21.15.3:0/1291904339 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc28055020 con 0x7efc40150690 2023-10-06T05:00:22.521 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T05:00:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efc20066f30 msgr2=0x7efc200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efc20066f30 0x7efc200693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efc380099f0 tx=0x7efc38018000 comp rx=0 tx=0).stop 2023-10-06T05:00:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc40150690 msgr2=0x7efc4014d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc40150690 0x7efc4014d9d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efc2800ec80 tx=0x7efc280091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 shutdown_connections 2023-10-06T05:00:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc400cba50 0x7efc4015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efc4015a720 0x7efc400b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 --2- 172.21.15.3:0/1291904339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc40150690 0x7efc4014d9d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 >> 172.21.15.3:0/1291904339 conn(0x7efc40070310 msgr2=0x7efc4014f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 shutdown_connections 2023-10-06T05:00:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.521+0000 7efc46c76700 1 -- 172.21.15.3:0/1291904339 wait complete. 2023-10-06T05:00:22.535 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:22.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.652+0000 7f73111a9700 1 Processor -- start 2023-10-06T05:00:22.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.652+0000 7f73111a9700 1 -- start start 2023-10-06T05:00:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f73111a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f730c157700 0x7f730c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f73111a9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f730c0b1b10 0x7f730c067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f73111a9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730c0b2e50 con 0x7f730c157ad0 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f73111a9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730c158290 con 0x7f730c157700 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f73111a9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730c0b1fc0 con 0x7f730c0b1b10 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f730ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f730c157700 0x7f730c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f730a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f730c157ad0 0x7f730c0b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52148/0 (socket says 172.21.15.3:52148) 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f730a59c700 1 -- 172.21.15.3:0/869754907 learned_addr learned my addr 172.21.15.3:0/869754907 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.653+0000 7f730ad9d700 1 --2- 172.21.15.3:0/869754907 >> v2:172.21.15.3:3300/0 conn(0x7f730c157700 0x7f730c0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33914/0 (socket says 172.21.15.3:33914) 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f730ad9d700 1 -- 172.21.15.3:0/869754907 >> v2:172.21.15.153:3300/0 conn(0x7f730c0b1b10 msgr2=0x7f730c067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f730ad9d700 1 --2- 172.21.15.3:0/869754907 >> v2:172.21.15.153:3300/0 conn(0x7f730c0b1b10 0x7f730c067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f730ad9d700 1 -- 172.21.15.3:0/869754907 >> v1:172.21.15.3:6789/0 conn(0x7f730c157ad0 legacy=0x7f730c0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f730ad9d700 1 -- 172.21.15.3:0/869754907 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c0b22a0 con 0x7f730c157700 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f7309d9b700 1 -- 172.21.15.3:0/869754907 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422600560 0 0) 0x7f730c0b2e50 con 0x7f730c157ad0 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f730ad9d700 1 --2- 172.21.15.3:0/869754907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157700 0x7f730c0cc270 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f730c0cbdb0 tx=0x7f72fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff45b7da773fccb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f7309d9b700 1 -- 172.21.15.3:0/869754907 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00b6b0 con 0x7f730c157700 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f7309d9b700 1 -- 172.21.15.3:0/869754907 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc00b810 con 0x7f730c157700 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f7309d9b700 1 -- 172.21.15.3:0/869754907 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00bbb0 con 0x7f730c157700 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 -- 172.21.15.3:0/869754907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157700 msgr2=0x7f730c0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 --2- 172.21.15.3:0/869754907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157700 0x7f730c0cc270 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f730c0cbdb0 tx=0x7f72fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 -- 172.21.15.3:0/869754907 shutdown_connections 2023-10-06T05:00:22.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 --2- 172.21.15.3:0/869754907 >> v2:172.21.15.153:3300/0 conn(0x7f730c0b1b10 0x7f730c067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 --2- 172.21.15.3:0/869754907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157700 0x7f730c0cc270 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 -- 172.21.15.3:0/869754907 >> 172.21.15.3:0/869754907 conn(0x7f730c0ce2c0 msgr2=0x7f730c0ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:22.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 -- 172.21.15.3:0/869754907 shutdown_connections 2023-10-06T05:00:22.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.654+0000 7f73111a9700 1 -- 172.21.15.3:0/869754907 wait complete. 2023-10-06T05:00:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 Processor -- start 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 -- start start 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f730c0b1b10 0x7f730c14c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f730c157700 0x7f730c158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157ad0 0x7f730c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c16e440 con 0x7f730c157ad0 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c16d7c0 con 0x7f730c0b1b10 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.656+0000 7f73111a9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c16daa0 con 0x7f730c157700 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157ad0 0x7f730c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157ad0 0x7f730c158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33922/0 (socket says 172.21.15.3:33922) 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 -- 172.21.15.3:0/1222620078 learned_addr learned my addr 172.21.15.3:0/1222620078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730a59c700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f730c157700 0x7f730c158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730ad9d700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f730c0b1b10 0x7f730c14c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 -- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f730c157700 msgr2=0x7f730c158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f730c157700 0x7f730c158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 -- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f730c0b1b10 msgr2=0x7f730c14c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f730c0b1b10 0x7f730c14c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 -- 172.21.15.3:0/1222620078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc009420 con 0x7f730c157ad0 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f730b59e700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157ad0 0x7f730c158f80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f730400bb60 tx=0x7f7304009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f7302ffd700 1 -- 172.21.15.3:0/1222620078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400c940 con 0x7f730c157ad0 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f7302ffd700 1 -- 172.21.15.3:0/1222620078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730400caa0 con 0x7f730c157ad0 2023-10-06T05:00:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730c16dd80 con 0x7f730c157ad0 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f7302ffd700 1 -- 172.21.15.3:0/1222620078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400ce60 con 0x7f730c157ad0 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.657+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c14cc10 con 0x7f730c157ad0 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.658+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c0644c0 con 0x7f730c157ad0 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.658+0000 7f7302ffd700 1 -- 172.21.15.3:0/1222620078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7304014620 con 0x7f730c157ad0 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.659+0000 7f7302ffd700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72ec066f30 0x7f72ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.659+0000 7f7302ffd700 1 -- 172.21.15.3:0/1222620078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f73040846a0 con 0x7f730c157ad0 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.659+0000 7f730ad9d700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72ec066f30 0x7f72ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.660+0000 7f730ad9d700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72ec066f30 0x7f72ec0693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0093f0 tx=0x7f72fc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:22.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.663+0000 7f7302ffd700 1 -- 172.21.15.3:0/1222620078 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f730404c5a0 con 0x7f730c157ad0 2023-10-06T05:00:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.902+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f730c16e580 con 0x7f730c157ad0 2023-10-06T05:00:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.903+0000 7f7302ffd700 1 -- 172.21.15.3:0/1222620078 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f73040517b0 con 0x7f730c157ad0 2023-10-06T05:00:22.905 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:22.905 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T05:00:19.596165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":9,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:00:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72ec066f30 msgr2=0x7f72ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72ec066f30 0x7f72ec0693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0093f0 tx=0x7f72fc017000 comp rx=0 tx=0).stop 2023-10-06T05:00:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157ad0 msgr2=0x7f730c158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157ad0 0x7f730c158f80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f730400bb60 tx=0x7f7304009810 comp rx=0 tx=0).stop 2023-10-06T05:00:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 shutdown_connections 2023-10-06T05:00:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f72ec066f30 0x7f72ec0693e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f730c0b1b10 0x7f730c14c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c157ad0 0x7f730c158f80 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 --2- 172.21.15.3:0/1222620078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f730c157700 0x7f730c158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:22.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 >> 172.21.15.3:0/1222620078 conn(0x7f730c0ce2c0 msgr2=0x7f730c14d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:22.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 shutdown_connections 2023-10-06T05:00:22.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:22.905+0000 7f73111a9700 1 -- 172.21.15.3:0/1222620078 wait complete. 2023-10-06T05:00:22.908 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 72 2023-10-06T05:00:22.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:00:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.039+0000 7f9aecab8700 1 Processor -- start 2023-10-06T05:00:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.039+0000 7f9aecab8700 1 -- start start 2023-10-06T05:00:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.040+0000 7f9aecab8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 0x7f9ae80aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.040+0000 7f9aecab8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9ae80ac310 0x7f9ae8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.040+0000 7f9aecab8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae80abee0 con 0x7f9ae806af30 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.040+0000 7f9aecab8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae806aaa0 con 0x7f9ae80ac310 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.040+0000 7f9aecab8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8159f80 con 0x7f9ae80a9ee0 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.040+0000 7f9ae7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9ae806af30 0x7f9ae806c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52164/0 (socket says 172.21.15.3:52164) 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.040+0000 7f9ae7fff700 1 -- 172.21.15.3:0/3152287186 learned_addr learned my addr 172.21.15.3:0/3152287186 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae77fe700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 0x7f9ae80aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/3152287186 >> v2:172.21.15.153:3300/0 conn(0x7f9ae80ac310 0x7f9ae8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350248476 0 0) 0x7f9ae80abee0 con 0x7f9ae806af30 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ad0003570 con 0x7f9ae806af30 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 145300956 0 0) 0x7f9ad0003570 con 0x7f9ae806af30 2023-10-06T05:00:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 msgr2=0x7f9ae80aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 0x7f9ae80aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 >> v2:172.21.15.153:3300/0 conn(0x7f9ae80ac310 msgr2=0x7f9ae8159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 --2- 172.21.15.3:0/3152287186 >> v2:172.21.15.153:3300/0 conn(0x7f9ae80ac310 0x7f9ae8159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae815a1e0 con 0x7f9ae806af30 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f9ae00017f0 con 0x7f9ae806af30 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 >> v1:172.21.15.3:6789/0 conn(0x7f9ae806af30 legacy=0x7f9ae806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 0x7f9ad0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ad0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad0004d50 0x7f9ad00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae806aaa0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8159f80 con 0x7f9ad0004d50 2023-10-06T05:00:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad000ba90 con 0x7f9ae80a9ee0 2023-10-06T05:00:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ad0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae77fe700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 0x7f9ad0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae7fff700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad0004d50 0x7f9ad00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae6ffd700 1 -- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 msgr2=0x7f9ad0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80a9ee0 0x7f9ad0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae6ffd700 1 -- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad0004d50 msgr2=0x7f9ad00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.041+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad0004d50 0x7f9ad00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9ae6ffd700 1 -- 172.21.15.3:0/3152287186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad0003570 con 0x7f9ad0003fe0 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9ae7fff700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad0004d50 0x7f9ad00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ad0004810 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009c00 tx=0x7f9adc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8152fbd258db90e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00e6b0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc00e810 con 0x7f9ad0003fe0 2023-10-06T05:00:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9ae67fc700 1 -- 172.21.15.3:0/3152287186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00ebd0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9aecab8700 1 -- 172.21.15.3:0/3152287186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 msgr2=0x7f9ad0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9aecab8700 1 --2- 172.21.15.3:0/3152287186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ad0004810 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009c00 tx=0x7f9adc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9ae7fff700 1 -- 172.21.15.3:0/3152287186 reap_dead start 2023-10-06T05:00:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9aecab8700 1 -- 172.21.15.3:0/3152287186 shutdown_connections 2023-10-06T05:00:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9aecab8700 1 -- 172.21.15.3:0/3152287186 >> 172.21.15.3:0/3152287186 conn(0x7f9ae80ccfd0 msgr2=0x7f9ae80cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9aecab8700 1 -- 172.21.15.3:0/3152287186 shutdown_connections 2023-10-06T05:00:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.042+0000 7f9aecab8700 1 -- 172.21.15.3:0/3152287186 wait complete. 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 Processor -- start 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 -- start start 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ae8169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ae81537f0 0x7f9ae80cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80cd920 0x7f9ae80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae806e0e0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae816a400 con 0x7f9ae81537f0 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9aecab8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80ca4f0 con 0x7f9ae80cd920 2023-10-06T05:00:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.044+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ae8169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ae8169ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33942/0 (socket says 172.21.15.3:33942) 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 -- 172.21.15.3:0/2905251045 learned_addr learned my addr 172.21.15.3:0/2905251045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae7fff700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80cd920 0x7f9ae80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ae81537f0 0x7f9ae80cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 -- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80cd920 msgr2=0x7f9ae80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80cd920 0x7f9ae80cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 -- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ae81537f0 msgr2=0x7f9ae80cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ae81537f0 0x7f9ae80cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 -- 172.21.15.3:0/2905251045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80c98e0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ae81537f0 0x7f9ae80cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9ae77fe700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ae8169ec0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011d40 tx=0x7f9ad80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9acf7fe700 1 -- 172.21.15.3:0/2905251045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800fac0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9acf7fe700 1 -- 172.21.15.3:0/2905251045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad800fc20 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae80c9bc0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9acf7fe700 1 -- 172.21.15.3:0/2905251045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8014440 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.045+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae80ca180 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.046+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae80ca7d0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.047+0000 7f9acf7fe700 1 -- 172.21.15.3:0/2905251045 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ad8014870 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.047+0000 7f9acf7fe700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ad4066f30 0x7f9ad40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.047+0000 7f9acf7fe700 1 -- 172.21.15.3:0/2905251045 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ad8018030 con 0x7f9ad0003fe0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.047+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ad4066f30 0x7f9ad40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.048+0000 7f9ae6ffd700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ad4066f30 0x7f9ad40693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0091e0 tx=0x7f9adc01a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.051+0000 7f9acf7fe700 1 -- 172.21.15.3:0/2905251045 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ad80521c0 con 0x7f9ad0003fe0 2023-10-06T05:00:23.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.246+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ae804c380 con 0x7f9ad0003fe0 2023-10-06T05:00:23.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.247+0000 7f9acf7fe700 1 -- 172.21.15.3:0/2905251045 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16832 (secure 0 0 0) 0x7f9ad8022090 con 0x7f9ad0003fe0 2023-10-06T05:00:23.249 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:23.249 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":108,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:55.572851+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:52.063522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:53.285974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:00:23.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ad4066f30 msgr2=0x7f9ad40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ad4066f30 0x7f9ad40693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0091e0 tx=0x7f9adc01a000 comp rx=0 tx=0).stop 2023-10-06T05:00:23.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 msgr2=0x7f9ae8169ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ae8169ec0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011d40 tx=0x7f9ad80091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 shutdown_connections 2023-10-06T05:00:23.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ad4066f30 0x7f9ad40693e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ae80cd920 0x7f9ae80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ad0003fe0 0x7f9ae8169ec0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 --2- 172.21.15.3:0/2905251045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ae81537f0 0x7f9ae80cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 >> 172.21.15.3:0/2905251045 conn(0x7f9ae80ccfd0 msgr2=0x7f9ae814dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:23.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 shutdown_connections 2023-10-06T05:00:23.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.249+0000 7f9aecab8700 1 -- 172.21.15.3:0/2905251045 wait complete. 2023-10-06T05:00:23.263 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:00:23.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:23.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.382+0000 7f9cf1932700 1 Processor -- start 2023-10-06T05:00:23.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.382+0000 7f9cf1932700 1 -- start start 2023-10-06T05:00:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9cf1932700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9cec0cb3b0 0x7f9cec0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9cf1932700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec0cf130 con 0x7f9cec0cb780 2023-10-06T05:00:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9cf1932700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec0ce520 con 0x7f9cec1596d0 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9cf1932700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec0ce800 con 0x7f9cec0cb3b0 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9cebfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9cec0cb780 0x7f9cec158fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52168/0 (socket says 172.21.15.3:52168) 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9cebfff700 1 -- 172.21.15.3:0/471361607 learned_addr learned my addr 172.21.15.3:0/471361607 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9cf0930700 1 --2- 172.21.15.3:0/471361607 >> v2:172.21.15.153:3300/0 conn(0x7f9cec0cb3b0 0x7f9cec0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412210729 0 0) 0x7f9cec0cf130 con 0x7f9cec0cb780 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.383+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cdc003570 con 0x7f9cec0cb780 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137826549 0 0) 0x7f9cdc003570 con 0x7f9cec0cb780 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 >> v2:172.21.15.153:3300/0 conn(0x7f9cec0cb3b0 msgr2=0x7f9cec0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 --2- 172.21.15.3:0/471361607 >> v2:172.21.15.153:3300/0 conn(0x7f9cec0cb3b0 0x7f9cec0cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 >> v1:172.21.15.153:6789/0 conn(0x7f9cec1596d0 legacy=0x7f9cec0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0ceae0 con 0x7f9cec0cb780 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f9cd8001470 con 0x7f9cec0cb780 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 >> v1:172.21.15.3:6789/0 conn(0x7f9cec0cb780 legacy=0x7f9cec158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cdc003a60 0x7f9cdc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cdc004810 0x7f9cdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc0051c0 0x7f9cdc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0ce800 con 0x7f9cdc004810 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc00c160 con 0x7f9cdc0051c0 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc00c320 con 0x7f9cdc003a60 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cebfff700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cdc004810 0x7f9cdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cf0930700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cdc003a60 0x7f9cdc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cf1131700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc0051c0 0x7f9cdc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cebfff700 1 -- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cdc003a60 msgr2=0x7f9cdc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cebfff700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cdc003a60 0x7f9cdc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cebfff700 1 -- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc0051c0 msgr2=0x7f9cdc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cebfff700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc0051c0 0x7f9cdc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cebfff700 1 -- 172.21.15.3:0/471361607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc003570 con 0x7f9cdc004810 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cf1131700 1 -- 172.21.15.3:0/471361607 reap_dead start 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9cebfff700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cdc004810 0x7f9cdc004c80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800c3c0 tx=0x7f9cd8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e756649f947a7409 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.384+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800cc90 con 0x7f9cdc004810 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd800cdf0 con 0x7f9cdc004810 2023-10-06T05:00:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9ceb7fe700 1 -- 172.21.15.3:0/471361607 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8005b50 con 0x7f9cdc004810 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9cf1932700 1 -- 172.21.15.3:0/471361607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cdc004810 msgr2=0x7f9cdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9cf1932700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cdc004810 0x7f9cdc004c80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800c3c0 tx=0x7f9cd8001e50 comp rx=0 tx=0).stop 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9cf1932700 1 -- 172.21.15.3:0/471361607 shutdown_connections 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9cf1932700 1 --2- 172.21.15.3:0/471361607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cdc004810 0x7f9cdc004c80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9cf1932700 1 -- 172.21.15.3:0/471361607 >> 172.21.15.3:0/471361607 conn(0x7f9cec070310 msgr2=0x7f9cec070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9cf1932700 1 -- 172.21.15.3:0/471361607 shutdown_connections 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.385+0000 7f9cf1932700 1 -- 172.21.15.3:0/471361607 wait complete. 2023-10-06T05:00:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.386+0000 7f9cf1932700 1 Processor -- start 2023-10-06T05:00:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.386+0000 7f9cf1932700 1 -- start start 2023-10-06T05:00:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1932700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc004810 0x7f9cec0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1932700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0c9eb0 0x7f9cec157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1932700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec156130 0x7f9cec0b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1932700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0c7360 con 0x7f9cec156130 2023-10-06T05:00:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1932700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0c75b0 con 0x7f9cdc004810 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1932700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0c7890 con 0x7f9cec0c9eb0 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1131700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec156130 0x7f9cec0b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1131700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec156130 0x7f9cec0b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33962/0 (socket says 172.21.15.3:33962) 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf1131700 1 -- 172.21.15.3:0/4266936589 learned_addr learned my addr 172.21.15.3:0/4266936589 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cebfff700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0c9eb0 0x7f9cec157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.387+0000 7f9cf0930700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc004810 0x7f9cec0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1131700 1 -- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0c9eb0 msgr2=0x7f9cec157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1131700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0c9eb0 0x7f9cec157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1131700 1 -- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc004810 msgr2=0x7f9cec0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1131700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc004810 0x7f9cec0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1131700 1 -- 172.21.15.3:0/4266936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0c7b70 con 0x7f9cec156130 2023-10-06T05:00:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1131700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec156130 0x7f9cec0b4b20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9cec089680 tx=0x7f9ce4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9ce8ff9700 1 -- 172.21.15.3:0/4266936589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4006db0 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9ce8ff9700 1 -- 172.21.15.3:0/4266936589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce400c5a0 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec0c7e50 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9ce8ff9700 1 -- 172.21.15.3:0/4266936589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400c960 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.388+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec158490 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.389+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec04c380 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.389+0000 7f9ce8ff9700 1 -- 172.21.15.3:0/4266936589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ce400cd50 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.390+0000 7f9ce8ff9700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cd0066f30 0x7f9cd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.390+0000 7f9ce8ff9700 1 -- 172.21.15.3:0/4266936589 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ce4084420 con 0x7f9cec156130 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.390+0000 7f9cf0930700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cd0066f30 0x7f9cd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.390+0000 7f9cf0930700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cd0066f30 0x7f9cd00693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9cec066cc0 tx=0x7f9ce00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.394+0000 7f9ce8ff9700 1 -- 172.21.15.3:0/4266936589 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ce4051530 con 0x7f9cec156130 2023-10-06T05:00:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.631+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cec0cedc0 con 0x7f9cec156130 2023-10-06T05:00:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.631+0000 7f9ce8ff9700 1 -- 172.21.15.3:0/4266936589 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f9ce4054b50 con 0x7f9cec156130 2023-10-06T05:00:23.633 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:23.633 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T05:00:19.596165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":9,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cd0066f30 msgr2=0x7f9cd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cd0066f30 0x7f9cd00693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9cec066cc0 tx=0x7f9ce00091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec156130 msgr2=0x7f9cec0b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec156130 0x7f9cec0b4b20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9cec089680 tx=0x7f9ce4001640 comp rx=0 tx=0).stop 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 shutdown_connections 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cd0066f30 0x7f9cd00693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cdc004810 0x7f9cec0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cec156130 0x7f9cec0b4b20 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 --2- 172.21.15.3:0/4266936589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0c9eb0 0x7f9cec157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 >> 172.21.15.3:0/4266936589 conn(0x7f9cec070310 msgr2=0x7f9cec154e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 shutdown_connections 2023-10-06T05:00:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.634+0000 7f9cf1932700 1 -- 172.21.15.3:0/4266936589 wait complete. 2023-10-06T05:00:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 72 2023-10-06T05:00:23.648 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:23.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.766+0000 7f1a4462e700 1 Processor -- start 2023-10-06T05:00:23.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.766+0000 7f1a4462e700 1 -- start start 2023-10-06T05:00:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.766+0000 7f1a4462e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1a3c0cd950 0x7f1a3c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.766+0000 7f1a4462e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a3c0b2740 0x7f1a3c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.766+0000 7f1a4462e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c0b0ce0 con 0x7f1a3c15cb50 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.766+0000 7f1a4462e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c0cb820 con 0x7f1a3c0b2740 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.766+0000 7f1a4462e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c0b0760 con 0x7f1a3c0cd950 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a3c0b2740 0x7f1a3c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a3c0b2740 0x7f1a3c0cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33976/0 (socket says 172.21.15.3:33976) 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 -- 172.21.15.3:0/3053154493 learned_addr learned my addr 172.21.15.3:0/3053154493 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a41bc9700 1 --2- 172.21.15.3:0/3053154493 >> v2:172.21.15.153:3300/0 conn(0x7f1a3c0cd950 0x7f1a3c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 -- 172.21.15.3:0/3053154493 >> v2:172.21.15.153:3300/0 conn(0x7f1a3c0cd950 msgr2=0x7f1a3c0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 --2- 172.21.15.3:0/3053154493 >> v2:172.21.15.153:3300/0 conn(0x7f1a3c0cd950 0x7f1a3c0b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 -- 172.21.15.3:0/3053154493 >> v1:172.21.15.153:6789/0 conn(0x7f1a3c15cb50 legacy=0x7f1a3c0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 -- 172.21.15.3:0/3053154493 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0b0a00 con 0x7f1a3c0b2740 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a42bcb700 1 --2- 172.21.15.3:0/3053154493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cbcb0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0c73f0 tx=0x7f1a34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d3495b439f7b90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a413c8700 1 -- 172.21.15.3:0/3053154493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400b860 con 0x7f1a3c0b2740 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a413c8700 1 -- 172.21.15.3:0/3053154493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3400b9c0 con 0x7f1a3c0b2740 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.767+0000 7f1a413c8700 1 -- 172.21.15.3:0/3053154493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400bd60 con 0x7f1a3c0b2740 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 -- 172.21.15.3:0/3053154493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 msgr2=0x7f1a3c0cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 --2- 172.21.15.3:0/3053154493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cbcb0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0c73f0 tx=0x7f1a34009250 comp rx=0 tx=0).stop 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 -- 172.21.15.3:0/3053154493 shutdown_connections 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 --2- 172.21.15.3:0/3053154493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cbcb0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 --2- 172.21.15.3:0/3053154493 >> v2:172.21.15.153:3300/0 conn(0x7f1a3c0cd950 0x7f1a3c0b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 -- 172.21.15.3:0/3053154493 >> 172.21.15.3:0/3053154493 conn(0x7f1a3c070310 msgr2=0x7f1a3c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 -- 172.21.15.3:0/3053154493 shutdown_connections 2023-10-06T05:00:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.768+0000 7f1a4462e700 1 -- 172.21.15.3:0/3053154493 wait complete. 2023-10-06T05:00:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.769+0000 7f1a4462e700 1 Processor -- start 2023-10-06T05:00:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.769+0000 7f1a4462e700 1 -- start start 2023-10-06T05:00:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a4462e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a4462e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a3c0cd950 0x7f1a3c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a4462e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3c15cb50 0x7f1a3c15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a4462e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c16a080 con 0x7f1a3c0b2740 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a4462e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0ce610 con 0x7f1a3c15cb50 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a4462e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0ce8f0 con 0x7f1a3c0cd950 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33980/0 (socket says 172.21.15.3:33980) 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 -- 172.21.15.3:0/2373487127 learned_addr learned my addr 172.21.15.3:0/2373487127 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a42bcb700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3c15cb50 0x7f1a3c15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a41bc9700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a3c0cd950 0x7f1a3c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 -- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a3c0cd950 msgr2=0x7f1a3c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a3c0cd950 0x7f1a3c0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 -- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3c15cb50 msgr2=0x7f1a3c15ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3c15cb50 0x7f1a3c15ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 -- 172.21.15.3:0/2373487127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340094f0 con 0x7f1a3c0b2740 2023-10-06T05:00:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a423ca700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cdf80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cc70 tx=0x7f1a2c001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a3a7fc700 1 -- 172.21.15.3:0/2373487127 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00a940 con 0x7f1a3c0b2740 2023-10-06T05:00:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.770+0000 7f1a3a7fc700 1 -- 172.21.15.3:0/2373487127 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c00aac0 con 0x7f1a3c0b2740 2023-10-06T05:00:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.771+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c0cebd0 con 0x7f1a3c0b2740 2023-10-06T05:00:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.771+0000 7f1a3a7fc700 1 -- 172.21.15.3:0/2373487127 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00ae80 con 0x7f1a3c0b2740 2023-10-06T05:00:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.771+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c169cd0 con 0x7f1a3c0b2740 2023-10-06T05:00:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.772+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c071680 con 0x7f1a3c0b2740 2023-10-06T05:00:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.772+0000 7f1a3a7fc700 1 -- 172.21.15.3:0/2373487127 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a2c012630 con 0x7f1a3c0b2740 2023-10-06T05:00:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.772+0000 7f1a3a7fc700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a28066f30 0x7f1a280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.772+0000 7f1a3a7fc700 1 -- 172.21.15.3:0/2373487127 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a2c084cd0 con 0x7f1a3c0b2740 2023-10-06T05:00:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.772+0000 7f1a41bc9700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a28066f30 0x7f1a280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.773+0000 7f1a41bc9700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a28066f30 0x7f1a280693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0d0f80 tx=0x7f1a30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:23.777+0000 7f1a3a7fc700 1 -- 172.21.15.3:0/2373487127 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a2c051de0 con 0x7f1a3c0b2740 2023-10-06T05:00:24.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.015+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a3c0cb1d0 con 0x7f1a3c0b2740 2023-10-06T05:00:24.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.016+0000 7f1a3a7fc700 1 -- 172.21.15.3:0/2373487127 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f1a2c055400 con 0x7f1a3c0b2740 2023-10-06T05:00:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:24.018 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:59:54.313543+0000","modified":"2023-10-06T05:00:19.596165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":9,"addr":"172.21.15.3:6835/237919975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":237919975},{"type":"v1","addr":"172.21.15.3:6835","nonce":237919975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:00:24.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a28066f30 msgr2=0x7f1a280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a28066f30 0x7f1a280693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0d0f80 tx=0x7f1a30008040 comp rx=0 tx=0).stop 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 msgr2=0x7f1a3c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cdf80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cc70 tx=0x7f1a2c001af0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 shutdown_connections 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a28066f30 0x7f1a280693e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3c15cb50 0x7f1a3c15ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3c0b2740 0x7f1a3c0cdf80 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 --2- 172.21.15.3:0/2373487127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a3c0cd950 0x7f1a3c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 >> 172.21.15.3:0/2373487127 conn(0x7f1a3c070310 msgr2=0x7f1a3c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 shutdown_connections 2023-10-06T05:00:24.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.018+0000 7f1a4462e700 1 -- 172.21.15.3:0/2373487127 wait complete. 2023-10-06T05:00:24.021 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 72 2023-10-06T05:00:24.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:00:24.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.148+0000 7f703e1a6700 1 Processor -- start 2023-10-06T05:00:24.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.148+0000 7f703e1a6700 1 -- start start 2023-10-06T05:00:24.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.149+0000 7f703e1a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7038156410 0x7f70380af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.149+0000 7f703e1a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70380b1c90 con 0x7f703806b7d0 2023-10-06T05:00:24.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.149+0000 7f703e1a6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70380b1080 con 0x7f70380b0900 2023-10-06T05:00:24.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.149+0000 7f703e1a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70380b1360 con 0x7f7038156410 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.149+0000 7f70377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7038156410 0x7f70380af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.149+0000 7f7037fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f703806b7d0 0x7f70380c9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52176/0 (socket says 172.21.15.3:52176) 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.149+0000 7f7037fff700 1 -- 172.21.15.3:0/3541328706 learned_addr learned my addr 172.21.15.3:0/3541328706 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70377fe700 1 -- 172.21.15.3:0/3541328706 >> v1:172.21.15.153:6789/0 conn(0x7f70380b0900 legacy=0x7f703806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70377fe700 1 -- 172.21.15.3:0/3541328706 >> v1:172.21.15.3:6789/0 conn(0x7f703806b7d0 legacy=0x7f70380c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70377fe700 1 -- 172.21.15.3:0/3541328706 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380b1640 con 0x7f7038156410 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70367fc700 1 -- 172.21.15.3:0/3541328706 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978445944 0 0) 0x7f70380b1c90 con 0x7f703806b7d0 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70377fe700 1 --2- 172.21.15.3:0/3541328706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7038156410 0x7f70380af480 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f702800bb90 tx=0x7f702800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b389942f4c9be995 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70367fc700 1 -- 172.21.15.3:0/3541328706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800db60 con 0x7f7038156410 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70367fc700 1 -- 172.21.15.3:0/3541328706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800dcc0 con 0x7f7038156410 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.150+0000 7f70367fc700 1 -- 172.21.15.3:0/3541328706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70280106a0 con 0x7f7038156410 2023-10-06T05:00:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.151+0000 7f703e1a6700 1 -- 172.21.15.3:0/3541328706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7038156410 msgr2=0x7f70380af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.151+0000 7f703e1a6700 1 --2- 172.21.15.3:0/3541328706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7038156410 0x7f70380af480 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f702800bb90 tx=0x7f702800b430 comp rx=0 tx=0).stop 2023-10-06T05:00:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.151+0000 7f703e1a6700 1 -- 172.21.15.3:0/3541328706 shutdown_connections 2023-10-06T05:00:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.151+0000 7f703e1a6700 1 --2- 172.21.15.3:0/3541328706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7038156410 0x7f70380af480 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.151+0000 7f703e1a6700 1 -- 172.21.15.3:0/3541328706 >> 172.21.15.3:0/3541328706 conn(0x7f70380cdc30 msgr2=0x7f703806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.151+0000 7f703e1a6700 1 -- 172.21.15.3:0/3541328706 shutdown_connections 2023-10-06T05:00:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.151+0000 7f703e1a6700 1 -- 172.21.15.3:0/3541328706 wait complete. 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.152+0000 7f703e1a6700 1 Processor -- start 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.152+0000 7f703e1a6700 1 -- start start 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f703806b7d0 0x7f703813d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70380b0900 0x7f703814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7038156410 0x7f70380674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7038158ab0 con 0x7f70380b0900 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7038157ea0 con 0x7f703806b7d0 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7038158180 con 0x7f7038156410 2023-10-06T05:00:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70380b0900 0x7f703814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70380b0900 0x7f703814c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33990/0 (socket says 172.21.15.3:33990) 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 -- 172.21.15.3:0/1205962970 learned_addr learned my addr 172.21.15.3:0/1205962970 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7037fff700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7038156410 0x7f70380674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f70377fe700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f703806b7d0 0x7f703813d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 -- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7038156410 msgr2=0x7f70380674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7038156410 0x7f70380674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 -- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f703806b7d0 msgr2=0x7f703813d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f703806b7d0 0x7f703813d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 -- 172.21.15.3:0/1205962970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702800b6d0 con 0x7f70380b0900 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f70377fe700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f703806b7d0 0x7f703813d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f7036ffd700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70380b0900 0x7f703814c180 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f702c00c5d0 tx=0x7f702c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f701f7fe700 1 -- 172.21.15.3:0/1205962970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011590 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f701f7fe700 1 -- 172.21.15.3:0/1205962970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c011710 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7038158460 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f701f7fe700 1 -- 172.21.15.3:0/1205962970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011ad0 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.153+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703814b9b0 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.155+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7038158c60 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.155+0000 7f701f7fe700 1 -- 172.21.15.3:0/1205962970 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f702c01c460 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.155+0000 7f701f7fe700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7024066f30 0x7f70240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.155+0000 7f701f7fe700 1 -- 172.21.15.3:0/1205962970 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f702c0843f0 con 0x7f70380b0900 2023-10-06T05:00:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.156+0000 7f70377fe700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7024066f30 0x7f70240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.156+0000 7f70377fe700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7024066f30 0x7f70240693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f702800b470 tx=0x7f702801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.159+0000 7f701f7fe700 1 -- 172.21.15.3:0/1205962970 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7038158c60 con 0x7f70380b0900 2023-10-06T05:00:24.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.353+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7038071680 con 0x7f70380b0900 2023-10-06T05:00:24.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.354+0000 7f701f7fe700 1 -- 172.21.15.3:0/1205962970 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16832 (secure 0 0 0) 0x7f702c051500 con 0x7f70380b0900 2023-10-06T05:00:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:24.356 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":108,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T04:59:55.572851+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:59:52.063522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T04:59:53.285974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:00:24.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7024066f30 msgr2=0x7f70240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7024066f30 0x7f70240693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f702800b470 tx=0x7f702801a000 comp rx=0 tx=0).stop 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70380b0900 msgr2=0x7f703814c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70380b0900 0x7f703814c180 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f702c00c5d0 tx=0x7f702c007f60 comp rx=0 tx=0).stop 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 shutdown_connections 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7024066f30 0x7f70240693e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f703806b7d0 0x7f703813d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7038156410 0x7f70380674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 --2- 172.21.15.3:0/1205962970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70380b0900 0x7f703814c180 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 >> 172.21.15.3:0/1205962970 conn(0x7f70380cdc30 msgr2=0x7f70380684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 shutdown_connections 2023-10-06T05:00:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.356+0000 7f703e1a6700 1 -- 172.21.15.3:0/1205962970 wait complete. 2023-10-06T05:00:24.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:00:24.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.490+0000 7f9f50e21700 1 Processor -- start 2023-10-06T05:00:24.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.490+0000 7f9f50e21700 1 -- start start 2023-10-06T05:00:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f50e21700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f50e21700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4c15aeb0 con 0x7f9f4c157030 2023-10-06T05:00:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f50e21700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4c15a7a0 con 0x7f9f4c157400 2023-10-06T05:00:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f50e21700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c15aa80 con 0x7f9f4c0b3d40 2023-10-06T05:00:24.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9f4c157030 0x7f9f4c159ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52180/0 (socket says 172.21.15.3:52180) 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4a59c700 1 -- 172.21.15.3:0/2551470541 learned_addr learned my addr 172.21.15.3:0/2551470541 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4ad9d700 1 --2- 172.21.15.3:0/2551470541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4959a700 1 -- 172.21.15.3:0/2551470541 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635138715 0 0) 0x7f9f4c15aeb0 con 0x7f9f4c157030 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4959a700 1 -- 172.21.15.3:0/2551470541 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f38003570 con 0x7f9f4c157030 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4959a700 1 -- 172.21.15.3:0/2551470541 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109086622 0 0) 0x7f9f4c15a7a0 con 0x7f9f4c157400 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4959a700 1 -- 172.21.15.3:0/2551470541 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f4c15aeb0 con 0x7f9f4c157400 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4ad9d700 1 -- 172.21.15.3:0/2551470541 >> v1:172.21.15.153:6789/0 conn(0x7f9f4c157400 legacy=0x7f9f4c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4ad9d700 1 -- 172.21.15.3:0/2551470541 >> v1:172.21.15.3:6789/0 conn(0x7f9f4c157030 legacy=0x7f9f4c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4ad9d700 1 -- 172.21.15.3:0/2551470541 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c06dba0 con 0x7f9f4c0b3d40 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.491+0000 7f9f4ad9d700 1 --2- 172.21.15.3:0/2551470541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c164320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009910 tx=0x7f9f3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd17b7efebfe5e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f4959a700 1 -- 172.21.15.3:0/2551470541 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c011630 con 0x7f9f4c0b3d40 2023-10-06T05:00:24.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f4959a700 1 -- 172.21.15.3:0/2551470541 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c011790 con 0x7f9f4c0b3d40 2023-10-06T05:00:24.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f4959a700 1 -- 172.21.15.3:0/2551470541 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c011b30 con 0x7f9f4c0b3d40 2023-10-06T05:00:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f50e21700 1 -- 172.21.15.3:0/2551470541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 msgr2=0x7f9f4c164320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f50e21700 1 --2- 172.21.15.3:0/2551470541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c164320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009910 tx=0x7f9f3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f50e21700 1 -- 172.21.15.3:0/2551470541 shutdown_connections 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f50e21700 1 --2- 172.21.15.3:0/2551470541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c164320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f50e21700 1 -- 172.21.15.3:0/2551470541 >> 172.21.15.3:0/2551470541 conn(0x7f9f4c0ce270 msgr2=0x7f9f4c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f50e21700 1 -- 172.21.15.3:0/2551470541 shutdown_connections 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.492+0000 7f9f50e21700 1 -- 172.21.15.3:0/2551470541 wait complete. 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.493+0000 7f9f50e21700 1 Processor -- start 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.493+0000 7f9f50e21700 1 -- start start 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f50e21700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f50e21700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f4c157030 0x7f9f4c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f50e21700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f4c157400 0x7f9f4c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f50e21700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0adc10 con 0x7f9f4c157400 2023-10-06T05:00:24.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f50e21700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0adef0 con 0x7f9f4c157030 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f50e21700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0cc040 con 0x7f9f4c0b3d40 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f4c157400 0x7f9f4c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f4c157400 0x7f9f4c0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33994/0 (socket says 172.21.15.3:33994) 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4ad9d700 1 -- 172.21.15.3:0/1963237035 learned_addr learned my addr 172.21.15.3:0/1963237035 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4a59c700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f49d9b700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f4c157030 0x7f9f4c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4ad9d700 1 -- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 msgr2=0x7f9f4c13fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4ad9d700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c13fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4ad9d700 1 -- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f4c157030 msgr2=0x7f9f4c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.494+0000 7f9f4ad9d700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f4c157030 0x7f9f4c0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f4ad9d700 1 -- 172.21.15.3:0/1963237035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009420 con 0x7f9f4c157400 2023-10-06T05:00:24.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f49d9b700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f4c157030 0x7f9f4c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f4a59c700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c13fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f4ad9d700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f4c157400 0x7f9f4c0cba60 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c15b060 tx=0x7f9f3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f427fc700 1 -- 172.21.15.3:0/1963237035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c01f070 con 0x7f9f4c157400 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f427fc700 1 -- 172.21.15.3:0/1963237035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c002f70 con 0x7f9f4c157400 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c0cc320 con 0x7f9f4c157400 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f427fc700 1 -- 172.21.15.3:0/1963237035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c011b30 con 0x7f9f4c157400 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.495+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c153dc0 con 0x7f9f4c157400 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.496+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c0644c0 con 0x7f9f4c157400 2023-10-06T05:00:24.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.496+0000 7f9f427fc700 1 -- 172.21.15.3:0/1963237035 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f3c023070 con 0x7f9f4c157400 2023-10-06T05:00:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.497+0000 7f9f427fc700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f2c066f30 0x7f9f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.497+0000 7f9f427fc700 1 -- 172.21.15.3:0/1963237035 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f3c085c80 con 0x7f9f4c157400 2023-10-06T05:00:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.497+0000 7f9f4a59c700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f2c066f30 0x7f9f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.497+0000 7f9f4a59c700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f2c066f30 0x7f9f2c0693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005c50 tx=0x7f9f44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.501+0000 7f9f427fc700 1 -- 172.21.15.3:0/1963237035 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f3c052d90 con 0x7f9f4c157400 2023-10-06T05:00:24.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.740+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9f4c0ae060 con 0x7f9f4c157400 2023-10-06T05:00:24.742 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T05:00:24.740+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:00:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.749+0000 7f9f427fc700 1 -- 172.21.15.3:0/1963237035 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9f3c02a020 con 0x7f9f4c157400 2023-10-06T05:00:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f2c066f30 msgr2=0x7f9f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f2c066f30 0x7f9f2c0693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005c50 tx=0x7f9f44009040 comp rx=0 tx=0).stop 2023-10-06T05:00:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f4c157400 msgr2=0x7f9f4c0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f4c157400 0x7f9f4c0cba60 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c15b060 tx=0x7f9f3c017000 comp rx=0 tx=0).stop 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 shutdown_connections 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9f2c066f30 0x7f9f2c0693e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f4c157400 0x7f9f4c0cba60 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f4c0b3d40 0x7f9f4c13fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 --2- 172.21.15.3:0/1963237035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f4c157030 0x7f9f4c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.751+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 >> 172.21.15.3:0/1963237035 conn(0x7f9f4c0ce270 msgr2=0x7f9f4c069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.752+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 shutdown_connections 2023-10-06T05:00:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.752+0000 7f9f50e21700 1 -- 172.21.15.3:0/1963237035 wait complete. 2023-10-06T05:00:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:00:24.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:00:24.782 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T05:00:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.888+0000 7fbde3a9e700 1 Processor -- start 2023-10-06T05:00:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.888+0000 7fbde3a9e700 1 -- start start 2023-10-06T05:00:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde3a9e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbddc159eb0 0x7fbddc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde3a9e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbddc0cef10 0x7fbddc15da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde3a9e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc0afca0 con 0x7fbddc0cfb80 2023-10-06T05:00:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde3a9e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc15d510 con 0x7fbddc0cef10 2023-10-06T05:00:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde3a9e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc15a480 con 0x7fbddc159eb0 2023-10-06T05:00:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbddc0cef10 0x7fbddc15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbddc0cef10 0x7fbddc15da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34038/0 (socket says 172.21.15.3:34038) 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 -- 172.21.15.3:0/2821139844 learned_addr learned my addr 172.21.15.3:0/2821139844 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde1039700 1 --2- 172.21.15.3:0/2821139844 >> v2:172.21.15.153:3300/0 conn(0x7fbddc159eb0 0x7fbddc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 -- 172.21.15.3:0/2821139844 >> v2:172.21.15.153:3300/0 conn(0x7fbddc159eb0 msgr2=0x7fbddc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 --2- 172.21.15.3:0/2821139844 >> v2:172.21.15.153:3300/0 conn(0x7fbddc159eb0 0x7fbddc0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 -- 172.21.15.3:0/2821139844 >> v1:172.21.15.153:6789/0 conn(0x7fbddc0cfb80 legacy=0x7fbddc0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 -- 172.21.15.3:0/2821139844 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc15a760 con 0x7fbddc0cef10 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde203b700 1 --2- 172.21.15.3:0/2821139844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cef10 0x7fbddc15da10 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0cb2e0 tx=0x7fbdd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbbd94bee81a3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde0838700 1 -- 172.21.15.3:0/2821139844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400e070 con 0x7fbddc0cef10 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde0838700 1 -- 172.21.15.3:0/2821139844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd400ba20 con 0x7fbddc0cef10 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.889+0000 7fbde0838700 1 -- 172.21.15.3:0/2821139844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40092e0 con 0x7fbddc0cef10 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 -- 172.21.15.3:0/2821139844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cef10 msgr2=0x7fbddc15da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/2821139844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cef10 0x7fbddc15da10 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0cb2e0 tx=0x7fbdd4009840 comp rx=0 tx=0).stop 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 -- 172.21.15.3:0/2821139844 shutdown_connections 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/2821139844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc0cef10 0x7fbddc15da10 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/2821139844 >> v2:172.21.15.153:3300/0 conn(0x7fbddc159eb0 0x7fbddc0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 -- 172.21.15.3:0/2821139844 >> 172.21.15.3:0/2821139844 conn(0x7fbddc070310 msgr2=0x7fbddc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:24.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 -- 172.21.15.3:0/2821139844 shutdown_connections 2023-10-06T05:00:24.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.890+0000 7fbde3a9e700 1 -- 172.21.15.3:0/2821139844 wait complete. 2023-10-06T05:00:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.891+0000 7fbde3a9e700 1 Processor -- start 2023-10-06T05:00:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.891+0000 7fbde3a9e700 1 -- start start 2023-10-06T05:00:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde3a9e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbddc0cef10 0x7fbddc154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde3a9e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbddc0cfb80 0x7fbddc0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde3a9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc159eb0 0x7fbddc0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde3a9e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc155350 con 0x7fbddc159eb0 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde3a9e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc1582e0 con 0x7fbddc0cfb80 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde3a9e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0c9a10 con 0x7fbddc0cef10 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc159eb0 0x7fbddc0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc159eb0 0x7fbddc0c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34044/0 (socket says 172.21.15.3:34044) 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 -- 172.21.15.3:0/3111244136 learned_addr learned my addr 172.21.15.3:0/3111244136 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde1039700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbddc0cfb80 0x7fbddc0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde183a700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbddc0cef10 0x7fbddc154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 -- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbddc0cef10 msgr2=0x7fbddc154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbddc0cef10 0x7fbddc154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 -- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbddc0cfb80 msgr2=0x7fbddc0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbddc0cfb80 0x7fbddc0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.892+0000 7fbde203b700 1 -- 172.21.15.3:0/3111244136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4009ae0 con 0x7fbddc159eb0 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.893+0000 7fbde203b700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc159eb0 0x7fbddc0c74f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009880 tx=0x7fbdd4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.893+0000 7fbdd1ffb700 1 -- 172.21.15.3:0/3111244136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4008a80 con 0x7fbddc159eb0 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.893+0000 7fbdd1ffb700 1 -- 172.21.15.3:0/3111244136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd4008be0 con 0x7fbddc159eb0 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.893+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc1579b0 con 0x7fbddc159eb0 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.893+0000 7fbdd1ffb700 1 -- 172.21.15.3:0/3111244136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4026460 con 0x7fbddc159eb0 2023-10-06T05:00:24.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.893+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc157f70 con 0x7fbddc159eb0 2023-10-06T05:00:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.894+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc158450 con 0x7fbddc159eb0 2023-10-06T05:00:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.894+0000 7fbdd1ffb700 1 -- 172.21.15.3:0/3111244136 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdd4003040 con 0x7fbddc159eb0 2023-10-06T05:00:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.895+0000 7fbdd1ffb700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.895+0000 7fbdd1ffb700 1 -- 172.21.15.3:0/3111244136 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fbdd4086510 con 0x7fbddc159eb0 2023-10-06T05:00:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.895+0000 7fbde183a700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.895+0000 7fbde183a700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbdc0066f30 0x7fbdc00693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0afab0 tx=0x7fbdd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:24.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:24.899+0000 7fbdd1ffb700 1 -- 172.21.15.3:0/3111244136 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdd40535d0 con 0x7fbddc159eb0 2023-10-06T05:00:25.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.137+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbddc04c380 con 0x7fbddc159eb0 2023-10-06T05:00:25.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:25.588 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.749+0000 7fbdd1ffb700 1 -- 172.21.15.3:0/3111244136 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdd4056bf0 con 0x7fbddc159eb0 2023-10-06T05:00:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbdc0066f30 msgr2=0x7fbdc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbdc0066f30 0x7fbdc00693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0afab0 tx=0x7fbdd8009040 comp rx=0 tx=0).stop 2023-10-06T05:00:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc159eb0 msgr2=0x7fbddc0c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc159eb0 0x7fbddc0c74f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009880 tx=0x7fbdd4018000 comp rx=0 tx=0).stop 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 shutdown_connections 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbddc159eb0 0x7fbddc0c74f0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbddc0cef10 0x7fbddc154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 --2- 172.21.15.3:0/3111244136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbddc0cfb80 0x7fbddc0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.751+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 >> 172.21.15.3:0/3111244136 conn(0x7fbddc070310 msgr2=0x7fbddc15c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.752+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 shutdown_connections 2023-10-06T05:00:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.752+0000 7fbde3a9e700 1 -- 172.21.15.3:0/3111244136 wait complete. 2023-10-06T05:00:25.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:00:25.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.890+0000 7f535acf7700 1 Processor -- start 2023-10-06T05:00:25.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.890+0000 7f535acf7700 1 -- start start 2023-10-06T05:00:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f535acf7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f535415c740 0x7f53540cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f535acf7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f53540afe60 0x7f53540ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f535acf7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53540ce7b0 con 0x7f535415cb10 2023-10-06T05:00:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f535acf7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535415cf70 con 0x7f535415c740 2023-10-06T05:00:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f535acf7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540cea90 con 0x7f53540afe60 2023-10-06T05:00:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f535415c740 0x7f53540cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f535415c740 0x7f53540cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34060/0 (socket says 172.21.15.3:34060) 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 -- 172.21.15.3:0/2302647325 learned_addr learned my addr 172.21.15.3:0/2302647325 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5359294700 1 --2- 172.21.15.3:0/2302647325 >> v2:172.21.15.153:3300/0 conn(0x7f53540afe60 0x7f53540ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 -- 172.21.15.3:0/2302647325 >> v2:172.21.15.153:3300/0 conn(0x7f53540afe60 msgr2=0x7f53540ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 --2- 172.21.15.3:0/2302647325 >> v2:172.21.15.153:3300/0 conn(0x7f53540afe60 0x7f53540ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 -- 172.21.15.3:0/2302647325 >> v1:172.21.15.153:6789/0 conn(0x7f535415cb10 legacy=0x7f535415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 -- 172.21.15.3:0/2302647325 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540ced70 con 0x7f535415c740 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.891+0000 7f5358a93700 1 --2- 172.21.15.3:0/2302647325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f53540cb3a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5354071290 tx=0x7f5344009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb64e3d6cbfda32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f53537fe700 1 -- 172.21.15.3:0/2302647325 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400b8b0 con 0x7f535415c740 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f53537fe700 1 -- 172.21.15.3:0/2302647325 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534400ba10 con 0x7f535415c740 2023-10-06T05:00:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f53537fe700 1 -- 172.21.15.3:0/2302647325 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400bdb0 con 0x7f535415c740 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 -- 172.21.15.3:0/2302647325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 msgr2=0x7f53540cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 --2- 172.21.15.3:0/2302647325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f53540cb3a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5354071290 tx=0x7f5344009250 comp rx=0 tx=0).stop 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 -- 172.21.15.3:0/2302647325 shutdown_connections 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 --2- 172.21.15.3:0/2302647325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f53540cb3a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 --2- 172.21.15.3:0/2302647325 >> v2:172.21.15.153:3300/0 conn(0x7f53540afe60 0x7f53540ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 -- 172.21.15.3:0/2302647325 >> 172.21.15.3:0/2302647325 conn(0x7f535406f960 msgr2=0x7f5354070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 -- 172.21.15.3:0/2302647325 shutdown_connections 2023-10-06T05:00:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.892+0000 7f535acf7700 1 -- 172.21.15.3:0/2302647325 wait complete. 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.893+0000 7f535acf7700 1 Processor -- start 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.893+0000 7f535acf7700 1 -- start start 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f535acf7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53540afe60 0x7f53540c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f535acf7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f5354152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f535acf7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f535415cb10 0x7f53540c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f535acf7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540c85f0 con 0x7f535415c740 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f535acf7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540b02c0 con 0x7f535415cb10 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f535acf7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53540b1120 con 0x7f53540afe60 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f5353fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f5354152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f5353fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f5354152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34062/0 (socket says 172.21.15.3:34062) 2023-10-06T05:00:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f5353fff700 1 -- 172.21.15.3:0/852434496 learned_addr learned my addr 172.21.15.3:0/852434496 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f5358a93700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53540afe60 0x7f53540c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.894+0000 7f5359294700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f535415cb10 0x7f53540c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5353fff700 1 -- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53540afe60 msgr2=0x7f53540c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5353fff700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53540afe60 0x7f53540c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5353fff700 1 -- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f535415cb10 msgr2=0x7f53540c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5353fff700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f535415cb10 0x7f53540c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5353fff700 1 -- 172.21.15.3:0/852434496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440094f0 con 0x7f535415c740 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5353fff700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f5354152000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f534800c5d0 tx=0x7f5348007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5350ff9700 1 -- 172.21.15.3:0/852434496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348011590 con 0x7f535415c740 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5350ff9700 1 -- 172.21.15.3:0/852434496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5348011710 con 0x7f535415c740 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53540b0510 con 0x7f535415c740 2023-10-06T05:00:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f5350ff9700 1 -- 172.21.15.3:0/852434496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348011ad0 con 0x7f535415c740 2023-10-06T05:00:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.895+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53540b0ad0 con 0x7f535415c740 2023-10-06T05:00:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.896+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53540b12b0 con 0x7f535415c740 2023-10-06T05:00:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.896+0000 7f5350ff9700 1 -- 172.21.15.3:0/852434496 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f534801c460 con 0x7f535415c740 2023-10-06T05:00:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.897+0000 7f5350ff9700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5340066f60 0x7f5340069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.897+0000 7f5350ff9700 1 -- 172.21.15.3:0/852434496 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5348084310 con 0x7f535415c740 2023-10-06T05:00:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.897+0000 7f5358a93700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5340066f60 0x7f5340069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.898+0000 7f5358a93700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5340066f60 0x7f5340069410 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5354071290 tx=0x7f5344016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:25.901+0000 7f5350ff9700 1 -- 172.21.15.3:0/852434496 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53480513d0 con 0x7f535415c740 2023-10-06T05:00:26.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.095+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53540cf0e0 con 0x7f535415c740 2023-10-06T05:00:26.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.749+0000 7f5350ff9700 1 -- 172.21.15.3:0/852434496 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7f53480549f0 con 0x7f535415c740 2023-10-06T05:00:26.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5340066f60 msgr2=0x7f5340069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:26.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5340066f60 0x7f5340069410 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5354071290 tx=0x7f5344016000 comp rx=0 tx=0).stop 2023-10-06T05:00:26.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 msgr2=0x7f5354152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:26.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f5354152000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f534800c5d0 tx=0x7f5348007f60 comp rx=0 tx=0).stop 2023-10-06T05:00:26.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 shutdown_connections 2023-10-06T05:00:26.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5340066f60 0x7f5340069410 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f535415cb10 0x7f53540c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53540afe60 0x7f53540c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 --2- 172.21.15.3:0/852434496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f535415c740 0x7f5354152000 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.753+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 >> 172.21.15.3:0/852434496 conn(0x7f535406f960 msgr2=0x7f535415ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:26.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.754+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 shutdown_connections 2023-10-06T05:00:26.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.754+0000 7f535acf7700 1 -- 172.21.15.3:0/852434496 wait complete. 2023-10-06T05:00:26.757 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:00:26.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:00:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.897+0000 7f43a9a14700 1 Processor -- start 2023-10-06T05:00:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.897+0000 7f43a9a14700 1 -- start start 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a9a14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a415d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a9a14700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43a415a560 0x7f43a4153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a9a14700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f43a40cce60 0x7f43a40cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a9a14700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a4153ff0 con 0x7f43a415a560 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a9a14700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a40cd820 con 0x7f43a40cce60 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a9a14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a415a9d0 con 0x7f43a41536e0 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43a415a560 0x7f43a4153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43a415a560 0x7f43a4153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34074/0 (socket says 172.21.15.3:34074) 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 -- 172.21.15.3:0/1185745145 learned_addr learned my addr 172.21.15.3:0/1185745145 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a37fe700 1 --2- 172.21.15.3:0/1185745145 >> v2:172.21.15.153:3300/0 conn(0x7f43a40cce60 0x7f43a40cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a2ffd700 1 --2- 172.21.15.3:0/1185745145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 -- 172.21.15.3:0/1185745145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 msgr2=0x7f43a415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 --2- 172.21.15.3:0/1185745145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a415d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 -- 172.21.15.3:0/1185745145 >> v2:172.21.15.153:3300/0 conn(0x7f43a40cce60 msgr2=0x7f43a40cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 --2- 172.21.15.3:0/1185745145 >> v2:172.21.15.153:3300/0 conn(0x7f43a40cce60 0x7f43a40cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 -- 172.21.15.3:0/1185745145 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a40ce6b0 con 0x7f43a415a560 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a37fe700 1 --2- 172.21.15.3:0/1185745145 >> v2:172.21.15.153:3300/0 conn(0x7f43a40cce60 0x7f43a40cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a27fc700 1 --2- 172.21.15.3:0/1185745145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a4153ab0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f43a40d0430 tx=0x7f4398009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df19f89ac7bca33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a1ffb700 1 -- 172.21.15.3:0/1185745145 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439800b820 con 0x7f43a415a560 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a1ffb700 1 -- 172.21.15.3:0/1185745145 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439800b980 con 0x7f43a415a560 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.898+0000 7f43a1ffb700 1 -- 172.21.15.3:0/1185745145 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439800bd20 con 0x7f43a415a560 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 -- 172.21.15.3:0/1185745145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 msgr2=0x7f43a4153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1185745145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a4153ab0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f43a40d0430 tx=0x7f4398009250 comp rx=0 tx=0).stop 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 -- 172.21.15.3:0/1185745145 shutdown_connections 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1185745145 >> v2:172.21.15.153:3300/0 conn(0x7f43a40cce60 0x7f43a40cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1185745145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1185745145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a4153ab0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 -- 172.21.15.3:0/1185745145 >> 172.21.15.3:0/1185745145 conn(0x7f43a406f950 msgr2=0x7f43a4070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 -- 172.21.15.3:0/1185745145 shutdown_connections 2023-10-06T05:00:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.899+0000 7f43a9a14700 1 -- 172.21.15.3:0/1185745145 wait complete. 2023-10-06T05:00:26.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.900+0000 7f43a9a14700 1 Processor -- start 2023-10-06T05:00:26.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.900+0000 7f43a9a14700 1 -- start start 2023-10-06T05:00:26.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a9a14700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a40cce60 0x7f43a40c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a9a14700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a41509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a9a14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a414fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a9a14700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40b3eb0 con 0x7f43a415a560 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a9a14700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40b32a0 con 0x7f43a40cce60 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a9a14700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40b3580 con 0x7f43a41536e0 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a414fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a414fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34086/0 (socket says 172.21.15.3:34086) 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a37fe700 1 -- 172.21.15.3:0/1659721760 learned_addr learned my addr 172.21.15.3:0/1659721760 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a27fc700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a41509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.901+0000 7f43a2ffd700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a40cce60 0x7f43a40c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a37fe700 1 -- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 msgr2=0x7f43a41509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a37fe700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a41509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a37fe700 1 -- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a40cce60 msgr2=0x7f43a40c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a37fe700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a40cce60 0x7f43a40c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a37fe700 1 -- 172.21.15.3:0/1659721760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43980094f0 con 0x7f43a415a560 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a27fc700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a41509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a2ffd700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a40cce60 0x7f43a40c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a37fe700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a414fe50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f439c00cef0 tx=0x7f439c007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f438affd700 1 -- 172.21.15.3:0/1659721760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00ae00 con 0x7f43a415a560 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f438affd700 1 -- 172.21.15.3:0/1659721760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c016420 con 0x7f43a415a560 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a40b3860 con 0x7f43a415a560 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f438affd700 1 -- 172.21.15.3:0/1659721760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0167e0 con 0x7f43a415a560 2023-10-06T05:00:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.902+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a40ca450 con 0x7f43a415a560 2023-10-06T05:00:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.903+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a4071680 con 0x7f43a415a560 2023-10-06T05:00:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.903+0000 7f438affd700 1 -- 172.21.15.3:0/1659721760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f439c016bd0 con 0x7f43a415a560 2023-10-06T05:00:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.904+0000 7f438affd700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4390066f30 0x7f43900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.904+0000 7f438affd700 1 -- 172.21.15.3:0/1659721760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f439c012070 con 0x7f43a415a560 2023-10-06T05:00:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.904+0000 7f43a2ffd700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4390066f30 0x7f43900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.905+0000 7f43a2ffd700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4390066f30 0x7f43900693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f439400ab70 tx=0x7f4394008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:26.908+0000 7f438affd700 1 -- 172.21.15.3:0/1659721760 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f439c04e5a0 con 0x7f43a415a560 2023-10-06T05:00:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.101+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f43a40cfbc0 con 0x7f43a415a560 2023-10-06T05:00:27.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.754+0000 7f438affd700 1 -- 172.21.15.3:0/1659721760 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7f439c0520f0 con 0x7f43a415a560 2023-10-06T05:00:27.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.759+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4390066f30 msgr2=0x7f43900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:27.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.759+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4390066f30 0x7f43900693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f439400ab70 tx=0x7f4394008040 comp rx=0 tx=0).stop 2023-10-06T05:00:27.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.759+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 msgr2=0x7f43a414fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:27.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.759+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a414fe50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f439c00cef0 tx=0x7f439c007860 comp rx=0 tx=0).stop 2023-10-06T05:00:27.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.761+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 shutdown_connections 2023-10-06T05:00:27.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.761+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4390066f30 0x7f43900693e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.761+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a40cce60 0x7f43a40c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.761+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43a415a560 0x7f43a414fe50 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.761+0000 7f43a9a14700 1 --2- 172.21.15.3:0/1659721760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a41536e0 0x7f43a41509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.761+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 >> 172.21.15.3:0/1659721760 conn(0x7f43a406f950 msgr2=0x7f43a40cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:27.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.765+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 shutdown_connections 2023-10-06T05:00:27.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.765+0000 7f43a9a14700 1 -- 172.21.15.3:0/1659721760 wait complete. 2023-10-06T05:00:27.770 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T05:00:27.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-10-06T05:00:27.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.911+0000 7f1e89615700 1 Processor -- start 2023-10-06T05:00:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.911+0000 7f1e89615700 1 -- start start 2023-10-06T05:00:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e89615700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 0x7f1e8415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e89615700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8415d180 con 0x7f1e84067dc0 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e89615700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e840691c0 con 0x7f1e8415cd40 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e89615700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840694a0 con 0x7f1e84068190 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e82ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1e84067dc0 0x7f1e840b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52196/0 (socket says 172.21.15.3:52196) 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e82ffd700 1 -- 172.21.15.3:0/3899390511 learned_addr learned my addr 172.21.15.3:0/3899390511 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e827fc700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 0x7f1e8415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666457140 0 0) 0x7f1e8415d180 con 0x7f1e84067dc0 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e70003570 con 0x7f1e84067dc0 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433786448 0 0) 0x7f1e840691c0 con 0x7f1e8415cd40 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e8415d180 con 0x7f1e8415cd40 2023-10-06T05:00:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1222462689 0 0) 0x7f1e70003570 con 0x7f1e84067dc0 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 msgr2=0x7f1e8415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.912+0000 7f1e81ffb700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 0x7f1e8415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 >> v1:172.21.15.153:6789/0 conn(0x7f1e8415cd40 legacy=0x7f1e8406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84069780 con 0x7f1e84067dc0 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f1e7c001970 con 0x7f1e84067dc0 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 >> v1:172.21.15.3:6789/0 conn(0x7f1e84067dc0 legacy=0x7f1e840b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e70003c20 0x7f1e70004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e70004810 0x7f1e70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 0x7f1e700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840694a0 con 0x7f1e70004810 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7000bb40 con 0x7f1e70003c20 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7000bd00 con 0x7f1e84068190 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e827fc700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e70004810 0x7f1e70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e82ffd700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e70003c20 0x7f1e70004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e837fe700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 0x7f1e700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e827fc700 1 -- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 msgr2=0x7f1e700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e827fc700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e84068190 0x7f1e700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e827fc700 1 -- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e70003c20 msgr2=0x7f1e70004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e827fc700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e70003c20 0x7f1e70004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e827fc700 1 -- 172.21.15.3:0/3899390511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70003570 con 0x7f1e70004810 2023-10-06T05:00:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e827fc700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e70004810 0x7f1e70004c80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009c00 tx=0x7f1e6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac99ce1498ec8c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00e710 con 0x7f1e70004810 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00e870 con 0x7f1e70004810 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.913+0000 7f1e81ffb700 1 -- 172.21.15.3:0/3899390511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00ec30 con 0x7f1e70004810 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.914+0000 7f1e89615700 1 -- 172.21.15.3:0/3899390511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e70004810 msgr2=0x7f1e70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.914+0000 7f1e89615700 1 --2- 172.21.15.3:0/3899390511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e70004810 0x7f1e70004c80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009c00 tx=0x7f1e6c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.914+0000 7f1e837fe700 1 -- 172.21.15.3:0/3899390511 reap_dead start 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.914+0000 7f1e89615700 1 -- 172.21.15.3:0/3899390511 shutdown_connections 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.914+0000 7f1e89615700 1 -- 172.21.15.3:0/3899390511 >> 172.21.15.3:0/3899390511 conn(0x7f1e840ce270 msgr2=0x7f1e840ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.914+0000 7f1e89615700 1 -- 172.21.15.3:0/3899390511 shutdown_connections 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.914+0000 7f1e89615700 1 -- 172.21.15.3:0/3899390511 wait complete. 2023-10-06T05:00:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.915+0000 7f1e89615700 1 Processor -- start 2023-10-06T05:00:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.915+0000 7f1e89615700 1 -- start start 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e89615700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e70004810 0x7f1e84156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e89615700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8416abe0 0x7f1e840cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e89615700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e840cc150 0x7f1e840a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e89615700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84156c90 con 0x7f1e8416abe0 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e89615700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840cd670 con 0x7f1e840cc150 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e89615700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840cc610 con 0x7f1e70004810 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8416abe0 0x7f1e840cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8416abe0 0x7f1e840cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34094/0 (socket says 172.21.15.3:34094) 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 -- 172.21.15.3:0/727406631 learned_addr learned my addr 172.21.15.3:0/727406631 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e82ffd700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e70004810 0x7f1e84156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e837fe700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e840cc150 0x7f1e840a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 -- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e70004810 msgr2=0x7f1e84156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e70004810 0x7f1e84156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 -- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e840cc150 msgr2=0x7f1e840a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e840cc150 0x7f1e840a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.916+0000 7f1e827fc700 1 -- 172.21.15.3:0/727406631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840cc8b0 con 0x7f1e8416abe0 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.917+0000 7f1e827fc700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8416abe0 0x7f1e840cbc10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000bb10 tx=0x7f1e6c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.917+0000 7f1e7affd700 1 -- 172.21.15.3:0/727406631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c022070 con 0x7f1e8416abe0 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.917+0000 7f1e7affd700 1 -- 172.21.15.3:0/727406631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00ebf0 con 0x7f1e8416abe0 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.917+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e840ccb90 con 0x7f1e8416abe0 2023-10-06T05:00:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.917+0000 7f1e7affd700 1 -- 172.21.15.3:0/727406631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c01c6a0 con 0x7f1e8416abe0 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.917+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e840cd150 con 0x7f1e8416abe0 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.918+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e840cd4c0 con 0x7f1e8416abe0 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.918+0000 7f1e7affd700 1 -- 172.21.15.3:0/727406631 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e6c01ca90 con 0x7f1e8416abe0 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.919+0000 7f1e7affd700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1e64066f30 0x7f1e640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.919+0000 7f1e7affd700 1 -- 172.21.15.3:0/727406631 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1e6c085be0 con 0x7f1e8416abe0 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.919+0000 7f1e82ffd700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1e64066f30 0x7f1e640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.919+0000 7f1e82ffd700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1e64066f30 0x7f1e640693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c001aa0 tx=0x7f1e7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:27.923+0000 7f1e7affd700 1 -- 172.21.15.3:0/727406631 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e6c052f90 con 0x7f1e8416abe0 2023-10-06T05:00:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.115+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f1e8404c380 con 0x7f1e8416abe0 2023-10-06T05:00:28.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.118+0000 7f1e7affd700 1 -- 172.21.15.3:0/727406631 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v49) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1e6c02b0b0 con 0x7f1e8416abe0 2023-10-06T05:00:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1e64066f30 msgr2=0x7f1e640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1e64066f30 0x7f1e640693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c001aa0 tx=0x7f1e7c009040 comp rx=0 tx=0).stop 2023-10-06T05:00:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8416abe0 msgr2=0x7f1e840cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8416abe0 0x7f1e840cbc10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000bb10 tx=0x7f1e6c01a000 comp rx=0 tx=0).stop 2023-10-06T05:00:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 shutdown_connections 2023-10-06T05:00:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1e64066f30 0x7f1e640693e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e840cc150 0x7f1e840a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e70004810 0x7f1e84156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 --2- 172.21.15.3:0/727406631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8416abe0 0x7f1e840cbc10 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 >> 172.21.15.3:0/727406631 conn(0x7f1e840ce270 msgr2=0x7f1e840b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 shutdown_connections 2023-10-06T05:00:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.120+0000 7f1e89615700 1 -- 172.21.15.3:0/727406631 wait complete. 2023-10-06T05:00:28.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:00:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.261+0000 7f591b05a700 1 Processor -- start 2023-10-06T05:00:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.261+0000 7f591b05a700 1 -- start start 2023-10-06T05:00:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.262+0000 7f591b05a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59140cba50 0x7f5914150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.262+0000 7f591b05a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591415a290 con 0x7f5914150690 2023-10-06T05:00:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.262+0000 7f591b05a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59140d0020 con 0x7f591415d540 2023-10-06T05:00:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.262+0000 7f591b05a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5914159f40 con 0x7f59140cba50 2023-10-06T05:00:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.262+0000 7f5913fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59140cba50 0x7f5914150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.262+0000 7f5918df6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5914150690 0x7f591415a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52210/0 (socket says 172.21.15.3:52210) 2023-10-06T05:00:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.262+0000 7f5918df6700 1 -- 172.21.15.3:0/254427875 learned_addr learned my addr 172.21.15.3:0/254427875 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.263+0000 7f5913fff700 1 -- 172.21.15.3:0/254427875 >> v1:172.21.15.153:6789/0 conn(0x7f591415d540 legacy=0x7f59140af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.263+0000 7f5913fff700 1 -- 172.21.15.3:0/254427875 >> v1:172.21.15.3:6789/0 conn(0x7f5914150690 legacy=0x7f591415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.263+0000 7f5913fff700 1 -- 172.21.15.3:0/254427875 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591415dcb0 con 0x7f59140cba50 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.263+0000 7f5913fff700 1 --2- 172.21.15.3:0/254427875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f5914150a60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f59080098e0 tx=0x7f59080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc35f0de01ffef4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.263+0000 7f59137fe700 1 -- 172.21.15.3:0/254427875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908009e80 con 0x7f59140cba50 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.263+0000 7f59137fe700 1 -- 172.21.15.3:0/254427875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590800b760 con 0x7f59140cba50 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.263+0000 7f59137fe700 1 -- 172.21.15.3:0/254427875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800bae0 con 0x7f59140cba50 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.264+0000 7f591b05a700 1 -- 172.21.15.3:0/254427875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 msgr2=0x7f5914150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.264+0000 7f591b05a700 1 --2- 172.21.15.3:0/254427875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f5914150a60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f59080098e0 tx=0x7f59080091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.264+0000 7f591b05a700 1 -- 172.21.15.3:0/254427875 shutdown_connections 2023-10-06T05:00:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.264+0000 7f591b05a700 1 --2- 172.21.15.3:0/254427875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f5914150a60 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.264+0000 7f591b05a700 1 -- 172.21.15.3:0/254427875 >> 172.21.15.3:0/254427875 conn(0x7f5914070310 msgr2=0x7f5914070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.264+0000 7f591b05a700 1 -- 172.21.15.3:0/254427875 shutdown_connections 2023-10-06T05:00:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.264+0000 7f591b05a700 1 -- 172.21.15.3:0/254427875 wait complete. 2023-10-06T05:00:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.265+0000 7f591b05a700 1 Processor -- start 2023-10-06T05:00:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.265+0000 7f591b05a700 1 -- start start 2023-10-06T05:00:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f591b05a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f591414ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f591b05a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914150690 0x7f59140cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f591b05a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f591415d540 0x7f59140cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f591b05a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591416d870 con 0x7f59140cba50 2023-10-06T05:00:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f591b05a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591416dac0 con 0x7f5914150690 2023-10-06T05:00:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f591b05a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591416dda0 con 0x7f591415d540 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f591414ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f591414ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34118/0 (socket says 172.21.15.3:34118) 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 -- 172.21.15.3:0/4147093102 learned_addr learned my addr 172.21.15.3:0/4147093102 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5913fff700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914150690 0x7f59140cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 -- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f591415d540 msgr2=0x7f59140cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f591415d540 0x7f59140cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 -- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914150690 msgr2=0x7f59140cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914150690 0x7f59140cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 -- 172.21.15.3:0/4147093102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5908009420 con 0x7f59140cba50 2023-10-06T05:00:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5918df6700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f591414ecb0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f590c00cec0 tx=0x7f590c001c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5910ff9700 1 -- 172.21.15.3:0/4147093102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c00aba0 con 0x7f59140cba50 2023-10-06T05:00:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5910ff9700 1 -- 172.21.15.3:0/4147093102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c00ad00 con 0x7f59140cba50 2023-10-06T05:00:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591416e080 con 0x7f59140cba50 2023-10-06T05:00:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.266+0000 7f5910ff9700 1 -- 172.21.15.3:0/4147093102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c007340 con 0x7f59140cba50 2023-10-06T05:00:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.267+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591414f1f0 con 0x7f59140cba50 2023-10-06T05:00:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.267+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591416e5a0 con 0x7f59140cba50 2023-10-06T05:00:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.268+0000 7f5910ff9700 1 -- 172.21.15.3:0/4147093102 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f590c0076f0 con 0x7f59140cba50 2023-10-06T05:00:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.268+0000 7f5910ff9700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.268+0000 7f5910ff9700 1 -- 172.21.15.3:0/4147093102 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f590c084a70 con 0x7f59140cba50 2023-10-06T05:00:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.269+0000 7f5913fff700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.269+0000 7f5913fff700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f59080098e0 tx=0x7f5908016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.273+0000 7f5910ff9700 1 -- 172.21.15.3:0/4147093102 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f591416e5a0 con 0x7f59140cba50 2023-10-06T05:00:28.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.466+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f591416e5a0 con 0x7f59140cba50 2023-10-06T05:00:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.470+0000 7f5910ff9700 1 -- 172.21.15.3:0/4147093102 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7f591416e5a0 con 0x7f59140cba50 2023-10-06T05:00:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 msgr2=0x7f59000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f59080098e0 tx=0x7f5908016000 comp rx=0 tx=0).stop 2023-10-06T05:00:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 msgr2=0x7f591414ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f591414ecb0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f590c00cec0 tx=0x7f590c001c20 comp rx=0 tx=0).stop 2023-10-06T05:00:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 shutdown_connections 2023-10-06T05:00:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f591415d540 0x7f59140cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59140cba50 0x7f591414ecb0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 --2- 172.21.15.3:0/4147093102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914150690 0x7f59140cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 >> 172.21.15.3:0/4147093102 conn(0x7f5914070310 msgr2=0x7f591415c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 shutdown_connections 2023-10-06T05:00:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.472+0000 7f591b05a700 1 -- 172.21.15.3:0/4147093102 wait complete. 2023-10-06T05:00:28.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-10-06T05:00:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.605+0000 7efd4f964700 1 Processor -- start 2023-10-06T05:00:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.606+0000 7efd4f964700 1 -- start start 2023-10-06T05:00:28.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.606+0000 7efd4f964700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd48140e70 0x7efd480ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.606+0000 7efd4f964700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd48069f10 con 0x7efd480cd440 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.606+0000 7efd4f964700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd480cca70 con 0x7efd480b16c0 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.606+0000 7efd4f964700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd480cc140 con 0x7efd48140e70 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.606+0000 7efd4d700700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd48140e70 0x7efd480ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.606+0000 7efd4ceff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efd480cd440 0x7efd480b0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52226/0 (socket says 172.21.15.3:52226) 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd4ceff700 1 -- 172.21.15.3:0/2548319618 learned_addr learned my addr 172.21.15.3:0/2548319618 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd4d700700 1 -- 172.21.15.3:0/2548319618 >> v1:172.21.15.153:6789/0 conn(0x7efd480b16c0 legacy=0x7efd48067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd4d700700 1 -- 172.21.15.3:0/2548319618 >> v1:172.21.15.3:6789/0 conn(0x7efd480cd440 legacy=0x7efd480b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd4d700700 1 -- 172.21.15.3:0/2548319618 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480cc420 con 0x7efd48140e70 2023-10-06T05:00:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd47fff700 1 -- 172.21.15.3:0/2548319618 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801713161 0 0) 0x7efd48069f10 con 0x7efd480cd440 2023-10-06T05:00:28.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd4d700700 1 --2- 172.21.15.3:0/2548319618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd48140e70 0x7efd480ccf00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7efd40009e60 tx=0x7efd40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e329a5147986f093 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd47fff700 1 -- 172.21.15.3:0/2548319618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000e070 con 0x7efd48140e70 2023-10-06T05:00:28.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd47fff700 1 -- 172.21.15.3:0/2548319618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4000bbf0 con 0x7efd48140e70 2023-10-06T05:00:28.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.607+0000 7efd47fff700 1 -- 172.21.15.3:0/2548319618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40012430 con 0x7efd48140e70 2023-10-06T05:00:28.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.608+0000 7efd4f964700 1 -- 172.21.15.3:0/2548319618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd48140e70 msgr2=0x7efd480ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.608+0000 7efd4f964700 1 --2- 172.21.15.3:0/2548319618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd48140e70 0x7efd480ccf00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7efd40009e60 tx=0x7efd40009780 comp rx=0 tx=0).stop 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.608+0000 7efd4f964700 1 -- 172.21.15.3:0/2548319618 shutdown_connections 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.608+0000 7efd4f964700 1 --2- 172.21.15.3:0/2548319618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd48140e70 0x7efd480ccf00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.608+0000 7efd4f964700 1 -- 172.21.15.3:0/2548319618 >> 172.21.15.3:0/2548319618 conn(0x7efd480ce210 msgr2=0x7efd480ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.608+0000 7efd4f964700 1 -- 172.21.15.3:0/2548319618 shutdown_connections 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.608+0000 7efd4f964700 1 -- 172.21.15.3:0/2548319618 wait complete. 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.609+0000 7efd4f964700 1 Processor -- start 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.609+0000 7efd4f964700 1 -- start start 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4f964700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd480b16c0 0x7efd48131f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4f964700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd480cd440 0x7efd480c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4f964700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd48140e70 0x7efd480c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4f964700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48157770 con 0x7efd480b16c0 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4f964700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48167b30 con 0x7efd480cd440 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4f964700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd48167e10 con 0x7efd48140e70 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd480b16c0 0x7efd48131f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd480b16c0 0x7efd48131f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34146/0 (socket says 172.21.15.3:34146) 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 -- 172.21.15.3:0/2131147111 learned_addr learned my addr 172.21.15.3:0/2131147111 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4ceff700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd480cd440 0x7efd480c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4df01700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd48140e70 0x7efd480c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 -- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd48140e70 msgr2=0x7efd480c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd48140e70 0x7efd480c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 -- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd480cd440 msgr2=0x7efd480c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd480cd440 0x7efd480c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 -- 172.21.15.3:0/2131147111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd40009a20 con 0x7efd480b16c0 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4df01700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd48140e70 0x7efd480c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.610+0000 7efd4d700700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd480b16c0 0x7efd48131f10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7efd4806b690 tx=0x7efd40018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.611+0000 7efd457fa700 1 -- 172.21.15.3:0/2131147111 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40027470 con 0x7efd480b16c0 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.611+0000 7efd457fa700 1 -- 172.21.15.3:0/2131147111 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd400275d0 con 0x7efd480b16c0 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.611+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd481680f0 con 0x7efd480b16c0 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.611+0000 7efd457fa700 1 -- 172.21.15.3:0/2131147111 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40027970 con 0x7efd480b16c0 2023-10-06T05:00:28.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.611+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd48167830 con 0x7efd480b16c0 2023-10-06T05:00:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.611+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4816afa0 con 0x7efd480b16c0 2023-10-06T05:00:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.612+0000 7efd457fa700 1 -- 172.21.15.3:0/2131147111 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd40024070 con 0x7efd480b16c0 2023-10-06T05:00:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.612+0000 7efd457fa700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd34066f30 0x7efd340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.612+0000 7efd457fa700 1 -- 172.21.15.3:0/2131147111 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7efd40086ed0 con 0x7efd480b16c0 2023-10-06T05:00:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.613+0000 7efd4ceff700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd34066f30 0x7efd340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.613+0000 7efd4ceff700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd34066f30 0x7efd340693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efd48068580 tx=0x7efd3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.617+0000 7efd457fa700 1 -- 172.21.15.3:0/2131147111 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd400542e0 con 0x7efd480b16c0 2023-10-06T05:00:28.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.810+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7efd4804c380 con 0x7efd480b16c0 2023-10-06T05:00:28.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.813+0000 7efd457fa700 1 -- 172.21.15.3:0/2131147111 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v51) v1 ==== 107+0+0 (secure 0 0 0) 0x7efd40057900 con 0x7efd480b16c0 2023-10-06T05:00:28.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.815+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd34066f30 msgr2=0x7efd340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.815+0000 7efd4f964700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd34066f30 0x7efd340693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efd48068580 tx=0x7efd3c009040 comp rx=0 tx=0).stop 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.815+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd480b16c0 msgr2=0x7efd48131f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.815+0000 7efd4f964700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd480b16c0 0x7efd48131f10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7efd4806b690 tx=0x7efd40018000 comp rx=0 tx=0).stop 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 shutdown_connections 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efd34066f30 0x7efd340693e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd48140e70 0x7efd480c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd480b16c0 0x7efd48131f10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 --2- 172.21.15.3:0/2131147111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd480cd440 0x7efd480c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 >> 172.21.15.3:0/2131147111 conn(0x7efd480ce210 msgr2=0x7efd480b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 shutdown_connections 2023-10-06T05:00:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.816+0000 7efd4f964700 1 -- 172.21.15.3:0/2131147111 wait complete. 2023-10-06T05:00:28.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-10-06T05:00:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.949+0000 7fe112f59700 1 Processor -- start 2023-10-06T05:00:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.949+0000 7fe112f59700 1 -- start start 2023-10-06T05:00:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe112f59700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe10c150690 0x7fe10c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe112f59700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe10c0cba50 0x7fe10c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe112f59700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c15da80 con 0x7fe10c15a720 2023-10-06T05:00:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe112f59700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c15a290 con 0x7fe10c150690 2023-10-06T05:00:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe112f59700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c0d0020 con 0x7fe10c0cba50 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe10c150690 0x7fe10c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe10c150690 0x7fe10c150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34158/0 (socket says 172.21.15.3:34158) 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 -- 172.21.15.3:0/451999260 learned_addr learned my addr 172.21.15.3:0/451999260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe10bfff700 1 --2- 172.21.15.3:0/451999260 >> v2:172.21.15.153:3300/0 conn(0x7fe10c0cba50 0x7fe10c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 -- 172.21.15.3:0/451999260 >> v2:172.21.15.153:3300/0 conn(0x7fe10c0cba50 msgr2=0x7fe10c15d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 --2- 172.21.15.3:0/451999260 >> v2:172.21.15.153:3300/0 conn(0x7fe10c0cba50 0x7fe10c15d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 -- 172.21.15.3:0/451999260 >> v1:172.21.15.153:6789/0 conn(0x7fe10c15a720 legacy=0x7fe10c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 -- 172.21.15.3:0/451999260 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c159f40 con 0x7fe10c150690 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe10bfff700 1 --2- 172.21.15.3:0/451999260 >> v2:172.21.15.153:3300/0 conn(0x7fe10c0cba50 0x7fe10c15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.950+0000 7fe110cf5700 1 --2- 172.21.15.3:0/451999260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c150690 0x7fe10c150a60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0ce1e0 tx=0x7fe104009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9a84d716a77b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe10b7fe700 1 -- 172.21.15.3:0/451999260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10400e070 con 0x7fe10c150690 2023-10-06T05:00:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe10b7fe700 1 -- 172.21.15.3:0/451999260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10400ba10 con 0x7fe10c150690 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe10b7fe700 1 -- 172.21.15.3:0/451999260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040092e0 con 0x7fe10c150690 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 -- 172.21.15.3:0/451999260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c150690 msgr2=0x7fe10c150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 --2- 172.21.15.3:0/451999260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c150690 0x7fe10c150a60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0ce1e0 tx=0x7fe104009840 comp rx=0 tx=0).stop 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 -- 172.21.15.3:0/451999260 shutdown_connections 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 --2- 172.21.15.3:0/451999260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c150690 0x7fe10c150a60 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 --2- 172.21.15.3:0/451999260 >> v2:172.21.15.153:3300/0 conn(0x7fe10c0cba50 0x7fe10c15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 -- 172.21.15.3:0/451999260 >> 172.21.15.3:0/451999260 conn(0x7fe10c070310 msgr2=0x7fe10c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 -- 172.21.15.3:0/451999260 shutdown_connections 2023-10-06T05:00:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.951+0000 7fe112f59700 1 -- 172.21.15.3:0/451999260 wait complete. 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.952+0000 7fe112f59700 1 Processor -- start 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe112f59700 1 -- start start 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe112f59700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe10c0cba50 0x7fe10c0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe112f59700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe10c150690 0x7fe10c16d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe112f59700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c15a720 0x7fe10c16aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe112f59700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c16e520 con 0x7fe10c15a720 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe112f59700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c150c90 con 0x7fe10c150690 2023-10-06T05:00:28.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe112f59700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c16af40 con 0x7fe10c0cba50 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe1114f6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c15a720 0x7fe10c16aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe1114f6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c15a720 0x7fe10c16aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34166/0 (socket says 172.21.15.3:34166) 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe1114f6700 1 -- 172.21.15.3:0/322416253 learned_addr learned my addr 172.21.15.3:0/322416253 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe110cf5700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe10c0cba50 0x7fe10c0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.953+0000 7fe10bfff700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe10c150690 0x7fe10c16d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe1114f6700 1 -- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe10c0cba50 msgr2=0x7fe10c0b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe1114f6700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe10c0cba50 0x7fe10c0b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe1114f6700 1 -- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe10c150690 msgr2=0x7fe10c16d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe1114f6700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe10c150690 0x7fe10c16d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe1114f6700 1 -- 172.21.15.3:0/322416253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104009ae0 con 0x7fe10c15a720 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe110cf5700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe10c0cba50 0x7fe10c0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe10bfff700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe10c150690 0x7fe10c16d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe1114f6700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c15a720 0x7fe10c16aa00 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b4e0 tx=0x7fe0fc006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe108ff9700 1 -- 172.21.15.3:0/322416253 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00f5e0 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe108ff9700 1 -- 172.21.15.3:0/322416253 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc00f740 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c16dc80 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe108ff9700 1 -- 172.21.15.3:0/322416253 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00fb00 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.954+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c16e240 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.955+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c150a60 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.955+0000 7fe108ff9700 1 -- 172.21.15.3:0/322416253 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0fc01a400 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.956+0000 7fe108ff9700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0ec066f30 0x7fe0ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.956+0000 7fe108ff9700 1 -- 172.21.15.3:0/322416253 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe0fc083f00 con 0x7fe10c15a720 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.956+0000 7fe110cf5700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0ec066f30 0x7fe0ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:28.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.957+0000 7fe110cf5700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0ec066f30 0x7fe0ec0693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe104009880 tx=0x7fe104017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:28.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:28.960+0000 7fe108ff9700 1 -- 172.21.15.3:0/322416253 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe10c150a60 con 0x7fe10c15a720 2023-10-06T05:00:29.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.151+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7fe10c150a60 con 0x7fe10c15a720 2023-10-06T05:00:29.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.157+0000 7fe108ff9700 1 -- 172.21.15.3:0/322416253 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v52) v1 ==== 102+0+0 (secure 0 0 0) 0x7fe10c150a60 con 0x7fe10c15a720 2023-10-06T05:00:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0ec066f30 msgr2=0x7fe0ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0ec066f30 0x7fe0ec0693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe104009880 tx=0x7fe104017000 comp rx=0 tx=0).stop 2023-10-06T05:00:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c15a720 msgr2=0x7fe10c16aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c15a720 0x7fe10c16aa00 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b4e0 tx=0x7fe0fc006f90 comp rx=0 tx=0).stop 2023-10-06T05:00:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 shutdown_connections 2023-10-06T05:00:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe0ec066f30 0x7fe0ec0693e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe10c15a720 0x7fe10c16aa00 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe10c0cba50 0x7fe10c0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 --2- 172.21.15.3:0/322416253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe10c150690 0x7fe10c16d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 >> 172.21.15.3:0/322416253 conn(0x7fe10c070310 msgr2=0x7fe10c159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 shutdown_connections 2023-10-06T05:00:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.159+0000 7fe112f59700 1 -- 172.21.15.3:0/322416253 wait complete. 2023-10-06T05:00:29.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-10-06T05:00:29.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.293+0000 7f10a86f4700 1 Processor -- start 2023-10-06T05:00:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.293+0000 7f10a86f4700 1 -- start start 2023-10-06T05:00:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a86f4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a86f4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10a00cb3c0 0x7f10a015db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a86f4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00cb810 con 0x7f10a00afec0 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a86f4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a00cf0f0 con 0x7f10a00cb3c0 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a86f4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a015e0c0 con 0x7f10a015c760 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a5c8f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10a00cb3c0 0x7f10a015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a6490700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a5c8f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10a00cb3c0 0x7f10a015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:33158/0 (socket says 172.21.15.3:33158) 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a6490700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a015cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:43642/0 (socket says 172.21.15.3:43642) 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a5c8f700 1 -- 172.21.15.3:0/4073328090 learned_addr learned my addr 172.21.15.3:0/4073328090 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.294+0000 7f10a6c91700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f10a00afec0 0x7f10a00cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:59450/0 (socket says 172.21.15.3:59450) 2023-10-06T05:00:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a548e700 1 -- 172.21.15.3:0/4073328090 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366444149 0 0) 0x7f1098001110 con 0x7f10a00afec0 2023-10-06T05:00:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a548e700 1 -- 172.21.15.3:0/4073328090 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1088003570 con 0x7f10a00afec0 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a6490700 1 -- 172.21.15.3:0/4073328090 >> v2:172.21.15.153:3300/0 conn(0x7f10a00cb3c0 msgr2=0x7f10a015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a6490700 1 --2- 172.21.15.3:0/4073328090 >> v2:172.21.15.153:3300/0 conn(0x7f10a00cb3c0 0x7f10a015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a6490700 1 -- 172.21.15.3:0/4073328090 >> v1:172.21.15.153:6789/0 conn(0x7f10a00afec0 legacy=0x7f10a00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a6490700 1 -- 172.21.15.3:0/4073328090 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00ce7c0 con 0x7f10a015c760 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a5c8f700 1 --2- 172.21.15.3:0/4073328090 >> v2:172.21.15.153:3300/0 conn(0x7f10a00cb3c0 0x7f10a015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a6490700 1 --2- 172.21.15.3:0/4073328090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a015cb30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1090009e60 tx=0x7f1090009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17707be95616b61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a548e700 1 -- 172.21.15.3:0/4073328090 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000e070 con 0x7f10a015c760 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a548e700 1 -- 172.21.15.3:0/4073328090 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109000bab0 con 0x7f10a015c760 2023-10-06T05:00:29.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.295+0000 7f10a548e700 1 -- 172.21.15.3:0/4073328090 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000be70 con 0x7f10a015c760 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 -- 172.21.15.3:0/4073328090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 msgr2=0x7f10a015cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 --2- 172.21.15.3:0/4073328090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a015cb30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1090009e60 tx=0x7f1090009780 comp rx=0 tx=0).stop 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 -- 172.21.15.3:0/4073328090 shutdown_connections 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 --2- 172.21.15.3:0/4073328090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a015cb30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 --2- 172.21.15.3:0/4073328090 >> v2:172.21.15.153:3300/0 conn(0x7f10a00cb3c0 0x7f10a015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 -- 172.21.15.3:0/4073328090 >> 172.21.15.3:0/4073328090 conn(0x7f10a006f940 msgr2=0x7f10a0070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 -- 172.21.15.3:0/4073328090 shutdown_connections 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.296+0000 7f10a86f4700 1 -- 172.21.15.3:0/4073328090 wait complete. 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.297+0000 7f10a86f4700 1 Processor -- start 2023-10-06T05:00:29.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.297+0000 7f10a86f4700 1 -- start start 2023-10-06T05:00:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a86f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10a00afec0 0x7f10a01578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a86f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10a00cb3c0 0x7f10a00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a86f4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a01531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a86f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00b3980 con 0x7f10a00afec0 2023-10-06T05:00:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a86f4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00b3c60 con 0x7f10a00cb3c0 2023-10-06T05:00:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a86f4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00c75a0 con 0x7f10a015c760 2023-10-06T05:00:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10a00afec0 0x7f10a01578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10a00afec0 0x7f10a01578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34168/0 (socket says 172.21.15.3:34168) 2023-10-06T05:00:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 -- 172.21.15.3:0/1633900375 learned_addr learned my addr 172.21.15.3:0/1633900375 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6c91700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a01531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a5c8f700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10a00cb3c0 0x7f10a00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 -- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 msgr2=0x7f10a01531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a01531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 -- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10a00cb3c0 msgr2=0x7f10a00b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10a00cb3c0 0x7f10a00b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 -- 172.21.15.3:0/1633900375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940094f0 con 0x7f10a00afec0 2023-10-06T05:00:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a5c8f700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10a00cb3c0 0x7f10a00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6c91700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a01531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a6490700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10a00afec0 0x7f10a01578b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f10900097c0 tx=0x7f1090019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f109e7fc700 1 -- 172.21.15.3:0/1633900375 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090012d00 con 0x7f10a00afec0 2023-10-06T05:00:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f109e7fc700 1 -- 172.21.15.3:0/1633900375 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090012e80 con 0x7f10a00afec0 2023-10-06T05:00:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f109e7fc700 1 -- 172.21.15.3:0/1633900375 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10900276c0 con 0x7f10a00afec0 2023-10-06T05:00:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.298+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1090009a20 con 0x7f10a00afec0 2023-10-06T05:00:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.299+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a00b3f40 con 0x7f10a00afec0 2023-10-06T05:00:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.299+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a004c380 con 0x7f10a00afec0 2023-10-06T05:00:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.300+0000 7f109e7fc700 1 -- 172.21.15.3:0/1633900375 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1090024070 con 0x7f10a00afec0 2023-10-06T05:00:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.300+0000 7f109e7fc700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f108c066f30 0x7f108c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.300+0000 7f109e7fc700 1 -- 172.21.15.3:0/1633900375 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1090086980 con 0x7f10a00afec0 2023-10-06T05:00:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.300+0000 7f10a5c8f700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f108c066f30 0x7f108c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.301+0000 7f10a5c8f700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f108c066f30 0x7f108c0693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f10940094c0 tx=0x7f1094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.305+0000 7f109e7fc700 1 -- 172.21.15.3:0/1633900375 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1090053d90 con 0x7f10a00afec0 2023-10-06T05:00:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.498+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f10a00ce120 con 0x7f10a00afec0 2023-10-06T05:00:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.505+0000 7f109e7fc700 1 -- 172.21.15.3:0/1633900375 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v53) v1 ==== 111+0+0 (secure 0 0 0) 0x7f109002c0b0 con 0x7f10a00afec0 2023-10-06T05:00:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f108c066f30 msgr2=0x7f108c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f108c066f30 0x7f108c0693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f10940094c0 tx=0x7f1094008040 comp rx=0 tx=0).stop 2023-10-06T05:00:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10a00afec0 msgr2=0x7f10a01578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10a00afec0 0x7f10a01578b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f10900097c0 tx=0x7f1090019000 comp rx=0 tx=0).stop 2023-10-06T05:00:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 shutdown_connections 2023-10-06T05:00:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f108c066f30 0x7f108c0693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f10a015c760 0x7f10a01531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10a00afec0 0x7f10a01578b0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 --2- 172.21.15.3:0/1633900375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10a00cb3c0 0x7f10a00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 >> 172.21.15.3:0/1633900375 conn(0x7f10a006f940 msgr2=0x7f10a0154120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 shutdown_connections 2023-10-06T05:00:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.507+0000 7f10a86f4700 1 -- 172.21.15.3:0/1633900375 wait complete. 2023-10-06T05:00:29.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2023-10-06T05:00:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.639+0000 7f3867f9f700 1 Processor -- start 2023-10-06T05:00:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.639+0000 7f3867f9f700 1 -- start start 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3867f9f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f38600d0870 0x7f38601644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3867f9f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600cf6c0 con 0x7f386015c8f0 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3867f9f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600cfb50 con 0x7f386015ccc0 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3867f9f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38600ceab0 con 0x7f38600d0870 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3865d3b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f386015c8f0 0x7f386015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52234/0 (socket says 172.21.15.3:52234) 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3865d3b700 1 -- 172.21.15.3:0/2981743359 learned_addr learned my addr 172.21.15.3:0/2981743359 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f386653c700 1 --2- 172.21.15.3:0/2981743359 >> v2:172.21.15.153:3300/0 conn(0x7f38600d0870 0x7f38601644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029434746 0 0) 0x7f38600cf6c0 con 0x7f386015c8f0 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3848003570 con 0x7f386015c8f0 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351262662 0 0) 0x7f3848003570 con 0x7f386015c8f0 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 >> v2:172.21.15.153:3300/0 conn(0x7f38600d0870 msgr2=0x7f38601644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 --2- 172.21.15.3:0/2981743359 >> v2:172.21.15.153:3300/0 conn(0x7f38600d0870 0x7f38601644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 >> v1:172.21.15.153:6789/0 conn(0x7f386015ccc0 legacy=0x7f38600d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600ced90 con 0x7f386015c8f0 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f3858001470 con 0x7f386015c8f0 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 >> v1:172.21.15.3:6789/0 conn(0x7f386015c8f0 legacy=0x7f386015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f3848003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.640+0000 7f3864d39700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3848004810 0x7f3848004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3864d39700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38480051c0 0x7f384800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600ceab0 con 0x7f3848003a60 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384800c0d0 con 0x7f3848004810 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384800c290 con 0x7f38480051c0 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3865d3b700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f3848003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f386553a700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3848004810 0x7f3848004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f386653c700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38480051c0 0x7f384800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3865d3b700 1 -- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38480051c0 msgr2=0x7f384800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3865d3b700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38480051c0 0x7f384800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3865d3b700 1 -- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3848004810 msgr2=0x7f3848004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3865d3b700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3848004810 0x7f3848004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3865d3b700 1 -- 172.21.15.3:0/2981743359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848003570 con 0x7f3848003a60 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f386553a700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3848004810 0x7f3848004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f386653c700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38480051c0 0x7f384800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f386653c700 1 -- 172.21.15.3:0/2981743359 reap_dead start 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3865d3b700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f3848003e70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f385800c3c0 tx=0x7f3858001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317fd30fb7f4a70d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800cc90 con 0x7f3848003a60 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800cdf0 con 0x7f3848003a60 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.641+0000 7f3864d39700 1 -- 172.21.15.3:0/2981743359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858005b50 con 0x7f3848003a60 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.642+0000 7f3867f9f700 1 -- 172.21.15.3:0/2981743359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 msgr2=0x7f3848003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.642+0000 7f3867f9f700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f3848003e70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f385800c3c0 tx=0x7f3858001e50 comp rx=0 tx=0).stop 2023-10-06T05:00:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.642+0000 7f3867f9f700 1 -- 172.21.15.3:0/2981743359 shutdown_connections 2023-10-06T05:00:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.642+0000 7f3867f9f700 1 --2- 172.21.15.3:0/2981743359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f3848003e70 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.642+0000 7f3867f9f700 1 -- 172.21.15.3:0/2981743359 >> 172.21.15.3:0/2981743359 conn(0x7f3860070310 msgr2=0x7f3860070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.642+0000 7f3867f9f700 1 -- 172.21.15.3:0/2981743359 shutdown_connections 2023-10-06T05:00:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.642+0000 7f3867f9f700 1 -- 172.21.15.3:0/2981743359 wait complete. 2023-10-06T05:00:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.643+0000 7f3867f9f700 1 Processor -- start 2023-10-06T05:00:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.643+0000 7f3867f9f700 1 -- start start 2023-10-06T05:00:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3867f9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f386014be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3867f9f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f386016a4a0 0x7f386016a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3867f9f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f386015b1b0 0x7f386015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3867f9f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860154da0 con 0x7f3848003a60 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3867f9f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386015bd30 con 0x7f386015b1b0 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3867f9f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3860154470 con 0x7f386016a4a0 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f386014be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f386014be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34196/0 (socket says 172.21.15.3:34196) 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 -- 172.21.15.3:0/3409711400 learned_addr learned my addr 172.21.15.3:0/3409711400 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 -- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f386016a4a0 msgr2=0x7f386016a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f386016a4a0 0x7f386016a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 -- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f386015b1b0 msgr2=0x7f386015b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f386015b1b0 0x7f386015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 -- 172.21.15.3:0/3409711400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860154750 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3865d3b700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f386014be40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f384800c0a0 tx=0x7f3858015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f385dffb700 1 -- 172.21.15.3:0/3409711400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800cc90 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f385dffb700 1 -- 172.21.15.3:0/3409711400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800c9e0 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860154a30 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.644+0000 7f385dffb700 1 -- 172.21.15.3:0/3409711400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858007530 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.645+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386014c380 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.646+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860071680 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.646+0000 7f385dffb700 1 -- 172.21.15.3:0/3409711400 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3858028460 con 0x7f3848003a60 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.646+0000 7f385dffb700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f384c066f30 0x7f384c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.646+0000 7f385dffb700 1 -- 172.21.15.3:0/3409711400 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3858085910 con 0x7f3848003a60 2023-10-06T05:00:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.647+0000 7f386553a700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f384c066f30 0x7f384c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.647+0000 7f386553a700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f384c066f30 0x7f384c0693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3854000c00 tx=0x7f3854009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.651+0000 7f385dffb700 1 -- 172.21.15.3:0/3409711400 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3858052d20 con 0x7f3848003a60 2023-10-06T05:00:29.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:29.926+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) v1 -- 0x7f3860156510 con 0x7f3848003a60 2023-10-06T05:00:30.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.763+0000 7f385dffb700 1 -- 172.21.15.3:0/3409711400 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v413) v1 ==== 151+0+0 (secure 0 0 0) 0x7f3858017020 con 0x7f3848003a60 2023-10-06T05:00:30.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f384c066f30 msgr2=0x7f384c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:30.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f384c066f30 0x7f384c0693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3854000c00 tx=0x7f3854009040 comp rx=0 tx=0).stop 2023-10-06T05:00:30.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 msgr2=0x7f386014be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:30.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f386014be40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f384800c0a0 tx=0x7f3858015000 comp rx=0 tx=0).stop 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 shutdown_connections 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f384c066f30 0x7f384c0693e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f386015b1b0 0x7f386015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3848003a60 0x7f386014be40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 --2- 172.21.15.3:0/3409711400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f386016a4a0 0x7f386016a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 >> 172.21.15.3:0/3409711400 conn(0x7f3860070310 msgr2=0x7f3860156fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 shutdown_connections 2023-10-06T05:00:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.766+0000 7f3867f9f700 1 -- 172.21.15.3:0/3409711400 wait complete. 2023-10-06T05:00:30.783 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-10-06T05:00:30.784 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T05:00:30.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2023-10-06T05:00:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.908+0000 7f9ba809b700 1 Processor -- start 2023-10-06T05:00:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.908+0000 7f9ba809b700 1 -- start start 2023-10-06T05:00:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba809b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba015cb50 0x7f9ba00b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba809b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ba00cd950 0x7f9ba00cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba809b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00b0ce0 con 0x7f9ba00b0590 2023-10-06T05:00:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba809b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba00cb820 con 0x7f9ba00cd950 2023-10-06T05:00:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba809b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00b0a00 con 0x7f9ba015cb50 2023-10-06T05:00:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba6638700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ba00cd950 0x7f9ba00cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba5636700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9ba00b0590 0x7f9ba00b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52240/0 (socket says 172.21.15.3:52240) 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba5636700 1 -- 172.21.15.3:0/3419725162 learned_addr learned my addr 172.21.15.3:0/3419725162 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba5e37700 1 --2- 172.21.15.3:0/3419725162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba015cb50 0x7f9ba00b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba6638700 1 -- 172.21.15.3:0/3419725162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba015cb50 msgr2=0x7f9ba00b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba6638700 1 --2- 172.21.15.3:0/3419725162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba015cb50 0x7f9ba00b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba6638700 1 -- 172.21.15.3:0/3419725162 >> v1:172.21.15.3:6789/0 conn(0x7f9ba00b0590 legacy=0x7f9ba00b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba6638700 1 -- 172.21.15.3:0/3419725162 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba00cb1d0 con 0x7f9ba00cd950 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba4e35700 1 -- 172.21.15.3:0/3419725162 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114040929 0 0) 0x7f9ba00b0ce0 con 0x7f9ba00b0590 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.909+0000 7f9ba5e37700 1 --2- 172.21.15.3:0/3419725162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba015cb50 0x7f9ba00b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba6638700 1 --2- 172.21.15.3:0/3419725162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00cd950 0x7f9ba00cbcb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9b980098e0 tx=0x7f9b980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a878c21680f8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba4e35700 1 -- 172.21.15.3:0/3419725162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98009e80 con 0x7f9ba00cd950 2023-10-06T05:00:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba4e35700 1 -- 172.21.15.3:0/3419725162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9800b860 con 0x7f9ba00cd950 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba4e35700 1 -- 172.21.15.3:0/3419725162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9800bbe0 con 0x7f9ba00cd950 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 -- 172.21.15.3:0/3419725162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00cd950 msgr2=0x7f9ba00cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3419725162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00cd950 0x7f9ba00cbcb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9b980098e0 tx=0x7f9b980091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 -- 172.21.15.3:0/3419725162 shutdown_connections 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3419725162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00cd950 0x7f9ba00cbcb0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3419725162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba015cb50 0x7f9ba00b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 -- 172.21.15.3:0/3419725162 >> 172.21.15.3:0/3419725162 conn(0x7f9ba0070310 msgr2=0x7f9ba0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 -- 172.21.15.3:0/3419725162 shutdown_connections 2023-10-06T05:00:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.910+0000 7f9ba809b700 1 -- 172.21.15.3:0/3419725162 wait complete. 2023-10-06T05:00:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.911+0000 7f9ba809b700 1 Processor -- start 2023-10-06T05:00:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba809b700 1 -- start start 2023-10-06T05:00:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba809b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00b0590 0x7f9ba00c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba809b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba00cd950 0x7f9ba014c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba809b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ba015cb50 0x7f9ba015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba809b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c6b00 con 0x7f9ba00b0590 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba809b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c72d0 con 0x7f9ba015cb50 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba809b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba014ce80 con 0x7f9ba00cd950 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba5e37700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00b0590 0x7f9ba00c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba5e37700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00b0590 0x7f9ba00c3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34208/0 (socket says 172.21.15.3:34208) 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.912+0000 7f9ba5e37700 1 -- 172.21.15.3:0/3258575298 learned_addr learned my addr 172.21.15.3:0/3258575298 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5636700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba00cd950 0x7f9ba014c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba6638700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ba015cb50 0x7f9ba015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5e37700 1 -- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba00cd950 msgr2=0x7f9ba014c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5e37700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba00cd950 0x7f9ba014c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5e37700 1 -- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ba015cb50 msgr2=0x7f9ba015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5e37700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ba015cb50 0x7f9ba015d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5e37700 1 -- 172.21.15.3:0/3258575298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98009420 con 0x7f9ba00b0590 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5636700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba00cd950 0x7f9ba014c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba5e37700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00b0590 0x7f9ba00c3e90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00a300 tx=0x7f9b9c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9b95ffb700 1 -- 172.21.15.3:0/3258575298 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c015440 con 0x7f9ba00b0590 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9b95ffb700 1 -- 172.21.15.3:0/3258575298 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0155a0 con 0x7f9ba00b0590 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba015df10 con 0x7f9ba00b0590 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9b95ffb700 1 -- 172.21.15.3:0/3258575298 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c015920 con 0x7f9ba00b0590 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.913+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba00c6ff0 con 0x7f9ba00b0590 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.914+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba0071680 con 0x7f9ba00b0590 2023-10-06T05:00:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.915+0000 7f9b95ffb700 1 -- 172.21.15.3:0/3258575298 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b9c015cd0 con 0x7f9ba00b0590 2023-10-06T05:00:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.915+0000 7f9b95ffb700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9b84066f30 0x7f9b840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.915+0000 7f9b95ffb700 1 -- 172.21.15.3:0/3258575298 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9b9c084a70 con 0x7f9ba00b0590 2023-10-06T05:00:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.915+0000 7f9ba5636700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9b84066f30 0x7f9b840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.916+0000 7f9ba5636700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9b84066f30 0x7f9b840693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9ba016c950 tx=0x7f9b90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:30.919+0000 7f9b95ffb700 1 -- 172.21.15.3:0/3258575298 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b9c04e5a0 con 0x7f9ba00b0590 2023-10-06T05:00:31.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.192+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) v1 -- 0x7f9ba01601c0 con 0x7f9ba00b0590 2023-10-06T05:00:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.770+0000 7f9b95ffb700 1 -- 172.21.15.3:0/3258575298 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v414) v1 ==== 151+0+0 (secure 0 0 0) 0x7f9b9c01a0d0 con 0x7f9ba00b0590 2023-10-06T05:00:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9b84066f30 msgr2=0x7f9b840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9b84066f30 0x7f9b840693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9ba016c950 tx=0x7f9b90009040 comp rx=0 tx=0).stop 2023-10-06T05:00:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00b0590 msgr2=0x7f9ba00c3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00b0590 0x7f9ba00c3e90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00a300 tx=0x7f9b9c0078c0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 shutdown_connections 2023-10-06T05:00:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9b84066f30 0x7f9b840693e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ba015cb50 0x7f9ba015d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00b0590 0x7f9ba00c3e90 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 --2- 172.21.15.3:0/3258575298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ba00cd950 0x7f9ba014c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 >> 172.21.15.3:0/3258575298 conn(0x7f9ba0070310 msgr2=0x7f9ba00b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 shutdown_connections 2023-10-06T05:00:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.772+0000 7f9ba809b700 1 -- 172.21.15.3:0/3258575298 wait complete. 2023-10-06T05:00:31.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:00:31.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.909+0000 7fc3eb922700 1 Processor -- start 2023-10-06T05:00:31.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.909+0000 7fc3eb922700 1 -- start start 2023-10-06T05:00:31.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.909+0000 7fc3eb922700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e4150690 0x7fc3e4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:31.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3eb922700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e415a290 con 0x7fc3e40cba50 2023-10-06T05:00:31.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3eb922700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e40d0020 con 0x7fc3e415d540 2023-10-06T05:00:31.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3eb922700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4159f40 con 0x7fc3e4150690 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e8ebd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc3e40cba50 0x7fc3e415a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52248/0 (socket says 172.21.15.3:52248) 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e8ebd700 1 -- 172.21.15.3:0/3313095320 learned_addr learned my addr 172.21.15.3:0/3313095320 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e96be700 1 --2- 172.21.15.3:0/3313095320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e4150690 0x7fc3e4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699525922 0 0) 0x7fc3e415a290 con 0x7fc3e40cba50 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3cc003570 con 0x7fc3e40cba50 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978563031 0 0) 0x7fc3e40d0020 con 0x7fc3e415d540 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3e415a290 con 0x7fc3e415d540 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e96be700 1 -- 172.21.15.3:0/3313095320 >> v1:172.21.15.153:6789/0 conn(0x7fc3e415d540 legacy=0x7fc3e40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e96be700 1 -- 172.21.15.3:0/3313095320 >> v1:172.21.15.3:6789/0 conn(0x7fc3e40cba50 legacy=0x7fc3e415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126106451 0 0) 0x7fc3cc003570 con 0x7fc3e40cba50 2023-10-06T05:00:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e96be700 1 -- 172.21.15.3:0/3313095320 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e415dcb0 con 0x7fc3e4150690 2023-10-06T05:00:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.910+0000 7fc3e96be700 1 --2- 172.21.15.3:0/3313095320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e4150690 0x7fc3e4150a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009e90 tx=0x7fc3d4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fe8cb8f99764c7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400e070 con 0x7fc3e4150690 2023-10-06T05:00:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d400b870 con 0x7fc3e4150690 2023-10-06T05:00:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3e3fff700 1 -- 172.21.15.3:0/3313095320 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8001470 con 0x7fc3e4150690 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3eb922700 1 -- 172.21.15.3:0/3313095320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e4150690 msgr2=0x7fc3e4150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3eb922700 1 --2- 172.21.15.3:0/3313095320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e4150690 0x7fc3e4150a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009e90 tx=0x7fc3d4009780 comp rx=0 tx=0).stop 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3eb922700 1 -- 172.21.15.3:0/3313095320 shutdown_connections 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3eb922700 1 --2- 172.21.15.3:0/3313095320 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e4150690 0x7fc3e4150a60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3eb922700 1 -- 172.21.15.3:0/3313095320 >> 172.21.15.3:0/3313095320 conn(0x7fc3e4070310 msgr2=0x7fc3e4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3eb922700 1 -- 172.21.15.3:0/3313095320 shutdown_connections 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.911+0000 7fc3eb922700 1 -- 172.21.15.3:0/3313095320 wait complete. 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.912+0000 7fc3eb922700 1 Processor -- start 2023-10-06T05:00:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.912+0000 7fc3eb922700 1 -- start start 2023-10-06T05:00:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3eb922700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3e40cba50 0x7fc3e40c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3eb922700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e4150690 0x7fc3e4152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3eb922700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e415d540 0x7fc3e4151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3eb922700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40c8ae0 con 0x7fc3e40cba50 2023-10-06T05:00:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3eb922700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4151dc0 con 0x7fc3e4150690 2023-10-06T05:00:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3eb922700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40b3360 con 0x7fc3e415d540 2023-10-06T05:00:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3e96be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3e40cba50 0x7fc3e40c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3e96be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3e40cba50 0x7fc3e40c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34222/0 (socket says 172.21.15.3:34222) 2023-10-06T05:00:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.913+0000 7fc3e96be700 1 -- 172.21.15.3:0/1522641979 learned_addr learned my addr 172.21.15.3:0/1522641979 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e9ebf700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e415d540 0x7fc3e4151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e96be700 1 -- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e415d540 msgr2=0x7fc3e4151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e96be700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e415d540 0x7fc3e4151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e8ebd700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e4150690 0x7fc3e4152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e96be700 1 -- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e4150690 msgr2=0x7fc3e4152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e96be700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e4150690 0x7fc3e4152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e96be700 1 -- 172.21.15.3:0/1522641979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d4009a20 con 0x7fc3e40cba50 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e96be700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3e40cba50 0x7fc3e40c85a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40c71f0 tx=0x7fc3d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e17fa700 1 -- 172.21.15.3:0/1522641979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4012c90 con 0x7fc3e40cba50 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e17fa700 1 -- 172.21.15.3:0/1522641979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d4012e10 con 0x7fc3e40cba50 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e40b3640 con 0x7fc3e40cba50 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3e17fa700 1 -- 172.21.15.3:0/1522641979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d401e6a0 con 0x7fc3e40cba50 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.914+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e41532e0 con 0x7fc3e40cba50 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.915+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e41534d0 con 0x7fc3e40cba50 2023-10-06T05:00:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.915+0000 7fc3e17fa700 1 -- 172.21.15.3:0/1522641979 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3d401ead0 con 0x7fc3e40cba50 2023-10-06T05:00:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.916+0000 7fc3e17fa700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3d0066f30 0x7fc3d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.916+0000 7fc3e17fa700 1 -- 172.21.15.3:0/1522641979 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc3d4086720 con 0x7fc3e40cba50 2023-10-06T05:00:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.916+0000 7fc3e8ebd700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3d0066f30 0x7fc3d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.917+0000 7fc3e8ebd700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3d0066f30 0x7fc3d00693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80011e0 tx=0x7fc3d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:31.920+0000 7fc3e17fa700 1 -- 172.21.15.3:0/1522641979 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3d4053b30 con 0x7fc3e40cba50 2023-10-06T05:00:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.113+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc3e404c380 con 0x7fc3e40cba50 2023-10-06T05:00:32.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.119+0000 7fc3e17fa700 1 -- 172.21.15.3:0/1522641979 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc3d4057150 con 0x7fc3e40cba50 2023-10-06T05:00:32.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3d0066f30 msgr2=0x7fc3d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3d0066f30 0x7fc3d00693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80011e0 tx=0x7fc3d8009040 comp rx=0 tx=0).stop 2023-10-06T05:00:32.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3e40cba50 msgr2=0x7fc3e40c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3e40cba50 0x7fc3e40c85a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40c71f0 tx=0x7fc3d4017000 comp rx=0 tx=0).stop 2023-10-06T05:00:32.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 shutdown_connections 2023-10-06T05:00:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3d0066f30 0x7fc3d00693e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3e415d540 0x7fc3e4151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3e40cba50 0x7fc3e40c85a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 --2- 172.21.15.3:0/1522641979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e4150690 0x7fc3e4152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 >> 172.21.15.3:0/1522641979 conn(0x7fc3e4070310 msgr2=0x7fc3e415cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 shutdown_connections 2023-10-06T05:00:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.121+0000 7fc3eb922700 1 -- 172.21.15.3:0/1522641979 wait complete. 2023-10-06T05:00:32.136 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:00:32.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:00:32.177 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:00:32.220 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:00:32.220 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:00:32.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:32.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.337+0000 7f7a33300700 1 Processor -- start 2023-10-06T05:00:32.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.337+0000 7f7a33300700 1 -- start start 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a33300700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a2c15c8f0 0x7f7a2c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a33300700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c0d0870 0x7f7a2c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a33300700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2c0cf6c0 con 0x7f7a2c15ccc0 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a33300700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2c0cfb50 con 0x7f7a2c15c8f0 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a33300700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c15e050 con 0x7f7a2c0d0870 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a2c15c8f0 0x7f7a2c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a2c15c8f0 0x7f7a2c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45792/0 (socket says 172.21.15.3:45792) 2023-10-06T05:00:32.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 -- 172.21.15.3:0/35365743 learned_addr learned my addr 172.21.15.3:0/35365743 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 -- 172.21.15.3:0/35365743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c0d0870 msgr2=0x7f7a2c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3189d700 1 --2- 172.21.15.3:0/35365743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c0d0870 0x7f7a2c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 --2- 172.21.15.3:0/35365743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c0d0870 0x7f7a2c0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 -- 172.21.15.3:0/35365743 >> v1:172.21.15.153:6789/0 conn(0x7f7a2c15ccc0 legacy=0x7f7a2c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 -- 172.21.15.3:0/35365743 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c0ceab0 con 0x7f7a2c15c8f0 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.338+0000 7f7a3109c700 1 --2- 172.21.15.3:0/35365743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c15c8f0 0x7f7a2c15db10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7a240099b0 tx=0x7f7a24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e887d9c6bbb94788 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a2bfff700 1 -- 172.21.15.3:0/35365743 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400b820 con 0x7f7a2c15c8f0 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a2bfff700 1 -- 172.21.15.3:0/35365743 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2400b980 con 0x7f7a2c15c8f0 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a2bfff700 1 -- 172.21.15.3:0/35365743 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400bd20 con 0x7f7a2c15c8f0 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 -- 172.21.15.3:0/35365743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c15c8f0 msgr2=0x7f7a2c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 --2- 172.21.15.3:0/35365743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c15c8f0 0x7f7a2c15db10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7a240099b0 tx=0x7f7a24009250 comp rx=0 tx=0).stop 2023-10-06T05:00:32.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 -- 172.21.15.3:0/35365743 shutdown_connections 2023-10-06T05:00:32.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 --2- 172.21.15.3:0/35365743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c0d0870 0x7f7a2c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 --2- 172.21.15.3:0/35365743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c15c8f0 0x7f7a2c15db10 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 -- 172.21.15.3:0/35365743 >> 172.21.15.3:0/35365743 conn(0x7f7a2c070310 msgr2=0x7f7a2c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:32.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 -- 172.21.15.3:0/35365743 shutdown_connections 2023-10-06T05:00:32.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.339+0000 7f7a33300700 1 -- 172.21.15.3:0/35365743 wait complete. 2023-10-06T05:00:32.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.340+0000 7f7a33300700 1 Processor -- start 2023-10-06T05:00:32.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.340+0000 7f7a33300700 1 -- start start 2023-10-06T05:00:32.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a33300700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c0d0870 0x7f7a2c0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a33300700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c15c8f0 0x7f7a2c16a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a33300700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a2c15ccc0 0x7f7a2c16ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a33300700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c0b5050 con 0x7f7a2c0d0870 2023-10-06T05:00:32.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a33300700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c16b2e0 con 0x7f7a2c15ccc0 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a33300700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c0b4440 con 0x7f7a2c15c8f0 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a3109c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c0d0870 0x7f7a2c0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a3109c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c0d0870 0x7f7a2c0b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45802/0 (socket says 172.21.15.3:45802) 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a3109c700 1 -- 172.21.15.3:0/1585696425 learned_addr learned my addr 172.21.15.3:0/1585696425 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.341+0000 7f7a3089b700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c15c8f0 0x7f7a2c16a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a3189d700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a2c15ccc0 0x7f7a2c16ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a3109c700 1 -- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c15c8f0 msgr2=0x7f7a2c16a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a3109c700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c15c8f0 0x7f7a2c16a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a3109c700 1 -- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a2c15ccc0 msgr2=0x7f7a2c16ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a3109c700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a2c15ccc0 0x7f7a2c16ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a3109c700 1 -- 172.21.15.3:0/1585696425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a240094f0 con 0x7f7a2c0d0870 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a3109c700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c0d0870 0x7f7a2c0b54e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009290 tx=0x7f7a24016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a297fa700 1 -- 172.21.15.3:0/1585696425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24011cc0 con 0x7f7a2c0d0870 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a297fa700 1 -- 172.21.15.3:0/1585696425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a24020070 con 0x7f7a2c0d0870 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a297fa700 1 -- 172.21.15.3:0/1585696425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2401b6a0 con 0x7f7a2c0d0870 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2c0b4720 con 0x7f7a2c0d0870 2023-10-06T05:00:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.342+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2c0b4ce0 con 0x7f7a2c0d0870 2023-10-06T05:00:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.343+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2c071680 con 0x7f7a2c0d0870 2023-10-06T05:00:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.343+0000 7f7a297fa700 1 -- 172.21.15.3:0/1585696425 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a24022030 con 0x7f7a2c0d0870 2023-10-06T05:00:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.344+0000 7f7a297fa700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a18066f30 0x7f7a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.344+0000 7f7a297fa700 1 -- 172.21.15.3:0/1585696425 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7a24086290 con 0x7f7a2c0d0870 2023-10-06T05:00:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.344+0000 7f7a3089b700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a18066f30 0x7f7a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.345+0000 7f7a3089b700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a18066f30 0x7f7a180693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c14dbc0 tx=0x7f7a200090f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.348+0000 7f7a297fa700 1 -- 172.21.15.3:0/1585696425 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a240536a0 con 0x7f7a2c0d0870 2023-10-06T05:00:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.586+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a2c071680 con 0x7f7a2c0d0870 2023-10-06T05:00:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.586+0000 7f7a297fa700 1 -- 172.21.15.3:0/1585696425 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f7a24009350 con 0x7f7a2c0d0870 2023-10-06T05:00:32.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:32.589 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3172458912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3172458912},{"type":"v1","addr":"172.21.15.153:6837","nonce":3172458912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1631923157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1631923157},{"type":"v1","addr":"172.21.15.153:6836","nonce":1631923157}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1051234909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1051234909},{"type":"v1","addr":"172.21.15.3:6834","nonce":1051234909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7908,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2925080339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2925080339},{"type":"v1","addr":"172.21.15.3:6835","nonce":2925080339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[]} 2023-10-06T05:00:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a18066f30 msgr2=0x7f7a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a18066f30 0x7f7a180693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c14dbc0 tx=0x7f7a200090f0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c0d0870 msgr2=0x7f7a2c0b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c0d0870 0x7f7a2c0b54e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009290 tx=0x7f7a24016000 comp rx=0 tx=0).stop 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 shutdown_connections 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7a18066f30 0x7f7a180693e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a2c15ccc0 0x7f7a2c16ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a2c0d0870 0x7f7a2c0b54e0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 --2- 172.21.15.3:0/1585696425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a2c15c8f0 0x7f7a2c16a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 >> 172.21.15.3:0/1585696425 conn(0x7f7a2c070310 msgr2=0x7f7a2c0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 shutdown_connections 2023-10-06T05:00:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.589+0000 7f7a33300700 1 -- 172.21.15.3:0/1585696425 wait complete. 2023-10-06T05:00:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 74 2023-10-06T05:00:32.603 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:00:32.604 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:00:32.604 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:00:32.604 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:00:32.604 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:00:32.604 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:00:32.604 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:00:32.605 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:00:32.605 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:00:32.605 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:00:32.605 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:00:32.605 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:00:32.605 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:00:32.605 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:00:32.606 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:00:32.606 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:00:32.612 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:00:32.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:00:32.613 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:00:32.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:00:32.619 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:00:32.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:00:32.620 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:00:32.620 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:00:32.654 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:00:32.655 DEBUG:teuthology.parallel:result is None 2023-10-06T05:00:32.655 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:00:32.655 DEBUG:teuthology.parallel:result is None 2023-10-06T05:00:32.662 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:00:32.662 DEBUG:teuthology.parallel:result is None 2023-10-06T05:00:32.663 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:00:32.663 DEBUG:teuthology.parallel:result is None 2023-10-06T05:00:32.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:00:32.731 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T05:00:32.732 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T05:00:32.742 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T05:00:32.743 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T05:00:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.791+0000 7fbadd665700 1 Processor -- start 2023-10-06T05:00:32.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.791+0000 7fbadd665700 1 -- start start 2023-10-06T05:00:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbadd665700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad80cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbadd665700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbad80cd360 0x7fbad806e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbadd665700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80a9860 con 0x7fbad806ed30 2023-10-06T05:00:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbadd665700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad806f1d0 con 0x7fbad80cd360 2023-10-06T05:00:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbadd665700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad806c6e0 con 0x7fbad8159260 2023-10-06T05:00:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbad80cd360 0x7fbad806e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad7fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad80cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbadce64700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbad806ed30 0x7fbad815cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:33778/0 (socket says 172.21.15.3:33778) 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbad80cd360 0x7fbad806e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:43032/0 (socket says 172.21.15.3:43032) 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbadce64700 1 -- 172.21.15.3:0/1042562829 learned_addr learned my addr 172.21.15.3:0/1042562829 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad7fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad80cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:37786/0 (socket says 172.21.15.3:37786) 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad6ffd700 1 -- 172.21.15.3:0/1042562829 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499139262 0 0) 0x7fbad80a9860 con 0x7fbad806ed30 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad6ffd700 1 -- 172.21.15.3:0/1042562829 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac0003570 con 0x7fbad806ed30 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad7fff700 1 -- 172.21.15.3:0/1042562829 >> v2:172.21.15.153:3300/0 conn(0x7fbad80cd360 msgr2=0x7fbad806e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad7fff700 1 --2- 172.21.15.3:0/1042562829 >> v2:172.21.15.153:3300/0 conn(0x7fbad80cd360 0x7fbad806e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad7fff700 1 -- 172.21.15.3:0/1042562829 >> v1:172.21.15.153:6789/0 conn(0x7fbad806ed30 legacy=0x7fbad815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad7fff700 1 -- 172.21.15.3:0/1042562829 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80c6de0 con 0x7fbad8159260 2023-10-06T05:00:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.792+0000 7fbad77fe700 1 --2- 172.21.15.3:0/1042562829 >> v2:172.21.15.153:3300/0 conn(0x7fbad80cd360 0x7fbad806e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbad7fff700 1 --2- 172.21.15.3:0/1042562829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad80cce20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbac80098e0 tx=0x7fbac80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2146b493ccd85291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbad6ffd700 1 -- 172.21.15.3:0/1042562829 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011630 con 0x7fbad8159260 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbad6ffd700 1 -- 172.21.15.3:0/1042562829 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac8011790 con 0x7fbad8159260 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbad6ffd700 1 -- 172.21.15.3:0/1042562829 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011b30 con 0x7fbad8159260 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 -- 172.21.15.3:0/1042562829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 msgr2=0x7fbad80cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 --2- 172.21.15.3:0/1042562829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad80cce20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbac80098e0 tx=0x7fbac80091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 -- 172.21.15.3:0/1042562829 shutdown_connections 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 --2- 172.21.15.3:0/1042562829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad80cce20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 --2- 172.21.15.3:0/1042562829 >> v2:172.21.15.153:3300/0 conn(0x7fbad80cd360 0x7fbad806e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 -- 172.21.15.3:0/1042562829 >> 172.21.15.3:0/1042562829 conn(0x7fbad8067a80 msgr2=0x7fbad80cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 -- 172.21.15.3:0/1042562829 shutdown_connections 2023-10-06T05:00:32.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.793+0000 7fbadd665700 1 -- 172.21.15.3:0/1042562829 wait complete. 2023-10-06T05:00:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 Processor -- start 2023-10-06T05:00:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 -- start start 2023-10-06T05:00:32.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad806ed30 0x7fbad80aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbad80cd360 0x7fbad815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b23f0 con 0x7fbad806ed30 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b1f60 con 0x7fbad80cd360 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.795+0000 7fbadd665700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b1350 con 0x7fbad8159260 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad806ed30 0x7fbad80aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad806ed30 0x7fbad80aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45858/0 (socket says 172.21.15.3:45858) 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 -- 172.21.15.3:0/1367182734 learned_addr learned my addr 172.21.15.3:0/1367182734 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbadce64700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad77fe700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbad80cd360 0x7fbad815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 -- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 msgr2=0x7fbad815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 -- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbad80cd360 msgr2=0x7fbad815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbad80cd360 0x7fbad815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 -- 172.21.15.3:0/1367182734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009420 con 0x7fbad806ed30 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbadce64700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad77fe700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbad80cd360 0x7fbad815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbad7fff700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad806ed30 0x7fbad80aa650 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbad806f620 tx=0x7fbac8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbabffff700 1 -- 172.21.15.3:0/1367182734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011b10 con 0x7fbad806ed30 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac8009420 con 0x7fbad806ed30 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbabffff700 1 -- 172.21.15.3:0/1367182734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac80233f0 con 0x7fbad806ed30 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbabffff700 1 -- 172.21.15.3:0/1367182734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8023770 con 0x7fbad806ed30 2023-10-06T05:00:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.796+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad80b1910 con 0x7fbad806ed30 2023-10-06T05:00:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.797+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad80b2240 con 0x7fbad806ed30 2023-10-06T05:00:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.798+0000 7fbabffff700 1 -- 172.21.15.3:0/1367182734 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbac802c3b0 con 0x7fbad806ed30 2023-10-06T05:00:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.798+0000 7fbabffff700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbac4066f30 0x7fbac40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.798+0000 7fbabffff700 1 -- 172.21.15.3:0/1367182734 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fbac80857f0 con 0x7fbad806ed30 2023-10-06T05:00:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.798+0000 7fbad77fe700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbac4066f30 0x7fbac40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.799+0000 7fbad77fe700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbac4066f30 0x7fbac40693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0093f0 tx=0x7fbacc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:32.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:32.802+0000 7fbabffff700 1 -- 172.21.15.3:0/1367182734 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbac8052c00 con 0x7fbad806ed30 2023-10-06T05:00:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.013+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbad804c380 con 0x7fbad806ed30 2023-10-06T05:00:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.782+0000 7fbabffff700 1 -- 172.21.15.3:0/1367182734 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbac8028080 con 0x7fbad806ed30 2023-10-06T05:00:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbac4066f30 msgr2=0x7fbac40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbac4066f30 0x7fbac40693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0093f0 tx=0x7fbacc008040 comp rx=0 tx=0).stop 2023-10-06T05:00:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad806ed30 msgr2=0x7fbad80aa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad806ed30 0x7fbad80aa650 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbad806f620 tx=0x7fbac8017000 comp rx=0 tx=0).stop 2023-10-06T05:00:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 shutdown_connections 2023-10-06T05:00:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbac4066f30 0x7fbac40693e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbad8159260 0x7fbad815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad806ed30 0x7fbad80aa650 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 --2- 172.21.15.3:0/1367182734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbad80cd360 0x7fbad815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 >> 172.21.15.3:0/1367182734 conn(0x7fbad8067a80 msgr2=0x7fbad80cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 shutdown_connections 2023-10-06T05:00:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.785+0000 7fbadd665700 1 -- 172.21.15.3:0/1367182734 wait complete. 2023-10-06T05:00:33.789 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T05:00:33.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:00:33.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.924+0000 7fc9eed65700 1 Processor -- start 2023-10-06T05:00:33.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.924+0000 7fc9eed65700 1 -- start start 2023-10-06T05:00:33.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9eed65700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc9e815cb50 0x7fc9e80b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:33.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9eed65700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:33.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9eed65700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e80b0ce0 con 0x7fc9e80cd950 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9eed65700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e80cccc0 con 0x7fc9e815cb50 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9eed65700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80cb820 con 0x7fc9e80b0590 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9ecb01700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc9e815cb50 0x7fc9e80b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9ecb01700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc9e815cb50 0x7fc9e80b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:43052/0 (socket says 172.21.15.3:43052) 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e80b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:37804/0 (socket says 172.21.15.3:37804) 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9ecb01700 1 -- 172.21.15.3:0/2736663032 learned_addr learned my addr 172.21.15.3:0/2736663032 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.925+0000 7fc9ed302700 1 --1- 172.21.15.3:0/2736663032 >> v1:172.21.15.153:6789/0 conn(0x7fc9e80cd950 0x7fc9e806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:33784/0 (socket says 172.21.15.3:33784) 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e7fff700 1 -- 172.21.15.3:0/2736663032 >> v2:172.21.15.153:3300/0 conn(0x7fc9e815cb50 msgr2=0x7fc9e80b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e7fff700 1 --2- 172.21.15.3:0/2736663032 >> v2:172.21.15.153:3300/0 conn(0x7fc9e815cb50 0x7fc9e80b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e7fff700 1 -- 172.21.15.3:0/2736663032 >> v1:172.21.15.153:6789/0 conn(0x7fc9e80cd950 legacy=0x7fc9e806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e77fe700 1 -- 172.21.15.3:0/2736663032 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103454281 0 0) 0x7fc9e80b0ce0 con 0x7fc9e80cd950 2023-10-06T05:00:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e7fff700 1 -- 172.21.15.3:0/2736663032 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e80b0a00 con 0x7fc9e80b0590 2023-10-06T05:00:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9ecb01700 1 --2- 172.21.15.3:0/2736663032 >> v2:172.21.15.153:3300/0 conn(0x7fc9e815cb50 0x7fc9e80b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e7fff700 1 --2- 172.21.15.3:0/2736663032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e80b2200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80098e0 tx=0x7fc9d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b35b7cd9a96cef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e77fe700 1 -- 172.21.15.3:0/2736663032 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800b640 con 0x7fc9e80b0590 2023-10-06T05:00:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e77fe700 1 -- 172.21.15.3:0/2736663032 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d800b7a0 con 0x7fc9e80b0590 2023-10-06T05:00:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9e77fe700 1 -- 172.21.15.3:0/2736663032 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800bb40 con 0x7fc9e80b0590 2023-10-06T05:00:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9eed65700 1 -- 172.21.15.3:0/2736663032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 msgr2=0x7fc9e80b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9eed65700 1 --2- 172.21.15.3:0/2736663032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e80b2200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80098e0 tx=0x7fc9d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9eed65700 1 -- 172.21.15.3:0/2736663032 shutdown_connections 2023-10-06T05:00:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9eed65700 1 --2- 172.21.15.3:0/2736663032 >> v2:172.21.15.153:3300/0 conn(0x7fc9e815cb50 0x7fc9e80b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9eed65700 1 --2- 172.21.15.3:0/2736663032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e80b2200 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.926+0000 7fc9eed65700 1 -- 172.21.15.3:0/2736663032 >> 172.21.15.3:0/2736663032 conn(0x7fc9e8070310 msgr2=0x7fc9e8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.927+0000 7fc9eed65700 1 -- 172.21.15.3:0/2736663032 shutdown_connections 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.927+0000 7fc9eed65700 1 -- 172.21.15.3:0/2736663032 wait complete. 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 Processor -- start 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 -- start start 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e816c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9e80cd950 0x7fc9e80d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e815cb50 0x7fc9e80d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8154340 con 0x7fc9e815cb50 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8150dc0 con 0x7fc9e80cd950 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.928+0000 7fc9eed65700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e81536c0 con 0x7fc9e80b0590 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e815cb50 0x7fc9e80d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e815cb50 0x7fc9e80d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45870/0 (socket says 172.21.15.3:45870) 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 -- 172.21.15.3:0/1502200668 learned_addr learned my addr 172.21.15.3:0/1502200668 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ecb01700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e816c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9e7fff700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9e80cd950 0x7fc9e80d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 -- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 msgr2=0x7fc9e816c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e816c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 -- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9e80cd950 msgr2=0x7fc9e80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9e80cd950 0x7fc9e80d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 -- 172.21.15.3:0/1502200668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0009480 con 0x7fc9e815cb50 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9e7fff700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9e80cd950 0x7fc9e80d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9ed302700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e815cb50 0x7fc9e80d07e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000cd50 tx=0x7fc9e0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9e4ff9700 1 -- 172.21.15.3:0/1502200668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0005df0 con 0x7fc9e815cb50 2023-10-06T05:00:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9e4ff9700 1 -- 172.21.15.3:0/1502200668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000a960 con 0x7fc9e815cb50 2023-10-06T05:00:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d8009420 con 0x7fc9e815cb50 2023-10-06T05:00:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9e4ff9700 1 -- 172.21.15.3:0/1502200668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000ad20 con 0x7fc9e815cb50 2023-10-06T05:00:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.929+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8153c80 con 0x7fc9e815cb50 2023-10-06T05:00:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.930+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e8071680 con 0x7fc9e815cb50 2023-10-06T05:00:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.931+0000 7fc9e4ff9700 1 -- 172.21.15.3:0/1502200668 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9e0007300 con 0x7fc9e815cb50 2023-10-06T05:00:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.931+0000 7fc9e4ff9700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9c8066f30 0x7fc9c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.931+0000 7fc9e4ff9700 1 -- 172.21.15.3:0/1502200668 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc9e00845d0 con 0x7fc9e815cb50 2023-10-06T05:00:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.931+0000 7fc9ecb01700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9c8066f30 0x7fc9c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.932+0000 7fc9ecb01700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9c8066f30 0x7fc9c80693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0006ce0 tx=0x7fc9d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:33.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:33.935+0000 7fc9e4ff9700 1 -- 172.21.15.3:0/1502200668 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9e0051ab0 con 0x7fc9e815cb50 2023-10-06T05:00:34.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.210+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc9e80cb1d0 con 0x7fc9e815cb50 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.210+0000 7fc9e4ff9700 1 -- 172.21.15.3:0/1502200668 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc9e00550d0 con 0x7fc9e815cb50 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T05:00:34.213 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.214 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.215 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:00:34.216 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.217 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T05:00:34.218 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.219 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.220 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.221 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.222 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:00:34.223 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.225 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.226 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.227 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T05:00:34.228 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T05:00:34.229 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:00:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9c8066f30 msgr2=0x7fc9c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9c8066f30 0x7fc9c80693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0006ce0 tx=0x7fc9d0008040 comp rx=0 tx=0).stop 2023-10-06T05:00:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e815cb50 msgr2=0x7fc9e80d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e815cb50 0x7fc9e80d07e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000cd50 tx=0x7fc9e0001b10 comp rx=0 tx=0).stop 2023-10-06T05:00:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 shutdown_connections 2023-10-06T05:00:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9c8066f30 0x7fc9c80693e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e815cb50 0x7fc9e80d07e0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9e80b0590 0x7fc9e816c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 --2- 172.21.15.3:0/1502200668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9e80cd950 0x7fc9e80d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 >> 172.21.15.3:0/1502200668 conn(0x7fc9e8070310 msgr2=0x7fc9e8151480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:34.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 shutdown_connections 2023-10-06T05:00:34.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.212+0000 7fc9eed65700 1 -- 172.21.15.3:0/1502200668 wait complete. 2023-10-06T05:00:34.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:00:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.349+0000 7fbbb37c4700 1 Processor -- start 2023-10-06T05:00:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.349+0000 7fbbb37c4700 1 -- start start 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb37c4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbbac1598d0 0x7fbbac0b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb37c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbac0cb440 0x7fbbac0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb37c4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbac0cc5c0 con 0x7fbbac159500 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb37c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac0cda60 con 0x7fbbac0cb440 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb37c4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac0b2ca0 con 0x7fbbac1598d0 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb1d61700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbac0cb440 0x7fbbac0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb1d61700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbac0cb440 0x7fbbac0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45876/0 (socket says 172.21.15.3:45876) 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.350+0000 7fbbb1d61700 1 -- 172.21.15.3:0/3613228011 learned_addr learned my addr 172.21.15.3:0/3613228011 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbb0d5f700 1 --2- 172.21.15.3:0/3613228011 >> v2:172.21.15.153:3300/0 conn(0x7fbbac1598d0 0x7fbbac0b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbb1d61700 1 -- 172.21.15.3:0/3613228011 >> v2:172.21.15.153:3300/0 conn(0x7fbbac1598d0 msgr2=0x7fbbac0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbb1d61700 1 --2- 172.21.15.3:0/3613228011 >> v2:172.21.15.153:3300/0 conn(0x7fbbac1598d0 0x7fbbac0b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbb1d61700 1 -- 172.21.15.3:0/3613228011 >> v1:172.21.15.153:6789/0 conn(0x7fbbac159500 legacy=0x7fbbac0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbb1d61700 1 -- 172.21.15.3:0/3613228011 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac0cb9b0 con 0x7fbbac0cb440 2023-10-06T05:00:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbb1d61700 1 --2- 172.21.15.3:0/3613228011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0cb440 0x7fbbac0b2760 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fbbac15d130 tx=0x7fbba4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4660dddfe98b118a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbabfff700 1 -- 172.21.15.3:0/3613228011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400b860 con 0x7fbbac0cb440 2023-10-06T05:00:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbabfff700 1 -- 172.21.15.3:0/3613228011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400b9c0 con 0x7fbbac0cb440 2023-10-06T05:00:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.351+0000 7fbbabfff700 1 -- 172.21.15.3:0/3613228011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400bd60 con 0x7fbbac0cb440 2023-10-06T05:00:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 -- 172.21.15.3:0/3613228011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0cb440 msgr2=0x7fbbac0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/3613228011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0cb440 0x7fbbac0b2760 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fbbac15d130 tx=0x7fbba4009250 comp rx=0 tx=0).stop 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 -- 172.21.15.3:0/3613228011 shutdown_connections 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/3613228011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac0cb440 0x7fbbac0b2760 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/3613228011 >> v2:172.21.15.153:3300/0 conn(0x7fbbac1598d0 0x7fbbac0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 -- 172.21.15.3:0/3613228011 >> 172.21.15.3:0/3613228011 conn(0x7fbbac070310 msgr2=0x7fbbac070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 -- 172.21.15.3:0/3613228011 shutdown_connections 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.352+0000 7fbbb37c4700 1 -- 172.21.15.3:0/3613228011 wait complete. 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.353+0000 7fbbb37c4700 1 Processor -- start 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.353+0000 7fbbb37c4700 1 -- start start 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb37c4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac0cb440 0x7fbbac14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb37c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac159500 0x7fbbac15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb37c4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac1598d0 0x7fbbac1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb37c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac15abb0 con 0x7fbbac159500 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb37c4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac14fea0 con 0x7fbbac1598d0 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb37c4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac159cc0 con 0x7fbbac0cb440 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac159500 0x7fbbac15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac159500 0x7fbbac15b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45884/0 (socket says 172.21.15.3:45884) 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 -- 172.21.15.3:0/223586385 learned_addr learned my addr 172.21.15.3:0/223586385 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 -- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac0cb440 msgr2=0x7fbbac14f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac0cb440 0x7fbbac14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 -- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac1598d0 msgr2=0x7fbbac1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac1598d0 0x7fbbac1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 -- 172.21.15.3:0/223586385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40094f0 con 0x7fbbac159500 2023-10-06T05:00:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb0d5f700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac159500 0x7fbbac15b040 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fbba000d860 tx=0x7fbba0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbba97fa700 1 -- 172.21.15.3:0/223586385 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba00125f0 con 0x7fbbac159500 2023-10-06T05:00:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbba97fa700 1 -- 172.21.15.3:0/223586385 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0012750 con 0x7fbbac159500 2023-10-06T05:00:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbba97fa700 1 -- 172.21.15.3:0/223586385 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0012b10 con 0x7fbbac159500 2023-10-06T05:00:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.354+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac159fa0 con 0x7fbbac159500 2023-10-06T05:00:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.355+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac15a560 con 0x7fbbac159500 2023-10-06T05:00:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.355+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac071680 con 0x7fbbac159500 2023-10-06T05:00:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.356+0000 7fbba97fa700 1 -- 172.21.15.3:0/223586385 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbba001e460 con 0x7fbbac159500 2023-10-06T05:00:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.356+0000 7fbba97fa700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.356+0000 7fbba97fa700 1 -- 172.21.15.3:0/223586385 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbba0084a70 con 0x7fbbac159500 2023-10-06T05:00:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.357+0000 7fbbb1560700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.357+0000 7fbbb1560700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001d90 tx=0x7fbb9c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:34.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.360+0000 7fbba97fa700 1 -- 172.21.15.3:0/223586385 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbba0051e30 con 0x7fbbac159500 2023-10-06T05:00:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.554+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbac0cbc90 con 0x7fbbac159500 2023-10-06T05:00:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.555+0000 7fbba97fa700 1 -- 172.21.15.3:0/223586385 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13612 (secure 0 0 0) 0x7fbba0055450 con 0x7fbbac159500 2023-10-06T05:00:34.558 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:34.558 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":116,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:00:33.781892+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:00:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 msgr2=0x7fbb980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001d90 tx=0x7fbb9c007ec0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac159500 msgr2=0x7fbbac15b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac159500 0x7fbbac15b040 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fbba000d860 tx=0x7fbba0008040 comp rx=0 tx=0).stop 2023-10-06T05:00:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 shutdown_connections 2023-10-06T05:00:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbb98066f30 0x7fbb980693e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac1598d0 0x7fbbac1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbac0cb440 0x7fbbac14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 --2- 172.21.15.3:0/223586385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbac159500 0x7fbbac15b040 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 >> 172.21.15.3:0/223586385 conn(0x7fbbac070310 msgr2=0x7fbbac0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 shutdown_connections 2023-10-06T05:00:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.557+0000 7fbbb37c4700 1 -- 172.21.15.3:0/223586385 wait complete. 2023-10-06T05:00:34.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:00:34.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:00:34.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 Processor -- start 2023-10-06T05:00:34.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 -- start start 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3250150690 0x7f3250150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32500cba50 0x7f325015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f325015a720 0x7f32500af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325015da80 con 0x7f3250150690 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325015a290 con 0x7f325015a720 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.693+0000 7f3255ddf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32500d0020 con 0x7f32500cba50 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3250150690 0x7f3250150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3250150690 0x7f3250150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45886/0 (socket says 172.21.15.3:45886) 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 -- 172.21.15.3:0/237312870 learned_addr learned my addr 172.21.15.3:0/237312870 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 -- 172.21.15.3:0/237312870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32500cba50 msgr2=0x7f325015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324effd700 1 --2- 172.21.15.3:0/237312870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32500cba50 0x7f325015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324ffff700 1 --2- 172.21.15.3:0/237312870 >> v2:172.21.15.153:3300/0 conn(0x7f325015a720 0x7f32500af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 --2- 172.21.15.3:0/237312870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32500cba50 0x7f325015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 -- 172.21.15.3:0/237312870 >> v2:172.21.15.153:3300/0 conn(0x7f325015a720 msgr2=0x7f32500af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 --2- 172.21.15.3:0/237312870 >> v2:172.21.15.153:3300/0 conn(0x7f325015a720 0x7f32500af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 -- 172.21.15.3:0/237312870 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3250159f40 con 0x7f3250150690 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324f7fe700 1 --2- 172.21.15.3:0/237312870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3250150690 0x7f3250150a60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3250157a70 tx=0x7f3240009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab21c057bd5a65f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324e7fc700 1 -- 172.21.15.3:0/237312870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324000b860 con 0x7f3250150690 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324e7fc700 1 -- 172.21.15.3:0/237312870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324000b9c0 con 0x7f3250150690 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.694+0000 7f324e7fc700 1 -- 172.21.15.3:0/237312870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324000bd60 con 0x7f3250150690 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 -- 172.21.15.3:0/237312870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3250150690 msgr2=0x7f3250150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 --2- 172.21.15.3:0/237312870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3250150690 0x7f3250150a60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3250157a70 tx=0x7f3240009250 comp rx=0 tx=0).stop 2023-10-06T05:00:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 -- 172.21.15.3:0/237312870 shutdown_connections 2023-10-06T05:00:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 --2- 172.21.15.3:0/237312870 >> v2:172.21.15.153:3300/0 conn(0x7f325015a720 0x7f32500af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 --2- 172.21.15.3:0/237312870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3250150690 0x7f3250150a60 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 --2- 172.21.15.3:0/237312870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32500cba50 0x7f325015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 -- 172.21.15.3:0/237312870 >> 172.21.15.3:0/237312870 conn(0x7f3250070310 msgr2=0x7f3250070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 -- 172.21.15.3:0/237312870 shutdown_connections 2023-10-06T05:00:34.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.695+0000 7f3255ddf700 1 -- 172.21.15.3:0/237312870 wait complete. 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.696+0000 7f3255ddf700 1 Processor -- start 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.696+0000 7f3255ddf700 1 -- start start 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f3255ddf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32500cba50 0x7f3250151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f3255ddf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250150690 0x7f3250150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f3255ddf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f325015a720 0x7f32500b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f3255ddf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3250151e40 con 0x7f32500cba50 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f3255ddf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3250152120 con 0x7f3250150690 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f3255ddf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3250152400 con 0x7f325015a720 2023-10-06T05:00:34.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32500cba50 0x7f3250151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32500cba50 0x7f3250151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45898/0 (socket says 172.21.15.3:45898) 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 -- 172.21.15.3:0/88744361 learned_addr learned my addr 172.21.15.3:0/88744361 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324ffff700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f325015a720 0x7f32500b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324effd700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250150690 0x7f3250150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 -- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f325015a720 msgr2=0x7f32500b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f325015a720 0x7f32500b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 -- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250150690 msgr2=0x7f3250150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250150690 0x7f3250150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 -- 172.21.15.3:0/88744361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32400094f0 con 0x7f32500cba50 2023-10-06T05:00:34.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f324f7fe700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32500cba50 0x7f3250151860 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f32500c71f0 tx=0x7f3240017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.697+0000 7f32377fe700 1 -- 172.21.15.3:0/88744361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240024440 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.698+0000 7f32377fe700 1 -- 172.21.15.3:0/88744361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32400245a0 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.698+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32501526e0 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.698+0000 7f32377fe700 1 -- 172.21.15.3:0/88744361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240024960 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.698+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32500b5c90 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.698+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3250152c00 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.699+0000 7f32377fe700 1 -- 172.21.15.3:0/88744361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f324001a420 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.699+0000 7f32377fe700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f323c066f30 0x7f323c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.699+0000 7f32377fe700 1 -- 172.21.15.3:0/88744361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3240086250 con 0x7f32500cba50 2023-10-06T05:00:34.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.699+0000 7f324effd700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f323c066f30 0x7f323c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:34.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.700+0000 7f324effd700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f323c066f30 0x7f323c0693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3250153750 tx=0x7f3244008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:34.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.704+0000 7f32377fe700 1 -- 172.21.15.3:0/88744361 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3250152c00 con 0x7f32500cba50 2023-10-06T05:00:34.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.898+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3250152c00 con 0x7f32500cba50 2023-10-06T05:00:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.898+0000 7f32377fe700 1 -- 172.21.15.3:0/88744361 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v116) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3250152c00 con 0x7f32500cba50 2023-10-06T05:00:34.901 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T05:00:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f323c066f30 msgr2=0x7f323c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f323c066f30 0x7f323c0693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3250153750 tx=0x7f3244008040 comp rx=0 tx=0).stop 2023-10-06T05:00:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32500cba50 msgr2=0x7f3250151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32500cba50 0x7f3250151860 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f32500c71f0 tx=0x7f3240017000 comp rx=0 tx=0).stop 2023-10-06T05:00:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 shutdown_connections 2023-10-06T05:00:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f323c066f30 0x7f323c0693e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f325015a720 0x7f32500b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32500cba50 0x7f3250151860 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 --2- 172.21.15.3:0/88744361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250150690 0x7f3250150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 >> 172.21.15.3:0/88744361 conn(0x7f3250070310 msgr2=0x7f32500b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 shutdown_connections 2023-10-06T05:00:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:34.900+0000 7f3255ddf700 1 -- 172.21.15.3:0/88744361 wait complete. 2023-10-06T05:00:34.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:00:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.035+0000 7fba57819700 1 Processor -- start 2023-10-06T05:00:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.035+0000 7fba57819700 1 -- start start 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba57819700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba50150690 0x7fba50150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba57819700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5015d540 0x7fba500af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba57819700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5015a290 con 0x7fba500cba50 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba57819700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba500d0020 con 0x7fba50150690 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba57819700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba50159f40 con 0x7fba5015d540 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba50150690 0x7fba50150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba50150690 0x7fba50150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45906/0 (socket says 172.21.15.3:45906) 2023-10-06T05:00:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 -- 172.21.15.3:0/1539188785 learned_addr learned my addr 172.21.15.3:0/1539188785 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 -- 172.21.15.3:0/1539188785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5015d540 msgr2=0x7fba500af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba55db6700 1 --2- 172.21.15.3:0/1539188785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5015d540 0x7fba500af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 --2- 172.21.15.3:0/1539188785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5015d540 0x7fba500af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 -- 172.21.15.3:0/1539188785 >> v1:172.21.15.153:6789/0 conn(0x7fba500cba50 legacy=0x7fba5015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 -- 172.21.15.3:0/1539188785 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5015dcb0 con 0x7fba50150690 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba555b5700 1 --2- 172.21.15.3:0/1539188785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba50150690 0x7fba50150a60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fba480099b0 tx=0x7fba48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8143b179019ebe5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba4ffff700 1 -- 172.21.15.3:0/1539188785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4800b820 con 0x7fba50150690 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba4ffff700 1 -- 172.21.15.3:0/1539188785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4800b980 con 0x7fba50150690 2023-10-06T05:00:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.036+0000 7fba4ffff700 1 -- 172.21.15.3:0/1539188785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4800bd20 con 0x7fba50150690 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 -- 172.21.15.3:0/1539188785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba50150690 msgr2=0x7fba50150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 --2- 172.21.15.3:0/1539188785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba50150690 0x7fba50150a60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fba480099b0 tx=0x7fba48009250 comp rx=0 tx=0).stop 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 -- 172.21.15.3:0/1539188785 shutdown_connections 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 --2- 172.21.15.3:0/1539188785 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5015d540 0x7fba500af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 --2- 172.21.15.3:0/1539188785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba50150690 0x7fba50150a60 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 -- 172.21.15.3:0/1539188785 >> 172.21.15.3:0/1539188785 conn(0x7fba50070310 msgr2=0x7fba50070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 -- 172.21.15.3:0/1539188785 shutdown_connections 2023-10-06T05:00:35.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.037+0000 7fba57819700 1 -- 172.21.15.3:0/1539188785 wait complete. 2023-10-06T05:00:35.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.038+0000 7fba57819700 1 Processor -- start 2023-10-06T05:00:35.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.038+0000 7fba57819700 1 -- start start 2023-10-06T05:00:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba57819700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba500cba50 0x7fba501527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba57819700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba50150690 0x7fba50153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba57819700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba5015d540 0x7fba500b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba57819700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba50152d30 con 0x7fba5015d540 2023-10-06T05:00:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba57819700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba50169e00 con 0x7fba50150690 2023-10-06T05:00:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba57819700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5016a0d0 con 0x7fba500cba50 2023-10-06T05:00:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba5015d540 0x7fba500b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba5015d540 0x7fba500b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45922/0 (socket says 172.21.15.3:45922) 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 -- 172.21.15.3:0/542998771 learned_addr learned my addr 172.21.15.3:0/542998771 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba555b5700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba500cba50 0x7fba501527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba54db4700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba50150690 0x7fba50153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 -- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba500cba50 msgr2=0x7fba501527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba500cba50 0x7fba501527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 -- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba50150690 msgr2=0x7fba50153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba50150690 0x7fba50153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.039+0000 7fba55db6700 1 -- 172.21.15.3:0/542998771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480094f0 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.040+0000 7fba55db6700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba5015d540 0x7fba500b5900 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fba4000cdb0 tx=0x7fba400077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.040+0000 7fba4d7fa700 1 -- 172.21.15.3:0/542998771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000a9d0 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.040+0000 7fba4d7fa700 1 -- 172.21.15.3:0/542998771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4000ab30 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.040+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5016a3b0 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.040+0000 7fba4d7fa700 1 -- 172.21.15.3:0/542998771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40017430 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.040+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5016a970 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.041+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5004c380 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.041+0000 7fba4d7fa700 1 -- 172.21.15.3:0/542998771 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba40003020 con 0x7fba5015d540 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.041+0000 7fba4d7fa700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba3c066f30 0x7fba3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.042+0000 7fba4d7fa700 1 -- 172.21.15.3:0/542998771 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fba40013070 con 0x7fba5015d540 2023-10-06T05:00:35.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.042+0000 7fba555b5700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba3c066f30 0x7fba3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.042+0000 7fba555b5700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba3c066f30 0x7fba3c0693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fba480099b0 tx=0x7fba48016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.046+0000 7fba4d7fa700 1 -- 172.21.15.3:0/542998771 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba40051bf0 con 0x7fba5015d540 2023-10-06T05:00:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.237+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba5015df90 con 0x7fba5015d540 2023-10-06T05:00:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.237+0000 7fba4d7fa700 1 -- 172.21.15.3:0/542998771 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba4001c020 con 0x7fba5015d540 2023-10-06T05:00:35.240 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T05:00:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba3c066f30 msgr2=0x7fba3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba3c066f30 0x7fba3c0693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fba480099b0 tx=0x7fba48016000 comp rx=0 tx=0).stop 2023-10-06T05:00:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba5015d540 msgr2=0x7fba500b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba5015d540 0x7fba500b5900 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fba4000cdb0 tx=0x7fba400077d0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 shutdown_connections 2023-10-06T05:00:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fba3c066f30 0x7fba3c0693e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba5015d540 0x7fba500b5900 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba500cba50 0x7fba501527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 --2- 172.21.15.3:0/542998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba50150690 0x7fba50153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 >> 172.21.15.3:0/542998771 conn(0x7fba50070310 msgr2=0x7fba500d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 shutdown_connections 2023-10-06T05:00:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.239+0000 7fba57819700 1 -- 172.21.15.3:0/542998771 wait complete. 2023-10-06T05:00:35.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:00:35.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.370+0000 7f4ca4510700 1 Processor -- start 2023-10-06T05:00:35.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.370+0000 7f4ca4510700 1 -- start start 2023-10-06T05:00:35.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.371+0000 7f4ca4510700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157700 0x7f4c9c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.371+0000 7f4ca4510700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c9c0b1b10 0x7f4c9c067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.371+0000 7f4ca4510700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c0b2e50 con 0x7f4c9c157ad0 2023-10-06T05:00:35.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.371+0000 7f4ca4510700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c158290 con 0x7f4c9c0b1b10 2023-10-06T05:00:35.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.371+0000 7f4ca4510700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b1fc0 con 0x7f4c9c157700 2023-10-06T05:00:35.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c9c0b1b10 0x7f4c9c067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca22ac700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157700 0x7f4c9c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c9c0b1b10 0x7f4c9c067b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:43122/0 (socket says 172.21.15.3:43122) 2023-10-06T05:00:35.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 -- 172.21.15.3:0/1974117317 learned_addr learned my addr 172.21.15.3:0/1974117317 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:35.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca1aab700 1 --1- 172.21.15.3:0/1974117317 >> v1:172.21.15.153:6789/0 conn(0x7f4c9c157ad0 0x7f4c9c0b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:33812/0 (socket says 172.21.15.3:33812) 2023-10-06T05:00:35.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 -- 172.21.15.3:0/1974117317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157700 msgr2=0x7f4c9c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:35.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 --2- 172.21.15.3:0/1974117317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157700 0x7f4c9c0cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 -- 172.21.15.3:0/1974117317 >> v1:172.21.15.153:6789/0 conn(0x7f4c9c157ad0 legacy=0x7f4c9c0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 -- 172.21.15.3:0/1974117317 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0b22a0 con 0x7f4c9c0b1b10 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca2aad700 1 --2- 172.21.15.3:0/1974117317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 0x7f4c9c067b30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009f20 tx=0x7f4c8c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f376b87220b772d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca12aa700 1 -- 172.21.15.3:0/1974117317 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00e070 con 0x7f4c9c0b1b10 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca12aa700 1 -- 172.21.15.3:0/1974117317 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00b9d0 con 0x7f4c9c0b1b10 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.372+0000 7f4ca12aa700 1 -- 172.21.15.3:0/1974117317 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0092e0 con 0x7f4c9c0b1b10 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 -- 172.21.15.3:0/1974117317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 msgr2=0x7f4c9c067b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1974117317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 0x7f4c9c067b30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009f20 tx=0x7f4c8c009840 comp rx=0 tx=0).stop 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 -- 172.21.15.3:0/1974117317 shutdown_connections 2023-10-06T05:00:35.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1974117317 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157700 0x7f4c9c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1974117317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 0x7f4c9c067b30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 -- 172.21.15.3:0/1974117317 >> 172.21.15.3:0/1974117317 conn(0x7f4c9c0ce2c0 msgr2=0x7f4c9c0ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:35.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 -- 172.21.15.3:0/1974117317 shutdown_connections 2023-10-06T05:00:35.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.373+0000 7f4ca4510700 1 -- 172.21.15.3:0/1974117317 wait complete. 2023-10-06T05:00:35.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.374+0000 7f4ca4510700 1 Processor -- start 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.374+0000 7f4ca4510700 1 -- start start 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca4510700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 0x7f4c9c16a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca4510700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c9c157700 0x7f4c9c0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca4510700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157ad0 0x7f4c9c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca4510700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1692c0 con 0x7f4c9c157700 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca4510700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1695a0 con 0x7f4c9c0b1b10 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca4510700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c169880 con 0x7f4c9c157ad0 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c9c157700 0x7f4c9c0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c9c157700 0x7f4c9c0ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45930/0 (socket says 172.21.15.3:45930) 2023-10-06T05:00:35.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 -- 172.21.15.3:0/1683852754 learned_addr learned my addr 172.21.15.3:0/1683852754 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca22ac700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 0x7f4c9c16a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca2aad700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157ad0 0x7f4c9c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 -- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157ad0 msgr2=0x7f4c9c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157ad0 0x7f4c9c0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 -- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 msgr2=0x7f4c9c16a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 0x7f4c9c16a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 -- 172.21.15.3:0/1683852754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009ae0 con 0x7f4c9c157700 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.375+0000 7f4ca1aab700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c9c157700 0x7f4c9c0ce690 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c560 tx=0x7f4c90007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.376+0000 7f4c9a7fc700 1 -- 172.21.15.3:0/1683852754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90005a70 con 0x7f4c9c157700 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.376+0000 7f4c9a7fc700 1 -- 172.21.15.3:0/1683852754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90005bd0 con 0x7f4c9c157700 2023-10-06T05:00:35.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.376+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c169b60 con 0x7f4c9c157700 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.376+0000 7f4c9a7fc700 1 -- 172.21.15.3:0/1683852754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900070d0 con 0x7f4c9c157700 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.376+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c168590 con 0x7f4c9c157700 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.377+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c16a080 con 0x7f4c9c157700 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.377+0000 7f4c9a7fc700 1 -- 172.21.15.3:0/1683852754 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c90007480 con 0x7f4c9c157700 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.377+0000 7f4c9a7fc700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c88066f30 0x7f4c880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.378+0000 7f4c9a7fc700 1 -- 172.21.15.3:0/1683852754 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4c90083850 con 0x7f4c9c157700 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.378+0000 7f4ca22ac700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c88066f30 0x7f4c880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.378+0000 7f4ca22ac700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c88066f30 0x7f4c880693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c15dd30 tx=0x7f4c94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.382+0000 7f4c9a7fc700 1 -- 172.21.15.3:0/1683852754 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c9c16a080 con 0x7f4c9c157700 2023-10-06T05:00:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.578+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4c9c16a080 con 0x7f4c9c157700 2023-10-06T05:00:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.579+0000 7f4c9a7fc700 1 -- 172.21.15.3:0/1683852754 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4c9c16a080 con 0x7f4c9c157700 2023-10-06T05:00:35.581 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:35.582 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T05:00:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c88066f30 msgr2=0x7f4c880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c88066f30 0x7f4c880693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c15dd30 tx=0x7f4c94008040 comp rx=0 tx=0).stop 2023-10-06T05:00:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c9c157700 msgr2=0x7f4c9c0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c9c157700 0x7f4c9c0ce690 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c560 tx=0x7f4c90007f70 comp rx=0 tx=0).stop 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 shutdown_connections 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4c88066f30 0x7f4c880693e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c0b1b10 0x7f4c9c16a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c9c157ad0 0x7f4c9c0cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 --2- 172.21.15.3:0/1683852754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c9c157700 0x7f4c9c0ce690 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 >> 172.21.15.3:0/1683852754 conn(0x7f4c9c0ce2c0 msgr2=0x7f4c9c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.580+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 shutdown_connections 2023-10-06T05:00:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.581+0000 7f4ca4510700 1 -- 172.21.15.3:0/1683852754 wait complete. 2023-10-06T05:00:35.595 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:00:35.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:00:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.716+0000 7fdb7f59e700 1 Processor -- start 2023-10-06T05:00:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.716+0000 7fdb7f59e700 1 -- start start 2023-10-06T05:00:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb800cd350 0x7fdb800cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb80159b70 con 0x7fdb8009da40 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7f59e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8015d8b0 con 0x7fdb8009d670 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8015d2d0 con 0x7fdb800cd350 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdb8009da40 0x7fdb800ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35584/0 (socket says 172.21.15.3:35584) 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7dd9b700 1 -- 172.21.15.3:0/4104998967 learned_addr learned my addr 172.21.15.3:0/4104998967 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7ed9d700 1 --2- 172.21.15.3:0/4104998967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb800cd350 0x7fdb800cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7d59a700 1 -- 172.21.15.3:0/4104998967 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769159705 0 0) 0x7fdb80159b70 con 0x7fdb8009da40 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.717+0000 7fdb7d59a700 1 -- 172.21.15.3:0/4104998967 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb6c003570 con 0x7fdb8009da40 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7d59a700 1 -- 172.21.15.3:0/4104998967 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239389857 0 0) 0x7fdb8015d8b0 con 0x7fdb8009d670 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7d59a700 1 -- 172.21.15.3:0/4104998967 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb80159b70 con 0x7fdb8009d670 2023-10-06T05:00:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7ed9d700 1 -- 172.21.15.3:0/4104998967 >> v1:172.21.15.153:6789/0 conn(0x7fdb8009d670 legacy=0x7fdb800b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7ed9d700 1 -- 172.21.15.3:0/4104998967 >> v1:172.21.15.3:6789/0 conn(0x7fdb8009da40 legacy=0x7fdb800ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7ed9d700 1 -- 172.21.15.3:0/4104998967 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb800cfcf0 con 0x7fdb800cd350 2023-10-06T05:00:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7ed9d700 1 --2- 172.21.15.3:0/4104998967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb800cd350 0x7fdb800cf6a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009e90 tx=0x7fdb70009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c9a03991498745 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7d59a700 1 -- 172.21.15.3:0/4104998967 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000e070 con 0x7fdb800cd350 2023-10-06T05:00:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7d59a700 1 -- 172.21.15.3:0/4104998967 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7000bb80 con 0x7fdb800cd350 2023-10-06T05:00:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7d59a700 1 -- 172.21.15.3:0/4104998967 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70012430 con 0x7fdb800cd350 2023-10-06T05:00:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4104998967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb800cd350 msgr2=0x7fdb800cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.718+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4104998967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb800cd350 0x7fdb800cf6a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009e90 tx=0x7fdb70009780 comp rx=0 tx=0).stop 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.719+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4104998967 shutdown_connections 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.719+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4104998967 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb800cd350 0x7fdb800cf6a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.719+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4104998967 >> 172.21.15.3:0/4104998967 conn(0x7fdb80070310 msgr2=0x7fdb80070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.719+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4104998967 shutdown_connections 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.719+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4104998967 wait complete. 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 Processor -- start 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 -- start start 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb8009d670 0x7fdb8016c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb8009da40 0x7fdb800accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800cd350 0x7fdb80098470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8016cdd0 con 0x7fdb800cd350 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb800ad720 con 0x7fdb8009da40 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb800ada00 con 0x7fdb8009d670 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800cd350 0x7fdb80098470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.720+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800cd350 0x7fdb80098470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45934/0 (socket says 172.21.15.3:45934) 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7ed9d700 1 -- 172.21.15.3:0/4253367148 learned_addr learned my addr 172.21.15.3:0/4253367148 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7e59c700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb8009d670 0x7fdb8016c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7dd9b700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb8009da40 0x7fdb800accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7ed9d700 1 -- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb8009d670 msgr2=0x7fdb8016c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7ed9d700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb8009d670 0x7fdb8016c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7ed9d700 1 -- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb8009da40 msgr2=0x7fdb800accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7ed9d700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb8009da40 0x7fdb800accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7ed9d700 1 -- 172.21.15.3:0/4253367148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70009a20 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7ed9d700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800cd350 0x7fdb80098470 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdb70013000 tx=0x7fdb70019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb767fc700 1 -- 172.21.15.3:0/4253367148 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70012e40 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb767fc700 1 -- 172.21.15.3:0/4253367148 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7001d630 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb800adce0 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb767fc700 1 -- 172.21.15.3:0/4253367148 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7001d9f0 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.721+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb800ae2a0 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.722+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8004c380 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.723+0000 7fdb767fc700 1 -- 172.21.15.3:0/4253367148 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb70024070 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.723+0000 7fdb767fc700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb60066f30 0x7fdb600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.723+0000 7fdb767fc700 1 -- 172.21.15.3:0/4253367148 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdb70086bb0 con 0x7fdb800cd350 2023-10-06T05:00:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.723+0000 7fdb7e59c700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb60066f30 0x7fdb600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:35.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.724+0000 7fdb7e59c700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb60066f30 0x7fdb600693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdb78007fd0 tx=0x7fdb78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:35.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.727+0000 7fdb767fc700 1 -- 172.21.15.3:0/4253367148 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb700540c0 con 0x7fdb800cd350 2023-10-06T05:00:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:35.922+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdb8015d540 con 0x7fdb800cd350 2023-10-06T05:00:36.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.797+0000 7fdb767fc700 1 -- 172.21.15.3:0/4253367148 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdb700576e0 con 0x7fdb800cd350 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb60066f30 msgr2=0x7fdb600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb60066f30 0x7fdb600693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdb78007fd0 tx=0x7fdb78009040 comp rx=0 tx=0).stop 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800cd350 msgr2=0x7fdb80098470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800cd350 0x7fdb80098470 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdb70013000 tx=0x7fdb70019000 comp rx=0 tx=0).stop 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 shutdown_connections 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdb60066f30 0x7fdb600693e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800cd350 0x7fdb80098470 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdb8009d670 0x7fdb8016c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 --2- 172.21.15.3:0/4253367148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb8009da40 0x7fdb800accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 >> 172.21.15.3:0/4253367148 conn(0x7fdb80070310 msgr2=0x7fdb80158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 shutdown_connections 2023-10-06T05:00:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.800+0000 7fdb7f59e700 1 -- 172.21.15.3:0/4253367148 wait complete. 2023-10-06T05:00:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T05:00:36.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:00:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.949+0000 7f90eeba4700 1 Processor -- start 2023-10-06T05:00:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.949+0000 7f90eeba4700 1 -- start start 2023-10-06T05:00:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90eeba4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f90e8140e70 0x7f90e80ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90eeba4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e80cd440 0x7f90e80b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90eeba4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e80b14f0 0x7f90e8067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90eeba4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e8069f10 con 0x7f90e80b14f0 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90eeba4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80cca70 con 0x7f90e8140e70 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90eeba4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80cc140 con 0x7f90e80cd440 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90ed141700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e80b14f0 0x7f90e8067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90ed141700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e80b14f0 0x7f90e8067ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45944/0 (socket says 172.21.15.3:45944) 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.950+0000 7f90ed141700 1 -- 172.21.15.3:0/4198745685 learned_addr learned my addr 172.21.15.3:0/4198745685 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90ec940700 1 --2- 172.21.15.3:0/4198745685 >> v2:172.21.15.153:3300/0 conn(0x7f90e8140e70 0x7f90e80ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90e7fff700 1 --2- 172.21.15.3:0/4198745685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e80cd440 0x7f90e80b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90ed141700 1 -- 172.21.15.3:0/4198745685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e80cd440 msgr2=0x7f90e80b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90ed141700 1 --2- 172.21.15.3:0/4198745685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e80cd440 0x7f90e80b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90ed141700 1 -- 172.21.15.3:0/4198745685 >> v2:172.21.15.153:3300/0 conn(0x7f90e8140e70 msgr2=0x7f90e80ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:36.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90ed141700 1 --2- 172.21.15.3:0/4198745685 >> v2:172.21.15.153:3300/0 conn(0x7f90e8140e70 0x7f90e80ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90ed141700 1 -- 172.21.15.3:0/4198745685 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80cc420 con 0x7f90e80b14f0 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90ed141700 1 --2- 172.21.15.3:0/4198745685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b14f0 0x7f90e8067ab0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f90e80cabe0 tx=0x7f90e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90b86b76348a923 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90e77fe700 1 -- 172.21.15.3:0/4198745685 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000e070 con 0x7f90e80b14f0 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90e77fe700 1 -- 172.21.15.3:0/4198745685 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e000b970 con 0x7f90e80b14f0 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.951+0000 7f90e77fe700 1 -- 172.21.15.3:0/4198745685 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e00092e0 con 0x7f90e80b14f0 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 -- 172.21.15.3:0/4198745685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b14f0 msgr2=0x7f90e8067ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 --2- 172.21.15.3:0/4198745685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b14f0 0x7f90e8067ab0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f90e80cabe0 tx=0x7f90e0009840 comp rx=0 tx=0).stop 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 -- 172.21.15.3:0/4198745685 shutdown_connections 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 --2- 172.21.15.3:0/4198745685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80b14f0 0x7f90e8067ab0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 --2- 172.21.15.3:0/4198745685 >> v2:172.21.15.153:3300/0 conn(0x7f90e8140e70 0x7f90e80ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 --2- 172.21.15.3:0/4198745685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e80cd440 0x7f90e80b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 -- 172.21.15.3:0/4198745685 >> 172.21.15.3:0/4198745685 conn(0x7f90e80ce210 msgr2=0x7f90e80ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 -- 172.21.15.3:0/4198745685 shutdown_connections 2023-10-06T05:00:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.952+0000 7f90eeba4700 1 -- 172.21.15.3:0/4198745685 wait complete. 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.953+0000 7f90eeba4700 1 Processor -- start 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.953+0000 7f90eeba4700 1 -- start start 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90eeba4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e80b14f0 0x7f90e8169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90eeba4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80cd440 0x7f90e8134750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90eeba4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e8140e70 0x7f90e8131c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90eeba4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8133150 con 0x7f90e80cd440 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90eeba4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8132540 con 0x7f90e80b14f0 2023-10-06T05:00:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90eeba4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8132820 con 0x7f90e8140e70 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80cd440 0x7f90e8134750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80cd440 0x7f90e8134750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45946/0 (socket says 172.21.15.3:45946) 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 -- 172.21.15.3:0/2329134823 learned_addr learned my addr 172.21.15.3:0/2329134823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90ed141700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e8140e70 0x7f90e8131c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90ec940700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e80b14f0 0x7f90e8169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 -- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e8140e70 msgr2=0x7f90e8131c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e8140e70 0x7f90e8131c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 -- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e80b14f0 msgr2=0x7f90e8169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e80b14f0 0x7f90e8169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 -- 172.21.15.3:0/2329134823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0009ae0 con 0x7f90e80cd440 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.954+0000 7f90e7fff700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80cd440 0x7f90e8134750 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00ec80 tx=0x7f90dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.955+0000 7f90e4ff9700 1 -- 172.21.15.3:0/2329134823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00c850 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.955+0000 7f90e4ff9700 1 -- 172.21.15.3:0/2329134823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc00c9b0 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.955+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e8132b00 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.955+0000 7f90e4ff9700 1 -- 172.21.15.3:0/2329134823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00cd50 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.955+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e8134c90 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.956+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e804c380 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.956+0000 7f90e4ff9700 1 -- 172.21.15.3:0/2329134823 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90dc013680 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.956+0000 7f90e4ff9700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90d4066f30 0x7f90d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.957+0000 7f90e4ff9700 1 -- 172.21.15.3:0/2329134823 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f90dc084730 con 0x7f90e80cd440 2023-10-06T05:00:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.957+0000 7f90ec940700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90d4066f30 0x7f90d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:36.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.957+0000 7f90ec940700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90d4066f30 0x7f90d40693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009d10 tx=0x7f90d80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:36.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:36.961+0000 7f90e4ff9700 1 -- 172.21.15.3:0/2329134823 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90dc051a20 con 0x7f90e80cd440 2023-10-06T05:00:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.156+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f90e8133290 con 0x7f90e80cd440 2023-10-06T05:00:37.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.806+0000 7f90e4ff9700 1 -- 172.21.15.3:0/2329134823 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7f90dc055040 con 0x7f90e80cd440 2023-10-06T05:00:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.813+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90d4066f30 msgr2=0x7f90d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90d4066f30 0x7f90d40693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009d10 tx=0x7f90d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80cd440 msgr2=0x7f90e8134750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80cd440 0x7f90e8134750 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00ec80 tx=0x7f90dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 shutdown_connections 2023-10-06T05:00:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90d4066f30 0x7f90d40693e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e80b14f0 0x7f90e8169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e8140e70 0x7f90e8131c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 --2- 172.21.15.3:0/2329134823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e80cd440 0x7f90e8134750 secure :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00ec80 tx=0x7f90dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.814+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 >> 172.21.15.3:0/2329134823 conn(0x7f90e80ce210 msgr2=0x7f90e813fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:37.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.816+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 shutdown_connections 2023-10-06T05:00:37.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.818+0000 7f90eeba4700 1 -- 172.21.15.3:0/2329134823 wait complete. 2023-10-06T05:00:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T05:00:37.839 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:00:37.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.961+0000 7f30b1787700 1 Processor -- start 2023-10-06T05:00:37.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.961+0000 7f30b1787700 1 -- start start 2023-10-06T05:00:37.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30b1787700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f30ac0ccc40 0x7f30ac15a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:37.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30b1787700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0af050 0x7f30ac0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:37.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30b1787700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac0ce760 con 0x7f30ac0cd010 2023-10-06T05:00:37.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30b1787700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac15d8d0 con 0x7f30ac0ccc40 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30b1787700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac15dbb0 con 0x7f30ac0af050 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0af050 0x7f30ac0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0af050 0x7f30ac0af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:37936/0 (socket says 172.21.15.3:37936) 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30ab7fe700 1 -- 172.21.15.3:0/2444102699 learned_addr learned my addr 172.21.15.3:0/2444102699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30aaffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f30ac0ccc40 0x7f30ac15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30ab7fe700 1 -- 172.21.15.3:0/2444102699 >> v2:172.21.15.153:3300/0 conn(0x7f30ac0ccc40 msgr2=0x7f30ac15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30ab7fe700 1 --2- 172.21.15.3:0/2444102699 >> v2:172.21.15.153:3300/0 conn(0x7f30ac0ccc40 0x7f30ac15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30ab7fe700 1 -- 172.21.15.3:0/2444102699 >> v1:172.21.15.153:6789/0 conn(0x7f30ac0cd010 legacy=0x7f30ac15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30ab7fe700 1 -- 172.21.15.3:0/2444102699 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac15de90 con 0x7f30ac0af050 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.962+0000 7f30aaffd700 1 --2- 172.21.15.3:0/2444102699 >> v2:172.21.15.153:3300/0 conn(0x7f30ac0ccc40 0x7f30ac15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30ab7fe700 1 --2- 172.21.15.3:0/2444102699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0af050 0x7f30ac0af4b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f309c0099b0 tx=0x7f309c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aefff4fb105c0240 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:37.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30a9ffb700 1 -- 172.21.15.3:0/2444102699 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c009ea0 con 0x7f30ac0af050 2023-10-06T05:00:37.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30a9ffb700 1 -- 172.21.15.3:0/2444102699 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c00b860 con 0x7f30ac0af050 2023-10-06T05:00:37.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30a9ffb700 1 -- 172.21.15.3:0/2444102699 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00bc00 con 0x7f30ac0af050 2023-10-06T05:00:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 -- 172.21.15.3:0/2444102699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0af050 msgr2=0x7f30ac0af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 --2- 172.21.15.3:0/2444102699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0af050 0x7f30ac0af4b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f309c0099b0 tx=0x7f309c009250 comp rx=0 tx=0).stop 2023-10-06T05:00:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 -- 172.21.15.3:0/2444102699 shutdown_connections 2023-10-06T05:00:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 --2- 172.21.15.3:0/2444102699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0af050 0x7f30ac0af4b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 --2- 172.21.15.3:0/2444102699 >> v2:172.21.15.153:3300/0 conn(0x7f30ac0ccc40 0x7f30ac15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 -- 172.21.15.3:0/2444102699 >> 172.21.15.3:0/2444102699 conn(0x7f30ac070da0 msgr2=0x7f30ac06ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 -- 172.21.15.3:0/2444102699 shutdown_connections 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.963+0000 7f30b1787700 1 -- 172.21.15.3:0/2444102699 wait complete. 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 Processor -- start 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 -- start start 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30ac0af050 0x7f30ac141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0ccc40 0x7f30ac0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30ac0cd010 0x7f30ac0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b58f0 con 0x7f30ac0cd010 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac157300 con 0x7f30ac0af050 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30b1787700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b0010 con 0x7f30ac0ccc40 2023-10-06T05:00:37.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.965+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30ac0cd010 0x7f30ac0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30ac0cd010 0x7f30ac0b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45954/0 (socket says 172.21.15.3:45954) 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 -- 172.21.15.3:0/3912743948 learned_addr learned my addr 172.21.15.3:0/3912743948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30aa7fc700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0ccc40 0x7f30ac0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30aaffd700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30ac0af050 0x7f30ac141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 -- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0ccc40 msgr2=0x7f30ac0b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0ccc40 0x7f30ac0b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 -- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30ac0af050 msgr2=0x7f30ac141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30ac0af050 0x7f30ac141660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 -- 172.21.15.3:0/3912743948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4009a20 con 0x7f30ac0cd010 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30ab7fe700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30ac0cd010 0x7f30ac0b53b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f30ac15dcf0 tx=0x7f309c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30a2ffd700 1 -- 172.21.15.3:0/3912743948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00baa0 con 0x7f30ac0cd010 2023-10-06T05:00:37.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30a2ffd700 1 -- 172.21.15.3:0/3912743948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c025420 con 0x7f30ac0cd010 2023-10-06T05:00:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c0094f0 con 0x7f30ac0cd010 2023-10-06T05:00:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30a2ffd700 1 -- 172.21.15.3:0/3912743948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00bc00 con 0x7f30ac0cd010 2023-10-06T05:00:37.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.966+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac0b05d0 con 0x7f30ac0cd010 2023-10-06T05:00:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.967+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ac04c380 con 0x7f30ac0cd010 2023-10-06T05:00:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.968+0000 7f30a2ffd700 1 -- 172.21.15.3:0/3912743948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f309c023070 con 0x7f30ac0cd010 2023-10-06T05:00:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.968+0000 7f30a2ffd700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f308c066f30 0x7f308c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.968+0000 7f30a2ffd700 1 -- 172.21.15.3:0/3912743948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f309c085b90 con 0x7f30ac0cd010 2023-10-06T05:00:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.968+0000 7f30aaffd700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f308c066f30 0x7f308c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:37.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.969+0000 7f30aaffd700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f308c066f30 0x7f308c0693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009fd0 tx=0x7f30a400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:37.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:37.973+0000 7f30a2ffd700 1 -- 172.21.15.3:0/3912743948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f309c052d80 con 0x7f30ac0cd010 2023-10-06T05:00:38.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.209+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f30ac0ce050 con 0x7f30ac0cd010 2023-10-06T05:00:38.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.807+0000 7f30a2ffd700 1 -- 172.21.15.3:0/3912743948 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7f309c056320 con 0x7f30ac0cd010 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f308c066f30 msgr2=0x7f308c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f308c066f30 0x7f308c0693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009fd0 tx=0x7f30a400b040 comp rx=0 tx=0).stop 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30ac0cd010 msgr2=0x7f30ac0b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30ac0cd010 0x7f30ac0b53b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f30ac15dcf0 tx=0x7f309c017000 comp rx=0 tx=0).stop 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 shutdown_connections 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f308c066f30 0x7f308c0693e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30ac0af050 0x7f30ac141660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30ac0cd010 0x7f30ac0b53b0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 --2- 172.21.15.3:0/3912743948 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30ac0ccc40 0x7f30ac0b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.817+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 >> 172.21.15.3:0/3912743948 conn(0x7f30ac070da0 msgr2=0x7f30ac0cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:38.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.818+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 shutdown_connections 2023-10-06T05:00:38.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.819+0000 7f30b1787700 1 -- 172.21.15.3:0/3912743948 wait complete. 2023-10-06T05:00:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 19 and data pool 20 2023-10-06T05:00:38.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:00:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.964+0000 7f869af10700 1 Processor -- start 2023-10-06T05:00:38.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.964+0000 7f869af10700 1 -- start start 2023-10-06T05:00:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869af10700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f86941593d0 0x7f86940c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869af10700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 0x7f86940c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869af10700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940cc910 con 0x7f86941597a0 2023-10-06T05:00:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869af10700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940cd920 con 0x7f86941593d0 2023-10-06T05:00:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869af10700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940cbd40 con 0x7f86940cd4c0 2023-10-06T05:00:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869970d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f86941597a0 0x7f86940ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35598/0 (socket says 172.21.15.3:35598) 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869970d700 1 -- 172.21.15.3:0/474933067 learned_addr learned my addr 172.21.15.3:0/474933067 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f8699f0e700 1 --2- 172.21.15.3:0/474933067 >> v2:172.21.15.153:3300/0 conn(0x7f86941593d0 0x7f86940c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f869a70f700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 0x7f86940c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330320237 0 0) 0x7f86940cc910 con 0x7f86941597a0 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.965+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f867c003570 con 0x7f86941597a0 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361387757 0 0) 0x7f867c003570 con 0x7f86941597a0 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 msgr2=0x7f86940c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 0x7f86940c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 >> v2:172.21.15.153:3300/0 conn(0x7f86941593d0 msgr2=0x7f86940c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 --2- 172.21.15.3:0/474933067 >> v2:172.21.15.153:3300/0 conn(0x7f86941593d0 0x7f86940c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940cbfe0 con 0x7f86941597a0 2023-10-06T05:00:38.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f8688001470 con 0x7f86941597a0 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 >> v1:172.21.15.3:6789/0 conn(0x7f86941597a0 legacy=0x7f86940ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c003a60 0x7f867c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f867c004810 0x7f867c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 0x7f867c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940cd920 con 0x7f867c003a60 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940cbd40 con 0x7f867c004810 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c00ba60 con 0x7f86940cd4c0 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8699f0e700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c003a60 0x7f867c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f869970d700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f867c004810 0x7f867c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f869a70f700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 0x7f867c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8699f0e700 1 -- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 msgr2=0x7f867c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:38.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8699f0e700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86940cd4c0 0x7f867c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8699f0e700 1 -- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f867c004810 msgr2=0x7f867c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8699f0e700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f867c004810 0x7f867c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8699f0e700 1 -- 172.21.15.3:0/474933067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c003570 con 0x7f867c003a60 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8699f0e700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c003a60 0x7f867c003e70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f868c009c30 tx=0x7f868c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ae85a794d9aa82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00e710 con 0x7f867c003a60 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00e870 con 0x7f867c003a60 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.966+0000 7f8698f0c700 1 -- 172.21.15.3:0/474933067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00ec30 con 0x7f867c003a60 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.967+0000 7f869af10700 1 -- 172.21.15.3:0/474933067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c003a60 msgr2=0x7f867c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.967+0000 7f869af10700 1 --2- 172.21.15.3:0/474933067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c003a60 0x7f867c003e70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f868c009c30 tx=0x7f868c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.967+0000 7f869a70f700 1 -- 172.21.15.3:0/474933067 reap_dead start 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.967+0000 7f869af10700 1 -- 172.21.15.3:0/474933067 shutdown_connections 2023-10-06T05:00:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.967+0000 7f869af10700 1 -- 172.21.15.3:0/474933067 >> 172.21.15.3:0/474933067 conn(0x7f8694089df0 msgr2=0x7f8694099710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.967+0000 7f869af10700 1 -- 172.21.15.3:0/474933067 shutdown_connections 2023-10-06T05:00:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.967+0000 7f869af10700 1 -- 172.21.15.3:0/474933067 wait complete. 2023-10-06T05:00:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.968+0000 7f869af10700 1 Processor -- start 2023-10-06T05:00:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.968+0000 7f869af10700 1 -- start start 2023-10-06T05:00:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869af10700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f867c003a60 0x7f8694144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869af10700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f869415b6e0 0x7f869415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869af10700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869406e850 0x7f869406ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869af10700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869406f230 con 0x7f869415b6e0 2023-10-06T05:00:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869af10700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869415c090 con 0x7f869406e850 2023-10-06T05:00:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869af10700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c7410 con 0x7f867c003a60 2023-10-06T05:00:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f869415b6e0 0x7f869415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f869415b6e0 0x7f869415bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45974/0 (socket says 172.21.15.3:45974) 2023-10-06T05:00:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 -- 172.21.15.3:0/1193049760 learned_addr learned my addr 172.21.15.3:0/1193049760 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869a70f700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869406e850 0x7f869406ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f8699f0e700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f867c003a60 0x7f8694144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 -- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f867c003a60 msgr2=0x7f8694144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f867c003a60 0x7f8694144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 -- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869406e850 msgr2=0x7f869406ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869406e850 0x7f869406ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 -- 172.21.15.3:0/1193049760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694154760 con 0x7f869415b6e0 2023-10-06T05:00:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869970d700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f869415b6e0 0x7f869415bb50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f868800eb90 tx=0x7f86880014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f8691ffb700 1 -- 172.21.15.3:0/1193049760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688005e90 con 0x7f869415b6e0 2023-10-06T05:00:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f8691ffb700 1 -- 172.21.15.3:0/1193049760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800eea0 con 0x7f869415b6e0 2023-10-06T05:00:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f8691ffb700 1 -- 172.21.15.3:0/1193049760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800c990 con 0x7f869415b6e0 2023-10-06T05:00:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.969+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869415c2c0 con 0x7f869415b6e0 2023-10-06T05:00:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.970+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869415c880 con 0x7f869415b6e0 2023-10-06T05:00:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.970+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869404c380 con 0x7f869415b6e0 2023-10-06T05:00:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.971+0000 7f8691ffb700 1 -- 172.21.15.3:0/1193049760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f868800cd40 con 0x7f869415b6e0 2023-10-06T05:00:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.971+0000 7f8691ffb700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8680066f30 0x7f86800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.971+0000 7f8691ffb700 1 -- 172.21.15.3:0/1193049760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8688084560 con 0x7f869415b6e0 2023-10-06T05:00:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.972+0000 7f8699f0e700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8680066f30 0x7f86800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.972+0000 7f8699f0e700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8680066f30 0x7f86800693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f868c0091e0 tx=0x7f868c01a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:38.976+0000 7f8691ffb700 1 -- 172.21.15.3:0/1193049760 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86880515d0 con 0x7f869415b6e0 2023-10-06T05:00:39.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.172+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86940c7550 con 0x7f869415b6e0 2023-10-06T05:00:39.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.172+0000 7f8691ffb700 1 -- 172.21.15.3:0/1193049760 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16845 (secure 0 0 0) 0x7f8688054bf0 con 0x7f869415b6e0 2023-10-06T05:00:39.175 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:39.175 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":119,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:00:38.211751+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:00:35.953744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:00:37.188090+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:00:39.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8680066f30 msgr2=0x7f86800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8680066f30 0x7f86800693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f868c0091e0 tx=0x7f868c01a000 comp rx=0 tx=0).stop 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f869415b6e0 msgr2=0x7f869415bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f869415b6e0 0x7f869415bb50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f868800eb90 tx=0x7f86880014c0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 shutdown_connections 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8680066f30 0x7f86800693e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869406e850 0x7f869406ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f867c003a60 0x7f8694144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 --2- 172.21.15.3:0/1193049760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f869415b6e0 0x7f869415bb50 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 >> 172.21.15.3:0/1193049760 conn(0x7f8694089df0 msgr2=0x7f8694158480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 shutdown_connections 2023-10-06T05:00:39.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.175+0000 7f869af10700 1 -- 172.21.15.3:0/1193049760 wait complete. 2023-10-06T05:00:39.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:00:39.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.306+0000 7ff97676f700 1 Processor -- start 2023-10-06T05:00:39.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.306+0000 7ff97676f700 1 -- start start 2023-10-06T05:00:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff97676f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff9700b2530 0x7ff9700b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff97676f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff970159080 0x7ff9701594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff97676f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff97015e0d0 con 0x7ff9700b2900 2023-10-06T05:00:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff97676f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700b0e00 con 0x7ff970159080 2023-10-06T05:00:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff97676f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700cbd70 con 0x7ff9700b2530 2023-10-06T05:00:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff974d0c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff970159080 0x7ff9701594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff974d0c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff970159080 0x7ff9701594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45980/0 (socket says 172.21.15.3:45980) 2023-10-06T05:00:39.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff974d0c700 1 -- 172.21.15.3:0/3152037887 learned_addr learned my addr 172.21.15.3:0/3152037887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:39.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.307+0000 7ff96ffff700 1 --2- 172.21.15.3:0/3152037887 >> v2:172.21.15.153:3300/0 conn(0x7ff9700b2530 0x7ff9700b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff974d0c700 1 -- 172.21.15.3:0/3152037887 >> v2:172.21.15.153:3300/0 conn(0x7ff9700b2530 msgr2=0x7ff9700b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff974d0c700 1 --2- 172.21.15.3:0/3152037887 >> v2:172.21.15.153:3300/0 conn(0x7ff9700b2530 0x7ff9700b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff974d0c700 1 -- 172.21.15.3:0/3152037887 >> v1:172.21.15.153:6789/0 conn(0x7ff9700b2900 legacy=0x7ff97015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff974d0c700 1 -- 172.21.15.3:0/3152037887 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700cb160 con 0x7ff970159080 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff974d0c700 1 --2- 172.21.15.3:0/3152037887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff9701594c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff9700649a0 tx=0x7ff968009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d73d73d6abdd93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff96effd700 1 -- 172.21.15.3:0/3152037887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800e070 con 0x7ff970159080 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff96effd700 1 -- 172.21.15.3:0/3152037887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96800ba20 con 0x7ff970159080 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff96effd700 1 -- 172.21.15.3:0/3152037887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9680092e0 con 0x7ff970159080 2023-10-06T05:00:39.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff97676f700 1 -- 172.21.15.3:0/3152037887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 msgr2=0x7ff9701594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff97676f700 1 --2- 172.21.15.3:0/3152037887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff9701594c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff9700649a0 tx=0x7ff968009840 comp rx=0 tx=0).stop 2023-10-06T05:00:39.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff97676f700 1 -- 172.21.15.3:0/3152037887 shutdown_connections 2023-10-06T05:00:39.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff97676f700 1 --2- 172.21.15.3:0/3152037887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff9701594c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff97676f700 1 --2- 172.21.15.3:0/3152037887 >> v2:172.21.15.153:3300/0 conn(0x7ff9700b2530 0x7ff9700b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.308+0000 7ff97676f700 1 -- 172.21.15.3:0/3152037887 >> 172.21.15.3:0/3152037887 conn(0x7ff970070310 msgr2=0x7ff970070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:39.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.309+0000 7ff97676f700 1 -- 172.21.15.3:0/3152037887 shutdown_connections 2023-10-06T05:00:39.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.309+0000 7ff97676f700 1 -- 172.21.15.3:0/3152037887 wait complete. 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 Processor -- start 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 -- start start 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9700b2530 0x7ff9700c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9700b2900 0x7ff9700c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff970155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97015a270 con 0x7ff970159080 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700c4d90 con 0x7ff9700b2530 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.310+0000 7ff97676f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700c5010 con 0x7ff9700b2900 2023-10-06T05:00:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff970155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff970155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45990/0 (socket says 172.21.15.3:45990) 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 -- 172.21.15.3:0/4007050285 learned_addr learned my addr 172.21.15.3:0/4007050285 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff96ffff700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9700b2530 0x7ff9700c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff96f7fe700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9700b2900 0x7ff9700c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 -- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9700b2900 msgr2=0x7ff9700c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9700b2900 0x7ff9700c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 -- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9700b2530 msgr2=0x7ff9700c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9700b2530 0x7ff9700c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 -- 172.21.15.3:0/4007050285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968009ae0 con 0x7ff970159080 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff974d0c700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff970155800 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b6c0 tx=0x7ff968018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff967fff700 1 -- 172.21.15.3:0/4007050285 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968008ae0 con 0x7ff970159080 2023-10-06T05:00:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff967fff700 1 -- 172.21.15.3:0/4007050285 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff968008c60 con 0x7ff970159080 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9700c52f0 con 0x7ff970159080 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff967fff700 1 -- 172.21.15.3:0/4007050285 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968026460 con 0x7ff970159080 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.311+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9700c58b0 con 0x7ff970159080 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.312+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97015a410 con 0x7ff970159080 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.313+0000 7ff967fff700 1 -- 172.21.15.3:0/4007050285 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff968003040 con 0x7ff970159080 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.313+0000 7ff967fff700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff950066f30 0x7ff9500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.313+0000 7ff967fff700 1 -- 172.21.15.3:0/4007050285 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff968086580 con 0x7ff970159080 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.313+0000 7ff96ffff700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff950066f30 0x7ff9500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.314+0000 7ff96ffff700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff950066f30 0x7ff9500693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff958009d10 tx=0x7ff9580091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:39.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.317+0000 7ff967fff700 1 -- 172.21.15.3:0/4007050285 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff968053690 con 0x7ff970159080 2023-10-06T05:00:39.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.511+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9700c5aa0 con 0x7ff970159080 2023-10-06T05:00:39.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.512+0000 7ff967fff700 1 -- 172.21.15.3:0/4007050285 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16845 (secure 0 0 0) 0x7ff968056cb0 con 0x7ff970159080 2023-10-06T05:00:39.515 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:39.515 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":119,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:00:38.211751+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:00:35.953744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:00:37.188090+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:00:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff950066f30 msgr2=0x7ff9500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff950066f30 0x7ff9500693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff958009d10 tx=0x7ff9580091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 msgr2=0x7ff970155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff970155800 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b6c0 tx=0x7ff968018000 comp rx=0 tx=0).stop 2023-10-06T05:00:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 shutdown_connections 2023-10-06T05:00:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff950066f30 0x7ff9500693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9700b2530 0x7ff9700c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff970159080 0x7ff970155800 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 --2- 172.21.15.3:0/4007050285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9700b2900 0x7ff9700c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 >> 172.21.15.3:0/4007050285 conn(0x7ff970070310 msgr2=0x7ff9700ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 shutdown_connections 2023-10-06T05:00:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.514+0000 7ff97676f700 1 -- 172.21.15.3:0/4007050285 wait complete. 2023-10-06T05:00:39.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:00:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.651+0000 7f719f627700 1 Processor -- start 2023-10-06T05:00:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.651+0000 7f719f627700 1 -- start start 2023-10-06T05:00:39.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f719f627700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f71980af670 0x7f71980ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f719f627700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71980ce420 con 0x7f71980cc510 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f719f627700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719815a880 con 0x7f71980cc140 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f719f627700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719815ab60 con 0x7f71980af670 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f719cbc2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f71980cc510 0x7f71980aef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35612/0 (socket says 172.21.15.3:35612) 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f719cbc2700 1 -- 172.21.15.3:0/4153888475 learned_addr learned my addr 172.21.15.3:0/4153888475 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197332472 0 0) 0x7f71980ce420 con 0x7f71980cc510 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f719dbc4700 1 --2- 172.21.15.3:0/4153888475 >> v2:172.21.15.153:3300/0 conn(0x7f71980af670 0x7f71980ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7180003570 con 0x7f71980cc510 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 148613975 0 0) 0x7f7180003570 con 0x7f71980cc510 2023-10-06T05:00:39.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 >> v2:172.21.15.153:3300/0 conn(0x7f71980af670 msgr2=0x7f71980ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 --2- 172.21.15.3:0/4153888475 >> v2:172.21.15.153:3300/0 conn(0x7f71980af670 0x7f71980ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 >> v1:172.21.15.153:6789/0 conn(0x7f71980cc140 legacy=0x7f7198159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980ce0b0 con 0x7f71980cc510 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f718c001470 con 0x7f71980cc510 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 >> v1:172.21.15.3:6789/0 conn(0x7f71980cc510 legacy=0x7f71980aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7180003b30 0x7f7180003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7180004810 0x7f7180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f718000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719815ab60 con 0x7f71800052e0 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718000c190 con 0x7f7180004810 2023-10-06T05:00:39.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.652+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718000c350 con 0x7f7180003b30 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f718000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719cbc2700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7180004810 0x7f7180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719d3c3700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7180003b30 0x7f7180003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 -- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7180003b30 msgr2=0x7f7180003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7180003b30 0x7f7180003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 -- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7180004810 msgr2=0x7f7180004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7180004810 0x7f7180004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 -- 172.21.15.3:0/4153888475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7180003570 con 0x7f71800052e0 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 -- 172.21.15.3:0/4153888475 reap_dead start 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f719dbc4700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f718000ba60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7190009c00 tx=0x7f71900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b5618f355b5f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000e710 con 0x7f71800052e0 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719000e890 con 0x7f71800052e0 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.653+0000 7f7197fff700 1 -- 172.21.15.3:0/4153888475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000ec50 con 0x7f71800052e0 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.654+0000 7f719f627700 1 -- 172.21.15.3:0/4153888475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 msgr2=0x7f718000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.654+0000 7f719f627700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f718000ba60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7190009c00 tx=0x7f71900091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.654+0000 7f719f627700 1 -- 172.21.15.3:0/4153888475 shutdown_connections 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.654+0000 7f719f627700 1 --2- 172.21.15.3:0/4153888475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f718000ba60 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.654+0000 7f719f627700 1 -- 172.21.15.3:0/4153888475 >> 172.21.15.3:0/4153888475 conn(0x7f71980d03e0 msgr2=0x7f71980d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.654+0000 7f719f627700 1 -- 172.21.15.3:0/4153888475 shutdown_connections 2023-10-06T05:00:39.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.654+0000 7f719f627700 1 -- 172.21.15.3:0/4153888475 wait complete. 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.655+0000 7f719f627700 1 Processor -- start 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.655+0000 7f719f627700 1 -- start start 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719f627700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f71980b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719f627700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f719816a950 0x7f71981526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719f627700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71980b0b50 0x7f71980b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719f627700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719816e1e0 con 0x7f71800052e0 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719f627700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71980b1510 con 0x7f719816a950 2023-10-06T05:00:39.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719f627700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719816d310 con 0x7f71980b0b50 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f71980b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f71980b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46000/0 (socket says 172.21.15.3:46000) 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 -- 172.21.15.3:0/435835420 learned_addr learned my addr 172.21.15.3:0/435835420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719cbc2700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f719816a950 0x7f71981526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719dbc4700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71980b0b50 0x7f71980b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 -- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71980b0b50 msgr2=0x7f71980b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71980b0b50 0x7f71980b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 -- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f719816a950 msgr2=0x7f71981526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f719816a950 0x7f71981526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 -- 172.21.15.3:0/435835420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980b6020 con 0x7f71800052e0 2023-10-06T05:00:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.656+0000 7f719d3c3700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f71980b5ae0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f718800c1c0 tx=0x7f7188001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.657+0000 7f71957fa700 1 -- 172.21.15.3:0/435835420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718800cdb0 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.657+0000 7f71957fa700 1 -- 172.21.15.3:0/435835420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7188015430 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.657+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719816d560 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.657+0000 7f71957fa700 1 -- 172.21.15.3:0/435835420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71880157f0 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.657+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719816db20 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.658+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719804c380 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.658+0000 7f71957fa700 1 -- 172.21.15.3:0/435835420 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7188015b60 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.659+0000 7f71957fa700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7184066f30 0x7f71840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.659+0000 7f71957fa700 1 -- 172.21.15.3:0/435835420 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f71880853c0 con 0x7f71800052e0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.659+0000 7f719cbc2700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7184066f30 0x7f71840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:39.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.659+0000 7f719cbc2700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7184066f30 0x7f71840693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71980cbb30 tx=0x7f718c001d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:39.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.663+0000 7f71957fa700 1 -- 172.21.15.3:0/435835420 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71880524d0 con 0x7f71800052e0 2023-10-06T05:00:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:39.903+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f71980d0100 con 0x7f71800052e0 2023-10-06T05:00:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.823+0000 7f71957fa700 1 -- 172.21.15.3:0/435835420 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7188055af0 con 0x7f71800052e0 2023-10-06T05:00:40.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7184066f30 msgr2=0x7f71840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:40.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7184066f30 0x7f71840693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71980cbb30 tx=0x7f718c001d80 comp rx=0 tx=0).stop 2023-10-06T05:00:40.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 msgr2=0x7f71980b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:40.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f71980b5ae0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f718800c1c0 tx=0x7f7188001a70 comp rx=0 tx=0).stop 2023-10-06T05:00:40.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 shutdown_connections 2023-10-06T05:00:40.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7184066f30 0x7f71840693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71980b0b50 0x7f71980b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71800052e0 0x7f71980b5ae0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 --2- 172.21.15.3:0/435835420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f719816a950 0x7f71981526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 >> 172.21.15.3:0/435835420 conn(0x7f71980d03e0 msgr2=0x7f719815c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:40.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 shutdown_connections 2023-10-06T05:00:40.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.826+0000 7f719f627700 1 -- 172.21.15.3:0/435835420 wait complete. 2023-10-06T05:00:40.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:00:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.971+0000 7fee4eb5a700 1 Processor -- start 2023-10-06T05:00:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.971+0000 7fee4eb5a700 1 -- start start 2023-10-06T05:00:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4eb5a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480b06b0 0x7fee480a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4eb5a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee480cc9f0 0x7fee480a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4eb5a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee480af210 con 0x7fee48159530 2023-10-06T05:00:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4eb5a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480ab7d0 con 0x7fee480cc9f0 2023-10-06T05:00:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4eb5a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee48159900 con 0x7fee480b06b0 2023-10-06T05:00:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4e359700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee480cc9f0 0x7fee480a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4e359700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee480cc9f0 0x7fee480a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46004/0 (socket says 172.21.15.3:46004) 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4e359700 1 -- 172.21.15.3:0/1835657693 learned_addr learned my addr 172.21.15.3:0/1835657693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4d357700 1 --2- 172.21.15.3:0/1835657693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480b06b0 0x7fee480a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4e359700 1 -- 172.21.15.3:0/1835657693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480b06b0 msgr2=0x7fee480a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4e359700 1 --2- 172.21.15.3:0/1835657693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480b06b0 0x7fee480a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4e359700 1 -- 172.21.15.3:0/1835657693 >> v1:172.21.15.153:6789/0 conn(0x7fee48159530 legacy=0x7fee480ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4e359700 1 -- 172.21.15.3:0/1835657693 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee480acd40 con 0x7fee480cc9f0 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.972+0000 7fee4d357700 1 --2- 172.21.15.3:0/1835657693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480b06b0 0x7fee480a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4e359700 1 --2- 172.21.15.3:0/1835657693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480cc9f0 0x7fee480a9d90 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fee40009f20 tx=0x7fee40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1746881bd243354 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4cb56700 1 -- 172.21.15.3:0/1835657693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4000e070 con 0x7fee480cc9f0 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4cb56700 1 -- 172.21.15.3:0/1835657693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4000ba80 con 0x7fee480cc9f0 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4cb56700 1 -- 172.21.15.3:0/1835657693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee400092e0 con 0x7fee480cc9f0 2023-10-06T05:00:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 -- 172.21.15.3:0/1835657693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480cc9f0 msgr2=0x7fee480a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/1835657693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480cc9f0 0x7fee480a9d90 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fee40009f20 tx=0x7fee40009840 comp rx=0 tx=0).stop 2023-10-06T05:00:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 -- 172.21.15.3:0/1835657693 shutdown_connections 2023-10-06T05:00:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/1835657693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480cc9f0 0x7fee480a9d90 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/1835657693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480b06b0 0x7fee480a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 -- 172.21.15.3:0/1835657693 >> 172.21.15.3:0/1835657693 conn(0x7fee4806f500 msgr2=0x7fee480cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 -- 172.21.15.3:0/1835657693 shutdown_connections 2023-10-06T05:00:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.973+0000 7fee4eb5a700 1 -- 172.21.15.3:0/1835657693 wait complete. 2023-10-06T05:00:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.974+0000 7fee4eb5a700 1 Processor -- start 2023-10-06T05:00:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4eb5a700 1 -- start start 2023-10-06T05:00:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4eb5a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480b06b0 0x7fee48149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4eb5a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480cc9f0 0x7fee4815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4eb5a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee48159530 0x7fee48168f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4eb5a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48169f40 con 0x7fee480b06b0 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4eb5a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4815af10 con 0x7fee48159530 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4eb5a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4814a1f0 con 0x7fee480cc9f0 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4db58700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480b06b0 0x7fee48149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4db58700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480b06b0 0x7fee48149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46022/0 (socket says 172.21.15.3:46022) 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.975+0000 7fee4db58700 1 -- 172.21.15.3:0/2726915828 learned_addr learned my addr 172.21.15.3:0/2726915828 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4d357700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480cc9f0 0x7fee4815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4e359700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee48159530 0x7fee48168f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4db58700 1 -- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480cc9f0 msgr2=0x7fee4815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4db58700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480cc9f0 0x7fee4815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4db58700 1 -- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee48159530 msgr2=0x7fee48168f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4db58700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee48159530 0x7fee48168f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4db58700 1 -- 172.21.15.3:0/2726915828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40009ae0 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4db58700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480b06b0 0x7fee48149cb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c590 tx=0x7fee3800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee45ffb700 1 -- 172.21.15.3:0/2726915828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38011590 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee45ffb700 1 -- 172.21.15.3:0/2726915828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380116f0 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee48169980 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee45ffb700 1 -- 172.21.15.3:0/2726915828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38011ab0 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.976+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4815aba0 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.977+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4815b1f0 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.977+0000 7fee45ffb700 1 -- 172.21.15.3:0/2726915828 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee3801b3b0 con 0x7fee480b06b0 2023-10-06T05:00:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.978+0000 7fee45ffb700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee34066f30 0x7fee340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.978+0000 7fee45ffb700 1 -- 172.21.15.3:0/2726915828 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fee38083c20 con 0x7fee480b06b0 2023-10-06T05:00:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.978+0000 7fee4d357700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee34066f30 0x7fee340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.979+0000 7fee4d357700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee34066f30 0x7fee340693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006f00 tx=0x7fee3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:40.982+0000 7fee45ffb700 1 -- 172.21.15.3:0/2726915828 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee38050d30 con 0x7fee480b06b0 2023-10-06T05:00:41.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.219+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fee4804c380 con 0x7fee480b06b0 2023-10-06T05:00:41.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.831+0000 7fee45ffb700 1 -- 172.21.15.3:0/2726915828 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7fee38017070 con 0x7fee480b06b0 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.833+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee34066f30 msgr2=0x7fee340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.833+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee34066f30 0x7fee340693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006f00 tx=0x7fee3c008040 comp rx=0 tx=0).stop 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480b06b0 msgr2=0x7fee48149cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480b06b0 0x7fee48149cb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c590 tx=0x7fee3800c950 comp rx=0 tx=0).stop 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 shutdown_connections 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee34066f30 0x7fee340693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee48159530 0x7fee48168f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee480b06b0 0x7fee48149cb0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 --2- 172.21.15.3:0/2726915828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee480cc9f0 0x7fee4815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 >> 172.21.15.3:0/2726915828 conn(0x7fee4806f500 msgr2=0x7fee481549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 shutdown_connections 2023-10-06T05:00:41.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.834+0000 7fee4eb5a700 1 -- 172.21.15.3:0/2726915828 wait complete. 2023-10-06T05:00:41.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:41.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.974+0000 7f2f7cc83700 1 Processor -- start 2023-10-06T05:00:41.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.974+0000 7f2f7cc83700 1 -- start start 2023-10-06T05:00:41.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f7cc83700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2f780cb3b0 0x7f2f780cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f7cc83700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 0x7f2f78158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f7cc83700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f780cf130 con 0x7f2f78159500 2023-10-06T05:00:41.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f7cc83700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780ce520 con 0x7f2f780cb3b0 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f7cc83700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780ce800 con 0x7f2f780cb780 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f77fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2f78159500 0x7f2f780cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35626/0 (socket says 172.21.15.3:35626) 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f77fff700 1 -- 172.21.15.3:0/3810674621 learned_addr learned my addr 172.21.15.3:0/3810674621 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f777fe700 1 --2- 172.21.15.3:0/3810674621 >> v2:172.21.15.153:3300/0 conn(0x7f2f780cb3b0 0x7f2f780cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 0x7f2f78158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262669424 0 0) 0x7f2f780cf130 con 0x7f2f78159500 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.975+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f60003570 con 0x7f2f78159500 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1831397700 0 0) 0x7f2f60003570 con 0x7f2f78159500 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 msgr2=0x7f2f78158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 0x7f2f78158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 >> v2:172.21.15.153:3300/0 conn(0x7f2f780cb3b0 msgr2=0x7f2f780cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 --2- 172.21.15.3:0/3810674621 >> v2:172.21.15.153:3300/0 conn(0x7f2f780cb3b0 0x7f2f780cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780ceae0 con 0x7f2f78159500 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f2f700017f0 con 0x7f2f78159500 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 >> v1:172.21.15.3:6789/0 conn(0x7f2f78159500 legacy=0x7f2f780cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 0x7f2f60003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fe0 0x7f2f60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f60004d50 0x7f2f600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780ce520 con 0x7f2f60003fe0 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780ce800 con 0x7f2f60004d50 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6000ba90 con 0x7f2f780cb780 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fe0 0x7f2f60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f777fe700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 0x7f2f60003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f77fff700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f60004d50 0x7f2f600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f76ffd700 1 -- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 msgr2=0x7f2f60003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 0x7f2f60003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f76ffd700 1 -- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f60004d50 msgr2=0x7f2f600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f60004d50 0x7f2f600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f76ffd700 1 -- 172.21.15.3:0/3810674621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f60003570 con 0x7f2f60003fe0 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f77fff700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f60004d50 0x7f2f600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f777fe700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f780cb780 0x7f2f60003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fe0 0x7f2f60004810 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009c00 tx=0x7f2f6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adf6b0cf85ef2c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00e6b0 con 0x7f2f60003fe0 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c00e810 con 0x7f2f60003fe0 2023-10-06T05:00:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.976+0000 7f2f767fc700 1 -- 172.21.15.3:0/3810674621 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00ebd0 con 0x7f2f60003fe0 2023-10-06T05:00:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.977+0000 7f2f7cc83700 1 -- 172.21.15.3:0/3810674621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fe0 msgr2=0x7f2f60004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.977+0000 7f2f7cc83700 1 --2- 172.21.15.3:0/3810674621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fe0 0x7f2f60004810 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009c00 tx=0x7f2f6c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.977+0000 7f2f77fff700 1 -- 172.21.15.3:0/3810674621 reap_dead start 2023-10-06T05:00:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.977+0000 7f2f7cc83700 1 -- 172.21.15.3:0/3810674621 shutdown_connections 2023-10-06T05:00:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.977+0000 7f2f7cc83700 1 -- 172.21.15.3:0/3810674621 >> 172.21.15.3:0/3810674621 conn(0x7f2f78070310 msgr2=0x7f2f78070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.977+0000 7f2f7cc83700 1 -- 172.21.15.3:0/3810674621 shutdown_connections 2023-10-06T05:00:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.977+0000 7f2f7cc83700 1 -- 172.21.15.3:0/3810674621 wait complete. 2023-10-06T05:00:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.978+0000 7f2f7cc83700 1 Processor -- start 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.978+0000 7f2f7cc83700 1 -- start start 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f7cc83700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f60003fe0 0x7f2f781539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f7cc83700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780c9db0 0x7f2f780b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f7cc83700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f780b54b0 0x7f2f780b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f7cc83700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780b5e90 con 0x7f2f780c9db0 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f7cc83700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781694b0 con 0x7f2f780b54b0 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f7cc83700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780b4620 con 0x7f2f60003fe0 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780c9db0 0x7f2f780b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780c9db0 0x7f2f780b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46046/0 (socket says 172.21.15.3:46046) 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 -- 172.21.15.3:0/1435911080 learned_addr learned my addr 172.21.15.3:0/1435911080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f77fff700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f780b54b0 0x7f2f780b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f777fe700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f60003fe0 0x7f2f781539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 -- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f60003fe0 msgr2=0x7f2f781539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f60003fe0 0x7f2f781539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 -- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f780b54b0 msgr2=0x7f2f780b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f780b54b0 0x7f2f780b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.979+0000 7f2f76ffd700 1 -- 172.21.15.3:0/1435911080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780b4900 con 0x7f2f780c9db0 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.980+0000 7f2f76ffd700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780c9db0 0x7f2f780b3cd0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2f60004730 tx=0x7f2f6c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.980+0000 7f2f5f7fe700 1 -- 172.21.15.3:0/1435911080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c022070 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.980+0000 7f2f5f7fe700 1 -- 172.21.15.3:0/1435911080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c00e980 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.980+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f780b4be0 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.980+0000 7f2f5f7fe700 1 -- 172.21.15.3:0/1435911080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c027470 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.980+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780b4210 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.981+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7804c380 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.981+0000 7f2f5f7fe700 1 -- 172.21.15.3:0/1435911080 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f6c01c450 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.981+0000 7f2f5f7fe700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f64066f30 0x7f2f640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.982+0000 7f2f5f7fe700 1 -- 172.21.15.3:0/1435911080 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2f6c0860f0 con 0x7f2f780c9db0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.982+0000 7f2f777fe700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f64066f30 0x7f2f640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.982+0000 7f2f777fe700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f64066f30 0x7f2f640693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009ee0 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:41.986+0000 7f2f5f7fe700 1 -- 172.21.15.3:0/1435911080 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f6c053200 con 0x7f2f780c9db0 2023-10-06T05:00:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.224+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f780cedc0 con 0x7f2f780c9db0 2023-10-06T05:00:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.225+0000 7f2f5f7fe700 1 -- 172.21.15.3:0/1435911080 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f2f6c056820 con 0x7f2f780c9db0 2023-10-06T05:00:42.228 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:42.228 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:41.823898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f64066f30 msgr2=0x7f2f640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f64066f30 0x7f2f640693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009ee0 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780c9db0 msgr2=0x7f2f780b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780c9db0 0x7f2f780b3cd0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2f60004730 tx=0x7f2f6c01a000 comp rx=0 tx=0).stop 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 shutdown_connections 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2f64066f30 0x7f2f640693e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f780b54b0 0x7f2f780b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f60003fe0 0x7f2f781539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 --2- 172.21.15.3:0/1435911080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780c9db0 0x7f2f780b3cd0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 >> 172.21.15.3:0/1435911080 conn(0x7f2f78070310 msgr2=0x7f2f780b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 shutdown_connections 2023-10-06T05:00:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.227+0000 7f2f7cc83700 1 -- 172.21.15.3:0/1435911080 wait complete. 2023-10-06T05:00:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 79 2023-10-06T05:00:42.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:00:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.359+0000 7fd00aa8e700 1 Processor -- start 2023-10-06T05:00:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.359+0000 7fd00aa8e700 1 -- start start 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd00aa8e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd004159500 0x7fd0040cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd00aa8e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0041598d0 0x7fd0040cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd00aa8e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0040cc5c0 con 0x7fd0040b2220 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd00aa8e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040cb7b0 con 0x7fd0041598d0 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd00aa8e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040cda60 con 0x7fd004159500 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0041598d0 0x7fd0040cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0041598d0 0x7fd0040cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36986/0 (socket says 172.21.15.3:36986) 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 -- 172.21.15.3:0/451466577 learned_addr learned my addr 172.21.15.3:0/451466577 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd00882a700 1 --2- 172.21.15.3:0/451466577 >> v2:172.21.15.153:3300/0 conn(0x7fd004159500 0x7fd0040cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 -- 172.21.15.3:0/451466577 >> v2:172.21.15.153:3300/0 conn(0x7fd004159500 msgr2=0x7fd0040cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 --2- 172.21.15.3:0/451466577 >> v2:172.21.15.153:3300/0 conn(0x7fd004159500 0x7fd0040cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 -- 172.21.15.3:0/451466577 >> v1:172.21.15.153:6789/0 conn(0x7fd0040b2220 legacy=0x7fd0040b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 -- 172.21.15.3:0/451466577 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040cb9b0 con 0x7fd0041598d0 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.360+0000 7fd003fff700 1 --2- 172.21.15.3:0/451466577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0041598d0 0x7fd0040cb270 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd004089390 tx=0x7fcff0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e89d27fba8f6e83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd0037fe700 1 -- 172.21.15.3:0/451466577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000b860 con 0x7fd0041598d0 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd0037fe700 1 -- 172.21.15.3:0/451466577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff000b9c0 con 0x7fd0041598d0 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd0037fe700 1 -- 172.21.15.3:0/451466577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000bd60 con 0x7fd0041598d0 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 -- 172.21.15.3:0/451466577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0041598d0 msgr2=0x7fd0040cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/451466577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0041598d0 0x7fd0040cb270 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd004089390 tx=0x7fcff0009250 comp rx=0 tx=0).stop 2023-10-06T05:00:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 -- 172.21.15.3:0/451466577 shutdown_connections 2023-10-06T05:00:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/451466577 >> v2:172.21.15.153:3300/0 conn(0x7fd004159500 0x7fd0040cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/451466577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0041598d0 0x7fd0040cb270 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 -- 172.21.15.3:0/451466577 >> 172.21.15.3:0/451466577 conn(0x7fd004070310 msgr2=0x7fd004070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 -- 172.21.15.3:0/451466577 shutdown_connections 2023-10-06T05:00:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.361+0000 7fd00aa8e700 1 -- 172.21.15.3:0/451466577 wait complete. 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.362+0000 7fd00aa8e700 1 Processor -- start 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.362+0000 7fd00aa8e700 1 -- start start 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00aa8e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0040b2220 0x7fd0040b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00aa8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd004159500 0x7fd004150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00aa8e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0041598d0 0x7fd0040d0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00aa8e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c45c0 con 0x7fd004159500 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00aa8e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004156270 con 0x7fd0041598d0 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00aa8e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd004155f90 con 0x7fd0040b2220 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd003fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd004159500 0x7fd004150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd003fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd004159500 0x7fd004150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36998/0 (socket says 172.21.15.3:36998) 2023-10-06T05:00:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd003fff700 1 -- 172.21.15.3:0/1188582924 learned_addr learned my addr 172.21.15.3:0/1188582924 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00882a700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0040b2220 0x7fd0040b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.363+0000 7fd00902b700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0041598d0 0x7fd0040d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd003fff700 1 -- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0040b2220 msgr2=0x7fd0040b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd003fff700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0040b2220 0x7fd0040b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd003fff700 1 -- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0041598d0 msgr2=0x7fd0040d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd003fff700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0041598d0 0x7fd0040d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd003fff700 1 -- 172.21.15.3:0/1188582924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff00094f0 con 0x7fd004159500 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd00902b700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0041598d0 0x7fd0040d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd003fff700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd004159500 0x7fd004150480 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fcff0006fd0 tx=0x7fcff0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd000ff9700 1 -- 172.21.15.3:0/1188582924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0024440 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd000ff9700 1 -- 172.21.15.3:0/1188582924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff00245a0 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd000ff9700 1 -- 172.21.15.3:0/1188582924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0024960 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd004156f70 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.364+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd004157530 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.365+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004156dd0 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.365+0000 7fd000ff9700 1 -- 172.21.15.3:0/1188582924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcff001a420 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.366+0000 7fd000ff9700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcfe8066f30 0x7fcfe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.366+0000 7fd000ff9700 1 -- 172.21.15.3:0/1188582924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcff0086500 con 0x7fd004159500 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.366+0000 7fd00882a700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcfe8066f30 0x7fcfe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.366+0000 7fd00882a700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcfe8066f30 0x7fcfe80693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcff8006fd0 tx=0x7fcff8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.370+0000 7fd000ff9700 1 -- 172.21.15.3:0/1188582924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd004156dd0 con 0x7fd004159500 2023-10-06T05:00:42.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.566+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd004156dd0 con 0x7fd004159500 2023-10-06T05:00:42.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.567+0000 7fd000ff9700 1 -- 172.21.15.3:0/1188582924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16832 (secure 0 0 0) 0x7fd004156dd0 con 0x7fd004159500 2023-10-06T05:00:42.569 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:42.570 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":120,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:00:39.812612+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:00:35.953744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:00:37.188090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:00:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcfe8066f30 msgr2=0x7fcfe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcfe8066f30 0x7fcfe80693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcff8006fd0 tx=0x7fcff8008040 comp rx=0 tx=0).stop 2023-10-06T05:00:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd004159500 msgr2=0x7fd004150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd004159500 0x7fd004150480 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fcff0006fd0 tx=0x7fcff0017000 comp rx=0 tx=0).stop 2023-10-06T05:00:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 shutdown_connections 2023-10-06T05:00:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcfe8066f30 0x7fcfe80693e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0041598d0 0x7fd0040d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd0040b2220 0x7fd0040b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 --2- 172.21.15.3:0/1188582924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd004159500 0x7fd004150480 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 >> 172.21.15.3:0/1188582924 conn(0x7fd004070310 msgr2=0x7fd00415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 shutdown_connections 2023-10-06T05:00:42.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.569+0000 7fd00aa8e700 1 -- 172.21.15.3:0/1188582924 wait complete. 2023-10-06T05:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:00:42.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:00:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 Processor -- start 2023-10-06T05:00:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 -- start start 2023-10-06T05:00:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1dc157030 0x7fc1dc159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc15a220 0x7fc1dc0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1dc157400 0x7fc1dc0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc0b40b0 con 0x7fc1dc157030 2023-10-06T05:00:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc15aeb0 con 0x7fc1dc157400 2023-10-06T05:00:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.702+0000 7fc1e3039700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc15a7a0 con 0x7fc1dc15a220 2023-10-06T05:00:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1dc157030 0x7fc1dc159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1dc157030 0x7fc1dc159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37004/0 (socket says 172.21.15.3:37004) 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 -- 172.21.15.3:0/158456896 learned_addr learned my addr 172.21.15.3:0/158456896 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e15d6700 1 --2- 172.21.15.3:0/158456896 >> v2:172.21.15.153:3300/0 conn(0x7fc1dc157400 0x7fc1dc0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1dbfff700 1 --2- 172.21.15.3:0/158456896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc15a220 0x7fc1dc0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 -- 172.21.15.3:0/158456896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc15a220 msgr2=0x7fc1dc0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 --2- 172.21.15.3:0/158456896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc15a220 0x7fc1dc0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 -- 172.21.15.3:0/158456896 >> v2:172.21.15.153:3300/0 conn(0x7fc1dc157400 msgr2=0x7fc1dc0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 --2- 172.21.15.3:0/158456896 >> v2:172.21.15.153:3300/0 conn(0x7fc1dc157400 0x7fc1dc0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 -- 172.21.15.3:0/158456896 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc15aa80 con 0x7fc1dc157030 2023-10-06T05:00:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1dbfff700 1 --2- 172.21.15.3:0/158456896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc15a220 0x7fc1dc0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1e0dd5700 1 --2- 172.21.15.3:0/158456896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc157030 0x7fc1dc159ce0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15a690 tx=0x7fc1cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a17d342fc32630 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1db7fe700 1 -- 172.21.15.3:0/158456896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc009e70 con 0x7fc1dc157030 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1db7fe700 1 -- 172.21.15.3:0/158456896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc00b860 con 0x7fc1dc157030 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.703+0000 7fc1db7fe700 1 -- 172.21.15.3:0/158456896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00bc00 con 0x7fc1dc157030 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 -- 172.21.15.3:0/158456896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc157030 msgr2=0x7fc1dc159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 --2- 172.21.15.3:0/158456896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc157030 0x7fc1dc159ce0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15a690 tx=0x7fc1cc009250 comp rx=0 tx=0).stop 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 -- 172.21.15.3:0/158456896 shutdown_connections 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 --2- 172.21.15.3:0/158456896 >> v2:172.21.15.153:3300/0 conn(0x7fc1dc157400 0x7fc1dc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 --2- 172.21.15.3:0/158456896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc157030 0x7fc1dc159ce0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 --2- 172.21.15.3:0/158456896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc15a220 0x7fc1dc0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 -- 172.21.15.3:0/158456896 >> 172.21.15.3:0/158456896 conn(0x7fc1dc0ce270 msgr2=0x7fc1dc0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 -- 172.21.15.3:0/158456896 shutdown_connections 2023-10-06T05:00:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.704+0000 7fc1e3039700 1 -- 172.21.15.3:0/158456896 wait complete. 2023-10-06T05:00:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.705+0000 7fc1e3039700 1 Processor -- start 2023-10-06T05:00:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.705+0000 7fc1e3039700 1 -- start start 2023-10-06T05:00:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e3039700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc157030 0x7fc1dc169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e3039700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1dc157400 0x7fc1dc1515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e3039700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc15a220 0x7fc1dc15e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e3039700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc15d880 con 0x7fc1dc15a220 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e3039700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc154f30 con 0x7fc1dc157400 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e3039700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc155210 con 0x7fc1dc157030 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc15a220 0x7fc1dc15e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc15a220 0x7fc1dc15e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37010/0 (socket says 172.21.15.3:37010) 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 -- 172.21.15.3:0/2529903802 learned_addr learned my addr 172.21.15.3:0/2529903802 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1dbfff700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1dc157400 0x7fc1dc1515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e0dd5700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc157030 0x7fc1dc169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 -- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc157030 msgr2=0x7fc1dc169530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc157030 0x7fc1dc169530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 -- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1dc157400 msgr2=0x7fc1dc1515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1dc157400 0x7fc1dc1515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 -- 172.21.15.3:0/2529903802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0094f0 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e0dd5700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc157030 0x7fc1dc169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.706+0000 7fc1e15d6700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc15a220 0x7fc1dc15e230 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009e90 tx=0x7fc1d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.707+0000 7fc1d8ff9700 1 -- 172.21.15.3:0/2529903802 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400cc20 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.707+0000 7fc1d8ff9700 1 -- 172.21.15.3:0/2529903802 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d400f040 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.707+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1dc1554f0 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.707+0000 7fc1d8ff9700 1 -- 172.21.15.3:0/2529903802 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4014480 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.707+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1dc169a70 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.708+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1dc169cf0 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.708+0000 7fc1d8ff9700 1 -- 172.21.15.3:0/2529903802 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1d4014830 con 0x7fc1dc15a220 2023-10-06T05:00:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.708+0000 7fc1d8ff9700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.709+0000 7fc1d8ff9700 1 -- 172.21.15.3:0/2529903802 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1d4085390 con 0x7fc1dc15a220 2023-10-06T05:00:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.709+0000 7fc1e0dd5700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.709+0000 7fc1e0dd5700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1bc066f30 0x7fc1bc0693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15a690 tx=0x7fc1cc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.713+0000 7fc1d8ff9700 1 -- 172.21.15.3:0/2529903802 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1d4055f30 con 0x7fc1dc15a220 2023-10-06T05:00:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.900+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1dc071680 con 0x7fc1bc066f30 2023-10-06T05:00:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.910+0000 7fc1d8ff9700 1 -- 172.21.15.3:0/2529903802 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312087 (secure 0 0 0) 0x7fc1dc071680 con 0x7fc1bc066f30 2023-10-06T05:00:42.916 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1bc066f30 msgr2=0x7fc1bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1bc066f30 0x7fc1bc0693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15a690 tx=0x7fc1cc017000 comp rx=0 tx=0).stop 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc15a220 msgr2=0x7fc1dc15e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc15a220 0x7fc1dc15e230 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009e90 tx=0x7fc1d400b040 comp rx=0 tx=0).stop 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 shutdown_connections 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1dc15a220 0x7fc1dc15e230 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1dc157030 0x7fc1dc169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 --2- 172.21.15.3:0/2529903802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1dc157400 0x7fc1dc1515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.913+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 >> 172.21.15.3:0/2529903802 conn(0x7fc1dc0ce270 msgr2=0x7fc1dc1536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.914+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 shutdown_connections 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:42.914+0000 7fc1e3039700 1 -- 172.21.15.3:0/2529903802 wait complete. 2023-10-06T05:00:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:00:42.928 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2023-10-06T05:00:41.547296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2448500,"kb_used_data":12852,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752526220,"statfs":{"total":773094113280,"available":770586849280,"internally_reserved":0,"allocated":13160448,"data_stored":6590731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.748223"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825458+0000","last_change":"2023-10-06T05:00:38.825458+0000","last_active":"2023-10-06T05:00:38.825458+0000","last_peered":"2023-10-06T05:00:38.825458+0000","last_clean":"2023-10-06T05:00:38.825458+0000","last_became_active":"2023-10-06T05:00:38.825076+0000","last_became_peered":"2023-10-06T05:00:38.825076+0000","last_unstale":"2023-10-06T05:00:38.825458+0000","last_undegraded":"2023-10-06T05:00:38.825458+0000","last_fullsized":"2023-10-06T05:00:38.825458+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:50.882176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833277+0000","last_change":"2023-10-06T05:00:38.832340+0000","last_active":"2023-10-06T05:00:38.833277+0000","last_peered":"2023-10-06T05:00:38.833277+0000","last_clean":"2023-10-06T05:00:38.833277+0000","last_became_active":"2023-10-06T05:00:38.831953+0000","last_became_peered":"2023-10-06T05:00:38.831953+0000","last_unstale":"2023-10-06T05:00:38.833277+0000","last_undegraded":"2023-10-06T05:00:38.833277+0000","last_fullsized":"2023-10-06T05:00:38.833277+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:11.089925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826988+0000","last_change":"2023-10-06T05:00:38.824669+0000","last_active":"2023-10-06T05:00:38.826988+0000","last_peered":"2023-10-06T05:00:38.826988+0000","last_clean":"2023-10-06T05:00:38.826988+0000","last_became_active":"2023-10-06T05:00:38.824091+0000","last_became_peered":"2023-10-06T05:00:38.824091+0000","last_unstale":"2023-10-06T05:00:38.826988+0000","last_undegraded":"2023-10-06T05:00:38.826988+0000","last_fullsized":"2023-10-06T05:00:38.826988+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:42.119873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832445+0000","last_change":"2023-10-06T05:00:38.827469+0000","last_active":"2023-10-06T05:00:38.832445+0000","last_peered":"2023-10-06T05:00:38.832445+0000","last_clean":"2023-10-06T05:00:38.832445+0000","last_became_active":"2023-10-06T05:00:38.827215+0000","last_became_peered":"2023-10-06T05:00:38.827215+0000","last_unstale":"2023-10-06T05:00:38.832445+0000","last_undegraded":"2023-10-06T05:00:38.832445+0000","last_fullsized":"2023-10-06T05:00:38.832445+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:48.164637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835737+0000","last_change":"2023-10-06T05:00:38.835737+0000","last_active":"2023-10-06T05:00:38.835737+0000","last_peered":"2023-10-06T05:00:38.835737+0000","last_clean":"2023-10-06T05:00:38.835737+0000","last_became_active":"2023-10-06T05:00:38.835489+0000","last_became_peered":"2023-10-06T05:00:38.835489+0000","last_unstale":"2023-10-06T05:00:38.835737+0000","last_undegraded":"2023-10-06T05:00:38.835737+0000","last_fullsized":"2023-10-06T05:00:38.835737+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:50.530154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824040+0000","last_change":"2023-10-06T05:00:38.824040+0000","last_active":"2023-10-06T05:00:38.824040+0000","last_peered":"2023-10-06T05:00:38.824040+0000","last_clean":"2023-10-06T05:00:38.824040+0000","last_became_active":"2023-10-06T05:00:38.823687+0000","last_became_peered":"2023-10-06T05:00:38.823687+0000","last_unstale":"2023-10-06T05:00:38.824040+0000","last_undegraded":"2023-10-06T05:00:38.824040+0000","last_fullsized":"2023-10-06T05:00:38.824040+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:26.130218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837362+0000","last_change":"2023-10-06T05:00:38.837362+0000","last_active":"2023-10-06T05:00:38.837362+0000","last_peered":"2023-10-06T05:00:38.837362+0000","last_clean":"2023-10-06T05:00:38.837362+0000","last_became_active":"2023-10-06T05:00:38.836250+0000","last_became_peered":"2023-10-06T05:00:38.836250+0000","last_unstale":"2023-10-06T05:00:38.837362+0000","last_undegraded":"2023-10-06T05:00:38.837362+0000","last_fullsized":"2023-10-06T05:00:38.837362+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:05.697530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832945+0000","last_change":"2023-10-06T05:00:38.829619+0000","last_active":"2023-10-06T05:00:38.832945+0000","last_peered":"2023-10-06T05:00:38.832945+0000","last_clean":"2023-10-06T05:00:38.832945+0000","last_became_active":"2023-10-06T05:00:38.829336+0000","last_became_peered":"2023-10-06T05:00:38.829336+0000","last_unstale":"2023-10-06T05:00:38.832945+0000","last_undegraded":"2023-10-06T05:00:38.832945+0000","last_fullsized":"2023-10-06T05:00:38.832945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:26.130218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.820268+0000","last_change":"2023-10-06T05:00:38.820268+0000","last_active":"2023-10-06T05:00:38.820268+0000","last_peered":"2023-10-06T05:00:38.820268+0000","last_clean":"2023-10-06T05:00:38.820268+0000","last_became_active":"2023-10-06T05:00:38.820042+0000","last_became_peered":"2023-10-06T05:00:38.820042+0000","last_unstale":"2023-10-06T05:00:38.820268+0000","last_undegraded":"2023-10-06T05:00:38.820268+0000","last_fullsized":"2023-10-06T05:00:38.820268+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:10.951487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835552+0000","last_change":"2023-10-06T05:00:38.834421+0000","last_active":"2023-10-06T05:00:38.835552+0000","last_peered":"2023-10-06T05:00:38.835552+0000","last_clean":"2023-10-06T05:00:38.835552+0000","last_became_active":"2023-10-06T05:00:38.834212+0000","last_became_peered":"2023-10-06T05:00:38.834212+0000","last_unstale":"2023-10-06T05:00:38.835552+0000","last_undegraded":"2023-10-06T05:00:38.835552+0000","last_fullsized":"2023-10-06T05:00:38.835552+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:48.142649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824354+0000","last_change":"2023-10-06T05:00:38.821599+0000","last_active":"2023-10-06T05:00:38.824354+0000","last_peered":"2023-10-06T05:00:38.824354+0000","last_clean":"2023-10-06T05:00:38.824354+0000","last_became_active":"2023-10-06T05:00:38.821323+0000","last_became_peered":"2023-10-06T05:00:38.821323+0000","last_unstale":"2023-10-06T05:00:38.824354+0000","last_undegraded":"2023-10-06T05:00:38.824354+0000","last_fullsized":"2023-10-06T05:00:38.824354+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:13.355262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824819+0000","last_change":"2023-10-06T05:00:38.824819+0000","last_active":"2023-10-06T05:00:38.824819+0000","last_peered":"2023-10-06T05:00:38.824819+0000","last_clean":"2023-10-06T05:00:38.824819+0000","last_became_active":"2023-10-06T05:00:38.824159+0000","last_became_peered":"2023-10-06T05:00:38.824159+0000","last_unstale":"2023-10-06T05:00:38.824819+0000","last_undegraded":"2023-10-06T05:00:38.824819+0000","last_fullsized":"2023-10-06T05:00:38.824819+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:04.585219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826887+0000","last_change":"2023-10-06T05:00:38.826887+0000","last_active":"2023-10-06T05:00:38.826887+0000","last_peered":"2023-10-06T05:00:38.826887+0000","last_clean":"2023-10-06T05:00:38.826887+0000","last_became_active":"2023-10-06T05:00:38.826597+0000","last_became_peered":"2023-10-06T05:00:38.826597+0000","last_unstale":"2023-10-06T05:00:38.826887+0000","last_undegraded":"2023-10-06T05:00:38.826887+0000","last_fullsized":"2023-10-06T05:00:38.826887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:11.497519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.829072+0000","last_change":"2023-10-06T05:00:38.829072+0000","last_active":"2023-10-06T05:00:38.829072+0000","last_peered":"2023-10-06T05:00:38.829072+0000","last_clean":"2023-10-06T05:00:38.829072+0000","last_became_active":"2023-10-06T05:00:38.828820+0000","last_became_peered":"2023-10-06T05:00:38.828820+0000","last_unstale":"2023-10-06T05:00:38.829072+0000","last_undegraded":"2023-10-06T05:00:38.829072+0000","last_fullsized":"2023-10-06T05:00:38.829072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:02.459001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.829615+0000","last_change":"2023-10-06T05:00:38.829615+0000","last_active":"2023-10-06T05:00:38.829615+0000","last_peered":"2023-10-06T05:00:38.829615+0000","last_clean":"2023-10-06T05:00:38.829615+0000","last_became_active":"2023-10-06T05:00:38.829339+0000","last_became_peered":"2023-10-06T05:00:38.829339+0000","last_unstale":"2023-10-06T05:00:38.829615+0000","last_undegraded":"2023-10-06T05:00:38.829615+0000","last_fullsized":"2023-10-06T05:00:38.829615+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:20.332378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825431+0000","last_change":"2023-10-06T05:00:38.825431+0000","last_active":"2023-10-06T05:00:38.825431+0000","last_peered":"2023-10-06T05:00:38.825431+0000","last_clean":"2023-10-06T05:00:38.825431+0000","last_became_active":"2023-10-06T05:00:38.825139+0000","last_became_peered":"2023-10-06T05:00:38.825139+0000","last_unstale":"2023-10-06T05:00:38.825431+0000","last_undegraded":"2023-10-06T05:00:38.825431+0000","last_fullsized":"2023-10-06T05:00:38.825431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:19.526639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.821941+0000","last_change":"2023-10-06T05:00:38.821941+0000","last_active":"2023-10-06T05:00:38.821941+0000","last_peered":"2023-10-06T05:00:38.821941+0000","last_clean":"2023-10-06T05:00:38.821941+0000","last_became_active":"2023-10-06T05:00:38.821627+0000","last_became_peered":"2023-10-06T05:00:38.821627+0000","last_unstale":"2023-10-06T05:00:38.821941+0000","last_undegraded":"2023-10-06T05:00:38.821941+0000","last_fullsized":"2023-10-06T05:00:38.821941+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:01.285988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836616+0000","last_change":"2023-10-06T05:00:38.836616+0000","last_active":"2023-10-06T05:00:38.836616+0000","last_peered":"2023-10-06T05:00:38.836616+0000","last_clean":"2023-10-06T05:00:38.836616+0000","last_became_active":"2023-10-06T05:00:38.836202+0000","last_became_peered":"2023-10-06T05:00:38.836202+0000","last_unstale":"2023-10-06T05:00:38.836616+0000","last_undegraded":"2023-10-06T05:00:38.836616+0000","last_fullsized":"2023-10-06T05:00:38.836616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:07.216936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837355+0000","last_change":"2023-10-06T05:00:38.837355+0000","last_active":"2023-10-06T05:00:38.837355+0000","last_peered":"2023-10-06T05:00:38.837355+0000","last_clean":"2023-10-06T05:00:38.837355+0000","last_became_active":"2023-10-06T05:00:38.835897+0000","last_became_peered":"2023-10-06T05:00:38.835897+0000","last_unstale":"2023-10-06T05:00:38.837355+0000","last_undegraded":"2023-10-06T05:00:38.837355+0000","last_fullsized":"2023-10-06T05:00:38.837355+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:46.088850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836357+0000","last_change":"2023-10-06T05:00:38.836357+0000","last_active":"2023-10-06T05:00:38.836357+0000","last_peered":"2023-10-06T05:00:38.836357+0000","last_clean":"2023-10-06T05:00:38.836357+0000","last_became_active":"2023-10-06T05:00:38.835298+0000","last_became_peered":"2023-10-06T05:00:38.835298+0000","last_unstale":"2023-10-06T05:00:38.836357+0000","last_undegraded":"2023-10-06T05:00:38.836357+0000","last_fullsized":"2023-10-06T05:00:38.836357+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:30.706310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832204+0000","last_change":"2023-10-06T05:00:38.825613+0000","last_active":"2023-10-06T05:00:38.832204+0000","last_peered":"2023-10-06T05:00:38.832204+0000","last_clean":"2023-10-06T05:00:38.832204+0000","last_became_active":"2023-10-06T05:00:38.825356+0000","last_became_peered":"2023-10-06T05:00:38.825356+0000","last_unstale":"2023-10-06T05:00:38.832204+0000","last_undegraded":"2023-10-06T05:00:38.832204+0000","last_fullsized":"2023-10-06T05:00:38.832204+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:35.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.820015+0000","last_change":"2023-10-06T05:00:38.820015+0000","last_active":"2023-10-06T05:00:38.820015+0000","last_peered":"2023-10-06T05:00:38.820015+0000","last_clean":"2023-10-06T05:00:38.820015+0000","last_became_active":"2023-10-06T05:00:38.819713+0000","last_became_peered":"2023-10-06T05:00:38.819713+0000","last_unstale":"2023-10-06T05:00:38.820015+0000","last_undegraded":"2023-10-06T05:00:38.820015+0000","last_fullsized":"2023-10-06T05:00:38.820015+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:42.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832627+0000","last_change":"2023-10-06T05:00:38.828283+0000","last_active":"2023-10-06T05:00:38.832627+0000","last_peered":"2023-10-06T05:00:38.832627+0000","last_clean":"2023-10-06T05:00:38.832627+0000","last_became_active":"2023-10-06T05:00:38.827962+0000","last_became_peered":"2023-10-06T05:00:38.827962+0000","last_unstale":"2023-10-06T05:00:38.832627+0000","last_undegraded":"2023-10-06T05:00:38.832627+0000","last_fullsized":"2023-10-06T05:00:38.832627+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:43.300647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831854+0000","last_change":"2023-10-06T05:00:38.831854+0000","last_active":"2023-10-06T05:00:38.831854+0000","last_peered":"2023-10-06T05:00:38.831854+0000","last_clean":"2023-10-06T05:00:38.831854+0000","last_became_active":"2023-10-06T05:00:38.831489+0000","last_became_peered":"2023-10-06T05:00:38.831489+0000","last_unstale":"2023-10-06T05:00:38.831854+0000","last_undegraded":"2023-10-06T05:00:38.831854+0000","last_fullsized":"2023-10-06T05:00:38.831854+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:51.040049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834433+0000","last_change":"2023-10-06T05:00:38.833438+0000","last_active":"2023-10-06T05:00:38.834433+0000","last_peered":"2023-10-06T05:00:38.834433+0000","last_clean":"2023-10-06T05:00:38.834433+0000","last_became_active":"2023-10-06T05:00:38.833147+0000","last_became_peered":"2023-10-06T05:00:38.833147+0000","last_unstale":"2023-10-06T05:00:38.834433+0000","last_undegraded":"2023-10-06T05:00:38.834433+0000","last_fullsized":"2023-10-06T05:00:38.834433+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:09.310003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831576+0000","last_change":"2023-10-06T05:00:38.829996+0000","last_active":"2023-10-06T05:00:38.831576+0000","last_peered":"2023-10-06T05:00:38.831576+0000","last_clean":"2023-10-06T05:00:38.831576+0000","last_became_active":"2023-10-06T05:00:38.829630+0000","last_became_peered":"2023-10-06T05:00:38.829630+0000","last_unstale":"2023-10-06T05:00:38.831576+0000","last_undegraded":"2023-10-06T05:00:38.831576+0000","last_fullsized":"2023-10-06T05:00:38.831576+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:57.708032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837567+0000","last_change":"2023-10-06T05:00:38.837567+0000","last_active":"2023-10-06T05:00:38.837567+0000","last_peered":"2023-10-06T05:00:38.837567+0000","last_clean":"2023-10-06T05:00:38.837567+0000","last_became_active":"2023-10-06T05:00:38.837165+0000","last_became_peered":"2023-10-06T05:00:38.837165+0000","last_unstale":"2023-10-06T05:00:38.837567+0000","last_undegraded":"2023-10-06T05:00:38.837567+0000","last_fullsized":"2023-10-06T05:00:38.837567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:18.883292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835617+0000","last_change":"2023-10-06T05:00:38.833175+0000","last_active":"2023-10-06T05:00:38.835617+0000","last_peered":"2023-10-06T05:00:38.835617+0000","last_clean":"2023-10-06T05:00:38.835617+0000","last_became_active":"2023-10-06T05:00:38.832927+0000","last_became_peered":"2023-10-06T05:00:38.832927+0000","last_unstale":"2023-10-06T05:00:38.835617+0000","last_undegraded":"2023-10-06T05:00:38.835617+0000","last_fullsized":"2023-10-06T05:00:38.835617+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:55.294468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826828+0000","last_change":"2023-10-06T05:00:38.821208+0000","last_active":"2023-10-06T05:00:38.826828+0000","last_peered":"2023-10-06T05:00:38.826828+0000","last_clean":"2023-10-06T05:00:38.826828+0000","last_became_active":"2023-10-06T05:00:38.820916+0000","last_became_peered":"2023-10-06T05:00:38.820916+0000","last_unstale":"2023-10-06T05:00:38.826828+0000","last_undegraded":"2023-10-06T05:00:38.826828+0000","last_fullsized":"2023-10-06T05:00:38.826828+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:49.254211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834768+0000","last_change":"2023-10-06T05:00:38.833563+0000","last_active":"2023-10-06T05:00:38.834768+0000","last_peered":"2023-10-06T05:00:38.834768+0000","last_clean":"2023-10-06T05:00:38.834768+0000","last_became_active":"2023-10-06T05:00:38.833009+0000","last_became_peered":"2023-10-06T05:00:38.833009+0000","last_unstale":"2023-10-06T05:00:38.834768+0000","last_undegraded":"2023-10-06T05:00:38.834768+0000","last_fullsized":"2023-10-06T05:00:38.834768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:36.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832853+0000","last_change":"2023-10-06T05:00:38.831045+0000","last_active":"2023-10-06T05:00:38.832853+0000","last_peered":"2023-10-06T05:00:38.832853+0000","last_clean":"2023-10-06T05:00:38.832853+0000","last_became_active":"2023-10-06T05:00:38.822224+0000","last_became_peered":"2023-10-06T05:00:38.822224+0000","last_unstale":"2023-10-06T05:00:38.832853+0000","last_undegraded":"2023-10-06T05:00:38.832853+0000","last_fullsized":"2023-10-06T05:00:38.832853+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:08.069022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832644+0000","last_change":"2023-10-06T05:00:38.831171+0000","last_active":"2023-10-06T05:00:38.832644+0000","last_peered":"2023-10-06T05:00:38.832644+0000","last_clean":"2023-10-06T05:00:38.832644+0000","last_became_active":"2023-10-06T05:00:38.830871+0000","last_became_peered":"2023-10-06T05:00:38.830871+0000","last_unstale":"2023-10-06T05:00:38.832644+0000","last_undegraded":"2023-10-06T05:00:38.832644+0000","last_fullsized":"2023-10-06T05:00:38.832644+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:55.935254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832963+0000","last_change":"2023-10-06T05:00:38.831028+0000","last_active":"2023-10-06T05:00:38.832963+0000","last_peered":"2023-10-06T05:00:38.832963+0000","last_clean":"2023-10-06T05:00:38.832963+0000","last_became_active":"2023-10-06T05:00:38.830765+0000","last_became_peered":"2023-10-06T05:00:38.830765+0000","last_unstale":"2023-10-06T05:00:38.832963+0000","last_undegraded":"2023-10-06T05:00:38.832963+0000","last_fullsized":"2023-10-06T05:00:38.832963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:19.526639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834817+0000","last_change":"2023-10-06T05:00:37.824624+0000","last_active":"2023-10-06T05:00:38.834817+0000","last_peered":"2023-10-06T05:00:38.834817+0000","last_clean":"2023-10-06T05:00:38.834817+0000","last_became_active":"2023-10-06T05:00:37.824294+0000","last_became_peered":"2023-10-06T05:00:37.824294+0000","last_unstale":"2023-10-06T05:00:38.834817+0000","last_undegraded":"2023-10-06T05:00:38.834817+0000","last_fullsized":"2023-10-06T05:00:38.834817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:19.326537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.819916+0000","last_change":"2023-10-06T05:00:38.819916+0000","last_active":"2023-10-06T05:00:38.819916+0000","last_peered":"2023-10-06T05:00:38.819916+0000","last_clean":"2023-10-06T05:00:38.819916+0000","last_became_active":"2023-10-06T05:00:38.819623+0000","last_became_peered":"2023-10-06T05:00:38.819623+0000","last_unstale":"2023-10-06T05:00:38.819916+0000","last_undegraded":"2023-10-06T05:00:38.819916+0000","last_fullsized":"2023-10-06T05:00:38.819916+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:17.692332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"119'1","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.814110+0000","last_change":"2023-10-06T05:00:37.822209+0000","last_active":"2023-10-06T05:00:38.814110+0000","last_peered":"2023-10-06T05:00:38.814110+0000","last_clean":"2023-10-06T05:00:38.814110+0000","last_became_active":"2023-10-06T05:00:37.821914+0000","last_became_peered":"2023-10-06T05:00:37.821914+0000","last_unstale":"2023-10-06T05:00:38.814110+0000","last_undegraded":"2023-10-06T05:00:38.814110+0000","last_fullsized":"2023-10-06T05:00:38.814110+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:03.591940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832500+0000","last_change":"2023-10-06T05:00:38.828319+0000","last_active":"2023-10-06T05:00:38.832500+0000","last_peered":"2023-10-06T05:00:38.832500+0000","last_clean":"2023-10-06T05:00:38.832500+0000","last_became_active":"2023-10-06T05:00:38.828029+0000","last_became_peered":"2023-10-06T05:00:38.828029+0000","last_unstale":"2023-10-06T05:00:38.832500+0000","last_undegraded":"2023-10-06T05:00:38.832500+0000","last_fullsized":"2023-10-06T05:00:38.832500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:31.918016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835960+0000","last_change":"2023-10-06T05:00:37.828303+0000","last_active":"2023-10-06T05:00:38.835960+0000","last_peered":"2023-10-06T05:00:38.835960+0000","last_clean":"2023-10-06T05:00:38.835960+0000","last_became_active":"2023-10-06T05:00:37.827998+0000","last_became_peered":"2023-10-06T05:00:37.827998+0000","last_unstale":"2023-10-06T05:00:38.835960+0000","last_undegraded":"2023-10-06T05:00:38.835960+0000","last_fullsized":"2023-10-06T05:00:38.835960+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:23.224800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827900+0000","last_change":"2023-10-06T05:00:38.827900+0000","last_active":"2023-10-06T05:00:38.827900+0000","last_peered":"2023-10-06T05:00:38.827900+0000","last_clean":"2023-10-06T05:00:38.827900+0000","last_became_active":"2023-10-06T05:00:38.827683+0000","last_became_peered":"2023-10-06T05:00:38.827683+0000","last_unstale":"2023-10-06T05:00:38.827900+0000","last_undegraded":"2023-10-06T05:00:38.827900+0000","last_fullsized":"2023-10-06T05:00:38.827900+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:48.609771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833306+0000","last_change":"2023-10-06T05:00:37.828214+0000","last_active":"2023-10-06T05:00:38.833306+0000","last_peered":"2023-10-06T05:00:38.833306+0000","last_clean":"2023-10-06T05:00:38.833306+0000","last_became_active":"2023-10-06T05:00:37.827756+0000","last_became_peered":"2023-10-06T05:00:37.827756+0000","last_unstale":"2023-10-06T05:00:38.833306+0000","last_undegraded":"2023-10-06T05:00:38.833306+0000","last_fullsized":"2023-10-06T05:00:38.833306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:03.506661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833242+0000","last_change":"2023-10-06T05:00:38.832073+0000","last_active":"2023-10-06T05:00:38.833242+0000","last_peered":"2023-10-06T05:00:38.833242+0000","last_clean":"2023-10-06T05:00:38.833242+0000","last_became_active":"2023-10-06T05:00:38.831786+0000","last_became_peered":"2023-10-06T05:00:38.831786+0000","last_unstale":"2023-10-06T05:00:38.833242+0000","last_undegraded":"2023-10-06T05:00:38.833242+0000","last_fullsized":"2023-10-06T05:00:38.833242+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:11.497519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833800+0000","last_change":"2023-10-06T05:00:37.824444+0000","last_active":"2023-10-06T05:00:38.833800+0000","last_peered":"2023-10-06T05:00:38.833800+0000","last_clean":"2023-10-06T05:00:38.833800+0000","last_became_active":"2023-10-06T05:00:37.824144+0000","last_became_peered":"2023-10-06T05:00:37.824144+0000","last_unstale":"2023-10-06T05:00:38.833800+0000","last_undegraded":"2023-10-06T05:00:38.833800+0000","last_fullsized":"2023-10-06T05:00:38.833800+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:17.632418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833176+0000","last_change":"2023-10-06T05:00:38.831780+0000","last_active":"2023-10-06T05:00:38.833176+0000","last_peered":"2023-10-06T05:00:38.833176+0000","last_clean":"2023-10-06T05:00:38.833176+0000","last_became_active":"2023-10-06T05:00:38.831536+0000","last_became_peered":"2023-10-06T05:00:38.831536+0000","last_unstale":"2023-10-06T05:00:38.833176+0000","last_undegraded":"2023-10-06T05:00:38.833176+0000","last_fullsized":"2023-10-06T05:00:38.833176+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:31.918016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"119'1","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812980+0000","last_change":"2023-10-06T05:00:37.822226+0000","last_active":"2023-10-06T05:00:38.812980+0000","last_peered":"2023-10-06T05:00:38.812980+0000","last_clean":"2023-10-06T05:00:38.812980+0000","last_became_active":"2023-10-06T05:00:37.821931+0000","last_became_peered":"2023-10-06T05:00:37.821931+0000","last_unstale":"2023-10-06T05:00:38.812980+0000","last_undegraded":"2023-10-06T05:00:38.812980+0000","last_fullsized":"2023-10-06T05:00:38.812980+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:18.345970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836493+0000","last_change":"2023-10-06T05:00:38.836493+0000","last_active":"2023-10-06T05:00:38.836493+0000","last_peered":"2023-10-06T05:00:38.836493+0000","last_clean":"2023-10-06T05:00:38.836493+0000","last_became_active":"2023-10-06T05:00:38.835950+0000","last_became_peered":"2023-10-06T05:00:38.835950+0000","last_unstale":"2023-10-06T05:00:38.836493+0000","last_undegraded":"2023-10-06T05:00:38.836493+0000","last_fullsized":"2023-10-06T05:00:38.836493+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:09.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.827976+0000","last_change":"2023-10-06T05:00:37.825234+0000","last_active":"2023-10-06T05:00:37.827976+0000","last_peered":"2023-10-06T05:00:37.827976+0000","last_clean":"2023-10-06T05:00:37.827976+0000","last_became_active":"2023-10-06T05:00:37.824944+0000","last_became_peered":"2023-10-06T05:00:37.824944+0000","last_unstale":"2023-10-06T05:00:37.827976+0000","last_undegraded":"2023-10-06T05:00:37.827976+0000","last_fullsized":"2023-10-06T05:00:37.827976+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:23.702952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836969+0000","last_change":"2023-10-06T05:00:38.836969+0000","last_active":"2023-10-06T05:00:38.836969+0000","last_peered":"2023-10-06T05:00:38.836969+0000","last_clean":"2023-10-06T05:00:38.836969+0000","last_became_active":"2023-10-06T05:00:38.836614+0000","last_became_peered":"2023-10-06T05:00:38.836614+0000","last_unstale":"2023-10-06T05:00:38.836969+0000","last_undegraded":"2023-10-06T05:00:38.836969+0000","last_fullsized":"2023-10-06T05:00:38.836969+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:43.932800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812220+0000","last_change":"2023-10-06T05:00:37.825045+0000","last_active":"2023-10-06T05:00:38.812220+0000","last_peered":"2023-10-06T05:00:38.812220+0000","last_clean":"2023-10-06T05:00:38.812220+0000","last_became_active":"2023-10-06T05:00:37.824754+0000","last_became_peered":"2023-10-06T05:00:37.824754+0000","last_unstale":"2023-10-06T05:00:38.812220+0000","last_undegraded":"2023-10-06T05:00:38.812220+0000","last_fullsized":"2023-10-06T05:00:38.812220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:03.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827028+0000","last_change":"2023-10-06T05:00:38.825299+0000","last_active":"2023-10-06T05:00:38.827028+0000","last_peered":"2023-10-06T05:00:38.827028+0000","last_clean":"2023-10-06T05:00:38.827028+0000","last_became_active":"2023-10-06T05:00:38.825036+0000","last_became_peered":"2023-10-06T05:00:38.825036+0000","last_unstale":"2023-10-06T05:00:38.827028+0000","last_undegraded":"2023-10-06T05:00:38.827028+0000","last_fullsized":"2023-10-06T05:00:38.827028+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:05.702451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825272+0000","last_change":"2023-10-06T05:00:37.821937+0000","last_active":"2023-10-06T05:00:38.825272+0000","last_peered":"2023-10-06T05:00:38.825272+0000","last_clean":"2023-10-06T05:00:38.825272+0000","last_became_active":"2023-10-06T05:00:37.821652+0000","last_became_peered":"2023-10-06T05:00:37.821652+0000","last_unstale":"2023-10-06T05:00:38.825272+0000","last_undegraded":"2023-10-06T05:00:38.825272+0000","last_fullsized":"2023-10-06T05:00:38.825272+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:58.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.838873+0000","last_change":"2023-10-06T05:00:38.838873+0000","last_active":"2023-10-06T05:00:38.838873+0000","last_peered":"2023-10-06T05:00:38.838873+0000","last_clean":"2023-10-06T05:00:38.838873+0000","last_became_active":"2023-10-06T05:00:38.838485+0000","last_became_peered":"2023-10-06T05:00:38.838485+0000","last_unstale":"2023-10-06T05:00:38.838873+0000","last_undegraded":"2023-10-06T05:00:38.838873+0000","last_fullsized":"2023-10-06T05:00:38.838873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:33.564757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825456+0000","last_change":"2023-10-06T05:00:37.824981+0000","last_active":"2023-10-06T05:00:38.825456+0000","last_peered":"2023-10-06T05:00:38.825456+0000","last_clean":"2023-10-06T05:00:38.825456+0000","last_became_active":"2023-10-06T05:00:37.824679+0000","last_became_peered":"2023-10-06T05:00:37.824679+0000","last_unstale":"2023-10-06T05:00:38.825456+0000","last_undegraded":"2023-10-06T05:00:38.825456+0000","last_fullsized":"2023-10-06T05:00:38.825456+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:44.060408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837945+0000","last_change":"2023-10-06T05:00:38.837945+0000","last_active":"2023-10-06T05:00:38.837945+0000","last_peered":"2023-10-06T05:00:38.837945+0000","last_clean":"2023-10-06T05:00:38.837945+0000","last_became_active":"2023-10-06T05:00:38.836603+0000","last_became_peered":"2023-10-06T05:00:38.836603+0000","last_unstale":"2023-10-06T05:00:38.837945+0000","last_undegraded":"2023-10-06T05:00:38.837945+0000","last_fullsized":"2023-10-06T05:00:38.837945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:43.324644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"119'1","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.814602+0000","last_change":"2023-10-06T05:00:37.822407+0000","last_active":"2023-10-06T05:00:38.814602+0000","last_peered":"2023-10-06T05:00:38.814602+0000","last_clean":"2023-10-06T05:00:38.814602+0000","last_became_active":"2023-10-06T05:00:37.822159+0000","last_became_peered":"2023-10-06T05:00:37.822159+0000","last_unstale":"2023-10-06T05:00:38.814602+0000","last_undegraded":"2023-10-06T05:00:38.814602+0000","last_fullsized":"2023-10-06T05:00:38.814602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:06.819261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831020+0000","last_change":"2023-10-06T05:00:38.828477+0000","last_active":"2023-10-06T05:00:38.831020+0000","last_peered":"2023-10-06T05:00:38.831020+0000","last_clean":"2023-10-06T05:00:38.831020+0000","last_became_active":"2023-10-06T05:00:38.828223+0000","last_became_peered":"2023-10-06T05:00:38.828223+0000","last_unstale":"2023-10-06T05:00:38.831020+0000","last_undegraded":"2023-10-06T05:00:38.831020+0000","last_fullsized":"2023-10-06T05:00:38.831020+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:30.706310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.826542+0000","last_change":"2023-10-06T05:00:37.824116+0000","last_active":"2023-10-06T05:00:37.826542+0000","last_peered":"2023-10-06T05:00:37.826542+0000","last_clean":"2023-10-06T05:00:37.826542+0000","last_became_active":"2023-10-06T05:00:37.823850+0000","last_became_peered":"2023-10-06T05:00:37.823850+0000","last_unstale":"2023-10-06T05:00:37.826542+0000","last_undegraded":"2023-10-06T05:00:37.826542+0000","last_fullsized":"2023-10-06T05:00:37.826542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:35.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827294+0000","last_change":"2023-10-06T05:00:38.827294+0000","last_active":"2023-10-06T05:00:38.827294+0000","last_peered":"2023-10-06T05:00:38.827294+0000","last_clean":"2023-10-06T05:00:38.827294+0000","last_became_active":"2023-10-06T05:00:38.827023+0000","last_became_peered":"2023-10-06T05:00:38.827023+0000","last_unstale":"2023-10-06T05:00:38.827294+0000","last_undegraded":"2023-10-06T05:00:38.827294+0000","last_fullsized":"2023-10-06T05:00:38.827294+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:14.205208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834262+0000","last_change":"2023-10-06T05:00:37.827647+0000","last_active":"2023-10-06T05:00:38.834262+0000","last_peered":"2023-10-06T05:00:38.834262+0000","last_clean":"2023-10-06T05:00:38.834262+0000","last_became_active":"2023-10-06T05:00:37.827345+0000","last_became_peered":"2023-10-06T05:00:37.827345+0000","last_unstale":"2023-10-06T05:00:38.834262+0000","last_undegraded":"2023-10-06T05:00:38.834262+0000","last_fullsized":"2023-10-06T05:00:38.834262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:30.972588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827382+0000","last_change":"2023-10-06T05:00:38.827382+0000","last_active":"2023-10-06T05:00:38.827382+0000","last_peered":"2023-10-06T05:00:38.827382+0000","last_clean":"2023-10-06T05:00:38.827382+0000","last_became_active":"2023-10-06T05:00:38.827127+0000","last_became_peered":"2023-10-06T05:00:38.827127+0000","last_unstale":"2023-10-06T05:00:38.827382+0000","last_undegraded":"2023-10-06T05:00:38.827382+0000","last_fullsized":"2023-10-06T05:00:38.827382+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:16.908090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.823315+0000","last_change":"2023-10-06T05:00:37.819650+0000","last_active":"2023-10-06T05:00:37.823315+0000","last_peered":"2023-10-06T05:00:37.823315+0000","last_clean":"2023-10-06T05:00:37.823315+0000","last_became_active":"2023-10-06T05:00:37.819374+0000","last_became_peered":"2023-10-06T05:00:37.819374+0000","last_unstale":"2023-10-06T05:00:37.823315+0000","last_undegraded":"2023-10-06T05:00:37.823315+0000","last_fullsized":"2023-10-06T05:00:37.823315+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:21.122839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.828029+0000","last_change":"2023-10-06T05:00:38.828029+0000","last_active":"2023-10-06T05:00:38.828029+0000","last_peered":"2023-10-06T05:00:38.828029+0000","last_clean":"2023-10-06T05:00:38.828029+0000","last_became_active":"2023-10-06T05:00:38.827777+0000","last_became_peered":"2023-10-06T05:00:38.827777+0000","last_unstale":"2023-10-06T05:00:38.828029+0000","last_undegraded":"2023-10-06T05:00:38.828029+0000","last_fullsized":"2023-10-06T05:00:38.828029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:48.142649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.830550+0000","last_change":"2023-10-06T05:00:37.828767+0000","last_active":"2023-10-06T05:00:37.830550+0000","last_peered":"2023-10-06T05:00:37.830550+0000","last_clean":"2023-10-06T05:00:37.830550+0000","last_became_active":"2023-10-06T05:00:37.827728+0000","last_became_peered":"2023-10-06T05:00:37.827728+0000","last_unstale":"2023-10-06T05:00:37.830550+0000","last_undegraded":"2023-10-06T05:00:37.830550+0000","last_fullsized":"2023-10-06T05:00:37.830550+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:50.034207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.829313+0000","last_change":"2023-10-06T05:00:38.825583+0000","last_active":"2023-10-06T05:00:38.829313+0000","last_peered":"2023-10-06T05:00:38.829313+0000","last_clean":"2023-10-06T05:00:38.829313+0000","last_became_active":"2023-10-06T05:00:38.825295+0000","last_became_peered":"2023-10-06T05:00:38.825295+0000","last_unstale":"2023-10-06T05:00:38.829313+0000","last_undegraded":"2023-10-06T05:00:38.829313+0000","last_fullsized":"2023-10-06T05:00:38.829313+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:54.806850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.827682+0000","last_change":"2023-10-06T05:00:37.824351+0000","last_active":"2023-10-06T05:00:37.827682+0000","last_peered":"2023-10-06T05:00:37.827682+0000","last_clean":"2023-10-06T05:00:37.827682+0000","last_became_active":"2023-10-06T05:00:37.823503+0000","last_became_peered":"2023-10-06T05:00:37.823503+0000","last_unstale":"2023-10-06T05:00:37.827682+0000","last_undegraded":"2023-10-06T05:00:37.827682+0000","last_fullsized":"2023-10-06T05:00:37.827682+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:41.167147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836606+0000","last_change":"2023-10-06T05:00:38.836606+0000","last_active":"2023-10-06T05:00:38.836606+0000","last_peered":"2023-10-06T05:00:38.836606+0000","last_clean":"2023-10-06T05:00:38.836606+0000","last_became_active":"2023-10-06T05:00:38.836411+0000","last_became_peered":"2023-10-06T05:00:38.836411+0000","last_unstale":"2023-10-06T05:00:38.836606+0000","last_undegraded":"2023-10-06T05:00:38.836606+0000","last_fullsized":"2023-10-06T05:00:38.836606+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:55.935254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.828304+0000","last_change":"2023-10-06T05:00:37.824564+0000","last_active":"2023-10-06T05:00:38.828304+0000","last_peered":"2023-10-06T05:00:38.828304+0000","last_clean":"2023-10-06T05:00:38.828304+0000","last_became_active":"2023-10-06T05:00:37.824261+0000","last_became_peered":"2023-10-06T05:00:37.824261+0000","last_unstale":"2023-10-06T05:00:38.828304+0000","last_undegraded":"2023-10-06T05:00:38.828304+0000","last_fullsized":"2023-10-06T05:00:38.828304+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:28.343985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.830834+0000","last_change":"2023-10-06T05:00:38.828608+0000","last_active":"2023-10-06T05:00:38.830834+0000","last_peered":"2023-10-06T05:00:38.830834+0000","last_clean":"2023-10-06T05:00:38.830834+0000","last_became_active":"2023-10-06T05:00:38.827802+0000","last_became_peered":"2023-10-06T05:00:38.827802+0000","last_unstale":"2023-10-06T05:00:38.830834+0000","last_undegraded":"2023-10-06T05:00:38.830834+0000","last_fullsized":"2023-10-06T05:00:38.830834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:01.708184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.830955+0000","last_change":"2023-10-06T05:00:37.821823+0000","last_active":"2023-10-06T05:00:38.830955+0000","last_peered":"2023-10-06T05:00:38.830955+0000","last_clean":"2023-10-06T05:00:38.830955+0000","last_became_active":"2023-10-06T05:00:37.821560+0000","last_became_peered":"2023-10-06T05:00:37.821560+0000","last_unstale":"2023-10-06T05:00:38.830955+0000","last_undegraded":"2023-10-06T05:00:38.830955+0000","last_fullsized":"2023-10-06T05:00:38.830955+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:05.196259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836743+0000","last_change":"2023-10-06T05:00:38.836743+0000","last_active":"2023-10-06T05:00:38.836743+0000","last_peered":"2023-10-06T05:00:38.836743+0000","last_clean":"2023-10-06T05:00:38.836743+0000","last_became_active":"2023-10-06T05:00:38.836207+0000","last_became_peered":"2023-10-06T05:00:38.836207+0000","last_unstale":"2023-10-06T05:00:38.836743+0000","last_undegraded":"2023-10-06T05:00:38.836743+0000","last_fullsized":"2023-10-06T05:00:38.836743+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:00.271426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.818240+0000","last_change":"2023-10-06T05:00:37.822392+0000","last_active":"2023-10-06T05:00:38.818240+0000","last_peered":"2023-10-06T05:00:38.818240+0000","last_clean":"2023-10-06T05:00:38.818240+0000","last_became_active":"2023-10-06T05:00:37.822087+0000","last_became_peered":"2023-10-06T05:00:37.822087+0000","last_unstale":"2023-10-06T05:00:38.818240+0000","last_undegraded":"2023-10-06T05:00:38.818240+0000","last_fullsized":"2023-10-06T05:00:38.818240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:18.281821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835140+0000","last_change":"2023-10-06T05:00:38.832121+0000","last_active":"2023-10-06T05:00:38.835140+0000","last_peered":"2023-10-06T05:00:38.835140+0000","last_clean":"2023-10-06T05:00:38.835140+0000","last_became_active":"2023-10-06T05:00:38.831830+0000","last_became_peered":"2023-10-06T05:00:38.831830+0000","last_unstale":"2023-10-06T05:00:38.835140+0000","last_undegraded":"2023-10-06T05:00:38.835140+0000","last_fullsized":"2023-10-06T05:00:38.835140+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:53.098939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.827902+0000","last_change":"2023-10-06T05:00:37.825614+0000","last_active":"2023-10-06T05:00:37.827902+0000","last_peered":"2023-10-06T05:00:37.827902+0000","last_clean":"2023-10-06T05:00:37.827902+0000","last_became_active":"2023-10-06T05:00:37.825274+0000","last_became_peered":"2023-10-06T05:00:37.825274+0000","last_unstale":"2023-10-06T05:00:37.827902+0000","last_undegraded":"2023-10-06T05:00:37.827902+0000","last_fullsized":"2023-10-06T05:00:37.827902+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:01.883453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826613+0000","last_change":"2023-10-06T05:00:38.826613+0000","last_active":"2023-10-06T05:00:38.826613+0000","last_peered":"2023-10-06T05:00:38.826613+0000","last_clean":"2023-10-06T05:00:38.826613+0000","last_became_active":"2023-10-06T05:00:38.826350+0000","last_became_peered":"2023-10-06T05:00:38.826350+0000","last_unstale":"2023-10-06T05:00:38.826613+0000","last_undegraded":"2023-10-06T05:00:38.826613+0000","last_fullsized":"2023-10-06T05:00:38.826613+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:00.189224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.814927+0000","last_change":"2023-10-06T05:00:37.818845+0000","last_active":"2023-10-06T05:00:38.814927+0000","last_peered":"2023-10-06T05:00:38.814927+0000","last_clean":"2023-10-06T05:00:38.814927+0000","last_became_active":"2023-10-06T05:00:37.818566+0000","last_became_peered":"2023-10-06T05:00:37.818566+0000","last_unstale":"2023-10-06T05:00:38.814927+0000","last_undegraded":"2023-10-06T05:00:38.814927+0000","last_fullsized":"2023-10-06T05:00:38.814927+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:04.696609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.821034+0000","last_change":"2023-10-06T05:00:38.821034+0000","last_active":"2023-10-06T05:00:38.821034+0000","last_peered":"2023-10-06T05:00:38.821034+0000","last_clean":"2023-10-06T05:00:38.821034+0000","last_became_active":"2023-10-06T05:00:38.820766+0000","last_became_peered":"2023-10-06T05:00:38.820766+0000","last_unstale":"2023-10-06T05:00:38.821034+0000","last_undegraded":"2023-10-06T05:00:38.821034+0000","last_fullsized":"2023-10-06T05:00:38.821034+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:35.409109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.830556+0000","last_change":"2023-10-06T05:00:37.828903+0000","last_active":"2023-10-06T05:00:37.830556+0000","last_peered":"2023-10-06T05:00:37.830556+0000","last_clean":"2023-10-06T05:00:37.830556+0000","last_became_active":"2023-10-06T05:00:37.828383+0000","last_became_peered":"2023-10-06T05:00:37.828383+0000","last_unstale":"2023-10-06T05:00:37.830556+0000","last_undegraded":"2023-10-06T05:00:37.830556+0000","last_fullsized":"2023-10-06T05:00:37.830556+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:59.183382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833341+0000","last_change":"2023-10-06T05:00:38.833341+0000","last_active":"2023-10-06T05:00:38.833341+0000","last_peered":"2023-10-06T05:00:38.833341+0000","last_clean":"2023-10-06T05:00:38.833341+0000","last_became_active":"2023-10-06T05:00:38.833026+0000","last_became_peered":"2023-10-06T05:00:38.833026+0000","last_unstale":"2023-10-06T05:00:38.833341+0000","last_undegraded":"2023-10-06T05:00:38.833341+0000","last_fullsized":"2023-10-06T05:00:38.833341+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:04.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.828243+0000","last_change":"2023-10-06T05:00:37.827775+0000","last_active":"2023-10-06T05:00:37.828243+0000","last_peered":"2023-10-06T05:00:37.828243+0000","last_clean":"2023-10-06T05:00:37.828243+0000","last_became_active":"2023-10-06T05:00:37.827412+0000","last_became_peered":"2023-10-06T05:00:37.827412+0000","last_unstale":"2023-10-06T05:00:37.828243+0000","last_undegraded":"2023-10-06T05:00:37.828243+0000","last_fullsized":"2023-10-06T05:00:37.828243+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:58.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837656+0000","last_change":"2023-10-06T05:00:38.837656+0000","last_active":"2023-10-06T05:00:38.837656+0000","last_peered":"2023-10-06T05:00:38.837656+0000","last_clean":"2023-10-06T05:00:38.837656+0000","last_became_active":"2023-10-06T05:00:38.836639+0000","last_became_peered":"2023-10-06T05:00:38.836639+0000","last_unstale":"2023-10-06T05:00:38.837656+0000","last_undegraded":"2023-10-06T05:00:38.837656+0000","last_fullsized":"2023-10-06T05:00:38.837656+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:22.707806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.826517+0000","last_change":"2023-10-06T05:00:37.824775+0000","last_active":"2023-10-06T05:00:37.826517+0000","last_peered":"2023-10-06T05:00:37.826517+0000","last_clean":"2023-10-06T05:00:37.826517+0000","last_became_active":"2023-10-06T05:00:37.824505+0000","last_became_peered":"2023-10-06T05:00:37.824505+0000","last_unstale":"2023-10-06T05:00:37.826517+0000","last_undegraded":"2023-10-06T05:00:37.826517+0000","last_fullsized":"2023-10-06T05:00:37.826517+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:05.083768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824684+0000","last_change":"2023-10-06T05:00:38.824684+0000","last_active":"2023-10-06T05:00:38.824684+0000","last_peered":"2023-10-06T05:00:38.824684+0000","last_clean":"2023-10-06T05:00:38.824684+0000","last_became_active":"2023-10-06T05:00:38.824244+0000","last_became_peered":"2023-10-06T05:00:38.824244+0000","last_unstale":"2023-10-06T05:00:38.824684+0000","last_undegraded":"2023-10-06T05:00:38.824684+0000","last_fullsized":"2023-10-06T05:00:38.824684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:45.066250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.816606+0000","last_change":"2023-10-06T05:00:37.824721+0000","last_active":"2023-10-06T05:00:38.816606+0000","last_peered":"2023-10-06T05:00:38.816606+0000","last_clean":"2023-10-06T05:00:38.816606+0000","last_became_active":"2023-10-06T05:00:37.824450+0000","last_became_peered":"2023-10-06T05:00:37.824450+0000","last_unstale":"2023-10-06T05:00:38.816606+0000","last_undegraded":"2023-10-06T05:00:38.816606+0000","last_fullsized":"2023-10-06T05:00:38.816606+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:47.079265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.818610+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:00:38.818610+0000","last_peered":"2023-10-06T05:00:38.818610+0000","last_clean":"2023-10-06T05:00:38.818610+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:00:38.818610+0000","last_undegraded":"2023-10-06T05:00:38.818610+0000","last_fullsized":"2023-10-06T05:00:38.818610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824825+0000","last_change":"2023-10-06T05:00:38.824825+0000","last_active":"2023-10-06T05:00:38.824825+0000","last_peered":"2023-10-06T05:00:38.824825+0000","last_clean":"2023-10-06T05:00:38.824825+0000","last_became_active":"2023-10-06T05:00:38.824549+0000","last_became_peered":"2023-10-06T05:00:38.824549+0000","last_unstale":"2023-10-06T05:00:38.824825+0000","last_undegraded":"2023-10-06T05:00:38.824825+0000","last_fullsized":"2023-10-06T05:00:38.824825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:44.948434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.823867+0000","last_change":"2023-10-06T05:00:37.821062+0000","last_active":"2023-10-06T05:00:37.823867+0000","last_peered":"2023-10-06T05:00:37.823867+0000","last_clean":"2023-10-06T05:00:37.823867+0000","last_became_active":"2023-10-06T05:00:37.820787+0000","last_became_peered":"2023-10-06T05:00:37.820787+0000","last_unstale":"2023-10-06T05:00:37.823867+0000","last_undegraded":"2023-10-06T05:00:37.823867+0000","last_fullsized":"2023-10-06T05:00:37.823867+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:59.171526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.817012+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:00:37.817012+0000","last_peered":"2023-10-06T05:00:37.817012+0000","last_clean":"2023-10-06T05:00:37.817012+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:00:37.817012+0000","last_undegraded":"2023-10-06T05:00:37.817012+0000","last_fullsized":"2023-10-06T05:00:37.817012+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836326+0000","last_change":"2023-10-06T05:00:38.836326+0000","last_active":"2023-10-06T05:00:38.836326+0000","last_peered":"2023-10-06T05:00:38.836326+0000","last_clean":"2023-10-06T05:00:38.836326+0000","last_became_active":"2023-10-06T05:00:38.835850+0000","last_became_peered":"2023-10-06T05:00:38.835850+0000","last_unstale":"2023-10-06T05:00:38.836326+0000","last_undegraded":"2023-10-06T05:00:38.836326+0000","last_fullsized":"2023-10-06T05:00:38.836326+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:16.829208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.811906+0000","last_change":"2023-10-06T05:00:37.826286+0000","last_active":"2023-10-06T05:00:38.811906+0000","last_peered":"2023-10-06T05:00:38.811906+0000","last_clean":"2023-10-06T05:00:38.811906+0000","last_became_active":"2023-10-06T05:00:37.825977+0000","last_became_peered":"2023-10-06T05:00:37.825977+0000","last_unstale":"2023-10-06T05:00:38.811906+0000","last_undegraded":"2023-10-06T05:00:38.811906+0000","last_fullsized":"2023-10-06T05:00:38.811906+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:49.876334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.812887+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:00:37.812887+0000","last_peered":"2023-10-06T05:00:37.812887+0000","last_clean":"2023-10-06T05:00:37.812887+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:00:37.812887+0000","last_undegraded":"2023-10-06T05:00:37.812887+0000","last_fullsized":"2023-10-06T05:00:37.812887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":242,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.819161+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:00:37.819161+0000","last_peered":"2023-10-06T05:00:37.819161+0000","last_clean":"2023-10-06T05:00:37.819161+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:00:37.819161+0000","last_undegraded":"2023-10-06T05:00:37.819161+0000","last_fullsized":"2023-10-06T05:00:37.819161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.822726+0000","last_change":"2023-10-06T05:00:38.822726+0000","last_active":"2023-10-06T05:00:38.822726+0000","last_peered":"2023-10-06T05:00:38.822726+0000","last_clean":"2023-10-06T05:00:38.822726+0000","last_became_active":"2023-10-06T05:00:38.822500+0000","last_became_peered":"2023-10-06T05:00:38.822500+0000","last_unstale":"2023-10-06T05:00:38.822726+0000","last_undegraded":"2023-10-06T05:00:38.822726+0000","last_fullsized":"2023-10-06T05:00:38.822726+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:22.707806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"119'8","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833808+0000","last_change":"2023-10-06T05:00:37.825037+0000","last_active":"2023-10-06T05:00:38.833808+0000","last_peered":"2023-10-06T05:00:38.833808+0000","last_clean":"2023-10-06T05:00:38.833808+0000","last_became_active":"2023-10-06T05:00:37.824634+0000","last_became_peered":"2023-10-06T05:00:37.824634+0000","last_unstale":"2023-10-06T05:00:38.833808+0000","last_undegraded":"2023-10-06T05:00:38.833808+0000","last_fullsized":"2023-10-06T05:00:38.833808+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:38.407633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":92,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.821735+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:00:37.821735+0000","last_peered":"2023-10-06T05:00:37.821735+0000","last_clean":"2023-10-06T05:00:37.821735+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:00:37.821735+0000","last_undegraded":"2023-10-06T05:00:37.821735+0000","last_fullsized":"2023-10-06T05:00:37.821735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827008+0000","last_change":"2023-10-06T05:00:38.824824+0000","last_active":"2023-10-06T05:00:38.827008+0000","last_peered":"2023-10-06T05:00:38.827008+0000","last_clean":"2023-10-06T05:00:38.827008+0000","last_became_active":"2023-10-06T05:00:38.824424+0000","last_became_peered":"2023-10-06T05:00:38.824424+0000","last_unstale":"2023-10-06T05:00:38.827008+0000","last_undegraded":"2023-10-06T05:00:38.827008+0000","last_fullsized":"2023-10-06T05:00:38.827008+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:51.075748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.830574+0000","last_change":"2023-10-06T05:00:37.829593+0000","last_active":"2023-10-06T05:00:37.830574+0000","last_peered":"2023-10-06T05:00:37.830574+0000","last_clean":"2023-10-06T05:00:37.830574+0000","last_became_active":"2023-10-06T05:00:37.829291+0000","last_became_peered":"2023-10-06T05:00:37.829291+0000","last_unstale":"2023-10-06T05:00:37.830574+0000","last_undegraded":"2023-10-06T05:00:37.830574+0000","last_fullsized":"2023-10-06T05:00:37.830574+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:11.848429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.819246+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:00:37.819246+0000","last_peered":"2023-10-06T05:00:37.819246+0000","last_clean":"2023-10-06T05:00:37.819246+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:00:37.819246+0000","last_undegraded":"2023-10-06T05:00:37.819246+0000","last_fullsized":"2023-10-06T05:00:37.819246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827214+0000","last_change":"2023-10-06T05:00:38.825431+0000","last_active":"2023-10-06T05:00:38.827214+0000","last_peered":"2023-10-06T05:00:38.827214+0000","last_clean":"2023-10-06T05:00:38.827214+0000","last_became_active":"2023-10-06T05:00:38.825147+0000","last_became_peered":"2023-10-06T05:00:38.825147+0000","last_unstale":"2023-10-06T05:00:38.827214+0000","last_undegraded":"2023-10-06T05:00:38.827214+0000","last_fullsized":"2023-10-06T05:00:38.827214+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:09.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825060+0000","last_change":"2023-10-06T05:00:37.821212+0000","last_active":"2023-10-06T05:00:38.825060+0000","last_peered":"2023-10-06T05:00:38.825060+0000","last_clean":"2023-10-06T05:00:38.825060+0000","last_became_active":"2023-10-06T05:00:37.820929+0000","last_became_peered":"2023-10-06T05:00:37.820929+0000","last_unstale":"2023-10-06T05:00:38.825060+0000","last_undegraded":"2023-10-06T05:00:38.825060+0000","last_fullsized":"2023-10-06T05:00:38.825060+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:39.259088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.816307+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:00:38.816307+0000","last_peered":"2023-10-06T05:00:38.816307+0000","last_clean":"2023-10-06T05:00:38.816307+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:00:38.816307+0000","last_undegraded":"2023-10-06T05:00:38.816307+0000","last_fullsized":"2023-10-06T05:00:38.816307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831091+0000","last_change":"2023-10-06T05:00:38.827789+0000","last_active":"2023-10-06T05:00:38.831091+0000","last_peered":"2023-10-06T05:00:38.831091+0000","last_clean":"2023-10-06T05:00:38.831091+0000","last_became_active":"2023-10-06T05:00:38.827488+0000","last_became_peered":"2023-10-06T05:00:38.827488+0000","last_unstale":"2023-10-06T05:00:38.831091+0000","last_undegraded":"2023-10-06T05:00:38.831091+0000","last_fullsized":"2023-10-06T05:00:38.831091+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:47.335919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"119'6","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.839100+0000","last_change":"2023-10-06T05:00:37.828413+0000","last_active":"2023-10-06T05:00:38.839100+0000","last_peered":"2023-10-06T05:00:38.839100+0000","last_clean":"2023-10-06T05:00:38.839100+0000","last_became_active":"2023-10-06T05:00:37.828035+0000","last_became_peered":"2023-10-06T05:00:37.828035+0000","last_unstale":"2023-10-06T05:00:38.839100+0000","last_undegraded":"2023-10-06T05:00:38.839100+0000","last_fullsized":"2023-10-06T05:00:38.839100+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:43.942592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812032+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:00:38.812032+0000","last_peered":"2023-10-06T05:00:38.812032+0000","last_clean":"2023-10-06T05:00:38.812032+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:00:38.812032+0000","last_undegraded":"2023-10-06T05:00:38.812032+0000","last_fullsized":"2023-10-06T05:00:38.812032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826591+0000","last_change":"2023-10-06T05:00:38.821172+0000","last_active":"2023-10-06T05:00:38.826591+0000","last_peered":"2023-10-06T05:00:38.826591+0000","last_clean":"2023-10-06T05:00:38.826591+0000","last_became_active":"2023-10-06T05:00:38.820895+0000","last_became_peered":"2023-10-06T05:00:38.820895+0000","last_unstale":"2023-10-06T05:00:38.826591+0000","last_undegraded":"2023-10-06T05:00:38.826591+0000","last_fullsized":"2023-10-06T05:00:38.826591+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:48.085106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"119'3","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836863+0000","last_change":"2023-10-06T05:00:37.828774+0000","last_active":"2023-10-06T05:00:38.836863+0000","last_peered":"2023-10-06T05:00:38.836863+0000","last_clean":"2023-10-06T05:00:38.836863+0000","last_became_active":"2023-10-06T05:00:37.828507+0000","last_became_peered":"2023-10-06T05:00:37.828507+0000","last_unstale":"2023-10-06T05:00:38.836863+0000","last_undegraded":"2023-10-06T05:00:38.836863+0000","last_fullsized":"2023-10-06T05:00:38.836863+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:09.945646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":94,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812144+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:00:38.812144+0000","last_peered":"2023-10-06T05:00:38.812144+0000","last_clean":"2023-10-06T05:00:38.812144+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:00:38.812144+0000","last_undegraded":"2023-10-06T05:00:38.812144+0000","last_fullsized":"2023-10-06T05:00:38.812144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307524,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064316,"statfs":{"total":96636764160,"available":96321859584,"internally_reserved":0,"allocated":2093056,"data_stored":1265161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.73899999999999999,"5min":0.86599999999999999,"15min":7.1779999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.73499999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.311},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":7.1879999999999997},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.073},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":1.1379999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.111},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.0880000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.874},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.214},"last":0.54900000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":681805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58799999999999997,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.73399999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63400000000000001,"15min":0.57499999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.21}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.65500000000000003,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":1.093},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.77900000000000003,"5min":1.234,"15min":1.234},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.63400000000000001,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70099999999999996,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.3120000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.755,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.238},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61199999999999999,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":1.175}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.68100000000000005,"5min":3.669,"15min":3.669},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.79300000000000004,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":1.147}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.274},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":1.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307512,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064328,"statfs":{"total":96636764160,"available":96321871872,"internally_reserved":0,"allocated":2080768,"data_stored":1263605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35599999999999998,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.216},"max":{"1min":0.46400000000000002,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.23499999999999999,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.56100000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":0.63800000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"last":0.316},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38700000000000001,"15min":0.40000000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.622,"5min":0.622,"15min":0.70399999999999996},"last":0.26300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42699999999999999,"15min":0.42599999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.433,"15min":0.433},"min":{"1min":0.313,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.57399999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.877,"15min":1.0780000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.434}]},{"osd":5,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.90300000000000002,"5min":0.998,"15min":0.998},"last":0.496},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.64800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39200000000000002},"max":{"1min":0.80900000000000005,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83199999999999996,"5min":0.96299999999999997,"15min":1.0900000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65100000000000002,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.44700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63900000000000001,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":0.87,"5min":0.98099999999999998,"15min":1.026},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.441},"max":{"1min":0.93700000000000006,"5min":1.022,"15min":1.0429999999999999},"last":0.56999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306940,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064900,"statfs":{"total":96636764160,"available":96322457600,"internally_reserved":0,"allocated":1495040,"data_stored":674576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":2,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.59199999999999997,"5min":0.628,"15min":0.628},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42999999999999999,"15min":0.38800000000000001},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46100000000000002,"15min":0.443},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.502}]},{"osd":4,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.98299999999999998},"last":0.629},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.85799999999999998,"15min":0.92300000000000004},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.91700000000000004},"last":0.749}]},{"osd":6,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.93400000000000005,"15min":1.0109999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.52200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.97599999999999998,"15min":1.0329999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":0.995},"last":0.88100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302844,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068996,"statfs":{"total":96636764160,"available":96326651904,"internally_reserved":0,"allocated":1495040,"data_stored":675108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.20300000000000001,"15min":0.186},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.048},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.39300000000000002},"min":{"1min":0.32300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.317}]},{"osd":1,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.216},"max":{"1min":0.52400000000000002,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.375},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.219},"max":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.27900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.36499999999999999},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.90200000000000002},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.622},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.2649999999999999},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.88100000000000001,"15min":0.93100000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":0.88300000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.82499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306948,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064892,"statfs":{"total":96636764160,"available":96322449408,"internally_reserved":0,"allocated":1503232,"data_stored":676126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56399999999999995,"5min":0.68200000000000005,"15min":2.008},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.53000000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":6.4370000000000003},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.224,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.67000000000000004,"15min":1.7649999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":2.1970000000000001},"last":0.33400000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42399999999999999,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.255},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":5.069},"last":0.315},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.441,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":2.246},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59499999999999997,"15min":0.76100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":14.646000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":14.606999999999999},"last":0.505}]},{"osd":5,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.81399999999999995,"5min":0.94699999999999995,"15min":2.2959999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":6.4009999999999998},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.73299999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.90100000000000002,"15min":5.0309999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":0.87,"5min":1.012,"15min":2.2749999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.75800000000000001},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.92600000000000005,"15min":6.367},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":5},"last":0.60499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":674520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.71199999999999997,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.73299999999999998},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.623}]},{"osd":1,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.76200000000000001},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.76700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.70999999999999996,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.64100000000000001},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.80000000000000004,"15min":5.415},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.64800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":4.202},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.622,"15min":0.70399999999999996},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.754,"5min":1.044,"15min":4.2130000000000001},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.497},"min":{"1min":0.27800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69799999999999995,"5min":0.73399999999999999,"15min":1.1279999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.76900000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.505},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78700000000000003,"5min":0.86199999999999999,"15min":0.872},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.49399999999999999,"5min":0.375,"15min":0.313},"max":{"1min":0.76300000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.54800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306944,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064896,"statfs":{"total":96636764160,"available":96322453504,"internally_reserved":0,"allocated":1499136,"data_stored":679830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.20499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":4.9039999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57499999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":6.7949999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.26200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.86599999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.501},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.86599999999999999},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.442}]},{"osd":4,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53800000000000003,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:00:42.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:00:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.085+0000 7fbf394da700 1 Processor -- start 2023-10-06T05:00:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.085+0000 7fbf394da700 1 -- start start 2023-10-06T05:00:43.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf394da700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbf3415c8f0 0x7fbf3415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf394da700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf340d0870 0x7fbf340b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf394da700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf340cf6c0 con 0x7fbf3415ccc0 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf394da700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf340cfb50 con 0x7fbf340d0870 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf394da700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3415e050 con 0x7fbf3415c8f0 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf337fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf340d0870 0x7fbf340b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf337fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf340d0870 0x7fbf340b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37020/0 (socket says 172.21.15.3:37020) 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf337fe700 1 -- 172.21.15.3:0/1113541959 learned_addr learned my addr 172.21.15.3:0/1113541959 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.086+0000 7fbf32ffd700 1 --2- 172.21.15.3:0/1113541959 >> v2:172.21.15.153:3300/0 conn(0x7fbf3415c8f0 0x7fbf3415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf337fe700 1 -- 172.21.15.3:0/1113541959 >> v2:172.21.15.153:3300/0 conn(0x7fbf3415c8f0 msgr2=0x7fbf3415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf337fe700 1 --2- 172.21.15.3:0/1113541959 >> v2:172.21.15.153:3300/0 conn(0x7fbf3415c8f0 0x7fbf3415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf337fe700 1 -- 172.21.15.3:0/1113541959 >> v1:172.21.15.153:6789/0 conn(0x7fbf3415ccc0 legacy=0x7fbf340d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf337fe700 1 -- 172.21.15.3:0/1113541959 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340ceab0 con 0x7fbf340d0870 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf337fe700 1 --2- 172.21.15.3:0/1113541959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf340d0870 0x7fbf340b3ba0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009f20 tx=0x7fbf24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f2571d6ea3ae6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf31ffb700 1 -- 172.21.15.3:0/1113541959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400e070 con 0x7fbf340d0870 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf31ffb700 1 -- 172.21.15.3:0/1113541959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2400ba40 con 0x7fbf340d0870 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf31ffb700 1 -- 172.21.15.3:0/1113541959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf240092e0 con 0x7fbf340d0870 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf394da700 1 -- 172.21.15.3:0/1113541959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf340d0870 msgr2=0x7fbf340b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.087+0000 7fbf394da700 1 --2- 172.21.15.3:0/1113541959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf340d0870 0x7fbf340b3ba0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009f20 tx=0x7fbf24009840 comp rx=0 tx=0).stop 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.088+0000 7fbf394da700 1 -- 172.21.15.3:0/1113541959 shutdown_connections 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.088+0000 7fbf394da700 1 --2- 172.21.15.3:0/1113541959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf340d0870 0x7fbf340b3ba0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.088+0000 7fbf394da700 1 --2- 172.21.15.3:0/1113541959 >> v2:172.21.15.153:3300/0 conn(0x7fbf3415c8f0 0x7fbf3415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.088+0000 7fbf394da700 1 -- 172.21.15.3:0/1113541959 >> 172.21.15.3:0/1113541959 conn(0x7fbf34070310 msgr2=0x7fbf34070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.088+0000 7fbf394da700 1 -- 172.21.15.3:0/1113541959 shutdown_connections 2023-10-06T05:00:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.088+0000 7fbf394da700 1 -- 172.21.15.3:0/1113541959 wait complete. 2023-10-06T05:00:43.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.089+0000 7fbf394da700 1 Processor -- start 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.089+0000 7fbf394da700 1 -- start start 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.089+0000 7fbf394da700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf340d0870 0x7fbf340b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf394da700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf3415c8f0 0x7fbf341500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf394da700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf3415ccc0 0x7fbf3414c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf394da700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3416cdc0 con 0x7fbf3415ccc0 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf394da700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34150600 con 0x7fbf340d0870 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf394da700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3414d400 con 0x7fbf3415c8f0 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf3415ccc0 0x7fbf3414c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf3415ccc0 0x7fbf3414c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37026/0 (socket says 172.21.15.3:37026) 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 -- 172.21.15.3:0/277903652 learned_addr learned my addr 172.21.15.3:0/277903652 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf32ffd700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf340d0870 0x7fbf340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 -- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf3415c8f0 msgr2=0x7fbf341500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf327fc700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf3415c8f0 0x7fbf341500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf3415c8f0 0x7fbf341500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 -- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf340d0870 msgr2=0x7fbf340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf340d0870 0x7fbf340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 -- 172.21.15.3:0/277903652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009ae0 con 0x7fbf3415ccc0 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf32ffd700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf340d0870 0x7fbf340b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:43.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf337fe700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf3415ccc0 0x7fbf3414c390 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbf24012000 tx=0x7fbf24018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf2affd700 1 -- 172.21.15.3:0/277903652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24008e40 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf2affd700 1 -- 172.21.15.3:0/277903652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24025460 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf2affd700 1 -- 172.21.15.3:0/277903652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24025800 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3416d8d0 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.090+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3416cfa0 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.091+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3416da60 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.092+0000 7fbf2affd700 1 -- 172.21.15.3:0/277903652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf24003040 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.092+0000 7fbf2affd700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf14066f30 0x7fbf140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.092+0000 7fbf2affd700 1 -- 172.21.15.3:0/277903652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbf24085d70 con 0x7fbf3415ccc0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.093+0000 7fbf32ffd700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf14066f30 0x7fbf140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.093+0000 7fbf32ffd700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf14066f30 0x7fbf140693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006fd0 tx=0x7fbf2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.097+0000 7fbf2affd700 1 -- 172.21.15.3:0/277903652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf24052e80 con 0x7fbf3415ccc0 2023-10-06T05:00:43.127 INFO:tasks.ceph.mds.d.smithi153.stderr:2023-10-06T05:00:43.126+0000 7fc93b6b6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:00:43.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.288+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf34071680 con 0x7fbf14066f30 2023-10-06T05:00:43.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.298+0000 7fbf2affd700 1 -- 172.21.15.3:0/277903652 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312087 (secure 0 0 0) 0x7fbf34071680 con 0x7fbf14066f30 2023-10-06T05:00:43.304 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:43.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf14066f30 msgr2=0x7fbf140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf14066f30 0x7fbf140693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006fd0 tx=0x7fbf2c008040 comp rx=0 tx=0).stop 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf3415ccc0 msgr2=0x7fbf3414c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf3415ccc0 0x7fbf3414c390 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbf24012000 tx=0x7fbf24018000 comp rx=0 tx=0).stop 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 shutdown_connections 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbf14066f30 0x7fbf140693e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf340d0870 0x7fbf340b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf3415ccc0 0x7fbf3414c390 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 --2- 172.21.15.3:0/277903652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf3415c8f0 0x7fbf341500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 >> 172.21.15.3:0/277903652 conn(0x7fbf34070310 msgr2=0x7fbf34070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 shutdown_connections 2023-10-06T05:00:43.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.301+0000 7fbf394da700 1 -- 172.21.15.3:0/277903652 wait complete. 2023-10-06T05:00:43.308 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:00:43.315 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2023-10-06T05:00:41.547296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2448500,"kb_used_data":12852,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752526220,"statfs":{"total":773094113280,"available":770586849280,"internally_reserved":0,"allocated":13160448,"data_stored":6590731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.748223"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825458+0000","last_change":"2023-10-06T05:00:38.825458+0000","last_active":"2023-10-06T05:00:38.825458+0000","last_peered":"2023-10-06T05:00:38.825458+0000","last_clean":"2023-10-06T05:00:38.825458+0000","last_became_active":"2023-10-06T05:00:38.825076+0000","last_became_peered":"2023-10-06T05:00:38.825076+0000","last_unstale":"2023-10-06T05:00:38.825458+0000","last_undegraded":"2023-10-06T05:00:38.825458+0000","last_fullsized":"2023-10-06T05:00:38.825458+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:50.882176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833277+0000","last_change":"2023-10-06T05:00:38.832340+0000","last_active":"2023-10-06T05:00:38.833277+0000","last_peered":"2023-10-06T05:00:38.833277+0000","last_clean":"2023-10-06T05:00:38.833277+0000","last_became_active":"2023-10-06T05:00:38.831953+0000","last_became_peered":"2023-10-06T05:00:38.831953+0000","last_unstale":"2023-10-06T05:00:38.833277+0000","last_undegraded":"2023-10-06T05:00:38.833277+0000","last_fullsized":"2023-10-06T05:00:38.833277+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:11.089925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826988+0000","last_change":"2023-10-06T05:00:38.824669+0000","last_active":"2023-10-06T05:00:38.826988+0000","last_peered":"2023-10-06T05:00:38.826988+0000","last_clean":"2023-10-06T05:00:38.826988+0000","last_became_active":"2023-10-06T05:00:38.824091+0000","last_became_peered":"2023-10-06T05:00:38.824091+0000","last_unstale":"2023-10-06T05:00:38.826988+0000","last_undegraded":"2023-10-06T05:00:38.826988+0000","last_fullsized":"2023-10-06T05:00:38.826988+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:42.119873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832445+0000","last_change":"2023-10-06T05:00:38.827469+0000","last_active":"2023-10-06T05:00:38.832445+0000","last_peered":"2023-10-06T05:00:38.832445+0000","last_clean":"2023-10-06T05:00:38.832445+0000","last_became_active":"2023-10-06T05:00:38.827215+0000","last_became_peered":"2023-10-06T05:00:38.827215+0000","last_unstale":"2023-10-06T05:00:38.832445+0000","last_undegraded":"2023-10-06T05:00:38.832445+0000","last_fullsized":"2023-10-06T05:00:38.832445+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:48.164637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835737+0000","last_change":"2023-10-06T05:00:38.835737+0000","last_active":"2023-10-06T05:00:38.835737+0000","last_peered":"2023-10-06T05:00:38.835737+0000","last_clean":"2023-10-06T05:00:38.835737+0000","last_became_active":"2023-10-06T05:00:38.835489+0000","last_became_peered":"2023-10-06T05:00:38.835489+0000","last_unstale":"2023-10-06T05:00:38.835737+0000","last_undegraded":"2023-10-06T05:00:38.835737+0000","last_fullsized":"2023-10-06T05:00:38.835737+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:50.530154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824040+0000","last_change":"2023-10-06T05:00:38.824040+0000","last_active":"2023-10-06T05:00:38.824040+0000","last_peered":"2023-10-06T05:00:38.824040+0000","last_clean":"2023-10-06T05:00:38.824040+0000","last_became_active":"2023-10-06T05:00:38.823687+0000","last_became_peered":"2023-10-06T05:00:38.823687+0000","last_unstale":"2023-10-06T05:00:38.824040+0000","last_undegraded":"2023-10-06T05:00:38.824040+0000","last_fullsized":"2023-10-06T05:00:38.824040+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:26.130218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837362+0000","last_change":"2023-10-06T05:00:38.837362+0000","last_active":"2023-10-06T05:00:38.837362+0000","last_peered":"2023-10-06T05:00:38.837362+0000","last_clean":"2023-10-06T05:00:38.837362+0000","last_became_active":"2023-10-06T05:00:38.836250+0000","last_became_peered":"2023-10-06T05:00:38.836250+0000","last_unstale":"2023-10-06T05:00:38.837362+0000","last_undegraded":"2023-10-06T05:00:38.837362+0000","last_fullsized":"2023-10-06T05:00:38.837362+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:05.697530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832945+0000","last_change":"2023-10-06T05:00:38.829619+0000","last_active":"2023-10-06T05:00:38.832945+0000","last_peered":"2023-10-06T05:00:38.832945+0000","last_clean":"2023-10-06T05:00:38.832945+0000","last_became_active":"2023-10-06T05:00:38.829336+0000","last_became_peered":"2023-10-06T05:00:38.829336+0000","last_unstale":"2023-10-06T05:00:38.832945+0000","last_undegraded":"2023-10-06T05:00:38.832945+0000","last_fullsized":"2023-10-06T05:00:38.832945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:26.130218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.820268+0000","last_change":"2023-10-06T05:00:38.820268+0000","last_active":"2023-10-06T05:00:38.820268+0000","last_peered":"2023-10-06T05:00:38.820268+0000","last_clean":"2023-10-06T05:00:38.820268+0000","last_became_active":"2023-10-06T05:00:38.820042+0000","last_became_peered":"2023-10-06T05:00:38.820042+0000","last_unstale":"2023-10-06T05:00:38.820268+0000","last_undegraded":"2023-10-06T05:00:38.820268+0000","last_fullsized":"2023-10-06T05:00:38.820268+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:10.951487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835552+0000","last_change":"2023-10-06T05:00:38.834421+0000","last_active":"2023-10-06T05:00:38.835552+0000","last_peered":"2023-10-06T05:00:38.835552+0000","last_clean":"2023-10-06T05:00:38.835552+0000","last_became_active":"2023-10-06T05:00:38.834212+0000","last_became_peered":"2023-10-06T05:00:38.834212+0000","last_unstale":"2023-10-06T05:00:38.835552+0000","last_undegraded":"2023-10-06T05:00:38.835552+0000","last_fullsized":"2023-10-06T05:00:38.835552+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:48.142649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824354+0000","last_change":"2023-10-06T05:00:38.821599+0000","last_active":"2023-10-06T05:00:38.824354+0000","last_peered":"2023-10-06T05:00:38.824354+0000","last_clean":"2023-10-06T05:00:38.824354+0000","last_became_active":"2023-10-06T05:00:38.821323+0000","last_became_peered":"2023-10-06T05:00:38.821323+0000","last_unstale":"2023-10-06T05:00:38.824354+0000","last_undegraded":"2023-10-06T05:00:38.824354+0000","last_fullsized":"2023-10-06T05:00:38.824354+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:13.355262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824819+0000","last_change":"2023-10-06T05:00:38.824819+0000","last_active":"2023-10-06T05:00:38.824819+0000","last_peered":"2023-10-06T05:00:38.824819+0000","last_clean":"2023-10-06T05:00:38.824819+0000","last_became_active":"2023-10-06T05:00:38.824159+0000","last_became_peered":"2023-10-06T05:00:38.824159+0000","last_unstale":"2023-10-06T05:00:38.824819+0000","last_undegraded":"2023-10-06T05:00:38.824819+0000","last_fullsized":"2023-10-06T05:00:38.824819+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:04.585219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826887+0000","last_change":"2023-10-06T05:00:38.826887+0000","last_active":"2023-10-06T05:00:38.826887+0000","last_peered":"2023-10-06T05:00:38.826887+0000","last_clean":"2023-10-06T05:00:38.826887+0000","last_became_active":"2023-10-06T05:00:38.826597+0000","last_became_peered":"2023-10-06T05:00:38.826597+0000","last_unstale":"2023-10-06T05:00:38.826887+0000","last_undegraded":"2023-10-06T05:00:38.826887+0000","last_fullsized":"2023-10-06T05:00:38.826887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:11.497519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.829072+0000","last_change":"2023-10-06T05:00:38.829072+0000","last_active":"2023-10-06T05:00:38.829072+0000","last_peered":"2023-10-06T05:00:38.829072+0000","last_clean":"2023-10-06T05:00:38.829072+0000","last_became_active":"2023-10-06T05:00:38.828820+0000","last_became_peered":"2023-10-06T05:00:38.828820+0000","last_unstale":"2023-10-06T05:00:38.829072+0000","last_undegraded":"2023-10-06T05:00:38.829072+0000","last_fullsized":"2023-10-06T05:00:38.829072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:02.459001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.829615+0000","last_change":"2023-10-06T05:00:38.829615+0000","last_active":"2023-10-06T05:00:38.829615+0000","last_peered":"2023-10-06T05:00:38.829615+0000","last_clean":"2023-10-06T05:00:38.829615+0000","last_became_active":"2023-10-06T05:00:38.829339+0000","last_became_peered":"2023-10-06T05:00:38.829339+0000","last_unstale":"2023-10-06T05:00:38.829615+0000","last_undegraded":"2023-10-06T05:00:38.829615+0000","last_fullsized":"2023-10-06T05:00:38.829615+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:20.332378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825431+0000","last_change":"2023-10-06T05:00:38.825431+0000","last_active":"2023-10-06T05:00:38.825431+0000","last_peered":"2023-10-06T05:00:38.825431+0000","last_clean":"2023-10-06T05:00:38.825431+0000","last_became_active":"2023-10-06T05:00:38.825139+0000","last_became_peered":"2023-10-06T05:00:38.825139+0000","last_unstale":"2023-10-06T05:00:38.825431+0000","last_undegraded":"2023-10-06T05:00:38.825431+0000","last_fullsized":"2023-10-06T05:00:38.825431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:19.526639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.821941+0000","last_change":"2023-10-06T05:00:38.821941+0000","last_active":"2023-10-06T05:00:38.821941+0000","last_peered":"2023-10-06T05:00:38.821941+0000","last_clean":"2023-10-06T05:00:38.821941+0000","last_became_active":"2023-10-06T05:00:38.821627+0000","last_became_peered":"2023-10-06T05:00:38.821627+0000","last_unstale":"2023-10-06T05:00:38.821941+0000","last_undegraded":"2023-10-06T05:00:38.821941+0000","last_fullsized":"2023-10-06T05:00:38.821941+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:01.285988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836616+0000","last_change":"2023-10-06T05:00:38.836616+0000","last_active":"2023-10-06T05:00:38.836616+0000","last_peered":"2023-10-06T05:00:38.836616+0000","last_clean":"2023-10-06T05:00:38.836616+0000","last_became_active":"2023-10-06T05:00:38.836202+0000","last_became_peered":"2023-10-06T05:00:38.836202+0000","last_unstale":"2023-10-06T05:00:38.836616+0000","last_undegraded":"2023-10-06T05:00:38.836616+0000","last_fullsized":"2023-10-06T05:00:38.836616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:07.216936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837355+0000","last_change":"2023-10-06T05:00:38.837355+0000","last_active":"2023-10-06T05:00:38.837355+0000","last_peered":"2023-10-06T05:00:38.837355+0000","last_clean":"2023-10-06T05:00:38.837355+0000","last_became_active":"2023-10-06T05:00:38.835897+0000","last_became_peered":"2023-10-06T05:00:38.835897+0000","last_unstale":"2023-10-06T05:00:38.837355+0000","last_undegraded":"2023-10-06T05:00:38.837355+0000","last_fullsized":"2023-10-06T05:00:38.837355+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:46.088850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836357+0000","last_change":"2023-10-06T05:00:38.836357+0000","last_active":"2023-10-06T05:00:38.836357+0000","last_peered":"2023-10-06T05:00:38.836357+0000","last_clean":"2023-10-06T05:00:38.836357+0000","last_became_active":"2023-10-06T05:00:38.835298+0000","last_became_peered":"2023-10-06T05:00:38.835298+0000","last_unstale":"2023-10-06T05:00:38.836357+0000","last_undegraded":"2023-10-06T05:00:38.836357+0000","last_fullsized":"2023-10-06T05:00:38.836357+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:30.706310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832204+0000","last_change":"2023-10-06T05:00:38.825613+0000","last_active":"2023-10-06T05:00:38.832204+0000","last_peered":"2023-10-06T05:00:38.832204+0000","last_clean":"2023-10-06T05:00:38.832204+0000","last_became_active":"2023-10-06T05:00:38.825356+0000","last_became_peered":"2023-10-06T05:00:38.825356+0000","last_unstale":"2023-10-06T05:00:38.832204+0000","last_undegraded":"2023-10-06T05:00:38.832204+0000","last_fullsized":"2023-10-06T05:00:38.832204+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:35.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.820015+0000","last_change":"2023-10-06T05:00:38.820015+0000","last_active":"2023-10-06T05:00:38.820015+0000","last_peered":"2023-10-06T05:00:38.820015+0000","last_clean":"2023-10-06T05:00:38.820015+0000","last_became_active":"2023-10-06T05:00:38.819713+0000","last_became_peered":"2023-10-06T05:00:38.819713+0000","last_unstale":"2023-10-06T05:00:38.820015+0000","last_undegraded":"2023-10-06T05:00:38.820015+0000","last_fullsized":"2023-10-06T05:00:38.820015+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:42.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832627+0000","last_change":"2023-10-06T05:00:38.828283+0000","last_active":"2023-10-06T05:00:38.832627+0000","last_peered":"2023-10-06T05:00:38.832627+0000","last_clean":"2023-10-06T05:00:38.832627+0000","last_became_active":"2023-10-06T05:00:38.827962+0000","last_became_peered":"2023-10-06T05:00:38.827962+0000","last_unstale":"2023-10-06T05:00:38.832627+0000","last_undegraded":"2023-10-06T05:00:38.832627+0000","last_fullsized":"2023-10-06T05:00:38.832627+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:43.300647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831854+0000","last_change":"2023-10-06T05:00:38.831854+0000","last_active":"2023-10-06T05:00:38.831854+0000","last_peered":"2023-10-06T05:00:38.831854+0000","last_clean":"2023-10-06T05:00:38.831854+0000","last_became_active":"2023-10-06T05:00:38.831489+0000","last_became_peered":"2023-10-06T05:00:38.831489+0000","last_unstale":"2023-10-06T05:00:38.831854+0000","last_undegraded":"2023-10-06T05:00:38.831854+0000","last_fullsized":"2023-10-06T05:00:38.831854+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:51.040049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834433+0000","last_change":"2023-10-06T05:00:38.833438+0000","last_active":"2023-10-06T05:00:38.834433+0000","last_peered":"2023-10-06T05:00:38.834433+0000","last_clean":"2023-10-06T05:00:38.834433+0000","last_became_active":"2023-10-06T05:00:38.833147+0000","last_became_peered":"2023-10-06T05:00:38.833147+0000","last_unstale":"2023-10-06T05:00:38.834433+0000","last_undegraded":"2023-10-06T05:00:38.834433+0000","last_fullsized":"2023-10-06T05:00:38.834433+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:09.310003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831576+0000","last_change":"2023-10-06T05:00:38.829996+0000","last_active":"2023-10-06T05:00:38.831576+0000","last_peered":"2023-10-06T05:00:38.831576+0000","last_clean":"2023-10-06T05:00:38.831576+0000","last_became_active":"2023-10-06T05:00:38.829630+0000","last_became_peered":"2023-10-06T05:00:38.829630+0000","last_unstale":"2023-10-06T05:00:38.831576+0000","last_undegraded":"2023-10-06T05:00:38.831576+0000","last_fullsized":"2023-10-06T05:00:38.831576+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:57.708032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837567+0000","last_change":"2023-10-06T05:00:38.837567+0000","last_active":"2023-10-06T05:00:38.837567+0000","last_peered":"2023-10-06T05:00:38.837567+0000","last_clean":"2023-10-06T05:00:38.837567+0000","last_became_active":"2023-10-06T05:00:38.837165+0000","last_became_peered":"2023-10-06T05:00:38.837165+0000","last_unstale":"2023-10-06T05:00:38.837567+0000","last_undegraded":"2023-10-06T05:00:38.837567+0000","last_fullsized":"2023-10-06T05:00:38.837567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:18.883292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835617+0000","last_change":"2023-10-06T05:00:38.833175+0000","last_active":"2023-10-06T05:00:38.835617+0000","last_peered":"2023-10-06T05:00:38.835617+0000","last_clean":"2023-10-06T05:00:38.835617+0000","last_became_active":"2023-10-06T05:00:38.832927+0000","last_became_peered":"2023-10-06T05:00:38.832927+0000","last_unstale":"2023-10-06T05:00:38.835617+0000","last_undegraded":"2023-10-06T05:00:38.835617+0000","last_fullsized":"2023-10-06T05:00:38.835617+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:55.294468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826828+0000","last_change":"2023-10-06T05:00:38.821208+0000","last_active":"2023-10-06T05:00:38.826828+0000","last_peered":"2023-10-06T05:00:38.826828+0000","last_clean":"2023-10-06T05:00:38.826828+0000","last_became_active":"2023-10-06T05:00:38.820916+0000","last_became_peered":"2023-10-06T05:00:38.820916+0000","last_unstale":"2023-10-06T05:00:38.826828+0000","last_undegraded":"2023-10-06T05:00:38.826828+0000","last_fullsized":"2023-10-06T05:00:38.826828+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:49.254211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834768+0000","last_change":"2023-10-06T05:00:38.833563+0000","last_active":"2023-10-06T05:00:38.834768+0000","last_peered":"2023-10-06T05:00:38.834768+0000","last_clean":"2023-10-06T05:00:38.834768+0000","last_became_active":"2023-10-06T05:00:38.833009+0000","last_became_peered":"2023-10-06T05:00:38.833009+0000","last_unstale":"2023-10-06T05:00:38.834768+0000","last_undegraded":"2023-10-06T05:00:38.834768+0000","last_fullsized":"2023-10-06T05:00:38.834768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:36.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832853+0000","last_change":"2023-10-06T05:00:38.831045+0000","last_active":"2023-10-06T05:00:38.832853+0000","last_peered":"2023-10-06T05:00:38.832853+0000","last_clean":"2023-10-06T05:00:38.832853+0000","last_became_active":"2023-10-06T05:00:38.822224+0000","last_became_peered":"2023-10-06T05:00:38.822224+0000","last_unstale":"2023-10-06T05:00:38.832853+0000","last_undegraded":"2023-10-06T05:00:38.832853+0000","last_fullsized":"2023-10-06T05:00:38.832853+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:08.069022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832644+0000","last_change":"2023-10-06T05:00:38.831171+0000","last_active":"2023-10-06T05:00:38.832644+0000","last_peered":"2023-10-06T05:00:38.832644+0000","last_clean":"2023-10-06T05:00:38.832644+0000","last_became_active":"2023-10-06T05:00:38.830871+0000","last_became_peered":"2023-10-06T05:00:38.830871+0000","last_unstale":"2023-10-06T05:00:38.832644+0000","last_undegraded":"2023-10-06T05:00:38.832644+0000","last_fullsized":"2023-10-06T05:00:38.832644+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:55.935254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832963+0000","last_change":"2023-10-06T05:00:38.831028+0000","last_active":"2023-10-06T05:00:38.832963+0000","last_peered":"2023-10-06T05:00:38.832963+0000","last_clean":"2023-10-06T05:00:38.832963+0000","last_became_active":"2023-10-06T05:00:38.830765+0000","last_became_peered":"2023-10-06T05:00:38.830765+0000","last_unstale":"2023-10-06T05:00:38.832963+0000","last_undegraded":"2023-10-06T05:00:38.832963+0000","last_fullsized":"2023-10-06T05:00:38.832963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:19.526639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834817+0000","last_change":"2023-10-06T05:00:37.824624+0000","last_active":"2023-10-06T05:00:38.834817+0000","last_peered":"2023-10-06T05:00:38.834817+0000","last_clean":"2023-10-06T05:00:38.834817+0000","last_became_active":"2023-10-06T05:00:37.824294+0000","last_became_peered":"2023-10-06T05:00:37.824294+0000","last_unstale":"2023-10-06T05:00:38.834817+0000","last_undegraded":"2023-10-06T05:00:38.834817+0000","last_fullsized":"2023-10-06T05:00:38.834817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:19.326537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.819916+0000","last_change":"2023-10-06T05:00:38.819916+0000","last_active":"2023-10-06T05:00:38.819916+0000","last_peered":"2023-10-06T05:00:38.819916+0000","last_clean":"2023-10-06T05:00:38.819916+0000","last_became_active":"2023-10-06T05:00:38.819623+0000","last_became_peered":"2023-10-06T05:00:38.819623+0000","last_unstale":"2023-10-06T05:00:38.819916+0000","last_undegraded":"2023-10-06T05:00:38.819916+0000","last_fullsized":"2023-10-06T05:00:38.819916+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:17.692332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"119'1","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.814110+0000","last_change":"2023-10-06T05:00:37.822209+0000","last_active":"2023-10-06T05:00:38.814110+0000","last_peered":"2023-10-06T05:00:38.814110+0000","last_clean":"2023-10-06T05:00:38.814110+0000","last_became_active":"2023-10-06T05:00:37.821914+0000","last_became_peered":"2023-10-06T05:00:37.821914+0000","last_unstale":"2023-10-06T05:00:38.814110+0000","last_undegraded":"2023-10-06T05:00:38.814110+0000","last_fullsized":"2023-10-06T05:00:38.814110+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:03.591940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.832500+0000","last_change":"2023-10-06T05:00:38.828319+0000","last_active":"2023-10-06T05:00:38.832500+0000","last_peered":"2023-10-06T05:00:38.832500+0000","last_clean":"2023-10-06T05:00:38.832500+0000","last_became_active":"2023-10-06T05:00:38.828029+0000","last_became_peered":"2023-10-06T05:00:38.828029+0000","last_unstale":"2023-10-06T05:00:38.832500+0000","last_undegraded":"2023-10-06T05:00:38.832500+0000","last_fullsized":"2023-10-06T05:00:38.832500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:31.918016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835960+0000","last_change":"2023-10-06T05:00:37.828303+0000","last_active":"2023-10-06T05:00:38.835960+0000","last_peered":"2023-10-06T05:00:38.835960+0000","last_clean":"2023-10-06T05:00:38.835960+0000","last_became_active":"2023-10-06T05:00:37.827998+0000","last_became_peered":"2023-10-06T05:00:37.827998+0000","last_unstale":"2023-10-06T05:00:38.835960+0000","last_undegraded":"2023-10-06T05:00:38.835960+0000","last_fullsized":"2023-10-06T05:00:38.835960+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:23.224800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827900+0000","last_change":"2023-10-06T05:00:38.827900+0000","last_active":"2023-10-06T05:00:38.827900+0000","last_peered":"2023-10-06T05:00:38.827900+0000","last_clean":"2023-10-06T05:00:38.827900+0000","last_became_active":"2023-10-06T05:00:38.827683+0000","last_became_peered":"2023-10-06T05:00:38.827683+0000","last_unstale":"2023-10-06T05:00:38.827900+0000","last_undegraded":"2023-10-06T05:00:38.827900+0000","last_fullsized":"2023-10-06T05:00:38.827900+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:48.609771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833306+0000","last_change":"2023-10-06T05:00:37.828214+0000","last_active":"2023-10-06T05:00:38.833306+0000","last_peered":"2023-10-06T05:00:38.833306+0000","last_clean":"2023-10-06T05:00:38.833306+0000","last_became_active":"2023-10-06T05:00:37.827756+0000","last_became_peered":"2023-10-06T05:00:37.827756+0000","last_unstale":"2023-10-06T05:00:38.833306+0000","last_undegraded":"2023-10-06T05:00:38.833306+0000","last_fullsized":"2023-10-06T05:00:38.833306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:03.506661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833242+0000","last_change":"2023-10-06T05:00:38.832073+0000","last_active":"2023-10-06T05:00:38.833242+0000","last_peered":"2023-10-06T05:00:38.833242+0000","last_clean":"2023-10-06T05:00:38.833242+0000","last_became_active":"2023-10-06T05:00:38.831786+0000","last_became_peered":"2023-10-06T05:00:38.831786+0000","last_unstale":"2023-10-06T05:00:38.833242+0000","last_undegraded":"2023-10-06T05:00:38.833242+0000","last_fullsized":"2023-10-06T05:00:38.833242+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:11.497519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833800+0000","last_change":"2023-10-06T05:00:37.824444+0000","last_active":"2023-10-06T05:00:38.833800+0000","last_peered":"2023-10-06T05:00:38.833800+0000","last_clean":"2023-10-06T05:00:38.833800+0000","last_became_active":"2023-10-06T05:00:37.824144+0000","last_became_peered":"2023-10-06T05:00:37.824144+0000","last_unstale":"2023-10-06T05:00:38.833800+0000","last_undegraded":"2023-10-06T05:00:38.833800+0000","last_fullsized":"2023-10-06T05:00:38.833800+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:17.632418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833176+0000","last_change":"2023-10-06T05:00:38.831780+0000","last_active":"2023-10-06T05:00:38.833176+0000","last_peered":"2023-10-06T05:00:38.833176+0000","last_clean":"2023-10-06T05:00:38.833176+0000","last_became_active":"2023-10-06T05:00:38.831536+0000","last_became_peered":"2023-10-06T05:00:38.831536+0000","last_unstale":"2023-10-06T05:00:38.833176+0000","last_undegraded":"2023-10-06T05:00:38.833176+0000","last_fullsized":"2023-10-06T05:00:38.833176+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:31.918016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"119'1","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812980+0000","last_change":"2023-10-06T05:00:37.822226+0000","last_active":"2023-10-06T05:00:38.812980+0000","last_peered":"2023-10-06T05:00:38.812980+0000","last_clean":"2023-10-06T05:00:38.812980+0000","last_became_active":"2023-10-06T05:00:37.821931+0000","last_became_peered":"2023-10-06T05:00:37.821931+0000","last_unstale":"2023-10-06T05:00:38.812980+0000","last_undegraded":"2023-10-06T05:00:38.812980+0000","last_fullsized":"2023-10-06T05:00:38.812980+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:18.345970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836493+0000","last_change":"2023-10-06T05:00:38.836493+0000","last_active":"2023-10-06T05:00:38.836493+0000","last_peered":"2023-10-06T05:00:38.836493+0000","last_clean":"2023-10-06T05:00:38.836493+0000","last_became_active":"2023-10-06T05:00:38.835950+0000","last_became_peered":"2023-10-06T05:00:38.835950+0000","last_unstale":"2023-10-06T05:00:38.836493+0000","last_undegraded":"2023-10-06T05:00:38.836493+0000","last_fullsized":"2023-10-06T05:00:38.836493+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:09.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.827976+0000","last_change":"2023-10-06T05:00:37.825234+0000","last_active":"2023-10-06T05:00:37.827976+0000","last_peered":"2023-10-06T05:00:37.827976+0000","last_clean":"2023-10-06T05:00:37.827976+0000","last_became_active":"2023-10-06T05:00:37.824944+0000","last_became_peered":"2023-10-06T05:00:37.824944+0000","last_unstale":"2023-10-06T05:00:37.827976+0000","last_undegraded":"2023-10-06T05:00:37.827976+0000","last_fullsized":"2023-10-06T05:00:37.827976+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:23.702952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836969+0000","last_change":"2023-10-06T05:00:38.836969+0000","last_active":"2023-10-06T05:00:38.836969+0000","last_peered":"2023-10-06T05:00:38.836969+0000","last_clean":"2023-10-06T05:00:38.836969+0000","last_became_active":"2023-10-06T05:00:38.836614+0000","last_became_peered":"2023-10-06T05:00:38.836614+0000","last_unstale":"2023-10-06T05:00:38.836969+0000","last_undegraded":"2023-10-06T05:00:38.836969+0000","last_fullsized":"2023-10-06T05:00:38.836969+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:43.932800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812220+0000","last_change":"2023-10-06T05:00:37.825045+0000","last_active":"2023-10-06T05:00:38.812220+0000","last_peered":"2023-10-06T05:00:38.812220+0000","last_clean":"2023-10-06T05:00:38.812220+0000","last_became_active":"2023-10-06T05:00:37.824754+0000","last_became_peered":"2023-10-06T05:00:37.824754+0000","last_unstale":"2023-10-06T05:00:38.812220+0000","last_undegraded":"2023-10-06T05:00:38.812220+0000","last_fullsized":"2023-10-06T05:00:38.812220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:03.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827028+0000","last_change":"2023-10-06T05:00:38.825299+0000","last_active":"2023-10-06T05:00:38.827028+0000","last_peered":"2023-10-06T05:00:38.827028+0000","last_clean":"2023-10-06T05:00:38.827028+0000","last_became_active":"2023-10-06T05:00:38.825036+0000","last_became_peered":"2023-10-06T05:00:38.825036+0000","last_unstale":"2023-10-06T05:00:38.827028+0000","last_undegraded":"2023-10-06T05:00:38.827028+0000","last_fullsized":"2023-10-06T05:00:38.827028+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:05.702451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825272+0000","last_change":"2023-10-06T05:00:37.821937+0000","last_active":"2023-10-06T05:00:38.825272+0000","last_peered":"2023-10-06T05:00:38.825272+0000","last_clean":"2023-10-06T05:00:38.825272+0000","last_became_active":"2023-10-06T05:00:37.821652+0000","last_became_peered":"2023-10-06T05:00:37.821652+0000","last_unstale":"2023-10-06T05:00:38.825272+0000","last_undegraded":"2023-10-06T05:00:38.825272+0000","last_fullsized":"2023-10-06T05:00:38.825272+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:58.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.838873+0000","last_change":"2023-10-06T05:00:38.838873+0000","last_active":"2023-10-06T05:00:38.838873+0000","last_peered":"2023-10-06T05:00:38.838873+0000","last_clean":"2023-10-06T05:00:38.838873+0000","last_became_active":"2023-10-06T05:00:38.838485+0000","last_became_peered":"2023-10-06T05:00:38.838485+0000","last_unstale":"2023-10-06T05:00:38.838873+0000","last_undegraded":"2023-10-06T05:00:38.838873+0000","last_fullsized":"2023-10-06T05:00:38.838873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:33.564757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825456+0000","last_change":"2023-10-06T05:00:37.824981+0000","last_active":"2023-10-06T05:00:38.825456+0000","last_peered":"2023-10-06T05:00:38.825456+0000","last_clean":"2023-10-06T05:00:38.825456+0000","last_became_active":"2023-10-06T05:00:37.824679+0000","last_became_peered":"2023-10-06T05:00:37.824679+0000","last_unstale":"2023-10-06T05:00:38.825456+0000","last_undegraded":"2023-10-06T05:00:38.825456+0000","last_fullsized":"2023-10-06T05:00:38.825456+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:44.060408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837945+0000","last_change":"2023-10-06T05:00:38.837945+0000","last_active":"2023-10-06T05:00:38.837945+0000","last_peered":"2023-10-06T05:00:38.837945+0000","last_clean":"2023-10-06T05:00:38.837945+0000","last_became_active":"2023-10-06T05:00:38.836603+0000","last_became_peered":"2023-10-06T05:00:38.836603+0000","last_unstale":"2023-10-06T05:00:38.837945+0000","last_undegraded":"2023-10-06T05:00:38.837945+0000","last_fullsized":"2023-10-06T05:00:38.837945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:43.324644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"119'1","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.814602+0000","last_change":"2023-10-06T05:00:37.822407+0000","last_active":"2023-10-06T05:00:38.814602+0000","last_peered":"2023-10-06T05:00:38.814602+0000","last_clean":"2023-10-06T05:00:38.814602+0000","last_became_active":"2023-10-06T05:00:37.822159+0000","last_became_peered":"2023-10-06T05:00:37.822159+0000","last_unstale":"2023-10-06T05:00:38.814602+0000","last_undegraded":"2023-10-06T05:00:38.814602+0000","last_fullsized":"2023-10-06T05:00:38.814602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:06.819261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831020+0000","last_change":"2023-10-06T05:00:38.828477+0000","last_active":"2023-10-06T05:00:38.831020+0000","last_peered":"2023-10-06T05:00:38.831020+0000","last_clean":"2023-10-06T05:00:38.831020+0000","last_became_active":"2023-10-06T05:00:38.828223+0000","last_became_peered":"2023-10-06T05:00:38.828223+0000","last_unstale":"2023-10-06T05:00:38.831020+0000","last_undegraded":"2023-10-06T05:00:38.831020+0000","last_fullsized":"2023-10-06T05:00:38.831020+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:30.706310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.826542+0000","last_change":"2023-10-06T05:00:37.824116+0000","last_active":"2023-10-06T05:00:37.826542+0000","last_peered":"2023-10-06T05:00:37.826542+0000","last_clean":"2023-10-06T05:00:37.826542+0000","last_became_active":"2023-10-06T05:00:37.823850+0000","last_became_peered":"2023-10-06T05:00:37.823850+0000","last_unstale":"2023-10-06T05:00:37.826542+0000","last_undegraded":"2023-10-06T05:00:37.826542+0000","last_fullsized":"2023-10-06T05:00:37.826542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:35.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827294+0000","last_change":"2023-10-06T05:00:38.827294+0000","last_active":"2023-10-06T05:00:38.827294+0000","last_peered":"2023-10-06T05:00:38.827294+0000","last_clean":"2023-10-06T05:00:38.827294+0000","last_became_active":"2023-10-06T05:00:38.827023+0000","last_became_peered":"2023-10-06T05:00:38.827023+0000","last_unstale":"2023-10-06T05:00:38.827294+0000","last_undegraded":"2023-10-06T05:00:38.827294+0000","last_fullsized":"2023-10-06T05:00:38.827294+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:14.205208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.834262+0000","last_change":"2023-10-06T05:00:37.827647+0000","last_active":"2023-10-06T05:00:38.834262+0000","last_peered":"2023-10-06T05:00:38.834262+0000","last_clean":"2023-10-06T05:00:38.834262+0000","last_became_active":"2023-10-06T05:00:37.827345+0000","last_became_peered":"2023-10-06T05:00:37.827345+0000","last_unstale":"2023-10-06T05:00:38.834262+0000","last_undegraded":"2023-10-06T05:00:38.834262+0000","last_fullsized":"2023-10-06T05:00:38.834262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:30.972588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827382+0000","last_change":"2023-10-06T05:00:38.827382+0000","last_active":"2023-10-06T05:00:38.827382+0000","last_peered":"2023-10-06T05:00:38.827382+0000","last_clean":"2023-10-06T05:00:38.827382+0000","last_became_active":"2023-10-06T05:00:38.827127+0000","last_became_peered":"2023-10-06T05:00:38.827127+0000","last_unstale":"2023-10-06T05:00:38.827382+0000","last_undegraded":"2023-10-06T05:00:38.827382+0000","last_fullsized":"2023-10-06T05:00:38.827382+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:16.908090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.823315+0000","last_change":"2023-10-06T05:00:37.819650+0000","last_active":"2023-10-06T05:00:37.823315+0000","last_peered":"2023-10-06T05:00:37.823315+0000","last_clean":"2023-10-06T05:00:37.823315+0000","last_became_active":"2023-10-06T05:00:37.819374+0000","last_became_peered":"2023-10-06T05:00:37.819374+0000","last_unstale":"2023-10-06T05:00:37.823315+0000","last_undegraded":"2023-10-06T05:00:37.823315+0000","last_fullsized":"2023-10-06T05:00:37.823315+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:21.122839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.828029+0000","last_change":"2023-10-06T05:00:38.828029+0000","last_active":"2023-10-06T05:00:38.828029+0000","last_peered":"2023-10-06T05:00:38.828029+0000","last_clean":"2023-10-06T05:00:38.828029+0000","last_became_active":"2023-10-06T05:00:38.827777+0000","last_became_peered":"2023-10-06T05:00:38.827777+0000","last_unstale":"2023-10-06T05:00:38.828029+0000","last_undegraded":"2023-10-06T05:00:38.828029+0000","last_fullsized":"2023-10-06T05:00:38.828029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:48.142649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.830550+0000","last_change":"2023-10-06T05:00:37.828767+0000","last_active":"2023-10-06T05:00:37.830550+0000","last_peered":"2023-10-06T05:00:37.830550+0000","last_clean":"2023-10-06T05:00:37.830550+0000","last_became_active":"2023-10-06T05:00:37.827728+0000","last_became_peered":"2023-10-06T05:00:37.827728+0000","last_unstale":"2023-10-06T05:00:37.830550+0000","last_undegraded":"2023-10-06T05:00:37.830550+0000","last_fullsized":"2023-10-06T05:00:37.830550+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:50.034207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.829313+0000","last_change":"2023-10-06T05:00:38.825583+0000","last_active":"2023-10-06T05:00:38.829313+0000","last_peered":"2023-10-06T05:00:38.829313+0000","last_clean":"2023-10-06T05:00:38.829313+0000","last_became_active":"2023-10-06T05:00:38.825295+0000","last_became_peered":"2023-10-06T05:00:38.825295+0000","last_unstale":"2023-10-06T05:00:38.829313+0000","last_undegraded":"2023-10-06T05:00:38.829313+0000","last_fullsized":"2023-10-06T05:00:38.829313+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:54.806850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.827682+0000","last_change":"2023-10-06T05:00:37.824351+0000","last_active":"2023-10-06T05:00:37.827682+0000","last_peered":"2023-10-06T05:00:37.827682+0000","last_clean":"2023-10-06T05:00:37.827682+0000","last_became_active":"2023-10-06T05:00:37.823503+0000","last_became_peered":"2023-10-06T05:00:37.823503+0000","last_unstale":"2023-10-06T05:00:37.827682+0000","last_undegraded":"2023-10-06T05:00:37.827682+0000","last_fullsized":"2023-10-06T05:00:37.827682+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:41.167147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836606+0000","last_change":"2023-10-06T05:00:38.836606+0000","last_active":"2023-10-06T05:00:38.836606+0000","last_peered":"2023-10-06T05:00:38.836606+0000","last_clean":"2023-10-06T05:00:38.836606+0000","last_became_active":"2023-10-06T05:00:38.836411+0000","last_became_peered":"2023-10-06T05:00:38.836411+0000","last_unstale":"2023-10-06T05:00:38.836606+0000","last_undegraded":"2023-10-06T05:00:38.836606+0000","last_fullsized":"2023-10-06T05:00:38.836606+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:55.935254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.828304+0000","last_change":"2023-10-06T05:00:37.824564+0000","last_active":"2023-10-06T05:00:38.828304+0000","last_peered":"2023-10-06T05:00:38.828304+0000","last_clean":"2023-10-06T05:00:38.828304+0000","last_became_active":"2023-10-06T05:00:37.824261+0000","last_became_peered":"2023-10-06T05:00:37.824261+0000","last_unstale":"2023-10-06T05:00:38.828304+0000","last_undegraded":"2023-10-06T05:00:38.828304+0000","last_fullsized":"2023-10-06T05:00:38.828304+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:28.343985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.830834+0000","last_change":"2023-10-06T05:00:38.828608+0000","last_active":"2023-10-06T05:00:38.830834+0000","last_peered":"2023-10-06T05:00:38.830834+0000","last_clean":"2023-10-06T05:00:38.830834+0000","last_became_active":"2023-10-06T05:00:38.827802+0000","last_became_peered":"2023-10-06T05:00:38.827802+0000","last_unstale":"2023-10-06T05:00:38.830834+0000","last_undegraded":"2023-10-06T05:00:38.830834+0000","last_fullsized":"2023-10-06T05:00:38.830834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:01.708184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.830955+0000","last_change":"2023-10-06T05:00:37.821823+0000","last_active":"2023-10-06T05:00:38.830955+0000","last_peered":"2023-10-06T05:00:38.830955+0000","last_clean":"2023-10-06T05:00:38.830955+0000","last_became_active":"2023-10-06T05:00:37.821560+0000","last_became_peered":"2023-10-06T05:00:37.821560+0000","last_unstale":"2023-10-06T05:00:38.830955+0000","last_undegraded":"2023-10-06T05:00:38.830955+0000","last_fullsized":"2023-10-06T05:00:38.830955+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:05.196259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836743+0000","last_change":"2023-10-06T05:00:38.836743+0000","last_active":"2023-10-06T05:00:38.836743+0000","last_peered":"2023-10-06T05:00:38.836743+0000","last_clean":"2023-10-06T05:00:38.836743+0000","last_became_active":"2023-10-06T05:00:38.836207+0000","last_became_peered":"2023-10-06T05:00:38.836207+0000","last_unstale":"2023-10-06T05:00:38.836743+0000","last_undegraded":"2023-10-06T05:00:38.836743+0000","last_fullsized":"2023-10-06T05:00:38.836743+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:00.271426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.818240+0000","last_change":"2023-10-06T05:00:37.822392+0000","last_active":"2023-10-06T05:00:38.818240+0000","last_peered":"2023-10-06T05:00:38.818240+0000","last_clean":"2023-10-06T05:00:38.818240+0000","last_became_active":"2023-10-06T05:00:37.822087+0000","last_became_peered":"2023-10-06T05:00:37.822087+0000","last_unstale":"2023-10-06T05:00:38.818240+0000","last_undegraded":"2023-10-06T05:00:38.818240+0000","last_fullsized":"2023-10-06T05:00:38.818240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:18.281821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.835140+0000","last_change":"2023-10-06T05:00:38.832121+0000","last_active":"2023-10-06T05:00:38.835140+0000","last_peered":"2023-10-06T05:00:38.835140+0000","last_clean":"2023-10-06T05:00:38.835140+0000","last_became_active":"2023-10-06T05:00:38.831830+0000","last_became_peered":"2023-10-06T05:00:38.831830+0000","last_unstale":"2023-10-06T05:00:38.835140+0000","last_undegraded":"2023-10-06T05:00:38.835140+0000","last_fullsized":"2023-10-06T05:00:38.835140+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:53.098939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.827902+0000","last_change":"2023-10-06T05:00:37.825614+0000","last_active":"2023-10-06T05:00:37.827902+0000","last_peered":"2023-10-06T05:00:37.827902+0000","last_clean":"2023-10-06T05:00:37.827902+0000","last_became_active":"2023-10-06T05:00:37.825274+0000","last_became_peered":"2023-10-06T05:00:37.825274+0000","last_unstale":"2023-10-06T05:00:37.827902+0000","last_undegraded":"2023-10-06T05:00:37.827902+0000","last_fullsized":"2023-10-06T05:00:37.827902+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:01.883453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826613+0000","last_change":"2023-10-06T05:00:38.826613+0000","last_active":"2023-10-06T05:00:38.826613+0000","last_peered":"2023-10-06T05:00:38.826613+0000","last_clean":"2023-10-06T05:00:38.826613+0000","last_became_active":"2023-10-06T05:00:38.826350+0000","last_became_peered":"2023-10-06T05:00:38.826350+0000","last_unstale":"2023-10-06T05:00:38.826613+0000","last_undegraded":"2023-10-06T05:00:38.826613+0000","last_fullsized":"2023-10-06T05:00:38.826613+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:00.189224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.814927+0000","last_change":"2023-10-06T05:00:37.818845+0000","last_active":"2023-10-06T05:00:38.814927+0000","last_peered":"2023-10-06T05:00:38.814927+0000","last_clean":"2023-10-06T05:00:38.814927+0000","last_became_active":"2023-10-06T05:00:37.818566+0000","last_became_peered":"2023-10-06T05:00:37.818566+0000","last_unstale":"2023-10-06T05:00:38.814927+0000","last_undegraded":"2023-10-06T05:00:38.814927+0000","last_fullsized":"2023-10-06T05:00:38.814927+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:04.696609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.821034+0000","last_change":"2023-10-06T05:00:38.821034+0000","last_active":"2023-10-06T05:00:38.821034+0000","last_peered":"2023-10-06T05:00:38.821034+0000","last_clean":"2023-10-06T05:00:38.821034+0000","last_became_active":"2023-10-06T05:00:38.820766+0000","last_became_peered":"2023-10-06T05:00:38.820766+0000","last_unstale":"2023-10-06T05:00:38.821034+0000","last_undegraded":"2023-10-06T05:00:38.821034+0000","last_fullsized":"2023-10-06T05:00:38.821034+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:35.409109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.830556+0000","last_change":"2023-10-06T05:00:37.828903+0000","last_active":"2023-10-06T05:00:37.830556+0000","last_peered":"2023-10-06T05:00:37.830556+0000","last_clean":"2023-10-06T05:00:37.830556+0000","last_became_active":"2023-10-06T05:00:37.828383+0000","last_became_peered":"2023-10-06T05:00:37.828383+0000","last_unstale":"2023-10-06T05:00:37.830556+0000","last_undegraded":"2023-10-06T05:00:37.830556+0000","last_fullsized":"2023-10-06T05:00:37.830556+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:59.183382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833341+0000","last_change":"2023-10-06T05:00:38.833341+0000","last_active":"2023-10-06T05:00:38.833341+0000","last_peered":"2023-10-06T05:00:38.833341+0000","last_clean":"2023-10-06T05:00:38.833341+0000","last_became_active":"2023-10-06T05:00:38.833026+0000","last_became_peered":"2023-10-06T05:00:38.833026+0000","last_unstale":"2023-10-06T05:00:38.833341+0000","last_undegraded":"2023-10-06T05:00:38.833341+0000","last_fullsized":"2023-10-06T05:00:38.833341+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:04.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.828243+0000","last_change":"2023-10-06T05:00:37.827775+0000","last_active":"2023-10-06T05:00:37.828243+0000","last_peered":"2023-10-06T05:00:37.828243+0000","last_clean":"2023-10-06T05:00:37.828243+0000","last_became_active":"2023-10-06T05:00:37.827412+0000","last_became_peered":"2023-10-06T05:00:37.827412+0000","last_unstale":"2023-10-06T05:00:37.828243+0000","last_undegraded":"2023-10-06T05:00:37.828243+0000","last_fullsized":"2023-10-06T05:00:37.828243+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:58.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.837656+0000","last_change":"2023-10-06T05:00:38.837656+0000","last_active":"2023-10-06T05:00:38.837656+0000","last_peered":"2023-10-06T05:00:38.837656+0000","last_clean":"2023-10-06T05:00:38.837656+0000","last_became_active":"2023-10-06T05:00:38.836639+0000","last_became_peered":"2023-10-06T05:00:38.836639+0000","last_unstale":"2023-10-06T05:00:38.837656+0000","last_undegraded":"2023-10-06T05:00:38.837656+0000","last_fullsized":"2023-10-06T05:00:38.837656+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:22.707806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.826517+0000","last_change":"2023-10-06T05:00:37.824775+0000","last_active":"2023-10-06T05:00:37.826517+0000","last_peered":"2023-10-06T05:00:37.826517+0000","last_clean":"2023-10-06T05:00:37.826517+0000","last_became_active":"2023-10-06T05:00:37.824505+0000","last_became_peered":"2023-10-06T05:00:37.824505+0000","last_unstale":"2023-10-06T05:00:37.826517+0000","last_undegraded":"2023-10-06T05:00:37.826517+0000","last_fullsized":"2023-10-06T05:00:37.826517+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:05.083768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824684+0000","last_change":"2023-10-06T05:00:38.824684+0000","last_active":"2023-10-06T05:00:38.824684+0000","last_peered":"2023-10-06T05:00:38.824684+0000","last_clean":"2023-10-06T05:00:38.824684+0000","last_became_active":"2023-10-06T05:00:38.824244+0000","last_became_peered":"2023-10-06T05:00:38.824244+0000","last_unstale":"2023-10-06T05:00:38.824684+0000","last_undegraded":"2023-10-06T05:00:38.824684+0000","last_fullsized":"2023-10-06T05:00:38.824684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:45.066250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.816606+0000","last_change":"2023-10-06T05:00:37.824721+0000","last_active":"2023-10-06T05:00:38.816606+0000","last_peered":"2023-10-06T05:00:38.816606+0000","last_clean":"2023-10-06T05:00:38.816606+0000","last_became_active":"2023-10-06T05:00:37.824450+0000","last_became_peered":"2023-10-06T05:00:37.824450+0000","last_unstale":"2023-10-06T05:00:38.816606+0000","last_undegraded":"2023-10-06T05:00:38.816606+0000","last_fullsized":"2023-10-06T05:00:38.816606+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:47.079265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.818610+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:00:38.818610+0000","last_peered":"2023-10-06T05:00:38.818610+0000","last_clean":"2023-10-06T05:00:38.818610+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:00:38.818610+0000","last_undegraded":"2023-10-06T05:00:38.818610+0000","last_fullsized":"2023-10-06T05:00:38.818610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.824825+0000","last_change":"2023-10-06T05:00:38.824825+0000","last_active":"2023-10-06T05:00:38.824825+0000","last_peered":"2023-10-06T05:00:38.824825+0000","last_clean":"2023-10-06T05:00:38.824825+0000","last_became_active":"2023-10-06T05:00:38.824549+0000","last_became_peered":"2023-10-06T05:00:38.824549+0000","last_unstale":"2023-10-06T05:00:38.824825+0000","last_undegraded":"2023-10-06T05:00:38.824825+0000","last_fullsized":"2023-10-06T05:00:38.824825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:44.948434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.823867+0000","last_change":"2023-10-06T05:00:37.821062+0000","last_active":"2023-10-06T05:00:37.823867+0000","last_peered":"2023-10-06T05:00:37.823867+0000","last_clean":"2023-10-06T05:00:37.823867+0000","last_became_active":"2023-10-06T05:00:37.820787+0000","last_became_peered":"2023-10-06T05:00:37.820787+0000","last_unstale":"2023-10-06T05:00:37.823867+0000","last_undegraded":"2023-10-06T05:00:37.823867+0000","last_fullsized":"2023-10-06T05:00:37.823867+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:59.171526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.817012+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:00:37.817012+0000","last_peered":"2023-10-06T05:00:37.817012+0000","last_clean":"2023-10-06T05:00:37.817012+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:00:37.817012+0000","last_undegraded":"2023-10-06T05:00:37.817012+0000","last_fullsized":"2023-10-06T05:00:37.817012+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836326+0000","last_change":"2023-10-06T05:00:38.836326+0000","last_active":"2023-10-06T05:00:38.836326+0000","last_peered":"2023-10-06T05:00:38.836326+0000","last_clean":"2023-10-06T05:00:38.836326+0000","last_became_active":"2023-10-06T05:00:38.835850+0000","last_became_peered":"2023-10-06T05:00:38.835850+0000","last_unstale":"2023-10-06T05:00:38.836326+0000","last_undegraded":"2023-10-06T05:00:38.836326+0000","last_fullsized":"2023-10-06T05:00:38.836326+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:16.829208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.811906+0000","last_change":"2023-10-06T05:00:37.826286+0000","last_active":"2023-10-06T05:00:38.811906+0000","last_peered":"2023-10-06T05:00:38.811906+0000","last_clean":"2023-10-06T05:00:38.811906+0000","last_became_active":"2023-10-06T05:00:37.825977+0000","last_became_peered":"2023-10-06T05:00:37.825977+0000","last_unstale":"2023-10-06T05:00:38.811906+0000","last_undegraded":"2023-10-06T05:00:38.811906+0000","last_fullsized":"2023-10-06T05:00:38.811906+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:49.876334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.812887+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:00:37.812887+0000","last_peered":"2023-10-06T05:00:37.812887+0000","last_clean":"2023-10-06T05:00:37.812887+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:00:37.812887+0000","last_undegraded":"2023-10-06T05:00:37.812887+0000","last_fullsized":"2023-10-06T05:00:37.812887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":242,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.819161+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:00:37.819161+0000","last_peered":"2023-10-06T05:00:37.819161+0000","last_clean":"2023-10-06T05:00:37.819161+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:00:37.819161+0000","last_undegraded":"2023-10-06T05:00:37.819161+0000","last_fullsized":"2023-10-06T05:00:37.819161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.822726+0000","last_change":"2023-10-06T05:00:38.822726+0000","last_active":"2023-10-06T05:00:38.822726+0000","last_peered":"2023-10-06T05:00:38.822726+0000","last_clean":"2023-10-06T05:00:38.822726+0000","last_became_active":"2023-10-06T05:00:38.822500+0000","last_became_peered":"2023-10-06T05:00:38.822500+0000","last_unstale":"2023-10-06T05:00:38.822726+0000","last_undegraded":"2023-10-06T05:00:38.822726+0000","last_fullsized":"2023-10-06T05:00:38.822726+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:22.707806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"119'8","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.833808+0000","last_change":"2023-10-06T05:00:37.825037+0000","last_active":"2023-10-06T05:00:38.833808+0000","last_peered":"2023-10-06T05:00:38.833808+0000","last_clean":"2023-10-06T05:00:38.833808+0000","last_became_active":"2023-10-06T05:00:37.824634+0000","last_became_peered":"2023-10-06T05:00:37.824634+0000","last_unstale":"2023-10-06T05:00:38.833808+0000","last_undegraded":"2023-10-06T05:00:38.833808+0000","last_fullsized":"2023-10-06T05:00:38.833808+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:38.407633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":92,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.821735+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:00:37.821735+0000","last_peered":"2023-10-06T05:00:37.821735+0000","last_clean":"2023-10-06T05:00:37.821735+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:00:37.821735+0000","last_undegraded":"2023-10-06T05:00:37.821735+0000","last_fullsized":"2023-10-06T05:00:37.821735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827008+0000","last_change":"2023-10-06T05:00:38.824824+0000","last_active":"2023-10-06T05:00:38.827008+0000","last_peered":"2023-10-06T05:00:38.827008+0000","last_clean":"2023-10-06T05:00:38.827008+0000","last_became_active":"2023-10-06T05:00:38.824424+0000","last_became_peered":"2023-10-06T05:00:38.824424+0000","last_unstale":"2023-10-06T05:00:38.827008+0000","last_undegraded":"2023-10-06T05:00:38.827008+0000","last_fullsized":"2023-10-06T05:00:38.827008+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:51.075748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.830574+0000","last_change":"2023-10-06T05:00:37.829593+0000","last_active":"2023-10-06T05:00:37.830574+0000","last_peered":"2023-10-06T05:00:37.830574+0000","last_clean":"2023-10-06T05:00:37.830574+0000","last_became_active":"2023-10-06T05:00:37.829291+0000","last_became_peered":"2023-10-06T05:00:37.829291+0000","last_unstale":"2023-10-06T05:00:37.830574+0000","last_undegraded":"2023-10-06T05:00:37.830574+0000","last_fullsized":"2023-10-06T05:00:37.830574+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:11.848429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-06T05:00:37.819246+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:00:37.819246+0000","last_peered":"2023-10-06T05:00:37.819246+0000","last_clean":"2023-10-06T05:00:37.819246+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:00:37.819246+0000","last_undegraded":"2023-10-06T05:00:37.819246+0000","last_fullsized":"2023-10-06T05:00:37.819246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.827214+0000","last_change":"2023-10-06T05:00:38.825431+0000","last_active":"2023-10-06T05:00:38.827214+0000","last_peered":"2023-10-06T05:00:38.827214+0000","last_clean":"2023-10-06T05:00:38.827214+0000","last_became_active":"2023-10-06T05:00:38.825147+0000","last_became_peered":"2023-10-06T05:00:38.825147+0000","last_unstale":"2023-10-06T05:00:38.827214+0000","last_undegraded":"2023-10-06T05:00:38.827214+0000","last_fullsized":"2023-10-06T05:00:38.827214+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:09.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"119'2","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.825060+0000","last_change":"2023-10-06T05:00:37.821212+0000","last_active":"2023-10-06T05:00:38.825060+0000","last_peered":"2023-10-06T05:00:38.825060+0000","last_clean":"2023-10-06T05:00:38.825060+0000","last_became_active":"2023-10-06T05:00:37.820929+0000","last_became_peered":"2023-10-06T05:00:37.820929+0000","last_unstale":"2023-10-06T05:00:38.825060+0000","last_undegraded":"2023-10-06T05:00:38.825060+0000","last_fullsized":"2023-10-06T05:00:38.825060+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:39.259088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.816307+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:00:38.816307+0000","last_peered":"2023-10-06T05:00:38.816307+0000","last_clean":"2023-10-06T05:00:38.816307+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:00:38.816307+0000","last_undegraded":"2023-10-06T05:00:38.816307+0000","last_fullsized":"2023-10-06T05:00:38.816307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.831091+0000","last_change":"2023-10-06T05:00:38.827789+0000","last_active":"2023-10-06T05:00:38.831091+0000","last_peered":"2023-10-06T05:00:38.831091+0000","last_clean":"2023-10-06T05:00:38.831091+0000","last_became_active":"2023-10-06T05:00:38.827488+0000","last_became_peered":"2023-10-06T05:00:38.827488+0000","last_unstale":"2023-10-06T05:00:38.831091+0000","last_undegraded":"2023-10-06T05:00:38.831091+0000","last_fullsized":"2023-10-06T05:00:38.831091+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:47.335919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"119'6","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.839100+0000","last_change":"2023-10-06T05:00:37.828413+0000","last_active":"2023-10-06T05:00:38.839100+0000","last_peered":"2023-10-06T05:00:38.839100+0000","last_clean":"2023-10-06T05:00:38.839100+0000","last_became_active":"2023-10-06T05:00:37.828035+0000","last_became_peered":"2023-10-06T05:00:37.828035+0000","last_unstale":"2023-10-06T05:00:38.839100+0000","last_undegraded":"2023-10-06T05:00:38.839100+0000","last_fullsized":"2023-10-06T05:00:38.839100+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:43.942592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812032+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:00:38.812032+0000","last_peered":"2023-10-06T05:00:38.812032+0000","last_clean":"2023-10-06T05:00:38.812032+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:00:38.812032+0000","last_undegraded":"2023-10-06T05:00:38.812032+0000","last_fullsized":"2023-10-06T05:00:38.812032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.826591+0000","last_change":"2023-10-06T05:00:38.821172+0000","last_active":"2023-10-06T05:00:38.826591+0000","last_peered":"2023-10-06T05:00:38.826591+0000","last_clean":"2023-10-06T05:00:38.826591+0000","last_became_active":"2023-10-06T05:00:38.820895+0000","last_became_peered":"2023-10-06T05:00:38.820895+0000","last_unstale":"2023-10-06T05:00:38.826591+0000","last_undegraded":"2023-10-06T05:00:38.826591+0000","last_fullsized":"2023-10-06T05:00:38.826591+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:37.796594+0000","last_clean_scrub_stamp":"2023-10-06T05:00:37.796594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:48.085106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"119'3","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.836863+0000","last_change":"2023-10-06T05:00:37.828774+0000","last_active":"2023-10-06T05:00:38.836863+0000","last_peered":"2023-10-06T05:00:38.836863+0000","last_clean":"2023-10-06T05:00:38.836863+0000","last_became_active":"2023-10-06T05:00:37.828507+0000","last_became_peered":"2023-10-06T05:00:37.828507+0000","last_unstale":"2023-10-06T05:00:38.836863+0000","last_undegraded":"2023-10-06T05:00:38.836863+0000","last_fullsized":"2023-10-06T05:00:38.836863+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:00:36.790752+0000","last_clean_scrub_stamp":"2023-10-06T05:00:36.790752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:09.945646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":94,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-06T05:00:38.812144+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:00:38.812144+0000","last_peered":"2023-10-06T05:00:38.812144+0000","last_clean":"2023-10-06T05:00:38.812144+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:00:38.812144+0000","last_undegraded":"2023-10-06T05:00:38.812144+0000","last_fullsized":"2023-10-06T05:00:38.812144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307524,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064316,"statfs":{"total":96636764160,"available":96321859584,"internally_reserved":0,"allocated":2093056,"data_stored":1265161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.73899999999999999,"5min":0.86599999999999999,"15min":7.1779999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.73499999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.311},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":7.1879999999999997},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.073},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":1.1379999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.111},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.0880000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.874},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.214},"last":0.54900000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":681805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58799999999999997,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.73399999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63400000000000001,"15min":0.57499999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.21}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.65500000000000003,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":1.093},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.77900000000000003,"5min":1.234,"15min":1.234},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.63400000000000001,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70099999999999996,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.3120000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.755,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.238},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61199999999999999,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":1.175}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.68100000000000005,"5min":3.669,"15min":3.669},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.79300000000000004,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":1.147}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.274},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":1.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307512,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064328,"statfs":{"total":96636764160,"available":96321871872,"internally_reserved":0,"allocated":2080768,"data_stored":1263605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35599999999999998,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.216},"max":{"1min":0.46400000000000002,"5min":0.61899999999999999,"15min":0.68400000000000005},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.23499999999999999,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.56100000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":0.63800000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"last":0.316},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38700000000000001,"15min":0.40000000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.622,"5min":0.622,"15min":0.70399999999999996},"last":0.26300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42699999999999999,"15min":0.42599999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.433,"15min":0.433},"min":{"1min":0.313,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.57399999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.877,"15min":1.0780000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.434}]},{"osd":5,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.90300000000000002,"5min":0.998,"15min":0.998},"last":0.496},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.64800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39200000000000002},"max":{"1min":0.80900000000000005,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.83199999999999996,"5min":0.96299999999999997,"15min":1.0900000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65100000000000002,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.44700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63900000000000001,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":0.87,"5min":0.98099999999999998,"15min":1.026},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.441},"max":{"1min":0.93700000000000006,"5min":1.022,"15min":1.0429999999999999},"last":0.56999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306940,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064900,"statfs":{"total":96636764160,"available":96322457600,"internally_reserved":0,"allocated":1495040,"data_stored":674576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":2,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.59199999999999997,"5min":0.628,"15min":0.628},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42999999999999999,"15min":0.38800000000000001},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46100000000000002,"15min":0.443},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.502}]},{"osd":4,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.98299999999999998},"last":0.629},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.85799999999999998,"15min":0.92300000000000004},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.91700000000000004},"last":0.749}]},{"osd":6,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.93400000000000005,"15min":1.0109999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.52200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.97599999999999998,"15min":1.0329999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":0.995},"last":0.88100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302844,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068996,"statfs":{"total":96636764160,"available":96326651904,"internally_reserved":0,"allocated":1495040,"data_stored":675108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.20300000000000001,"15min":0.186},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.048},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.39300000000000002},"min":{"1min":0.32300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.317}]},{"osd":1,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.216},"max":{"1min":0.52400000000000002,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.375},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.219},"max":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.27900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.36499999999999999},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.90200000000000002},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.622},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.2649999999999999},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.88100000000000001,"15min":0.93100000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":0.88300000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.82499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306948,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064892,"statfs":{"total":96636764160,"available":96322449408,"internally_reserved":0,"allocated":1503232,"data_stored":676126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56399999999999995,"5min":0.68200000000000005,"15min":2.008},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.53000000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":6.4370000000000003},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.224,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.67000000000000004,"15min":1.7649999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":2.1970000000000001},"last":0.33400000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42399999999999999,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.255},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":5.069},"last":0.315},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.441,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":2.246},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59499999999999997,"15min":0.76100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":14.646000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":14.606999999999999},"last":0.505}]},{"osd":5,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.81399999999999995,"5min":0.94699999999999995,"15min":2.2959999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":6.4009999999999998},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.73299999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.90100000000000002,"15min":5.0309999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":0.87,"5min":1.012,"15min":2.2749999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.75800000000000001},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.92600000000000005,"15min":6.367},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":5},"last":0.60499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":674520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.71199999999999997,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.73299999999999998},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.623}]},{"osd":1,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.76200000000000001},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.76700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.70999999999999996,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.64100000000000001},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.80000000000000004,"15min":5.415},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.64800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":4.202},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.622,"15min":0.70399999999999996},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.754,"5min":1.044,"15min":4.2130000000000001},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.497},"min":{"1min":0.27800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69799999999999995,"5min":0.73399999999999999,"15min":1.1279999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.76900000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.505},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78700000000000003,"5min":0.86199999999999999,"15min":0.872},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.49399999999999999,"5min":0.375,"15min":0.313},"max":{"1min":0.76300000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.54800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306944,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064896,"statfs":{"total":96636764160,"available":96322453504,"internally_reserved":0,"allocated":1499136,"data_stored":679830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.20499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":4.9039999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57499999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":6.7949999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.26200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.86599999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.501},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.86599999999999999},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.442}]},{"osd":4,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53800000000000003,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:00:43.357 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:00:43.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:00:43.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.482+0000 7fc740d1d700 1 Processor -- start 2023-10-06T05:00:43.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.482+0000 7fc740d1d700 1 -- start start 2023-10-06T05:00:43.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.482+0000 7fc740d1d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 0x7fc73c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.482+0000 7fc740d1d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc73c15cf10 0x7fc73c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.482+0000 7fc740d1d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73c0691c0 con 0x7fc73c0b1b00 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.482+0000 7fc740d1d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c0694a0 con 0x7fc73c15cf10 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.482+0000 7fc740d1d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c069780 con 0x7fc73c067dc0 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc731d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc73c0b1b00 0x7fc73c15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60974/0 (socket says 172.21.15.3:60974) 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc731d9b700 1 -- 172.21.15.3:0/990693440 learned_addr learned my addr 172.21.15.3:0/990693440 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc73a59c700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 0x7fc73c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc73ad9d700 1 --2- 172.21.15.3:0/990693440 >> v2:172.21.15.153:3300/0 conn(0x7fc73c15cf10 0x7fc73c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025610687 0 0) 0x7fc73c0691c0 con 0x7fc73c0b1b00 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc728003570 con 0x7fc73c0b1b00 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3753272224 0 0) 0x7fc728003570 con 0x7fc73c0b1b00 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 msgr2=0x7fc73c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 0x7fc73c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 >> v2:172.21.15.153:3300/0 conn(0x7fc73c15cf10 msgr2=0x7fc73c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 --2- 172.21.15.3:0/990693440 >> v2:172.21.15.153:3300/0 conn(0x7fc73c15cf10 0x7fc73c06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c069a60 con 0x7fc73c0b1b00 2023-10-06T05:00:43.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc724001470 con 0x7fc73c0b1b00 2023-10-06T05:00:43.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 >> v1:172.21.15.3:6789/0 conn(0x7fc73c0b1b00 legacy=0x7fc73c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc728003a60 0x7fc728003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 0x7fc728004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc728004d50 0x7fc7280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0694a0 con 0x7fc728003a60 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c069780 con 0x7fc728004d50 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72800ba60 con 0x7fc73c067dc0 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc73a59c700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc728003a60 0x7fc728003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc731d9b700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 0x7fc728004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.483+0000 7fc73ad9d700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc728004d50 0x7fc7280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73a59c700 1 -- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 msgr2=0x7fc728004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73a59c700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc73c067dc0 0x7fc728004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73a59c700 1 -- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc728004d50 msgr2=0x7fc7280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73a59c700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc728004d50 0x7fc7280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73a59c700 1 -- 172.21.15.3:0/990693440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728003570 con 0x7fc728003a60 2023-10-06T05:00:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73ad9d700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc728004d50 0x7fc7280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73a59c700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc728003a60 0x7fc728003e70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009bf0 tx=0x7fc72c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d543af7f4885033 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c005a00 con 0x7fc728003a60 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c005b60 con 0x7fc728003a60 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc739d9b700 1 -- 172.21.15.3:0/990693440 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00e6b0 con 0x7fc728003a60 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc740d1d700 1 -- 172.21.15.3:0/990693440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc728003a60 msgr2=0x7fc728003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc740d1d700 1 --2- 172.21.15.3:0/990693440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc728003a60 0x7fc728003e70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009bf0 tx=0x7fc72c009120 comp rx=0 tx=0).stop 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.484+0000 7fc73ad9d700 1 -- 172.21.15.3:0/990693440 reap_dead start 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.485+0000 7fc740d1d700 1 -- 172.21.15.3:0/990693440 shutdown_connections 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.485+0000 7fc740d1d700 1 -- 172.21.15.3:0/990693440 >> 172.21.15.3:0/990693440 conn(0x7fc73c0ce270 msgr2=0x7fc73c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.485+0000 7fc740d1d700 1 -- 172.21.15.3:0/990693440 shutdown_connections 2023-10-06T05:00:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.485+0000 7fc740d1d700 1 -- 172.21.15.3:0/990693440 wait complete. 2023-10-06T05:00:43.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.486+0000 7fc740d1d700 1 Processor -- start 2023-10-06T05:00:43.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.486+0000 7fc740d1d700 1 -- start start 2023-10-06T05:00:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.486+0000 7fc740d1d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc728003a60 0x7fc73c14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.486+0000 7fc740d1d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc73c16ad30 0x7fc73c1528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc740d1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73c152e20 0x7fc73c15b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc740d1d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c153610 con 0x7fc73c152e20 2023-10-06T05:00:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc740d1d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c16a8a0 con 0x7fc73c16ad30 2023-10-06T05:00:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc740d1d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c169fe0 con 0x7fc728003a60 2023-10-06T05:00:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73c152e20 0x7fc73c15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73c152e20 0x7fc73c15b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37052/0 (socket says 172.21.15.3:37052) 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 -- 172.21.15.3:0/369583004 learned_addr learned my addr 172.21.15.3:0/369583004 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc731d9b700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc73c16ad30 0x7fc73c1528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73a59c700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc728003a60 0x7fc73c14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 -- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc728003a60 msgr2=0x7fc73c14cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc728003a60 0x7fc73c14cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 -- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc73c16ad30 msgr2=0x7fc73c1528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc73c16ad30 0x7fc73c1528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 -- 172.21.15.3:0/369583004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c153330 con 0x7fc73c152e20 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc731d9b700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc73c16ad30 0x7fc73c1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc73ad9d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73c152e20 0x7fc73c15b920 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc73c06f660 tx=0x7fc73400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc732ffd700 1 -- 172.21.15.3:0/369583004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73400e030 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc732ffd700 1 -- 172.21.15.3:0/369583004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc734011d30 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73c16a250 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc732ffd700 1 -- 172.21.15.3:0/369583004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7340166c0 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.487+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c14d4d0 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.488+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73c16aa50 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.489+0000 7fc732ffd700 1 -- 172.21.15.3:0/369583004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc734016ab0 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.489+0000 7fc732ffd700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc71c066f30 0x7fc71c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.489+0000 7fc732ffd700 1 -- 172.21.15.3:0/369583004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc73400f070 con 0x7fc73c152e20 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.489+0000 7fc73a59c700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc71c066f30 0x7fc71c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.490+0000 7fc73a59c700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc71c066f30 0x7fc71c0693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009160 tx=0x7fc72c019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.493+0000 7fc732ffd700 1 -- 172.21.15.3:0/369583004 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc73c16aa50 con 0x7fc73c152e20 2023-10-06T05:00:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.766+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc73c16aa50 con 0x7fc73c152e20 2023-10-06T05:00:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.770+0000 7fc732ffd700 1 -- 172.21.15.3:0/369583004 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc73c16aa50 con 0x7fc73c152e20 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc71c066f30 msgr2=0x7fc71c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc71c066f30 0x7fc71c0693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009160 tx=0x7fc72c019000 comp rx=0 tx=0).stop 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73c152e20 msgr2=0x7fc73c15b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73c152e20 0x7fc73c15b920 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc73c06f660 tx=0x7fc73400b040 comp rx=0 tx=0).stop 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 shutdown_connections 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc71c066f30 0x7fc71c0693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73c152e20 0x7fc73c15b920 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc728003a60 0x7fc73c14cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 --2- 172.21.15.3:0/369583004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc73c16ad30 0x7fc73c1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 >> 172.21.15.3:0/369583004 conn(0x7fc73c0ce270 msgr2=0x7fc73c0af890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 shutdown_connections 2023-10-06T05:00:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.772+0000 7fc740d1d700 1 -- 172.21.15.3:0/369583004 wait complete. 2023-10-06T05:00:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T05:00:43.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:43.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.906+0000 7f0070d84700 1 Processor -- start 2023-10-06T05:00:43.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.906+0000 7f0070d84700 1 -- start start 2023-10-06T05:00:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0070d84700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0070d84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f006c15ccc0 0x7f006c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0070d84700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f006c0d06a0 0x7f006c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0070d84700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c0cf6c0 con 0x7f006c15ccc0 2023-10-06T05:00:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0070d84700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c0cfb50 con 0x7f006c0d06a0 2023-10-06T05:00:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0070d84700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c15e050 con 0x7f006c15c8f0 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f006c15ccc0 0x7f006c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f006c15ccc0 0x7f006c0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37054/0 (socket says 172.21.15.3:37054) 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 -- 172.21.15.3:0/3002039523 learned_addr learned my addr 172.21.15.3:0/3002039523 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 -- 172.21.15.3:0/3002039523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 msgr2=0x7f006c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f006a59c700 1 --2- 172.21.15.3:0/3002039523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f006ad9d700 1 --2- 172.21.15.3:0/3002039523 >> v2:172.21.15.153:3300/0 conn(0x7f006c0d06a0 0x7f006c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 --2- 172.21.15.3:0/3002039523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 -- 172.21.15.3:0/3002039523 >> v2:172.21.15.153:3300/0 conn(0x7f006c0d06a0 msgr2=0x7f006c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 --2- 172.21.15.3:0/3002039523 >> v2:172.21.15.153:3300/0 conn(0x7f006c0d06a0 0x7f006c06d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 -- 172.21.15.3:0/3002039523 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0d0af0 con 0x7f006c15ccc0 2023-10-06T05:00:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f006ad9d700 1 --2- 172.21.15.3:0/3002039523 >> v2:172.21.15.153:3300/0 conn(0x7f006c0d06a0 0x7f006c06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f006a59c700 1 --2- 172.21.15.3:0/3002039523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f0069d9b700 1 --2- 172.21.15.3:0/3002039523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0d0160 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f006c0af8a0 tx=0x7f0054009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283f6f9d9874889c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.907+0000 7f006959a700 1 -- 172.21.15.3:0/3002039523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400b860 con 0x7f006c15ccc0 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f006959a700 1 -- 172.21.15.3:0/3002039523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005400b9c0 con 0x7f006c15ccc0 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f006959a700 1 -- 172.21.15.3:0/3002039523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400bd60 con 0x7f006c15ccc0 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 -- 172.21.15.3:0/3002039523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 msgr2=0x7f006c0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 --2- 172.21.15.3:0/3002039523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0d0160 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f006c0af8a0 tx=0x7f0054009250 comp rx=0 tx=0).stop 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 -- 172.21.15.3:0/3002039523 shutdown_connections 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 --2- 172.21.15.3:0/3002039523 >> v2:172.21.15.153:3300/0 conn(0x7f006c0d06a0 0x7f006c06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 --2- 172.21.15.3:0/3002039523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 --2- 172.21.15.3:0/3002039523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0d0160 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 -- 172.21.15.3:0/3002039523 >> 172.21.15.3:0/3002039523 conn(0x7f006c070310 msgr2=0x7f006c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:43.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 -- 172.21.15.3:0/3002039523 shutdown_connections 2023-10-06T05:00:43.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.908+0000 7f0070d84700 1 -- 172.21.15.3:0/3002039523 wait complete. 2023-10-06T05:00:43.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.909+0000 7f0070d84700 1 Processor -- start 2023-10-06T05:00:43.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.909+0000 7f0070d84700 1 -- start start 2023-10-06T05:00:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0070d84700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f006c0d06a0 0x7f006c146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0070d84700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0070d84700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0070d84700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0cbf80 con 0x7f006c15ccc0 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0070d84700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c146950 con 0x7f006c0d06a0 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0070d84700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0cb370 con 0x7f006c15c8f0 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37070/0 (socket says 172.21.15.3:37070) 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 -- 172.21.15.3:0/1614527389 learned_addr learned my addr 172.21.15.3:0/1614527389 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006a59c700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f006c0d06a0 0x7f006c146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0069d9b700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 -- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 msgr2=0x7f006c0ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c0ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 -- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f006c0d06a0 msgr2=0x7f006c146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f006c0d06a0 0x7f006c146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006ad9d700 1 -- 172.21.15.3:0/1614527389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540094f0 con 0x7f006c15ccc0 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f006a59c700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f006c0d06a0 0x7f006c146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.910+0000 7f0069d9b700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.911+0000 7f006ad9d700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0cab50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f006400c8b0 tx=0x7f0064008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.911+0000 7f00627fc700 1 -- 172.21.15.3:0/1614527389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064011600 con 0x7f006c15ccc0 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.911+0000 7f00627fc700 1 -- 172.21.15.3:0/1614527389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0064011780 con 0x7f006c15ccc0 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.911+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c0cb650 con 0x7f006c15ccc0 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.911+0000 7f00627fc700 1 -- 172.21.15.3:0/1614527389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064011b40 con 0x7f006c15ccc0 2023-10-06T05:00:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.911+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c0cbc10 con 0x7f006c15ccc0 2023-10-06T05:00:43.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.912+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c0cc130 con 0x7f006c15ccc0 2023-10-06T05:00:43.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.912+0000 7f00627fc700 1 -- 172.21.15.3:0/1614527389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00640193b0 con 0x7f006c15ccc0 2023-10-06T05:00:43.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.912+0000 7f00627fc700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f004c066f30 0x7f004c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:43.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.913+0000 7f00627fc700 1 -- 172.21.15.3:0/1614527389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0064084640 con 0x7f006c15ccc0 2023-10-06T05:00:43.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.913+0000 7f006a59c700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f004c066f30 0x7f004c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:43.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.913+0000 7f006a59c700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f004c066f30 0x7f004c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d640 tx=0x7f005c007670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:43.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:43.917+0000 7f00627fc700 1 -- 172.21.15.3:0/1614527389 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f006c0cc130 con 0x7f006c15ccc0 2023-10-06T05:00:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.152+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f006c0cc130 con 0x7f006c15ccc0 2023-10-06T05:00:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.153+0000 7f00627fc700 1 -- 172.21.15.3:0/1614527389 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f006c0cc130 con 0x7f006c15ccc0 2023-10-06T05:00:44.156 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:44.156 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:41.823898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:00:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f004c066f30 msgr2=0x7f004c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f004c066f30 0x7f004c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d640 tx=0x7f005c007670 comp rx=0 tx=0).stop 2023-10-06T05:00:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 msgr2=0x7f006c0cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0cab50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f006400c8b0 tx=0x7f0064008040 comp rx=0 tx=0).stop 2023-10-06T05:00:44.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 shutdown_connections 2023-10-06T05:00:44.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f004c066f30 0x7f004c0693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f006c0d06a0 0x7f006c146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f006c15ccc0 0x7f006c0cab50 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 --2- 172.21.15.3:0/1614527389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f006c15c8f0 0x7f006c0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 >> 172.21.15.3:0/1614527389 conn(0x7f006c070310 msgr2=0x7f006c0b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 shutdown_connections 2023-10-06T05:00:44.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.155+0000 7f0070d84700 1 -- 172.21.15.3:0/1614527389 wait complete. 2023-10-06T05:00:44.160 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 79 2023-10-06T05:00:44.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:00:38.212073+0000', 'modified': '2023-10-06T05:00:41.823898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8010}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8010': {'gid': 8010, 'name': 'd', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6837/3912023637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6835', 'nonce': 3912023637}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3912023637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:00:44.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:00:44.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:00:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.693+0000 7f07f73c8700 1 Processor -- start 2023-10-06T05:00:44.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.693+0000 7f07f73c8700 1 -- start start 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.694+0000 7f07f73c8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f00b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.694+0000 7f07f73c8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f00b0ce0 con 0x7f07f00b0590 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.694+0000 7f07f73c8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f00cb820 con 0x7f07f00cd950 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.694+0000 7f07f73c8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00b0a00 con 0x7f07f015cb50 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.694+0000 7f07f4963700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f07f00b0590 0x7f07f00b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60986/0 (socket says 172.21.15.3:60986) 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.694+0000 7f07f4963700 1 -- 172.21.15.3:0/240698916 learned_addr learned my addr 172.21.15.3:0/240698916 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.694+0000 7f07f5164700 1 --2- 172.21.15.3:0/240698916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f00b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07effff700 1 -- 172.21.15.3:0/240698916 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635746402 0 0) 0x7f07f00b0ce0 con 0x7f07f00b0590 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07effff700 1 -- 172.21.15.3:0/240698916 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d8003570 con 0x7f07f00b0590 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07effff700 1 -- 172.21.15.3:0/240698916 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221588629 0 0) 0x7f07f00cb820 con 0x7f07f00cd950 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07effff700 1 -- 172.21.15.3:0/240698916 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f00b0ce0 con 0x7f07f00cd950 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07f5164700 1 -- 172.21.15.3:0/240698916 >> v1:172.21.15.153:6789/0 conn(0x7f07f00cd950 legacy=0x7f07f00cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07f5164700 1 -- 172.21.15.3:0/240698916 >> v1:172.21.15.3:6789/0 conn(0x7f07f00b0590 legacy=0x7f07f00b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07f5164700 1 -- 172.21.15.3:0/240698916 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f00cb1d0 con 0x7f07f015cb50 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07f5164700 1 --2- 172.21.15.3:0/240698916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f00b0050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009900 tx=0x7f07e8009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bef7b6bbf636688 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07effff700 1 -- 172.21.15.3:0/240698916 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4001470 con 0x7f07f015cb50 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07effff700 1 -- 172.21.15.3:0/240698916 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e8011420 con 0x7f07f015cb50 2023-10-06T05:00:44.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.695+0000 7f07effff700 1 -- 172.21.15.3:0/240698916 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80117c0 con 0x7f07f015cb50 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.696+0000 7f07f73c8700 1 -- 172.21.15.3:0/240698916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 msgr2=0x7f07f00b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.696+0000 7f07f73c8700 1 --2- 172.21.15.3:0/240698916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f00b0050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009900 tx=0x7f07e8009120 comp rx=0 tx=0).stop 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.696+0000 7f07f73c8700 1 -- 172.21.15.3:0/240698916 shutdown_connections 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.696+0000 7f07f73c8700 1 --2- 172.21.15.3:0/240698916 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f00b0050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.696+0000 7f07f73c8700 1 -- 172.21.15.3:0/240698916 >> 172.21.15.3:0/240698916 conn(0x7f07f0070310 msgr2=0x7f07f0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.696+0000 7f07f73c8700 1 -- 172.21.15.3:0/240698916 shutdown_connections 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.696+0000 7f07f73c8700 1 -- 172.21.15.3:0/240698916 wait complete. 2023-10-06T05:00:44.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.697+0000 7f07f73c8700 1 Processor -- start 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.697+0000 7f07f73c8700 1 -- start start 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f73c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07f00b0590 0x7f07f00c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f73c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f00cd950 0x7f07f016a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f73c8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f0158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f73c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00c6ce0 con 0x7f07f00cd950 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f73c8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01552b0 con 0x7f07f00b0590 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f73c8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f016a8e0 con 0x7f07f015cb50 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f00cd950 0x7f07f016a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f00cd950 0x7f07f016a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37074/0 (socket says 172.21.15.3:37074) 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 -- 172.21.15.3:0/377371597 learned_addr learned my addr 172.21.15.3:0/377371597 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:44.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f5164700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07f00b0590 0x7f07f00c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f5965700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f0158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 -- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 msgr2=0x7f07f0158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f0158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 -- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07f00b0590 msgr2=0x7f07f00c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07f00b0590 0x7f07f00c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.698+0000 7f07f4963700 1 -- 172.21.15.3:0/377371597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80093c0 con 0x7f07f00cd950 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.699+0000 7f07f4963700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f00cd950 0x7f07f016a3a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f07e40011e0 tx=0x7f07e4001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.699+0000 7f07ed7fa700 1 -- 172.21.15.3:0/377371597 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e400ce40 con 0x7f07f00cd950 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.699+0000 7f07ed7fa700 1 -- 172.21.15.3:0/377371597 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e4007360 con 0x7f07f00cd950 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.699+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f00c6160 con 0x7f07f00cd950 2023-10-06T05:00:44.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.699+0000 7f07ed7fa700 1 -- 172.21.15.3:0/377371597 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4007720 con 0x7f07f00cd950 2023-10-06T05:00:44.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.699+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f00c6720 con 0x7f07f00cd950 2023-10-06T05:00:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.700+0000 7f07ed7fa700 1 -- 172.21.15.3:0/377371597 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07e4004a40 con 0x7f07f00cd950 2023-10-06T05:00:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.701+0000 7f07ed7fa700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07dc066f30 0x7f07dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.701+0000 7f07ed7fa700 1 -- 172.21.15.3:0/377371597 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f07e4083fe0 con 0x7f07f00cd950 2023-10-06T05:00:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.701+0000 7f07f5164700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07dc066f30 0x7f07dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.702+0000 7f07f5164700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07dc066f30 0x7f07dc0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009850 tx=0x7f07e8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 Processor -- start 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 -- start start 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f61de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f5d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f556c80 con 0x55a55f5d99a0 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f52da70 con 0x55a55f61da90 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.703+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55f52b270 con 0x55a55f5d7bd0 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37078/0 (socket says 172.21.15.3:37078) 2023-10-06T05:00:44.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 -- 172.21.15.3:0/782756721 learned_addr learned my addr 172.21.15.3:0/782756721 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f5164700 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f61de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f5965700 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f5d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 -- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 msgr2=0x55a55f5d6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f5d6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 -- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 msgr2=0x55a55f61de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f61de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 -- 172.21.15.3:0/782756721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a55f3c1270 con 0x55a55f5d99a0 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f5164700 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f61de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f5965700 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f5d6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07f4963700 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5d6580 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f07e4054d60 tx=0x7f07e4057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e9eb419701df07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07d67fc700 1 -- 172.21.15.3:0/782756721 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e405a0f0 con 0x55a55f5d99a0 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07d67fc700 1 -- 172.21.15.3:0/782756721 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e406b170 con 0x55a55f5d99a0 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.704+0000 7f07d67fc700 1 -- 172.21.15.3:0/782756721 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4053170 con 0x55a55f5d99a0 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 -- 172.21.15.3:0/782756721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 msgr2=0x55a55f5d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5d6580 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f07e4054d60 tx=0x7f07e4057040 comp rx=0 tx=0).stop 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 -- 172.21.15.3:0/782756721 shutdown_connections 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f61de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f5d6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 --2- 172.21.15.3:0/782756721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5d6580 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 -- 172.21.15.3:0/782756721 >> 172.21.15.3:0/782756721 conn(0x55a55f5d8070 msgr2=0x55a55f5d8440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 -- 172.21.15.3:0/782756721 shutdown_connections 2023-10-06T05:00:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 -- 172.21.15.3:0/782756721 wait complete. 2023-10-06T05:00:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.705+0000 7f08006c0b80 1 -- start start 2023-10-06T05:00:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 1 Processor -- start 2023-10-06T05:00:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f61aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f61da90 0x55a55f5ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f527060 con 0x55a55f5d99a0 2023-10-06T05:00:44.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f4d2890 con 0x55a55f5d7bd0 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55f4edea0 con 0x55a55f61da90 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f08006c0b80 10 client.0 _ll_register_callbacks cb 0x55a55f3c3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.706+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37094/0 (socket says 172.21.15.3:37094) 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 -- 172.21.15.3:0/3898712574 learned_addr learned my addr 172.21.15.3:0/3898712574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f5164700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f61aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f5965700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f61da90 0x55a55f5ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 -- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f61da90 msgr2=0x55a55f5ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f61da90 0x55a55f5ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 -- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 msgr2=0x55a55f61aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f61aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 -- 172.21.15.3:0/3898712574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f00c6720 con 0x55a55f5d99a0 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f5164700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f61aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f5965700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f61da90 0x55a55f5ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07f4963700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5feba0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f07e4055680 tx=0x7f07e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e063c86fec6cb708 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07cf7fe700 1 -- 172.21.15.3:0/3898712574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4052f10 con 0x55a55f5d99a0 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07cf7fe700 1 -- 172.21.15.3:0/3898712574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f07e405a070 con 0x55a55f5d99a0 2023-10-06T05:00:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a55f5cb690 con 0x55a55f5d99a0 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07cf7fe700 1 -- 172.21.15.3:0/3898712574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e405b8e0 con 0x55a55f5d99a0 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 10 client.8127 fetch_fsmap learned FSMap version 79 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55a55f363fe0 con 0x55a55f5d99a0 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07cf7fe700 1 -- 172.21.15.3:0/3898712574 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a55f5cb690 con 0x55a55f5d99a0 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f07cf7fe700 1 -- 172.21.15.3:0/3898712574 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 79) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f07e4067240 con 0x55a55f5d99a0 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 10 client.8127 fetch_fsmap finished waiting for FSMap version 79 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 10 client.8127 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 20 client.8127 populate_metadata read hostname 'smithi003' 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f617300 0x55a55f600b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 4 client.8127 mds_command: new command op to 8010 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.707+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 --> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a55f5a85c0 con 0x55a55f617300 2023-10-06T05:00:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.708+0000 7f07f5164700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f617300 0x55a55f600b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.708+0000 7f07cf7fe700 10 client.8127 ms_handle_connect on v2:172.21.15.153:6835/3912023637 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.708+0000 7f07f5164700 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f617300 0x55a55f600b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e99f5eeb15a772b4 server_cookie=e8d44819ee45e67a in_seq=0 out_seq=0 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f07cf7fe700 1 -- 172.21.15.3:0/3898712574 <== mds.0 v2:172.21.15.153:6835/3912023637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a55f5a85c0 con 0x55a55f617300 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f07cf7fe700 10 client.8127 handle_command_reply: tid=0 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 client.8127 shutdown 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 msgr2=0x55a55f5feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5feba0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f07e4055680 tx=0x7f07e4013000 comp rx=0 tx=0).stop 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 shutdown_connections 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f617300 0x55a55f600b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f61aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f07cf7fe700 0 client.8127 ms_handle_reset on v2:172.21.15.153:6835/3912023637 2023-10-06T05:00:44.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f61da90 0x55a55f5ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3898712574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f5feba0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 >> 172.21.15.3:0/3898712574 conn(0x55a55f5d8070 msgr2=0x55a55f61cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 shutdown_connections 2023-10-06T05:00:44.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 1 -- 172.21.15.3:0/3898712574 wait complete. 2023-10-06T05:00:44.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.709+0000 7f08006c0b80 20 client.8127 trim_cache size 0 max 16384 2023-10-06T05:00:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.726+0000 7f08006c0b80 1 Processor -- start 2023-10-06T05:00:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.726+0000 7f08006c0b80 1 -- start start 2023-10-06T05:00:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f600d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f620c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f6211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f08006c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f3a9e30 con 0x55a55f5d99a0 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f6147b0 con 0x55a55f61da90 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55f5d9310 con 0x55a55f5d7bd0 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f620c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f620c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37098/0 (socket says 172.21.15.3:37098) 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 -- 172.21.15.3:0/3292977562 learned_addr learned my addr 172.21.15.3:0/3292977562 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f5164700 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f600d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f5965700 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f6211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 -- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 msgr2=0x55a55f600d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f600d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 -- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 msgr2=0x55a55f6211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f6211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 -- 172.21.15.3:0/3292977562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a55f5d5970 con 0x55a55f5d99a0 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f5164700 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f600d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f5965700 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f6211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07f4963700 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f620c80 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f07e405bc70 tx=0x7f07e4057070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f688157066529309 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07d6ffd700 1 -- 172.21.15.3:0/3292977562 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e405b8e0 con 0x55a55f5d99a0 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.727+0000 7f07d6ffd700 1 -- 172.21.15.3:0/3292977562 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e4001d90 con 0x55a55f5d99a0 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f07d6ffd700 1 -- 172.21.15.3:0/3292977562 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4052f10 con 0x55a55f5d99a0 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 -- 172.21.15.3:0/3292977562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 msgr2=0x55a55f620c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f620c80 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f07e405bc70 tx=0x7f07e4057070 comp rx=0 tx=0).stop 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 -- 172.21.15.3:0/3292977562 shutdown_connections 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f61da90 0x55a55f6211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d7bd0 0x55a55f600d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 --2- 172.21.15.3:0/3292977562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f5d99a0 0x55a55f620c80 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 -- 172.21.15.3:0/3292977562 >> 172.21.15.3:0/3292977562 conn(0x55a55f5d8070 msgr2=0x55a55f61a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 -- 172.21.15.3:0/3292977562 shutdown_connections 2023-10-06T05:00:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.728+0000 7f08006c0b80 1 -- 172.21.15.3:0/3292977562 wait complete. 2023-10-06T05:00:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.729+0000 7f08006c0b80 1 -- start start 2023-10-06T05:00:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.729+0000 7f08006c0b80 1 Processor -- start 2023-10-06T05:00:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f67c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d99a0 0x55a55f618050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f61da90 0x55a55f618590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f606d10 con 0x55a55f61da90 2023-10-06T05:00:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f6020b0 con 0x55a55f5d7bd0 2023-10-06T05:00:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55f5d4130 con 0x55a55f5d99a0 2023-10-06T05:00:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 10 client.0 _ll_register_callbacks cb 0x55a55f3c3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f61da90 0x55a55f618590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f61da90 0x55a55f618590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37106/0 (socket says 172.21.15.3:37106) 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 -- 172.21.15.3:0/916519714 learned_addr learned my addr 172.21.15.3:0/916519714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f4963700 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d99a0 0x55a55f618050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5164700 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f67c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 -- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d99a0 msgr2=0x55a55f618050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d99a0 0x55a55f618050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 -- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 msgr2=0x55a55f67c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f67c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 -- 172.21.15.3:0/916519714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a55f363fe0 con 0x55a55f61da90 2023-10-06T05:00:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07f5965700 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f61da90 0x55a55f618590 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x55a55f61def0 tx=0x7f07e0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1957512781c50790 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07cf7fe700 1 -- 172.21.15.3:0/916519714 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0015630 con 0x55a55f61da90 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f07cf7fe700 1 -- 172.21.15.3:0/916519714 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f07e001a030 con 0x55a55f61da90 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.730+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07e406b170 con 0x55a55f61da90 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f07cf7fe700 1 -- 172.21.15.3:0/916519714 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e00159a0 con 0x55a55f61da90 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 10 client.8133 fetch_fsmap learned FSMap version 79 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55a55f4f5780 con 0x55a55f61da90 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f07cf7fe700 1 -- 172.21.15.3:0/916519714 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e406b170 con 0x55a55f61da90 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f07cf7fe700 1 -- 172.21.15.3:0/916519714 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 79) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f07e0015b00 con 0x55a55f61da90 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 10 client.8133 fetch_fsmap finished waiting for FSMap version 79 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 10 client.8133 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:00:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 20 client.8133 populate_metadata read hostname 'smithi003' 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f5fc400 0x55a55f5fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 4 client.8133 mds_command: new command op to 8010 tid=0[{"prefix": "status"}] 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 --> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a55f4c9e00 con 0x55a55f5fc400 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f07f5164700 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f5fc400 0x55a55f5fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f07cf7fe700 10 client.8133 ms_handle_connect on v2:172.21.15.153:6835/3912023637 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.731+0000 7f07f5164700 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f5fc400 0x55a55f5fc7d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b625586c430ddce4 server_cookie=cad81cad39283190 in_seq=0 out_seq=0 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f07cf7fe700 1 -- 172.21.15.3:0/916519714 <== mds.0 v2:172.21.15.153:6835/3912023637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a55f4c9e00 con 0x55a55f5fc400 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f07cf7fe700 10 client.8133 handle_command_reply: tid=0 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 client.8133 shutdown 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f61da90 msgr2=0x55a55f618590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f61da90 0x55a55f618590 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x55a55f61def0 tx=0x7f07e0009040 comp rx=0 tx=0).stop 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 shutdown_connections 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:6835/3912023637,v1:172.21.15.153:6837/3912023637] conn(0x55a55f5fc400 0x55a55f5fc7d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a55f5d7bd0 0x55a55f67c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f07cf7fe700 0 client.8133 ms_handle_reset on v2:172.21.15.153:6835/3912023637 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a55f61da90 0x55a55f618590 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 --2- 172.21.15.3:0/916519714 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a55f5d99a0 0x55a55f618050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 >> 172.21.15.3:0/916519714 conn(0x55a55f5d8070 msgr2=0x55a55f5d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 shutdown_connections 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 1 -- 172.21.15.3:0/916519714 wait complete. 2023-10-06T05:00:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.732+0000 7f08006c0b80 20 client.8133 trim_cache size 0 max 16384 2023-10-06T05:00:44.743 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:00:44.743 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T05:00:44.743 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T05:00:44.743 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 8010, 2023-10-06T05:00:44.743 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T05:00:44.743 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T05:00:44.744 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T05:00:44.744 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 5.9272847779999998, 2023-10-06T05:00:44.744 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 79, 2023-10-06T05:00:44.744 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 120, 2023-10-06T05:00:44.744 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 120, 2023-10-06T05:00:44.744 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 11.990165501 2023-10-06T05:00:44.744 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:00:44.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07dc066f30 msgr2=0x7f07dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07dc066f30 0x7f07dc0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009850 tx=0x7f07e8017000 comp rx=0 tx=0).stop 2023-10-06T05:00:44.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f00cd950 msgr2=0x7f07f016a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f00cd950 0x7f07f016a3a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f07e40011e0 tx=0x7f07e4001c30 comp rx=0 tx=0).stop 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 shutdown_connections 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f07dc066f30 0x7f07dc0693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07f00b0590 0x7f07f00c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f07f015cb50 0x7f07f0158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 --2- 172.21.15.3:0/377371597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f00cd950 0x7f07f016a3a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 >> 172.21.15.3:0/377371597 conn(0x7f07f0070310 msgr2=0x7f07f015a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 shutdown_connections 2023-10-06T05:00:44.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.733+0000 7f07f73c8700 1 -- 172.21.15.3:0/377371597 wait complete. 2023-10-06T05:00:44.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:00:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.864+0000 7f3d84673700 1 Processor -- start 2023-10-06T05:00:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.864+0000 7f3d84673700 1 -- start start 2023-10-06T05:00:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.865+0000 7f3d84673700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3d7c140dc0 0x7f3d7c0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.865+0000 7f3d84673700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cd380 con 0x7f3d7c0b3a40 2023-10-06T05:00:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.865+0000 7f3d84673700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cd660 con 0x7f3d7c0b3e10 2023-10-06T05:00:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.865+0000 7f3d84673700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c0cd940 con 0x7f3d7c140dc0 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.865+0000 7f3d8240f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3d7c0b3a40 0x7f3d7c0680b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60990/0 (socket says 172.21.15.3:60990) 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.865+0000 7f3d8240f700 1 -- 172.21.15.3:0/2781292902 learned_addr learned my addr 172.21.15.3:0/2781292902 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d82c10700 1 --2- 172.21.15.3:0/2781292902 >> v2:172.21.15.153:3300/0 conn(0x7f3d7c140dc0 0x7f3d7c0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d8140d700 1 -- 172.21.15.3:0/2781292902 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701011488 0 0) 0x7f3d7c0cd380 con 0x7f3d7c0b3a40 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d8140d700 1 -- 172.21.15.3:0/2781292902 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d64003570 con 0x7f3d7c0b3a40 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d82c10700 1 -- 172.21.15.3:0/2781292902 >> v1:172.21.15.153:6789/0 conn(0x7f3d7c0b3e10 legacy=0x7f3d7c1406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d8140d700 1 -- 172.21.15.3:0/2781292902 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022852493 0 0) 0x7f3d7c0cd660 con 0x7f3d7c0b3e10 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d82c10700 1 -- 172.21.15.3:0/2781292902 >> v1:172.21.15.3:6789/0 conn(0x7f3d7c0b3a40 legacy=0x7f3d7c0680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d82c10700 1 -- 172.21.15.3:0/2781292902 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0cdc20 con 0x7f3d7c140dc0 2023-10-06T05:00:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d82c10700 1 --2- 172.21.15.3:0/2781292902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c0ccd30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009910 tx=0x7f3d740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f122527244309fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d8140d700 1 -- 172.21.15.3:0/2781292902 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0017f0 con 0x7f3d7c140dc0 2023-10-06T05:00:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d8140d700 1 -- 172.21.15.3:0/2781292902 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74011420 con 0x7f3d7c140dc0 2023-10-06T05:00:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.866+0000 7f3d8140d700 1 -- 172.21.15.3:0/2781292902 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d740117c0 con 0x7f3d7c140dc0 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.867+0000 7f3d84673700 1 -- 172.21.15.3:0/2781292902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c140dc0 msgr2=0x7f3d7c0ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.867+0000 7f3d84673700 1 --2- 172.21.15.3:0/2781292902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c0ccd30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009910 tx=0x7f3d740091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.867+0000 7f3d84673700 1 -- 172.21.15.3:0/2781292902 shutdown_connections 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.867+0000 7f3d84673700 1 --2- 172.21.15.3:0/2781292902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c0ccd30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.867+0000 7f3d84673700 1 -- 172.21.15.3:0/2781292902 >> 172.21.15.3:0/2781292902 conn(0x7f3d7c0ce270 msgr2=0x7f3d7c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.867+0000 7f3d84673700 1 -- 172.21.15.3:0/2781292902 shutdown_connections 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.867+0000 7f3d84673700 1 -- 172.21.15.3:0/2781292902 wait complete. 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.868+0000 7f3d84673700 1 Processor -- start 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.868+0000 7f3d84673700 1 -- start start 2023-10-06T05:00:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d84673700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d7c0b3a40 0x7f3d7c1685c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d84673700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c0b3e10 0x7f3d7c0c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d84673700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d84673700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c3550 con 0x7f3d7c140dc0 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d84673700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c3780 con 0x7f3d7c0b3e10 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d84673700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c3a60 con 0x7f3d7c0b3a40 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37110/0 (socket says 172.21.15.3:37110) 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 -- 172.21.15.3:0/2148319325 learned_addr learned my addr 172.21.15.3:0/2148319325 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d8240f700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d7c0b3a40 0x7f3d7c1685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d81c0e700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c0b3e10 0x7f3d7c0c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 -- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d7c0b3a40 msgr2=0x7f3d7c1685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d7c0b3a40 0x7f3d7c1685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 -- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c0b3e10 msgr2=0x7f3d7c0c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c0b3e10 0x7f3d7c0c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 -- 172.21.15.3:0/2148319325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d74009420 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.869+0000 7f3d82c10700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c156d70 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f3d740098e0 tx=0x7f3d74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.870+0000 7f3d7a7fc700 1 -- 172.21.15.3:0/2148319325 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74010cb0 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.870+0000 7f3d7a7fc700 1 -- 172.21.15.3:0/2148319325 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74010e30 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.870+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c0c3d40 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.870+0000 7f3d7a7fc700 1 -- 172.21.15.3:0/2148319325 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7401d680 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.870+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c0c4300 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.871+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c0b34d0 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.871+0000 7f3d7a7fc700 1 -- 172.21.15.3:0/2148319325 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d7401da70 con 0x7f3d7c140dc0 2023-10-06T05:00:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.871+0000 7f3d7a7fc700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d68066f30 0x7f3d680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:44.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.871+0000 7f3d7a7fc700 1 -- 172.21.15.3:0/2148319325 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d740866d0 con 0x7f3d7c140dc0 2023-10-06T05:00:44.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.871+0000 7f3d8240f700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d68066f30 0x7f3d680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:44.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.872+0000 7f3d8240f700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d68066f30 0x7f3d680693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c001560 tx=0x7f3d6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:44.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:44.876+0000 7f3d7a7fc700 1 -- 172.21.15.3:0/2148319325 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d740537e0 con 0x7f3d7c140dc0 2023-10-06T05:00:45.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.115+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3d7c0644c0 con 0x7f3d7c140dc0 2023-10-06T05:00:45.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.115+0000 7f3d7a7fc700 1 -- 172.21.15.3:0/2148319325 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3d74056e00 con 0x7f3d7c140dc0 2023-10-06T05:00:45.118 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:00:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d68066f30 msgr2=0x7f3d680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d68066f30 0x7f3d680693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c001560 tx=0x7f3d6c009040 comp rx=0 tx=0).stop 2023-10-06T05:00:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d7c140dc0 msgr2=0x7f3d7c156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c156d70 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f3d740098e0 tx=0x7f3d74017000 comp rx=0 tx=0).stop 2023-10-06T05:00:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 shutdown_connections 2023-10-06T05:00:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d68066f30 0x7f3d680693e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d7c140dc0 0x7f3d7c156d70 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d7c0b3a40 0x7f3d7c1685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 --2- 172.21.15.3:0/2148319325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c0b3e10 0x7f3d7c0c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 >> 172.21.15.3:0/2148319325 conn(0x7f3d7c0ce270 msgr2=0x7f3d7c0680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 shutdown_connections 2023-10-06T05:00:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:45.117+0000 7f3d84673700 1 -- 172.21.15.3:0/2148319325 wait complete. 2023-10-06T05:00:45.132 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:00:45.132 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:00:45.132 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T05:00:45.132 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:00:45.132 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:45.132 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:00:45.132 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:00:45.133 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:00:45.133 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:00:45.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:45.133 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T05:00:45.148 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:00:45.149 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41985sec preferred_lft 41985sec 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T05:00:45.150 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:00:45.152 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T05:00:45.152 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:45.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:45.152 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:00:45.213 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T05:00:45.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:45.218 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T05:00:45.274 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T05:00:45.274 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:00:45.274 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:00:45.274 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:00:45.274 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:00:45.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:45.275 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T05:00:45.275 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T05:00:45.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:00:45.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:00:45.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:00:45.275 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T05:00:45.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:45.377 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T05:00:45.391 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:00:45.391 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:00:45.391 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:00:45.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:45.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T05:00:45.457 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:00:45.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:45.457 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:45.512 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:00:45.513 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:45.513 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:45.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:00:45.632 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:45.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:45.698 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:45.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:45.699 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:00:45.754 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:00:45.754 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:00:45.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:00:45.821 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:45.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:45.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T05:00:45.879+0000 7fd79f45e540 -1 init, newargv = 0x55e38b6bfa60 newargc=15 2023-10-06T05:00:45.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[126814]: starting ceph client 2023-10-06T05:00:45.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[126814]: starting fuse 2023-10-06T05:00:45.887 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:45.888 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:45.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:45.889 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:00:45.944 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T05:00:45.945 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:00:45.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:45.945 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T05:00:45.945 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T05:00:45.945 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:00:45.945 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T05:00:45.945 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T05:00:45.946 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T05:00:45.947 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:00:46.068 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.126814.asok 2023-10-06T05:00:46.250 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.126814.asok 2023-10-06T05:00:46.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:46.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.126814.asok status 2023-10-06T05:00:46.404 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:00:46.404 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "126814", 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 8148, 2023-10-06T05:00:46.405 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 8148 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2559196242 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:46.406 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2559196242 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.8148 192.168.0.1:0/2559196242", 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2559196242", 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 79, 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 120, 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:00:46.407 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T05:00:46.408 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T05:00:46.408 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:00:46.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:46.415 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T05:00:46.415 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:46.430 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T05:00:46.430 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:46.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:46.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:00:46.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:00:46.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.653+0000 7f6556302700 1 Processor -- start 2023-10-06T05:00:46.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.653+0000 7f6556302700 1 -- start start 2023-10-06T05:00:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f6556302700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f655015ccc0 0x7f65500d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f6556302700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65500d06a0 0x7f65500b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f6556302700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65500cf6c0 con 0x7f655015c8f0 2023-10-06T05:00:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f6556302700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65500cfb50 con 0x7f655015ccc0 2023-10-06T05:00:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f6556302700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65500d0ad0 con 0x7f65500d06a0 2023-10-06T05:00:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f655015ccc0 0x7f65500d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:46.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f655015ccc0 0x7f65500d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37126/0 (socket says 172.21.15.3:37126) 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 -- 172.21.15.3:0/1153409825 learned_addr learned my addr 172.21.15.3:0/1153409825 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f655489f700 1 --2- 172.21.15.3:0/1153409825 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65500d06a0 0x7f65500b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 -- 172.21.15.3:0/1153409825 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65500d06a0 msgr2=0x7f65500b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 --2- 172.21.15.3:0/1153409825 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65500d06a0 0x7f65500b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 -- 172.21.15.3:0/1153409825 >> v1:172.21.15.153:6789/0 conn(0x7f655015c8f0 legacy=0x7f655015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 -- 172.21.15.3:0/1153409825 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500ceab0 con 0x7f655015ccc0 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654f7fe700 1 --2- 172.21.15.3:0/1153409825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f655015ccc0 0x7f65500d0160 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f65440099b0 tx=0x7f6544009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb444b91346c6be7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654effd700 1 -- 172.21.15.3:0/1153409825 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400b820 con 0x7f655015ccc0 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654effd700 1 -- 172.21.15.3:0/1153409825 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400b980 con 0x7f655015ccc0 2023-10-06T05:00:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.654+0000 7f654effd700 1 -- 172.21.15.3:0/1153409825 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400bd20 con 0x7f655015ccc0 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 -- 172.21.15.3:0/1153409825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f655015ccc0 msgr2=0x7f65500d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 --2- 172.21.15.3:0/1153409825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f655015ccc0 0x7f65500d0160 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f65440099b0 tx=0x7f6544009250 comp rx=0 tx=0).stop 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 -- 172.21.15.3:0/1153409825 shutdown_connections 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 --2- 172.21.15.3:0/1153409825 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f65500d06a0 0x7f65500b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 --2- 172.21.15.3:0/1153409825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f655015ccc0 0x7f65500d0160 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 -- 172.21.15.3:0/1153409825 >> 172.21.15.3:0/1153409825 conn(0x7f6550070310 msgr2=0x7f6550070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 -- 172.21.15.3:0/1153409825 shutdown_connections 2023-10-06T05:00:46.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.655+0000 7f6556302700 1 -- 172.21.15.3:0/1153409825 wait complete. 2023-10-06T05:00:46.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.656+0000 7f6556302700 1 Processor -- start 2023-10-06T05:00:46.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.656+0000 7f6556302700 1 -- start start 2023-10-06T05:00:46.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f6556302700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65500d06a0 0x7f655016ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:46.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f6556302700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f655015c8f0 0x7f655016cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:46.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f6556302700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655015ccc0 0x7f65500c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:46.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f6556302700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65501558e0 con 0x7f65500d06a0 2023-10-06T05:00:46.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f6556302700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550159560 con 0x7f655015ccc0 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f6556302700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6550158c90 con 0x7f655015c8f0 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65500d06a0 0x7f655016ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65500d06a0 0x7f655016ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37136/0 (socket says 172.21.15.3:37136) 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 -- 172.21.15.3:0/4146428534 learned_addr learned my addr 172.21.15.3:0/4146428534 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654f7fe700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f655015c8f0 0x7f655016cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f655489f700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655015ccc0 0x7f65500c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 -- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f655015c8f0 msgr2=0x7f655016cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f655015c8f0 0x7f655016cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 -- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655015ccc0 msgr2=0x7f65500c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655015ccc0 0x7f65500c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.657+0000 7f654ffff700 1 -- 172.21.15.3:0/4146428534 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440094f0 con 0x7f65500d06a0 2023-10-06T05:00:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.658+0000 7f654ffff700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65500d06a0 0x7f655016ae00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f654000c5d0 tx=0x7f6540007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.658+0000 7f6537fff700 1 -- 172.21.15.3:0/4146428534 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540011590 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.658+0000 7f6537fff700 1 -- 172.21.15.3:0/4146428534 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6540011710 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.658+0000 7f6537fff700 1 -- 172.21.15.3:0/4146428534 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540011ad0 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.658+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65500c8010 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.658+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65501591f0 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.659+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6550071680 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.659+0000 7f6537fff700 1 -- 172.21.15.3:0/4146428534 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f654001c460 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.660+0000 7f6537fff700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f653c066f30 0x7f653c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.660+0000 7f6537fff700 1 -- 172.21.15.3:0/4146428534 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6540084310 con 0x7f65500d06a0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.660+0000 7f654f7fe700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f653c066f30 0x7f653c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.660+0000 7f654f7fe700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f653c066f30 0x7f653c0693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f65440099b0 tx=0x7f6544016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:46.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.664+0000 7f6537fff700 1 -- 172.21.15.3:0/4146428534 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6540051420 con 0x7f65500d06a0 2023-10-06T05:00:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.906+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f65500afd40 con 0x7f65500d06a0 2023-10-06T05:00:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.906+0000 7f6537fff700 1 -- 172.21.15.3:0/4146428534 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6540054a40 con 0x7f65500d06a0 2023-10-06T05:00:46.909 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:00:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f653c066f30 msgr2=0x7f653c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f653c066f30 0x7f653c0693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f65440099b0 tx=0x7f6544016000 comp rx=0 tx=0).stop 2023-10-06T05:00:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65500d06a0 msgr2=0x7f655016ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65500d06a0 0x7f655016ae00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f654000c5d0 tx=0x7f6540007f60 comp rx=0 tx=0).stop 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 shutdown_connections 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f653c066f30 0x7f653c0693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655015ccc0 0x7f65500c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65500d06a0 0x7f655016ae00 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 --2- 172.21.15.3:0/4146428534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f655015c8f0 0x7f655016cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 >> 172.21.15.3:0/4146428534 conn(0x7f6550070310 msgr2=0x7f6550070930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.908+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 shutdown_connections 2023-10-06T05:00:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:46.909+0000 7f6556302700 1 -- 172.21.15.3:0/4146428534 wait complete. 2023-10-06T05:00:46.923 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:00:46.923 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:00:46.923 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T05:00:46.923 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:00:46.923 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:46.923 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:00:46.924 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:00:46.924 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:00:46.924 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:00:46.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:46.924 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:46.943 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:00:46.944 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 41986sec preferred_lft 41986sec 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:00:46.945 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T05:00:46.946 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:46.946 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:00:46.946 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:00:46.946 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T05:00:46.946 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:00:46.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:46.946 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:00:47.011 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T05:00:47.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:47.013 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T05:00:47.071 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T05:00:47.072 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:00:47.072 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:00:47.072 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:00:47.072 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:00:47.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:47.072 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T05:00:47.073 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T05:00:47.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:00:47.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:00:47.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:00:47.073 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T05:00:47.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:47.179 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T05:00:47.194 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:00:47.194 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:00:47.194 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:00:47.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:00:47.194 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T05:00:47.259 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T05:00:47.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:47.259 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:47.314 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:00:47.314 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:00:47.314 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:47.369 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T05:00:47.435 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:47.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:47.499 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:47.500 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:47.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:47.500 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T05:00:47.556 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:00:47.556 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T05:00:47.598 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T05:00:47.622 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:00:47.622 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:00:47.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T05:00:47.677+0000 7fbda23ab540 -1 init, newargv = 0x561fa70b39b0 newargc=15 2023-10-06T05:00:47.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[102508]: starting ceph client 2023-10-06T05:00:47.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[102508]: starting fuse 2023-10-06T05:00:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:00:47.689 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:00:47.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:47.690 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T05:00:47.744 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T05:00:47.745 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T05:00:47.745 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:00:47.746 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T05:00:47.747 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T05:00:47.747 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T05:00:47.747 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:00:47.747 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:00:47.747 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T05:00:47.747 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T05:00:47.867 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.102508.asok 2023-10-06T05:00:48.047 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.102508.asok 2023-10-06T05:00:48.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:48.048 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.102508.asok status 2023-10-06T05:00:48.198 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T05:00:48.198 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T05:00:48.198 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:00:48.198 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:00:48.198 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "102508", 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 7480, 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T05:00:48.199 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 7480 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 4115743899 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:48.200 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 4115743899 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.7480 192.168.0.1:0/4115743899", 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/4115743899", 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 79, 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 120, 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T05:00:48.201 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T05:00:48.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:48.207 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T05:00:48.207 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:48.222 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T05:00:48.223 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:48.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:48.223 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:00:48.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-10-06T05:00:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.421+0000 7fed858c1700 1 Processor -- start 2023-10-06T05:00:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.421+0000 7fed858c1700 1 -- start start 2023-10-06T05:00:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed858c1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed80159140 0x7fed80159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed858c1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed800affa0 0x7fed800b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed858c1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed800cb8c0 con 0x7fed800b1fe0 2023-10-06T05:00:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed858c1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed800caf90 con 0x7fed800affa0 2023-10-06T05:00:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed858c1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed800cb270 con 0x7fed80159140 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed7e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed800affa0 0x7fed800b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed7e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed800affa0 0x7fed800b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37138/0 (socket says 172.21.15.3:37138) 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed7e7fc700 1 -- 172.21.15.3:0/280323030 learned_addr learned my addr 172.21.15.3:0/280323030 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.422+0000 7fed7effd700 1 --2- 172.21.15.3:0/280323030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed80159140 0x7fed80159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7e7fc700 1 -- 172.21.15.3:0/280323030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed80159140 msgr2=0x7fed80159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7e7fc700 1 --2- 172.21.15.3:0/280323030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed80159140 0x7fed80159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7e7fc700 1 -- 172.21.15.3:0/280323030 >> v1:172.21.15.153:6789/0 conn(0x7fed800b1fe0 legacy=0x7fed800b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7e7fc700 1 -- 172.21.15.3:0/280323030 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed800cb550 con 0x7fed800affa0 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7e7fc700 1 --2- 172.21.15.3:0/280323030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed800affa0 0x7fed800b0440 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fed680099b0 tx=0x7fed68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab4a6c1e99046c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7dffb700 1 -- 172.21.15.3:0/280323030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68009ea0 con 0x7fed800affa0 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7dffb700 1 -- 172.21.15.3:0/280323030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6800b860 con 0x7fed800affa0 2023-10-06T05:00:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.423+0000 7fed7dffb700 1 -- 172.21.15.3:0/280323030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800bc00 con 0x7fed800affa0 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 -- 172.21.15.3:0/280323030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed800affa0 msgr2=0x7fed800b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 --2- 172.21.15.3:0/280323030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed800affa0 0x7fed800b0440 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fed680099b0 tx=0x7fed68009250 comp rx=0 tx=0).stop 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 -- 172.21.15.3:0/280323030 shutdown_connections 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 --2- 172.21.15.3:0/280323030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed80159140 0x7fed80159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 --2- 172.21.15.3:0/280323030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed800affa0 0x7fed800b0440 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 -- 172.21.15.3:0/280323030 >> 172.21.15.3:0/280323030 conn(0x7fed80070310 msgr2=0x7fed80070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 -- 172.21.15.3:0/280323030 shutdown_connections 2023-10-06T05:00:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.424+0000 7fed858c1700 1 -- 172.21.15.3:0/280323030 wait complete. 2023-10-06T05:00:48.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.425+0000 7fed858c1700 1 Processor -- start 2023-10-06T05:00:48.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.425+0000 7fed858c1700 1 -- start start 2023-10-06T05:00:48.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed858c1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed800affa0 0x7fed8015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed858c1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed800b1fe0 0x7fed800c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed858c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed80159140 0x7fed800c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed858c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed800c5bc0 con 0x7fed80159140 2023-10-06T05:00:48.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed858c1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed80156ab0 con 0x7fed800b1fe0 2023-10-06T05:00:48.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed858c1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed800c4cd0 con 0x7fed800affa0 2023-10-06T05:00:48.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed80159140 0x7fed800c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed80159140 0x7fed800c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37150/0 (socket says 172.21.15.3:37150) 2023-10-06T05:00:48.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 -- 172.21.15.3:0/3321594519 learned_addr learned my addr 172.21.15.3:0/3321594519 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:48.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7e7fc700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed800b1fe0 0x7fed800c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7effd700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed800affa0 0x7fed8015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 -- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed800affa0 msgr2=0x7fed8015c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed800affa0 0x7fed8015c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 -- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed800b1fe0 msgr2=0x7fed800c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed800b1fe0 0x7fed800c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 -- 172.21.15.3:0/3321594519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed680094f0 con 0x7fed80159140 2023-10-06T05:00:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7effd700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed800affa0 0x7fed8015c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed7f7fe700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed80159140 0x7fed800c8510 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c8e0 tx=0x7fed78001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed76ffd700 1 -- 172.21.15.3:0/3321594519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78012630 con 0x7fed80159140 2023-10-06T05:00:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed76ffd700 1 -- 172.21.15.3:0/3321594519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed780127b0 con 0x7fed80159140 2023-10-06T05:00:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed800c4fb0 con 0x7fed80159140 2023-10-06T05:00:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.426+0000 7fed76ffd700 1 -- 172.21.15.3:0/3321594519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78012b70 con 0x7fed80159140 2023-10-06T05:00:48.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.427+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed800c5570 con 0x7fed80159140 2023-10-06T05:00:48.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.428+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed800c5d30 con 0x7fed80159140 2023-10-06T05:00:48.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.428+0000 7fed76ffd700 1 -- 172.21.15.3:0/3321594519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed78003030 con 0x7fed80159140 2023-10-06T05:00:48.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.428+0000 7fed76ffd700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed60066f30 0x7fed600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.429+0000 7fed76ffd700 1 -- 172.21.15.3:0/3321594519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fed780845f0 con 0x7fed80159140 2023-10-06T05:00:48.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.429+0000 7fed7effd700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed60066f30 0x7fed600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.429+0000 7fed7effd700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed60066f30 0x7fed600693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fed70009d80 tx=0x7fed70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:48.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.433+0000 7fed76ffd700 1 -- 172.21.15.3:0/3321594519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed78051700 con 0x7fed80159140 2023-10-06T05:00:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.626+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7fed8004c380 con 0x7fed80159140 2023-10-06T05:00:48.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.630+0000 7fed76ffd700 1 -- 172.21.15.3:0/3321594519 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v55)=0 v55) v1 ==== 121+0+0 (secure 0 0 0) 0x7fed78054d20 con 0x7fed80159140 2023-10-06T05:00:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed60066f30 msgr2=0x7fed600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed60066f30 0x7fed600693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fed70009d80 tx=0x7fed70008040 comp rx=0 tx=0).stop 2023-10-06T05:00:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed80159140 msgr2=0x7fed800c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed80159140 0x7fed800c8510 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c8e0 tx=0x7fed78001710 comp rx=0 tx=0).stop 2023-10-06T05:00:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 shutdown_connections 2023-10-06T05:00:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed60066f30 0x7fed600693e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed80159140 0x7fed800c8510 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed800affa0 0x7fed8015c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 --2- 172.21.15.3:0/3321594519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed800b1fe0 0x7fed800c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 >> 172.21.15.3:0/3321594519 conn(0x7fed80070310 msgr2=0x7fed800cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 shutdown_connections 2023-10-06T05:00:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.632+0000 7fed858c1700 1 -- 172.21.15.3:0/3321594519 wait complete. 2023-10-06T05:00:48.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2023-10-06T05:00:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.766+0000 7f741a31e700 1 Processor -- start 2023-10-06T05:00:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.766+0000 7f741a31e700 1 -- start start 2023-10-06T05:00:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f741a31e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f741415cb30 0x7f74140cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f741a31e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74140afec0 0x7f74140cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f741a31e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74140cf0f0 con 0x7f741415c760 2023-10-06T05:00:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f741a31e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741415cf80 con 0x7f741415cb30 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f741a31e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74140ce7c0 con 0x7f74140afec0 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f74137fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f741415cb30 0x7f74140cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f7413fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f741415c760 0x7f741415db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60996/0 (socket says 172.21.15.3:60996) 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f7413fff700 1 -- 172.21.15.3:0/3303687755 learned_addr learned my addr 172.21.15.3:0/3303687755 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.767+0000 7f74188bb700 1 --2- 172.21.15.3:0/3303687755 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74140afec0 0x7f74140cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f74137fe700 1 -- 172.21.15.3:0/3303687755 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74140afec0 msgr2=0x7f74140cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f74137fe700 1 --2- 172.21.15.3:0/3303687755 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74140afec0 0x7f74140cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f74137fe700 1 -- 172.21.15.3:0/3303687755 >> v1:172.21.15.3:6789/0 conn(0x7f741415c760 legacy=0x7f741415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f7412ffd700 1 -- 172.21.15.3:0/3303687755 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949196233 0 0) 0x7f74140cf0f0 con 0x7f741415c760 2023-10-06T05:00:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f74137fe700 1 -- 172.21.15.3:0/3303687755 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74140ceaa0 con 0x7f741415cb30 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f74137fe700 1 --2- 172.21.15.3:0/3303687755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74140cb3c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f74040098e0 tx=0x7f74040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a15c1ffac95d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f7412ffd700 1 -- 172.21.15.3:0/3303687755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404009e80 con 0x7f741415cb30 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f7412ffd700 1 -- 172.21.15.3:0/3303687755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400b860 con 0x7f741415cb30 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f7412ffd700 1 -- 172.21.15.3:0/3303687755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400bbe0 con 0x7f741415cb30 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f741a31e700 1 -- 172.21.15.3:0/3303687755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 msgr2=0x7f74140cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f741a31e700 1 --2- 172.21.15.3:0/3303687755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74140cb3c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f74040098e0 tx=0x7f74040091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f741a31e700 1 -- 172.21.15.3:0/3303687755 shutdown_connections 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f741a31e700 1 --2- 172.21.15.3:0/3303687755 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74140afec0 0x7f74140cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f741a31e700 1 --2- 172.21.15.3:0/3303687755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74140cb3c0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.768+0000 7f741a31e700 1 -- 172.21.15.3:0/3303687755 >> 172.21.15.3:0/3303687755 conn(0x7f741406f940 msgr2=0x7f7414070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.769+0000 7f741a31e700 1 -- 172.21.15.3:0/3303687755 shutdown_connections 2023-10-06T05:00:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.769+0000 7f741a31e700 1 -- 172.21.15.3:0/3303687755 wait complete. 2023-10-06T05:00:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 Processor -- start 2023-10-06T05:00:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 -- start start 2023-10-06T05:00:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74140afec0 0x7f74140b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f741415c760 0x7f741416a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74141589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741416aa80 con 0x7f741415cb30 2023-10-06T05:00:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74140c7a30 con 0x7f74140afec0 2023-10-06T05:00:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.770+0000 7f741a31e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74140b5bb0 con 0x7f741415c760 2023-10-06T05:00:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74141589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74141589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37182/0 (socket says 172.21.15.3:37182) 2023-10-06T05:00:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 -- 172.21.15.3:0/96679759 learned_addr learned my addr 172.21.15.3:0/96679759 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f7413fff700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74140afec0 0x7f74140b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74137fe700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f741415c760 0x7f741416a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 -- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f741415c760 msgr2=0x7f741416a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f741415c760 0x7f741416a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 -- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74140afec0 msgr2=0x7f74140b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74140afec0 0x7f74140b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 -- 172.21.15.3:0/96679759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404009420 con 0x7f741415cb30 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f74188bb700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74141589d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f740c009fa0 tx=0x7f740c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f740bfff700 1 -- 172.21.15.3:0/96679759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00ebc0 con 0x7f741415cb30 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f740bfff700 1 -- 172.21.15.3:0/96679759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c00ed20 con 0x7f741415cb30 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f740bfff700 1 -- 172.21.15.3:0/96679759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c014670 con 0x7f741415cb30 2023-10-06T05:00:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74140b5e90 con 0x7f741415cb30 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.771+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7414158f10 con 0x7f741415cb30 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.772+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74140c7500 con 0x7f741415cb30 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.773+0000 7f740bfff700 1 -- 172.21.15.3:0/96679759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f740c0149e0 con 0x7f741415cb30 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.773+0000 7f740bfff700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f73f4066f30 0x7f73f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.773+0000 7f740bfff700 1 -- 172.21.15.3:0/96679759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f740c084d30 con 0x7f741415cb30 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.773+0000 7f7413fff700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f73f4066f30 0x7f73f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.774+0000 7f7413fff700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f73f4066f30 0x7f73f40693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f74140aee90 tx=0x7f73fc007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.777+0000 7f740bfff700 1 -- 172.21.15.3:0/96679759 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f740c051e40 con 0x7f741415cb30 2023-10-06T05:00:48.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.971+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7f74140b6080 con 0x7f741415cb30 2023-10-06T05:00:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.974+0000 7f740bfff700 1 -- 172.21.15.3:0/96679759 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v56)=0 v56) v1 ==== 119+0+0 (secure 0 0 0) 0x7f740c055460 con 0x7f741415cb30 2023-10-06T05:00:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f73f4066f30 msgr2=0x7f73f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f73f4066f30 0x7f73f40693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f74140aee90 tx=0x7f73fc007d30 comp rx=0 tx=0).stop 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 msgr2=0x7f74141589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74141589d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f740c009fa0 tx=0x7f740c009900 comp rx=0 tx=0).stop 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 shutdown_connections 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f73f4066f30 0x7f73f40693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74140afec0 0x7f74140b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741415cb30 0x7f74141589d0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 --2- 172.21.15.3:0/96679759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f741415c760 0x7f741416a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 >> 172.21.15.3:0/96679759 conn(0x7f741406f940 msgr2=0x7f741415a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 shutdown_connections 2023-10-06T05:00:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:48.977+0000 7f741a31e700 1 -- 172.21.15.3:0/96679759 wait complete. 2023-10-06T05:00:48.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2023-10-06T05:00:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.110+0000 7fc6317e9700 1 Processor -- start 2023-10-06T05:00:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.110+0000 7fc6317e9700 1 -- start start 2023-10-06T05:00:49.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.111+0000 7fc6317e9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc62c157980 0x7fc62c0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.111+0000 7fc6317e9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 0x7fc62c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.111+0000 7fc6317e9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c0cb0f0 con 0x7fc62c1575b0 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.111+0000 7fc6317e9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c06de00 con 0x7fc62c157980 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.111+0000 7fc6317e9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0b24c0 con 0x7fc62c0b2040 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.111+0000 7fc62affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc62c1575b0 0x7fc62c06cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:32768/0 (socket says 172.21.15.3:32768) 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.111+0000 7fc62affd700 1 -- 172.21.15.3:0/2216526452 learned_addr learned my addr 172.21.15.3:0/2216526452 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc62a7fc700 1 --2- 172.21.15.3:0/2216526452 >> v2:172.21.15.153:3300/0 conn(0x7fc62c157980 0x7fc62c0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc62b7fe700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 0x7fc62c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154353629 0 0) 0x7fc62c0cb0f0 con 0x7fc62c1575b0 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc618003570 con 0x7fc62c1575b0 2023-10-06T05:00:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1278392937 0 0) 0x7fc618003570 con 0x7fc62c1575b0 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 msgr2=0x7fc62c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 0x7fc62c0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 >> v2:172.21.15.153:3300/0 conn(0x7fc62c157980 msgr2=0x7fc62c0b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 --2- 172.21.15.3:0/2216526452 >> v2:172.21.15.153:3300/0 conn(0x7fc62c157980 0x7fc62c0b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c0caae0 con 0x7fc62c1575b0 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc6240017f0 con 0x7fc62c1575b0 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc62a7fc700 1 --2- 172.21.15.3:0/2216526452 >> v2:172.21.15.153:3300/0 conn(0x7fc62c157980 0x7fc62c0b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 >> v1:172.21.15.3:6789/0 conn(0x7fc62c1575b0 legacy=0x7fc62c06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc618003a90 0x7fc618003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc618004810 0x7fc618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 0x7fc6180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0b24c0 con 0x7fc618004810 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61800ba60 con 0x7fc618003a90 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61800bbd0 con 0x7fc62c0b2040 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc62a7fc700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc618004810 0x7fc618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc62b7fe700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 0x7fc6180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.112+0000 7fc62affd700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc618003a90 0x7fc618003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62a7fc700 1 -- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 msgr2=0x7fc6180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62a7fc700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc62c0b2040 0x7fc6180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62a7fc700 1 -- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc618003a90 msgr2=0x7fc618003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62a7fc700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc618003a90 0x7fc618003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62a7fc700 1 -- 172.21.15.3:0/2216526452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc618003570 con 0x7fc618004810 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62affd700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc618003a90 0x7fc618003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62a7fc700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc618004810 0x7fc618004c80 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fc614009d70 tx=0x7fc6140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b4fb3cc2e1e316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614015070 con 0x7fc618004810 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614009740 con 0x7fc618004810 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc629ffb700 1 -- 172.21.15.3:0/2216526452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614010e60 con 0x7fc618004810 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc6317e9700 1 -- 172.21.15.3:0/2216526452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc618004810 msgr2=0x7fc618004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc6317e9700 1 --2- 172.21.15.3:0/2216526452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc618004810 0x7fc618004c80 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fc614009d70 tx=0x7fc6140091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc62b7fe700 1 -- 172.21.15.3:0/2216526452 reap_dead start 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc6317e9700 1 -- 172.21.15.3:0/2216526452 shutdown_connections 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc6317e9700 1 -- 172.21.15.3:0/2216526452 >> 172.21.15.3:0/2216526452 conn(0x7fc62c066fb0 msgr2=0x7fc62c0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:49.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.113+0000 7fc6317e9700 1 -- 172.21.15.3:0/2216526452 shutdown_connections 2023-10-06T05:00:49.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.114+0000 7fc6317e9700 1 -- 172.21.15.3:0/2216526452 wait complete. 2023-10-06T05:00:49.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.115+0000 7fc6317e9700 1 Processor -- start 2023-10-06T05:00:49.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.115+0000 7fc6317e9700 1 -- start start 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.115+0000 7fc6317e9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc618004810 0x7fc62c163950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.115+0000 7fc6317e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc62c166ee0 0x7fc62c166760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc6317e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc62c0c19c0 0x7fc62c1651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc6317e9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c166ca0 con 0x7fc62c0c19c0 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc6317e9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c14fc60 con 0x7fc62c166ee0 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc6317e9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c167350 con 0x7fc618004810 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc62c0c19c0 0x7fc62c1651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc62c0c19c0 0x7fc62c1651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37208/0 (socket says 172.21.15.3:37208) 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 -- 172.21.15.3:0/52174792 learned_addr learned my addr 172.21.15.3:0/52174792 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62affd700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc618004810 0x7fc62c163950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62a7fc700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc62c166ee0 0x7fc62c166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 -- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc618004810 msgr2=0x7fc62c163950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc618004810 0x7fc62c163950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 -- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc62c166ee0 msgr2=0x7fc62c166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc62c166ee0 0x7fc62c166760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 -- 172.21.15.3:0/52174792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c167630 con 0x7fc62c0c19c0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc62b7fe700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc62c0c19c0 0x7fc62c1651b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc62c069960 tx=0x7fc61c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc622ffd700 1 -- 172.21.15.3:0/52174792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011ed0 con 0x7fc62c0c19c0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc622ffd700 1 -- 172.21.15.3:0/52174792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c00f910 con 0x7fc62c0c19c0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c167910 con 0x7fc62c0c19c0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc622ffd700 1 -- 172.21.15.3:0/52174792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00fcb0 con 0x7fc62c0c19c0 2023-10-06T05:00:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.116+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c167ed0 con 0x7fc62c0c19c0 2023-10-06T05:00:49.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.117+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c04c380 con 0x7fc62c0c19c0 2023-10-06T05:00:49.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.118+0000 7fc622ffd700 1 -- 172.21.15.3:0/52174792 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc61c014650 con 0x7fc62c0c19c0 2023-10-06T05:00:49.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.118+0000 7fc622ffd700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc60c066f30 0x7fc60c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.118+0000 7fc622ffd700 1 -- 172.21.15.3:0/52174792 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc61c084c40 con 0x7fc62c0c19c0 2023-10-06T05:00:49.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.119+0000 7fc62affd700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc60c066f30 0x7fc60c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.119+0000 7fc62affd700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc60c066f30 0x7fc60c0693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc624002120 tx=0x7fc624009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.122+0000 7fc622ffd700 1 -- 172.21.15.3:0/52174792 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc61c051d30 con 0x7fc62c0c19c0 2023-10-06T05:00:49.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.316+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7fc62c06d840 con 0x7fc62c0c19c0 2023-10-06T05:00:49.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.322+0000 7fc622ffd700 1 -- 172.21.15.3:0/52174792 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v57)=0 v57) v1 ==== 128+0+0 (secure 0 0 0) 0x7fc61c0220c0 con 0x7fc62c0c19c0 2023-10-06T05:00:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc60c066f30 msgr2=0x7fc60c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc60c066f30 0x7fc60c0693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc624002120 tx=0x7fc624009040 comp rx=0 tx=0).stop 2023-10-06T05:00:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc62c0c19c0 msgr2=0x7fc62c1651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc62c0c19c0 0x7fc62c1651b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc62c069960 tx=0x7fc61c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 shutdown_connections 2023-10-06T05:00:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc60c066f30 0x7fc60c0693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc62c0c19c0 0x7fc62c1651b0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc618004810 0x7fc62c163950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 --2- 172.21.15.3:0/52174792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc62c166ee0 0x7fc62c166760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 >> 172.21.15.3:0/52174792 conn(0x7fc62c066fb0 msgr2=0x7fc62c151c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 shutdown_connections 2023-10-06T05:00:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.324+0000 7fc6317e9700 1 -- 172.21.15.3:0/52174792 wait complete. 2023-10-06T05:00:49.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-10-06T05:00:49.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.458+0000 7f698759e700 1 Processor -- start 2023-10-06T05:00:49.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.458+0000 7f698759e700 1 -- start start 2023-10-06T05:00:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 0x7f69880698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698759e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f69880c86f0 0x7f69880c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698759e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6988068e90 con 0x7f69880679c0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698759e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6988069450 con 0x7f69880c86f0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6988069e20 con 0x7f69881591b0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f6985d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f69880679c0 0x7f6988067df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:32780/0 (socket says 172.21.15.3:32780) 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f6985d9b700 1 -- 172.21.15.3:0/2761952153 learned_addr learned my addr 172.21.15.3:0/2761952153 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698659c700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 0x7f69880698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f6986d9d700 1 --2- 172.21.15.3:0/2761952153 >> v2:172.21.15.153:3300/0 conn(0x7f69880c86f0 0x7f69880c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566601121 0 0) 0x7f6988068e90 con 0x7f69880679c0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.459+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6974003570 con 0x7f69880679c0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3167578885 0 0) 0x7f6974003570 con 0x7f69880679c0 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 msgr2=0x7f69880698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 0x7f69880698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 >> v2:172.21.15.153:3300/0 conn(0x7f69880c86f0 msgr2=0x7f69880c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 --2- 172.21.15.3:0/2761952153 >> v2:172.21.15.153:3300/0 conn(0x7f69880c86f0 0x7f69880c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880689c0 con 0x7f69880679c0 2023-10-06T05:00:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6970001470 con 0x7f69880679c0 2023-10-06T05:00:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 0x7f69880698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f6986d9d700 1 --2- 172.21.15.3:0/2761952153 >> v2:172.21.15.153:3300/0 conn(0x7f69880c86f0 0x7f69880c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 >> v1:172.21.15.3:6789/0 conn(0x7f69880679c0 legacy=0x7f6988067df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f6974003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 0x7f6974004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6974004d50 0x7f69740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988069450 con 0x7f6974003a60 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988069e20 con 0x7f6974004d50 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697400ba60 con 0x7f69881591b0 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f6974003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f6985d9b700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 0x7f6974004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f6986d9d700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6974004d50 0x7f69740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 -- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 msgr2=0x7f6974004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69881591b0 0x7f6974004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 -- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6974004d50 msgr2=0x7f69740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6974004d50 0x7f69740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 -- 172.21.15.3:0/2761952153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980009a20 con 0x7f6974003a60 2023-10-06T05:00:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698659c700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f6974003e70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f698000e3c0 tx=0x7f698000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7822e46214f5edcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.460+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6980009c50 con 0x7f6974003a60 2023-10-06T05:00:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698000f040 con 0x7f6974003a60 2023-10-06T05:00:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698559a700 1 -- 172.21.15.3:0/2761952153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000ee40 con 0x7f6974003a60 2023-10-06T05:00:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698759e700 1 -- 172.21.15.3:0/2761952153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 msgr2=0x7f6974003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698759e700 1 --2- 172.21.15.3:0/2761952153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f6974003e70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f698000e3c0 tx=0x7f698000b040 comp rx=0 tx=0).stop 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f6986d9d700 1 -- 172.21.15.3:0/2761952153 reap_dead start 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698759e700 1 -- 172.21.15.3:0/2761952153 shutdown_connections 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698759e700 1 -- 172.21.15.3:0/2761952153 >> 172.21.15.3:0/2761952153 conn(0x7f69880ce0b0 msgr2=0x7f69880ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698759e700 1 -- 172.21.15.3:0/2761952153 shutdown_connections 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.461+0000 7f698759e700 1 -- 172.21.15.3:0/2761952153 wait complete. 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.462+0000 7f698759e700 1 Processor -- start 2023-10-06T05:00:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.462+0000 7f698759e700 1 -- start start 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f69880b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69880b2d80 0x7f698813bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f698813e760 0x7f698813ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698813f100 con 0x7f6974003a60 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698759e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b2360 con 0x7f69880b2d80 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b3db0 con 0x7f698813e760 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f69880b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f69880b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37224/0 (socket says 172.21.15.3:37224) 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 -- 172.21.15.3:0/259310530 learned_addr learned my addr 172.21.15.3:0/259310530 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f6985d9b700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69880b2d80 0x7f698813bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f6986d9d700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f698813e760 0x7f698813ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 -- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f698813e760 msgr2=0x7f698813ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f698813e760 0x7f698813ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 -- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69880b2d80 msgr2=0x7f698813bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69880b2d80 0x7f698813bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 -- 172.21.15.3:0/259310530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880b1750 con 0x7f6974003a60 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.463+0000 7f698659c700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f69880b10c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f69800099f0 tx=0x7f698001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.464+0000 7f697e7fc700 1 -- 172.21.15.3:0/259310530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6980029480 con 0x7f6974003a60 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.464+0000 7f697e7fc700 1 -- 172.21.15.3:0/259310530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6980029600 con 0x7f6974003a60 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.464+0000 7f697e7fc700 1 -- 172.21.15.3:0/259310530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69800299c0 con 0x7f6974003a60 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.464+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69880b1a30 con 0x7f6974003a60 2023-10-06T05:00:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.464+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69880b1ff0 con 0x7f6974003a60 2023-10-06T05:00:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.465+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698804c380 con 0x7f6974003a60 2023-10-06T05:00:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.465+0000 7f697e7fc700 1 -- 172.21.15.3:0/259310530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6980003040 con 0x7f6974003a60 2023-10-06T05:00:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.465+0000 7f697e7fc700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.465+0000 7f697e7fc700 1 -- 172.21.15.3:0/259310530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6980086980 con 0x7f6974003a60 2023-10-06T05:00:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.466+0000 7f6985d9b700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.466+0000 7f6985d9b700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6968066f30 0x7f69680693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6970001620 tx=0x7f69880693c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.470+0000 7f697e7fc700 1 -- 172.21.15.3:0/259310530 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6980053a90 con 0x7f6974003a60 2023-10-06T05:00:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.666+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f69880c6630 con 0x7f6974003a60 2023-10-06T05:00:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.666+0000 7f697e7fc700 1 -- 172.21.15.3:0/259310530 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v57) v1 ==== 106+0+4 (secure 0 0 0) 0x7f69800570b0 con 0x7f6974003a60 2023-10-06T05:00:49.670 INFO:teuthology.orchestra.run.smithi003.stdout:100 2023-10-06T05:00:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6968066f30 msgr2=0x7f69680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6968066f30 0x7f69680693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6970001620 tx=0x7f69880693c0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 msgr2=0x7f69880b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f69880b10c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f69800099f0 tx=0x7f698001a000 comp rx=0 tx=0).stop 2023-10-06T05:00:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 shutdown_connections 2023-10-06T05:00:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f698813e760 0x7f698813ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6974003a60 0x7f69880b10c0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 --2- 172.21.15.3:0/259310530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69880b2d80 0x7f698813bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 >> 172.21.15.3:0/259310530 conn(0x7f69880ce0b0 msgr2=0x7f698813ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 shutdown_connections 2023-10-06T05:00:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.668+0000 7f698759e700 1 -- 172.21.15.3:0/259310530 wait complete. 2023-10-06T05:00:49.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-10-06T05:00:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.803+0000 7fdc46354700 1 Processor -- start 2023-10-06T05:00:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.803+0000 7fdc46354700 1 -- start start 2023-10-06T05:00:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.803+0000 7fdc46354700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdc40159eb0 0x7fdc400ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.803+0000 7fdc46354700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc400cef10 0x7fdc4015da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc46354700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc400afca0 con 0x7fdc400cfb80 2023-10-06T05:00:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc46354700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4015d510 con 0x7fdc400cef10 2023-10-06T05:00:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc46354700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4015a480 con 0x7fdc40159eb0 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc400cef10 0x7fdc4015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc400cef10 0x7fdc4015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37240/0 (socket says 172.21.15.3:37240) 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 -- 172.21.15.3:0/4025626607 learned_addr learned my addr 172.21.15.3:0/4025626607 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc3f7fe700 1 --2- 172.21.15.3:0/4025626607 >> v2:172.21.15.153:3300/0 conn(0x7fdc40159eb0 0x7fdc400ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 -- 172.21.15.3:0/4025626607 >> v2:172.21.15.153:3300/0 conn(0x7fdc40159eb0 msgr2=0x7fdc400ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 --2- 172.21.15.3:0/4025626607 >> v2:172.21.15.153:3300/0 conn(0x7fdc40159eb0 0x7fdc400ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 -- 172.21.15.3:0/4025626607 >> v1:172.21.15.153:6789/0 conn(0x7fdc400cfb80 legacy=0x7fdc400af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 -- 172.21.15.3:0/4025626607 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4015a760 con 0x7fdc400cef10 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc3f7fe700 1 --2- 172.21.15.3:0/4025626607 >> v2:172.21.15.153:3300/0 conn(0x7fdc40159eb0 0x7fdc400ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc448f1700 1 --2- 172.21.15.3:0/4025626607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cef10 0x7fdc4015da10 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdc400cb2e0 tx=0x7fdc38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52cf6f6a162f1e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc3effd700 1 -- 172.21.15.3:0/4025626607 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800e070 con 0x7fdc400cef10 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc3effd700 1 -- 172.21.15.3:0/4025626607 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3800ba10 con 0x7fdc400cef10 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.804+0000 7fdc3effd700 1 -- 172.21.15.3:0/4025626607 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380092e0 con 0x7fdc400cef10 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 -- 172.21.15.3:0/4025626607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cef10 msgr2=0x7fdc4015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 --2- 172.21.15.3:0/4025626607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cef10 0x7fdc4015da10 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdc400cb2e0 tx=0x7fdc38009840 comp rx=0 tx=0).stop 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 -- 172.21.15.3:0/4025626607 shutdown_connections 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 --2- 172.21.15.3:0/4025626607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cef10 0x7fdc4015da10 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 --2- 172.21.15.3:0/4025626607 >> v2:172.21.15.153:3300/0 conn(0x7fdc40159eb0 0x7fdc400ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 -- 172.21.15.3:0/4025626607 >> 172.21.15.3:0/4025626607 conn(0x7fdc40070310 msgr2=0x7fdc40070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 -- 172.21.15.3:0/4025626607 shutdown_connections 2023-10-06T05:00:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.805+0000 7fdc46354700 1 -- 172.21.15.3:0/4025626607 wait complete. 2023-10-06T05:00:49.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.806+0000 7fdc46354700 1 Processor -- start 2023-10-06T05:00:49.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.806+0000 7fdc46354700 1 -- start start 2023-10-06T05:00:49.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc46354700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc400cef10 0x7fdc400c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc46354700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cfb80 0x7fdc401553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc46354700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdc40159eb0 0x7fdc40160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc46354700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400c7a30 con 0x7fdc400cfb80 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc46354700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400b43e0 con 0x7fdc400cef10 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc46354700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400b4f10 con 0x7fdc40159eb0 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cfb80 0x7fdc401553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cfb80 0x7fdc401553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37248/0 (socket says 172.21.15.3:37248) 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 -- 172.21.15.3:0/2999416435 learned_addr learned my addr 172.21.15.3:0/2999416435 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3ffff700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc400cef10 0x7fdc400c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc448f1700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdc40159eb0 0x7fdc40160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 -- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdc40159eb0 msgr2=0x7fdc40160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdc40159eb0 0x7fdc40160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 -- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc400cef10 msgr2=0x7fdc400c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc400cef10 0x7fdc400c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc3f7fe700 1 -- 172.21.15.3:0/2999416435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38009ae0 con 0x7fdc400cfb80 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.807+0000 7fdc448f1700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdc40159eb0 0x7fdc40160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.808+0000 7fdc3f7fe700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cfb80 0x7fdc401553e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c880 tx=0x7fdc28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.808+0000 7fdc37fff700 1 -- 172.21.15.3:0/2999416435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28011600 con 0x7fdc400cfb80 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.808+0000 7fdc37fff700 1 -- 172.21.15.3:0/2999416435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc28011780 con 0x7fdc400cfb80 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.808+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc40155920 con 0x7fdc400cfb80 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.808+0000 7fdc37fff700 1 -- 172.21.15.3:0/2999416435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28011b40 con 0x7fdc400cfb80 2023-10-06T05:00:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.808+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400b4670 con 0x7fdc400cfb80 2023-10-06T05:00:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.809+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4004c380 con 0x7fdc400cfb80 2023-10-06T05:00:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.809+0000 7fdc37fff700 1 -- 172.21.15.3:0/2999416435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc28018350 con 0x7fdc400cfb80 2023-10-06T05:00:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.810+0000 7fdc37fff700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc20066f30 0x7fdc200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.810+0000 7fdc37fff700 1 -- 172.21.15.3:0/2999416435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdc28084530 con 0x7fdc400cfb80 2023-10-06T05:00:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.810+0000 7fdc3ffff700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc20066f30 0x7fdc200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.810+0000 7fdc3ffff700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc20066f30 0x7fdc200693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdc30003a30 tx=0x7fdc30007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:49.814+0000 7fdc37fff700 1 -- 172.21.15.3:0/2999416435 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc28051640 con 0x7fdc400cfb80 2023-10-06T05:00:50.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.007+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7fdc400b5050 con 0x7fdc400cfb80 2023-10-06T05:00:50.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.013+0000 7fdc37fff700 1 -- 172.21.15.3:0/2999416435 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v58)=0 v58) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdc28054c60 con 0x7fdc400cfb80 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc20066f30 msgr2=0x7fdc200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc20066f30 0x7fdc200693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdc30003a30 tx=0x7fdc30007ea0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cfb80 msgr2=0x7fdc401553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cfb80 0x7fdc401553e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c880 tx=0x7fdc28008040 comp rx=0 tx=0).stop 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 shutdown_connections 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fdc20066f30 0x7fdc200693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc400cef10 0x7fdc400c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdc40159eb0 0x7fdc40160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 --2- 172.21.15.3:0/2999416435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc400cfb80 0x7fdc401553e0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 >> 172.21.15.3:0/2999416435 conn(0x7fdc40070310 msgr2=0x7fdc4015c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 shutdown_connections 2023-10-06T05:00:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.015+0000 7fdc46354700 1 -- 172.21.15.3:0/2999416435 wait complete. 2023-10-06T05:00:50.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-10-06T05:00:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.148+0000 7f66e4e1d700 1 Processor -- start 2023-10-06T05:00:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.148+0000 7f66e4e1d700 1 -- start start 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66e4e1d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f66e00af290 0x7f66e00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66e4e1d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e00cb190 con 0x7f66e00af660 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66e4e1d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e0159950 con 0x7f66e00c5c20 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66e4e1d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e00cd6a0 con 0x7f66e00af290 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66deffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66e00af660 0x7f66e00ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:32782/0 (socket says 172.21.15.3:32782) 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66deffd700 1 -- 172.21.15.3:0/168428774 learned_addr learned my addr 172.21.15.3:0/168428774 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66df7fe700 1 --2- 172.21.15.3:0/168428774 >> v2:172.21.15.153:3300/0 conn(0x7f66e00af290 0x7f66e00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264845985 0 0) 0x7f66e00cb190 con 0x7f66e00af660 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.149+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c8003570 con 0x7f66e00af660 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231381327 0 0) 0x7f66e0159950 con 0x7f66e00c5c20 2023-10-06T05:00:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e00cb190 con 0x7f66e00c5c20 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1081616625 0 0) 0x7f66c8003570 con 0x7f66e00af660 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 >> v2:172.21.15.153:3300/0 conn(0x7f66e00af290 msgr2=0x7f66e00ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 --2- 172.21.15.3:0/168428774 >> v2:172.21.15.153:3300/0 conn(0x7f66e00af290 0x7f66e00ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 >> v1:172.21.15.153:6789/0 conn(0x7f66e00c5c20 legacy=0x7f66e00cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e00cb3f0 con 0x7f66e00af660 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f66d40015f0 con 0x7f66e00af660 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 >> v1:172.21.15.3:6789/0 conn(0x7f66e00af660 legacy=0x7f66e00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66c8004050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66c8004810 0x7f66c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66c8005210 0x7f66c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e00cd6a0 con 0x7f66c8003c40 2023-10-06T05:00:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c800c130 con 0x7f66c8005210 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c800c2f0 con 0x7f66c8004810 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66df7fe700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66c8004050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66deffd700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66c8004810 0x7f66c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66dffff700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66c8005210 0x7f66c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66df7fe700 1 -- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66c8004810 msgr2=0x7f66c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66df7fe700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66c8004810 0x7f66c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66df7fe700 1 -- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66c8005210 msgr2=0x7f66c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66df7fe700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66c8005210 0x7f66c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66df7fe700 1 -- 172.21.15.3:0/168428774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8003570 con 0x7f66c8003c40 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66dffff700 1 -- 172.21.15.3:0/168428774 reap_dead start 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.150+0000 7f66df7fe700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66c8004050 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009bf0 tx=0x7f66d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214f469132073e61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0005a00 con 0x7f66c8003c40 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d0005b60 con 0x7f66c8003c40 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66de7fc700 1 -- 172.21.15.3:0/168428774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000e710 con 0x7f66c8003c40 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66e4e1d700 1 -- 172.21.15.3:0/168428774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 msgr2=0x7f66c8004050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66c8004050 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009bf0 tx=0x7f66d0009120 comp rx=0 tx=0).stop 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66e4e1d700 1 -- 172.21.15.3:0/168428774 shutdown_connections 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/168428774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66c8004050 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66e4e1d700 1 -- 172.21.15.3:0/168428774 >> 172.21.15.3:0/168428774 conn(0x7f66e0089dd0 msgr2=0x7f66e00996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66e4e1d700 1 -- 172.21.15.3:0/168428774 shutdown_connections 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.151+0000 7f66e4e1d700 1 -- 172.21.15.3:0/168428774 wait complete. 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.152+0000 7f66e4e1d700 1 Processor -- start 2023-10-06T05:00:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.152+0000 7f66e4e1d700 1 -- start start 2023-10-06T05:00:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66e4e1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66e0147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66e4e1d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66e0146c00 0x7f66e00d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66e4e1d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66e0147070 0x7f66e00d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66e4e1d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e00c8010 con 0x7f66c8003c40 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66e4e1d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0157c60 con 0x7f66e0147070 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66e4e1d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e00c7400 con 0x7f66e0146c00 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66df7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66e0147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66df7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66e0147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37274/0 (socket says 172.21.15.3:37274) 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66df7fe700 1 -- 172.21.15.3:0/1002251936 learned_addr learned my addr 172.21.15.3:0/1002251936 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66deffd700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66e0146c00 0x7f66e00d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.153+0000 7f66dffff700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66e0147070 0x7f66e00d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66df7fe700 1 -- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66e0146c00 msgr2=0x7f66e00d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66df7fe700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66e0146c00 0x7f66e00d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66df7fe700 1 -- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66e0147070 msgr2=0x7f66e00d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66df7fe700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66e0147070 0x7f66e00d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66df7fe700 1 -- 172.21.15.3:0/1002251936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e00c76e0 con 0x7f66c8003c40 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66df7fe700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66e0147790 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f66c80057d0 tx=0x7f66d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66c77fe700 1 -- 172.21.15.3:0/1002251936 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0022070 con 0x7f66c8003c40 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66c77fe700 1 -- 172.21.15.3:0/1002251936 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d0006240 con 0x7f66c8003c40 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e00c79c0 con 0x7f66c8003c40 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66c77fe700 1 -- 172.21.15.3:0/1002251936 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0006620 con 0x7f66c8003c40 2023-10-06T05:00:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.154+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e0147cd0 con 0x7f66c8003c40 2023-10-06T05:00:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.155+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e00c82f0 con 0x7f66c8003c40 2023-10-06T05:00:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.155+0000 7f66c77fe700 1 -- 172.21.15.3:0/1002251936 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66d0013470 con 0x7f66c8003c40 2023-10-06T05:00:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.156+0000 7f66c77fe700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66cc066f30 0x7f66cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.156+0000 7f66c77fe700 1 -- 172.21.15.3:0/1002251936 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f66d0085b70 con 0x7f66c8003c40 2023-10-06T05:00:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.156+0000 7f66deffd700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66cc066f30 0x7f66cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.157+0000 7f66deffd700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66cc066f30 0x7f66cc0693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f66d40012a0 tx=0x7f66d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.160+0000 7f66c77fe700 1 -- 172.21.15.3:0/1002251936 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66d0052c80 con 0x7f66c8003c40 2023-10-06T05:00:50.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.353+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7f66e004c380 con 0x7f66c8003c40 2023-10-06T05:00:50.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.354+0000 7f66c77fe700 1 -- 172.21.15.3:0/1002251936 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v58) v1 ==== 106+0+8 (secure 0 0 0) 0x7f66d00562a0 con 0x7f66c8003c40 2023-10-06T05:00:50.357 INFO:teuthology.orchestra.run.smithi003.stdout:1048576 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66cc066f30 msgr2=0x7f66cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66cc066f30 0x7f66cc0693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f66d40012a0 tx=0x7f66d4009040 comp rx=0 tx=0).stop 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 msgr2=0x7f66e0147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66e0147790 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f66c80057d0 tx=0x7f66d0019000 comp rx=0 tx=0).stop 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 shutdown_connections 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f66cc066f30 0x7f66cc0693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66e0147070 0x7f66e00d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c8003c40 0x7f66e0147790 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 --2- 172.21.15.3:0/1002251936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f66e0146c00 0x7f66e00d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 >> 172.21.15.3:0/1002251936 conn(0x7f66e0089dd0 msgr2=0x7f66e0159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 shutdown_connections 2023-10-06T05:00:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.356+0000 7f66e4e1d700 1 -- 172.21.15.3:0/1002251936 wait complete. 2023-10-06T05:00:50.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-10-06T05:00:50.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 Processor -- start 2023-10-06T05:00:50.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 -- start start 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa7100698f0 0x7fa710069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa71006e120 0x7fa710157b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa71006e590 0x7fa710158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71006a200 con 0x7fa71006e590 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71006eb00 con 0x7fa7100698f0 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.489+0000 7fa71629e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71006ede0 con 0x7fa71006e120 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa71006e590 0x7fa710158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa71006e590 0x7fa710158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37288/0 (socket says 172.21.15.3:37288) 2023-10-06T05:00:50.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 -- 172.21.15.3:0/604656357 learned_addr learned my addr 172.21.15.3:0/604656357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa70ffff700 1 --2- 172.21.15.3:0/604656357 >> v2:172.21.15.153:3300/0 conn(0x7fa7100698f0 0x7fa710069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa70f7fe700 1 --2- 172.21.15.3:0/604656357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa71006e120 0x7fa710157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 -- 172.21.15.3:0/604656357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa71006e120 msgr2=0x7fa710157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 --2- 172.21.15.3:0/604656357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa71006e120 0x7fa710157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 -- 172.21.15.3:0/604656357 >> v2:172.21.15.153:3300/0 conn(0x7fa7100698f0 msgr2=0x7fa710069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 --2- 172.21.15.3:0/604656357 >> v2:172.21.15.153:3300/0 conn(0x7fa7100698f0 0x7fa710069cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 -- 172.21.15.3:0/604656357 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71006f0c0 con 0x7fa71006e590 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa70f7fe700 1 --2- 172.21.15.3:0/604656357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa71006e120 0x7fa710157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa71483b700 1 --2- 172.21.15.3:0/604656357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e590 0x7fa710158040 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa710067ae0 tx=0x7fa708009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4bdfa53562d4ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa70effd700 1 -- 172.21.15.3:0/604656357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70800e070 con 0x7fa71006e590 2023-10-06T05:00:50.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa70effd700 1 -- 172.21.15.3:0/604656357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70800ba50 con 0x7fa71006e590 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.490+0000 7fa70effd700 1 -- 172.21.15.3:0/604656357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080092e0 con 0x7fa71006e590 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 -- 172.21.15.3:0/604656357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e590 msgr2=0x7fa710158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 --2- 172.21.15.3:0/604656357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e590 0x7fa710158040 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa710067ae0 tx=0x7fa708009840 comp rx=0 tx=0).stop 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 -- 172.21.15.3:0/604656357 shutdown_connections 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 --2- 172.21.15.3:0/604656357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e590 0x7fa710158040 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 --2- 172.21.15.3:0/604656357 >> v2:172.21.15.153:3300/0 conn(0x7fa7100698f0 0x7fa710069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 --2- 172.21.15.3:0/604656357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa71006e120 0x7fa710157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 -- 172.21.15.3:0/604656357 >> 172.21.15.3:0/604656357 conn(0x7fa71006ce40 msgr2=0x7fa7100cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 -- 172.21.15.3:0/604656357 shutdown_connections 2023-10-06T05:00:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.491+0000 7fa71629e700 1 -- 172.21.15.3:0/604656357 wait complete. 2023-10-06T05:00:50.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.492+0000 7fa71629e700 1 Processor -- start 2023-10-06T05:00:50.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.492+0000 7fa71629e700 1 -- start start 2023-10-06T05:00:50.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa71629e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7100698f0 0x7fa7100c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa71629e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e120 0x7fa7101395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa71629e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa71006e590 0x7fa71015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa71629e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710156b10 con 0x7fa71006e120 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa71629e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100c5fa0 con 0x7fa71006e590 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa71629e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100c6220 con 0x7fa7100698f0 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e120 0x7fa7101395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e120 0x7fa7101395c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37298/0 (socket says 172.21.15.3:37298) 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 -- 172.21.15.3:0/1964148893 learned_addr learned my addr 172.21.15.3:0/1964148893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 -- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7100698f0 msgr2=0x7fa7100c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa71483b700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa71006e590 0x7fa71015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7100698f0 0x7fa7100c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 -- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa71006e590 msgr2=0x7fa71015b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa71006e590 0x7fa71015b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 -- 172.21.15.3:0/1964148893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708009ae0 con 0x7fa71006e120 2023-10-06T05:00:50.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa70f7fe700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e120 0x7fa7101395c0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa70400c830 tx=0x7fa704008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:50.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa6f7fff700 1 -- 172.21.15.3:0/1964148893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704011600 con 0x7fa71006e120 2023-10-06T05:00:50.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.493+0000 7fa6f7fff700 1 -- 172.21.15.3:0/1964148893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa704011760 con 0x7fa71006e120 2023-10-06T05:00:50.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.494+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7100c64e0 con 0x7fa71006e120 2023-10-06T05:00:50.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.494+0000 7fa6f7fff700 1 -- 172.21.15.3:0/1964148893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704011b20 con 0x7fa71006e120 2023-10-06T05:00:50.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.494+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7100c6aa0 con 0x7fa71006e120 2023-10-06T05:00:50.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.495+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71004c380 con 0x7fa71006e120 2023-10-06T05:00:50.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.495+0000 7fa6f7fff700 1 -- 172.21.15.3:0/1964148893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa70400a3e0 con 0x7fa71006e120 2023-10-06T05:00:50.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.495+0000 7fa6f7fff700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6fc066f30 0x7fa6fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:50.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.495+0000 7fa6f7fff700 1 -- 172.21.15.3:0/1964148893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa704084780 con 0x7fa71006e120 2023-10-06T05:00:50.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.496+0000 7fa70ffff700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6fc066f30 0x7fa6fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:50.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.496+0000 7fa70ffff700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6fc066f30 0x7fa6fc0693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa71006a1a0 tx=0x7fa700008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:50.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.500+0000 7fa6f7fff700 1 -- 172.21.15.3:0/1964148893 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa704051890 con 0x7fa71006e120 2023-10-06T05:00:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.694+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7fa71006b3c0 con 0x7fa71006e120 2023-10-06T05:00:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.694+0000 7fa6f7fff700 1 -- 172.21.15.3:0/1964148893 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v58) v1 ==== 112+0+10 (secure 0 0 0) 0x7fa704054eb0 con 0x7fa71006e120 2023-10-06T05:00:50.697 INFO:teuthology.orchestra.run.smithi003.stdout:60.000000 2023-10-06T05:00:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6fc066f30 msgr2=0x7fa6fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6fc066f30 0x7fa6fc0693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa71006a1a0 tx=0x7fa700008040 comp rx=0 tx=0).stop 2023-10-06T05:00:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e120 msgr2=0x7fa7101395c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e120 0x7fa7101395c0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa70400c830 tx=0x7fa704008040 comp rx=0 tx=0).stop 2023-10-06T05:00:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 shutdown_connections 2023-10-06T05:00:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6fc066f30 0x7fa6fc0693e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa71006e590 0x7fa71015b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7100698f0 0x7fa7100c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 --2- 172.21.15.3:0/1964148893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006e120 0x7fa7101395c0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 >> 172.21.15.3:0/1964148893 conn(0x7fa71006ce40 msgr2=0x7fa7100c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 shutdown_connections 2023-10-06T05:00:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:50.696+0000 7fa71629e700 1 -- 172.21.15.3:0/1964148893 wait complete. 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:00:50.711 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T05:00:50.712 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T05:00:50.713 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T05:00:50.713 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:00:50.713 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:50.713 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T05:00:50.713 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:00:50.796 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.126814.asok 2023-10-06T05:00:50.978 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.126814.asok 2023-10-06T05:00:50.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:50.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.126814.asok mds_sessions 2023-10-06T05:00:51.132 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:00:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 8148, 2023-10-06T05:00:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 8148 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2559196242 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.8148 v1:192.168.0.1:0/2559196242", 2023-10-06T05:00:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/2559196242", 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6835", 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3912023637 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:00:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6837", 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3912023637 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 0, 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2023-10-06T05:01:46.882641+0000", 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2023-10-06T05:00:46.882641+0000", 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 1, 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2023-10-06T05:00:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:00:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-06T05:00:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 79 2023-10-06T05:00:51.136 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:00:51.142 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> import sys 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> import time 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> n = 400 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> if not os.path.exists(abs_path): 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> os.makedirs(abs_path) 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:51.143 DEBUG:teuthology.orchestra.run.smithi003:> handles = [] 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> for i in range(0, n): 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> fname = "file_"+str(i) 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> path = os.path.join(abs_path, fname) 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> handles.append(open(path, '"'"'w'"'"')) 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> while True: 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> time.sleep(1) 2023-10-06T05:00:51.144 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:00:51.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.267+0000 7f9fd7c07700 1 Processor -- start 2023-10-06T05:00:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.267+0000 7f9fd7c07700 1 -- start start 2023-10-06T05:00:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd7c07700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9fd0156410 0x7f9fd00af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd7c07700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9fd00b0900 0x7f9fd006b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd7c07700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd006b600 0x7f9fd00adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd7c07700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd006ba80 con 0x7f9fd0156410 2023-10-06T05:00:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd7c07700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd00b1c90 con 0x7f9fd00b0900 2023-10-06T05:00:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd7c07700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd00b1080 con 0x7f9fd006b600 2023-10-06T05:00:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9fd0156410 0x7f9fd00af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9fd0156410 0x7f9fd00af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37308/0 (socket says 172.21.15.3:37308) 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 -- 172.21.15.3:0/2706345995 learned_addr learned my addr 172.21.15.3:0/2706345995 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd51a2700 1 --2- 172.21.15.3:0/2706345995 >> v2:172.21.15.153:3300/0 conn(0x7f9fd00b0900 0x7f9fd006b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd61a4700 1 --2- 172.21.15.3:0/2706345995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd006b600 0x7f9fd00adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 -- 172.21.15.3:0/2706345995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd006b600 msgr2=0x7f9fd00adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 --2- 172.21.15.3:0/2706345995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd006b600 0x7f9fd00adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 -- 172.21.15.3:0/2706345995 >> v2:172.21.15.153:3300/0 conn(0x7f9fd00b0900 msgr2=0x7f9fd006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 --2- 172.21.15.3:0/2706345995 >> v2:172.21.15.153:3300/0 conn(0x7f9fd00b0900 0x7f9fd006b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.268+0000 7f9fd59a3700 1 -- 172.21.15.3:0/2706345995 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00b1360 con 0x7f9fd0156410 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd59a3700 1 --2- 172.21.15.3:0/2706345995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd0156410 0x7f9fd00af480 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00acaf0 tx=0x7f9fcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3c9fc1cd49456e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd49a1700 1 -- 172.21.15.3:0/2706345995 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00b640 con 0x7f9fd0156410 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd49a1700 1 -- 172.21.15.3:0/2706345995 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc00b7a0 con 0x7f9fd0156410 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd49a1700 1 -- 172.21.15.3:0/2706345995 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00bb60 con 0x7f9fd0156410 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 -- 172.21.15.3:0/2706345995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd0156410 msgr2=0x7f9fd00af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/2706345995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd0156410 0x7f9fd00af480 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00acaf0 tx=0x7f9fcc009250 comp rx=0 tx=0).stop 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 -- 172.21.15.3:0/2706345995 shutdown_connections 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/2706345995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd006b600 0x7f9fd00adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/2706345995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd0156410 0x7f9fd00af480 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/2706345995 >> v2:172.21.15.153:3300/0 conn(0x7f9fd00b0900 0x7f9fd006b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 -- 172.21.15.3:0/2706345995 >> 172.21.15.3:0/2706345995 conn(0x7f9fd00cdc30 msgr2=0x7f9fd006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 -- 172.21.15.3:0/2706345995 shutdown_connections 2023-10-06T05:00:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.269+0000 7f9fd7c07700 1 -- 172.21.15.3:0/2706345995 wait complete. 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 Processor -- start 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 -- start start 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fd006b600 0x7f9fd00a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd00b0900 0x7f9fd01577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd0156410 0x7f9fd0157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd01567e0 con 0x7f9fd00b0900 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0157320 con 0x7f9fd006b600 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd7c07700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd01569f0 con 0x7f9fd0156410 2023-10-06T05:00:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd51a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd00b0900 0x7f9fd01577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd51a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd00b0900 0x7f9fd01577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37324/0 (socket says 172.21.15.3:37324) 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd51a2700 1 -- 172.21.15.3:0/502458634 learned_addr learned my addr 172.21.15.3:0/502458634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.271+0000 7f9fd59a3700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fd006b600 0x7f9fd00a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd61a4700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd0156410 0x7f9fd0157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd51a2700 1 -- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd0156410 msgr2=0x7f9fd0157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd51a2700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd0156410 0x7f9fd0157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd51a2700 1 -- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fd006b600 msgr2=0x7f9fd00a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd51a2700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fd006b600 0x7f9fd00a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd51a2700 1 -- 172.21.15.3:0/502458634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc0094f0 con 0x7f9fd00b0900 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd51a2700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd00b0900 0x7f9fd01577b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0010e90 tx=0x7f9fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fc5ffb700 1 -- 172.21.15.3:0/502458634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000ed40 con 0x7f9fd00b0900 2023-10-06T05:00:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fc5ffb700 1 -- 172.21.15.3:0/502458634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc0011450 con 0x7f9fd00b0900 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd0156cd0 con 0x7f9fd00b0900 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fc5ffb700 1 -- 172.21.15.3:0/502458634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000eea0 con 0x7f9fd00b0900 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.272+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd00abf40 con 0x7f9fd00b0900 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.273+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd0071680 con 0x7f9fd00b0900 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.274+0000 7f9fc5ffb700 1 -- 172.21.15.3:0/502458634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fc0011a00 con 0x7f9fd00b0900 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.274+0000 7f9fc5ffb700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.274+0000 7f9fc5ffb700 1 -- 172.21.15.3:0/502458634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9fc0086a60 con 0x7f9fd00b0900 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.274+0000 7f9fd59a3700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.275+0000 7f9fd59a3700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00acaf0 tx=0x7f9fcc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.278+0000 7f9fc5ffb700 1 -- 172.21.15.3:0/502458634 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fc0053b70 con 0x7f9fd00b0900 2023-10-06T05:00:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.546+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fd00707c0 con 0x7f9fd00b0900 2023-10-06T05:00:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.547+0000 7f9fc5ffb700 1 -- 172.21.15.3:0/502458634 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f9fc001e020 con 0x7f9fd00b0900 2023-10-06T05:00:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:51.550 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:41.823898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9fb4066f30 msgr2=0x7f9fb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00acaf0 tx=0x7f9fcc017000 comp rx=0 tx=0).stop 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd00b0900 msgr2=0x7f9fd01577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd00b0900 0x7f9fd01577b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0010e90 tx=0x7f9fc00091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 shutdown_connections 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fd006b600 0x7f9fd00a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9fd0156410 0x7f9fd0157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 --2- 172.21.15.3:0/502458634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd00b0900 0x7f9fd01577b0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:51.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 >> 172.21.15.3:0/502458634 conn(0x7f9fd00cdc30 msgr2=0x7f9fd0151830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:51.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 shutdown_connections 2023-10-06T05:00:51.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:51.549+0000 7f9fd7c07700 1 -- 172.21.15.3:0/502458634 wait complete. 2023-10-06T05:00:51.553 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 79 2023-10-06T05:00:51.566 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T05:00:51.705 INFO:teuthology.orchestra.run.smithi153.stdout:[{"id":7480,"entity":{"name":{"type":"client","num":7480},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4115743899}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.0149291529999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9885449630928362,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7480 192.168.0.1:0/4115743899","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"102508","root":"/"}},{"id":8148,"entity":{"name":{"type":"client","num":8148},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2559196242}},"state":"open","num_leases":240,"num_caps":241,"request_load_avg":720,"uptime":5.8108974609999997,"requests_in_flight":239,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":720.59939476169245,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8148 192.168.0.1:0/2559196242","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"126814","root":"/"}}] 2023-10-06T05:00:51.709 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "102508", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4115743899, "type": "any" }, "name": { "num": 7480, "type": "client" } }, "id": 7480, "inst": "client.7480 192.168.0.1:0/4115743899", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9885449630928362 }, "state": "open", "uptime": 4.014929153 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "126814", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2559196242, "type": "any" }, "name": { "num": 8148, "type": "client" } }, "id": 8148, "inst": "client.8148 192.168.0.1:0/2559196242", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 241, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 240, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 720, "requests_in_flight": 239, "session_cache_liveness": { "halflife": 300, "value": 720.5993947616925 }, "state": "open", "uptime": 5.810897461 } ] 2023-10-06T05:00:51.709 DEBUG:tasks.ceph_test_case:wait_until_equal: 241 != 401, waiting (timeout=600)... 2023-10-06T05:00:55.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:55.654 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:56.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:00:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.841+0000 7f9c14bf6700 1 Processor -- start 2023-10-06T05:00:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.842+0000 7f9c14bf6700 1 -- start start 2023-10-06T05:00:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.842+0000 7f9c14bf6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c1015cb50 0x7f9c100b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c14bf6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c100b0ce0 con 0x7f9c100b0590 2023-10-06T05:00:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c14bf6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c100cb820 con 0x7f9c100cd950 2023-10-06T05:00:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c14bf6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100b0a00 con 0x7f9c1015cb50 2023-10-06T05:00:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c100b0590 0x7f9c100b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:32968/0 (socket says 172.21.15.3:32968) 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0dd9b700 1 -- 172.21.15.3:0/3551704405 learned_addr learned my addr 172.21.15.3:0/3551704405 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0e59c700 1 --2- 172.21.15.3:0/3551704405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c1015cb50 0x7f9c100b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182836068 0 0) 0x7f9c100b0ce0 con 0x7f9c100b0590 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bfc003570 con 0x7f9c100b0590 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929236549 0 0) 0x7f9c100cb820 con 0x7f9c100cd950 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c100b0ce0 con 0x7f9c100cd950 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0e59c700 1 -- 172.21.15.3:0/3551704405 >> v1:172.21.15.153:6789/0 conn(0x7f9c100cd950 legacy=0x7f9c100cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0e59c700 1 -- 172.21.15.3:0/3551704405 >> v1:172.21.15.3:6789/0 conn(0x7f9c100b0590 legacy=0x7f9c100b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1335523137 0 0) 0x7f9bfc003570 con 0x7f9c100b0590 2023-10-06T05:00:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.843+0000 7f9c0e59c700 1 -- 172.21.15.3:0/3551704405 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100cb1d0 con 0x7f9c1015cb50 2023-10-06T05:00:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c0e59c700 1 --2- 172.21.15.3:0/3551704405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c1015cb50 0x7f9c100b0050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009ee0 tx=0x7f9c00009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=787d415beca8bc6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000e070 con 0x7f9c1015cb50 2023-10-06T05:00:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000bb80 con 0x7f9c1015cb50 2023-10-06T05:00:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c0d59a700 1 -- 172.21.15.3:0/3551704405 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00012430 con 0x7f9c1015cb50 2023-10-06T05:00:56.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c14bf6700 1 -- 172.21.15.3:0/3551704405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c1015cb50 msgr2=0x7f9c100b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/3551704405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c1015cb50 0x7f9c100b0050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009ee0 tx=0x7f9c00009780 comp rx=0 tx=0).stop 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c14bf6700 1 -- 172.21.15.3:0/3551704405 shutdown_connections 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/3551704405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c1015cb50 0x7f9c100b0050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c14bf6700 1 -- 172.21.15.3:0/3551704405 >> 172.21.15.3:0/3551704405 conn(0x7f9c10070310 msgr2=0x7f9c10070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c14bf6700 1 -- 172.21.15.3:0/3551704405 shutdown_connections 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.844+0000 7f9c14bf6700 1 -- 172.21.15.3:0/3551704405 wait complete. 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.845+0000 7f9c14bf6700 1 Processor -- start 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c14bf6700 1 -- start start 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c14bf6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c100b0590 0x7f9c10158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:56.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c14bf6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c100cd950 0x7f9c100c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c14bf6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c1015cb50 0x7f9c10157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c14bf6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1016a020 con 0x7f9c1015cb50 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c14bf6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1016a4b0 con 0x7f9c100b0590 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c14bf6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c10158c20 con 0x7f9c100cd950 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c0ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c1015cb50 0x7f9c10157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c0ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c1015cb50 0x7f9c10157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60784/0 (socket says 172.21.15.3:60784) 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.846+0000 7f9c0ed9d700 1 -- 172.21.15.3:0/1557979121 learned_addr learned my addr 172.21.15.3:0/1557979121 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0e59c700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c100b0590 0x7f9c10158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0dd9b700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c100cd950 0x7f9c100c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:56.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0ed9d700 1 -- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c100cd950 msgr2=0x7f9c100c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0ed9d700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c100cd950 0x7f9c100c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0ed9d700 1 -- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c100b0590 msgr2=0x7f9c10158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0ed9d700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c100b0590 0x7f9c10158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0ed9d700 1 -- 172.21.15.3:0/1557979121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c00009a20 con 0x7f9c1015cb50 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c0ed9d700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c1015cb50 0x7f9c10157b90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c590 tx=0x7f9c08007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c067fc700 1 -- 172.21.15.3:0/1557979121 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08016070 con 0x7f9c1015cb50 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c067fc700 1 -- 172.21.15.3:0/1557979121 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08011590 con 0x7f9c1015cb50 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c100cdf80 con 0x7f9c1015cb50 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c067fc700 1 -- 172.21.15.3:0/1557979121 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08011970 con 0x7f9c1015cb50 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.847+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c101696f0 con 0x7f9c1015cb50 2023-10-06T05:00:56.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.848+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1016a190 con 0x7f9c1015cb50 2023-10-06T05:00:56.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.849+0000 7f9c067fc700 1 -- 172.21.15.3:0/1557979121 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c08012420 con 0x7f9c1015cb50 2023-10-06T05:00:56.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.849+0000 7f9c067fc700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9bf0066f30 0x7f9bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:56.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.849+0000 7f9c067fc700 1 -- 172.21.15.3:0/1557979121 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9c08084190 con 0x7f9c1015cb50 2023-10-06T05:00:56.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.849+0000 7f9c0e59c700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9bf0066f30 0x7f9bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:56.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.850+0000 7f9c0e59c700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9bf0066f30 0x7f9bf00693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009e30 tx=0x7f9c00019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:56.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:56.853+0000 7f9c067fc700 1 -- 172.21.15.3:0/1557979121 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c080512a0 con 0x7f9c1015cb50 2023-10-06T05:00:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.095+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c10071680 con 0x7f9c1015cb50 2023-10-06T05:00:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.095+0000 7f9c067fc700 1 -- 172.21.15.3:0/1557979121 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f9c080548c0 con 0x7f9c1015cb50 2023-10-06T05:00:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:57.097 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:55.606634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:00:57.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9bf0066f30 msgr2=0x7f9bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9bf0066f30 0x7f9bf00693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009e30 tx=0x7f9c00019000 comp rx=0 tx=0).stop 2023-10-06T05:00:57.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c1015cb50 msgr2=0x7f9c10157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c1015cb50 0x7f9c10157b90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c590 tx=0x7f9c08007d80 comp rx=0 tx=0).stop 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 shutdown_connections 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9bf0066f30 0x7f9bf00693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c100b0590 0x7f9c10158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c1015cb50 0x7f9c10157b90 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 --2- 172.21.15.3:0/1557979121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c100cd950 0x7f9c100c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.097+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 >> 172.21.15.3:0/1557979121 conn(0x7f9c10070310 msgr2=0x7f9c100b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.098+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 shutdown_connections 2023-10-06T05:00:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.098+0000 7f9c14bf6700 1 -- 172.21.15.3:0/1557979121 wait complete. 2023-10-06T05:00:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 80 2023-10-06T05:00:57.111 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T05:00:57.252 INFO:teuthology.orchestra.run.smithi153.stdout:[{"id":7480,"entity":{"name":{"type":"client","num":7480},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4115743899}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":9.5618312749999994,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9504879673045448,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7480 192.168.0.1:0/4115743899","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"102508","root":"/"}},{"id":8148,"entity":{"name":{"type":"client","num":8148},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2559196242}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1127,"uptime":11.357799583,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1160.3416258374982,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":416.27048105798735,"halflife":1.5},"recall_caps_throttle2o":{"value":147.89570647315625,"halflife":0.5},"session_cache_liveness":{"value":1187.497517626193,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1203,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8148 192.168.0.1:0/2559196242","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"126814","root":"/"}}] 2023-10-06T05:00:57.253 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "102508", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4115743899, "type": "any" }, "name": { "num": 7480, "type": "client" } }, "id": 7480, "inst": "client.7480 192.168.0.1:0/4115743899", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9504879673045448 }, "state": "open", "uptime": 9.561831275 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "126814", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2559196242, "type": "any" }, "name": { "num": 8148, "type": "client" } }, "id": 8148, "inst": "client.8148 192.168.0.1:0/2559196242", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1203, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1160.3416258374982 }, "recall_caps_throttle": { "halflife": 1.5, "value": 416.27048105798735 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 147.89570647315625 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1127, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1187.497517626193 }, "state": "open", "uptime": 11.357799583 } ] 2023-10-06T05:00:57.253 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-10-06T05:00:57.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:00:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.372+0000 7f1fc7cc2700 1 Processor -- start 2023-10-06T05:00:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.372+0000 7f1fc7cc2700 1 -- start start 2023-10-06T05:00:57.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc7cc2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1fc015d240 0x7f1fc015a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc7cc2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc015d6d0 con 0x7f1fc00cd320 2023-10-06T05:00:57.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc7cc2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc015d9b0 con 0x7f1fc00b27b0 2023-10-06T05:00:57.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc7cc2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc015aba0 con 0x7f1fc015d240 2023-10-06T05:00:57.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc625f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1fc00cd320 0x7f1fc00aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:32982/0 (socket says 172.21.15.3:32982) 2023-10-06T05:00:57.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc625f700 1 -- 172.21.15.3:0/1574317878 learned_addr learned my addr 172.21.15.3:0/1574317878 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934635276 0 0) 0x7f1fc015d6d0 con 0x7f1fc00cd320 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fa8003570 con 0x7f1fc00cd320 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc525d700 1 --2- 172.21.15.3:0/1574317878 >> v2:172.21.15.153:3300/0 conn(0x7f1fc015d240 0x7f1fc015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489484590 0 0) 0x7f1fc015d9b0 con 0x7f1fc00b27b0 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc015d6d0 con 0x7f1fc00b27b0 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2010040129 0 0) 0x7f1fa8003570 con 0x7f1fc00cd320 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 >> v2:172.21.15.153:3300/0 conn(0x7f1fc015d240 msgr2=0x7f1fc015a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 --2- 172.21.15.3:0/1574317878 >> v2:172.21.15.153:3300/0 conn(0x7f1fc015d240 0x7f1fc015a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 >> v1:172.21.15.153:6789/0 conn(0x7f1fc00b27b0 legacy=0x7f1fc00ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc015dc90 con 0x7f1fc00cd320 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f1fb80017f0 con 0x7f1fc00cd320 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 >> v1:172.21.15.3:6789/0 conn(0x7f1fc00cd320 legacy=0x7f1fc00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fa8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.373+0000 7f1fc4a5c700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fa8004810 0x7f1fa8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc4a5c700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fa8005210 0x7f1fa800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa800c180 con 0x7f1fa8003c40 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa800c2f0 con 0x7f1fa8005210 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa800c4b0 con 0x7f1fa8004810 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc5a5e700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fa8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc625f700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fa8005210 0x7f1fa800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc525d700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fa8004810 0x7f1fa8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc5a5e700 1 -- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fa8004810 msgr2=0x7f1fa8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc5a5e700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fa8004810 0x7f1fa8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc5a5e700 1 -- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fa8005210 msgr2=0x7f1fa800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc5a5e700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fa8005210 0x7f1fa800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc5a5e700 1 -- 172.21.15.3:0/1574317878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8003570 con 0x7f1fa8003c40 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc625f700 1 -- 172.21.15.3:0/1574317878 reap_dead start 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc525d700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fa8004810 0x7f1fa8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc5a5e700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fa8004030 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0007fd0 tx=0x7f1fb0007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a7f576c3292c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00013f0 con 0x7f1fa8003c40 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb000c5d0 con 0x7f1fa8003c40 2023-10-06T05:00:57.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.374+0000 7f1fc4a5c700 1 -- 172.21.15.3:0/1574317878 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000c990 con 0x7f1fa8003c40 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.375+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1574317878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 msgr2=0x7f1fa8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.375+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fa8004030 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0007fd0 tx=0x7f1fb0007d80 comp rx=0 tx=0).stop 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.375+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1574317878 shutdown_connections 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.375+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1574317878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fa8004030 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.375+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1574317878 >> 172.21.15.3:0/1574317878 conn(0x7f1fc0070310 msgr2=0x7f1fc0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.375+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1574317878 shutdown_connections 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.375+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1574317878 wait complete. 2023-10-06T05:00:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.376+0000 7f1fc7cc2700 1 Processor -- start 2023-10-06T05:00:57.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.376+0000 7f1fc7cc2700 1 -- start start 2023-10-06T05:00:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc7cc2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fc00b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc7cc2700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fc013c200 0x7f1fc00b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc7cc2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fc00d0290 0x7f1fc00d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc7cc2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00d0c40 con 0x7f1fa8003c40 2023-10-06T05:00:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc7cc2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00c7be0 con 0x7f1fc00d0290 2023-10-06T05:00:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc7cc2700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00c75d0 con 0x7f1fc013c200 2023-10-06T05:00:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fc00b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fc00b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60808/0 (socket says 172.21.15.3:60808) 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 -- 172.21.15.3:0/1857823546 learned_addr learned my addr 172.21.15.3:0/1857823546 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc525d700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fc013c200 0x7f1fc00b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc625f700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fc00d0290 0x7f1fc00d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 -- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fc013c200 msgr2=0x7f1fc00b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fc013c200 0x7f1fc00b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 -- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fc00d0290 msgr2=0x7f1fc00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fc00d0290 0x7f1fc00d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 -- 172.21.15.3:0/1857823546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00b5e80 con 0x7f1fa8003c40 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc625f700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fc00d0290 0x7f1fc00d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc525d700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fc013c200 0x7f1fc00b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc5a5e700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fc00b32d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c3c0 tx=0x7f1fb0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fbdffb700 1 -- 172.21.15.3:0/1857823546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000c740 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fbdffb700 1 -- 172.21.15.3:0/1857823546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb000c8a0 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc00c7870 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.377+0000 7f1fbdffb700 1 -- 172.21.15.3:0/1857823546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0006ac0 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.378+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc013c670 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.379+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc00c7a00 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.379+0000 7f1fbdffb700 1 -- 172.21.15.3:0/1857823546 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1fb0019070 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.379+0000 7f1fbdffb700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1fac066f30 0x7f1fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.379+0000 7f1fbdffb700 1 -- 172.21.15.3:0/1857823546 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1fb0001e40 con 0x7f1fa8003c40 2023-10-06T05:00:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.380+0000 7f1fc525d700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1fac066f30 0x7f1fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:57.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.380+0000 7f1fc525d700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1fac066f30 0x7f1fac0693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00ca5a0 tx=0x7f1fb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:57.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.384+0000 7f1fbdffb700 1 -- 172.21.15.3:0/1857823546 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fb0053120 con 0x7f1fa8003c40 2023-10-06T05:00:57.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.617+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1fc0071680 con 0x7f1fa8003c40 2023-10-06T05:00:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.617+0000 7f1fbdffb700 1 -- 172.21.15.3:0/1857823546 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f1fb0017080 con 0x7f1fa8003c40 2023-10-06T05:00:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:00:57.619 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1fac066f30 msgr2=0x7f1fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1fac066f30 0x7f1fac0693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00ca5a0 tx=0x7f1fb4008040 comp rx=0 tx=0).stop 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 msgr2=0x7f1fc00b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fc00b32d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c3c0 tx=0x7f1fb0015000 comp rx=0 tx=0).stop 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 shutdown_connections 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1fac066f30 0x7f1fac0693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fc00d0290 0x7f1fc00d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa8003c40 0x7f1fc00b32d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 --2- 172.21.15.3:0/1857823546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1fc013c200 0x7f1fc00b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 >> 172.21.15.3:0/1857823546 conn(0x7f1fc0070310 msgr2=0x7f1fc00c63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 shutdown_connections 2023-10-06T05:00:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:00:57.619+0000 7f1fc7cc2700 1 -- 172.21.15.3:0/1857823546 wait complete. 2023-10-06T05:00:57.632 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-10-06T05:00:57.633 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-10-06T05:01:02.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:01:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.766+0000 7fe03623d700 1 Processor -- start 2023-10-06T05:01:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.766+0000 7fe03623d700 1 -- start start 2023-10-06T05:01:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe03623d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe030157c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe03623d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe0300b1840 0x7fe030158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe03623d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0301589c0 con 0x7fe030140a60 2023-10-06T05:01:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe03623d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0300c9ef0 con 0x7fe0300b1840 2023-10-06T05:01:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe03623d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300cbed0 con 0x7fe030140e30 2023-10-06T05:01:02.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe02effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe030157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe02ffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe0300b1840 0x7fe030158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe02f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe030140a60 0x7fe0300b1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53906/0 (socket says 172.21.15.3:53906) 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.767+0000 7fe02f7fe700 1 -- 172.21.15.3:0/1427299669 learned_addr learned my addr 172.21.15.3:0/1427299669 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02e7fc700 1 -- 172.21.15.3:0/1427299669 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350621981 0 0) 0x7fe0301589c0 con 0x7fe030140a60 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02e7fc700 1 -- 172.21.15.3:0/1427299669 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe018003570 con 0x7fe030140a60 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02ffff700 1 -- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 msgr2=0x7fe030157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02ffff700 1 --2- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe030157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02ffff700 1 -- 172.21.15.3:0/1427299669 >> v1:172.21.15.153:6789/0 conn(0x7fe030140a60 legacy=0x7fe0300b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02ffff700 1 -- 172.21.15.3:0/1427299669 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300caa30 con 0x7fe0300b1840 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02effd700 1 --2- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe030157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:02.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02ffff700 1 --2- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0300b1840 0x7fe030158170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe028009e60 tx=0x7fe028009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9168e95eb7c13183 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02e7fc700 1 -- 172.21.15.3:0/1427299669 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02800e070 con 0x7fe0300b1840 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02e7fc700 1 -- 172.21.15.3:0/1427299669 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02800bab0 con 0x7fe0300b1840 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.768+0000 7fe02e7fc700 1 -- 172.21.15.3:0/1427299669 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02800be30 con 0x7fe0300b1840 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 -- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0300b1840 msgr2=0x7fe030158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 --2- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0300b1840 0x7fe030158170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe028009e60 tx=0x7fe028009780 comp rx=0 tx=0).stop 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 -- 172.21.15.3:0/1427299669 shutdown_connections 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 --2- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0300b1840 0x7fe030158170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 --2- 172.21.15.3:0/1427299669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe030157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 -- 172.21.15.3:0/1427299669 >> 172.21.15.3:0/1427299669 conn(0x7fe03006c370 msgr2=0x7fe03006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 -- 172.21.15.3:0/1427299669 shutdown_connections 2023-10-06T05:01:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.769+0000 7fe03623d700 1 -- 172.21.15.3:0/1427299669 wait complete. 2023-10-06T05:01:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.770+0000 7fe03623d700 1 Processor -- start 2023-10-06T05:01:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.770+0000 7fe03623d700 1 -- start start 2023-10-06T05:01:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe03623d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0300b1840 0x7fe0300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe03623d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030140a60 0x7fe03015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe03623d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe03015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe03623d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030167cb0 con 0x7fe0300b1840 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe03623d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030167f70 con 0x7fe030140a60 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe03623d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe030168250 con 0x7fe030140e30 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0300b1840 0x7fe0300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0300b1840 0x7fe0300a6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54718/0 (socket says 172.21.15.3:54718) 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 -- 172.21.15.3:0/3666948724 learned_addr learned my addr 172.21.15.3:0/3666948724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02effd700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030140a60 0x7fe03015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02ffff700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe03015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 -- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 msgr2=0x7fe03015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe03015cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 -- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030140a60 msgr2=0x7fe03015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030140a60 0x7fe03015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02f7fe700 1 -- 172.21.15.3:0/3666948724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe024009420 con 0x7fe0300b1840 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.771+0000 7fe02ffff700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe03015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.772+0000 7fe02f7fe700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0300b1840 0x7fe0300a6210 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe0200011e0 tx=0x7fe020007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.772+0000 7fe0177fe700 1 -- 172.21.15.3:0/3666948724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020005b30 con 0x7fe0300b1840 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.772+0000 7fe0177fe700 1 -- 172.21.15.3:0/3666948724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020005c90 con 0x7fe0300b1840 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.772+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe028009a20 con 0x7fe0300b1840 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.772+0000 7fe0177fe700 1 -- 172.21.15.3:0/3666948724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020007320 con 0x7fe0300b1840 2023-10-06T05:01:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.772+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030168810 con 0x7fe0300b1840 2023-10-06T05:01:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.773+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe030071680 con 0x7fe0300b1840 2023-10-06T05:01:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.773+0000 7fe0177fe700 1 -- 172.21.15.3:0/3666948724 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe020007690 con 0x7fe0300b1840 2023-10-06T05:01:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.774+0000 7fe0177fe700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe01c066f30 0x7fe01c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.774+0000 7fe0177fe700 1 -- 172.21.15.3:0/3666948724 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe020083b20 con 0x7fe0300b1840 2023-10-06T05:01:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.774+0000 7fe02effd700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe01c066f30 0x7fe01c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.775+0000 7fe02effd700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe01c066f30 0x7fe01c0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe03015bc70 tx=0x7fe0240091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:02.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:02.778+0000 7fe0177fe700 1 -- 172.21.15.3:0/3666948724 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe020050c30 con 0x7fe0300b1840 2023-10-06T05:01:03.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.018+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe0300ca100 con 0x7fe0300b1840 2023-10-06T05:01:03.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.019+0000 7fe0177fe700 1 -- 172.21.15.3:0/3666948724 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7fe020017080 con 0x7fe0300b1840 2023-10-06T05:01:03.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:03.021 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe01c066f30 msgr2=0x7fe01c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe01c066f30 0x7fe01c0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe03015bc70 tx=0x7fe0240091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0300b1840 msgr2=0x7fe0300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0300b1840 0x7fe0300a6210 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe0200011e0 tx=0x7fe020007df0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 shutdown_connections 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe01c066f30 0x7fe01c0693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe030140e30 0x7fe03015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0300b1840 0x7fe0300a6210 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 --2- 172.21.15.3:0/3666948724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030140a60 0x7fe03015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 >> 172.21.15.3:0/3666948724 conn(0x7fe03006c370 msgr2=0x7fe030156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 shutdown_connections 2023-10-06T05:01:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.021+0000 7fe03623d700 1 -- 172.21.15.3:0/3666948724 wait complete. 2023-10-06T05:01:03.034 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:01:03.034 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-10-06T05:01:03.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:01:03.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.151+0000 7fcd7cf25700 1 Processor -- start 2023-10-06T05:01:03.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.151+0000 7fcd7cf25700 1 -- start start 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd7cf25700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcd780c8520 0x7fcd781572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd7cf25700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd7806ec80 0x7fcd780af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd7cf25700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd78157800 con 0x7fcd780acb90 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd7cf25700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780ad070 con 0x7fcd780c8520 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd7cf25700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780ad350 con 0x7fcd7806ec80 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd7659c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcd780c8520 0x7fcd781572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd76d9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd7806ec80 0x7fcd780af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd75d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcd780acb90 0x7fcd7806e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:53922/0 (socket says 172.21.15.3:53922) 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.152+0000 7fcd75d9b700 1 -- 172.21.15.3:0/1063627557 learned_addr learned my addr 172.21.15.3:0/1063627557 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd7559a700 1 -- 172.21.15.3:0/1063627557 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171951942 0 0) 0x7fcd78157800 con 0x7fcd780acb90 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd7559a700 1 -- 172.21.15.3:0/1063627557 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd64003570 con 0x7fcd780acb90 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd76d9d700 1 -- 172.21.15.3:0/1063627557 >> v2:172.21.15.153:3300/0 conn(0x7fcd780c8520 msgr2=0x7fcd781572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd76d9d700 1 --2- 172.21.15.3:0/1063627557 >> v2:172.21.15.153:3300/0 conn(0x7fcd780c8520 0x7fcd781572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd76d9d700 1 -- 172.21.15.3:0/1063627557 >> v1:172.21.15.153:6789/0 conn(0x7fcd780acb90 legacy=0x7fcd7806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd76d9d700 1 -- 172.21.15.3:0/1063627557 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd780ad630 con 0x7fcd7806ec80 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd7659c700 1 --2- 172.21.15.3:0/1063627557 >> v2:172.21.15.153:3300/0 conn(0x7fcd780c8520 0x7fcd781572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd76d9d700 1 --2- 172.21.15.3:0/1063627557 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd7806ec80 0x7fcd780af200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd680098e0 tx=0x7fcd680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6c033242dbe454b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd7559a700 1 -- 172.21.15.3:0/1063627557 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68011630 con 0x7fcd7806ec80 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd7559a700 1 -- 172.21.15.3:0/1063627557 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68011790 con 0x7fcd7806ec80 2023-10-06T05:01:03.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.153+0000 7fcd7559a700 1 -- 172.21.15.3:0/1063627557 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68011b10 con 0x7fcd7806ec80 2023-10-06T05:01:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 -- 172.21.15.3:0/1063627557 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd7806ec80 msgr2=0x7fcd780af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/1063627557 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd7806ec80 0x7fcd780af200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd680098e0 tx=0x7fcd680091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 -- 172.21.15.3:0/1063627557 shutdown_connections 2023-10-06T05:01:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/1063627557 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd7806ec80 0x7fcd780af200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/1063627557 >> v2:172.21.15.153:3300/0 conn(0x7fcd780c8520 0x7fcd781572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 -- 172.21.15.3:0/1063627557 >> 172.21.15.3:0/1063627557 conn(0x7fcd7806c370 msgr2=0x7fcd7806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 -- 172.21.15.3:0/1063627557 shutdown_connections 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.154+0000 7fcd7cf25700 1 -- 172.21.15.3:0/1063627557 wait complete. 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.155+0000 7fcd7cf25700 1 Processor -- start 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.155+0000 7fcd7cf25700 1 -- start start 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7cf25700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7806ec80 0x7fcd78149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7cf25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd780acb90 0x7fcd781572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7cf25700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd780c8520 0x7fcd7814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7cf25700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78149580 con 0x7fcd780acb90 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7cf25700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd781644f0 con 0x7fcd7806ec80 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7cf25700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78157db0 con 0x7fcd780c8520 2023-10-06T05:01:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd780acb90 0x7fcd781572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd780acb90 0x7fcd781572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54728/0 (socket says 172.21.15.3:54728) 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 -- 172.21.15.3:0/792140442 learned_addr learned my addr 172.21.15.3:0/792140442 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd76d9d700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd780c8520 0x7fcd7814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7659c700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7806ec80 0x7fcd78149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 -- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd780c8520 msgr2=0x7fcd7814a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd780c8520 0x7fcd7814a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 -- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7806ec80 msgr2=0x7fcd78149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7806ec80 0x7fcd78149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 -- 172.21.15.3:0/792140442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70009a20 con 0x7fcd780acb90 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd75d9b700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd780acb90 0x7fcd781572f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fcd600011e0 tx=0x7fcd60007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd6e7fc700 1 -- 172.21.15.3:0/792140442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60005b30 con 0x7fcd780acb90 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd6e7fc700 1 -- 172.21.15.3:0/792140442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd60005c90 con 0x7fcd780acb90 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd68009420 con 0x7fcd780acb90 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.156+0000 7fcd6e7fc700 1 -- 172.21.15.3:0/792140442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600072e0 con 0x7fcd780acb90 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.157+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd78158370 con 0x7fcd780acb90 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.157+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd781646b0 con 0x7fcd780acb90 2023-10-06T05:01:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.158+0000 7fcd6e7fc700 1 -- 172.21.15.3:0/792140442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd60007650 con 0x7fcd780acb90 2023-10-06T05:01:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.158+0000 7fcd6e7fc700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd58066f30 0x7fcd580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.158+0000 7fcd6e7fc700 1 -- 172.21.15.3:0/792140442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcd60083ac0 con 0x7fcd780acb90 2023-10-06T05:01:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.158+0000 7fcd7659c700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd58066f30 0x7fcd580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.159+0000 7fcd7659c700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd58066f30 0x7fcd580693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcd700099f0 tx=0x7fcd7000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:03.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.162+0000 7fcd6e7fc700 1 -- 172.21.15.3:0/792140442 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd6004b5a0 con 0x7fcd780acb90 2023-10-06T05:01:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.396+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcd78071680 con 0x7fcd780acb90 2023-10-06T05:01:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.396+0000 7fcd6e7fc700 1 -- 172.21.15.3:0/792140442 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7fcd60083e60 con 0x7fcd780acb90 2023-10-06T05:01:03.398 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-10-06T05:01:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.398+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd58066f30 msgr2=0x7fcd580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd58066f30 0x7fcd580693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcd700099f0 tx=0x7fcd7000b040 comp rx=0 tx=0).stop 2023-10-06T05:01:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd780acb90 msgr2=0x7fcd781572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd780acb90 0x7fcd781572f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fcd600011e0 tx=0x7fcd60007df0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 shutdown_connections 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd58066f30 0x7fcd580693e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7806ec80 0x7fcd78149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd780c8520 0x7fcd7814a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 --2- 172.21.15.3:0/792140442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd780acb90 0x7fcd781572f0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 >> 172.21.15.3:0/792140442 conn(0x7fcd7806c370 msgr2=0x7fcd780afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 shutdown_connections 2023-10-06T05:01:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.399+0000 7fcd7cf25700 1 -- 172.21.15.3:0/792140442 wait complete. 2023-10-06T05:01:03.412 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:01:03.412 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-10-06T05:01:03.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:03.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.586+0000 7f3c798ac700 1 Processor -- start 2023-10-06T05:01:03.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.586+0000 7f3c798ac700 1 -- start start 2023-10-06T05:01:03.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c798ac700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3c741591a0 0x7f3c740affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c798ac700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c740b0a90 con 0x7f3c74159570 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c798ac700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c740b3110 con 0x7f3c7415dbd0 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c798ac700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c740b33f0 con 0x7f3c741591a0 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c737fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3c74159570 0x7f3c740b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42242/0 (socket says 172.21.15.3:42242) 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c737fe700 1 -- 172.21.15.3:0/669123503 learned_addr learned my addr 172.21.15.3:0/669123503 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c72ffd700 1 --2- 172.21.15.3:0/669123503 >> v2:172.21.15.153:3300/0 conn(0x7f3c741591a0 0x7f3c740affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758147249 0 0) 0x7f3c740b0a90 con 0x7f3c74159570 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.587+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c60003570 con 0x7f3c74159570 2023-10-06T05:01:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3348179293 0 0) 0x7f3c60003570 con 0x7f3c74159570 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 >> v2:172.21.15.153:3300/0 conn(0x7f3c741591a0 msgr2=0x7f3c740affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 --2- 172.21.15.3:0/669123503 >> v2:172.21.15.153:3300/0 conn(0x7f3c741591a0 0x7f3c740affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 >> v1:172.21.15.153:6789/0 conn(0x7f3c7415dbd0 legacy=0x7f3c740b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c740b36d0 con 0x7f3c74159570 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f3c640017f0 con 0x7f3c74159570 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 >> v1:172.21.15.3:6789/0 conn(0x7f3c74159570 legacy=0x7f3c740b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60003a60 0x7f3c60003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c60004810 0x7f3c60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c600051c0 0x7f3c6000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740b33f0 con 0x7f3c600051c0 2023-10-06T05:01:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6000c0d0 con 0x7f3c60003a60 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6000c290 con 0x7f3c60004810 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c600051c0 0x7f3c6000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c72ffd700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60003a60 0x7f3c60003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c727fc700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c60004810 0x7f3c60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 -- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c60004810 msgr2=0x7f3c60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c60004810 0x7f3c60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 -- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60003a60 msgr2=0x7f3c60003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60003a60 0x7f3c60003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 -- 172.21.15.3:0/669123503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c60003570 con 0x7f3c600051c0 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c727fc700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c60004810 0x7f3c60004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 -- 172.21.15.3:0/669123503 reap_dead start 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.588+0000 7f3c737fe700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c600051c0 0x7f3c6000ba60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400a930 tx=0x7f3c640021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf78b746bec42b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c640070d0 con 0x7f3c600051c0 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c64007230 con 0x7f3c600051c0 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c71ffb700 1 -- 172.21.15.3:0/669123503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c640075d0 con 0x7f3c600051c0 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c798ac700 1 -- 172.21.15.3:0/669123503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c600051c0 msgr2=0x7f3c6000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c798ac700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c600051c0 0x7f3c6000ba60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400a930 tx=0x7f3c640021d0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c798ac700 1 -- 172.21.15.3:0/669123503 shutdown_connections 2023-10-06T05:01:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c798ac700 1 --2- 172.21.15.3:0/669123503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c600051c0 0x7f3c6000ba60 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c798ac700 1 -- 172.21.15.3:0/669123503 >> 172.21.15.3:0/669123503 conn(0x7f3c74070310 msgr2=0x7f3c74070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c798ac700 1 -- 172.21.15.3:0/669123503 shutdown_connections 2023-10-06T05:01:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.589+0000 7f3c798ac700 1 -- 172.21.15.3:0/669123503 wait complete. 2023-10-06T05:01:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.590+0000 7f3c798ac700 1 Processor -- start 2023-10-06T05:01:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.590+0000 7f3c798ac700 1 -- start start 2023-10-06T05:01:03.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c798ac700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c600051c0 0x7f3c7415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c798ac700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c74158510 0x7f3c740c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c798ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c74157260 0x7f3c741542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c798ac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7415b200 con 0x7f3c74157260 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c798ac700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74157af0 con 0x7f3c74158510 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c798ac700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7415bd60 con 0x7f3c600051c0 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c737fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c74157260 0x7f3c741542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c737fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c74157260 0x7f3c741542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54756/0 (socket says 172.21.15.3:54756) 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c737fe700 1 -- 172.21.15.3:0/3304875018 learned_addr learned my addr 172.21.15.3:0/3304875018 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c72ffd700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c600051c0 0x7f3c7415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.591+0000 7f3c727fc700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c74158510 0x7f3c740c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c737fe700 1 -- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c600051c0 msgr2=0x7f3c7415c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c737fe700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c600051c0 0x7f3c7415c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c737fe700 1 -- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c74158510 msgr2=0x7f3c740c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c737fe700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c74158510 0x7f3c740c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c737fe700 1 -- 172.21.15.3:0/3304875018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7415c730 con 0x7f3c74157260 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c737fe700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c74157260 0x7f3c741542c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3c740b1870 tx=0x7f3c64016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c6affd700 1 -- 172.21.15.3:0/3304875018 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c640070d0 con 0x7f3c74157260 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c6affd700 1 -- 172.21.15.3:0/3304875018 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c64001250 con 0x7f3c74157260 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7415b430 con 0x7f3c74157260 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c6affd700 1 -- 172.21.15.3:0/3304875018 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c640017f0 con 0x7f3c74157260 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.592+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7415b9f0 con 0x7f3c74157260 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.593+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7415bef0 con 0x7f3c74157260 2023-10-06T05:01:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.595+0000 7f3c6affd700 1 -- 172.21.15.3:0/3304875018 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c6401eb20 con 0x7f3c74157260 2023-10-06T05:01:03.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.595+0000 7f3c6affd700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c54066f30 0x7f3c540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:03.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.596+0000 7f3c6affd700 1 -- 172.21.15.3:0/3304875018 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3c64087720 con 0x7f3c74157260 2023-10-06T05:01:03.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.596+0000 7f3c72ffd700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c54066f30 0x7f3c540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:03.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.596+0000 7f3c72ffd700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c54066f30 0x7f3c540693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c006f00 tx=0x7f3c6c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:03.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.599+0000 7f3c6affd700 1 -- 172.21.15.3:0/3304875018 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c64054830 con 0x7f3c74157260 2023-10-06T05:01:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.839+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c7404c380 con 0x7f3c74157260 2023-10-06T05:01:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.840+0000 7f3c6affd700 1 -- 172.21.15.3:0/3304875018 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f3c64005b00 con 0x7f3c74157260 2023-10-06T05:01:03.842 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:03.842 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:59.668266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":8,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c54066f30 msgr2=0x7f3c540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c54066f30 0x7f3c540693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c006f00 tx=0x7f3c6c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c74157260 msgr2=0x7f3c741542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c74157260 0x7f3c741542c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3c740b1870 tx=0x7f3c64016000 comp rx=0 tx=0).stop 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 shutdown_connections 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3c54066f30 0x7f3c540693e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c74157260 0x7f3c741542c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c600051c0 0x7f3c7415c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 --2- 172.21.15.3:0/3304875018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c74158510 0x7f3c740c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 >> 172.21.15.3:0/3304875018 conn(0x7f3c74070310 msgr2=0x7f3c740d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 shutdown_connections 2023-10-06T05:01:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:03.842+0000 7f3c798ac700 1 -- 172.21.15.3:0/3304875018 wait complete. 2023-10-06T05:01:03.845 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 81 2023-10-06T05:01:03.856 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T05:01:04.003 INFO:teuthology.orchestra.run.smithi153.stdout:[{"id":7480,"entity":{"name":{"type":"client","num":7480},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4115743899}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":16.312712153,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9048237653618396,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7480 192.168.0.1:0/4115743899","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"1","hostname":"smithi153","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"102508","root":"/"}},{"id":8148,"entity":{"name":{"type":"client","num":8148},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2559196242}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1042,"uptime":18.108680460999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2423.2429336449386,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":497.61108781907632,"halflife":1.5},"recall_caps_throttle2o":{"value":209.78744551395843,"halflife":0.5},"session_cache_liveness":{"value":1169.1188199150877,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1203,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8148 192.168.0.1:0/2559196242","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"126814","root":"/"}}] 2023-10-06T05:01:04.004 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi153", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "102508", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4115743899, "type": "any" }, "name": { "num": 7480, "type": "client" } }, "id": 7480, "inst": "client.7480 192.168.0.1:0/4115743899", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9048237653618396 }, "state": "open", "uptime": 16.312712153 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "126814", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2559196242, "type": "any" }, "name": { "num": 8148, "type": "client" } }, "id": 8148, "inst": "client.8148 192.168.0.1:0/2559196242", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1203, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 2423.2429336449386 }, "recall_caps_throttle": { "halflife": 1.5, "value": 497.6110878190763 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 209.78744551395843 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1042, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1169.1188199150877 }, "state": "open", "uptime": 18.108680461 } ] 2023-10-06T05:01:04.004 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:01:04.005 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:01:04.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:01:04.042 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:01:04.079 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:01:04.079 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:01:04.080 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T05:01:04.080 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:01:04.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:04.127 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[126814]: fuse finished with error 0 and tester_r 0 2023-10-06T05:01:04.129 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:01:04.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:04.129 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:01:04.257 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T05:01:04.257 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:01:04.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T05:01:04.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[102508]: fuse finished with error 0 and tester_r 0 2023-10-06T05:01:04.290 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:01:04.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:04.291 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:01:04.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:01:04.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:04.456 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:01:04.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:01:04.472 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:01:04.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:01:04.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:04.472 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:01:04.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:01:04.510 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:01:04.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:04.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.632+0000 7f7b16f7c700 1 Processor -- start 2023-10-06T05:01:04.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.632+0000 7f7b16f7c700 1 -- start start 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b16f7c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7b100cd950 0x7f7b100b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b16f7c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b100b0ce0 con 0x7f7b100b2740 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b16f7c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b100cb820 con 0x7f7b1015cb50 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b16f7c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b100b0760 con 0x7f7b100cd950 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b15519700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7b100b2740 0x7f7b100cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42258/0 (socket says 172.21.15.3:42258) 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b15519700 1 -- 172.21.15.3:0/3667005166 learned_addr learned my addr 172.21.15.3:0/3667005166 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034116953 0 0) 0x7f7b100b0ce0 con 0x7f7b100b2740 2023-10-06T05:01:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0ffff700 1 --2- 172.21.15.3:0/3667005166 >> v2:172.21.15.153:3300/0 conn(0x7f7b100cd950 0x7f7b100b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7afc003570 con 0x7f7b100b2740 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 218055607 0 0) 0x7f7afc003570 con 0x7f7b100b2740 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 >> v2:172.21.15.153:3300/0 conn(0x7f7b100cd950 msgr2=0x7f7b100b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 --2- 172.21.15.3:0/3667005166 >> v2:172.21.15.153:3300/0 conn(0x7f7b100cd950 0x7f7b100b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 >> v1:172.21.15.153:6789/0 conn(0x7f7b1015cb50 legacy=0x7f7b100b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b100b0a00 con 0x7f7b100b2740 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f7af8001470 con 0x7f7b100b2740 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 >> v1:172.21.15.3:6789/0 conn(0x7f7b100b2740 legacy=0x7f7b100cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7afc003a60 0x7f7afc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc004810 0x7f7afc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7afc0051c0 0x7f7afc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100b0760 con 0x7f7afc0051c0 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc00c160 con 0x7f7afc004810 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc00c320 con 0x7f7afc003a60 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.633+0000 7f7b15519700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7afc0051c0 0x7f7afc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b0ffff700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc004810 0x7f7afc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b14d18700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7afc003a60 0x7f7afc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b15519700 1 -- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7afc003a60 msgr2=0x7f7afc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b15519700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7afc003a60 0x7f7afc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b15519700 1 -- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc004810 msgr2=0x7f7afc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b15519700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc004810 0x7f7afc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b15519700 1 -- 172.21.15.3:0/3667005166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc003570 con 0x7f7afc0051c0 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b15519700 1 -- 172.21.15.3:0/3667005166 reap_dead start 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b15519700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7afc0051c0 0x7f7afc00ba60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c3c0 tx=0x7f7af8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684c3d7d2acb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800cc90 con 0x7f7afc0051c0 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af800cdf0 con 0x7f7afc0051c0 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b0f7fe700 1 -- 172.21.15.3:0/3667005166 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af8005b50 con 0x7f7afc0051c0 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.634+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3667005166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7afc0051c0 msgr2=0x7f7afc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.635+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7afc0051c0 0x7f7afc00ba60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c3c0 tx=0x7f7af8001e50 comp rx=0 tx=0).stop 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.635+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3667005166 shutdown_connections 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.635+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3667005166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7afc0051c0 0x7f7afc00ba60 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.635+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3667005166 >> 172.21.15.3:0/3667005166 conn(0x7f7b10070310 msgr2=0x7f7b10070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.635+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3667005166 shutdown_connections 2023-10-06T05:01:04.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.635+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3667005166 wait complete. 2023-10-06T05:01:04.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.636+0000 7f7b16f7c700 1 Processor -- start 2023-10-06T05:01:04.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.636+0000 7f7b16f7c700 1 -- start start 2023-10-06T05:01:04.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc0051c0 0x7f7b1016c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1015b450 0x7f7b1015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b10169520 0x7f7b10169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1016a4d0 con 0x7f7b1015b450 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10148150 con 0x7f7afc0051c0 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100cf7b0 con 0x7f7b10169520 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1015b450 0x7f7b1015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1015b450 0x7f7b1015b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54766/0 (socket says 172.21.15.3:54766) 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 -- 172.21.15.3:0/3510325653 learned_addr learned my addr 172.21.15.3:0/3510325653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b14d18700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc0051c0 0x7f7b1016c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b15519700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b10169520 0x7f7b10169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 -- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b10169520 msgr2=0x7f7b10169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b10169520 0x7f7b10169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 -- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc0051c0 msgr2=0x7f7b1016c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc0051c0 0x7f7b1016c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 -- 172.21.15.3:0/3510325653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b100ce940 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0ffff700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1015b450 0x7f7b1015b8c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7b00011ac0 tx=0x7f7b000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0cff9700 1 -- 172.21.15.3:0/3510325653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000f940 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0cff9700 1 -- 172.21.15.3:0/3510325653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0000faa0 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1016c890 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b0cff9700 1 -- 172.21.15.3:0/3510325653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000fe20 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.637+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b100cee80 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.638+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b100cf8f0 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.639+0000 7f7b0cff9700 1 -- 172.21.15.3:0/3510325653 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b0001d690 con 0x7f7b1015b450 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.639+0000 7f7b0cff9700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7af0066f30 0x7f7af00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.639+0000 7f7b0cff9700 1 -- 172.21.15.3:0/3510325653 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7b00018070 con 0x7f7b1015b450 2023-10-06T05:01:04.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.640+0000 7f7b14d18700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7af0066f30 0x7f7af00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:04.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.640+0000 7f7b14d18700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7af0066f30 0x7f7af00693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7b100cca20 tx=0x7f7b08001a70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:04.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.644+0000 7f7b0cff9700 1 -- 172.21.15.3:0/3510325653 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b100cf8f0 con 0x7f7b1015b450 2023-10-06T05:01:04.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.884+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b100cf8f0 con 0x7f7b1015b450 2023-10-06T05:01:04.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.884+0000 7f7b0cff9700 1 -- 172.21.15.3:0/3510325653 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7b100cf8f0 con 0x7f7b1015b450 2023-10-06T05:01:04.886 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:04.886 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:59.668266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":8,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:01:04.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7af0066f30 msgr2=0x7f7af00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:04.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7af0066f30 0x7f7af00693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7b100cca20 tx=0x7f7b08001a70 comp rx=0 tx=0).stop 2023-10-06T05:01:04.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1015b450 msgr2=0x7f7b1015b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:04.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1015b450 0x7f7b1015b8c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7b00011ac0 tx=0x7f7b000091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 shutdown_connections 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7af0066f30 0x7f7af00693e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc0051c0 0x7f7b1016c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b10169520 0x7f7b10169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 --2- 172.21.15.3:0/3510325653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1015b450 0x7f7b1015b8c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 >> 172.21.15.3:0/3510325653 conn(0x7f7b10070310 msgr2=0x7f7b100b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 shutdown_connections 2023-10-06T05:01:04.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:04.886+0000 7f7b16f7c700 1 -- 172.21.15.3:0/3510325653 wait complete. 2023-10-06T05:01:04.890 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 81 2023-10-06T05:01:04.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:01:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.018+0000 7f95f486b700 1 Processor -- start 2023-10-06T05:01:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.018+0000 7f95f486b700 1 -- start start 2023-10-06T05:01:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95f486b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f95f00cb780 0x7f95f0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95f486b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95f0159500 0x7f95f00cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95f486b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f00cf130 con 0x7f95f00cb3b0 2023-10-06T05:01:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95f486b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f00ce520 con 0x7f95f0159500 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95f486b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f00ce800 con 0x7f95f00cb780 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95effff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95f0159500 0x7f95f00cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95ef7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f95f00cb3b0 0x7f95f00cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42266/0 (socket says 172.21.15.3:42266) 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95ef7fe700 1 -- 172.21.15.3:0/4227744629 learned_addr learned my addr 172.21.15.3:0/4227744629 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95effff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95f0159500 0x7f95f00cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54768/0 (socket says 172.21.15.3:54768) 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95eeffd700 1 --2- 172.21.15.3:0/4227744629 >> v2:172.21.15.153:3300/0 conn(0x7f95f00cb780 0x7f95f0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95effff700 1 -- 172.21.15.3:0/4227744629 >> v2:172.21.15.153:3300/0 conn(0x7f95f00cb780 msgr2=0x7f95f0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95effff700 1 --2- 172.21.15.3:0/4227744629 >> v2:172.21.15.153:3300/0 conn(0x7f95f00cb780 0x7f95f0158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95effff700 1 -- 172.21.15.3:0/4227744629 >> v1:172.21.15.3:6789/0 conn(0x7f95f00cb3b0 legacy=0x7f95f00cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.019+0000 7f95effff700 1 -- 172.21.15.3:0/4227744629 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f00ceae0 con 0x7f95f0159500 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95effff700 1 --2- 172.21.15.3:0/4227744629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0159500 0x7f95f00cccc0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f95f00892b0 tx=0x7f95e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb65498e4023323 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95ee7fc700 1 -- 172.21.15.3:0/4227744629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e800e070 con 0x7f95f0159500 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95ee7fc700 1 -- 172.21.15.3:0/4227744629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e800bc30 con 0x7f95f0159500 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95ee7fc700 1 -- 172.21.15.3:0/4227744629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8012430 con 0x7f95f0159500 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 -- 172.21.15.3:0/4227744629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0159500 msgr2=0x7f95f00cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 --2- 172.21.15.3:0/4227744629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0159500 0x7f95f00cccc0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f95f00892b0 tx=0x7f95e8009780 comp rx=0 tx=0).stop 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 -- 172.21.15.3:0/4227744629 shutdown_connections 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 --2- 172.21.15.3:0/4227744629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0159500 0x7f95f00cccc0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 --2- 172.21.15.3:0/4227744629 >> v2:172.21.15.153:3300/0 conn(0x7f95f00cb780 0x7f95f0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 -- 172.21.15.3:0/4227744629 >> 172.21.15.3:0/4227744629 conn(0x7f95f0070310 msgr2=0x7f95f0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 -- 172.21.15.3:0/4227744629 shutdown_connections 2023-10-06T05:01:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.020+0000 7f95f486b700 1 -- 172.21.15.3:0/4227744629 wait complete. 2023-10-06T05:01:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.021+0000 7f95f486b700 1 Processor -- start 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95f486b700 1 -- start start 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95f486b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95f00cb3b0 0x7f95f01496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95f486b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f00cb780 0x7f95f00c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95f486b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95f0159500 0x7f95f00c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95f486b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00c8410 con 0x7f95f00cb780 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95f486b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00c86f0 con 0x7f95f00cb3b0 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95f486b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00c89d0 con 0x7f95f0159500 2023-10-06T05:01:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95eeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f00cb780 0x7f95f00c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95eeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f00cb780 0x7f95f00c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54776/0 (socket says 172.21.15.3:54776) 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95eeffd700 1 -- 172.21.15.3:0/2715770022 learned_addr learned my addr 172.21.15.3:0/2715770022 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95effff700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95f0159500 0x7f95f00c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.022+0000 7f95ef7fe700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95f00cb3b0 0x7f95f01496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95eeffd700 1 -- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95f0159500 msgr2=0x7f95f00c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95eeffd700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95f0159500 0x7f95f00c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95eeffd700 1 -- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95f00cb3b0 msgr2=0x7f95f01496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95eeffd700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95f00cb3b0 0x7f95f01496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95eeffd700 1 -- 172.21.15.3:0/2715770022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e8009a20 con 0x7f95f00cb780 2023-10-06T05:01:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95eeffd700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f00cb780 0x7f95f00c7360 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f95e400c7d0 tx=0x7f95e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95d77fe700 1 -- 172.21.15.3:0/2715770022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4011600 con 0x7f95f00cb780 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95d77fe700 1 -- 172.21.15.3:0/2715770022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e4011760 con 0x7f95f00cb780 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f014c8e0 con 0x7f95f00cb780 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95d77fe700 1 -- 172.21.15.3:0/2715770022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4011b00 con 0x7f95f00cb780 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.023+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f014cfb0 con 0x7f95f00cb780 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.024+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f00b3410 con 0x7f95f00cb780 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.024+0000 7f95d77fe700 1 -- 172.21.15.3:0/2715770022 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95e4018350 con 0x7f95f00cb780 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.025+0000 7f95d77fe700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f95dc066f30 0x7f95dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.025+0000 7f95d77fe700 1 -- 172.21.15.3:0/2715770022 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f95e40844b0 con 0x7f95f00cb780 2023-10-06T05:01:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.025+0000 7f95ef7fe700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f95dc066f30 0x7f95dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.025+0000 7f95ef7fe700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f95dc066f30 0x7f95dc0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f95e0007130 tx=0x7f95e0007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.029+0000 7f95d77fe700 1 -- 172.21.15.3:0/2715770022 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95f00b3410 con 0x7f95f00cb780 2023-10-06T05:01:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.225+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95f00b3410 con 0x7f95f00cb780 2023-10-06T05:01:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.226+0000 7f95d77fe700 1 -- 172.21.15.3:0/2715770022 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16832 (secure 0 0 0) 0x7f95f00b3410 con 0x7f95f00cb780 2023-10-06T05:01:05.228 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:05.228 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":120,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:00:39.812612+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:00:35.953744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:00:37.188090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:01:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f95dc066f30 msgr2=0x7f95dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f95dc066f30 0x7f95dc0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f95e0007130 tx=0x7f95e0007eb0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f00cb780 msgr2=0x7f95f00c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f00cb780 0x7f95f00c7360 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f95e400c7d0 tx=0x7f95e4008040 comp rx=0 tx=0).stop 2023-10-06T05:01:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 shutdown_connections 2023-10-06T05:01:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f95dc066f30 0x7f95dc0693e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95f00cb3b0 0x7f95f01496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95f0159500 0x7f95f00c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 --2- 172.21.15.3:0/2715770022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f00cb780 0x7f95f00c7360 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 >> 172.21.15.3:0/2715770022 conn(0x7f95f0070310 msgr2=0x7f95f00cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 shutdown_connections 2023-10-06T05:01:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.228+0000 7f95f486b700 1 -- 172.21.15.3:0/2715770022 wait complete. 2023-10-06T05:01:05.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:00:35.953744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:00:37.188090+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:01:05.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:01:05.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.363+0000 7f4a2f59e700 1 Processor -- start 2023-10-06T05:01:05.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.363+0000 7f4a2f59e700 1 -- start start 2023-10-06T05:01:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4a30067ca0 0x7f4a300b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a30068070 0x7f4a30156d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2f59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a300b3e80 con 0x7f4a30068070 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a300b0f90 con 0x7f4a30067ca0 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300b1ad0 con 0x7f4a30157290 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a30068070 0x7f4a30156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a30068070 0x7f4a30156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54788/0 (socket says 172.21.15.3:54788) 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.364+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2941242894 learned_addr learned my addr 172.21.15.3:0/2941242894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2e59c700 1 --2- 172.21.15.3:0/2941242894 >> v2:172.21.15.153:3300/0 conn(0x7f4a30067ca0 0x7f4a300b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2ed9d700 1 --2- 172.21.15.3:0/2941242894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2941242894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 msgr2=0x7f4a300b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2dd9b700 1 --2- 172.21.15.3:0/2941242894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2941242894 >> v2:172.21.15.153:3300/0 conn(0x7f4a30067ca0 msgr2=0x7f4a300b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2dd9b700 1 --2- 172.21.15.3:0/2941242894 >> v2:172.21.15.153:3300/0 conn(0x7f4a30067ca0 0x7f4a300b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2941242894 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300b11a0 con 0x7f4a30068070 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2dd9b700 1 --2- 172.21.15.3:0/2941242894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a30156d50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f4a300cf240 tx=0x7f4a18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7206d487f4bcc485 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2d59a700 1 -- 172.21.15.3:0/2941242894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800b640 con 0x7f4a30068070 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2d59a700 1 -- 172.21.15.3:0/2941242894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1800b7a0 con 0x7f4a30068070 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.365+0000 7f4a2d59a700 1 -- 172.21.15.3:0/2941242894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800bb60 con 0x7f4a30068070 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2941242894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 msgr2=0x7f4a30156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2941242894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a30156d50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f4a300cf240 tx=0x7f4a18009250 comp rx=0 tx=0).stop 2023-10-06T05:01:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2941242894 shutdown_connections 2023-10-06T05:01:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2941242894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2941242894 >> v2:172.21.15.153:3300/0 conn(0x7f4a30067ca0 0x7f4a300b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2941242894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a30156d50 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2941242894 >> 172.21.15.3:0/2941242894 conn(0x7f4a3006ce40 msgr2=0x7f4a300afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2941242894 shutdown_connections 2023-10-06T05:01:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.366+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2941242894 wait complete. 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.367+0000 7f4a2f59e700 1 Processor -- start 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.367+0000 7f4a2f59e700 1 -- start start 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a30067ca0 0x7f4a300cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a3014f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300ccbf0 con 0x7f4a30068070 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3014fbd0 con 0x7f4a30067ca0 2023-10-06T05:01:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300c7380 con 0x7f4a30157290 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a3014f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a3014f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54794/0 (socket says 172.21.15.3:54794) 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2760557396 learned_addr learned my addr 172.21.15.3:0/2760557396 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2e59c700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a30067ca0 0x7f4a300cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2ed9d700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 msgr2=0x7f4a300cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a30067ca0 msgr2=0x7f4a300cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a30067ca0 0x7f4a300cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 -- 172.21.15.3:0/2760557396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180094f0 con 0x7f4a30068070 2023-10-06T05:01:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2e59c700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a30067ca0 0x7f4a300cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2ed9d700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2dd9b700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a3014f690 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4a3006b400 tx=0x7f4a18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a267fc700 1 -- 172.21.15.3:0/2760557396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18000d10 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a267fc700 1 -- 172.21.15.3:0/2760557396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1800bb20 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3006dee0 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a267fc700 1 -- 172.21.15.3:0/2760557396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18011670 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.368+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3006d5b0 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.369+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3006e070 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.370+0000 7f4a267fc700 1 -- 172.21.15.3:0/2760557396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a180119e0 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.370+0000 7f4a267fc700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a10066f30 0x7f4a100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.370+0000 7f4a267fc700 1 -- 172.21.15.3:0/2760557396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4a18085f60 con 0x7f4a30068070 2023-10-06T05:01:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.371+0000 7f4a2e59c700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a10066f30 0x7f4a100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.371+0000 7f4a2e59c700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a10066f30 0x7f4a100693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4a200076c0 tx=0x7f4a2000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.375+0000 7f4a267fc700 1 -- 172.21.15.3:0/2760557396 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a18052fd0 con 0x7f4a30068070 2023-10-06T05:01:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.567+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4a300b1480 con 0x7f4a30068070 2023-10-06T05:01:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.567+0000 7f4a267fc700 1 -- 172.21.15.3:0/2760557396 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v120) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4a180565f0 con 0x7f4a30068070 2023-10-06T05:01:05.569 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T05:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a10066f30 msgr2=0x7f4a100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a10066f30 0x7f4a100693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4a200076c0 tx=0x7f4a2000b040 comp rx=0 tx=0).stop 2023-10-06T05:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 msgr2=0x7f4a3014f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a3014f690 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4a3006b400 tx=0x7f4a18017000 comp rx=0 tx=0).stop 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 shutdown_connections 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a10066f30 0x7f4a100693e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a30067ca0 0x7f4a300cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a30157290 0x7f4a300cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 --2- 172.21.15.3:0/2760557396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a30068070 0x7f4a3014f690 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 >> 172.21.15.3:0/2760557396 conn(0x7f4a3006ce40 msgr2=0x7f4a30157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 shutdown_connections 2023-10-06T05:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.569+0000 7f4a2f59e700 1 -- 172.21.15.3:0/2760557396 wait complete. 2023-10-06T05:01:05.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:01:05.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.703+0000 7fb4ab47e700 1 Processor -- start 2023-10-06T05:01:05.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.703+0000 7fb4ab47e700 1 -- start start 2023-10-06T05:01:05.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.704+0000 7fb4ab47e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4a41593f0 0x7fb4a406c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.704+0000 7fb4ab47e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.704+0000 7fb4ab47e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb4a40a9ad0 0x7fb4a40a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.704+0000 7fb4ab47e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a406b720 con 0x7fb4a41593f0 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.704+0000 7fb4ab47e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a406ab50 con 0x7fb4a40a9ad0 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.704+0000 7fb4ab47e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a406adf0 con 0x7fb4a41597c0 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.704+0000 7fb4a921a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4a41593f0 0x7fb4a406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4a41593f0 0x7fb4a406c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54798/0 (socket says 172.21.15.3:54798) 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 -- 172.21.15.3:0/1495695045 learned_addr learned my addr 172.21.15.3:0/1495695045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a9a1b700 1 --2- 172.21.15.3:0/1495695045 >> v2:172.21.15.153:3300/0 conn(0x7fb4a40a9ad0 0x7fb4a40a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a8a19700 1 --2- 172.21.15.3:0/1495695045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 -- 172.21.15.3:0/1495695045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 msgr2=0x7fb4a406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 --2- 172.21.15.3:0/1495695045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 -- 172.21.15.3:0/1495695045 >> v2:172.21.15.153:3300/0 conn(0x7fb4a40a9ad0 msgr2=0x7fb4a40a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 --2- 172.21.15.3:0/1495695045 >> v2:172.21.15.153:3300/0 conn(0x7fb4a40a9ad0 0x7fb4a40a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 -- 172.21.15.3:0/1495695045 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a406b0d0 con 0x7fb4a41593f0 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a921a700 1 --2- 172.21.15.3:0/1495695045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a406c370 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb494009f20 tx=0x7fb494009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2df8eacbad347 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a3fff700 1 -- 172.21.15.3:0/1495695045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49400e070 con 0x7fb4a41593f0 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a3fff700 1 -- 172.21.15.3:0/1495695045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49400ba40 con 0x7fb4a41593f0 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.705+0000 7fb4a3fff700 1 -- 172.21.15.3:0/1495695045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4940092e0 con 0x7fb4a41593f0 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 -- 172.21.15.3:0/1495695045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 msgr2=0x7fb4a406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/1495695045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a406c370 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb494009f20 tx=0x7fb494009840 comp rx=0 tx=0).stop 2023-10-06T05:01:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 -- 172.21.15.3:0/1495695045 shutdown_connections 2023-10-06T05:01:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/1495695045 >> v2:172.21.15.153:3300/0 conn(0x7fb4a40a9ad0 0x7fb4a40a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/1495695045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a406c370 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/1495695045 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 -- 172.21.15.3:0/1495695045 >> 172.21.15.3:0/1495695045 conn(0x7fb4a406df50 msgr2=0x7fb4a40aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 -- 172.21.15.3:0/1495695045 shutdown_connections 2023-10-06T05:01:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.706+0000 7fb4ab47e700 1 -- 172.21.15.3:0/1495695045 wait complete. 2023-10-06T05:01:05.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.707+0000 7fb4ab47e700 1 Processor -- start 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.707+0000 7fb4ab47e700 1 -- start start 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4a40a9ad0 0x7fb4a41439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40ae600 con 0x7fb4a41593f0 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40ad9f0 con 0x7fb4a40a9ad0 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40adcd0 con 0x7fb4a41597c0 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54808/0 (socket says 172.21.15.3:54808) 2023-10-06T05:01:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 -- 172.21.15.3:0/2396428021 learned_addr learned my addr 172.21.15.3:0/2396428021 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a921a700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4a40a9ad0 0x7fb4a41439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a9a1b700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 -- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 msgr2=0x7fb4a415c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a415c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 -- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4a40a9ad0 msgr2=0x7fb4a41439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4a40a9ad0 0x7fb4a41439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 -- 172.21.15.3:0/2396428021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb494009ae0 con 0x7fb4a41593f0 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a921a700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4a40a9ad0 0x7fb4a41439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a8a19700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a415bee0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb49800d860 tx=0x7fb498008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a17fa700 1 -- 172.21.15.3:0/2396428021 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498009610 con 0x7fb4a41593f0 2023-10-06T05:01:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a17fa700 1 -- 172.21.15.3:0/2396428021 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb498009770 con 0x7fb4a41593f0 2023-10-06T05:01:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a40adfb0 con 0x7fb4a41593f0 2023-10-06T05:01:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4a17fa700 1 -- 172.21.15.3:0/2396428021 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498009b30 con 0x7fb4a41593f0 2023-10-06T05:01:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.708+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a414fb40 con 0x7fb4a41593f0 2023-10-06T05:01:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.709+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a40ae7b0 con 0x7fb4a41593f0 2023-10-06T05:01:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.710+0000 7fb4a17fa700 1 -- 172.21.15.3:0/2396428021 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb49801e460 con 0x7fb4a41593f0 2023-10-06T05:01:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.710+0000 7fb4a17fa700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb490066f30 0x7fb4900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.710+0000 7fb4a17fa700 1 -- 172.21.15.3:0/2396428021 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb498084d50 con 0x7fb4a41593f0 2023-10-06T05:01:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.710+0000 7fb4a921a700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb490066f30 0x7fb4900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.711+0000 7fb4a921a700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb490066f30 0x7fb4900693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb494009880 tx=0x7fb494018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.715+0000 7fb4a17fa700 1 -- 172.21.15.3:0/2396428021 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4a40ae7b0 con 0x7fb4a41593f0 2023-10-06T05:01:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.910+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4a40b11f0 con 0x7fb4a41593f0 2023-10-06T05:01:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.910+0000 7fb4a17fa700 1 -- 172.21.15.3:0/2396428021 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb498051dc0 con 0x7fb4a41593f0 2023-10-06T05:01:05.912 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T05:01:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb490066f30 msgr2=0x7fb4900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb490066f30 0x7fb4900693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb494009880 tx=0x7fb494018000 comp rx=0 tx=0).stop 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 msgr2=0x7fb4a415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a415bee0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb49800d860 tx=0x7fb498008040 comp rx=0 tx=0).stop 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 shutdown_connections 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb490066f30 0x7fb4900693e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4a40a9ad0 0x7fb4a41439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4a41597c0 0x7fb4a415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 --2- 172.21.15.3:0/2396428021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a41593f0 0x7fb4a415bee0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 >> 172.21.15.3:0/2396428021 conn(0x7fb4a406df50 msgr2=0x7fb4a4155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 shutdown_connections 2023-10-06T05:01:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:05.913+0000 7fb4ab47e700 1 -- 172.21.15.3:0/2396428021 wait complete. 2023-10-06T05:01:05.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:01:06.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.046+0000 7fdea8ff8700 1 Processor -- start 2023-10-06T05:01:06.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.046+0000 7fdea8ff8700 1 -- start start 2023-10-06T05:01:06.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.047+0000 7fdea8ff8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 0x7fdea40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.047+0000 7fdea8ff8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea40afca0 con 0x7fdea40cfb80 2023-10-06T05:01:06.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.047+0000 7fdea8ff8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea415d510 con 0x7fdea40cef10 2023-10-06T05:01:06.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.047+0000 7fdea8ff8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea415a480 con 0x7fdea4159eb0 2023-10-06T05:01:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.047+0000 7fdea259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdea40cfb80 0x7fdea40af590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42280/0 (socket says 172.21.15.3:42280) 2023-10-06T05:01:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.047+0000 7fdea259c700 1 -- 172.21.15.3:0/2701138769 learned_addr learned my addr 172.21.15.3:0/2701138769 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 0x7fdea40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576983163 0 0) 0x7fdea40afca0 con 0x7fdea40cfb80 2023-10-06T05:01:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde90003570 con 0x7fdea40cfb80 2023-10-06T05:01:06.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3185924380 0 0) 0x7fde90003570 con 0x7fdea40cfb80 2023-10-06T05:01:06.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 msgr2=0x7fdea40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:06.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 0x7fdea40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 >> v1:172.21.15.153:6789/0 conn(0x7fdea40cef10 legacy=0x7fdea415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea415a760 con 0x7fdea40cfb80 2023-10-06T05:01:06.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fde94001470 con 0x7fdea40cfb80 2023-10-06T05:01:06.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 >> v1:172.21.15.3:6789/0 conn(0x7fdea40cfb80 legacy=0x7fdea40af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 0x7fde90003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde90004010 0x7fde90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004d50 0x7fde900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea415a480 con 0x7fde90004010 2023-10-06T05:01:06.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9000bbf0 con 0x7fde90004d50 2023-10-06T05:01:06.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9000bdb0 con 0x7fdea4159eb0 2023-10-06T05:01:06.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde90004010 0x7fde90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea259c700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 0x7fde90003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea2d9d700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004d50 0x7fde900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea1d9b700 1 -- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 msgr2=0x7fde90003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:06.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea4159eb0 0x7fde90003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea1d9b700 1 -- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004d50 msgr2=0x7fde900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004d50 0x7fde900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea1d9b700 1 -- 172.21.15.3:0/2701138769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90003570 con 0x7fde90004010 2023-10-06T05:01:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.048+0000 7fdea2d9d700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004d50 0x7fde900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde90004010 0x7fde90004810 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fde8c006f00 tx=0x7fde8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c475fdba2af4dfc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00e710 con 0x7fde90004010 2023-10-06T05:01:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c00e870 con 0x7fde90004010 2023-10-06T05:01:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea159a700 1 -- 172.21.15.3:0/2701138769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00ec30 con 0x7fde90004010 2023-10-06T05:01:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea8ff8700 1 -- 172.21.15.3:0/2701138769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde90004010 msgr2=0x7fde90004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea8ff8700 1 --2- 172.21.15.3:0/2701138769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde90004010 0x7fde90004810 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fde8c006f00 tx=0x7fde8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea2d9d700 1 -- 172.21.15.3:0/2701138769 reap_dead start 2023-10-06T05:01:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea8ff8700 1 -- 172.21.15.3:0/2701138769 shutdown_connections 2023-10-06T05:01:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea8ff8700 1 -- 172.21.15.3:0/2701138769 >> 172.21.15.3:0/2701138769 conn(0x7fdea4070310 msgr2=0x7fdea4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea8ff8700 1 -- 172.21.15.3:0/2701138769 shutdown_connections 2023-10-06T05:01:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.049+0000 7fdea8ff8700 1 -- 172.21.15.3:0/2701138769 wait complete. 2023-10-06T05:01:06.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 Processor -- start 2023-10-06T05:01:06.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 -- start start 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004010 0x7fdea40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdea40b31e0 0x7fdea4156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea415c3e0 0x7fdea415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea415da10 con 0x7fdea40b31e0 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea415cd90 con 0x7fde90004010 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea8ff8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40b3750 con 0x7fdea415c3e0 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdea40b31e0 0x7fdea4156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdea40b31e0 0x7fdea4156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54832/0 (socket says 172.21.15.3:54832) 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.051+0000 7fdea1d9b700 1 -- 172.21.15.3:0/3386419510 learned_addr learned my addr 172.21.15.3:0/3386419510 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea2d9d700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea415c3e0 0x7fdea415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea259c700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004010 0x7fdea40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea1d9b700 1 -- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea415c3e0 msgr2=0x7fdea415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea415c3e0 0x7fdea415c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea1d9b700 1 -- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004010 msgr2=0x7fdea40ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004010 0x7fdea40ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea1d9b700 1 -- 172.21.15.3:0/3386419510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40b3a30 con 0x7fdea40b31e0 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea259c700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004010 0x7fdea40ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea1d9b700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdea40b31e0 0x7fdea4156d30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fde900057d0 tx=0x7fde8c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fde9a7fc700 1 -- 172.21.15.3:0/3386419510 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c022070 con 0x7fdea40b31e0 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fde9a7fc700 1 -- 172.21.15.3:0/3386419510 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c00e9e0 con 0x7fdea40b31e0 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea40b3d10 con 0x7fdea40b31e0 2023-10-06T05:01:06.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fde9a7fc700 1 -- 172.21.15.3:0/3386419510 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c01c450 con 0x7fdea40b31e0 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.052+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea40ca660 con 0x7fdea40b31e0 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.053+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea415dbc0 con 0x7fdea40b31e0 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.053+0000 7fde9a7fc700 1 -- 172.21.15.3:0/3386419510 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde8c01c7c0 con 0x7fdea40b31e0 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.054+0000 7fde9a7fc700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde84066f30 0x7fde840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.054+0000 7fde9a7fc700 1 -- 172.21.15.3:0/3386419510 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde8c085ed0 con 0x7fdea40b31e0 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.054+0000 7fdea259c700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde84066f30 0x7fde840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.054+0000 7fdea259c700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde84066f30 0x7fde840693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fde94000e10 tx=0x7fde94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.058+0000 7fde9a7fc700 1 -- 172.21.15.3:0/3386419510 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdea415dbc0 con 0x7fdea40b31e0 2023-10-06T05:01:06.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.248+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdea404c380 con 0x7fdea40b31e0 2023-10-06T05:01:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.248+0000 7fde9a7fc700 1 -- 172.21.15.3:0/3386419510 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde8c017020 con 0x7fdea40b31e0 2023-10-06T05:01:06.250 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T05:01:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde84066f30 msgr2=0x7fde840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde84066f30 0x7fde840693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fde94000e10 tx=0x7fde94009040 comp rx=0 tx=0).stop 2023-10-06T05:01:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdea40b31e0 msgr2=0x7fdea4156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdea40b31e0 0x7fdea4156d30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fde900057d0 tx=0x7fde8c01a000 comp rx=0 tx=0).stop 2023-10-06T05:01:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 shutdown_connections 2023-10-06T05:01:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fde84066f30 0x7fde840693e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde90004010 0x7fdea40ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdea415c3e0 0x7fdea415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 --2- 172.21.15.3:0/3386419510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdea40b31e0 0x7fdea4156d30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 >> 172.21.15.3:0/3386419510 conn(0x7fdea4070310 msgr2=0x7fdea41588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 shutdown_connections 2023-10-06T05:01:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.250+0000 7fdea8ff8700 1 -- 172.21.15.3:0/3386419510 wait complete. 2023-10-06T05:01:06.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:01:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.384+0000 7f40154c1700 1 Processor -- start 2023-10-06T05:01:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.384+0000 7f40154c1700 1 -- start start 2023-10-06T05:01:06.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f40154c1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40100af660 0x7f40100ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f40154c1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40100cd190 0x7f40100c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f40154c1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40100cb190 con 0x7f40100af290 2023-10-06T05:01:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f40154c1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4010159950 con 0x7f40100cd190 2023-10-06T05:01:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f40154c1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40100cd6a0 con 0x7f40100af660 2023-10-06T05:01:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f4014cc0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40100cd190 0x7f40100c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f400ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40100af290 0x7f40100ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42286/0 (socket says 172.21.15.3:42286) 2023-10-06T05:01:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f400ffff700 1 -- 172.21.15.3:0/3940552713 learned_addr learned my addr 172.21.15.3:0/3940552713 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f400f7fe700 1 --2- 172.21.15.3:0/3940552713 >> v2:172.21.15.153:3300/0 conn(0x7f40100af660 0x7f40100ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f4014cc0700 1 -- 172.21.15.3:0/3940552713 >> v2:172.21.15.153:3300/0 conn(0x7f40100af660 msgr2=0x7f40100ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f4014cc0700 1 --2- 172.21.15.3:0/3940552713 >> v2:172.21.15.153:3300/0 conn(0x7f40100af660 0x7f40100ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f4014cc0700 1 -- 172.21.15.3:0/3940552713 >> v1:172.21.15.3:6789/0 conn(0x7f40100af290 legacy=0x7f40100ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f400effd700 1 -- 172.21.15.3:0/3940552713 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783037495 0 0) 0x7f40100cb190 con 0x7f40100af290 2023-10-06T05:01:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.385+0000 7f4014cc0700 1 -- 172.21.15.3:0/3940552713 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40100cb3f0 con 0x7f40100cd190 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f4014cc0700 1 --2- 172.21.15.3:0/3940552713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100c5c20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f40100d12c0 tx=0x7f4008009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bea66f0a46f835d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f400effd700 1 -- 172.21.15.3:0/3940552713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400800e070 con 0x7f40100cd190 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f400effd700 1 -- 172.21.15.3:0/3940552713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400800bb60 con 0x7f40100cd190 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f400effd700 1 -- 172.21.15.3:0/3940552713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008012430 con 0x7f40100cd190 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 -- 172.21.15.3:0/3940552713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 msgr2=0x7f40100c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 --2- 172.21.15.3:0/3940552713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100c5c20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f40100d12c0 tx=0x7f4008009780 comp rx=0 tx=0).stop 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 -- 172.21.15.3:0/3940552713 shutdown_connections 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 --2- 172.21.15.3:0/3940552713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100c5c20 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 --2- 172.21.15.3:0/3940552713 >> v2:172.21.15.153:3300/0 conn(0x7f40100af660 0x7f40100ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 -- 172.21.15.3:0/3940552713 >> 172.21.15.3:0/3940552713 conn(0x7f4010089dd0 msgr2=0x7f40100996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 -- 172.21.15.3:0/3940552713 shutdown_connections 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.386+0000 7f40154c1700 1 -- 172.21.15.3:0/3940552713 wait complete. 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.387+0000 7f40154c1700 1 Processor -- start 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.388+0000 7f40154c1700 1 -- start start 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.388+0000 7f40154c1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40100af290 0x7f40100b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.388+0000 7f40154c1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40100af660 0x7f401014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.388+0000 7f40154c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.388+0000 7f40154c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40100cff50 con 0x7f40100cd190 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.388+0000 7f40154c1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40100b5540 con 0x7f40100af660 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.388+0000 7f40154c1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40100b4a70 con 0x7f40100af290 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54858/0 (socket says 172.21.15.3:54858) 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 -- 172.21.15.3:0/113103372 learned_addr learned my addr 172.21.15.3:0/113103372 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f400ffff700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40100af290 0x7f40100b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f400f7fe700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40100af660 0x7f401014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 -- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40100af290 msgr2=0x7f40100b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40100af290 0x7f40100b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 -- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40100af660 msgr2=0x7f401014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40100af660 0x7f401014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 -- 172.21.15.3:0/113103372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4008009a20 con 0x7f40100cd190 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f4014cc0700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100b3050 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f40080099f0 tx=0x7f4008019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f3ff7fff700 1 -- 172.21.15.3:0/113103372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008012e80 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f3ff7fff700 1 -- 172.21.15.3:0/113103372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400801d630 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401015dd30 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f3ff7fff700 1 -- 172.21.15.3:0/113103372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400801da10 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.389+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401014c320 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.390+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4010159430 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.390+0000 7f3ff7fff700 1 -- 172.21.15.3:0/113103372 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4008024070 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.391+0000 7f3ff7fff700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ffc066f30 0x7f3ffc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.391+0000 7f3ff7fff700 1 -- 172.21.15.3:0/113103372 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4008086e90 con 0x7f40100cd190 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.391+0000 7f400ffff700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ffc066f30 0x7f3ffc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.392+0000 7f400ffff700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ffc066f30 0x7f3ffc0693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4000001360 tx=0x7f4000009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.395+0000 7f3ff7fff700 1 -- 172.21.15.3:0/113103372 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4008053fa0 con 0x7f40100cd190 2023-10-06T05:01:06.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.586+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f401015dec0 con 0x7f40100cd190 2023-10-06T05:01:06.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.586+0000 7f3ff7fff700 1 -- 172.21.15.3:0/113103372 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40080575c0 con 0x7f40100cd190 2023-10-06T05:01:06.588 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T05:01:06.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ffc066f30 msgr2=0x7f3ffc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ffc066f30 0x7f3ffc0693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4000001360 tx=0x7f4000009040 comp rx=0 tx=0).stop 2023-10-06T05:01:06.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 msgr2=0x7f40100b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100b3050 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f40080099f0 tx=0x7f4008019000 comp rx=0 tx=0).stop 2023-10-06T05:01:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 shutdown_connections 2023-10-06T05:01:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3ffc066f30 0x7f3ffc0693e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40100cd190 0x7f40100b3050 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40100af290 0x7f40100b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 --2- 172.21.15.3:0/113103372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40100af660 0x7f401014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 >> 172.21.15.3:0/113103372 conn(0x7f4010089dd0 msgr2=0x7f40100ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:06.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 shutdown_connections 2023-10-06T05:01:06.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.588+0000 7f40154c1700 1 -- 172.21.15.3:0/113103372 wait complete. 2023-10-06T05:01:06.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.719+0000 7ff11a9ad700 1 Processor -- start 2023-10-06T05:01:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.719+0000 7ff11a9ad700 1 -- start start 2023-10-06T05:01:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.720+0000 7ff11a9ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff11415c8f0 0x7ff11415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.720+0000 7ff11a9ad700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1140cf6c0 con 0x7ff11415ccc0 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.720+0000 7ff11a9ad700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1140cfb50 con 0x7ff1140d0870 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.720+0000 7ff11a9ad700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11415e050 con 0x7ff11415c8f0 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.720+0000 7ff113fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff11415c8f0 0x7ff11415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.720+0000 7ff1137fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff11415ccc0 0x7ff1140d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42298/0 (socket says 172.21.15.3:42298) 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.720+0000 7ff1137fe700 1 -- 172.21.15.3:0/2655127867 learned_addr learned my addr 172.21.15.3:0/2655127867 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff113fff700 1 -- 172.21.15.3:0/2655127867 >> v1:172.21.15.153:6789/0 conn(0x7ff1140d0870 legacy=0x7ff1140b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff113fff700 1 -- 172.21.15.3:0/2655127867 >> v1:172.21.15.3:6789/0 conn(0x7ff11415ccc0 legacy=0x7ff1140d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff112ffd700 1 -- 172.21.15.3:0/2655127867 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590394020 0 0) 0x7ff1140cf6c0 con 0x7ff11415ccc0 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff113fff700 1 -- 172.21.15.3:0/2655127867 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1140ceab0 con 0x7ff11415c8f0 2023-10-06T05:01:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff113fff700 1 --2- 172.21.15.3:0/2655127867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11415db10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009910 tx=0x7ff0fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bcd5fcb819d98f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff112ffd700 1 -- 172.21.15.3:0/2655127867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00b820 con 0x7ff11415c8f0 2023-10-06T05:01:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff112ffd700 1 -- 172.21.15.3:0/2655127867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00b980 con 0x7ff11415c8f0 2023-10-06T05:01:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff112ffd700 1 -- 172.21.15.3:0/2655127867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00bd20 con 0x7ff11415c8f0 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff11a9ad700 1 -- 172.21.15.3:0/2655127867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 msgr2=0x7ff11415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.721+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/2655127867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11415db10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009910 tx=0x7ff0fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.722+0000 7ff11a9ad700 1 -- 172.21.15.3:0/2655127867 shutdown_connections 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.722+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/2655127867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11415db10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.722+0000 7ff11a9ad700 1 -- 172.21.15.3:0/2655127867 >> 172.21.15.3:0/2655127867 conn(0x7ff114070310 msgr2=0x7ff114070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.722+0000 7ff11a9ad700 1 -- 172.21.15.3:0/2655127867 shutdown_connections 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.722+0000 7ff11a9ad700 1 -- 172.21.15.3:0/2655127867 wait complete. 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.723+0000 7ff11a9ad700 1 Processor -- start 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.723+0000 7ff11a9ad700 1 -- start start 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1140d0870 0x7ff11416af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11416d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff11415ccc0 0x7ff11416d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140cc110 con 0x7ff11415c8f0 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11415a670 con 0x7ff1140d0870 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff114159d40 con 0x7ff11415ccc0 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11416d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11416d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54876/0 (socket says 172.21.15.3:54876) 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 -- 172.21.15.3:0/266951712 learned_addr learned my addr 172.21.15.3:0/266951712 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 -- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff11415ccc0 msgr2=0x7ff11416d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff11415ccc0 0x7ff11416d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff113fff700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1140d0870 0x7ff11416af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 -- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1140d0870 msgr2=0x7ff11416af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1140d0870 0x7ff11416af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 -- 172.21.15.3:0/266951712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc009420 con 0x7ff11415c8f0 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff1137fe700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11416d380 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c7e0 tx=0x7ff104007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff10bfff700 1 -- 172.21.15.3:0/266951712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104005c00 con 0x7ff11415c8f0 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff10bfff700 1 -- 172.21.15.3:0/266951712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff104005d80 con 0x7ff11415c8f0 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11415a020 con 0x7ff11415c8f0 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff10bfff700 1 -- 172.21.15.3:0/266951712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104007320 con 0x7ff11415c8f0 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.724+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11416b490 con 0x7ff11415c8f0 2023-10-06T05:01:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.725+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11415a820 con 0x7ff11415c8f0 2023-10-06T05:01:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.726+0000 7ff10bfff700 1 -- 172.21.15.3:0/266951712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff10401b3b0 con 0x7ff11415c8f0 2023-10-06T05:01:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.726+0000 7ff10bfff700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff0f4066f30 0x7ff0f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.726+0000 7ff10bfff700 1 -- 172.21.15.3:0/266951712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff104083c40 con 0x7ff11415c8f0 2023-10-06T05:01:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.726+0000 7ff113fff700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff0f4066f30 0x7ff0f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.727+0000 7ff113fff700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff0f4066f30 0x7ff0f40693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff11415ba80 tx=0x7ff0fc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:06.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.730+0000 7ff10bfff700 1 -- 172.21.15.3:0/266951712 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff11415a820 con 0x7ff11415c8f0 2023-10-06T05:01:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.969+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff11415a820 con 0x7ff11415c8f0 2023-10-06T05:01:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.969+0000 7ff10bfff700 1 -- 172.21.15.3:0/266951712 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4596 (secure 0 0 0) 0x7ff11415a820 con 0x7ff11415c8f0 2023-10-06T05:01:06.971 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:06.971 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:59.668266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":8,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:01:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff0f4066f30 msgr2=0x7ff0f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff0f4066f30 0x7ff0f40693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff11415ba80 tx=0x7ff0fc017000 comp rx=0 tx=0).stop 2023-10-06T05:01:06.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 msgr2=0x7ff11416d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:06.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11416d380 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c7e0 tx=0x7ff104007d70 comp rx=0 tx=0).stop 2023-10-06T05:01:06.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 shutdown_connections 2023-10-06T05:01:06.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff0f4066f30 0x7ff0f40693e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1140d0870 0x7ff11416af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff11415ccc0 0x7ff11416d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 --2- 172.21.15.3:0/266951712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff11415c8f0 0x7ff11416d380 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 >> 172.21.15.3:0/266951712 conn(0x7ff114070310 msgr2=0x7ff1140b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 shutdown_connections 2023-10-06T05:01:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:06.972+0000 7ff11a9ad700 1 -- 172.21.15.3:0/266951712 wait complete. 2023-10-06T05:01:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 81 2023-10-06T05:01:06.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:01:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.103+0000 7fadb099f700 1 Processor -- start 2023-10-06T05:01:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.103+0000 7fadb099f700 1 -- start start 2023-10-06T05:01:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.104+0000 7fadb099f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fadac150100 0x7fadac0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.104+0000 7fadb099f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac0afec0 0x7fadac1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.104+0000 7fadb099f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadac157cb0 con 0x7fadac14fd30 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.104+0000 7fadb099f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac156df0 con 0x7fadac150100 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.104+0000 7fadb099f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac1570a0 con 0x7fadac0afec0 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fadac150100 0x7fadac0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fadac150100 0x7fadac0b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:35876/0 (socket says 172.21.15.3:35876) 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 -- 172.21.15.3:0/297937291 learned_addr learned my addr 172.21.15.3:0/297937291 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadabfff700 1 --2- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac0afec0 0x7fadac1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 -- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac0afec0 msgr2=0x7fadac1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 --2- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac0afec0 0x7fadac1568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 -- 172.21.15.3:0/297937291 >> v1:172.21.15.153:6789/0 conn(0x7fadac14fd30 legacy=0x7fadac0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 -- 172.21.15.3:0/297937291 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac157380 con 0x7fadac150100 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaaffd700 1 --2- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac150100 0x7fadac0b19c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fada00099b0 tx=0x7fada0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbaa66af71d15183 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaa7fc700 1 -- 172.21.15.3:0/297937291 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000b820 con 0x7fadac150100 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaa7fc700 1 -- 172.21.15.3:0/297937291 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada000b980 con 0x7fadac150100 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.105+0000 7fadaa7fc700 1 -- 172.21.15.3:0/297937291 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000bd20 con 0x7fadac150100 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 -- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac150100 msgr2=0x7fadac0b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 --2- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac150100 0x7fadac0b19c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fada00099b0 tx=0x7fada0009250 comp rx=0 tx=0).stop 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 -- 172.21.15.3:0/297937291 shutdown_connections 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 --2- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac0afec0 0x7fadac1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 --2- 172.21.15.3:0/297937291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac150100 0x7fadac0b19c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 -- 172.21.15.3:0/297937291 >> 172.21.15.3:0/297937291 conn(0x7fadac06c370 msgr2=0x7fadac06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 -- 172.21.15.3:0/297937291 shutdown_connections 2023-10-06T05:01:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.106+0000 7fadb099f700 1 -- 172.21.15.3:0/297937291 wait complete. 2023-10-06T05:01:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.107+0000 7fadb099f700 1 Processor -- start 2023-10-06T05:01:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.107+0000 7fadb099f700 1 -- start start 2023-10-06T05:01:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadb099f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac0afec0 0x7fadac14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadb099f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac14fd30 0x7fadac1472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadb099f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadac14ac70 0x7fadac147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadb099f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac147d60 con 0x7fadac14ac70 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadb099f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac154860 con 0x7fadac0afec0 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadb099f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac154b40 con 0x7fadac14fd30 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadac14ac70 0x7fadac147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadac14ac70 0x7fadac147820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54888/0 (socket says 172.21.15.3:54888) 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadabfff700 1 -- 172.21.15.3:0/2585392401 learned_addr learned my addr 172.21.15.3:0/2585392401 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadab7fe700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac0afec0 0x7fadac14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadaaffd700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac14fd30 0x7fadac1472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadabfff700 1 -- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac14fd30 msgr2=0x7fadac1472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadabfff700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac14fd30 0x7fadac1472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadabfff700 1 -- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac0afec0 msgr2=0x7fadac14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.108+0000 7fadabfff700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac0afec0 0x7fadac14a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.109+0000 7fadabfff700 1 -- 172.21.15.3:0/2585392401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00094f0 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.109+0000 7fadabfff700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadac14ac70 0x7fadac147820 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fada4010320 tx=0x7fada4009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.109+0000 7fad937fe700 1 -- 172.21.15.3:0/2585392401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada4010e60 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.109+0000 7fad937fe700 1 -- 172.21.15.3:0/2585392401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada4013470 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.109+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac151a80 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.109+0000 7fad937fe700 1 -- 172.21.15.3:0/2585392401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada4013810 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.109+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac154e20 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.110+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac151fa0 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.110+0000 7fad937fe700 1 -- 172.21.15.3:0/2585392401 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fada4013b80 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.111+0000 7fad937fe700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.111+0000 7fad937fe700 1 -- 172.21.15.3:0/2585392401 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fada4017030 con 0x7fadac14ac70 2023-10-06T05:01:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.111+0000 7fadab7fe700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.111+0000 7fadab7fe700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0039a0 tx=0x7fad9c007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.115+0000 7fad937fe700 1 -- 172.21.15.3:0/2585392401 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadac151fa0 con 0x7fadac14ac70 2023-10-06T05:01:07.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.310+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadac151fa0 con 0x7fadac14ac70 2023-10-06T05:01:07.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.311+0000 7fad937fe700 1 -- 172.21.15.3:0/2585392401 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16832 (secure 0 0 0) 0x7fadac151fa0 con 0x7fadac14ac70 2023-10-06T05:01:07.313 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:07.313 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":120,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:00:39.812612+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:00:35.953744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:00:37.188090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:01:07.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad98066f30 msgr2=0x7fad980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0039a0 tx=0x7fad9c007ea0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadac14ac70 msgr2=0x7fadac147820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadac14ac70 0x7fadac147820 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fada4010320 tx=0x7fada4009990 comp rx=0 tx=0).stop 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 shutdown_connections 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadac0afec0 0x7fadac14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadac14ac70 0x7fadac147820 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 --2- 172.21.15.3:0/2585392401 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadac14fd30 0x7fadac1472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 >> 172.21.15.3:0/2585392401 conn(0x7fadac06c370 msgr2=0x7fadac06ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 shutdown_connections 2023-10-06T05:01:07.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.313+0000 7fadb099f700 1 -- 172.21.15.3:0/2585392401 wait complete. 2023-10-06T05:01:07.327 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:01:07.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:07.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.445+0000 7f263f04c700 1 Processor -- start 2023-10-06T05:01:07.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.445+0000 7f263f04c700 1 -- start start 2023-10-06T05:01:07.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f263f04c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2638159500 0x7f26380cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f263f04c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26381598d0 0x7f26380cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f263f04c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380cc5c0 con 0x7f26380b2220 2023-10-06T05:01:07.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f263f04c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380cb7b0 con 0x7f26381598d0 2023-10-06T05:01:07.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f263f04c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380cda60 con 0x7f2638159500 2023-10-06T05:01:07.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f2637fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26381598d0 0x7f26380cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f2637fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26381598d0 0x7f26380cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54900/0 (socket says 172.21.15.3:54900) 2023-10-06T05:01:07.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.446+0000 7f2637fff700 1 -- 172.21.15.3:0/1117145025 learned_addr learned my addr 172.21.15.3:0/1117145025 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f263cde8700 1 --2- 172.21.15.3:0/1117145025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2638159500 0x7f26380cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f2637fff700 1 -- 172.21.15.3:0/1117145025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2638159500 msgr2=0x7f26380cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f2637fff700 1 --2- 172.21.15.3:0/1117145025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2638159500 0x7f26380cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f2637fff700 1 -- 172.21.15.3:0/1117145025 >> v1:172.21.15.153:6789/0 conn(0x7f26380b2220 legacy=0x7f26380b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f2637fff700 1 -- 172.21.15.3:0/1117145025 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380cb9b0 con 0x7f26381598d0 2023-10-06T05:01:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f263cde8700 1 --2- 172.21.15.3:0/1117145025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2638159500 0x7f26380cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f2637fff700 1 --2- 172.21.15.3:0/1117145025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26381598d0 0x7f26380cb270 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f26280099b0 tx=0x7f2628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a26bd83e59596 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f26377fe700 1 -- 172.21.15.3:0/1117145025 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800b820 con 0x7f26381598d0 2023-10-06T05:01:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f26377fe700 1 -- 172.21.15.3:0/1117145025 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262800b980 con 0x7f26381598d0 2023-10-06T05:01:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.447+0000 7f26377fe700 1 -- 172.21.15.3:0/1117145025 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800bd20 con 0x7f26381598d0 2023-10-06T05:01:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 -- 172.21.15.3:0/1117145025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26381598d0 msgr2=0x7f26380cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 --2- 172.21.15.3:0/1117145025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26381598d0 0x7f26380cb270 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f26280099b0 tx=0x7f2628009250 comp rx=0 tx=0).stop 2023-10-06T05:01:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 -- 172.21.15.3:0/1117145025 shutdown_connections 2023-10-06T05:01:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 --2- 172.21.15.3:0/1117145025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2638159500 0x7f26380cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 --2- 172.21.15.3:0/1117145025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26381598d0 0x7f26380cb270 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 -- 172.21.15.3:0/1117145025 >> 172.21.15.3:0/1117145025 conn(0x7f2638070310 msgr2=0x7f2638070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 -- 172.21.15.3:0/1117145025 shutdown_connections 2023-10-06T05:01:07.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.448+0000 7f263f04c700 1 -- 172.21.15.3:0/1117145025 wait complete. 2023-10-06T05:01:07.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.449+0000 7f263f04c700 1 Processor -- start 2023-10-06T05:01:07.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.449+0000 7f263f04c700 1 -- start start 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263f04c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26380b2220 0x7f26380b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263f04c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2638159500 0x7f2638155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263f04c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381598d0 0x7f26380c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263f04c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b42e0 con 0x7f2638159500 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263f04c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b45c0 con 0x7f26381598d0 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263f04c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b48a0 con 0x7f26380b2220 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2638159500 0x7f2638155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2638159500 0x7f2638155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54904/0 (socket says 172.21.15.3:54904) 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 -- 172.21.15.3:0/1786759300 learned_addr learned my addr 172.21.15.3:0/1786759300 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263cde8700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26380b2220 0x7f26380b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 -- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26380b2220 msgr2=0x7f26380b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26380b2220 0x7f26380b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263d5e9700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381598d0 0x7f26380c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 -- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381598d0 msgr2=0x7f26380c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381598d0 0x7f26380c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 -- 172.21.15.3:0/1786759300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280094f0 con 0x7f2638159500 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263cde8700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26380b2220 0x7f26380b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f263d5e9700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381598d0 0x7f26380c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2637fff700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2638159500 0x7f2638155bc0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2628009290 tx=0x7f2628016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.450+0000 7f2634ff9700 1 -- 172.21.15.3:0/1786759300 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628011d70 con 0x7f2638159500 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.451+0000 7f2634ff9700 1 -- 172.21.15.3:0/1786759300 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628020070 con 0x7f2638159500 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.451+0000 7f2634ff9700 1 -- 172.21.15.3:0/1786759300 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262801b6a0 con 0x7f2638159500 2023-10-06T05:01:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.451+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26380b4b80 con 0x7f2638159500 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.451+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2638156100 con 0x7f2638159500 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.451+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26380b50a0 con 0x7f2638159500 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.452+0000 7f2634ff9700 1 -- 172.21.15.3:0/1786759300 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2628022030 con 0x7f2638159500 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.452+0000 7f2634ff9700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2618066f30 0x7f26180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.452+0000 7f2634ff9700 1 -- 172.21.15.3:0/1786759300 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2628086550 con 0x7f2638159500 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.452+0000 7f263cde8700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2618066f30 0x7f26180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.453+0000 7f263cde8700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2618066f30 0x7f26180693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2620008b20 tx=0x7f2620008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.457+0000 7f2634ff9700 1 -- 172.21.15.3:0/1786759300 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26380b50a0 con 0x7f2638159500 2023-10-06T05:01:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.695+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26380b50a0 con 0x7f2638159500 2023-10-06T05:01:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.696+0000 7f2634ff9700 1 -- 172.21.15.3:0/1786759300 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f26380b50a0 con 0x7f2638159500 2023-10-06T05:01:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:07.698 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:59.668266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":8,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:01:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2618066f30 msgr2=0x7f26180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2618066f30 0x7f26180693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2620008b20 tx=0x7f2620008040 comp rx=0 tx=0).stop 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2638159500 msgr2=0x7f2638155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2638159500 0x7f2638155bc0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2628009290 tx=0x7f2628016000 comp rx=0 tx=0).stop 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 shutdown_connections 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2618066f30 0x7f26180693e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381598d0 0x7f26380c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26380b2220 0x7f26380b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 --2- 172.21.15.3:0/1786759300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2638159500 0x7f2638155bc0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 >> 172.21.15.3:0/1786759300 conn(0x7f2638070310 msgr2=0x7f26380cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 shutdown_connections 2023-10-06T05:01:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.698+0000 7f263f04c700 1 -- 172.21.15.3:0/1786759300 wait complete. 2023-10-06T05:01:07.702 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 81 2023-10-06T05:01:07.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.829+0000 7fa62a1a3700 1 Processor -- start 2023-10-06T05:01:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.829+0000 7fa62a1a3700 1 -- start start 2023-10-06T05:01:07.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa62a1a3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa62406b310 0x7fa6241586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa62a1a3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa6240af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa62a1a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa624151600 0x7fa624151a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa62a1a3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa624158c00 con 0x7fa624151600 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa62a1a3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62415b190 con 0x7fa62406b310 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa62a1a3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62415ba30 con 0x7fa62406b6e0 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa623fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa624151600 0x7fa624151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa623fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa624151600 0x7fa624151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54918/0 (socket says 172.21.15.3:54918) 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.830+0000 7fa623fff700 1 -- 172.21.15.3:0/1649071608 learned_addr learned my addr 172.21.15.3:0/1649071608 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa6237fe700 1 --2- 172.21.15.3:0/1649071608 >> v2:172.21.15.153:3300/0 conn(0x7fa62406b310 0x7fa6241586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa622ffd700 1 --2- 172.21.15.3:0/1649071608 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa6240af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa623fff700 1 -- 172.21.15.3:0/1649071608 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 msgr2=0x7fa6240af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa623fff700 1 --2- 172.21.15.3:0/1649071608 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa6240af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa623fff700 1 -- 172.21.15.3:0/1649071608 >> v2:172.21.15.153:3300/0 conn(0x7fa62406b310 msgr2=0x7fa6241586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa623fff700 1 --2- 172.21.15.3:0/1649071608 >> v2:172.21.15.153:3300/0 conn(0x7fa62406b310 0x7fa6241586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa623fff700 1 -- 172.21.15.3:0/1649071608 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6240af960 con 0x7fa624151600 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa622ffd700 1 --2- 172.21.15.3:0/1649071608 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa6240af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa623fff700 1 --2- 172.21.15.3:0/1649071608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624151a40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa624070d50 tx=0x7fa61c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4960b4e93288d837 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa6227fc700 1 -- 172.21.15.3:0/1649071608 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00e070 con 0x7fa624151600 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa6227fc700 1 -- 172.21.15.3:0/1649071608 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c00ba50 con 0x7fa624151600 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa6227fc700 1 -- 172.21.15.3:0/1649071608 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0092e0 con 0x7fa624151600 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.831+0000 7fa62a1a3700 1 -- 172.21.15.3:0/1649071608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 msgr2=0x7fa624151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/1649071608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624151a40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa624070d50 tx=0x7fa61c009840 comp rx=0 tx=0).stop 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 -- 172.21.15.3:0/1649071608 shutdown_connections 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/1649071608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624151a40 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/1649071608 >> v2:172.21.15.153:3300/0 conn(0x7fa62406b310 0x7fa6241586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/1649071608 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa6240af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 -- 172.21.15.3:0/1649071608 >> 172.21.15.3:0/1649071608 conn(0x7fa6240cdc30 msgr2=0x7fa62406a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 -- 172.21.15.3:0/1649071608 shutdown_connections 2023-10-06T05:01:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.832+0000 7fa62a1a3700 1 -- 172.21.15.3:0/1649071608 wait complete. 2023-10-06T05:01:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.833+0000 7fa62a1a3700 1 Processor -- start 2023-10-06T05:01:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.833+0000 7fa62a1a3700 1 -- start start 2023-10-06T05:01:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa62406b310 0x7fa62414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa624156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624167c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624168cb0 con 0x7fa624151600 2023-10-06T05:01:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624155f30 con 0x7fa62406b310 2023-10-06T05:01:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa624156790 con 0x7fa62406b6e0 2023-10-06T05:01:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624167c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54924/0 (socket says 172.21.15.3:54924) 2023-10-06T05:01:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 -- 172.21.15.3:0/2473664090 learned_addr learned my addr 172.21.15.3:0/2473664090 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 -- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 msgr2=0x7fa624156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa624156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 -- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa62406b310 msgr2=0x7fa62414cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa62406b310 0x7fa62414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 -- 172.21.15.3:0/2473664090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c009ae0 con 0x7fa624151600 2023-10-06T05:01:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa623fff700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624167c80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa624070d80 tx=0x7fa61c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa60b7fe700 1 -- 172.21.15.3:0/2473664090 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0032f0 con 0x7fa624151600 2023-10-06T05:01:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa60b7fe700 1 -- 172.21.15.3:0/2473664090 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c007e70 con 0x7fa624151600 2023-10-06T05:01:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6241542e0 con 0x7fa624151600 2023-10-06T05:01:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa60b7fe700 1 -- 172.21.15.3:0/2473664090 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c01d650 con 0x7fa624151600 2023-10-06T05:01:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.834+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa624156420 con 0x7fa624151600 2023-10-06T05:01:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.835+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62414d310 con 0x7fa624151600 2023-10-06T05:01:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.836+0000 7fa60b7fe700 1 -- 172.21.15.3:0/2473664090 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa61c003040 con 0x7fa624151600 2023-10-06T05:01:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.836+0000 7fa60b7fe700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.836+0000 7fa60b7fe700 1 -- 172.21.15.3:0/2473664090 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa61c085ec0 con 0x7fa624151600 2023-10-06T05:01:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.837+0000 7fa6237fe700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.837+0000 7fa6237fe700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa6240c1fb0 tx=0x7fa614008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:07.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:07.841+0000 7fa60b7fe700 1 -- 172.21.15.3:0/2473664090 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa61c052fd0 con 0x7fa624151600 2023-10-06T05:01:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.080+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa624071680 con 0x7fa624151600 2023-10-06T05:01:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.081+0000 7fa60b7fe700 1 -- 172.21.15.3:0/2473664090 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fa61c009940 con 0x7fa624151600 2023-10-06T05:01:08.083 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:08.083 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:00:38.212073+0000","modified":"2023-10-06T05:00:59.668266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8010":{"gid":8010,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":8,"addr":"172.21.15.153:6837/3912023637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3912023637},{"type":"v1","addr":"172.21.15.153:6837","nonce":3912023637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:01:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa610066f30 msgr2=0x7fa6100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa6240c1fb0 tx=0x7fa614008040 comp rx=0 tx=0).stop 2023-10-06T05:01:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 msgr2=0x7fa624167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624167c80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa624070d80 tx=0x7fa61c018000 comp rx=0 tx=0).stop 2023-10-06T05:01:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 shutdown_connections 2023-10-06T05:01:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa62406b310 0x7fa62414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa624151600 0x7fa624167c80 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 --2- 172.21.15.3:0/2473664090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa62406b6e0 0x7fa624156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 >> 172.21.15.3:0/2473664090 conn(0x7fa6240cdc30 msgr2=0x7fa62406a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 shutdown_connections 2023-10-06T05:01:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.083+0000 7fa62a1a3700 1 -- 172.21.15.3:0/2473664090 wait complete. 2023-10-06T05:01:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 81 2023-10-06T05:01:08.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:01:08.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.215+0000 7f17d6073700 1 Processor -- start 2023-10-06T05:01:08.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.215+0000 7f17d6073700 1 -- start start 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17d6073700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f17d015c8f0 0x7f17d015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17d6073700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17d00d0870 0x7f17d00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17d6073700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00cf6c0 con 0x7f17d015ccc0 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17d6073700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d00cfb50 con 0x7f17d00d0870 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17d6073700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d015e050 con 0x7f17d015c8f0 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17d00d0870 0x7f17d00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17d00d0870 0x7f17d00b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54930/0 (socket says 172.21.15.3:54930) 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 -- 172.21.15.3:0/2940521646 learned_addr learned my addr 172.21.15.3:0/2940521646 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cf7fe700 1 --2- 172.21.15.3:0/2940521646 >> v2:172.21.15.153:3300/0 conn(0x7f17d015c8f0 0x7f17d015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 -- 172.21.15.3:0/2940521646 >> v2:172.21.15.153:3300/0 conn(0x7f17d015c8f0 msgr2=0x7f17d015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 --2- 172.21.15.3:0/2940521646 >> v2:172.21.15.153:3300/0 conn(0x7f17d015c8f0 0x7f17d015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 -- 172.21.15.3:0/2940521646 >> v1:172.21.15.153:6789/0 conn(0x7f17d015ccc0 legacy=0x7f17d00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 -- 172.21.15.3:0/2940521646 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00ceab0 con 0x7f17d00d0870 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17cffff700 1 --2- 172.21.15.3:0/2940521646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d00d0870 0x7f17d00b3ba0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009f20 tx=0x7f17c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88586b8528967ad8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17ce7fc700 1 -- 172.21.15.3:0/2940521646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800e070 con 0x7f17d00d0870 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17ce7fc700 1 -- 172.21.15.3:0/2940521646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c800b990 con 0x7f17d00d0870 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.216+0000 7f17ce7fc700 1 -- 172.21.15.3:0/2940521646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80092e0 con 0x7f17d00d0870 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 -- 172.21.15.3:0/2940521646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d00d0870 msgr2=0x7f17d00b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 --2- 172.21.15.3:0/2940521646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d00d0870 0x7f17d00b3ba0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009f20 tx=0x7f17c8009840 comp rx=0 tx=0).stop 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 -- 172.21.15.3:0/2940521646 shutdown_connections 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 --2- 172.21.15.3:0/2940521646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d00d0870 0x7f17d00b3ba0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 --2- 172.21.15.3:0/2940521646 >> v2:172.21.15.153:3300/0 conn(0x7f17d015c8f0 0x7f17d015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 -- 172.21.15.3:0/2940521646 >> 172.21.15.3:0/2940521646 conn(0x7f17d0070310 msgr2=0x7f17d0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:08.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 -- 172.21.15.3:0/2940521646 shutdown_connections 2023-10-06T05:01:08.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.217+0000 7f17d6073700 1 -- 172.21.15.3:0/2940521646 wait complete. 2023-10-06T05:01:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.218+0000 7f17d6073700 1 Processor -- start 2023-10-06T05:01:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.218+0000 7f17d6073700 1 -- start start 2023-10-06T05:01:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17d6073700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d00d0870 0x7f17d01539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17d6073700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17d015c8f0 0x7f17d0154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17d6073700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d015ccc0 0x7f17d014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17d6073700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0155630 con 0x7f17d015ccc0 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17d6073700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0154780 con 0x7f17d00d0870 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17d6073700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0154a20 con 0x7f17d015c8f0 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d015ccc0 0x7f17d014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d015ccc0 0x7f17d014f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54936/0 (socket says 172.21.15.3:54936) 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 -- 172.21.15.3:0/2654480478 learned_addr learned my addr 172.21.15.3:0/2654480478 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cf7fe700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d00d0870 0x7f17d01539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17ceffd700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17d015c8f0 0x7f17d0154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 -- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17d015c8f0 msgr2=0x7f17d0154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17d015c8f0 0x7f17d0154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 -- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d00d0870 msgr2=0x7f17d01539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d00d0870 0x7f17d01539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.219+0000 7f17cffff700 1 -- 172.21.15.3:0/2654480478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8009ae0 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.220+0000 7f17cffff700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d015ccc0 0x7f17d014f0c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f17c8012000 tx=0x7f17c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.220+0000 7f17c77fe700 1 -- 172.21.15.3:0/2654480478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8008ad0 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.220+0000 7f17c77fe700 1 -- 172.21.15.3:0/2654480478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c8008c70 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.220+0000 7f17c77fe700 1 -- 172.21.15.3:0/2654480478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c801c410 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.220+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d0154d00 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.220+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d01552c0 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.221+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d01557e0 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.221+0000 7f17c77fe700 1 -- 172.21.15.3:0/2654480478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17c8003040 con 0x7f17d015ccc0 2023-10-06T05:01:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.222+0000 7f17c77fe700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f17b0066f30 0x7f17b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.222+0000 7f17c77fe700 1 -- 172.21.15.3:0/2654480478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f17c8085d80 con 0x7f17d015ccc0 2023-10-06T05:01:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.222+0000 7f17cf7fe700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f17b0066f30 0x7f17b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.222+0000 7f17cf7fe700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f17b0066f30 0x7f17b00693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009db0 tx=0x7f17c00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.226+0000 7f17c77fe700 1 -- 172.21.15.3:0/2654480478 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17d01557e0 con 0x7f17d015ccc0 2023-10-06T05:01:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.420+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17d0071680 con 0x7f17d015ccc0 2023-10-06T05:01:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.421+0000 7f17c77fe700 1 -- 172.21.15.3:0/2654480478 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16832 (secure 0 0 0) 0x7f17c80721d0 con 0x7f17d015ccc0 2023-10-06T05:01:08.423 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:08.423 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":120,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:00:39.812612+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:00:35.953744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:00:37.188090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f17b0066f30 msgr2=0x7f17b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f17b0066f30 0x7f17b00693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009db0 tx=0x7f17c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d015ccc0 msgr2=0x7f17d014f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d015ccc0 0x7f17d014f0c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f17c8012000 tx=0x7f17c8018000 comp rx=0 tx=0).stop 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 shutdown_connections 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f17b0066f30 0x7f17b00693e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d00d0870 0x7f17d01539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17d015ccc0 0x7f17d014f0c0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 --2- 172.21.15.3:0/2654480478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17d015c8f0 0x7f17d0154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 >> 172.21.15.3:0/2654480478 conn(0x7f17d0070310 msgr2=0x7f17d00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 shutdown_connections 2023-10-06T05:01:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.423+0000 7f17d6073700 1 -- 172.21.15.3:0/2654480478 wait complete. 2023-10-06T05:01:08.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:01:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.556+0000 7fa6fed59700 1 Processor -- start 2023-10-06T05:01:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.556+0000 7fa6fed59700 1 -- start start 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fed59700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6f815c8f0 0x7fa6f815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fed59700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815ccc0 0x7fa6f80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fed59700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6f80d06a0 0x7fa6f806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fed59700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80cf6c0 con 0x7fa6f80d06a0 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fed59700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80cfb50 con 0x7fa6f815c8f0 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fed59700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815e050 con 0x7fa6f815ccc0 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fd2f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6f80d06a0 0x7fa6f806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fd2f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6f80d06a0 0x7fa6f806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54950/0 (socket says 172.21.15.3:54950) 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.557+0000 7fa6fd2f6700 1 -- 172.21.15.3:0/105984064 learned_addr learned my addr 172.21.15.3:0/105984064 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6f7fff700 1 --2- 172.21.15.3:0/105984064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815ccc0 0x7fa6f80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6fcaf5700 1 --2- 172.21.15.3:0/105984064 >> v2:172.21.15.153:3300/0 conn(0x7fa6f815c8f0 0x7fa6f815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6fd2f6700 1 -- 172.21.15.3:0/105984064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815ccc0 msgr2=0x7fa6f80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6fd2f6700 1 --2- 172.21.15.3:0/105984064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815ccc0 0x7fa6f80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6fd2f6700 1 -- 172.21.15.3:0/105984064 >> v2:172.21.15.153:3300/0 conn(0x7fa6f815c8f0 msgr2=0x7fa6f815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6fd2f6700 1 --2- 172.21.15.3:0/105984064 >> v2:172.21.15.153:3300/0 conn(0x7fa6f815c8f0 0x7fa6f815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6fd2f6700 1 -- 172.21.15.3:0/105984064 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80d0af0 con 0x7fa6f80d06a0 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6fd2f6700 1 --2- 172.21.15.3:0/105984064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f806d950 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80d1550 tx=0x7fa6f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a787293b172dd516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6f77fe700 1 -- 172.21.15.3:0/105984064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000e070 con 0x7fa6f80d06a0 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6f77fe700 1 -- 172.21.15.3:0/105984064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000b970 con 0x7fa6f80d06a0 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.558+0000 7fa6f77fe700 1 -- 172.21.15.3:0/105984064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00092e0 con 0x7fa6f80d06a0 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 -- 172.21.15.3:0/105984064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 msgr2=0x7fa6f806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 --2- 172.21.15.3:0/105984064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f806d950 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80d1550 tx=0x7fa6f0009840 comp rx=0 tx=0).stop 2023-10-06T05:01:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 -- 172.21.15.3:0/105984064 shutdown_connections 2023-10-06T05:01:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 --2- 172.21.15.3:0/105984064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f806d950 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 --2- 172.21.15.3:0/105984064 >> v2:172.21.15.153:3300/0 conn(0x7fa6f815c8f0 0x7fa6f815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 --2- 172.21.15.3:0/105984064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815ccc0 0x7fa6f80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 -- 172.21.15.3:0/105984064 >> 172.21.15.3:0/105984064 conn(0x7fa6f8070310 msgr2=0x7fa6f8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 -- 172.21.15.3:0/105984064 shutdown_connections 2023-10-06T05:01:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.559+0000 7fa6fed59700 1 -- 172.21.15.3:0/105984064 wait complete. 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.560+0000 7fa6fed59700 1 Processor -- start 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.560+0000 7fa6fed59700 1 -- start start 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fed59700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f8156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fed59700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815c8f0 0x7fa6f80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fed59700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f815ccc0 0x7fa6f80c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fed59700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c5550 con 0x7fa6f80d06a0 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fed59700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8156c40 con 0x7fa6f815ccc0 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fed59700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f816cf70 con 0x7fa6f815c8f0 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f8156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f8156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54960/0 (socket says 172.21.15.3:54960) 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 -- 172.21.15.3:0/532113196 learned_addr learned my addr 172.21.15.3:0/532113196 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6f7fff700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815c8f0 0x7fa6f80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fd2f6700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f815ccc0 0x7fa6f80c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 -- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815c8f0 msgr2=0x7fa6f80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815c8f0 0x7fa6f80c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 -- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f815ccc0 msgr2=0x7fa6f80c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f815ccc0 0x7fa6f80c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 -- 172.21.15.3:0/532113196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0009ae0 con 0x7fa6f80d06a0 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fcaf5700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f8156700 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00ec80 tx=0x7fa6ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6f4ff9700 1 -- 172.21.15.3:0/532113196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00c850 con 0x7fa6f80d06a0 2023-10-06T05:01:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6f4ff9700 1 -- 172.21.15.3:0/532113196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec00c9b0 con 0x7fa6f80d06a0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f816d250 con 0x7fa6f80d06a0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.561+0000 7fa6f4ff9700 1 -- 172.21.15.3:0/532113196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00cd50 con 0x7fa6f80d06a0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.562+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f816d810 con 0x7fa6f80d06a0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.562+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8071680 con 0x7fa6f80d06a0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.563+0000 7fa6f4ff9700 1 -- 172.21.15.3:0/532113196 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6ec013680 con 0x7fa6f80d06a0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.563+0000 7fa6f4ff9700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6dc066f30 0x7fa6dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.563+0000 7fa6f4ff9700 1 -- 172.21.15.3:0/532113196 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa6ec084880 con 0x7fa6f80d06a0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.564+0000 7fa6f7fff700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6dc066f30 0x7fa6dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.564+0000 7fa6f7fff700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6dc066f30 0x7fa6dc0693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa6f806e4a0 tx=0x7fa6e40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.568+0000 7fa6f4ff9700 1 -- 172.21.15.3:0/532113196 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6ec051990 con 0x7fa6f80d06a0 2023-10-06T05:01:08.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.808+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa6f80cdf90 con 0x7fa6f80d06a0 2023-10-06T05:01:08.810 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T05:01:08.808+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:01:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.814+0000 7fa6f4ff9700 1 -- 172.21.15.3:0/532113196 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa6ec054fb0 con 0x7fa6f80d06a0 2023-10-06T05:01:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6dc066f30 msgr2=0x7fa6dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6dc066f30 0x7fa6dc0693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa6f806e4a0 tx=0x7fa6e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 msgr2=0x7fa6f8156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f8156700 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00ec80 tx=0x7fa6ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 shutdown_connections 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa6dc066f30 0x7fa6dc0693e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f815ccc0 0x7fa6f80c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6f80d06a0 0x7fa6f8156700 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 --2- 172.21.15.3:0/532113196 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6f815c8f0 0x7fa6f80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 >> 172.21.15.3:0/532113196 conn(0x7fa6f8070310 msgr2=0x7fa6f80b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 shutdown_connections 2023-10-06T05:01:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.816+0000 7fa6fed59700 1 -- 172.21.15.3:0/532113196 wait complete. 2023-10-06T05:01:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:01:08.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:01:08.853 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T05:01:08.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.949+0000 7f90e978a700 1 Processor -- start 2023-10-06T05:01:08.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.949+0000 7f90e978a700 1 -- start start 2023-10-06T05:01:08.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.950+0000 7f90e978a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e40af670 0x7f90e40ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.950+0000 7f90e978a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e40ce420 con 0x7f90e40cc140 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.950+0000 7f90e978a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e415a880 con 0x7f90e40cc510 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.950+0000 7f90e978a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e415ab60 con 0x7f90e40af670 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.950+0000 7f90e37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e40af670 0x7f90e40ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.950+0000 7f90e2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f90e40cc140 0x7f90e4159df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42310/0 (socket says 172.21.15.3:42310) 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.950+0000 7f90e2ffd700 1 -- 172.21.15.3:0/2299534926 learned_addr learned my addr 172.21.15.3:0/2299534926 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.951+0000 7f90e37fe700 1 -- 172.21.15.3:0/2299534926 >> v1:172.21.15.153:6789/0 conn(0x7f90e40cc510 legacy=0x7f90e40aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.951+0000 7f90e37fe700 1 -- 172.21.15.3:0/2299534926 >> v1:172.21.15.3:6789/0 conn(0x7f90e40cc140 legacy=0x7f90e4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.951+0000 7f90e37fe700 1 -- 172.21.15.3:0/2299534926 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e40ce0b0 con 0x7f90e40af670 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.951+0000 7f90e37fe700 1 --2- 172.21.15.3:0/2299534926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40af670 0x7f90e40ace50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009e60 tx=0x7f90dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf942047ad58fe84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.951+0000 7f90e1ffb700 1 -- 172.21.15.3:0/2299534926 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00e070 con 0x7f90e40af670 2023-10-06T05:01:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.951+0000 7f90e1ffb700 1 -- 172.21.15.3:0/2299534926 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc00bc30 con 0x7f90e40af670 2023-10-06T05:01:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.951+0000 7f90e1ffb700 1 -- 172.21.15.3:0/2299534926 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc012430 con 0x7f90e40af670 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.952+0000 7f90e978a700 1 -- 172.21.15.3:0/2299534926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40af670 msgr2=0x7f90e40ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.952+0000 7f90e978a700 1 --2- 172.21.15.3:0/2299534926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40af670 0x7f90e40ace50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009e60 tx=0x7f90dc009780 comp rx=0 tx=0).stop 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.952+0000 7f90e978a700 1 -- 172.21.15.3:0/2299534926 shutdown_connections 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.952+0000 7f90e978a700 1 --2- 172.21.15.3:0/2299534926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40af670 0x7f90e40ace50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.952+0000 7f90e978a700 1 -- 172.21.15.3:0/2299534926 >> 172.21.15.3:0/2299534926 conn(0x7f90e40d03e0 msgr2=0x7f90e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.952+0000 7f90e978a700 1 -- 172.21.15.3:0/2299534926 shutdown_connections 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.952+0000 7f90e978a700 1 -- 172.21.15.3:0/2299534926 wait complete. 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.953+0000 7f90e978a700 1 Processor -- start 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.953+0000 7f90e978a700 1 -- start start 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e40af670 0x7f90e4151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40cc140 0x7f90e40b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e40cc510 0x7f90e40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c4790 con 0x7f90e40cc140 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c92e0 con 0x7f90e40af670 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c9dd0 con 0x7f90e40cc510 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40cc140 0x7f90e40b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40cc140 0x7f90e40b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54972/0 (socket says 172.21.15.3:54972) 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 -- 172.21.15.3:0/269144046 learned_addr learned my addr 172.21.15.3:0/269144046 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e37fe700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e40cc510 0x7f90e40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e2ffd700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e40af670 0x7f90e4151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 -- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e40cc510 msgr2=0x7f90e40b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e40cc510 0x7f90e40b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 -- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e40af670 msgr2=0x7f90e4151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e40af670 0x7f90e4151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 -- 172.21.15.3:0/269144046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc009a20 con 0x7f90e40cc140 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e37fe700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e40cc510 0x7f90e40b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e2ffd700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e40af670 0x7f90e4151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e27fc700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40cc140 0x7f90e40b4a10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00c5c0 tx=0x7f90cc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90daffd700 1 -- 172.21.15.3:0/269144046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc011590 con 0x7f90e40cc140 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90daffd700 1 -- 172.21.15.3:0/269144046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc011730 con 0x7f90e40cc140 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e40c9530 con 0x7f90e40cc140 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90daffd700 1 -- 172.21.15.3:0/269144046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc011af0 con 0x7f90e40cc140 2023-10-06T05:01:08.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.954+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e40c9af0 con 0x7f90e40cc140 2023-10-06T05:01:08.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.955+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e40c48d0 con 0x7f90e40cc140 2023-10-06T05:01:08.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.956+0000 7f90daffd700 1 -- 172.21.15.3:0/269144046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90cc012420 con 0x7f90e40cc140 2023-10-06T05:01:08.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.956+0000 7f90daffd700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90c4066f30 0x7f90c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:08.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.956+0000 7f90daffd700 1 -- 172.21.15.3:0/269144046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f90cc0839f0 con 0x7f90e40cc140 2023-10-06T05:01:08.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.956+0000 7f90e2ffd700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90c4066f30 0x7f90c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:08.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.957+0000 7f90e2ffd700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90c4066f30 0x7f90c40693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000c00 tx=0x7f90d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:08.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:08.960+0000 7f90daffd700 1 -- 172.21.15.3:0/269144046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90cc04b5a0 con 0x7f90e40cc140 2023-10-06T05:01:09.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.198+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f90e404c380 con 0x7f90e40cc140 2023-10-06T05:01:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.821+0000 7f90daffd700 1 -- 172.21.15.3:0/269144046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 104+0+0 (secure 0 0 0) 0x7f90cc017020 con 0x7f90e40cc140 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90c4066f30 msgr2=0x7f90c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90c4066f30 0x7f90c40693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000c00 tx=0x7f90d4009040 comp rx=0 tx=0).stop 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40cc140 msgr2=0x7f90e40b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40cc140 0x7f90e40b4a10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00c5c0 tx=0x7f90cc007f30 comp rx=0 tx=0).stop 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 shutdown_connections 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f90c4066f30 0x7f90c40693e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90e40af670 0x7f90e4151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f90e40cc510 0x7f90e40b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 --2- 172.21.15.3:0/269144046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e40cc140 0x7f90e40b4a10 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 >> 172.21.15.3:0/269144046 conn(0x7f90e40d03e0 msgr2=0x7f90e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:09.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 shutdown_connections 2023-10-06T05:01:09.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.824+0000 7f90e978a700 1 -- 172.21.15.3:0/269144046 wait complete. 2023-10-06T05:01:09.841 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:01:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.967+0000 7fee34115700 1 Processor -- start 2023-10-06T05:01:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.967+0000 7fee34115700 1 -- start start 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee34115700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee2c15dca0 0x7fee2c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee34115700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee2c0d1170 0x7fee2c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee34115700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2c0d15f0 con 0x7fee2c1598f0 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee34115700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c15e400 con 0x7fee2c0d1170 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee34115700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0ccda0 con 0x7fee2c15dca0 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee326b2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee2c0d1170 0x7fee2c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee31eb1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee2c1598f0 0x7fee2c15d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42318/0 (socket says 172.21.15.3:42318) 2023-10-06T05:01:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee31eb1700 1 -- 172.21.15.3:0/52901623 learned_addr learned my addr 172.21.15.3:0/52901623 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee326b2700 1 -- 172.21.15.3:0/52901623 >> v2:172.21.15.153:3300/0 conn(0x7fee2c15dca0 msgr2=0x7fee2c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee316b0700 1 --2- 172.21.15.3:0/52901623 >> v2:172.21.15.153:3300/0 conn(0x7fee2c15dca0 0x7fee2c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee30eaf700 1 -- 172.21.15.3:0/52901623 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168565298 0 0) 0x7fee28001110 con 0x7fee2c1598f0 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee326b2700 1 --2- 172.21.15.3:0/52901623 >> v2:172.21.15.153:3300/0 conn(0x7fee2c15dca0 0x7fee2c0cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee326b2700 1 -- 172.21.15.3:0/52901623 >> v1:172.21.15.3:6789/0 conn(0x7fee2c1598f0 legacy=0x7fee2c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee326b2700 1 -- 172.21.15.3:0/52901623 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c0cf7d0 con 0x7fee2c0d1170 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee326b2700 1 --2- 172.21.15.3:0/52901623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c0d1170 0x7fee2c159cc0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fee2c06db00 tx=0x7fee24009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5017ae6ec51c91d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.968+0000 7fee30eaf700 1 -- 172.21.15.3:0/52901623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400e070 con 0x7fee2c0d1170 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee30eaf700 1 -- 172.21.15.3:0/52901623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2400bcc0 con 0x7fee2c0d1170 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee30eaf700 1 -- 172.21.15.3:0/52901623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24012680 con 0x7fee2c0d1170 2023-10-06T05:01:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 -- 172.21.15.3:0/52901623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c0d1170 msgr2=0x7fee2c159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 --2- 172.21.15.3:0/52901623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c0d1170 0x7fee2c159cc0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fee2c06db00 tx=0x7fee24009780 comp rx=0 tx=0).stop 2023-10-06T05:01:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 -- 172.21.15.3:0/52901623 shutdown_connections 2023-10-06T05:01:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 --2- 172.21.15.3:0/52901623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c0d1170 0x7fee2c159cc0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 --2- 172.21.15.3:0/52901623 >> v2:172.21.15.153:3300/0 conn(0x7fee2c15dca0 0x7fee2c0cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 -- 172.21.15.3:0/52901623 >> 172.21.15.3:0/52901623 conn(0x7fee2c070af0 msgr2=0x7fee2c0d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 -- 172.21.15.3:0/52901623 shutdown_connections 2023-10-06T05:01:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.969+0000 7fee34115700 1 -- 172.21.15.3:0/52901623 wait complete. 2023-10-06T05:01:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.970+0000 7fee34115700 1 Processor -- start 2023-10-06T05:01:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.970+0000 7fee34115700 1 -- start start 2023-10-06T05:01:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee34115700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee2c0d1170 0x7fee2c14d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee34115700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c1598f0 0x7fee2c1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee34115700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c15dca0 0x7fee2c157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee34115700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c14de40 con 0x7fee2c15dca0 2023-10-06T05:01:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee34115700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c14eca0 con 0x7fee2c1598f0 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee34115700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c14e090 con 0x7fee2c0d1170 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c15dca0 0x7fee2c157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c15dca0 0x7fee2c157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54990/0 (socket says 172.21.15.3:54990) 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 -- 172.21.15.3:0/2916117978 learned_addr learned my addr 172.21.15.3:0/2916117978 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee31eb1700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee2c0d1170 0x7fee2c14d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 -- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee2c0d1170 msgr2=0x7fee2c14d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee2c0d1170 0x7fee2c14d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee316b0700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c1598f0 0x7fee2c1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 -- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c1598f0 msgr2=0x7fee2c1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c1598f0 0x7fee2c1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee326b2700 1 -- 172.21.15.3:0/2916117978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24009a20 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee31eb1700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee2c0d1170 0x7fee2c14d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.971+0000 7fee316b0700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c1598f0 0x7fee2c1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.972+0000 7fee326b2700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c15dca0 0x7fee2c157920 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fee2c088810 tx=0x7fee24019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.972+0000 7fee21ffb700 1 -- 172.21.15.3:0/2916117978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2401d630 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.972+0000 7fee21ffb700 1 -- 172.21.15.3:0/2916117978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2401d790 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.972+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c14e370 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.972+0000 7fee21ffb700 1 -- 172.21.15.3:0/2916117978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2401db50 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.972+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c14e930 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.973+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c14ee50 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.973+0000 7fee21ffb700 1 -- 172.21.15.3:0/2916117978 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee24024070 con 0x7fee2c15dca0 2023-10-06T05:01:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.974+0000 7fee21ffb700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee10066f30 0x7fee100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.974+0000 7fee21ffb700 1 -- 172.21.15.3:0/2916117978 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fee24087140 con 0x7fee2c15dca0 2023-10-06T05:01:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.974+0000 7fee31eb1700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee10066f30 0x7fee100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.975+0000 7fee31eb1700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee10066f30 0x7fee100693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee280018c0 tx=0x7fee28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:09.978+0000 7fee21ffb700 1 -- 172.21.15.3:0/2916117978 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee2c14ee50 con 0x7fee2c15dca0 2023-10-06T05:01:10.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.172+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fee2c14ee10 con 0x7fee2c15dca0 2023-10-06T05:01:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.826+0000 7fee21ffb700 1 -- 172.21.15.3:0/2916117978 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) v1 ==== 181+0+0 (secure 0 0 0) 0x7fee24054200 con 0x7fee2c15dca0 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee10066f30 msgr2=0x7fee100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee10066f30 0x7fee100693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee280018c0 tx=0x7fee28009040 comp rx=0 tx=0).stop 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c15dca0 msgr2=0x7fee2c157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c15dca0 0x7fee2c157920 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fee2c088810 tx=0x7fee24019000 comp rx=0 tx=0).stop 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 shutdown_connections 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fee10066f30 0x7fee100693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee2c15dca0 0x7fee2c157920 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee2c0d1170 0x7fee2c14d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 --2- 172.21.15.3:0/2916117978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c1598f0 0x7fee2c1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 >> 172.21.15.3:0/2916117978 conn(0x7fee2c070af0 msgr2=0x7fee2c0c5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 shutdown_connections 2023-10-06T05:01:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.831+0000 7fee34115700 1 -- 172.21.15.3:0/2916117978 wait complete. 2023-10-06T05:01:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:01:10.848 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:01:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.977+0000 7f56655c4700 1 Processor -- start 2023-10-06T05:01:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.977+0000 7f56655c4700 1 -- start start 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f56655c4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 0x7f56600d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f56655c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56600ccda0 con 0x7f56601598f0 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f56655c4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56600cf7d0 con 0x7f566015dca0 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f56655c4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566015e400 con 0x7f56600cebd0 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f565effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f56601598f0 0x7f566015d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42330/0 (socket says 172.21.15.3:42330) 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f565effd700 1 -- 172.21.15.3:0/3047073491 learned_addr learned my addr 172.21.15.3:0/3047073491 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f565f7fe700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 0x7f56600d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017071947 0 0) 0x7f56600ccda0 con 0x7f56601598f0 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.978+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f564c003570 con 0x7f56601598f0 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414675436 0 0) 0x7f564c003570 con 0x7f56601598f0 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 msgr2=0x7f56600d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 0x7f56600d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 >> v1:172.21.15.153:6789/0 conn(0x7f566015dca0 legacy=0x7f5660159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600cf210 con 0x7f56601598f0 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f5658001470 con 0x7f56601598f0 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 >> v1:172.21.15.3:6789/0 conn(0x7f56601598f0 legacy=0x7f566015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 0x7f564c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c004010 0x7f564c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c004d50 0x7f564c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566015e400 con 0x7f564c004d50 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c00bbf0 con 0x7f564c004010 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c00bdb0 con 0x7f56600cebd0 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565f7fe700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c004d50 0x7f564c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565effd700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 0x7f564c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565e7fc700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c004010 0x7f564c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565f7fe700 1 -- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 msgr2=0x7f564c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565f7fe700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600cebd0 0x7f564c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565f7fe700 1 -- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c004010 msgr2=0x7f564c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565f7fe700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c004010 0x7f564c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565f7fe700 1 -- 172.21.15.3:0/3047073491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c003570 con 0x7f564c004d50 2023-10-06T05:01:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565f7fe700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c004d50 0x7f564c0051c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f56500094d0 tx=0x7f565000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8b30cfd838fb52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565000ea90 con 0x7f564c004d50 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565000f040 con 0x7f564c004d50 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.979+0000 7f565dffb700 1 -- 172.21.15.3:0/3047073491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565000ee20 con 0x7f564c004d50 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.980+0000 7f56655c4700 1 -- 172.21.15.3:0/3047073491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c004d50 msgr2=0x7f564c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.980+0000 7f56655c4700 1 --2- 172.21.15.3:0/3047073491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c004d50 0x7f564c0051c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f56500094d0 tx=0x7f565000b040 comp rx=0 tx=0).stop 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.980+0000 7f565f7fe700 1 -- 172.21.15.3:0/3047073491 reap_dead start 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.980+0000 7f56655c4700 1 -- 172.21.15.3:0/3047073491 shutdown_connections 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.980+0000 7f56655c4700 1 -- 172.21.15.3:0/3047073491 >> 172.21.15.3:0/3047073491 conn(0x7f5660070af0 msgr2=0x7f56600d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.980+0000 7f56655c4700 1 -- 172.21.15.3:0/3047073491 shutdown_connections 2023-10-06T05:01:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.980+0000 7f56655c4700 1 -- 172.21.15.3:0/3047073491 wait complete. 2023-10-06T05:01:10.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.981+0000 7f56655c4700 1 Processor -- start 2023-10-06T05:01:10.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.981+0000 7f56655c4700 1 -- start start 2023-10-06T05:01:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f56655c4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5660147270 0x7f5660154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f56655c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56600d0650 0x7f56600d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f56655c4700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600ccc60 0x7f56600cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f56655c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56600d1000 con 0x7f56600d0650 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f56655c4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660155580 con 0x7f5660147270 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f56655c4700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5660155a10 con 0x7f56600ccc60 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f565e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56600d0650 0x7f56600d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f565e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56600d0650 0x7f56600d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55014/0 (socket says 172.21.15.3:55014) 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f565e7fc700 1 -- 172.21.15.3:0/3403589460 learned_addr learned my addr 172.21.15.3:0/3403589460 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f565effd700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5660147270 0x7f5660154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.982+0000 7f565f7fe700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600ccc60 0x7f56600cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f565e7fc700 1 -- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600ccc60 msgr2=0x7f56600cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f565e7fc700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600ccc60 0x7f56600cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f565e7fc700 1 -- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5660147270 msgr2=0x7f5660154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f565e7fc700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5660147270 0x7f5660154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f565e7fc700 1 -- 172.21.15.3:0/3403589460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5660154970 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f565e7fc700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56600d0650 0x7f56600d0ac0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5648002950 tx=0x7f5648007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f5656ffd700 1 -- 172.21.15.3:0/3403589460 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648013600 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f5656ffd700 1 -- 172.21.15.3:0/3403589460 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5648013760 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5660154c50 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f5656ffd700 1 -- 172.21.15.3:0/3403589460 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648013b20 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.983+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5660155210 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.984+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56601556f0 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.985+0000 7f5656ffd700 1 -- 172.21.15.3:0/3403589460 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f564801c400 con 0x7f56600d0650 2023-10-06T05:01:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.985+0000 7f5656ffd700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5640066f30 0x7f56400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.986+0000 7f5656ffd700 1 -- 172.21.15.3:0/3403589460 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f5648084850 con 0x7f56600d0650 2023-10-06T05:01:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.986+0000 7f565effd700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5640066f30 0x7f56400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.986+0000 7f565effd700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5640066f30 0x7f56400693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5658001050 tx=0x7f5658001e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:10.991+0000 7f5656ffd700 1 -- 172.21.15.3:0/3403589460 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5648051b70 con 0x7f56600d0650 2023-10-06T05:01:11.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.185+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5660071680 con 0x7f56600d0650 2023-10-06T05:01:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.830+0000 7f5656ffd700 1 -- 172.21.15.3:0/3403589460 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5648055190 con 0x7f56600d0650 2023-10-06T05:01:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5640066f30 msgr2=0x7f56400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5640066f30 0x7f56400693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5658001050 tx=0x7f5658001e50 comp rx=0 tx=0).stop 2023-10-06T05:01:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56600d0650 msgr2=0x7f56600d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56600d0650 0x7f56600d0ac0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5648002950 tx=0x7f5648007d30 comp rx=0 tx=0).stop 2023-10-06T05:01:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 shutdown_connections 2023-10-06T05:01:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5640066f30 0x7f56400693e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5660147270 0x7f5660154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56600ccc60 0x7f56600cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 --2- 172.21.15.3:0/3403589460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56600d0650 0x7f56600d0ac0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 >> 172.21.15.3:0/3403589460 conn(0x7f5660070af0 msgr2=0x7f56600c53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 shutdown_connections 2023-10-06T05:01:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.835+0000 7f56655c4700 1 -- 172.21.15.3:0/3403589460 wait complete. 2023-10-06T05:01:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T05:01:11.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-10-06T05:01:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.976+0000 7f5a12a97700 1 Processor -- start 2023-10-06T05:01:11.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.977+0000 7f5a12a97700 1 -- start start 2023-10-06T05:01:11.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.977+0000 7f5a12a97700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5a0c0c5c20 0x7f5a0c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.977+0000 7f5a12a97700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c0cb190 con 0x7f5a0c0af660 2023-10-06T05:01:11.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.977+0000 7f5a12a97700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c0c65a0 con 0x7f5a0c0af290 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.977+0000 7f5a12a97700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c159950 con 0x7f5a0c0c5c20 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.977+0000 7f5a11294700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5a0c0af660 0x7f5a0c0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42338/0 (socket says 172.21.15.3:42338) 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a11294700 1 -- 172.21.15.3:0/1094876804 learned_addr learned my addr 172.21.15.3:0/1094876804 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313460500 0 0) 0x7f5a0c0cb190 con 0x7f5a0c0af660 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a12296700 1 --2- 172.21.15.3:0/1094876804 >> v2:172.21.15.153:3300/0 conn(0x7f5a0c0c5c20 0x7f5a0c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f4003570 con 0x7f5a0c0af660 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 36563287 0 0) 0x7f59f4003570 con 0x7f5a0c0af660 2023-10-06T05:01:11.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 >> v2:172.21.15.153:3300/0 conn(0x7f5a0c0c5c20 msgr2=0x7f5a0c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 --2- 172.21.15.3:0/1094876804 >> v2:172.21.15.153:3300/0 conn(0x7f5a0c0c5c20 0x7f5a0c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 >> v1:172.21.15.153:6789/0 conn(0x7f5a0c0af290 legacy=0x7f5a0c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0cd6a0 con 0x7f5a0c0af660 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f5a000015f0 con 0x7f5a0c0af660 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 >> v1:172.21.15.3:6789/0 conn(0x7f5a0c0af660 legacy=0x7f5a0c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59f4003a60 0x7f59f4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59f4004810 0x7f59f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f40051c0 0x7f59f400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c159950 con 0x7f59f4004810 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f400c0d0 con 0x7f59f40051c0 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f400c290 con 0x7f59f4003a60 2023-10-06T05:01:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a11294700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59f4004810 0x7f59f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a11a95700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59f4003a60 0x7f59f4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a12296700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f40051c0 0x7f59f400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.978+0000 7f5a11294700 1 -- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59f4003a60 msgr2=0x7f59f4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a11294700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f59f4003a60 0x7f59f4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a11294700 1 -- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f40051c0 msgr2=0x7f59f400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a11294700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f40051c0 0x7f59f400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a11294700 1 -- 172.21.15.3:0/1094876804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4003570 con 0x7f59f4004810 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12296700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f40051c0 0x7f59f400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12296700 1 -- 172.21.15.3:0/1094876804 reap_dead start 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a11294700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59f4004810 0x7f59f4004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c2d0 tx=0x7f5a00007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2a331e285cc567 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:11.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000cba0 con 0x7f59f4004810 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0000cd20 con 0x7f59f4004810 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a10a93700 1 -- 172.21.15.3:0/1094876804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00007320 con 0x7f59f4004810 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12a97700 1 -- 172.21.15.3:0/1094876804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59f4004810 msgr2=0x7f59f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59f4004810 0x7f59f4004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c2d0 tx=0x7f5a00007920 comp rx=0 tx=0).stop 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12a97700 1 -- 172.21.15.3:0/1094876804 shutdown_connections 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1094876804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59f4004810 0x7f59f4004c80 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12a97700 1 -- 172.21.15.3:0/1094876804 >> 172.21.15.3:0/1094876804 conn(0x7f5a0c089dd0 msgr2=0x7f5a0c0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.979+0000 7f5a12a97700 1 -- 172.21.15.3:0/1094876804 shutdown_connections 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.980+0000 7f5a12a97700 1 -- 172.21.15.3:0/1094876804 wait complete. 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 Processor -- start 2023-10-06T05:01:11.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 -- start start 2023-10-06T05:01:11.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4004810 0x7f5a0c0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a0c0ce1b0 0x7f5a0c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c0b0ae0 0x7f5a0c0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0b3d50 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0b2490 con 0x7f59f4004810 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12a97700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0b3fa0 con 0x7f5a0c0ce1b0 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.981+0000 7f5a12296700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c0b0ae0 0x7f5a0c0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c0b0ae0 0x7f5a0c0b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55032/0 (socket says 172.21.15.3:55032) 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 -- 172.21.15.3:0/1442709014 learned_addr learned my addr 172.21.15.3:0/1442709014 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a11a95700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4004810 0x7f5a0c0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 -- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a0c0ce1b0 msgr2=0x7f5a0c0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a11294700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a0c0ce1b0 0x7f5a0c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a0c0ce1b0 0x7f5a0c0b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 -- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4004810 msgr2=0x7f5a0c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4004810 0x7f5a0c0ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 -- 172.21.15.3:0/1442709014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0b4280 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a11a95700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4004810 0x7f5a0c0ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12296700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c0b0ae0 0x7f5a0c0b0f50 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c08abf0 tx=0x7f59fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a09ffb700 1 -- 172.21.15.3:0/1442709014 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc015670 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a09ffb700 1 -- 172.21.15.3:0/1442709014 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc0157f0 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c0b4560 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a09ffb700 1 -- 172.21.15.3:0/1442709014 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc015bb0 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.982+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c0b52a0 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.983+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c04c380 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.984+0000 7f5a09ffb700 1 -- 172.21.15.3:0/1442709014 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59fc01f410 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.984+0000 7f5a09ffb700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59f8066f30 0x7f59f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.984+0000 7f5a09ffb700 1 -- 172.21.15.3:0/1442709014 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f59fc084230 con 0x7f5a0c0b0ae0 2023-10-06T05:01:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.984+0000 7f5a11a95700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59f8066f30 0x7f59f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.985+0000 7f5a11a95700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59f8066f30 0x7f59f80693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5a040042e0 tx=0x7f5a04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:11.988+0000 7f5a09ffb700 1 -- 172.21.15.3:0/1442709014 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59fc051640 con 0x7f5a0c0b0ae0 2023-10-06T05:01:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.183+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f5a0c06ed10 con 0x7f5a0c0b0ae0 2023-10-06T05:01:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.187+0000 7f5a09ffb700 1 -- 172.21.15.3:0/1442709014 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v59) v1 ==== 105+0+0 (secure 0 0 0) 0x7f59fc054c60 con 0x7f5a0c0b0ae0 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59f8066f30 msgr2=0x7f59f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59f8066f30 0x7f59f80693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5a040042e0 tx=0x7f5a04009040 comp rx=0 tx=0).stop 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c0b0ae0 msgr2=0x7f5a0c0b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c0b0ae0 0x7f5a0c0b0f50 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c08abf0 tx=0x7f59fc008040 comp rx=0 tx=0).stop 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 shutdown_connections 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f59f8066f30 0x7f59f80693e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4004810 0x7f5a0c0ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c0b0ae0 0x7f5a0c0b0f50 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 --2- 172.21.15.3:0/1442709014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a0c0ce1b0 0x7f5a0c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 >> 172.21.15.3:0/1442709014 conn(0x7f5a0c089dd0 msgr2=0x7f5a0c0c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 shutdown_connections 2023-10-06T05:01:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.189+0000 7f5a12a97700 1 -- 172.21.15.3:0/1442709014 wait complete. 2023-10-06T05:01:12.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:01:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.322+0000 7f6d51c9c700 1 Processor -- start 2023-10-06T05:01:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.322+0000 7f6d51c9c700 1 -- start start 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d51c9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c0af290 0x7f6d4c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d51c9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6d4c0c5c20 0x7f6d4c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d51c9c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c0cb190 con 0x7f6d4c0af660 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d51c9c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c0c65a0 con 0x7f6d4c0c5c20 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d51c9c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c159950 con 0x7f6d4c0af290 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d4bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d4c0af660 0x7f6d4c0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45546/0 (socket says 172.21.15.3:45546) 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d4bfff700 1 -- 172.21.15.3:0/454711119 learned_addr learned my addr 172.21.15.3:0/454711119 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834914571 0 0) 0x7f6d4c0cb190 con 0x7f6d4c0af660 2023-10-06T05:01:12.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d50c9a700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c0af290 0x7f6d4c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d34003570 con 0x7f6d4c0af660 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.323+0000 7f6d5149b700 1 --2- 172.21.15.3:0/454711119 >> v2:172.21.15.153:3300/0 conn(0x7f6d4c0c5c20 0x7f6d4c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 908313538 0 0) 0x7f6d34003570 con 0x7f6d4c0af660 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c0af290 msgr2=0x7f6d4c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c0af290 0x7f6d4c0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 >> v2:172.21.15.153:3300/0 conn(0x7f6d4c0c5c20 msgr2=0x7f6d4c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 --2- 172.21.15.3:0/454711119 >> v2:172.21.15.153:3300/0 conn(0x7f6d4c0c5c20 0x7f6d4c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0cd6a0 con 0x7f6d4c0af660 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6d40001610 con 0x7f6d4c0af660 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 >> v1:172.21.15.3:6789/0 conn(0x7f6d4c0af660 legacy=0x7f6d4c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d34003a60 0x7f6d34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c0af290 0x7f6d34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d34004d50 0x7f6d340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0c65a0 con 0x7f6d34004d50 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c159950 con 0x7f6d34003a60 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3400bae0 con 0x7f6d4c0af290 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d5149b700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d34004d50 0x7f6d340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d50c9a700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d34003a60 0x7f6d34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d5149b700 1 -- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c0af290 msgr2=0x7f6d34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d5149b700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c0af290 0x7f6d34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d5149b700 1 -- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d34003a60 msgr2=0x7f6d34003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:12.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d5149b700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d34003a60 0x7f6d34003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d5149b700 1 -- 172.21.15.3:0/454711119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34003570 con 0x7f6d34004d50 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d50c9a700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d34003a60 0x7f6d34003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d5149b700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d34004d50 0x7f6d340051c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007440 tx=0x7f6d44009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed17946299c3114e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44009a40 con 0x7f6d34004d50 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d44007cf0 con 0x7f6d34004d50 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.324+0000 7f6d4b7fe700 1 -- 172.21.15.3:0/454711119 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44014630 con 0x7f6d34004d50 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.325+0000 7f6d51c9c700 1 -- 172.21.15.3:0/454711119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d34004d50 msgr2=0x7f6d340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.325+0000 7f6d51c9c700 1 --2- 172.21.15.3:0/454711119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d34004d50 0x7f6d340051c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007440 tx=0x7f6d44009710 comp rx=0 tx=0).stop 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.325+0000 7f6d5149b700 1 -- 172.21.15.3:0/454711119 reap_dead start 2023-10-06T05:01:12.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.325+0000 7f6d51c9c700 1 -- 172.21.15.3:0/454711119 shutdown_connections 2023-10-06T05:01:12.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.325+0000 7f6d51c9c700 1 -- 172.21.15.3:0/454711119 >> 172.21.15.3:0/454711119 conn(0x7f6d4c089dd0 msgr2=0x7f6d4c0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:12.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.325+0000 7f6d51c9c700 1 -- 172.21.15.3:0/454711119 shutdown_connections 2023-10-06T05:01:12.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.325+0000 7f6d51c9c700 1 -- 172.21.15.3:0/454711119 wait complete. 2023-10-06T05:01:12.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.326+0000 7f6d51c9c700 1 Processor -- start 2023-10-06T05:01:12.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.326+0000 7f6d51c9c700 1 -- start start 2023-10-06T05:01:12.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d51c9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c14dda0 0x7f6d4c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d51c9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d4c0b0790 0x7f6d4c0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d51c9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0c7e60 0x7f6d4c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d51c9c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0cf480 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d51c9c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0b0c00 con 0x7f6d4c0b0790 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d51c9c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0ce870 con 0x7f6d4c14dda0 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0c7e60 0x7f6d4c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0c7e60 0x7f6d4c0b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59794/0 (socket says 172.21.15.3:59794) 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 -- 172.21.15.3:0/2290180611 learned_addr learned my addr 172.21.15.3:0/2290180611 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d50c9a700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c14dda0 0x7f6d4c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d4bfff700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d4c0b0790 0x7f6d4c0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 -- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c14dda0 msgr2=0x7f6d4c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c14dda0 0x7f6d4c0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 -- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d4c0b0790 msgr2=0x7f6d4c0c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d4c0b0790 0x7f6d4c0c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 -- 172.21.15.3:0/2290180611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0ceb50 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d50c9a700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c14dda0 0x7f6d4c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d4bfff700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d4c0b0790 0x7f6d4c0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.327+0000 7f6d5149b700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0c7e60 0x7f6d4c0b4fb0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6d44002230 tx=0x7f6d44018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.328+0000 7f6d48ff9700 1 -- 172.21.15.3:0/2290180611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44007e60 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.328+0000 7f6d48ff9700 1 -- 172.21.15.3:0/2290180611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4401f630 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.328+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c0cee30 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.328+0000 7f6d48ff9700 1 -- 172.21.15.3:0/2290180611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4401f9f0 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.328+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c14e170 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.329+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c0cf2d0 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.329+0000 7f6d48ff9700 1 -- 172.21.15.3:0/2290180611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d44003020 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.329+0000 7f6d48ff9700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6d38066f30 0x7f6d380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.329+0000 7f6d48ff9700 1 -- 172.21.15.3:0/2290180611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f6d44086480 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.330+0000 7f6d50c9a700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6d38066f30 0x7f6d380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.331+0000 7f6d50c9a700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6d38066f30 0x7f6d380693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c006aa0 tx=0x7f6d3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:12.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.334+0000 7f6d48ff9700 1 -- 172.21.15.3:0/2290180611 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d44053890 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.527+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6d4c04c380 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.531+0000 7f6d48ff9700 1 -- 172.21.15.3:0/2290180611 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d44056eb0 con 0x7f6d4c0c7e60 2023-10-06T05:01:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6d38066f30 msgr2=0x7f6d380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6d38066f30 0x7f6d380693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c006aa0 tx=0x7f6d3c008040 comp rx=0 tx=0).stop 2023-10-06T05:01:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0c7e60 msgr2=0x7f6d4c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0c7e60 0x7f6d4c0b4fb0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6d44002230 tx=0x7f6d44018000 comp rx=0 tx=0).stop 2023-10-06T05:01:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 shutdown_connections 2023-10-06T05:01:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6d38066f30 0x7f6d380693e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0c7e60 0x7f6d4c0b4fb0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d4c14dda0 0x7f6d4c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 --2- 172.21.15.3:0/2290180611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d4c0b0790 0x7f6d4c0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 >> 172.21.15.3:0/2290180611 conn(0x7f6d4c089dd0 msgr2=0x7f6d4c0acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 shutdown_connections 2023-10-06T05:01:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.533+0000 7f6d51c9c700 1 -- 172.21.15.3:0/2290180611 wait complete. 2023-10-06T05:01:12.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-10-06T05:01:12.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.674+0000 7fe0bc9c5700 1 Processor -- start 2023-10-06T05:01:12.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.674+0000 7fe0bc9c5700 1 -- start start 2023-10-06T05:01:12.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0bc9c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0b8142d70 0x7fe0b80cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0bc9c5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0bc9c5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b8069a80 con 0x7fe0b8143140 2023-10-06T05:01:12.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0bc9c5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b80cc880 con 0x7fe0b8142d70 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0bc9c5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b8069f10 con 0x7fe0b80681c0 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0b659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0b8142d70 0x7fe0b80cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0b659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0b8142d70 0x7fe0b80cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59810/0 (socket says 172.21.15.3:59810) 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0b659c700 1 -- 172.21.15.3:0/3398915762 learned_addr learned my addr 172.21.15.3:0/3398915762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0b659c700 1 -- 172.21.15.3:0/3398915762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 msgr2=0x7fe0b806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0b6d9d700 1 --2- 172.21.15.3:0/3398915762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0b659c700 1 --2- 172.21.15.3:0/3398915762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b806d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.675+0000 7fe0b659c700 1 -- 172.21.15.3:0/3398915762 >> v1:172.21.15.153:6789/0 conn(0x7fe0b8143140 legacy=0x7fe0b8067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0b659c700 1 -- 172.21.15.3:0/3398915762 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8069150 con 0x7fe0b8142d70 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0b659c700 1 --2- 172.21.15.3:0/3398915762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8142d70 0x7fe0b80cbde0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009f50 tx=0x7fe0b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac7a46966daf3e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0b559a700 1 -- 172.21.15.3:0/3398915762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000e070 con 0x7fe0b8142d70 2023-10-06T05:01:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0b559a700 1 -- 172.21.15.3:0/3398915762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b000ba30 con 0x7fe0b8142d70 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0b559a700 1 -- 172.21.15.3:0/3398915762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00092e0 con 0x7fe0b8142d70 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/3398915762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8142d70 msgr2=0x7fe0b80cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/3398915762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8142d70 0x7fe0b80cbde0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009f50 tx=0x7fe0b0009840 comp rx=0 tx=0).stop 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/3398915762 shutdown_connections 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/3398915762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/3398915762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8142d70 0x7fe0b80cbde0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.676+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/3398915762 >> 172.21.15.3:0/3398915762 conn(0x7fe0b80ce210 msgr2=0x7fe0b80ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.677+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/3398915762 shutdown_connections 2023-10-06T05:01:12.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.677+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/3398915762 wait complete. 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 Processor -- start 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 -- start start 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b8142d70 0x7fe0b80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8143140 0x7fe0b8159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80b4090 con 0x7fe0b8143140 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b815d4e0 con 0x7fe0b8142d70 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.678+0000 7fe0bc9c5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b815d7c0 con 0x7fe0b80681c0 2023-10-06T05:01:12.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8143140 0x7fe0b8159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8143140 0x7fe0b8159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59814/0 (socket says 172.21.15.3:59814) 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 -- 172.21.15.3:0/842936746 learned_addr learned my addr 172.21.15.3:0/842936746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b659c700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b5d9b700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b8142d70 0x7fe0b80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 -- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 msgr2=0x7fe0b815ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b815ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 -- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b8142d70 msgr2=0x7fe0b80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b8142d70 0x7fe0b80b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 -- 172.21.15.3:0/842936746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b0009ae0 con 0x7fe0b8143140 2023-10-06T05:01:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b5d9b700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b8142d70 0x7fe0b80b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b659c700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0b6d9d700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8143140 0x7fe0b8159f70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c880 tx=0x7fe0a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0ae7fc700 1 -- 172.21.15.3:0/842936746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8011600 con 0x7fe0b8143140 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0ae7fc700 1 -- 172.21.15.3:0/842936746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a8011760 con 0x7fe0b8143140 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b815daa0 con 0x7fe0b8143140 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0ae7fc700 1 -- 172.21.15.3:0/842936746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8011b20 con 0x7fe0b8143140 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.679+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b815e060 con 0x7fe0b8143140 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.680+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b804c380 con 0x7fe0b8143140 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.681+0000 7fe0ae7fc700 1 -- 172.21.15.3:0/842936746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0a8018360 con 0x7fe0b8143140 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.681+0000 7fe0ae7fc700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe098066f30 0x7fe0980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:12.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.681+0000 7fe0ae7fc700 1 -- 172.21.15.3:0/842936746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe0a8084070 con 0x7fe0b8143140 2023-10-06T05:01:12.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.681+0000 7fe0b659c700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe098066f30 0x7fe0980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:12.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.682+0000 7fe0b659c700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe098066f30 0x7fe0980693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009880 tx=0x7fe0b0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:12.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.685+0000 7fe0ae7fc700 1 -- 172.21.15.3:0/842936746 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0a8051480 con 0x7fe0b8143140 2023-10-06T05:01:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.880+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7fe0b8069430 con 0x7fe0b8143140 2023-10-06T05:01:12.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.883+0000 7fe0ae7fc700 1 -- 172.21.15.3:0/842936746 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v61) v1 ==== 107+0+0 (secure 0 0 0) 0x7fe0a8054aa0 con 0x7fe0b8143140 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe098066f30 msgr2=0x7fe0980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe098066f30 0x7fe0980693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009880 tx=0x7fe0b0017000 comp rx=0 tx=0).stop 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8143140 msgr2=0x7fe0b8159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8143140 0x7fe0b8159f70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c880 tx=0x7fe0a8008040 comp rx=0 tx=0).stop 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 shutdown_connections 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe098066f30 0x7fe0980693e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0b8143140 0x7fe0b8159f70 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0b80681c0 0x7fe0b815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 --2- 172.21.15.3:0/842936746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b8142d70 0x7fe0b80b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 >> 172.21.15.3:0/842936746 conn(0x7fe0b80ce210 msgr2=0x7fe0b8141ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 shutdown_connections 2023-10-06T05:01:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:12.885+0000 7fe0bc9c5700 1 -- 172.21.15.3:0/842936746 wait complete. 2023-10-06T05:01:12.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-10-06T05:01:13.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.017+0000 7fa9202e9700 1 Processor -- start 2023-10-06T05:01:13.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.017+0000 7fa9202e9700 1 -- start start 2023-10-06T05:01:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.018+0000 7fa9202e9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa918143140 0x7fa918067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.018+0000 7fa9202e9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa918067ff0 0x7fa91806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.018+0000 7fa9202e9700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa918069a80 con 0x7fa918142d70 2023-10-06T05:01:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.018+0000 7fa9202e9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9180cc880 con 0x7fa918143140 2023-10-06T05:01:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.018+0000 7fa9202e9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa918069f10 con 0x7fa918067ff0 2023-10-06T05:01:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.018+0000 7fa91d884700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa918143140 0x7fa918067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.018+0000 7fa91d884700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa918143140 0x7fa918067ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59816/0 (socket says 172.21.15.3:59816) 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d884700 1 -- 172.21.15.3:0/2818477534 learned_addr learned my addr 172.21.15.3:0/2818477534 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91e886700 1 --2- 172.21.15.3:0/2818477534 >> v2:172.21.15.153:3300/0 conn(0x7fa918067ff0 0x7fa91806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d884700 1 -- 172.21.15.3:0/2818477534 >> v2:172.21.15.153:3300/0 conn(0x7fa918067ff0 msgr2=0x7fa91806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d884700 1 --2- 172.21.15.3:0/2818477534 >> v2:172.21.15.153:3300/0 conn(0x7fa918067ff0 0x7fa91806d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d884700 1 -- 172.21.15.3:0/2818477534 >> v1:172.21.15.153:6789/0 conn(0x7fa918142d70 legacy=0x7fa9180cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d884700 1 -- 172.21.15.3:0/2818477534 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918068410 con 0x7fa918143140 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d884700 1 --2- 172.21.15.3:0/2818477534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918143140 0x7fa918067ab0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa9180b41f0 tx=0x7fa90c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248f0a0143781016 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d083700 1 -- 172.21.15.3:0/2818477534 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00b820 con 0x7fa918143140 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d083700 1 -- 172.21.15.3:0/2818477534 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00b980 con 0x7fa918143140 2023-10-06T05:01:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.019+0000 7fa91d083700 1 -- 172.21.15.3:0/2818477534 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00bd20 con 0x7fa918143140 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 -- 172.21.15.3:0/2818477534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918143140 msgr2=0x7fa918067ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 --2- 172.21.15.3:0/2818477534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918143140 0x7fa918067ab0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa9180b41f0 tx=0x7fa90c009250 comp rx=0 tx=0).stop 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 -- 172.21.15.3:0/2818477534 shutdown_connections 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 --2- 172.21.15.3:0/2818477534 >> v2:172.21.15.153:3300/0 conn(0x7fa918067ff0 0x7fa91806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 --2- 172.21.15.3:0/2818477534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918143140 0x7fa918067ab0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 -- 172.21.15.3:0/2818477534 >> 172.21.15.3:0/2818477534 conn(0x7fa9180ce210 msgr2=0x7fa9180ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 -- 172.21.15.3:0/2818477534 shutdown_connections 2023-10-06T05:01:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.020+0000 7fa9202e9700 1 -- 172.21.15.3:0/2818477534 wait complete. 2023-10-06T05:01:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.021+0000 7fa9202e9700 1 Processor -- start 2023-10-06T05:01:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.021+0000 7fa9202e9700 1 -- start start 2023-10-06T05:01:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa9202e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918067ff0 0x7fa91816aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa9202e9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa918142d70 0x7fa91813efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa9202e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918143140 0x7fa9181414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa9202e9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91813fea0 con 0x7fa918067ff0 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa9202e9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91816afc0 con 0x7fa918143140 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa9202e9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180c9d50 con 0x7fa918142d70 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918067ff0 0x7fa91816aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918067ff0 0x7fa91816aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59826/0 (socket says 172.21.15.3:59826) 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 -- 172.21.15.3:0/1286467716 learned_addr learned my addr 172.21.15.3:0/1286467716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e886700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918143140 0x7fa9181414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91d884700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa918142d70 0x7fa91813efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 -- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa918142d70 msgr2=0x7fa91813efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa918142d70 0x7fa91813efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 -- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918143140 msgr2=0x7fa9181414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918143140 0x7fa9181414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 -- 172.21.15.3:0/1286467716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0094f0 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91d884700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa918142d70 0x7fa91813efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa91e085700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918067ff0 0x7fa91816aa80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c590 tx=0x7fa910007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.022+0000 7fa9167fc700 1 -- 172.21.15.3:0/1286467716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011590 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.023+0000 7fa9167fc700 1 -- 172.21.15.3:0/1286467716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910011730 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.023+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9180c9800 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.023+0000 7fa9167fc700 1 -- 172.21.15.3:0/1286467716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011af0 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.023+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91813fac0 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.023+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91804c380 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.024+0000 7fa9167fc700 1 -- 172.21.15.3:0/1286467716 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa910012420 con 0x7fa918067ff0 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.024+0000 7fa9167fc700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa904066f30 0x7fa9040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.024+0000 7fa9167fc700 1 -- 172.21.15.3:0/1286467716 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa910083630 con 0x7fa918067ff0 2023-10-06T05:01:13.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.025+0000 7fa91d884700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa904066f30 0x7fa9040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.025+0000 7fa91d884700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa904066f30 0x7fa9040693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa918143590 tx=0x7fa90c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.028+0000 7fa9167fc700 1 -- 172.21.15.3:0/1286467716 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa91004b5a0 con 0x7fa918067ff0 2023-10-06T05:01:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.222+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7fa918069150 con 0x7fa918067ff0 2023-10-06T05:01:13.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.225+0000 7fa9167fc700 1 -- 172.21.15.3:0/1286467716 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v62) v1 ==== 102+0+0 (secure 0 0 0) 0x7fa910017020 con 0x7fa918067ff0 2023-10-06T05:01:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa904066f30 msgr2=0x7fa9040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa904066f30 0x7fa9040693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa918143590 tx=0x7fa90c017000 comp rx=0 tx=0).stop 2023-10-06T05:01:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918067ff0 msgr2=0x7fa91816aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918067ff0 0x7fa91816aa80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c590 tx=0x7fa910007f30 comp rx=0 tx=0).stop 2023-10-06T05:01:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 shutdown_connections 2023-10-06T05:01:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa904066f30 0x7fa9040693e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918143140 0x7fa9181414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa918067ff0 0x7fa91816aa80 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 --2- 172.21.15.3:0/1286467716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa918142d70 0x7fa91813efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 >> 172.21.15.3:0/1286467716 conn(0x7fa9180ce210 msgr2=0x7fa9180c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 shutdown_connections 2023-10-06T05:01:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.227+0000 7fa9202e9700 1 -- 172.21.15.3:0/1286467716 wait complete. 2023-10-06T05:01:13.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-10-06T05:01:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.360+0000 7f878d423700 1 Processor -- start 2023-10-06T05:01:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.360+0000 7f878d423700 1 -- start start 2023-10-06T05:01:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f878d423700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f878d423700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f87880cba50 0x7f878815d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f878d423700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878815da80 con 0x7f878815a720 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f878d423700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878815a290 con 0x7f87880cba50 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f878d423700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880d0020 con 0x7f8788150690 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f87867fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f87880cba50 0x7f878815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:44624/0 (socket says 172.21.15.3:44624) 2023-10-06T05:01:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f8786ffd700 1 -- 172.21.15.3:0/3255740232 learned_addr learned my addr 172.21.15.3:0/3255740232 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f8786ffd700 1 -- 172.21.15.3:0/3255740232 >> v2:172.21.15.153:3300/0 conn(0x7f87880cba50 msgr2=0x7f878815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f8786ffd700 1 --2- 172.21.15.3:0/3255740232 >> v2:172.21.15.153:3300/0 conn(0x7f87880cba50 0x7f878815d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f8786ffd700 1 -- 172.21.15.3:0/3255740232 >> v1:172.21.15.153:6789/0 conn(0x7f878815a720 legacy=0x7f87880af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f8786ffd700 1 -- 172.21.15.3:0/3255740232 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788159f40 con 0x7f8788150690 2023-10-06T05:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.361+0000 7f87867fc700 1 --2- 172.21.15.3:0/3255740232 >> v2:172.21.15.153:3300/0 conn(0x7f87880cba50 0x7f878815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f8786ffd700 1 --2- 172.21.15.3:0/3255740232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87780099b0 tx=0x7f8778009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87ddafb3b72c9a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f8785ffb700 1 -- 172.21.15.3:0/3255740232 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778009ea0 con 0x7f8788150690 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f8785ffb700 1 -- 172.21.15.3:0/3255740232 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877800b860 con 0x7f8788150690 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f8785ffb700 1 -- 172.21.15.3:0/3255740232 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877800bc00 con 0x7f8788150690 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 -- 172.21.15.3:0/3255740232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 msgr2=0x7f8788150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 --2- 172.21.15.3:0/3255740232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87780099b0 tx=0x7f8778009250 comp rx=0 tx=0).stop 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 -- 172.21.15.3:0/3255740232 shutdown_connections 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 --2- 172.21.15.3:0/3255740232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 --2- 172.21.15.3:0/3255740232 >> v2:172.21.15.153:3300/0 conn(0x7f87880cba50 0x7f878815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 -- 172.21.15.3:0/3255740232 >> 172.21.15.3:0/3255740232 conn(0x7f8788070310 msgr2=0x7f8788070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 -- 172.21.15.3:0/3255740232 shutdown_connections 2023-10-06T05:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.362+0000 7f878d423700 1 -- 172.21.15.3:0/3255740232 wait complete. 2023-10-06T05:01:13.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 Processor -- start 2023-10-06T05:01:13.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 -- start start 2023-10-06T05:01:13.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87880cba50 0x7f878814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f878815a720 0x7f8788154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8788155ad0 con 0x7f87880cba50 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880b4800 con 0x7f878815a720 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.364+0000 7f878d423700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880b5340 con 0x7f8788150690 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87880cba50 0x7f878814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87880cba50 0x7f878814fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59834/0 (socket says 172.21.15.3:59834) 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 -- 172.21.15.3:0/1968014928 learned_addr learned my addr 172.21.15.3:0/1968014928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f87867fc700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f87877fe700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f878815a720 0x7f8788154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 -- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 msgr2=0x7f8788150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 -- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f878815a720 msgr2=0x7f8788154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f878815a720 0x7f8788154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 -- 172.21.15.3:0/1968014928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770009420 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f8786ffd700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87880cba50 0x7f878814fdc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f87880d0160 tx=0x7f8778017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f877effd700 1 -- 172.21.15.3:0/1968014928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877800bbe0 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f877effd700 1 -- 172.21.15.3:0/1968014928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8778011420 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87780094f0 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f877effd700 1 -- 172.21.15.3:0/1968014928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87780117c0 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.365+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87880b4cf0 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.366+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878804c380 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.367+0000 7f877effd700 1 -- 172.21.15.3:0/1968014928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8778011bf0 con 0x7f87880cba50 2023-10-06T05:01:13.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.367+0000 7f877effd700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8768066f30 0x7f87680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.367+0000 7f877effd700 1 -- 172.21.15.3:0/1968014928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f8778085bf0 con 0x7f87880cba50 2023-10-06T05:01:13.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.367+0000 7f87867fc700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8768066f30 0x7f87680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.368+0000 7f87867fc700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8768066f30 0x7f87680693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f87880b2a90 tx=0x7f87700091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.371+0000 7f877effd700 1 -- 172.21.15.3:0/1968014928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8778053000 con 0x7f87880cba50 2023-10-06T05:01:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.568+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f878815dcb0 con 0x7f87880cba50 2023-10-06T05:01:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.574+0000 7f877effd700 1 -- 172.21.15.3:0/1968014928 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v63) v1 ==== 111+0+0 (secure 0 0 0) 0x7f8778056620 con 0x7f87880cba50 2023-10-06T05:01:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8768066f30 msgr2=0x7f87680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8768066f30 0x7f87680693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f87880b2a90 tx=0x7f87700091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87880cba50 msgr2=0x7f878814fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87880cba50 0x7f878814fdc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f87880d0160 tx=0x7f8778017000 comp rx=0 tx=0).stop 2023-10-06T05:01:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 shutdown_connections 2023-10-06T05:01:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8768066f30 0x7f87680693e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f878815a720 0x7f8788154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87880cba50 0x7f878814fdc0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 --2- 172.21.15.3:0/1968014928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8788150690 0x7f8788150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 >> 172.21.15.3:0/1968014928 conn(0x7f8788070310 msgr2=0x7f87881591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 shutdown_connections 2023-10-06T05:01:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.576+0000 7f878d423700 1 -- 172.21.15.3:0/1968014928 wait complete. 2023-10-06T05:01:13.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2023-10-06T05:01:13.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.708+0000 7fbc4079f700 1 Processor -- start 2023-10-06T05:01:13.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.708+0000 7fbc4079f700 1 -- start start 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc4079f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbc3815c8f0 0x7fbc3815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc4079f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc3815ccc0 0x7fbc380d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc4079f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc380cf6c0 con 0x7fbc380d06a0 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc4079f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc380cfb50 con 0x7fbc3815ccc0 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc4079f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc380d0b20 con 0x7fbc3815c8f0 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3dd3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc3815ccc0 0x7fbc380d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3ed3c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbc380d06a0 0x7fbc3806d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45554/0 (socket says 172.21.15.3:45554) 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3ed3c700 1 -- 172.21.15.3:0/1900864551 learned_addr learned my addr 172.21.15.3:0/1900864551 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:13.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3e53b700 1 --2- 172.21.15.3:0/1900864551 >> v2:172.21.15.153:3300/0 conn(0x7fbc3815c8f0 0x7fbc3815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3dd3a700 1 -- 172.21.15.3:0/1900864551 >> v2:172.21.15.153:3300/0 conn(0x7fbc3815c8f0 msgr2=0x7fbc3815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3dd3a700 1 --2- 172.21.15.3:0/1900864551 >> v2:172.21.15.153:3300/0 conn(0x7fbc3815c8f0 0x7fbc3815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3dd3a700 1 -- 172.21.15.3:0/1900864551 >> v1:172.21.15.3:6789/0 conn(0x7fbc380d06a0 legacy=0x7fbc3806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3dd3a700 1 -- 172.21.15.3:0/1900864551 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3815e050 con 0x7fbc3815ccc0 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3e53b700 1 --2- 172.21.15.3:0/1900864551 >> v2:172.21.15.153:3300/0 conn(0x7fbc3815c8f0 0x7fbc3815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3dd3a700 1 --2- 172.21.15.3:0/1900864551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc3815ccc0 0x7fbc380d0160 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098e0 tx=0x7fbc280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf1cf3c64f55123 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3d539700 1 -- 172.21.15.3:0/1900864551 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28009e80 con 0x7fbc3815ccc0 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3d539700 1 -- 172.21.15.3:0/1900864551 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2800b820 con 0x7fbc3815ccc0 2023-10-06T05:01:13.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.709+0000 7fbc3d539700 1 -- 172.21.15.3:0/1900864551 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800bba0 con 0x7fbc3815ccc0 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 -- 172.21.15.3:0/1900864551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc3815ccc0 msgr2=0x7fbc380d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 --2- 172.21.15.3:0/1900864551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc3815ccc0 0x7fbc380d0160 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098e0 tx=0x7fbc280091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 -- 172.21.15.3:0/1900864551 shutdown_connections 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 --2- 172.21.15.3:0/1900864551 >> v2:172.21.15.153:3300/0 conn(0x7fbc3815c8f0 0x7fbc3815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 --2- 172.21.15.3:0/1900864551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc3815ccc0 0x7fbc380d0160 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 -- 172.21.15.3:0/1900864551 >> 172.21.15.3:0/1900864551 conn(0x7fbc38070310 msgr2=0x7fbc38070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 -- 172.21.15.3:0/1900864551 shutdown_connections 2023-10-06T05:01:13.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.710+0000 7fbc4079f700 1 -- 172.21.15.3:0/1900864551 wait complete. 2023-10-06T05:01:13.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.711+0000 7fbc4079f700 1 Processor -- start 2023-10-06T05:01:13.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.711+0000 7fbc4079f700 1 -- start start 2023-10-06T05:01:13.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc4079f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc380d06a0 0x7fbc3814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc4079f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc3815c8f0 0x7fbc38169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc4079f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc3815ccc0 0x7fbc38155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc4079f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38152fe0 con 0x7fbc380d06a0 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc4079f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc381523d0 con 0x7fbc3815c8f0 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc4079f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc381526b0 con 0x7fbc3815ccc0 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc380d06a0 0x7fbc3814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc380d06a0 0x7fbc3814cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59866/0 (socket says 172.21.15.3:59866) 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 -- 172.21.15.3:0/4238789634 learned_addr learned my addr 172.21.15.3:0/4238789634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3ed3c700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc3815ccc0 0x7fbc38155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3dd3a700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc3815c8f0 0x7fbc38169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 -- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc3815ccc0 msgr2=0x7fbc38155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc3815ccc0 0x7fbc38155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 -- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc3815c8f0 msgr2=0x7fbc38169d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:13.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc3815c8f0 0x7fbc38169d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 -- 172.21.15.3:0/4238789634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc28009420 con 0x7fbc380d06a0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3ed3c700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc3815ccc0 0x7fbc38155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.712+0000 7fbc3e53b700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc380d06a0 0x7fbc3814cf00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c850 tx=0x7fbc34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.713+0000 7fbc2e7fc700 1 -- 172.21.15.3:0/4238789634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34011600 con 0x7fbc380d06a0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.713+0000 7fbc2e7fc700 1 -- 172.21.15.3:0/4238789634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34011760 con 0x7fbc380d06a0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.713+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc38152990 con 0x7fbc380d06a0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.713+0000 7fbc2e7fc700 1 -- 172.21.15.3:0/4238789634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34011b00 con 0x7fbc380d06a0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.713+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3814d440 con 0x7fbc380d06a0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.713+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc38071680 con 0x7fbc380d06a0 2023-10-06T05:01:13.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.714+0000 7fbc2e7fc700 1 -- 172.21.15.3:0/4238789634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc340073b0 con 0x7fbc380d06a0 2023-10-06T05:01:13.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.714+0000 7fbc2e7fc700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbc1c066f30 0x7fbc1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:13.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.714+0000 7fbc2e7fc700 1 -- 172.21.15.3:0/4238789634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fbc34084310 con 0x7fbc380d06a0 2023-10-06T05:01:13.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.715+0000 7fbc3dd3a700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbc1c066f30 0x7fbc1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:13.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.715+0000 7fbc3dd3a700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbc1c066f30 0x7fbc1c0693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098e0 tx=0x7fbc28017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:13.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.719+0000 7fbc2e7fc700 1 -- 172.21.15.3:0/4238789634 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc340551b0 con 0x7fbc380d06a0 2023-10-06T05:01:13.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:13.995+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) v1 -- 0x7fbc38154120 con 0x7fbc380d06a0 2023-10-06T05:01:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.834+0000 7fbc2e7fc700 1 -- 172.21.15.3:0/4238789634 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v453) v1 ==== 148+0+0 (secure 0 0 0) 0x7fbc34054d40 con 0x7fbc380d06a0 2023-10-06T05:01:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbc1c066f30 msgr2=0x7fbc1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbc1c066f30 0x7fbc1c0693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098e0 tx=0x7fbc28017000 comp rx=0 tx=0).stop 2023-10-06T05:01:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc380d06a0 msgr2=0x7fbc3814cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc380d06a0 0x7fbc3814cf00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c850 tx=0x7fbc34008040 comp rx=0 tx=0).stop 2023-10-06T05:01:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 shutdown_connections 2023-10-06T05:01:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fbc1c066f30 0x7fbc1c0693e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc3815ccc0 0x7fbc38155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc380d06a0 0x7fbc3814cf00 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 --2- 172.21.15.3:0/4238789634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc3815c8f0 0x7fbc38169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 >> 172.21.15.3:0/4238789634 conn(0x7fbc38070310 msgr2=0x7fbc3806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 shutdown_connections 2023-10-06T05:01:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.837+0000 7fbc4079f700 1 -- 172.21.15.3:0/4238789634 wait complete. 2023-10-06T05:01:14.854 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-10-06T05:01:14.854 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) When a client has a bug (which we will simulate) preventing it from releasing caps, 2023-10-06T05:01:14.854 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-06T05:01:14.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2023-10-06T05:01:14.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.981+0000 7f9cd6485700 1 Processor -- start 2023-10-06T05:01:14.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.981+0000 7f9cd6485700 1 -- start start 2023-10-06T05:01:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.981+0000 7f9cd6485700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cd0159500 0x7f9cd00cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.981+0000 7f9cd6485700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd00cc5c0 con 0x7f9cd00b2930 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.981+0000 7f9cd6485700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd00cb9b0 con 0x7f9cd01598d0 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.981+0000 7f9cd6485700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00cbc90 con 0x7f9cd0159500 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9ccffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cd0159500 0x7f9cd00cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9cd4a22700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9cd00b2930 0x7f9cd00cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45570/0 (socket says 172.21.15.3:45570) 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9cd4a22700 1 -- 172.21.15.3:0/3925978137 learned_addr learned my addr 172.21.15.3:0/3925978137 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9ccffff700 1 -- 172.21.15.3:0/3925978137 >> v1:172.21.15.153:6789/0 conn(0x7f9cd01598d0 legacy=0x7f9cd00b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9ccffff700 1 -- 172.21.15.3:0/3925978137 >> v1:172.21.15.3:6789/0 conn(0x7f9cd00b2930 legacy=0x7f9cd00cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9ccffff700 1 -- 172.21.15.3:0/3925978137 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00cbf70 con 0x7f9cd0159500 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9ccffff700 1 --2- 172.21.15.3:0/3925978137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0159500 0x7f9cd00cad30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098e0 tx=0x7f9cb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c9ad8f821de516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9cceffd700 1 -- 172.21.15.3:0/3925978137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8009e80 con 0x7f9cd0159500 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9cceffd700 1 -- 172.21.15.3:0/3925978137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb800b860 con 0x7f9cd0159500 2023-10-06T05:01:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.982+0000 7f9cceffd700 1 -- 172.21.15.3:0/3925978137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800bbe0 con 0x7f9cd0159500 2023-10-06T05:01:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.983+0000 7f9cd6485700 1 -- 172.21.15.3:0/3925978137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0159500 msgr2=0x7f9cd00cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.983+0000 7f9cd6485700 1 --2- 172.21.15.3:0/3925978137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0159500 0x7f9cd00cad30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098e0 tx=0x7f9cb80091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.983+0000 7f9cd6485700 1 -- 172.21.15.3:0/3925978137 shutdown_connections 2023-10-06T05:01:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.983+0000 7f9cd6485700 1 --2- 172.21.15.3:0/3925978137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd0159500 0x7f9cd00cad30 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.983+0000 7f9cd6485700 1 -- 172.21.15.3:0/3925978137 >> 172.21.15.3:0/3925978137 conn(0x7f9cd0070310 msgr2=0x7f9cd0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.983+0000 7f9cd6485700 1 -- 172.21.15.3:0/3925978137 shutdown_connections 2023-10-06T05:01:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.983+0000 7f9cd6485700 1 -- 172.21.15.3:0/3925978137 wait complete. 2023-10-06T05:01:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.984+0000 7f9cd6485700 1 Processor -- start 2023-10-06T05:01:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.984+0000 7f9cd6485700 1 -- start start 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9cd6485700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd00b2930 0x7f9cd01509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9cd6485700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cd0159500 0x7f9cd00b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9cd6485700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cd01598d0 0x7f9cd0157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9cd6485700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0150f10 con 0x7f9cd00b2930 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9cd6485700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00b5f40 con 0x7f9cd01598d0 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9cd6485700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00b5330 con 0x7f9cd0159500 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd00b2930 0x7f9cd01509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd00b2930 0x7f9cd01509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59888/0 (socket says 172.21.15.3:59888) 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 -- 172.21.15.3:0/1246602201 learned_addr learned my addr 172.21.15.3:0/1246602201 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9cd4a22700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cd01598d0 0x7f9cd0157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccf7fe700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cd0159500 0x7f9cd00b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 -- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cd0159500 msgr2=0x7f9cd00b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cd0159500 0x7f9cd00b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 -- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cd01598d0 msgr2=0x7f9cd0157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cd01598d0 0x7f9cd0157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 -- 172.21.15.3:0/1246602201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8009420 con 0x7f9cd00b2930 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.985+0000 7f9ccffff700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd00b2930 0x7f9cd01509d0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9cd006ea80 tx=0x7f9cb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.986+0000 7f9cc7fff700 1 -- 172.21.15.3:0/1246602201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800bb20 con 0x7f9cd00b2930 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.986+0000 7f9cc7fff700 1 -- 172.21.15.3:0/1246602201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8025420 con 0x7f9cd00b2930 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.986+0000 7f9cc7fff700 1 -- 172.21.15.3:0/1246602201 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80257c0 con 0x7f9cd00b2930 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.986+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd00b5610 con 0x7f9cd00b2930 2023-10-06T05:01:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.986+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd00b5bd0 con 0x7f9cd00b2930 2023-10-06T05:01:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.987+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd0071680 con 0x7f9cd00b2930 2023-10-06T05:01:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.987+0000 7f9cc7fff700 1 -- 172.21.15.3:0/1246602201 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9cb8023070 con 0x7f9cd00b2930 2023-10-06T05:01:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.987+0000 7f9cc7fff700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.988+0000 7f9cc7fff700 1 -- 172.21.15.3:0/1246602201 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9cb8085a90 con 0x7f9cd00b2930 2023-10-06T05:01:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.988+0000 7f9ccf7fe700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.988+0000 7f9ccf7fe700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cb0066f30 0x7f9cb00693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0156fb0 tx=0x7f9cc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:14.992+0000 7f9cc7fff700 1 -- 172.21.15.3:0/1246602201 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cb8052ea0 con 0x7f9cd00b2930 2023-10-06T05:01:15.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.263+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) v1 -- 0x7f9cd00cab10 con 0x7f9cd00b2930 2023-10-06T05:01:15.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.843+0000 7f9cc7fff700 1 -- 172.21.15.3:0/1246602201 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v454) v1 ==== 154+0+0 (secure 0 0 0) 0x7f9cb80564c0 con 0x7f9cd00b2930 2023-10-06T05:01:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.845+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cb0066f30 msgr2=0x7f9cb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.845+0000 7f9cd6485700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cb0066f30 0x7f9cb00693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0156fb0 tx=0x7f9cc0009040 comp rx=0 tx=0).stop 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.845+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd00b2930 msgr2=0x7f9cd01509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd00b2930 0x7f9cd01509d0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9cd006ea80 tx=0x7f9cb8017000 comp rx=0 tx=0).stop 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 shutdown_connections 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cd01598d0 0x7f9cd0157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cd00b2930 0x7f9cd01509d0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 --2- 172.21.15.3:0/1246602201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cd0159500 0x7f9cd00b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 >> 172.21.15.3:0/1246602201 conn(0x7f9cd0070310 msgr2=0x7f9cd0158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 shutdown_connections 2023-10-06T05:01:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.846+0000 7f9cd6485700 1 -- 172.21.15.3:0/1246602201 wait complete. 2023-10-06T05:01:15.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:01:15.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.985+0000 7f8148853700 1 Processor -- start 2023-10-06T05:01:15.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.985+0000 7f8148853700 1 -- start start 2023-10-06T05:01:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.986+0000 7f8148853700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81440b0fb0 0x7f81440b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.986+0000 7f8148853700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8144067ab0 0x7f8144067f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.986+0000 7f8148853700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8144069f10 con 0x7f8144140e70 2023-10-06T05:01:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.986+0000 7f8148853700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8144068470 con 0x7f8144067ab0 2023-10-06T05:01:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.986+0000 7f8148853700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81440cca70 con 0x7f81440b0fb0 2023-10-06T05:01:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81440b0fb0 0x7f81440b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8142d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8144067ab0 0x7f8144067f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:15.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81440b0fb0 0x7f81440b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:44660/0 (socket says 172.21.15.3:44660) 2023-10-06T05:01:15.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 -- 172.21.15.3:0/3538356339 learned_addr learned my addr 172.21.15.3:0/3538356339 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 -- 172.21.15.3:0/3538356339 >> v2:172.21.15.153:3300/0 conn(0x7f8144067ab0 msgr2=0x7f8144067f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 --2- 172.21.15.3:0/3538356339 >> v2:172.21.15.153:3300/0 conn(0x7f8144067ab0 0x7f8144067f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 -- 172.21.15.3:0/3538356339 >> v1:172.21.15.153:6789/0 conn(0x7f8144140e70 legacy=0x7f81440ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 -- 172.21.15.3:0/3538356339 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81440cc140 con 0x7f81440b0fb0 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8142d9d700 1 --2- 172.21.15.3:0/3538356339 >> v2:172.21.15.153:3300/0 conn(0x7f8144067ab0 0x7f8144067f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f8141d9b700 1 --2- 172.21.15.3:0/3538356339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81440b0fb0 0x7f81440b1420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f812c0099b0 tx=0x7f812c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0fcd04007a8bbd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f814159a700 1 -- 172.21.15.3:0/3538356339 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c009ea0 con 0x7f81440b0fb0 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f814159a700 1 -- 172.21.15.3:0/3538356339 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c00b860 con 0x7f81440b0fb0 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.987+0000 7f814159a700 1 -- 172.21.15.3:0/3538356339 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c00bc00 con 0x7f81440b0fb0 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 -- 172.21.15.3:0/3538356339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81440b0fb0 msgr2=0x7f81440b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 --2- 172.21.15.3:0/3538356339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81440b0fb0 0x7f81440b1420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f812c0099b0 tx=0x7f812c009250 comp rx=0 tx=0).stop 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 -- 172.21.15.3:0/3538356339 shutdown_connections 2023-10-06T05:01:15.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 --2- 172.21.15.3:0/3538356339 >> v2:172.21.15.153:3300/0 conn(0x7f8144067ab0 0x7f8144067f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 --2- 172.21.15.3:0/3538356339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81440b0fb0 0x7f81440b1420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 -- 172.21.15.3:0/3538356339 >> 172.21.15.3:0/3538356339 conn(0x7f81440ce210 msgr2=0x7f81440ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:15.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 -- 172.21.15.3:0/3538356339 shutdown_connections 2023-10-06T05:01:15.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.988+0000 7f8148853700 1 -- 172.21.15.3:0/3538356339 wait complete. 2023-10-06T05:01:15.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.989+0000 7f8148853700 1 Processor -- start 2023-10-06T05:01:15.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.989+0000 7f8148853700 1 -- start start 2023-10-06T05:01:15.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8148853700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8144067ab0 0x7f814416d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:15.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8148853700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81440b0fb0 0x7f814416ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:15.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8148853700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8144140e70 0x7f814416b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8148853700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81440c7740 con 0x7f81440b0fb0 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8148853700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814416b2c0 con 0x7f8144067ab0 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8148853700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81440c6b30 con 0x7f8144140e70 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81440b0fb0 0x7f814416ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81440b0fb0 0x7f814416ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59898/0 (socket says 172.21.15.3:59898) 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 -- 172.21.15.3:0/604937478 learned_addr learned my addr 172.21.15.3:0/604937478 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8142d9d700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8144140e70 0x7f814416b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f814259c700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8144067ab0 0x7f814416d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 -- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8144140e70 msgr2=0x7f814416b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8144140e70 0x7f814416b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 -- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8144067ab0 msgr2=0x7f814416d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:15.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8144067ab0 0x7f814416d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.990+0000 7f8141d9b700 1 -- 172.21.15.3:0/604937478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134009420 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.991+0000 7f8141d9b700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81440b0fb0 0x7f814416ad80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8144068cf0 tx=0x7f812c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.991+0000 7f813a7fc700 1 -- 172.21.15.3:0/604937478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c00bbe0 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.991+0000 7f813a7fc700 1 -- 172.21.15.3:0/604937478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c011420 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.991+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c0094f0 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.991+0000 7f813a7fc700 1 -- 172.21.15.3:0/604937478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c0117c0 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.991+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81440c70f0 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.992+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81440c78f0 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.992+0000 7f813a7fc700 1 -- 172.21.15.3:0/604937478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f812c011bf0 con 0x7f81440b0fb0 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.993+0000 7f813a7fc700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8124066f30 0x7f81240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.993+0000 7f813a7fc700 1 -- 172.21.15.3:0/604937478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f812c085b20 con 0x7f81440b0fb0 2023-10-06T05:01:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.993+0000 7f814259c700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8124066f30 0x7f81240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.993+0000 7f814259c700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8124066f30 0x7f81240693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f813c007fd0 tx=0x7f813c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:15.997+0000 7f813a7fc700 1 -- 172.21.15.3:0/604937478 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81440c78f0 con 0x7f81440b0fb0 2023-10-06T05:01:16.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.189+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f81440c78f0 con 0x7f81440b0fb0 2023-10-06T05:01:16.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.194+0000 7f813a7fc700 1 -- 172.21.15.3:0/604937478 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81440c78f0 con 0x7f81440b0fb0 2023-10-06T05:01:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8124066f30 msgr2=0x7f81240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8124066f30 0x7f81240693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f813c007fd0 tx=0x7f813c009040 comp rx=0 tx=0).stop 2023-10-06T05:01:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81440b0fb0 msgr2=0x7f814416ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81440b0fb0 0x7f814416ad80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8144068cf0 tx=0x7f812c017000 comp rx=0 tx=0).stop 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 shutdown_connections 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8124066f30 0x7f81240693e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8144067ab0 0x7f814416d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8144140e70 0x7f814416b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 --2- 172.21.15.3:0/604937478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81440b0fb0 0x7f814416ad80 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 >> 172.21.15.3:0/604937478 conn(0x7f81440ce210 msgr2=0x7f81440b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 shutdown_connections 2023-10-06T05:01:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.196+0000 7f8148853700 1 -- 172.21.15.3:0/604937478 wait complete. 2023-10-06T05:01:16.210 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:01:16.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:01:16.250 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:01:16.298 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:01:16.298 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:01:16.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.415+0000 7fc41547a700 1 Processor -- start 2023-10-06T05:01:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.415+0000 7fc41547a700 1 -- start start 2023-10-06T05:01:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc41547a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41015c8f0 0x7fc41015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc41547a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc41015ccc0 0x7fc4100d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc41547a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4100d06a0 0x7fc41006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc41547a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100cf6c0 con 0x7fc41015c8f0 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc41547a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100cfb50 con 0x7fc41015ccc0 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc41547a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100d0b20 con 0x7fc4100d06a0 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc40effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41015c8f0 0x7fc41015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc40effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41015c8f0 0x7fc41015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59914/0 (socket says 172.21.15.3:59914) 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.416+0000 7fc40effd700 1 -- 172.21.15.3:0/2728593634 learned_addr learned my addr 172.21.15.3:0/2728593634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40effd700 1 -- 172.21.15.3:0/2728593634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4100d06a0 msgr2=0x7fc41006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40effd700 1 --2- 172.21.15.3:0/2728593634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4100d06a0 0x7fc41006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40effd700 1 -- 172.21.15.3:0/2728593634 >> v2:172.21.15.153:3300/0 conn(0x7fc41015ccc0 msgr2=0x7fc4100d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40e7fc700 1 --2- 172.21.15.3:0/2728593634 >> v2:172.21.15.153:3300/0 conn(0x7fc41015ccc0 0x7fc4100d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40effd700 1 --2- 172.21.15.3:0/2728593634 >> v2:172.21.15.153:3300/0 conn(0x7fc41015ccc0 0x7fc4100d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40effd700 1 -- 172.21.15.3:0/2728593634 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41015e050 con 0x7fc41015c8f0 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40effd700 1 --2- 172.21.15.3:0/2728593634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015c8f0 0x7fc41015db10 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc4100b6550 tx=0x7fc400009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bd7c464d5e52b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40dffb700 1 -- 172.21.15.3:0/2728593634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40000e070 con 0x7fc41015c8f0 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40dffb700 1 -- 172.21.15.3:0/2728593634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40000ba50 con 0x7fc41015c8f0 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc40dffb700 1 -- 172.21.15.3:0/2728593634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000092e0 con 0x7fc41015c8f0 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 -- 172.21.15.3:0/2728593634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015c8f0 msgr2=0x7fc41015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 --2- 172.21.15.3:0/2728593634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015c8f0 0x7fc41015db10 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc4100b6550 tx=0x7fc400009840 comp rx=0 tx=0).stop 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 -- 172.21.15.3:0/2728593634 shutdown_connections 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 --2- 172.21.15.3:0/2728593634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4100d06a0 0x7fc41006d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 --2- 172.21.15.3:0/2728593634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015c8f0 0x7fc41015db10 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 --2- 172.21.15.3:0/2728593634 >> v2:172.21.15.153:3300/0 conn(0x7fc41015ccc0 0x7fc4100d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 -- 172.21.15.3:0/2728593634 >> 172.21.15.3:0/2728593634 conn(0x7fc410070310 msgr2=0x7fc410070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:16.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.417+0000 7fc41547a700 1 -- 172.21.15.3:0/2728593634 shutdown_connections 2023-10-06T05:01:16.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.418+0000 7fc41547a700 1 -- 172.21.15.3:0/2728593634 wait complete. 2023-10-06T05:01:16.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 Processor -- start 2023-10-06T05:01:16.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 -- start start 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4100d06a0 0x7fc41014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41015c8f0 0x7fc4101527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015ccc0 0x7fc410152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41014df60 con 0x7fc41015ccc0 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc410150100 con 0x7fc4100d06a0 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.419+0000 7fc41547a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41014d950 con 0x7fc41015c8f0 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015ccc0 0x7fc410152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015ccc0 0x7fc410152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59926/0 (socket says 172.21.15.3:59926) 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 -- 172.21.15.3:0/2962264846 learned_addr learned my addr 172.21.15.3:0/2962264846 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 -- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41015c8f0 msgr2=0x7fc4101527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41015c8f0 0x7fc4101527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 -- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4100d06a0 msgr2=0x7fc41014cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40effd700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4100d06a0 0x7fc41014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4100d06a0 0x7fc41014cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 -- 172.21.15.3:0/2962264846 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc400009ae0 con 0x7fc41015ccc0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc40f7fe700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015ccc0 0x7fc410152d30 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc40800b5c0 tx=0x7fc408006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc406ffd700 1 -- 172.21.15.3:0/2962264846 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800f510 con 0x7fc41015ccc0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc406ffd700 1 -- 172.21.15.3:0/2962264846 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800f670 con 0x7fc41015ccc0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41016a3b0 con 0x7fc41015ccc0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc406ffd700 1 -- 172.21.15.3:0/2962264846 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800fa30 con 0x7fc41015ccc0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.420+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41016a690 con 0x7fc41015ccc0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.421+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410071680 con 0x7fc41015ccc0 2023-10-06T05:01:16.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.422+0000 7fc406ffd700 1 -- 172.21.15.3:0/2962264846 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc40801a400 con 0x7fc41015ccc0 2023-10-06T05:01:16.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.422+0000 7fc406ffd700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f0066f30 0x7fc3f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.422+0000 7fc406ffd700 1 -- 172.21.15.3:0/2962264846 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc408083f10 con 0x7fc41015ccc0 2023-10-06T05:01:16.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.422+0000 7fc40effd700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f0066f30 0x7fc3f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.423+0000 7fc40effd700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f0066f30 0x7fc3f00693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc4100b0ba0 tx=0x7fc400018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:16.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.427+0000 7fc406ffd700 1 -- 172.21.15.3:0/2962264846 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4080513a0 con 0x7fc41015ccc0 2023-10-06T05:01:16.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.662+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4100b35a0 con 0x7fc41015ccc0 2023-10-06T05:01:16.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.663+0000 7fc406ffd700 1 -- 172.21.15.3:0/2962264846 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fc4080549c0 con 0x7fc41015ccc0 2023-10-06T05:01:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:16.665 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1133577782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1133577782},{"type":"v1","addr":"172.21.15.153:6836","nonce":1133577782}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":7630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1934314758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":1934314758},{"type":"v1","addr":"172.21.15.153:6837","nonce":1934314758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/438241764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":438241764},{"type":"v1","addr":"172.21.15.3:6835","nonce":438241764}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2668975615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2668975615},{"type":"v1","addr":"172.21.15.3:6834","nonce":2668975615}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[]} 2023-10-06T05:01:16.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f0066f30 msgr2=0x7fc3f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f0066f30 0x7fc3f00693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc4100b0ba0 tx=0x7fc400018000 comp rx=0 tx=0).stop 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015ccc0 msgr2=0x7fc410152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015ccc0 0x7fc410152d30 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc40800b5c0 tx=0x7fc408006f90 comp rx=0 tx=0).stop 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 shutdown_connections 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc3f0066f30 0x7fc3f00693e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4100d06a0 0x7fc41014cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015ccc0 0x7fc410152d30 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 --2- 172.21.15.3:0/2962264846 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc41015c8f0 0x7fc4101527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 >> 172.21.15.3:0/2962264846 conn(0x7fc410070310 msgr2=0x7fc4100cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 shutdown_connections 2023-10-06T05:01:16.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.665+0000 7fc41547a700 1 -- 172.21.15.3:0/2962264846 wait complete. 2023-10-06T05:01:16.668 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 83 2023-10-06T05:01:16.679 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:01:16.679 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:01:16.679 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:01:16.679 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:01:16.679 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:01:16.680 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:01:16.680 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:01:16.680 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:01:16.680 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:01:16.680 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:01:16.680 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:01:16.681 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:01:16.681 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:01:16.681 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:01:16.681 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:01:16.681 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:01:16.688 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:01:16.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:01:16.689 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:01:16.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:01:16.694 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:01:16.694 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:01:16.695 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:01:16.695 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:01:16.731 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:01:16.731 DEBUG:teuthology.parallel:result is None 2023-10-06T05:01:16.732 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:01:16.732 DEBUG:teuthology.parallel:result is None 2023-10-06T05:01:16.737 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:01:16.738 DEBUG:teuthology.parallel:result is None 2023-10-06T05:01:16.738 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:01:16.738 DEBUG:teuthology.parallel:result is None 2023-10-06T05:01:16.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:01:16.808 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-06T05:01:16.808 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-10-06T05:01:16.818 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T05:01:16.818 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-10-06T05:01:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.865+0000 7f8746b8d700 1 Processor -- start 2023-10-06T05:01:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.865+0000 7f8746b8d700 1 -- start start 2023-10-06T05:01:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f8746b8d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f874015ccc0 0x7f87400d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f8746b8d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f87400d06a0 0x7f87400b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f8746b8d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400cf6c0 con 0x7f874015c8f0 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f8746b8d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400cfb50 con 0x7f87400d06a0 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f8746b8d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400d0b20 con 0x7f874015ccc0 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f8744929700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f874015c8f0 0x7f874015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45582/0 (socket says 172.21.15.3:45582) 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f8744929700 1 -- 172.21.15.3:0/1171892790 learned_addr learned my addr 172.21.15.3:0/1171892790 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f874512a700 1 --2- 172.21.15.3:0/1171892790 >> v2:172.21.15.153:3300/0 conn(0x7f87400d06a0 0x7f87400b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.866+0000 7f873ffff700 1 --2- 172.21.15.3:0/1171892790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f874015ccc0 0x7f87400d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873f7fe700 1 -- 172.21.15.3:0/1171892790 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191134340 0 0) 0x7f87400cf6c0 con 0x7f874015c8f0 2023-10-06T05:01:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873f7fe700 1 -- 172.21.15.3:0/1171892790 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8730003570 con 0x7f874015c8f0 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873ffff700 1 -- 172.21.15.3:0/1171892790 >> v2:172.21.15.153:3300/0 conn(0x7f87400d06a0 msgr2=0x7f87400b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873ffff700 1 --2- 172.21.15.3:0/1171892790 >> v2:172.21.15.153:3300/0 conn(0x7f87400d06a0 0x7f87400b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873ffff700 1 -- 172.21.15.3:0/1171892790 >> v1:172.21.15.3:6789/0 conn(0x7f874015c8f0 legacy=0x7f874015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873ffff700 1 -- 172.21.15.3:0/1171892790 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400ceab0 con 0x7f874015ccc0 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f874512a700 1 --2- 172.21.15.3:0/1171892790 >> v2:172.21.15.153:3300/0 conn(0x7f87400d06a0 0x7f87400b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873ffff700 1 --2- 172.21.15.3:0/1171892790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f874015ccc0 0x7f87400d0160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f872c0098e0 tx=0x7f872c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1a3c99609ae10f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873f7fe700 1 -- 172.21.15.3:0/1171892790 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c011630 con 0x7f874015ccc0 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873f7fe700 1 -- 172.21.15.3:0/1171892790 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c011790 con 0x7f874015ccc0 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.867+0000 7f873f7fe700 1 -- 172.21.15.3:0/1171892790 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c011b30 con 0x7f874015ccc0 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 -- 172.21.15.3:0/1171892790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f874015ccc0 msgr2=0x7f87400d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 --2- 172.21.15.3:0/1171892790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f874015ccc0 0x7f87400d0160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f872c0098e0 tx=0x7f872c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 -- 172.21.15.3:0/1171892790 shutdown_connections 2023-10-06T05:01:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 --2- 172.21.15.3:0/1171892790 >> v2:172.21.15.153:3300/0 conn(0x7f87400d06a0 0x7f87400b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 --2- 172.21.15.3:0/1171892790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f874015ccc0 0x7f87400d0160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 -- 172.21.15.3:0/1171892790 >> 172.21.15.3:0/1171892790 conn(0x7f8740070310 msgr2=0x7f8740070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 -- 172.21.15.3:0/1171892790 shutdown_connections 2023-10-06T05:01:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.868+0000 7f8746b8d700 1 -- 172.21.15.3:0/1171892790 wait complete. 2023-10-06T05:01:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.869+0000 7f8746b8d700 1 Processor -- start 2023-10-06T05:01:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.869+0000 7f8746b8d700 1 -- start start 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f8746b8d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87400d06a0 0x7f87400c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f8746b8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874015c8f0 0x7f874015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f8746b8d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f874015ccc0 0x7f874015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f8746b8d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874015ac40 con 0x7f874015c8f0 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f8746b8d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8740159490 con 0x7f874015ccc0 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f8746b8d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400c5cf0 con 0x7f87400d06a0 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874015c8f0 0x7f874015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874015c8f0 0x7f874015a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59956/0 (socket says 172.21.15.3:59956) 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 -- 172.21.15.3:0/881286683 learned_addr learned my addr 172.21.15.3:0/881286683 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f8744929700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87400d06a0 0x7f87400c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f874512a700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f874015ccc0 0x7f874015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 -- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87400d06a0 msgr2=0x7f87400c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87400d06a0 0x7f87400c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 -- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f874015ccc0 msgr2=0x7f874015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f874015ccc0 0x7f874015a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 -- 172.21.15.3:0/881286683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738009a20 con 0x7f874015c8f0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.870+0000 7f873ffff700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874015c8f0 0x7f874015a060 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f872c013000 tx=0x7f872c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.871+0000 7f873cff9700 1 -- 172.21.15.3:0/881286683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c011900 con 0x7f874015c8f0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.871+0000 7f873cff9700 1 -- 172.21.15.3:0/881286683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c011a60 con 0x7f874015c8f0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.871+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c009420 con 0x7f874015c8f0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.871+0000 7f873cff9700 1 -- 172.21.15.3:0/881286683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c025670 con 0x7f874015c8f0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.871+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87400c5980 con 0x7f874015c8f0 2023-10-06T05:01:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.872+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87400c5ea0 con 0x7f874015c8f0 2023-10-06T05:01:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.872+0000 7f873cff9700 1 -- 172.21.15.3:0/881286683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f872c023070 con 0x7f874015c8f0 2023-10-06T05:01:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.873+0000 7f873cff9700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8724066f30 0x7f87240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.873+0000 7f873cff9700 1 -- 172.21.15.3:0/881286683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f872c085810 con 0x7f874015c8f0 2023-10-06T05:01:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.873+0000 7f8744929700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8724066f30 0x7f87240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.873+0000 7f8744929700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8724066f30 0x7f87240693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8734007fd0 tx=0x7f8734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:16.877+0000 7f873cff9700 1 -- 172.21.15.3:0/881286683 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87400c5ea0 con 0x7f874015c8f0 2023-10-06T05:01:17.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.085+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f87400c5e60 con 0x7f874015c8f0 2023-10-06T05:01:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.866+0000 7f873cff9700 1 -- 172.21.15.3:0/881286683 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7f872c02a080 con 0x7f874015c8f0 2023-10-06T05:01:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8724066f30 msgr2=0x7f87240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8724066f30 0x7f87240693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8734007fd0 tx=0x7f8734009040 comp rx=0 tx=0).stop 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874015c8f0 msgr2=0x7f874015a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874015c8f0 0x7f874015a060 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f872c013000 tx=0x7f872c017000 comp rx=0 tx=0).stop 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 shutdown_connections 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8724066f30 0x7f87240693e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f874015ccc0 0x7f874015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87400d06a0 0x7f87400c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 --2- 172.21.15.3:0/881286683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874015c8f0 0x7f874015a060 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 >> 172.21.15.3:0/881286683 conn(0x7f8740070310 msgr2=0x7f87400b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 shutdown_connections 2023-10-06T05:01:17.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:17.869+0000 7f8746b8d700 1 -- 172.21.15.3:0/881286683 wait complete. 2023-10-06T05:01:17.872 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-10-06T05:01:17.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:01:18.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.010+0000 7febcd794700 1 Processor -- start 2023-10-06T05:01:18.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.010+0000 7febcd794700 1 -- start start 2023-10-06T05:01:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febcd794700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7febc815ccc0 0x7febc80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febcd794700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febc80d06a0 0x7febc80b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febcd794700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc80cf6c0 con 0x7febc815c8f0 2023-10-06T05:01:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febcd794700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80cfb50 con 0x7febc80d06a0 2023-10-06T05:01:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febcd794700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80d0b00 con 0x7febc815ccc0 2023-10-06T05:01:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febc80d06a0 0x7febc80b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febc80d06a0 0x7febc80b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59958/0 (socket says 172.21.15.3:59958) 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc6ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7febc815c8f0 0x7febc815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45598/0 (socket says 172.21.15.3:45598) 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc77fe700 1 -- 172.21.15.3:0/3913866074 learned_addr learned my addr 172.21.15.3:0/3913866074 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc67fc700 1 --2- 172.21.15.3:0/3913866074 >> v2:172.21.15.153:3300/0 conn(0x7febc815ccc0 0x7febc80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc77fe700 1 -- 172.21.15.3:0/3913866074 >> v2:172.21.15.153:3300/0 conn(0x7febc815ccc0 msgr2=0x7febc80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc77fe700 1 --2- 172.21.15.3:0/3913866074 >> v2:172.21.15.153:3300/0 conn(0x7febc815ccc0 0x7febc80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc77fe700 1 -- 172.21.15.3:0/3913866074 >> v1:172.21.15.3:6789/0 conn(0x7febc815c8f0 legacy=0x7febc815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc77fe700 1 -- 172.21.15.3:0/3913866074 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80ceab0 con 0x7febc80d06a0 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.011+0000 7febc5ffb700 1 -- 172.21.15.3:0/3913866074 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624742958 0 0) 0x7febc80cf6c0 con 0x7febc815c8f0 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febc77fe700 1 --2- 172.21.15.3:0/3913866074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc80d06a0 0x7febc80b3ba0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7febc80b6550 tx=0x7febb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb25d5ed1575c05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febc5ffb700 1 -- 172.21.15.3:0/3913866074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb800b860 con 0x7febc80d06a0 2023-10-06T05:01:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febc5ffb700 1 -- 172.21.15.3:0/3913866074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb800b9c0 con 0x7febc80d06a0 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febc5ffb700 1 -- 172.21.15.3:0/3913866074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb800bd60 con 0x7febc80d06a0 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 -- 172.21.15.3:0/3913866074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc80d06a0 msgr2=0x7febc80b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 --2- 172.21.15.3:0/3913866074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc80d06a0 0x7febc80b3ba0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7febc80b6550 tx=0x7febb8009250 comp rx=0 tx=0).stop 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 -- 172.21.15.3:0/3913866074 shutdown_connections 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 --2- 172.21.15.3:0/3913866074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc80d06a0 0x7febc80b3ba0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 --2- 172.21.15.3:0/3913866074 >> v2:172.21.15.153:3300/0 conn(0x7febc815ccc0 0x7febc80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 -- 172.21.15.3:0/3913866074 >> 172.21.15.3:0/3913866074 conn(0x7febc8070310 msgr2=0x7febc8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 -- 172.21.15.3:0/3913866074 shutdown_connections 2023-10-06T05:01:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.012+0000 7febcd794700 1 -- 172.21.15.3:0/3913866074 wait complete. 2023-10-06T05:01:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 Processor -- start 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 -- start start 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febc80d06a0 0x7febc816d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc815c8f0 0x7febc80b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc815ccc0 0x7febc80c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80c56f0 con 0x7febc815c8f0 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc816d5c0 con 0x7febc815ccc0 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febcd794700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc816d890 con 0x7febc80d06a0 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febc67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc815c8f0 0x7febc80b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febc67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc815c8f0 0x7febc80b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59968/0 (socket says 172.21.15.3:59968) 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.014+0000 7febc67fc700 1 -- 172.21.15.3:0/2828078210 learned_addr learned my addr 172.21.15.3:0/2828078210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc6ffd700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febc80d06a0 0x7febc816d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc77fe700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc815ccc0 0x7febc80c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc67fc700 1 -- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febc80d06a0 msgr2=0x7febc816d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc67fc700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febc80d06a0 0x7febc816d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc67fc700 1 -- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc815ccc0 msgr2=0x7febc80c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc67fc700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc815ccc0 0x7febc80c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc67fc700 1 -- 172.21.15.3:0/2828078210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80094f0 con 0x7febc815c8f0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febc67fc700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc815c8f0 0x7febc80b20d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7febb000d860 tx=0x7febb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febbeffd700 1 -- 172.21.15.3:0/2828078210 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb00125f0 con 0x7febc815c8f0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febbeffd700 1 -- 172.21.15.3:0/2828078210 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb0012750 con 0x7febc815c8f0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc816db70 con 0x7febc815c8f0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febbeffd700 1 -- 172.21.15.3:0/2828078210 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0012b10 con 0x7febc815c8f0 2023-10-06T05:01:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.015+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc80c5940 con 0x7febc815c8f0 2023-10-06T05:01:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.016+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8071680 con 0x7febc815c8f0 2023-10-06T05:01:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.017+0000 7febbeffd700 1 -- 172.21.15.3:0/2828078210 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febb001e460 con 0x7febc815c8f0 2023-10-06T05:01:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.017+0000 7febbeffd700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feba8066f30 0x7feba80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.017+0000 7febbeffd700 1 -- 172.21.15.3:0/2828078210 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5206+0+0 (secure 0 0 0) 0x7febb0084920 con 0x7febc815c8f0 2023-10-06T05:01:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.017+0000 7febc6ffd700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feba8066f30 0x7feba80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.018+0000 7febc6ffd700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feba8066f30 0x7feba80693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7febc0001560 tx=0x7febc0001b00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.021+0000 7febbeffd700 1 -- 172.21.15.3:0/2828078210 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febb0051d80 con 0x7febc815c8f0 2023-10-06T05:01:18.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.300+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7febc815de60 con 0x7febc815c8f0 2023-10-06T05:01:18.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.300+0000 7febbeffd700 1 -- 172.21.15.3:0/2828078210 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7febb00553a0 con 0x7febc815c8f0 2023-10-06T05:01:18.302 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:18.302 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:01:18.302 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-10-06T05:01:18.302 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lXvTAGRAA7yPQ0X2dD0rfp4jUzegRJQ==", 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.303 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lkVduJRAAl5uXCL0PNSylCLJt8osHtg==", 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-10-06T05:01:18.304 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lvKq7IBAAW/vpQOwzRX4L/TbqSzAZ5g==", 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ll4NaLBAAuT3w8qZJ5dgbVmHLfr2PLg==", 2023-10-06T05:01:18.305 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBSkh9l0rXmCBAAlj3B2POXMZzxtry3Yx61gg==", 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBVkh9lpYeHBxAAwu6HZJFBd1aBFFEr0odsPg==", 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBYkh9lWfZ0BRAAa+lafUWesJUYVbMudO5kug==", 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBbkh9lQi+FBBAAujTGaWrER0diXnqt4Erhmw==", 2023-10-06T05:01:18.308 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBhkh9lesn0BRAAlLupoEq1lwMqJ+ap3bpmcA==", 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.309 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBkkh9l/jA/BBAAK7ssFclBNluspIdh9RvXBA==", 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBnkh9l/809AhAAknCZ5Po/VE0oU03Ucc6UTQ==", 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBqkh9lZqNqABAAqi92lTaM/j4BVVEd4uouwg==", 2023-10-06T05:01:18.311 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9l8rBHMBAABBnPwpqGOrwsY2WRPhCn1g==", 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lJb7IOBAA8TwGa6l1Z4bOtozZslMGxQ==", 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.313 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9lRRe/NRAAoqS5lOS3eJSsGGO6KvYxGw==", 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.314 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBPkh9lgiqUAhAA2ErdXEtMIrsWGzpro1TzTg==", 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-10-06T05:01:18.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBNkh9lh8jeMRAAP6vC9rVY6eFeebMkct4qow==", 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.316 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ldKKJFhAA+Qo0+vd1++ghZZysb8Sfyw==", 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-10-06T05:01:18.317 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBOkh9ljvGoCxAAj1wAqckZ+dEy1VoTYNw9rA==", 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T05:01:18.318 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feba8066f30 msgr2=0x7feba80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feba8066f30 0x7feba80693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7febc0001560 tx=0x7febc0001b00 comp rx=0 tx=0).stop 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc815c8f0 msgr2=0x7febc80b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc815c8f0 0x7febc80b20d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7febb000d860 tx=0x7febb0008040 comp rx=0 tx=0).stop 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 shutdown_connections 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7feba8066f30 0x7feba80693e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc815ccc0 0x7febc80c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febc80d06a0 0x7febc816d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 --2- 172.21.15.3:0/2828078210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc815c8f0 0x7febc80b20d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 >> 172.21.15.3:0/2828078210 conn(0x7febc8070310 msgr2=0x7febc80b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:18.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 shutdown_connections 2023-10-06T05:01:18.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.303+0000 7febcd794700 1 -- 172.21.15.3:0/2828078210 wait complete. 2023-10-06T05:01:18.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:01:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.438+0000 7fc894983700 1 Processor -- start 2023-10-06T05:01:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.438+0000 7fc894983700 1 -- start start 2023-10-06T05:01:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc894983700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc890157030 0x7fc890159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc894983700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc894983700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89015aeb0 con 0x7fc89015a220 2023-10-06T05:01:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc894983700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89015a7a0 con 0x7fc890157030 2023-10-06T05:01:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc894983700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89015aa80 con 0x7fc890157400 2023-10-06T05:01:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc88e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc890157030 0x7fc890159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc88dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc89015a220 0x7fc8900b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45614/0 (socket says 172.21.15.3:45614) 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc88dd9b700 1 -- 172.21.15.3:0/2967011661 learned_addr learned my addr 172.21.15.3:0/2967011661 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.439+0000 7fc88ed9d700 1 --2- 172.21.15.3:0/2967011661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88e59c700 1 -- 172.21.15.3:0/2967011661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 msgr2=0x7fc89006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88e59c700 1 --2- 172.21.15.3:0/2967011661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89006e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88e59c700 1 -- 172.21.15.3:0/2967011661 >> v1:172.21.15.3:6789/0 conn(0x7fc89015a220 legacy=0x7fc8900b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88e59c700 1 -- 172.21.15.3:0/2967011661 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89006dba0 con 0x7fc890157030 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88d59a700 1 -- 172.21.15.3:0/2967011661 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645782469 0 0) 0x7fc89015aeb0 con 0x7fc89015a220 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88ed9d700 1 --2- 172.21.15.3:0/2967011661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89006e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88e59c700 1 --2- 172.21.15.3:0/2967011661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc890159ce0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc8880098e0 tx=0x7fc8880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d719bcf4592a8bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88d59a700 1 -- 172.21.15.3:0/2967011661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888009e80 con 0x7fc890157030 2023-10-06T05:01:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88d59a700 1 -- 172.21.15.3:0/2967011661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88800b860 con 0x7fc890157030 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc88d59a700 1 -- 172.21.15.3:0/2967011661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88800bbe0 con 0x7fc890157030 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc894983700 1 -- 172.21.15.3:0/2967011661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 msgr2=0x7fc890159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc894983700 1 --2- 172.21.15.3:0/2967011661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc890159ce0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc8880098e0 tx=0x7fc8880091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc894983700 1 -- 172.21.15.3:0/2967011661 shutdown_connections 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc894983700 1 --2- 172.21.15.3:0/2967011661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89006e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc894983700 1 --2- 172.21.15.3:0/2967011661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc890159ce0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.440+0000 7fc894983700 1 -- 172.21.15.3:0/2967011661 >> 172.21.15.3:0/2967011661 conn(0x7fc8900ce270 msgr2=0x7fc8900ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.441+0000 7fc894983700 1 -- 172.21.15.3:0/2967011661 shutdown_connections 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.441+0000 7fc894983700 1 -- 172.21.15.3:0/2967011661 wait complete. 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 Processor -- start 2023-10-06T05:01:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 -- start start 2023-10-06T05:01:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc89015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc89015a220 0x7fc8900c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89015bd20 con 0x7fc890157030 2023-10-06T05:01:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89013f570 con 0x7fc89015a220 2023-10-06T05:01:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.442+0000 7fc894983700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89015d030 con 0x7fc890157400 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc89015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc89015c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59990/0 (socket says 172.21.15.3:59990) 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 -- 172.21.15.3:0/1047652469 learned_addr learned my addr 172.21.15.3:0/1047652469 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88dd9b700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88ed9d700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc89015a220 0x7fc8900c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 -- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 msgr2=0x7fc89015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 -- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc89015a220 msgr2=0x7fc8900c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc89015a220 0x7fc8900c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 -- 172.21.15.3:0/1047652469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc888009420 con 0x7fc890157030 2023-10-06T05:01:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc88e59c700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc89015c890 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc8900676c0 tx=0x7fc888017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc8867fc700 1 -- 172.21.15.3:0/1047652469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88800bbe0 con 0x7fc890157030 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc8867fc700 1 -- 172.21.15.3:0/1047652469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888011420 con 0x7fc890157030 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc890151ca0 con 0x7fc890157030 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc8867fc700 1 -- 172.21.15.3:0/1047652469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8880117e0 con 0x7fc890157030 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.443+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89015b760 con 0x7fc890157030 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.444+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8900644c0 con 0x7fc890157030 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.445+0000 7fc8867fc700 1 -- 172.21.15.3:0/1047652469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc888011bd0 con 0x7fc890157030 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.445+0000 7fc8867fc700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc870066f30 0x7fc8700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.445+0000 7fc8867fc700 1 -- 172.21.15.3:0/1047652469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc888085e80 con 0x7fc890157030 2023-10-06T05:01:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.445+0000 7fc88dd9b700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc870066f30 0x7fc8700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.446+0000 7fc88dd9b700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc870066f30 0x7fc8700693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc89015cdd0 tx=0x7fc878009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.449+0000 7fc8867fc700 1 -- 172.21.15.3:0/1047652469 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8880532e0 con 0x7fc890157030 2023-10-06T05:01:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.644+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc890071680 con 0x7fc890157030 2023-10-06T05:01:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.645+0000 7fc8867fc700 1 -- 172.21.15.3:0/1047652469 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13612 (secure 0 0 0) 0x7fc888056900 con 0x7fc890157030 2023-10-06T05:01:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:18.647 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":128,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:01:17.863508+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc870066f30 msgr2=0x7fc8700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc870066f30 0x7fc8700693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc89015cdd0 tx=0x7fc878009040 comp rx=0 tx=0).stop 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 msgr2=0x7fc89015c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc89015c890 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc8900676c0 tx=0x7fc888017000 comp rx=0 tx=0).stop 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 shutdown_connections 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc870066f30 0x7fc8700693e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc89015a220 0x7fc8900c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc890157030 0x7fc89015c890 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 --2- 172.21.15.3:0/1047652469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc890157400 0x7fc89015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 >> 172.21.15.3:0/1047652469 conn(0x7fc8900ce270 msgr2=0x7fc890153980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 shutdown_connections 2023-10-06T05:01:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.647+0000 7fc894983700 1 -- 172.21.15.3:0/1047652469 wait complete. 2023-10-06T05:01:18.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:01:18.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:01:18.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.782+0000 7fd3a759e700 1 Processor -- start 2023-10-06T05:01:18.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.782+0000 7fd3a759e700 1 -- start start 2023-10-06T05:01:18.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a80af290 0x7fd3a80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a759e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a80cb190 con 0x7fd3a80c5c20 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a759e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a8159950 con 0x7fd3a80af660 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80cd6a0 con 0x7fd3a80af290 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a6d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3a80c5c20 0x7fd3a80cbed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45622/0 (socket says 172.21.15.3:45622) 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a6d9d700 1 -- 172.21.15.3:0/4074971956 learned_addr learned my addr 172.21.15.3:0/4074971956 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a659c700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a80af290 0x7fd3a80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095351408 0 0) 0x7fd3a80cb190 con 0x7fd3a80c5c20 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.783+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd394003570 con 0x7fd3a80c5c20 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 229977706 0 0) 0x7fd394003570 con 0x7fd3a80c5c20 2023-10-06T05:01:18.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a80af290 msgr2=0x7fd3a80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a80af290 0x7fd3a80ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 >> v1:172.21.15.153:6789/0 conn(0x7fd3a80af660 legacy=0x7fd3a80ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a80cb3f0 con 0x7fd3a80c5c20 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fd398001810 con 0x7fd3a80c5c20 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 >> v1:172.21.15.3:6789/0 conn(0x7fd3a80c5c20 legacy=0x7fd3a80cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394003a90 0x7fd394003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd394004810 0x7fd394004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a80af290 0x7fd3940051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80cd6a0 con 0x7fd394004810 2023-10-06T05:01:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39400bb60 con 0x7fd394003a90 2023-10-06T05:01:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39400bd20 con 0x7fd3a80af290 2023-10-06T05:01:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a5d9b700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd394004810 0x7fd394004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a659c700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394003a90 0x7fd394003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a5d9b700 1 -- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a80af290 msgr2=0x7fd3940051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a5d9b700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a80af290 0x7fd3940051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a5d9b700 1 -- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394003a90 msgr2=0x7fd394003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a5d9b700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394003a90 0x7fd394003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a5d9b700 1 -- 172.21.15.3:0/4074971956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394003570 con 0x7fd394004810 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a659c700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394003a90 0x7fd394003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a5d9b700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd394004810 0x7fd394004c80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd3900018d0 tx=0x7fd390007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba6015070acc99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390001290 con 0x7fd394004810 2023-10-06T05:01:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd390012420 con 0x7fd394004810 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.784+0000 7fd3a559a700 1 -- 172.21.15.3:0/4074971956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900127e0 con 0x7fd394004810 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.785+0000 7fd3a759e700 1 -- 172.21.15.3:0/4074971956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd394004810 msgr2=0x7fd394004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.785+0000 7fd3a759e700 1 --2- 172.21.15.3:0/4074971956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd394004810 0x7fd394004c80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd3900018d0 tx=0x7fd390007d80 comp rx=0 tx=0).stop 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.785+0000 7fd3a6d9d700 1 -- 172.21.15.3:0/4074971956 reap_dead start 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.785+0000 7fd3a759e700 1 -- 172.21.15.3:0/4074971956 shutdown_connections 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.785+0000 7fd3a759e700 1 -- 172.21.15.3:0/4074971956 >> 172.21.15.3:0/4074971956 conn(0x7fd3a8089dd0 msgr2=0x7fd3a80996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.785+0000 7fd3a759e700 1 -- 172.21.15.3:0/4074971956 shutdown_connections 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.785+0000 7fd3a759e700 1 -- 172.21.15.3:0/4074971956 wait complete. 2023-10-06T05:01:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.786+0000 7fd3a759e700 1 Processor -- start 2023-10-06T05:01:18.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.786+0000 7fd3a759e700 1 -- start start 2023-10-06T05:01:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394004810 0x7fd3a8152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a8151090 0x7fd3a81692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a80b51a0 0x7fd3a80b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80b5b50 con 0x7fd3a80b51a0 2023-10-06T05:01:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a759e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8151500 con 0x7fd394004810 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8169be0 con 0x7fd3a8151090 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a80b51a0 0x7fd3a80b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a80b51a0 0x7fd3a80b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60008/0 (socket says 172.21.15.3:60008) 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 -- 172.21.15.3:0/939408609 learned_addr learned my addr 172.21.15.3:0/939408609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a659c700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394004810 0x7fd3a8152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a5d9b700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a8151090 0x7fd3a81692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 -- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a8151090 msgr2=0x7fd3a81692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a8151090 0x7fd3a81692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 -- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394004810 msgr2=0x7fd3a8152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394004810 0x7fd3a8152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 -- 172.21.15.3:0/939408609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a8169ec0 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a659c700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394004810 0x7fd3a8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.787+0000 7fd3a6d9d700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a80b51a0 0x7fd3a80b5610 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80c6920 tx=0x7fd398007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.788+0000 7fd39e7fc700 1 -- 172.21.15.3:0/939408609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398004dc0 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.788+0000 7fd39e7fc700 1 -- 172.21.15.3:0/939408609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39800a6a0 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.788+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a816a1a0 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.788+0000 7fd39e7fc700 1 -- 172.21.15.3:0/939408609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800aa60 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.788+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a80b1520 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.789+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a80b1740 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.789+0000 7fd39e7fc700 1 -- 172.21.15.3:0/939408609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd398007190 con 0x7fd3a80b51a0 2023-10-06T05:01:18.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.789+0000 7fd39e7fc700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd388066f30 0x7fd3880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:18.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.789+0000 7fd39e7fc700 1 -- 172.21.15.3:0/939408609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd398084260 con 0x7fd3a80b51a0 2023-10-06T05:01:18.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.790+0000 7fd3a659c700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd388066f30 0x7fd3880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:18.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.790+0000 7fd3a659c700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd388066f30 0x7fd3880693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009e30 tx=0x7fd3a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:18.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.794+0000 7fd39e7fc700 1 -- 172.21.15.3:0/939408609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd398051740 con 0x7fd3a80b51a0 2023-10-06T05:01:18.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.986+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3a804c380 con 0x7fd3a80b51a0 2023-10-06T05:01:18.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.987+0000 7fd39e7fc700 1 -- 172.21.15.3:0/939408609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v128) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd398054d60 con 0x7fd3a80b51a0 2023-10-06T05:01:18.988 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T05:01:18.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd388066f30 msgr2=0x7fd3880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd388066f30 0x7fd3880693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009e30 tx=0x7fd3a0008040 comp rx=0 tx=0).stop 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a80b51a0 msgr2=0x7fd3a80b5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a80b51a0 0x7fd3a80b5610 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80c6920 tx=0x7fd398007c70 comp rx=0 tx=0).stop 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 shutdown_connections 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd388066f30 0x7fd3880693e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd394004810 0x7fd3a8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a80b51a0 0x7fd3a80b5610 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 --2- 172.21.15.3:0/939408609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3a8151090 0x7fd3a81692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 >> 172.21.15.3:0/939408609 conn(0x7fd3a8089dd0 msgr2=0x7fd3a80cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:18.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 shutdown_connections 2023-10-06T05:01:18.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:18.989+0000 7fd3a759e700 1 -- 172.21.15.3:0/939408609 wait complete. 2023-10-06T05:01:19.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:01:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.123+0000 7f62810cc700 1 Processor -- start 2023-10-06T05:01:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.123+0000 7f62810cc700 1 -- start start 2023-10-06T05:01:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62810cc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f627c15cae0 0x7f627c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62810cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f627c06e010 0x7f627c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62810cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c0b1e70 con 0x7f627c06d480 2023-10-06T05:01:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62810cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c0b1860 con 0x7f627c06e010 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62810cc700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c06eaf0 con 0x7f627c15cae0 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62808cb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f627c06e010 0x7f627c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f627b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f627c06d480 0x7f627c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45628/0 (socket says 172.21.15.3:45628) 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62808cb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f627c06e010 0x7f627c0b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60020/0 (socket says 172.21.15.3:60020) 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f627b7fe700 1 -- 172.21.15.3:0/1922287345 learned_addr learned my addr 172.21.15.3:0/1922287345 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f627bfff700 1 --2- 172.21.15.3:0/1922287345 >> v2:172.21.15.153:3300/0 conn(0x7f627c15cae0 0x7f627c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62808cb700 1 -- 172.21.15.3:0/1922287345 >> v2:172.21.15.153:3300/0 conn(0x7f627c15cae0 msgr2=0x7f627c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62808cb700 1 --2- 172.21.15.3:0/1922287345 >> v2:172.21.15.153:3300/0 conn(0x7f627c15cae0 0x7f627c0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62808cb700 1 -- 172.21.15.3:0/1922287345 >> v1:172.21.15.3:6789/0 conn(0x7f627c06d480 legacy=0x7f627c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f627affd700 1 -- 172.21.15.3:0/1922287345 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486571495 0 0) 0x7f627c0b1e70 con 0x7f627c06d480 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.124+0000 7f62808cb700 1 -- 172.21.15.3:0/1922287345 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c06e530 con 0x7f627c06e010 2023-10-06T05:01:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62808cb700 1 --2- 172.21.15.3:0/1922287345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0b2300 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6274009e60 tx=0x7f6274009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337f8538a40923f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f627affd700 1 -- 172.21.15.3:0/1922287345 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400e070 con 0x7f627c06e010 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f627affd700 1 -- 172.21.15.3:0/1922287345 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627400bb20 con 0x7f627c06e010 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f627affd700 1 -- 172.21.15.3:0/1922287345 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274012430 con 0x7f627c06e010 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 -- 172.21.15.3:0/1922287345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 msgr2=0x7f627c0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 --2- 172.21.15.3:0/1922287345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0b2300 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6274009e60 tx=0x7f6274009780 comp rx=0 tx=0).stop 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 -- 172.21.15.3:0/1922287345 shutdown_connections 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 --2- 172.21.15.3:0/1922287345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0b2300 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 --2- 172.21.15.3:0/1922287345 >> v2:172.21.15.153:3300/0 conn(0x7f627c15cae0 0x7f627c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 -- 172.21.15.3:0/1922287345 >> 172.21.15.3:0/1922287345 conn(0x7f627c06cb50 msgr2=0x7f627c0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 -- 172.21.15.3:0/1922287345 shutdown_connections 2023-10-06T05:01:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.125+0000 7f62810cc700 1 -- 172.21.15.3:0/1922287345 wait complete. 2023-10-06T05:01:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.126+0000 7f62810cc700 1 Processor -- start 2023-10-06T05:01:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f62810cc700 1 -- start start 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f62810cc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c06d480 0x7f627c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f62810cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f62810cc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627c15cae0 0x7f627c14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f62810cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c16a040 con 0x7f627c06e010 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f62810cc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0997f0 con 0x7f627c06d480 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f62810cc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c14da40 con 0x7f627c15cae0 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.127+0000 7f627b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0c2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60028/0 (socket says 172.21.15.3:60028) 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 -- 172.21.15.3:0/293191789 learned_addr learned my addr 172.21.15.3:0/293191789 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f62808cb700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627c15cae0 0x7f627c14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627bfff700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c06d480 0x7f627c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 -- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627c15cae0 msgr2=0x7f627c14d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627c15cae0 0x7f627c14d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 -- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c06d480 msgr2=0x7f627c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c06d480 0x7f627c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 -- 172.21.15.3:0/293191789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009a20 con 0x7f627c06e010 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f627b7fe700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0c2e30 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f627000c810 tx=0x7f6270007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f6263fff700 1 -- 172.21.15.3:0/293191789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270005b30 con 0x7f627c06e010 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f6263fff700 1 -- 172.21.15.3:0/293191789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6270005c90 con 0x7f627c06e010 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c16a510 con 0x7f627c06e010 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f6263fff700 1 -- 172.21.15.3:0/293191789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270007300 con 0x7f627c06e010 2023-10-06T05:01:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.128+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c0991a0 con 0x7f627c06e010 2023-10-06T05:01:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.129+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c099980 con 0x7f627c06e010 2023-10-06T05:01:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.130+0000 7f6263fff700 1 -- 172.21.15.3:0/293191789 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f627001b400 con 0x7f627c06e010 2023-10-06T05:01:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.130+0000 7f6263fff700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6268066f30 0x7f62680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.130+0000 7f6263fff700 1 -- 172.21.15.3:0/293191789 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6270083ec0 con 0x7f627c06e010 2023-10-06T05:01:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.130+0000 7f627bfff700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6268066f30 0x7f62680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.131+0000 7f627bfff700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6268066f30 0x7f62680693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f626c009c90 tx=0x7f626c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.134+0000 7f6263fff700 1 -- 172.21.15.3:0/293191789 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62700513a0 con 0x7f627c06e010 2023-10-06T05:01:19.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.326+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f627c04c380 con 0x7f627c06e010 2023-10-06T05:01:19.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.327+0000 7f6263fff700 1 -- 172.21.15.3:0/293191789 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62700549c0 con 0x7f627c06e010 2023-10-06T05:01:19.329 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6268066f30 msgr2=0x7f62680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6268066f30 0x7f62680693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f626c009c90 tx=0x7f626c008040 comp rx=0 tx=0).stop 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 msgr2=0x7f627c0c2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0c2e30 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f627000c810 tx=0x7f6270007df0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 shutdown_connections 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6268066f30 0x7f62680693e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c06d480 0x7f627c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f627c15cae0 0x7f627c14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 --2- 172.21.15.3:0/293191789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c06e010 0x7f627c0c2e30 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 >> 172.21.15.3:0/293191789 conn(0x7f627c06cb50 msgr2=0x7f627c0b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 shutdown_connections 2023-10-06T05:01:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.329+0000 7f62810cc700 1 -- 172.21.15.3:0/293191789 wait complete. 2023-10-06T05:01:19.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:01:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.460+0000 7fc1394a8700 1 Processor -- start 2023-10-06T05:01:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.460+0000 7fc1394a8700 1 -- start start 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1394a8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1341591a0 0x7fc1340affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1394a8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 0x7fc1340b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1394a8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340b0a90 con 0x7fc134159570 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1394a8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340b3110 con 0x7fc1341591a0 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1394a8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b33f0 con 0x7fc13415dbd0 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1337fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc134159570 0x7fc13406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45634/0 (socket says 172.21.15.3:45634) 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1337fe700 1 -- 172.21.15.3:0/3329754250 learned_addr learned my addr 172.21.15.3:0/3329754250 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc1327fc700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 0x7fc1340b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc132ffd700 1 --2- 172.21.15.3:0/3329754250 >> v2:172.21.15.153:3300/0 conn(0x7fc1341591a0 0x7fc1340affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640957660 0 0) 0x7fc1340b0a90 con 0x7fc134159570 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.461+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc120003570 con 0x7fc134159570 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338238724 0 0) 0x7fc120003570 con 0x7fc134159570 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 msgr2=0x7fc1340b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 0x7fc1340b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 >> v2:172.21.15.153:3300/0 conn(0x7fc1341591a0 msgr2=0x7fc1340affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 --2- 172.21.15.3:0/3329754250 >> v2:172.21.15.153:3300/0 conn(0x7fc1341591a0 0x7fc1340affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340b36d0 con 0x7fc134159570 2023-10-06T05:01:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc1240017f0 con 0x7fc134159570 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 >> v1:172.21.15.3:6789/0 conn(0x7fc134159570 legacy=0x7fc13406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 0x7fc120003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc120003fe0 0x7fc120004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120004d50 0x7fc1200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b3110 con 0x7fc120004d50 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b33f0 con 0x7fc120003fe0 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12000ba90 con 0x7fc13415dbd0 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1337fe700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120004d50 0x7fc1200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1327fc700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc120003fe0 0x7fc120004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc132ffd700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 0x7fc120003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1337fe700 1 -- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 msgr2=0x7fc120003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1337fe700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc13415dbd0 0x7fc120003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1337fe700 1 -- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc120003fe0 msgr2=0x7fc120004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1337fe700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc120003fe0 0x7fc120004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1337fe700 1 -- 172.21.15.3:0/3329754250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc120003570 con 0x7fc120004d50 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.462+0000 7fc1327fc700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc120003fe0 0x7fc120004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1337fe700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120004d50 0x7fc1200051c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc12400a970 tx=0x7fc124002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fa81bea8d10925 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124012440 con 0x7fc120004d50 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1240125c0 con 0x7fc120004d50 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc131ffb700 1 -- 172.21.15.3:0/3329754250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124012980 con 0x7fc120004d50 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1394a8700 1 -- 172.21.15.3:0/3329754250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120004d50 msgr2=0x7fc1200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1394a8700 1 --2- 172.21.15.3:0/3329754250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120004d50 0x7fc1200051c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc12400a970 tx=0x7fc124002210 comp rx=0 tx=0).stop 2023-10-06T05:01:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1337fe700 1 -- 172.21.15.3:0/3329754250 reap_dead start 2023-10-06T05:01:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1394a8700 1 -- 172.21.15.3:0/3329754250 shutdown_connections 2023-10-06T05:01:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1394a8700 1 -- 172.21.15.3:0/3329754250 >> 172.21.15.3:0/3329754250 conn(0x7fc134070310 msgr2=0x7fc134070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1394a8700 1 -- 172.21.15.3:0/3329754250 shutdown_connections 2023-10-06T05:01:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.463+0000 7fc1394a8700 1 -- 172.21.15.3:0/3329754250 wait complete. 2023-10-06T05:01:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.464+0000 7fc1394a8700 1 Processor -- start 2023-10-06T05:01:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.464+0000 7fc1394a8700 1 -- start start 2023-10-06T05:01:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1394a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340b03f0 0x7fc1340c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1394a8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1340c82f0 0x7fc1341455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1394a8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc134145b30 0x7fc1340a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1394a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340c8110 con 0x7fc1340b03f0 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1394a8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340c9400 con 0x7fc134145b30 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1394a8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b07c0 con 0x7fc1340c82f0 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc132ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340b03f0 0x7fc1340c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc132ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340b03f0 0x7fc1340c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60044/0 (socket says 172.21.15.3:60044) 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc132ffd700 1 -- 172.21.15.3:0/1887874206 learned_addr learned my addr 172.21.15.3:0/1887874206 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1337fe700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc134145b30 0x7fc1340a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc1327fc700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1340c82f0 0x7fc1341455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc132ffd700 1 -- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1340c82f0 msgr2=0x7fc1341455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc132ffd700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1340c82f0 0x7fc1341455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc132ffd700 1 -- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc134145b30 msgr2=0x7fc1340a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.465+0000 7fc132ffd700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc134145b30 0x7fc1340a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc132ffd700 1 -- 172.21.15.3:0/1887874206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc134156a40 con 0x7fc1340b03f0 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc1327fc700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1340c82f0 0x7fc1341455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc132ffd700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340b03f0 0x7fc1340c7bd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006ac0 tx=0x7fc12c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc12affd700 1 -- 172.21.15.3:0/1887874206 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00f9b0 con 0x7fc1340b03f0 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc12affd700 1 -- 172.21.15.3:0/1887874206 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c00fb10 con 0x7fc1340b03f0 2023-10-06T05:01:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1340c87f0 con 0x7fc1340b03f0 2023-10-06T05:01:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc12affd700 1 -- 172.21.15.3:0/1887874206 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00fe90 con 0x7fc1340b03f0 2023-10-06T05:01:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.466+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1340c8db0 con 0x7fc1340b03f0 2023-10-06T05:01:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.467+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13404c380 con 0x7fc1340b03f0 2023-10-06T05:01:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.467+0000 7fc12affd700 1 -- 172.21.15.3:0/1887874206 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc12c014650 con 0x7fc1340b03f0 2023-10-06T05:01:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.467+0000 7fc12affd700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc114066f30 0x7fc1140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.468+0000 7fc12affd700 1 -- 172.21.15.3:0/1887874206 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc12c018030 con 0x7fc1340b03f0 2023-10-06T05:01:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.468+0000 7fc1327fc700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc114066f30 0x7fc1140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.468+0000 7fc1327fc700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc114066f30 0x7fc1140693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0093f0 tx=0x7fc11c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.472+0000 7fc12affd700 1 -- 172.21.15.3:0/1887874206 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc12c052170 con 0x7fc1340b03f0 2023-10-06T05:01:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.668+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc1340b39b0 con 0x7fc1340b03f0 2023-10-06T05:01:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.669+0000 7fc12affd700 1 -- 172.21.15.3:0/1887874206 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc12c022090 con 0x7fc1340b03f0 2023-10-06T05:01:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-10-06T05:01:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:19.672 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-06T05:01:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc114066f30 msgr2=0x7fc1140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc114066f30 0x7fc1140693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0093f0 tx=0x7fc11c008040 comp rx=0 tx=0).stop 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340b03f0 msgr2=0x7fc1340c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340b03f0 0x7fc1340c7bd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006ac0 tx=0x7fc12c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 shutdown_connections 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc114066f30 0x7fc1140693e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc134145b30 0x7fc1340a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340b03f0 0x7fc1340c7bd0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 --2- 172.21.15.3:0/1887874206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1340c82f0 0x7fc1341455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 >> 172.21.15.3:0/1887874206 conn(0x7fc134070310 msgr2=0x7fc134157820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 shutdown_connections 2023-10-06T05:01:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.671+0000 7fc1394a8700 1 -- 172.21.15.3:0/1887874206 wait complete. 2023-10-06T05:01:19.684 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:01:19.685 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:01:19.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.806+0000 7f484f226700 1 Processor -- start 2023-10-06T05:01:19.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.806+0000 7f484f226700 1 -- start start 2023-10-06T05:01:19.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f484f226700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f48480af980 0x7f484806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f484f226700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48480a8b40 0x7f484806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f484f226700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484806e8e0 con 0x7f48480afd50 2023-10-06T05:01:19.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f484f226700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484806f2b0 con 0x7f48480a8b40 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f484f226700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484806dcd0 con 0x7f48480af980 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f4847fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48480a8b40 0x7f484806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f4847fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48480a8b40 0x7f484806d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60054/0 (socket says 172.21.15.3:60054) 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.807+0000 7f4847fff700 1 -- 172.21.15.3:0/1641971570 learned_addr learned my addr 172.21.15.3:0/1641971570 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f484cfc2700 1 --2- 172.21.15.3:0/1641971570 >> v2:172.21.15.153:3300/0 conn(0x7f48480af980 0x7f484806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f4847fff700 1 -- 172.21.15.3:0/1641971570 >> v2:172.21.15.153:3300/0 conn(0x7f48480af980 msgr2=0x7f484806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f4847fff700 1 --2- 172.21.15.3:0/1641971570 >> v2:172.21.15.153:3300/0 conn(0x7f48480af980 0x7f484806ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f4847fff700 1 -- 172.21.15.3:0/1641971570 >> v1:172.21.15.153:6789/0 conn(0x7f48480afd50 legacy=0x7f48480b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f4847fff700 1 -- 172.21.15.3:0/1641971570 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484806dfb0 con 0x7f48480a8b40 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f4847fff700 1 --2- 172.21.15.3:0/1641971570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480a8b40 0x7f484806d480 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f48480713c0 tx=0x7f4838009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd89305f3da05c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f48477fe700 1 -- 172.21.15.3:0/1641971570 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800b820 con 0x7f48480a8b40 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f48477fe700 1 -- 172.21.15.3:0/1641971570 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483800b980 con 0x7f48480a8b40 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f48477fe700 1 -- 172.21.15.3:0/1641971570 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800bd20 con 0x7f48480a8b40 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.808+0000 7f484f226700 1 -- 172.21.15.3:0/1641971570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480a8b40 msgr2=0x7f484806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.809+0000 7f484f226700 1 --2- 172.21.15.3:0/1641971570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480a8b40 0x7f484806d480 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f48480713c0 tx=0x7f4838009250 comp rx=0 tx=0).stop 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.809+0000 7f484f226700 1 -- 172.21.15.3:0/1641971570 shutdown_connections 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.809+0000 7f484f226700 1 --2- 172.21.15.3:0/1641971570 >> v2:172.21.15.153:3300/0 conn(0x7f48480af980 0x7f484806ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.809+0000 7f484f226700 1 --2- 172.21.15.3:0/1641971570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480a8b40 0x7f484806d480 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.809+0000 7f484f226700 1 -- 172.21.15.3:0/1641971570 >> 172.21.15.3:0/1641971570 conn(0x7f484806c370 msgr2=0x7f484806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.809+0000 7f484f226700 1 -- 172.21.15.3:0/1641971570 shutdown_connections 2023-10-06T05:01:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.809+0000 7f484f226700 1 -- 172.21.15.3:0/1641971570 wait complete. 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.810+0000 7f484f226700 1 Processor -- start 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.810+0000 7f484f226700 1 -- start start 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48480a8b40 0x7f4848156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480af980 0x7f484815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48480afd50 0x7f4848099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484815c5c0 con 0x7f48480af980 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848156e40 con 0x7f48480afd50 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4848096f20 con 0x7f48480a8b40 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480af980 0x7f484815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480af980 0x7f484815bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60070/0 (socket says 172.21.15.3:60070) 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 -- 172.21.15.3:0/1398661731 learned_addr learned my addr 172.21.15.3:0/1398661731 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484cfc2700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48480a8b40 0x7f4848156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484d7c3700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48480afd50 0x7f4848099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 -- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48480a8b40 msgr2=0x7f4848156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48480a8b40 0x7f4848156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 -- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48480afd50 msgr2=0x7f4848099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48480afd50 0x7f4848099810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 -- 172.21.15.3:0/1398661731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380094f0 con 0x7f48480af980 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4847fff700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480af980 0x7f484815bad0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4838009290 tx=0x7f4838016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4844ff9700 1 -- 172.21.15.3:0/1398661731 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838011e20 con 0x7f48480af980 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4844ff9700 1 -- 172.21.15.3:0/1398661731 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838020070 con 0x7f48480af980 2023-10-06T05:01:19.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4848097200 con 0x7f48480af980 2023-10-06T05:01:19.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f4844ff9700 1 -- 172.21.15.3:0/1398661731 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483801b6a0 con 0x7f48480af980 2023-10-06T05:01:19.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.811+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48480977c0 con 0x7f48480af980 2023-10-06T05:01:19.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.812+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4848071680 con 0x7f48480af980 2023-10-06T05:01:19.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.813+0000 7f4844ff9700 1 -- 172.21.15.3:0/1398661731 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4838022030 con 0x7f48480af980 2023-10-06T05:01:19.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.813+0000 7f4844ff9700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4828066f30 0x7f48280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:19.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.813+0000 7f4844ff9700 1 -- 172.21.15.3:0/1398661731 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f48380862a0 con 0x7f48480af980 2023-10-06T05:01:19.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.814+0000 7f484cfc2700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4828066f30 0x7f48280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:19.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.814+0000 7f484cfc2700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4828066f30 0x7f48280693e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f484000e5f0 tx=0x7f48400076d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:19.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:19.818+0000 7f4844ff9700 1 -- 172.21.15.3:0/1398661731 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4838053700 con 0x7f48480af980 2023-10-06T05:01:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.013+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4848071680 con 0x7f48480af980 2023-10-06T05:01:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.878+0000 7f4844ff9700 1 -- 172.21.15.3:0/1398661731 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4838056d20 con 0x7f48480af980 2023-10-06T05:01:20.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4828066f30 msgr2=0x7f48280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:20.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4828066f30 0x7f48280693e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f484000e5f0 tx=0x7f48400076d0 comp rx=0 tx=0).stop 2023-10-06T05:01:20.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480af980 msgr2=0x7f484815bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:20.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480af980 0x7f484815bad0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4838009290 tx=0x7f4838016000 comp rx=0 tx=0).stop 2023-10-06T05:01:20.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 shutdown_connections 2023-10-06T05:01:20.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4828066f30 0x7f48280693e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:20.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48480afd50 0x7f4848099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:20.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48480a8b40 0x7f4848156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:20.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 --2- 172.21.15.3:0/1398661731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48480af980 0x7f484815bad0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:20.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.883+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 >> 172.21.15.3:0/1398661731 conn(0x7f484806c370 msgr2=0x7f48481574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:20.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.884+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 shutdown_connections 2023-10-06T05:01:20.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:20.884+0000 7f484f226700 1 -- 172.21.15.3:0/1398661731 wait complete. 2023-10-06T05:01:20.888 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-06T05:01:20.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:01:21.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.032+0000 7fcd551d7700 1 Processor -- start 2023-10-06T05:01:21.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.032+0000 7fcd551d7700 1 -- start start 2023-10-06T05:01:21.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd551d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd500af8a0 0x7fcd50150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:21.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd551d7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015a290 con 0x7fcd50150690 2023-10-06T05:01:21.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd551d7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500d0020 con 0x7fcd500cba50 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd551d7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd50159f40 con 0x7fcd500af8a0 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd4f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd500af8a0 0x7fcd50150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd4f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd500af8a0 0x7fcd50150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60082/0 (socket says 172.21.15.3:60082) 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd4ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcd50150690 0x7fcd5015a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45638/0 (socket says 172.21.15.3:45638) 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.033+0000 7fcd4f59e700 1 -- 172.21.15.3:0/353243989 learned_addr learned my addr 172.21.15.3:0/353243989 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4f59e700 1 -- 172.21.15.3:0/353243989 >> v1:172.21.15.153:6789/0 conn(0x7fcd500cba50 legacy=0x7fcd500af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4f59e700 1 -- 172.21.15.3:0/353243989 >> v1:172.21.15.3:6789/0 conn(0x7fcd50150690 legacy=0x7fcd5015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4f59e700 1 -- 172.21.15.3:0/353243989 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5015dcb0 con 0x7fcd500af8a0 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4dd9b700 1 -- 172.21.15.3:0/353243989 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878919221 0 0) 0x7fcd5015a290 con 0x7fcd50150690 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4f59e700 1 --2- 172.21.15.3:0/353243989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd50150a60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fcd480099b0 tx=0x7fcd48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35a09354ac083a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4dd9b700 1 -- 172.21.15.3:0/353243989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800b820 con 0x7fcd500af8a0 2023-10-06T05:01:21.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4dd9b700 1 -- 172.21.15.3:0/353243989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4800b980 con 0x7fcd500af8a0 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.034+0000 7fcd4dd9b700 1 -- 172.21.15.3:0/353243989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800bd20 con 0x7fcd500af8a0 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.035+0000 7fcd551d7700 1 -- 172.21.15.3:0/353243989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 msgr2=0x7fcd50150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.035+0000 7fcd551d7700 1 --2- 172.21.15.3:0/353243989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd50150a60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fcd480099b0 tx=0x7fcd48009250 comp rx=0 tx=0).stop 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.035+0000 7fcd551d7700 1 -- 172.21.15.3:0/353243989 shutdown_connections 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.035+0000 7fcd551d7700 1 --2- 172.21.15.3:0/353243989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd50150a60 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.035+0000 7fcd551d7700 1 -- 172.21.15.3:0/353243989 >> 172.21.15.3:0/353243989 conn(0x7fcd50070310 msgr2=0x7fcd50070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.035+0000 7fcd551d7700 1 -- 172.21.15.3:0/353243989 shutdown_connections 2023-10-06T05:01:21.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.035+0000 7fcd551d7700 1 -- 172.21.15.3:0/353243989 wait complete. 2023-10-06T05:01:21.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.036+0000 7fcd551d7700 1 Processor -- start 2023-10-06T05:01:21.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.036+0000 7fcd551d7700 1 -- start start 2023-10-06T05:01:21.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd5016aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:21.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd500cba50 0x7fcd50150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:21.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd50150690 0x7fcd50150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c93a0 con 0x7fcd500af8a0 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50153af0 con 0x7fcd500cba50 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c8a70 con 0x7fcd50150690 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd5016aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd5016aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60096/0 (socket says 172.21.15.3:60096) 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 -- 172.21.15.3:0/2789440786 learned_addr learned my addr 172.21.15.3:0/2789440786 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 -- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd50150690 msgr2=0x7fcd50150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd50150690 0x7fcd50150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 -- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd500cba50 msgr2=0x7fcd50150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd500cba50 0x7fcd50150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 -- 172.21.15.3:0/2789440786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd480094f0 con 0x7fcd500af8a0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd4ed9d700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd5016aaf0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000cd30 tx=0x7fcd40001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd46ffd700 1 -- 172.21.15.3:0/2789440786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40005df0 con 0x7fcd500af8a0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd46ffd700 1 -- 172.21.15.3:0/2789440786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4000a960 con 0x7fcd500af8a0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd500c8d50 con 0x7fcd500af8a0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd46ffd700 1 -- 172.21.15.3:0/2789440786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000ad20 con 0x7fcd500af8a0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.037+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5016b030 con 0x7fcd500af8a0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.038+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5004c380 con 0x7fcd500af8a0 2023-10-06T05:01:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.039+0000 7fcd46ffd700 1 -- 172.21.15.3:0/2789440786 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd40007300 con 0x7fcd500af8a0 2023-10-06T05:01:21.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.039+0000 7fcd46ffd700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd30066f30 0x7fcd300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:21.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.040+0000 7fcd46ffd700 1 -- 172.21.15.3:0/2789440786 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fcd40084720 con 0x7fcd500af8a0 2023-10-06T05:01:21.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.040+0000 7fcd4e59c700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd30066f30 0x7fcd300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:21.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.040+0000 7fcd4e59c700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd30066f30 0x7fcd300693e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcd38007fd0 tx=0x7fcd38007b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.044+0000 7fcd46ffd700 1 -- 172.21.15.3:0/2789440786 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd400519f0 con 0x7fcd500af8a0 2023-10-06T05:01:21.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.239+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcd5015df90 con 0x7fcd500af8a0 2023-10-06T05:01:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.887+0000 7fcd46ffd700 1 -- 172.21.15.3:0/2789440786 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcd40055010 con 0x7fcd500af8a0 2023-10-06T05:01:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd30066f30 msgr2=0x7fcd300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd30066f30 0x7fcd300693e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcd38007fd0 tx=0x7fcd38007b20 comp rx=0 tx=0).stop 2023-10-06T05:01:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 msgr2=0x7fcd5016aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd5016aaf0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000cd30 tx=0x7fcd40001af0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 shutdown_connections 2023-10-06T05:01:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcd30066f30 0x7fcd300693e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd50150690 0x7fcd50150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd500af8a0 0x7fcd5016aaf0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 --2- 172.21.15.3:0/2789440786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd500cba50 0x7fcd50150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:21.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.895+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 >> 172.21.15.3:0/2789440786 conn(0x7fcd50070310 msgr2=0x7fcd500cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:21.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.897+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 shutdown_connections 2023-10-06T05:01:21.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:21.898+0000 7fcd551d7700 1 -- 172.21.15.3:0/2789440786 wait complete. 2023-10-06T05:01:21.904 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-06T05:01:21.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:01:22.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.045+0000 7f190359e700 1 Processor -- start 2023-10-06T05:01:22.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.045+0000 7f190359e700 1 -- start start 2023-10-06T05:01:22.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f190359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19040af290 0x7f19040ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:22.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f190359e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19040af660 0x7f19040ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f190359e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19040cb190 con 0x7f19040cd190 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f190359e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1904159950 con 0x7f19040af290 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f190359e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19040cd6a0 con 0x7f19040af660 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f190259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19040af290 0x7f19040ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f1902d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f19040cd190 0x7f19040c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45652/0 (socket says 172.21.15.3:45652) 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f190259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19040af290 0x7f19040ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60108/0 (socket says 172.21.15.3:60108) 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.046+0000 7f1902d9d700 1 -- 172.21.15.3:0/1824588488 learned_addr learned my addr 172.21.15.3:0/1824588488 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190259c700 1 -- 172.21.15.3:0/1824588488 >> v2:172.21.15.153:3300/0 conn(0x7f19040af660 msgr2=0x7f19040ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190259c700 1 --2- 172.21.15.3:0/1824588488 >> v2:172.21.15.153:3300/0 conn(0x7f19040af660 0x7f19040ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190259c700 1 -- 172.21.15.3:0/1824588488 >> v1:172.21.15.3:6789/0 conn(0x7f19040cd190 legacy=0x7f19040c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190259c700 1 -- 172.21.15.3:0/1824588488 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040cb3f0 con 0x7f19040af290 2023-10-06T05:01:22.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190159a700 1 -- 172.21.15.3:0/1824588488 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793192671 0 0) 0x7f19040cb190 con 0x7f19040cd190 2023-10-06T05:01:22.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190259c700 1 --2- 172.21.15.3:0/1824588488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af290 0x7f19040ce1b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f19040d12c0 tx=0x7f18fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2b1a2266d8ba5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:22.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190159a700 1 -- 172.21.15.3:0/1824588488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00e070 con 0x7f19040af290 2023-10-06T05:01:22.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190159a700 1 -- 172.21.15.3:0/1824588488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00bc30 con 0x7f19040af290 2023-10-06T05:01:22.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190159a700 1 -- 172.21.15.3:0/1824588488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc012430 con 0x7f19040af290 2023-10-06T05:01:22.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190359e700 1 -- 172.21.15.3:0/1824588488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af290 msgr2=0x7f19040ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:22.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190359e700 1 --2- 172.21.15.3:0/1824588488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af290 0x7f19040ce1b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f19040d12c0 tx=0x7f18fc009780 comp rx=0 tx=0).stop 2023-10-06T05:01:22.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190359e700 1 -- 172.21.15.3:0/1824588488 shutdown_connections 2023-10-06T05:01:22.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190359e700 1 --2- 172.21.15.3:0/1824588488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af290 0x7f19040ce1b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:22.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190359e700 1 --2- 172.21.15.3:0/1824588488 >> v2:172.21.15.153:3300/0 conn(0x7f19040af660 0x7f19040ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.047+0000 7f190359e700 1 -- 172.21.15.3:0/1824588488 >> 172.21.15.3:0/1824588488 conn(0x7f1904089dd0 msgr2=0x7f19040996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.048+0000 7f190359e700 1 -- 172.21.15.3:0/1824588488 shutdown_connections 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.048+0000 7f190359e700 1 -- 172.21.15.3:0/1824588488 wait complete. 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 Processor -- start 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 -- start start 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19040af290 0x7f19040b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af660 0x7f1904147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19040cd190 0x7f19040d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040c7cb0 con 0x7f19040af660 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040cf130 con 0x7f19040af290 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f190359e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19040cf410 con 0x7f19040cd190 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.049+0000 7f1901d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af660 0x7f1904147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af660 0x7f1904147c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60122/0 (socket says 172.21.15.3:60122) 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 -- 172.21.15.3:0/2178640819 learned_addr learned my addr 172.21.15.3:0/2178640819 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f190259c700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19040af290 0x7f19040b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1902d9d700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19040cd190 0x7f19040d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 -- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19040cd190 msgr2=0x7f19040d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19040cd190 0x7f19040d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 -- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19040af290 msgr2=0x7f19040b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19040af290 0x7f19040b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 -- 172.21.15.3:0/2178640819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc009a20 con 0x7f19040af660 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f190259c700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19040af290 0x7f19040b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f1901d9b700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af660 0x7f1904147c20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b810 tx=0x7f18ec009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f18fa7fc700 1 -- 172.21.15.3:0/2178640819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00f5f0 con 0x7f19040af660 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f18fa7fc700 1 -- 172.21.15.3:0/2178640819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00f770 con 0x7f19040af660 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19040cf6f0 con 0x7f19040af660 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f18fa7fc700 1 -- 172.21.15.3:0/2178640819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00fb30 con 0x7f19040af660 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.050+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19040b20a0 con 0x7f19040af660 2023-10-06T05:01:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.051+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190404c380 con 0x7f19040af660 2023-10-06T05:01:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.052+0000 7f18fa7fc700 1 -- 172.21.15.3:0/2178640819 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18ec010480 con 0x7f19040af660 2023-10-06T05:01:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.052+0000 7f18fa7fc700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f18e4066f30 0x7f18e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.052+0000 7f18fa7fc700 1 -- 172.21.15.3:0/2178640819 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f18ec0845c0 con 0x7f19040af660 2023-10-06T05:01:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.052+0000 7f190259c700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f18e4066f30 0x7f18e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.053+0000 7f190259c700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f18e4066f30 0x7f18e40693e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f19040d12c0 tx=0x7f18fc019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.057+0000 7f18fa7fc700 1 -- 172.21.15.3:0/2178640819 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18ec0517b0 con 0x7f19040af660 2023-10-06T05:01:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.297+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f19040c7e40 con 0x7f19040af660 2023-10-06T05:01:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.896+0000 7f18fa7fc700 1 -- 172.21.15.3:0/2178640819 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v85) v1 ==== 174+0+0 (secure 0 0 0) 0x7f18ec054dd0 con 0x7f19040af660 2023-10-06T05:01:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f18e4066f30 msgr2=0x7f18e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f18e4066f30 0x7f18e40693e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f19040d12c0 tx=0x7f18fc019000 comp rx=0 tx=0).stop 2023-10-06T05:01:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af660 msgr2=0x7f1904147c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af660 0x7f1904147c20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b810 tx=0x7f18ec009220 comp rx=0 tx=0).stop 2023-10-06T05:01:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 shutdown_connections 2023-10-06T05:01:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f18e4066f30 0x7f18e40693e0 secure :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f19040d12c0 tx=0x7f18fc019000 comp rx=0 tx=0).stop 2023-10-06T05:01:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19040af290 0x7f19040b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19040cd190 0x7f19040d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 --2- 172.21.15.3:0/2178640819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040af660 0x7f1904147c20 secure :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b810 tx=0x7f18ec009220 comp rx=0 tx=0).stop 2023-10-06T05:01:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.903+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 >> 172.21.15.3:0/2178640819 conn(0x7f1904089dd0 msgr2=0x7f19040ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.904+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 shutdown_connections 2023-10-06T05:01:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:22.905+0000 7f190359e700 1 -- 172.21.15.3:0/2178640819 wait complete. 2023-10-06T05:01:22.911 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 21 and data pool 22 2023-10-06T05:01:22.927 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:01:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.046+0000 7fac93585700 1 Processor -- start 2023-10-06T05:01:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.046+0000 7fac93585700 1 -- start start 2023-10-06T05:01:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac93585700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac8c15c8f0 0x7fac8c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac93585700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fac8c0d0870 0x7fac8c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac93585700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c0cf6c0 con 0x7fac8c15ccc0 2023-10-06T05:01:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac93585700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8c0cfb50 con 0x7fac8c15c8f0 2023-10-06T05:01:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac93585700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8c15e050 con 0x7fac8c0d0870 2023-10-06T05:01:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac91321700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac8c15c8f0 0x7fac8c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac91321700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac8c15c8f0 0x7fac8c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50346/0 (socket says 172.21.15.3:50346) 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.047+0000 7fac91321700 1 -- 172.21.15.3:0/1240419370 learned_addr learned my addr 172.21.15.3:0/1240419370 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac91321700 1 -- 172.21.15.3:0/1240419370 >> v2:172.21.15.153:3300/0 conn(0x7fac8c0d0870 msgr2=0x7fac8c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac91321700 1 --2- 172.21.15.3:0/1240419370 >> v2:172.21.15.153:3300/0 conn(0x7fac8c0d0870 0x7fac8c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac91321700 1 -- 172.21.15.3:0/1240419370 >> v1:172.21.15.153:6789/0 conn(0x7fac8c15ccc0 legacy=0x7fac8c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac91321700 1 -- 172.21.15.3:0/1240419370 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c0ceab0 con 0x7fac8c15c8f0 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac91321700 1 --2- 172.21.15.3:0/1240419370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c15db10 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0099b0 tx=0x7fac7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4117d9e2057348 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac8bfff700 1 -- 172.21.15.3:0/1240419370 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00b7e0 con 0x7fac8c15c8f0 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac8bfff700 1 -- 172.21.15.3:0/1240419370 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c00b940 con 0x7fac8c15c8f0 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac8bfff700 1 -- 172.21.15.3:0/1240419370 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00bce0 con 0x7fac8c15c8f0 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac93585700 1 -- 172.21.15.3:0/1240419370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 msgr2=0x7fac8c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac93585700 1 --2- 172.21.15.3:0/1240419370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c15db10 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0099b0 tx=0x7fac7c009250 comp rx=0 tx=0).stop 2023-10-06T05:01:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac93585700 1 -- 172.21.15.3:0/1240419370 shutdown_connections 2023-10-06T05:01:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac93585700 1 --2- 172.21.15.3:0/1240419370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c15db10 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac93585700 1 --2- 172.21.15.3:0/1240419370 >> v2:172.21.15.153:3300/0 conn(0x7fac8c0d0870 0x7fac8c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.048+0000 7fac93585700 1 -- 172.21.15.3:0/1240419370 >> 172.21.15.3:0/1240419370 conn(0x7fac8c070310 msgr2=0x7fac8c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.049+0000 7fac93585700 1 -- 172.21.15.3:0/1240419370 shutdown_connections 2023-10-06T05:01:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.049+0000 7fac93585700 1 -- 172.21.15.3:0/1240419370 wait complete. 2023-10-06T05:01:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.050+0000 7fac93585700 1 Processor -- start 2023-10-06T05:01:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.050+0000 7fac93585700 1 -- start start 2023-10-06T05:01:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac93585700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac8c0d0870 0x7fac8c14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac93585700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac93585700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac8c15ccc0 0x7fac8c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac93585700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c14c9c0 con 0x7fac8c15c8f0 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac93585700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c14c7d0 con 0x7fac8c0d0870 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac93585700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c16aae0 con 0x7fac8c15ccc0 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50356/0 (socket says 172.21.15.3:50356) 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 -- 172.21.15.3:0/2512033851 learned_addr learned my addr 172.21.15.3:0/2512033851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac91321700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac8c0d0870 0x7fac8c14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac91b22700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac8c15ccc0 0x7fac8c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 -- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac8c15ccc0 msgr2=0x7fac8c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac8c15ccc0 0x7fac8c0b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 -- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac8c0d0870 msgr2=0x7fac8c14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac8c0d0870 0x7fac8c14c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 -- 172.21.15.3:0/2512033851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c0094f0 con 0x7fac8c15c8f0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac90b20700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c152b00 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fac8000b840 tx=0x7fac800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac897fa700 1 -- 172.21.15.3:0/2512033851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8000f510 con 0x7fac8c15c8f0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac897fa700 1 -- 172.21.15.3:0/2512033851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8000f670 con 0x7fac8c15c8f0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.051+0000 7fac897fa700 1 -- 172.21.15.3:0/2512033851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8000fa10 con 0x7fac8c15c8f0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.052+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8c14cc10 con 0x7fac8c15c8f0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.052+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8c14d1d0 con 0x7fac8c15c8f0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.053+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8c071680 con 0x7fac8c15c8f0 2023-10-06T05:01:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.053+0000 7fac897fa700 1 -- 172.21.15.3:0/2512033851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac8001a400 con 0x7fac8c15c8f0 2023-10-06T05:01:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.053+0000 7fac897fa700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac78066f60 0x7fac78069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.053+0000 7fac897fa700 1 -- 172.21.15.3:0/2512033851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fac80084170 con 0x7fac8c15c8f0 2023-10-06T05:01:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.054+0000 7fac91321700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac78066f60 0x7fac78069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.054+0000 7fac91321700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac78066f60 0x7fac78069410 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0099b0 tx=0x7fac7c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.058+0000 7fac897fa700 1 -- 172.21.15.3:0/2512033851 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac80054d90 con 0x7fac8c15c8f0 2023-10-06T05:01:23.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.254+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac8c0b35a0 con 0x7fac8c15c8f0 2023-10-06T05:01:23.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.255+0000 7fac897fa700 1 -- 172.21.15.3:0/2512033851 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16845 (secure 0 0 0) 0x7fac80054920 con 0x7fac8c15c8f0 2023-10-06T05:01:23.256 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:23.257 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":131,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:01:22.298778+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:01:20.044822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:01:21.270200+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac78066f60 msgr2=0x7fac78069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac78066f60 0x7fac78069410 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0099b0 tx=0x7fac7c016000 comp rx=0 tx=0).stop 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 msgr2=0x7fac8c152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c152b00 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fac8000b840 tx=0x7fac800091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 shutdown_connections 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fac78066f60 0x7fac78069410 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac8c0d0870 0x7fac8c14c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fac8c15ccc0 0x7fac8c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 --2- 172.21.15.3:0/2512033851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac8c15c8f0 0x7fac8c152b00 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 >> 172.21.15.3:0/2512033851 conn(0x7fac8c070310 msgr2=0x7fac8c155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:23.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 shutdown_connections 2023-10-06T05:01:23.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.257+0000 7fac93585700 1 -- 172.21.15.3:0/2512033851 wait complete. 2023-10-06T05:01:23.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:01:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.389+0000 7fed31acf700 1 Processor -- start 2023-10-06T05:01:23.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.389+0000 7fed31acf700 1 -- start start 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.390+0000 7fed31acf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 0x7fed2c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.390+0000 7fed31acf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c0cf6c0 con 0x7fed2c15c8f0 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.390+0000 7fed31acf700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c0cfb50 con 0x7fed2c0d06a0 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.390+0000 7fed31acf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0d0b20 con 0x7fed2c15ccc0 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.390+0000 7fed2b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed2c15c8f0 0x7fed2c15db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40158/0 (socket says 172.21.15.3:40158) 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.390+0000 7fed2b7fe700 1 -- 172.21.15.3:0/2392489 learned_addr learned my addr 172.21.15.3:0/2392489 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2affd700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 0x7fed2c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202532974 0 0) 0x7fed2c0cf6c0 con 0x7fed2c15c8f0 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed14003570 con 0x7fed2c15c8f0 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 907065100 0 0) 0x7fed14003570 con 0x7fed2c15c8f0 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 msgr2=0x7fed2c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 0x7fed2c0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 >> v1:172.21.15.153:6789/0 conn(0x7fed2c0d06a0 legacy=0x7fed2c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0ceab0 con 0x7fed2c15c8f0 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fed1c001470 con 0x7fed2c15c8f0 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 >> v1:172.21.15.3:6789/0 conn(0x7fed2c15c8f0 legacy=0x7fed2c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 0x7fed14003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed14003fd0 0x7fed14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed14004d50 0x7fed140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0d0b20 con 0x7fed14004d50 2023-10-06T05:01:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1400bb40 con 0x7fed14003fd0 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1400bd00 con 0x7fed2c15ccc0 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2bfff700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed14004d50 0x7fed140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2b7fe700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 0x7fed14003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2affd700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed14003fd0 0x7fed14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2bfff700 1 -- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 msgr2=0x7fed14003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2bfff700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c15ccc0 0x7fed14003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2bfff700 1 -- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed14003fd0 msgr2=0x7fed14004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2bfff700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed14003fd0 0x7fed14004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.391+0000 7fed2bfff700 1 -- 172.21.15.3:0/2392489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed14003570 con 0x7fed14004d50 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed2bfff700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed14004d50 0x7fed140051c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fed240020a0 tx=0x7fed240016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e94677fc7ded935 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240019e0 con 0x7fed14004d50 2023-10-06T05:01:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2400c8b0 con 0x7fed14004d50 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed2a7fc700 1 -- 172.21.15.3:0/2392489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2400cc50 con 0x7fed14004d50 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed31acf700 1 -- 172.21.15.3:0/2392489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed14004d50 msgr2=0x7fed140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed31acf700 1 --2- 172.21.15.3:0/2392489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed14004d50 0x7fed140051c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fed240020a0 tx=0x7fed240016d0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed2bfff700 1 -- 172.21.15.3:0/2392489 reap_dead start 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed31acf700 1 -- 172.21.15.3:0/2392489 shutdown_connections 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed31acf700 1 -- 172.21.15.3:0/2392489 >> 172.21.15.3:0/2392489 conn(0x7fed2c070310 msgr2=0x7fed2c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed31acf700 1 -- 172.21.15.3:0/2392489 shutdown_connections 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.392+0000 7fed31acf700 1 -- 172.21.15.3:0/2392489 wait complete. 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.393+0000 7fed31acf700 1 Processor -- start 2023-10-06T05:01:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed31acf700 1 -- start start 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed31acf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2c13a3a0 0x7fed2c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed31acf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c13a770 0x7fed2c152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed31acf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed2c15f1b0 0x7fed2c15f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed31acf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c13ac80 con 0x7fed2c13a3a0 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed31acf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c15fb70 con 0x7fed2c15f1b0 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed31acf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c14f680 con 0x7fed2c13a770 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed2b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2c13a3a0 0x7fed2c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed2b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2c13a3a0 0x7fed2c151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50374/0 (socket says 172.21.15.3:50374) 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.394+0000 7fed2b7fe700 1 -- 172.21.15.3:0/1907096776 learned_addr learned my addr 172.21.15.3:0/1907096776 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2bfff700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed2c15f1b0 0x7fed2c15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2affd700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c13a770 0x7fed2c152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2b7fe700 1 -- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c13a770 msgr2=0x7fed2c152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2b7fe700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c13a770 0x7fed2c152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2b7fe700 1 -- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed2c15f1b0 msgr2=0x7fed2c15f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2b7fe700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed2c15f1b0 0x7fed2c15f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2b7fe700 1 -- 172.21.15.3:0/1907096776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c1504e0 con 0x7fed2c13a3a0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed2b7fe700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2c13a3a0 0x7fed2c151ce0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fed1c007390 tx=0x7fed1c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed137fe700 1 -- 172.21.15.3:0/1907096776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00f870 con 0x7fed2c13a3a0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed137fe700 1 -- 172.21.15.3:0/1907096776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c00f9d0 con 0x7fed2c13a3a0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c14f8d0 con 0x7fed2c13a3a0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed137fe700 1 -- 172.21.15.3:0/1907096776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00fd70 con 0x7fed2c13a3a0 2023-10-06T05:01:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.395+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c14fe90 con 0x7fed2c13a3a0 2023-10-06T05:01:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.396+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2c071680 con 0x7fed2c13a3a0 2023-10-06T05:01:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.396+0000 7fed137fe700 1 -- 172.21.15.3:0/1907096776 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed1c013600 con 0x7fed2c13a3a0 2023-10-06T05:01:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.397+0000 7fed137fe700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed18066f30 0x7fed180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.397+0000 7fed137fe700 1 -- 172.21.15.3:0/1907096776 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fed1c0847c0 con 0x7fed2c13a3a0 2023-10-06T05:01:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.397+0000 7fed2affd700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed18066f30 0x7fed180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.397+0000 7fed2affd700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed18066f30 0x7fed180693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fed2c155220 tx=0x7fed20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.401+0000 7fed137fe700 1 -- 172.21.15.3:0/1907096776 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed1c0518d0 con 0x7fed2c13a3a0 2023-10-06T05:01:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.599+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed2c0b35a0 con 0x7fed2c13a3a0 2023-10-06T05:01:23.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.600+0000 7fed137fe700 1 -- 172.21.15.3:0/1907096776 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16845 (secure 0 0 0) 0x7fed1c054ef0 con 0x7fed2c13a3a0 2023-10-06T05:01:23.601 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:23.602 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":131,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:01:22.298778+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:01:20.044822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:01:21.270200+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed18066f30 msgr2=0x7fed180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed18066f30 0x7fed180693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fed2c155220 tx=0x7fed20008040 comp rx=0 tx=0).stop 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2c13a3a0 msgr2=0x7fed2c151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2c13a3a0 0x7fed2c151ce0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fed1c007390 tx=0x7fed1c001e70 comp rx=0 tx=0).stop 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 shutdown_connections 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fed18066f30 0x7fed180693e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed2c15f1b0 0x7fed2c15f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2c13a3a0 0x7fed2c151ce0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 --2- 172.21.15.3:0/1907096776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed2c13a770 0x7fed2c152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 >> 172.21.15.3:0/1907096776 conn(0x7fed2c070310 msgr2=0x7fed2c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:23.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 shutdown_connections 2023-10-06T05:01:23.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.602+0000 7fed31acf700 1 -- 172.21.15.3:0/1907096776 wait complete. 2023-10-06T05:01:23.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:01:23.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.736+0000 7f7f58849700 1 Processor -- start 2023-10-06T05:01:23.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.736+0000 7f7f58849700 1 -- start start 2023-10-06T05:01:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f58849700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f540af290 0x7f7f540ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f58849700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7f540c5c20 0x7f7f540c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f58849700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f540cb190 con 0x7f7f540af660 2023-10-06T05:01:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f58849700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f540c65a0 con 0x7f7f540af290 2023-10-06T05:01:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f58849700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f54159950 con 0x7f7f540c5c20 2023-10-06T05:01:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f52d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f540af290 0x7f7f540ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f52d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f540af290 0x7f7f540ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50388/0 (socket says 172.21.15.3:50388) 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f52d9d700 1 -- 172.21.15.3:0/4072878930 learned_addr learned my addr 172.21.15.3:0/4072878930 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f5359e700 1 --2- 172.21.15.3:0/4072878930 >> v2:172.21.15.153:3300/0 conn(0x7f7f540c5c20 0x7f7f540c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f52d9d700 1 -- 172.21.15.3:0/4072878930 >> v2:172.21.15.153:3300/0 conn(0x7f7f540c5c20 msgr2=0x7f7f540c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f52d9d700 1 --2- 172.21.15.3:0/4072878930 >> v2:172.21.15.153:3300/0 conn(0x7f7f540c5c20 0x7f7f540c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f52d9d700 1 -- 172.21.15.3:0/4072878930 >> v1:172.21.15.153:6789/0 conn(0x7f7f540af660 legacy=0x7f7f540ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.737+0000 7f7f52d9d700 1 -- 172.21.15.3:0/4072878930 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540cd6a0 con 0x7f7f540af290 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f52d9d700 1 --2- 172.21.15.3:0/4072878930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af290 0x7f7f540ce1b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7f540c6920 tx=0x7f7f44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b46230be668fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f51d9b700 1 -- 172.21.15.3:0/4072878930 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400e070 con 0x7f7f540af290 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f51d9b700 1 -- 172.21.15.3:0/4072878930 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4400ba90 con 0x7f7f540af290 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f51d9b700 1 -- 172.21.15.3:0/4072878930 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f440092e0 con 0x7f7f540af290 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 -- 172.21.15.3:0/4072878930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af290 msgr2=0x7f7f540ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 --2- 172.21.15.3:0/4072878930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af290 0x7f7f540ce1b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7f540c6920 tx=0x7f7f44009840 comp rx=0 tx=0).stop 2023-10-06T05:01:23.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 -- 172.21.15.3:0/4072878930 shutdown_connections 2023-10-06T05:01:23.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 --2- 172.21.15.3:0/4072878930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af290 0x7f7f540ce1b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 --2- 172.21.15.3:0/4072878930 >> v2:172.21.15.153:3300/0 conn(0x7f7f540c5c20 0x7f7f540c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 -- 172.21.15.3:0/4072878930 >> 172.21.15.3:0/4072878930 conn(0x7f7f54089dd0 msgr2=0x7f7f540996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:23.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 -- 172.21.15.3:0/4072878930 shutdown_connections 2023-10-06T05:01:23.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.738+0000 7f7f58849700 1 -- 172.21.15.3:0/4072878930 wait complete. 2023-10-06T05:01:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.739+0000 7f7f58849700 1 Processor -- start 2023-10-06T05:01:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.739+0000 7f7f58849700 1 -- start start 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f58849700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f540af290 0x7f7f540cff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f58849700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af660 0x7f7f54157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f58849700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f540c5c20 0x7f7f54157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f58849700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540c7b50 con 0x7f7f540af660 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f58849700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540d0570 con 0x7f7f540c5c20 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f58849700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540c7500 con 0x7f7f540af290 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f5259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af660 0x7f7f54157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f5259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af660 0x7f7f54157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50396/0 (socket says 172.21.15.3:50396) 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f5259c700 1 -- 172.21.15.3:0/2125777173 learned_addr learned my addr 172.21.15.3:0/2125777173 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f52d9d700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f540af290 0x7f7f540cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.740+0000 7f7f5359e700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f540c5c20 0x7f7f54157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f5259c700 1 -- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f540af290 msgr2=0x7f7f540cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f5259c700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f540af290 0x7f7f540cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f5259c700 1 -- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f540c5c20 msgr2=0x7f7f54157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f5259c700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f540c5c20 0x7f7f54157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f5259c700 1 -- 172.21.15.3:0/2125777173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f44009ae0 con 0x7f7f540af660 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f5259c700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af660 0x7f7f54157b40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c510 tx=0x7f7f3c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f4affd700 1 -- 172.21.15.3:0/2125777173 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c011590 con 0x7f7f540af660 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f4affd700 1 -- 172.21.15.3:0/2125777173 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c011710 con 0x7f7f540af660 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f540c77e0 con 0x7f7f540af660 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f4affd700 1 -- 172.21.15.3:0/2125777173 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c011ad0 con 0x7f7f540af660 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.741+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f54158330 con 0x7f7f540af660 2023-10-06T05:01:23.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.742+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f540c7d00 con 0x7f7f540af660 2023-10-06T05:01:23.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.742+0000 7f7f4affd700 1 -- 172.21.15.3:0/2125777173 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f3c012420 con 0x7f7f540af660 2023-10-06T05:01:23.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.743+0000 7f7f4affd700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:23.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.743+0000 7f7f4affd700 1 -- 172.21.15.3:0/2125777173 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f3c083ff0 con 0x7f7f540af660 2023-10-06T05:01:23.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.743+0000 7f7f52d9d700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:23.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.744+0000 7f7f52d9d700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f34066f30 0x7f7f340693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009880 tx=0x7f7f44017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:23.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.747+0000 7f7f4affd700 1 -- 172.21.15.3:0/2125777173 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f540c7d00 con 0x7f7f540af660 2023-10-06T05:01:23.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:23.988+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7f540c7340 con 0x7f7f540af660 2023-10-06T05:01:24.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.911+0000 7f7f4affd700 1 -- 172.21.15.3:0/2125777173 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7f3c051100 con 0x7f7f540af660 2023-10-06T05:01:24.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f34066f30 msgr2=0x7f7f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f34066f30 0x7f7f340693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009880 tx=0x7f7f44017000 comp rx=0 tx=0).stop 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af660 msgr2=0x7f7f54157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af660 0x7f7f54157b40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c510 tx=0x7f7f3c00c820 comp rx=0 tx=0).stop 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 shutdown_connections 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f540c5c20 0x7f7f54157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f540af290 0x7f7f540cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 --2- 172.21.15.3:0/2125777173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f540af660 0x7f7f54157b40 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 >> 172.21.15.3:0/2125777173 conn(0x7f7f54089dd0 msgr2=0x7f7f540acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 shutdown_connections 2023-10-06T05:01:24.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:24.914+0000 7f7f58849700 1 -- 172.21.15.3:0/2125777173 wait complete. 2023-10-06T05:01:24.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:01:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.057+0000 7f155dbfd700 1 Processor -- start 2023-10-06T05:01:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.057+0000 7f155dbfd700 1 -- start start 2023-10-06T05:01:25.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f155dbfd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1558067dc0 0x7f15580b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:25.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f155dbfd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f155815c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:25.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f155dbfd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1558068190 0x7f155815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:25.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f155dbfd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15580691c0 con 0x7f1558068190 2023-10-06T05:01:25.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f155dbfd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15580694a0 con 0x7f1558067dc0 2023-10-06T05:01:25.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f155dbfd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1558069780 con 0x7f15580b1b00 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1558068190 0x7f155815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1558068190 0x7f155815cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50400/0 (socket says 172.21.15.3:50400) 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 -- 172.21.15.3:0/4282345149 learned_addr learned my addr 172.21.15.3:0/4282345149 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1556ffd700 1 --2- 172.21.15.3:0/4282345149 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f155815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 -- 172.21.15.3:0/4282345149 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 msgr2=0x7f155815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 --2- 172.21.15.3:0/4282345149 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f155815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 -- 172.21.15.3:0/4282345149 >> v2:172.21.15.153:3300/0 conn(0x7f1558067dc0 msgr2=0x7f15580b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 --2- 172.21.15.3:0/4282345149 >> v2:172.21.15.153:3300/0 conn(0x7f1558067dc0 0x7f15580b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 -- 172.21.15.3:0/4282345149 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1558069a60 con 0x7f1558068190 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.058+0000 7f1557fff700 1 --2- 172.21.15.3:0/4282345149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f155815cd40 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f155806b5f0 tx=0x7f1550009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca7bd369e524322 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f15567fc700 1 -- 172.21.15.3:0/4282345149 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000e070 con 0x7f1558068190 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f15567fc700 1 -- 172.21.15.3:0/4282345149 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155000ba50 con 0x7f1558068190 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f15567fc700 1 -- 172.21.15.3:0/4282345149 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15500092e0 con 0x7f1558068190 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 -- 172.21.15.3:0/4282345149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 msgr2=0x7f155815cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 --2- 172.21.15.3:0/4282345149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f155815cd40 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f155806b5f0 tx=0x7f1550009840 comp rx=0 tx=0).stop 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 -- 172.21.15.3:0/4282345149 shutdown_connections 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 --2- 172.21.15.3:0/4282345149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f155815cd40 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 --2- 172.21.15.3:0/4282345149 >> v2:172.21.15.153:3300/0 conn(0x7f1558067dc0 0x7f15580b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 --2- 172.21.15.3:0/4282345149 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f155815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 -- 172.21.15.3:0/4282345149 >> 172.21.15.3:0/4282345149 conn(0x7f15580ce270 msgr2=0x7f15580ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 -- 172.21.15.3:0/4282345149 shutdown_connections 2023-10-06T05:01:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.059+0000 7f155dbfd700 1 -- 172.21.15.3:0/4282345149 wait complete. 2023-10-06T05:01:25.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.060+0000 7f155dbfd700 1 Processor -- start 2023-10-06T05:01:25.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.060+0000 7f155dbfd700 1 -- start start 2023-10-06T05:01:25.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f155dbfd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558067dc0 0x7f155813c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:25.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f155dbfd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f15581518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:25.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f155dbfd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f1558152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f155dbfd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155816d4c0 con 0x7f1558068190 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f155dbfd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15580aeb20 con 0x7f1558067dc0 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f155dbfd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15580aee00 con 0x7f15580b1b00 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f15581518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f15581518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50408/0 (socket says 172.21.15.3:50408) 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f1556ffd700 1 -- 172.21.15.3:0/2181070291 learned_addr learned my addr 172.21.15.3:0/2181070291 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f15577fe700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558067dc0 0x7f155813c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.061+0000 7f1557fff700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f1558152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f1556ffd700 1 -- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 msgr2=0x7f1558152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f1556ffd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f1558152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f1556ffd700 1 -- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558067dc0 msgr2=0x7f155813c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f1556ffd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558067dc0 0x7f155813c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f1556ffd700 1 -- 172.21.15.3:0/2181070291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550009ae0 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f1556ffd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f15581518d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f154c00d860 tx=0x7f154c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f153f7fe700 1 -- 172.21.15.3:0/2181070291 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c0125f0 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f153f7fe700 1 -- 172.21.15.3:0/2181070291 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c012750 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15580af0e0 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f153f7fe700 1 -- 172.21.15.3:0/2181070291 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c012b10 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.062+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15580af3c0 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.063+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155816d630 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.063+0000 7f153f7fe700 1 -- 172.21.15.3:0/2181070291 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f154c01e460 con 0x7f1558068190 2023-10-06T05:01:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.064+0000 7f153f7fe700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.064+0000 7f153f7fe700 1 -- 172.21.15.3:0/2181070291 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f154c084bc0 con 0x7f1558068190 2023-10-06T05:01:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.064+0000 7f15577fe700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.065+0000 7f15577fe700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1544066f30 0x7f15440693e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f155816e3e0 tx=0x7f15480091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.068+0000 7f153f7fe700 1 -- 172.21.15.3:0/2181070291 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f155816d630 con 0x7f1558068190 2023-10-06T05:01:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.307+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f155816d630 con 0x7f1558068190 2023-10-06T05:01:25.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:25.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:25.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.917+0000 7f153f7fe700 1 -- 172.21.15.3:0/2181070291 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) v1 ==== 122+0+0 (secure 0 0 0) 0x7f155816d630 con 0x7f1558068190 2023-10-06T05:01:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1544066f30 msgr2=0x7f15440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1544066f30 0x7f15440693e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f155816e3e0 tx=0x7f15480091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 msgr2=0x7f15581518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f15581518d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f154c00d860 tx=0x7f154c008040 comp rx=0 tx=0).stop 2023-10-06T05:01:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 shutdown_connections 2023-10-06T05:01:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558067dc0 0x7f155813c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15580b1b00 0x7f1558152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 --2- 172.21.15.3:0/2181070291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1558068190 0x7f15581518d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 >> 172.21.15.3:0/2181070291 conn(0x7f15580ce270 msgr2=0x7f15580ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 shutdown_connections 2023-10-06T05:01:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:25.919+0000 7f155dbfd700 1 -- 172.21.15.3:0/2181070291 wait complete. 2023-10-06T05:01:25.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:26.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.053+0000 7f4a91fab700 1 Processor -- start 2023-10-06T05:01:26.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.053+0000 7f4a91fab700 1 -- start start 2023-10-06T05:01:26.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a91fab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a8c159500 0x7f4a8c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a91fab700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4a8c1598d0 0x7f4a8c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a91fab700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c0cc5c0 con 0x7f4a8c0b2220 2023-10-06T05:01:26.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a91fab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c0cb7b0 con 0x7f4a8c159500 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a91fab700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c0cda60 con 0x7f4a8c1598d0 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a8c159500 0x7f4a8c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a8c159500 0x7f4a8c0cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50424/0 (socket says 172.21.15.3:50424) 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4a8c0b2220 0x7f4a8c0b2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40166/0 (socket says 172.21.15.3:40166) 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/3927356629 learned_addr learned my addr 172.21.15.3:0/3927356629 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/3927356629 >> v2:172.21.15.153:3300/0 conn(0x7f4a8c1598d0 msgr2=0x7f4a8c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8affd700 1 --2- 172.21.15.3:0/3927356629 >> v2:172.21.15.153:3300/0 conn(0x7f4a8c1598d0 0x7f4a8c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8b7fe700 1 --2- 172.21.15.3:0/3927356629 >> v2:172.21.15.153:3300/0 conn(0x7f4a8c1598d0 0x7f4a8c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8a7fc700 1 -- 172.21.15.3:0/3927356629 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187088419 0 0) 0x7f4a8c0cc5c0 con 0x7f4a8c0b2220 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/3927356629 >> v1:172.21.15.3:6789/0 conn(0x7f4a8c0b2220 legacy=0x7f4a8c0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/3927356629 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0cb9b0 con 0x7f4a8c159500 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.054+0000 7f4a8affd700 1 --2- 172.21.15.3:0/3927356629 >> v2:172.21.15.153:3300/0 conn(0x7f4a8c1598d0 0x7f4a8c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a8b7fe700 1 --2- 172.21.15.3:0/3927356629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c159500 0x7f4a8c0cad30 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c15d130 tx=0x7f4a7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d862e59f282366c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a8a7fc700 1 -- 172.21.15.3:0/3927356629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00b860 con 0x7f4a8c159500 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a8a7fc700 1 -- 172.21.15.3:0/3927356629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c00b9c0 con 0x7f4a8c159500 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a8a7fc700 1 -- 172.21.15.3:0/3927356629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00bd60 con 0x7f4a8c159500 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 -- 172.21.15.3:0/3927356629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c159500 msgr2=0x7f4a8c0cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 --2- 172.21.15.3:0/3927356629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c159500 0x7f4a8c0cad30 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c15d130 tx=0x7f4a7c009250 comp rx=0 tx=0).stop 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 -- 172.21.15.3:0/3927356629 shutdown_connections 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 --2- 172.21.15.3:0/3927356629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c159500 0x7f4a8c0cad30 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 --2- 172.21.15.3:0/3927356629 >> v2:172.21.15.153:3300/0 conn(0x7f4a8c1598d0 0x7f4a8c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 -- 172.21.15.3:0/3927356629 >> 172.21.15.3:0/3927356629 conn(0x7f4a8c070310 msgr2=0x7f4a8c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 -- 172.21.15.3:0/3927356629 shutdown_connections 2023-10-06T05:01:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.055+0000 7f4a91fab700 1 -- 172.21.15.3:0/3927356629 wait complete. 2023-10-06T05:01:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 Processor -- start 2023-10-06T05:01:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 -- start start 2023-10-06T05:01:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c0b2220 0x7f4a8c0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a8c159500 0x7f4a8c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a8c1598d0 0x7f4a8c14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0c9d20 con 0x7f4a8c0b2220 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0b60a0 con 0x7f4a8c1598d0 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a91fab700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c15b410 con 0x7f4a8c159500 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a8b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c0b2220 0x7f4a8c0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a8b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c0b2220 0x7f4a8c0c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50434/0 (socket says 172.21.15.3:50434) 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.057+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/2068200077 learned_addr learned my addr 172.21.15.3:0/2068200077 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a8bfff700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a8c1598d0 0x7f4a8c14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a8c159500 msgr2=0x7f4a8c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a8b7fe700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a8c159500 0x7f4a8c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a8c1598d0 msgr2=0x7f4a8c14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a8b7fe700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a8c1598d0 0x7f4a8c14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a8b7fe700 1 -- 172.21.15.3:0/2068200077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c0094f0 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a8b7fe700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c0b2220 0x7f4a8c0c7440 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c06ea80 tx=0x7f4a7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a737fe700 1 -- 172.21.15.3:0/2068200077 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c026450 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a737fe700 1 -- 172.21.15.3:0/2068200077 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c0265b0 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c15b9d0 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a737fe700 1 -- 172.21.15.3:0/2068200077 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c026950 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.058+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c0ca210 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.059+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c071680 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.059+0000 7f4a737fe700 1 -- 172.21.15.3:0/2068200077 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a7c01a420 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.060+0000 7f4a737fe700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a78066f30 0x7f4a780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.060+0000 7f4a737fe700 1 -- 172.21.15.3:0/2068200077 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4a7c0864d0 con 0x7f4a8c0b2220 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.060+0000 7f4a8affd700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a78066f30 0x7f4a780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.061+0000 7f4a8affd700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a78066f30 0x7f4a780693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4a80006fd0 tx=0x7f4a80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.064+0000 7f4a737fe700 1 -- 172.21.15.3:0/2068200077 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a7c053660 con 0x7f4a8c0b2220 2023-10-06T05:01:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.308+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a8c0cab10 con 0x7f4a8c0b2220 2023-10-06T05:01:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.308+0000 7f4a737fe700 1 -- 172.21.15.3:0/2068200077 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f4a7c056c80 con 0x7f4a8c0b2220 2023-10-06T05:01:26.310 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:26.310 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:01:25.910940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a78066f30 msgr2=0x7f4a780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a78066f30 0x7f4a780693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4a80006fd0 tx=0x7f4a80008040 comp rx=0 tx=0).stop 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c0b2220 msgr2=0x7f4a8c0c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c0b2220 0x7f4a8c0c7440 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c06ea80 tx=0x7f4a7c017000 comp rx=0 tx=0).stop 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 shutdown_connections 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4a78066f30 0x7f4a780693e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a8c1598d0 0x7f4a8c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a8c0b2220 0x7f4a8c0c7440 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 --2- 172.21.15.3:0/2068200077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a8c159500 0x7f4a8c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 >> 172.21.15.3:0/2068200077 conn(0x7f4a8c070310 msgr2=0x7f4a8c150920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 shutdown_connections 2023-10-06T05:01:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.310+0000 7f4a91fab700 1 -- 172.21.15.3:0/2068200077 wait complete. 2023-10-06T05:01:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 89 2023-10-06T05:01:26.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:01:26.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.454+0000 7f833f59e700 1 Processor -- start 2023-10-06T05:01:26.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.454+0000 7f833f59e700 1 -- start start 2023-10-06T05:01:26.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8340153c30 0x7f8340157310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8340154000 0x7f8340159de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833f59e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400c9b20 con 0x7f83400ae9b0 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834015a550 con 0x7f8340154000 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834015a830 con 0x7f8340153c30 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833e59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8340153c30 0x7f8340157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8340154000 0x7f8340159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8340154000 0x7f8340159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:42298/0 (socket says 172.21.15.3:42298) 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833ed9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f83400ae9b0 0x7f83400aed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:42064/0 (socket says 172.21.15.3:42064) 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833dd9b700 1 -- 172.21.15.3:0/1111878903 learned_addr learned my addr 172.21.15.3:0/1111878903 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.455+0000 7f833e59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8340153c30 0x7f8340157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:37944/0 (socket says 172.21.15.3:37944) 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833e59c700 1 -- 172.21.15.3:0/1111878903 >> v2:172.21.15.153:3300/0 conn(0x7f8340154000 msgr2=0x7f8340159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:26.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833e59c700 1 --2- 172.21.15.3:0/1111878903 >> v2:172.21.15.153:3300/0 conn(0x7f8340154000 0x7f8340159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833e59c700 1 -- 172.21.15.3:0/1111878903 >> v1:172.21.15.153:6789/0 conn(0x7f83400ae9b0 legacy=0x7f83400aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833e59c700 1 -- 172.21.15.3:0/1111878903 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834015a320 con 0x7f8340153c30 2023-10-06T05:01:26.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833dd9b700 1 --2- 172.21.15.3:0/1111878903 >> v2:172.21.15.153:3300/0 conn(0x7f8340154000 0x7f8340159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:26.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833e59c700 1 --2- 172.21.15.3:0/1111878903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8340153c30 0x7f8340157310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f83380098e0 tx=0x7f83380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d22d833a0bcda0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833d59a700 1 -- 172.21.15.3:0/1111878903 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833800b6d0 con 0x7f8340153c30 2023-10-06T05:01:26.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833d59a700 1 -- 172.21.15.3:0/1111878903 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833800b830 con 0x7f8340153c30 2023-10-06T05:01:26.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.456+0000 7f833d59a700 1 -- 172.21.15.3:0/1111878903 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833800bbd0 con 0x7f8340153c30 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 -- 172.21.15.3:0/1111878903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8340153c30 msgr2=0x7f8340157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 --2- 172.21.15.3:0/1111878903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8340153c30 0x7f8340157310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f83380098e0 tx=0x7f83380091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 -- 172.21.15.3:0/1111878903 shutdown_connections 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 --2- 172.21.15.3:0/1111878903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8340153c30 0x7f8340157310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 --2- 172.21.15.3:0/1111878903 >> v2:172.21.15.153:3300/0 conn(0x7f8340154000 0x7f8340159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 -- 172.21.15.3:0/1111878903 >> 172.21.15.3:0/1111878903 conn(0x7f834006d480 msgr2=0x7f834006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 -- 172.21.15.3:0/1111878903 shutdown_connections 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.457+0000 7f833f59e700 1 -- 172.21.15.3:0/1111878903 wait complete. 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.458+0000 7f833f59e700 1 Processor -- start 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.458+0000 7f833f59e700 1 -- start start 2023-10-06T05:01:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83400ae9b0 0x7f834016cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8340153c30 0x7f8340150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340154000 0x7f834016a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834016abe0 con 0x7f8340154000 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400aa2b0 con 0x7f8340153c30 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83401514c0 con 0x7f83400ae9b0 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340154000 0x7f834016a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340154000 0x7f834016a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50440/0 (socket says 172.21.15.3:50440) 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 -- 172.21.15.3:0/2597474167 learned_addr learned my addr 172.21.15.3:0/2597474167 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833dd9b700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8340153c30 0x7f8340150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833e59c700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83400ae9b0 0x7f834016cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 -- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83400ae9b0 msgr2=0x7f834016cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83400ae9b0 0x7f834016cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 -- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8340153c30 msgr2=0x7f8340150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8340153c30 0x7f8340150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 -- 172.21.15.3:0/2597474167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83280094f0 con 0x7f8340154000 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.459+0000 7f833ed9d700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340154000 0x7f834016a6a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f833000cca0 tx=0x7f8330001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.460+0000 7f83367fc700 1 -- 172.21.15.3:0/2597474167 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330005df0 con 0x7f8340154000 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.460+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8338009420 con 0x7f8340154000 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.460+0000 7f83367fc700 1 -- 172.21.15.3:0/2597474167 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833000a5f0 con 0x7f8340154000 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.460+0000 7f83367fc700 1 -- 172.21.15.3:0/2597474167 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833000a9b0 con 0x7f8340154000 2023-10-06T05:01:26.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.460+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400a9c60 con 0x7f8340154000 2023-10-06T05:01:26.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.461+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834004c380 con 0x7f8340154000 2023-10-06T05:01:26.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.461+0000 7f83367fc700 1 -- 172.21.15.3:0/2597474167 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83300070f0 con 0x7f8340154000 2023-10-06T05:01:26.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.461+0000 7f83367fc700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8320066f30 0x7f83200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.461+0000 7f83367fc700 1 -- 172.21.15.3:0/2597474167 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8330084640 con 0x7f8340154000 2023-10-06T05:01:26.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.462+0000 7f833e59c700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8320066f30 0x7f83200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.462+0000 7f833e59c700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8320066f30 0x7f83200693e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f83380093f0 tx=0x7f8338017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.466+0000 7f83367fc700 1 -- 172.21.15.3:0/2597474167 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8330051750 con 0x7f8340154000 2023-10-06T05:01:26.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.660+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83400aa3f0 con 0x7f8340154000 2023-10-06T05:01:26.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.661+0000 7f83367fc700 1 -- 172.21.15.3:0/2597474167 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16832 (secure 0 0 0) 0x7f8330054d70 con 0x7f8340154000 2023-10-06T05:01:26.663 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:26.663 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":132,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:01:23.899107+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:01:20.044822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:01:21.270200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:01:26.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8320066f30 msgr2=0x7f83200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8320066f30 0x7f83200693e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f83380093f0 tx=0x7f8338017000 comp rx=0 tx=0).stop 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340154000 msgr2=0x7f834016a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340154000 0x7f834016a6a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f833000cca0 tx=0x7f8330001a70 comp rx=0 tx=0).stop 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 shutdown_connections 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8320066f30 0x7f83200693e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340154000 0x7f834016a6a0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83400ae9b0 0x7f834016cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 --2- 172.21.15.3:0/2597474167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8340153c30 0x7f8340150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 >> 172.21.15.3:0/2597474167 conn(0x7f834006d480 msgr2=0x7f8340159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 shutdown_connections 2023-10-06T05:01:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.663+0000 7f833f59e700 1 -- 172.21.15.3:0/2597474167 wait complete. 2023-10-06T05:01:26.677 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:01:26.677 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:01:26.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.795+0000 7f943d7c8700 1 Processor -- start 2023-10-06T05:01:26.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.795+0000 7f943d7c8700 1 -- start start 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.795+0000 7f943d7c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9438159500 0x7f94380cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.795+0000 7f943d7c8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f94380b2930 0x7f94380cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f943d7c8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380cc5c0 con 0x7f94381598d0 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f943d7c8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94380cb7b0 con 0x7f9438159500 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f943d7c8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94380cda60 con 0x7f94380b2930 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9436ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9438159500 0x7f94380cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f94367fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94381598d0 0x7f94380b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40182/0 (socket says 172.21.15.3:40182) 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f94367fc700 1 -- 172.21.15.3:0/3254837829 learned_addr learned my addr 172.21.15.3:0/3254837829 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f94377fe700 1 --2- 172.21.15.3:0/3254837829 >> v2:172.21.15.153:3300/0 conn(0x7f94380b2930 0x7f94380cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9435ffb700 1 -- 172.21.15.3:0/3254837829 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659429335 0 0) 0x7f94380cc5c0 con 0x7f94381598d0 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9435ffb700 1 -- 172.21.15.3:0/3254837829 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9420003570 con 0x7f94381598d0 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9436ffd700 1 -- 172.21.15.3:0/3254837829 >> v2:172.21.15.153:3300/0 conn(0x7f94380b2930 msgr2=0x7f94380cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9436ffd700 1 --2- 172.21.15.3:0/3254837829 >> v2:172.21.15.153:3300/0 conn(0x7f94380b2930 0x7f94380cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9436ffd700 1 -- 172.21.15.3:0/3254837829 >> v1:172.21.15.3:6789/0 conn(0x7f94381598d0 legacy=0x7f94380b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9436ffd700 1 -- 172.21.15.3:0/3254837829 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380cb9b0 con 0x7f9438159500 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9436ffd700 1 --2- 172.21.15.3:0/3254837829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9438159500 0x7f94380cad30 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f94380c70d0 tx=0x7f94280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adee2c7ea7a057aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9435ffb700 1 -- 172.21.15.3:0/3254837829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011630 con 0x7f9438159500 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9435ffb700 1 -- 172.21.15.3:0/3254837829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94280117b0 con 0x7f9438159500 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.796+0000 7f9435ffb700 1 -- 172.21.15.3:0/3254837829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011b70 con 0x7f9438159500 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 -- 172.21.15.3:0/3254837829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9438159500 msgr2=0x7f94380cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 --2- 172.21.15.3:0/3254837829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9438159500 0x7f94380cad30 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f94380c70d0 tx=0x7f94280091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 -- 172.21.15.3:0/3254837829 shutdown_connections 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 --2- 172.21.15.3:0/3254837829 >> v2:172.21.15.153:3300/0 conn(0x7f94380b2930 0x7f94380cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 --2- 172.21.15.3:0/3254837829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9438159500 0x7f94380cad30 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 -- 172.21.15.3:0/3254837829 >> 172.21.15.3:0/3254837829 conn(0x7f9438070310 msgr2=0x7f9438070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 -- 172.21.15.3:0/3254837829 shutdown_connections 2023-10-06T05:01:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.797+0000 7f943d7c8700 1 -- 172.21.15.3:0/3254837829 wait complete. 2023-10-06T05:01:26.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.798+0000 7f943d7c8700 1 Processor -- start 2023-10-06T05:01:26.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.798+0000 7f943d7c8700 1 -- start start 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f943d7c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94380b2930 0x7f943815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f943d7c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9438159500 0x7f94380cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f943d7c8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94381598d0 0x7f94380c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f943d7c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380b36e0 con 0x7f94380b2930 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f943d7c8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380b39c0 con 0x7f9438159500 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f943d7c8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380b3ca0 con 0x7f94381598d0 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94380b2930 0x7f943815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94380b2930 0x7f943815bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50460/0 (socket says 172.21.15.3:50460) 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 -- 172.21.15.3:0/2448233648 learned_addr learned my addr 172.21.15.3:0/2448233648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f94367fc700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9438159500 0x7f94380cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 -- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94381598d0 msgr2=0x7f94380c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f94377fe700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94381598d0 0x7f94380c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94381598d0 0x7f94380c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 -- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9438159500 msgr2=0x7f94380cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9438159500 0x7f94380cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 -- 172.21.15.3:0/2448233648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9428009420 con 0x7f94380b2930 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f94367fc700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9438159500 0x7f94380cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.799+0000 7f9436ffd700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94380b2930 0x7f943815bbe0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9428013000 tx=0x7f9428017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.800+0000 7f941effd700 1 -- 172.21.15.3:0/2448233648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942801f070 con 0x7f94380b2930 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.800+0000 7f941effd700 1 -- 172.21.15.3:0/2448233648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9428000bc0 con 0x7f94380b2930 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.800+0000 7f941effd700 1 -- 172.21.15.3:0/2448233648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011b70 con 0x7f94380b2930 2023-10-06T05:01:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.800+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94380b3f80 con 0x7f94380b2930 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.800+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943815c120 con 0x7f94380b2930 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.801+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9438071680 con 0x7f94380b2930 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.801+0000 7f941effd700 1 -- 172.21.15.3:0/2448233648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9428023070 con 0x7f94380b2930 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.801+0000 7f941effd700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.802+0000 7f941effd700 1 -- 172.21.15.3:0/2448233648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9428085a20 con 0x7f94380b2930 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.802+0000 7f94367fc700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.802+0000 7f94367fc700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9424066f30 0x7f94240693e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f942c007fd0 tx=0x7f942c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.806+0000 7f941effd700 1 -- 172.21.15.3:0/2448233648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9428052b30 con 0x7f94380b2930 2023-10-06T05:01:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:26.998+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94380cab10 con 0x7f9424066f30 2023-10-06T05:01:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.007+0000 7f941effd700 1 -- 172.21.15.3:0/2448233648 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308531 (secure 0 0 0) 0x7f94380cab10 con 0x7f9424066f30 2023-10-06T05:01:27.012 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9424066f30 msgr2=0x7f94240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9424066f30 0x7f94240693e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f942c007fd0 tx=0x7f942c009040 comp rx=0 tx=0).stop 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94380b2930 msgr2=0x7f943815bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94380b2930 0x7f943815bbe0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9428013000 tx=0x7f9428017000 comp rx=0 tx=0).stop 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 shutdown_connections 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94381598d0 0x7f94380c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94380b2930 0x7f943815bbe0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 --2- 172.21.15.3:0/2448233648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9438159500 0x7f94380cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 >> 172.21.15.3:0/2448233648 conn(0x7f9438070310 msgr2=0x7f94380cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 shutdown_connections 2023-10-06T05:01:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.010+0000 7f943d7c8700 1 -- 172.21.15.3:0/2448233648 wait complete. 2023-10-06T05:01:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:01:27.024 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2023-10-06T05:01:25.558118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449676,"kb_used_data":14028,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752525044,"statfs":{"total":773094113280,"available":770585645056,"internally_reserved":0,"allocated":14364672,"data_stored":7156765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.679338"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919337+0000","last_change":"2023-10-06T05:01:22.917546+0000","last_active":"2023-10-06T05:01:22.919337+0000","last_peered":"2023-10-06T05:01:22.919337+0000","last_clean":"2023-10-06T05:01:22.919337+0000","last_became_active":"2023-10-06T05:01:22.916374+0000","last_became_peered":"2023-10-06T05:01:22.916374+0000","last_unstale":"2023-10-06T05:01:22.919337+0000","last_undegraded":"2023-10-06T05:01:22.919337+0000","last_fullsized":"2023-10-06T05:01:22.919337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917323+0000","last_change":"2023-10-06T05:01:22.915942+0000","last_active":"2023-10-06T05:01:22.917323+0000","last_peered":"2023-10-06T05:01:22.917323+0000","last_clean":"2023-10-06T05:01:22.917323+0000","last_became_active":"2023-10-06T05:01:22.915602+0000","last_became_peered":"2023-10-06T05:01:22.915602+0000","last_unstale":"2023-10-06T05:01:22.917323+0000","last_undegraded":"2023-10-06T05:01:22.917323+0000","last_fullsized":"2023-10-06T05:01:22.917323+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:39.759900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917901+0000","last_change":"2023-10-06T05:01:22.916859+0000","last_active":"2023-10-06T05:01:22.917901+0000","last_peered":"2023-10-06T05:01:22.917901+0000","last_clean":"2023-10-06T05:01:22.917901+0000","last_became_active":"2023-10-06T05:01:22.916504+0000","last_became_peered":"2023-10-06T05:01:22.916504+0000","last_unstale":"2023-10-06T05:01:22.917901+0000","last_undegraded":"2023-10-06T05:01:22.917901+0000","last_fullsized":"2023-10-06T05:01:22.917901+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:36.075945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912802+0000","last_change":"2023-10-06T05:01:22.908378+0000","last_active":"2023-10-06T05:01:22.912802+0000","last_peered":"2023-10-06T05:01:22.912802+0000","last_clean":"2023-10-06T05:01:22.912802+0000","last_became_active":"2023-10-06T05:01:22.907429+0000","last_became_peered":"2023-10-06T05:01:22.907429+0000","last_unstale":"2023-10-06T05:01:22.912802+0000","last_undegraded":"2023-10-06T05:01:22.912802+0000","last_fullsized":"2023-10-06T05:01:22.912802+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:32.443484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912753+0000","last_change":"2023-10-06T05:01:22.907760+0000","last_active":"2023-10-06T05:01:22.912753+0000","last_peered":"2023-10-06T05:01:22.912753+0000","last_clean":"2023-10-06T05:01:22.912753+0000","last_became_active":"2023-10-06T05:01:22.907466+0000","last_became_peered":"2023-10-06T05:01:22.907466+0000","last_unstale":"2023-10-06T05:01:22.912753+0000","last_undegraded":"2023-10-06T05:01:22.912753+0000","last_fullsized":"2023-10-06T05:01:22.912753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:21.629837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908240+0000","last_change":"2023-10-06T05:01:22.907593+0000","last_active":"2023-10-06T05:01:22.908240+0000","last_peered":"2023-10-06T05:01:22.908240+0000","last_clean":"2023-10-06T05:01:22.908240+0000","last_became_active":"2023-10-06T05:01:22.907290+0000","last_became_peered":"2023-10-06T05:01:22.907290+0000","last_unstale":"2023-10-06T05:01:22.908240+0000","last_undegraded":"2023-10-06T05:01:22.908240+0000","last_fullsized":"2023-10-06T05:01:22.908240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910976+0000","last_change":"2023-10-06T05:01:22.906539+0000","last_active":"2023-10-06T05:01:22.910976+0000","last_peered":"2023-10-06T05:01:22.910976+0000","last_clean":"2023-10-06T05:01:22.910976+0000","last_became_active":"2023-10-06T05:01:22.905906+0000","last_became_peered":"2023-10-06T05:01:22.905906+0000","last_unstale":"2023-10-06T05:01:22.910976+0000","last_undegraded":"2023-10-06T05:01:22.910976+0000","last_fullsized":"2023-10-06T05:01:22.910976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:33.022370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908499+0000","last_change":"2023-10-06T05:01:22.906059+0000","last_active":"2023-10-06T05:01:22.908499+0000","last_peered":"2023-10-06T05:01:22.908499+0000","last_clean":"2023-10-06T05:01:22.908499+0000","last_became_active":"2023-10-06T05:01:22.905782+0000","last_became_peered":"2023-10-06T05:01:22.905782+0000","last_unstale":"2023-10-06T05:01:22.908499+0000","last_undegraded":"2023-10-06T05:01:22.908499+0000","last_fullsized":"2023-10-06T05:01:22.908499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:31.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916007+0000","last_change":"2023-10-06T05:01:22.914642+0000","last_active":"2023-10-06T05:01:22.916007+0000","last_peered":"2023-10-06T05:01:22.916007+0000","last_clean":"2023-10-06T05:01:22.916007+0000","last_became_active":"2023-10-06T05:01:22.913781+0000","last_became_peered":"2023-10-06T05:01:22.913781+0000","last_unstale":"2023-10-06T05:01:22.916007+0000","last_undegraded":"2023-10-06T05:01:22.916007+0000","last_fullsized":"2023-10-06T05:01:22.916007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:39.375404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909396+0000","last_change":"2023-10-06T05:01:22.906295+0000","last_active":"2023-10-06T05:01:22.909396+0000","last_peered":"2023-10-06T05:01:22.909396+0000","last_clean":"2023-10-06T05:01:22.909396+0000","last_became_active":"2023-10-06T05:01:22.905241+0000","last_became_peered":"2023-10-06T05:01:22.905241+0000","last_unstale":"2023-10-06T05:01:22.909396+0000","last_undegraded":"2023-10-06T05:01:22.909396+0000","last_fullsized":"2023-10-06T05:01:22.909396+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:56.323569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921375+0000","last_change":"2023-10-06T05:01:22.920640+0000","last_active":"2023-10-06T05:01:22.921375+0000","last_peered":"2023-10-06T05:01:22.921375+0000","last_clean":"2023-10-06T05:01:22.921375+0000","last_became_active":"2023-10-06T05:01:22.920435+0000","last_became_peered":"2023-10-06T05:01:22.920435+0000","last_unstale":"2023-10-06T05:01:22.921375+0000","last_undegraded":"2023-10-06T05:01:22.921375+0000","last_fullsized":"2023-10-06T05:01:22.921375+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:21.298643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921735+0000","last_change":"2023-10-06T05:01:22.921275+0000","last_active":"2023-10-06T05:01:22.921735+0000","last_peered":"2023-10-06T05:01:22.921735+0000","last_clean":"2023-10-06T05:01:22.921735+0000","last_became_active":"2023-10-06T05:01:22.920953+0000","last_became_peered":"2023-10-06T05:01:22.920953+0000","last_unstale":"2023-10-06T05:01:22.921735+0000","last_undegraded":"2023-10-06T05:01:22.921735+0000","last_fullsized":"2023-10-06T05:01:22.921735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:22.455454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912183+0000","last_change":"2023-10-06T05:01:22.910088+0000","last_active":"2023-10-06T05:01:22.912183+0000","last_peered":"2023-10-06T05:01:22.912183+0000","last_clean":"2023-10-06T05:01:22.912183+0000","last_became_active":"2023-10-06T05:01:22.909830+0000","last_became_peered":"2023-10-06T05:01:22.909830+0000","last_unstale":"2023-10-06T05:01:22.912183+0000","last_undegraded":"2023-10-06T05:01:22.912183+0000","last_fullsized":"2023-10-06T05:01:22.912183+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:14.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920682+0000","last_change":"2023-10-06T05:01:22.915966+0000","last_active":"2023-10-06T05:01:22.920682+0000","last_peered":"2023-10-06T05:01:22.920682+0000","last_clean":"2023-10-06T05:01:22.920682+0000","last_became_active":"2023-10-06T05:01:22.915477+0000","last_became_peered":"2023-10-06T05:01:22.915477+0000","last_unstale":"2023-10-06T05:01:22.920682+0000","last_undegraded":"2023-10-06T05:01:22.920682+0000","last_fullsized":"2023-10-06T05:01:22.920682+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:24.762820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908898+0000","last_change":"2023-10-06T05:01:22.905656+0000","last_active":"2023-10-06T05:01:22.908898+0000","last_peered":"2023-10-06T05:01:22.908898+0000","last_clean":"2023-10-06T05:01:22.908898+0000","last_became_active":"2023-10-06T05:01:22.905402+0000","last_became_peered":"2023-10-06T05:01:22.905402+0000","last_unstale":"2023-10-06T05:01:22.908898+0000","last_undegraded":"2023-10-06T05:01:22.908898+0000","last_fullsized":"2023-10-06T05:01:22.908898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.573131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912788+0000","last_change":"2023-10-06T05:01:22.908184+0000","last_active":"2023-10-06T05:01:22.912788+0000","last_peered":"2023-10-06T05:01:22.912788+0000","last_clean":"2023-10-06T05:01:22.912788+0000","last_became_active":"2023-10-06T05:01:22.904908+0000","last_became_peered":"2023-10-06T05:01:22.904908+0000","last_unstale":"2023-10-06T05:01:22.912788+0000","last_undegraded":"2023-10-06T05:01:22.912788+0000","last_fullsized":"2023-10-06T05:01:22.912788+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:42.892786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912280+0000","last_change":"2023-10-06T05:01:22.908024+0000","last_active":"2023-10-06T05:01:22.912280+0000","last_peered":"2023-10-06T05:01:22.912280+0000","last_clean":"2023-10-06T05:01:22.912280+0000","last_became_active":"2023-10-06T05:01:22.907701+0000","last_became_peered":"2023-10-06T05:01:22.907701+0000","last_unstale":"2023-10-06T05:01:22.912280+0000","last_undegraded":"2023-10-06T05:01:22.912280+0000","last_fullsized":"2023-10-06T05:01:22.912280+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910490+0000","last_change":"2023-10-06T05:01:22.902838+0000","last_active":"2023-10-06T05:01:22.910490+0000","last_peered":"2023-10-06T05:01:22.910490+0000","last_clean":"2023-10-06T05:01:22.910490+0000","last_became_active":"2023-10-06T05:01:22.902549+0000","last_became_peered":"2023-10-06T05:01:22.902549+0000","last_unstale":"2023-10-06T05:01:22.910490+0000","last_undegraded":"2023-10-06T05:01:22.910490+0000","last_fullsized":"2023-10-06T05:01:22.910490+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:26.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912308+0000","last_change":"2023-10-06T05:01:22.908586+0000","last_active":"2023-10-06T05:01:22.912308+0000","last_peered":"2023-10-06T05:01:22.912308+0000","last_clean":"2023-10-06T05:01:22.912308+0000","last_became_active":"2023-10-06T05:01:22.908289+0000","last_became_peered":"2023-10-06T05:01:22.908289+0000","last_unstale":"2023-10-06T05:01:22.912308+0000","last_undegraded":"2023-10-06T05:01:22.912308+0000","last_fullsized":"2023-10-06T05:01:22.912308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:38.888355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914573+0000","last_change":"2023-10-06T05:01:22.910843+0000","last_active":"2023-10-06T05:01:22.914573+0000","last_peered":"2023-10-06T05:01:22.914573+0000","last_clean":"2023-10-06T05:01:22.914573+0000","last_became_active":"2023-10-06T05:01:22.910470+0000","last_became_peered":"2023-10-06T05:01:22.910470+0000","last_unstale":"2023-10-06T05:01:22.914573+0000","last_undegraded":"2023-10-06T05:01:22.914573+0000","last_fullsized":"2023-10-06T05:01:22.914573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912812+0000","last_change":"2023-10-06T05:01:22.908293+0000","last_active":"2023-10-06T05:01:22.912812+0000","last_peered":"2023-10-06T05:01:22.912812+0000","last_clean":"2023-10-06T05:01:22.912812+0000","last_became_active":"2023-10-06T05:01:22.907986+0000","last_became_peered":"2023-10-06T05:01:22.907986+0000","last_unstale":"2023-10-06T05:01:22.912812+0000","last_undegraded":"2023-10-06T05:01:22.912812+0000","last_fullsized":"2023-10-06T05:01:22.912812+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:58.242153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910601+0000","last_change":"2023-10-06T05:01:22.905470+0000","last_active":"2023-10-06T05:01:22.910601+0000","last_peered":"2023-10-06T05:01:22.910601+0000","last_clean":"2023-10-06T05:01:22.910601+0000","last_became_active":"2023-10-06T05:01:22.905162+0000","last_became_peered":"2023-10-06T05:01:22.905162+0000","last_unstale":"2023-10-06T05:01:22.910601+0000","last_undegraded":"2023-10-06T05:01:22.910601+0000","last_fullsized":"2023-10-06T05:01:22.910601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:53.259645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912590+0000","last_change":"2023-10-06T05:01:22.911945+0000","last_active":"2023-10-06T05:01:22.912590+0000","last_peered":"2023-10-06T05:01:22.912590+0000","last_clean":"2023-10-06T05:01:22.912590+0000","last_became_active":"2023-10-06T05:01:22.911675+0000","last_became_peered":"2023-10-06T05:01:22.911675+0000","last_unstale":"2023-10-06T05:01:22.912590+0000","last_undegraded":"2023-10-06T05:01:22.912590+0000","last_fullsized":"2023-10-06T05:01:22.912590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:32.223585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914827+0000","last_change":"2023-10-06T05:01:22.913460+0000","last_active":"2023-10-06T05:01:22.914827+0000","last_peered":"2023-10-06T05:01:22.914827+0000","last_clean":"2023-10-06T05:01:22.914827+0000","last_became_active":"2023-10-06T05:01:22.913178+0000","last_became_peered":"2023-10-06T05:01:22.913178+0000","last_unstale":"2023-10-06T05:01:22.914827+0000","last_undegraded":"2023-10-06T05:01:22.914827+0000","last_fullsized":"2023-10-06T05:01:22.914827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:16.124524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909358+0000","last_change":"2023-10-06T05:01:22.906868+0000","last_active":"2023-10-06T05:01:22.909358+0000","last_peered":"2023-10-06T05:01:22.909358+0000","last_clean":"2023-10-06T05:01:22.909358+0000","last_became_active":"2023-10-06T05:01:22.906368+0000","last_became_peered":"2023-10-06T05:01:22.906368+0000","last_unstale":"2023-10-06T05:01:22.909358+0000","last_undegraded":"2023-10-06T05:01:22.909358+0000","last_fullsized":"2023-10-06T05:01:22.909358+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:45.514209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910892+0000","last_change":"2023-10-06T05:01:21.905167+0000","last_active":"2023-10-06T05:01:22.910892+0000","last_peered":"2023-10-06T05:01:22.910892+0000","last_clean":"2023-10-06T05:01:22.910892+0000","last_became_active":"2023-10-06T05:01:21.904856+0000","last_became_peered":"2023-10-06T05:01:21.904856+0000","last_unstale":"2023-10-06T05:01:22.910892+0000","last_undegraded":"2023-10-06T05:01:22.910892+0000","last_fullsized":"2023-10-06T05:01:22.910892+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916718+0000","last_change":"2023-10-06T05:01:22.915682+0000","last_active":"2023-10-06T05:01:22.916718+0000","last_peered":"2023-10-06T05:01:22.916718+0000","last_clean":"2023-10-06T05:01:22.916718+0000","last_became_active":"2023-10-06T05:01:22.915425+0000","last_became_peered":"2023-10-06T05:01:22.915425+0000","last_unstale":"2023-10-06T05:01:22.916718+0000","last_undegraded":"2023-10-06T05:01:22.916718+0000","last_fullsized":"2023-10-06T05:01:22.916718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913713+0000","last_change":"2023-10-06T05:01:21.909299+0000","last_active":"2023-10-06T05:01:22.913713+0000","last_peered":"2023-10-06T05:01:22.913713+0000","last_clean":"2023-10-06T05:01:22.913713+0000","last_became_active":"2023-10-06T05:01:21.908989+0000","last_became_peered":"2023-10-06T05:01:21.908989+0000","last_unstale":"2023-10-06T05:01:22.913713+0000","last_undegraded":"2023-10-06T05:01:22.913713+0000","last_fullsized":"2023-10-06T05:01:22.913713+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914828+0000","last_change":"2023-10-06T05:01:21.909301+0000","last_active":"2023-10-06T05:01:22.914828+0000","last_peered":"2023-10-06T05:01:22.914828+0000","last_clean":"2023-10-06T05:01:22.914828+0000","last_became_active":"2023-10-06T05:01:21.908984+0000","last_became_peered":"2023-10-06T05:01:21.908984+0000","last_unstale":"2023-10-06T05:01:22.914828+0000","last_undegraded":"2023-10-06T05:01:22.914828+0000","last_fullsized":"2023-10-06T05:01:22.914828+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:26.376744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920398+0000","last_change":"2023-10-06T05:01:22.910823+0000","last_active":"2023-10-06T05:01:22.920398+0000","last_peered":"2023-10-06T05:01:22.920398+0000","last_clean":"2023-10-06T05:01:22.920398+0000","last_became_active":"2023-10-06T05:01:22.910158+0000","last_became_peered":"2023-10-06T05:01:22.910158+0000","last_unstale":"2023-10-06T05:01:22.920398+0000","last_undegraded":"2023-10-06T05:01:22.920398+0000","last_fullsized":"2023-10-06T05:01:22.920398+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:33.254913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917199+0000","last_change":"2023-10-06T05:01:21.898248+0000","last_active":"2023-10-06T05:01:22.917199+0000","last_peered":"2023-10-06T05:01:22.917199+0000","last_clean":"2023-10-06T05:01:22.917199+0000","last_became_active":"2023-10-06T05:01:21.897180+0000","last_became_peered":"2023-10-06T05:01:21.897180+0000","last_unstale":"2023-10-06T05:01:22.917199+0000","last_undegraded":"2023-10-06T05:01:22.917199+0000","last_fullsized":"2023-10-06T05:01:22.917199+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:59.743528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909059+0000","last_change":"2023-10-06T05:01:22.905942+0000","last_active":"2023-10-06T05:01:22.909059+0000","last_peered":"2023-10-06T05:01:22.909059+0000","last_clean":"2023-10-06T05:01:22.909059+0000","last_became_active":"2023-10-06T05:01:22.905653+0000","last_became_peered":"2023-10-06T05:01:22.905653+0000","last_unstale":"2023-10-06T05:01:22.909059+0000","last_undegraded":"2023-10-06T05:01:22.909059+0000","last_fullsized":"2023-10-06T05:01:22.909059+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:12.300608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916407+0000","last_change":"2023-10-06T05:01:21.906758+0000","last_active":"2023-10-06T05:01:22.916407+0000","last_peered":"2023-10-06T05:01:22.916407+0000","last_clean":"2023-10-06T05:01:22.916407+0000","last_became_active":"2023-10-06T05:01:21.906463+0000","last_became_peered":"2023-10-06T05:01:21.906463+0000","last_unstale":"2023-10-06T05:01:22.916407+0000","last_undegraded":"2023-10-06T05:01:22.916407+0000","last_fullsized":"2023-10-06T05:01:22.916407+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:42.205607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908789+0000","last_change":"2023-10-06T05:01:22.905020+0000","last_active":"2023-10-06T05:01:22.908789+0000","last_peered":"2023-10-06T05:01:22.908789+0000","last_clean":"2023-10-06T05:01:22.908789+0000","last_became_active":"2023-10-06T05:01:22.904762+0000","last_became_peered":"2023-10-06T05:01:22.904762+0000","last_unstale":"2023-10-06T05:01:22.908789+0000","last_undegraded":"2023-10-06T05:01:22.908789+0000","last_fullsized":"2023-10-06T05:01:22.908789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900865+0000","last_change":"2023-10-06T05:01:21.907467+0000","last_active":"2023-10-06T05:01:22.900865+0000","last_peered":"2023-10-06T05:01:22.900865+0000","last_clean":"2023-10-06T05:01:22.900865+0000","last_became_active":"2023-10-06T05:01:21.907169+0000","last_became_peered":"2023-10-06T05:01:21.907169+0000","last_unstale":"2023-10-06T05:01:22.900865+0000","last_undegraded":"2023-10-06T05:01:22.900865+0000","last_fullsized":"2023-10-06T05:01:22.900865+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.309675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918810+0000","last_change":"2023-10-06T05:01:22.916673+0000","last_active":"2023-10-06T05:01:22.918810+0000","last_peered":"2023-10-06T05:01:22.918810+0000","last_clean":"2023-10-06T05:01:22.918810+0000","last_became_active":"2023-10-06T05:01:22.916393+0000","last_became_peered":"2023-10-06T05:01:22.916393+0000","last_unstale":"2023-10-06T05:01:22.918810+0000","last_undegraded":"2023-10-06T05:01:22.918810+0000","last_fullsized":"2023-10-06T05:01:22.918810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:00.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.890628+0000","last_change":"2023-10-06T05:01:21.904853+0000","last_active":"2023-10-06T05:01:22.890628+0000","last_peered":"2023-10-06T05:01:22.890628+0000","last_clean":"2023-10-06T05:01:22.890628+0000","last_became_active":"2023-10-06T05:01:21.904448+0000","last_became_peered":"2023-10-06T05:01:21.904448+0000","last_unstale":"2023-10-06T05:01:22.890628+0000","last_undegraded":"2023-10-06T05:01:22.890628+0000","last_fullsized":"2023-10-06T05:01:22.890628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.567032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915557+0000","last_change":"2023-10-06T05:01:22.909027+0000","last_active":"2023-10-06T05:01:22.915557+0000","last_peered":"2023-10-06T05:01:22.915557+0000","last_clean":"2023-10-06T05:01:22.915557+0000","last_became_active":"2023-10-06T05:01:22.908783+0000","last_became_peered":"2023-10-06T05:01:22.908783+0000","last_unstale":"2023-10-06T05:01:22.915557+0000","last_undegraded":"2023-10-06T05:01:22.915557+0000","last_fullsized":"2023-10-06T05:01:22.915557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:48.927255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914458+0000","last_change":"2023-10-06T05:01:21.901342+0000","last_active":"2023-10-06T05:01:22.914458+0000","last_peered":"2023-10-06T05:01:22.914458+0000","last_clean":"2023-10-06T05:01:22.914458+0000","last_became_active":"2023-10-06T05:01:21.901020+0000","last_became_peered":"2023-10-06T05:01:21.901020+0000","last_unstale":"2023-10-06T05:01:22.914458+0000","last_undegraded":"2023-10-06T05:01:22.914458+0000","last_fullsized":"2023-10-06T05:01:22.914458+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:26.399481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920662+0000","last_change":"2023-10-06T05:01:22.915799+0000","last_active":"2023-10-06T05:01:22.920662+0000","last_peered":"2023-10-06T05:01:22.920662+0000","last_clean":"2023-10-06T05:01:22.920662+0000","last_became_active":"2023-10-06T05:01:22.915224+0000","last_became_peered":"2023-10-06T05:01:22.915224+0000","last_unstale":"2023-10-06T05:01:22.920662+0000","last_undegraded":"2023-10-06T05:01:22.920662+0000","last_fullsized":"2023-10-06T05:01:22.920662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:24.443047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909917+0000","last_change":"2023-10-06T05:01:21.903666+0000","last_active":"2023-10-06T05:01:22.909917+0000","last_peered":"2023-10-06T05:01:22.909917+0000","last_clean":"2023-10-06T05:01:22.909917+0000","last_became_active":"2023-10-06T05:01:21.903416+0000","last_became_peered":"2023-10-06T05:01:21.903416+0000","last_unstale":"2023-10-06T05:01:22.909917+0000","last_undegraded":"2023-10-06T05:01:22.909917+0000","last_fullsized":"2023-10-06T05:01:22.909917+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:38.753801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913218+0000","last_change":"2023-10-06T05:01:22.908994+0000","last_active":"2023-10-06T05:01:22.913218+0000","last_peered":"2023-10-06T05:01:22.913218+0000","last_clean":"2023-10-06T05:01:22.913218+0000","last_became_active":"2023-10-06T05:01:22.908627+0000","last_became_peered":"2023-10-06T05:01:22.908627+0000","last_unstale":"2023-10-06T05:01:22.913218+0000","last_undegraded":"2023-10-06T05:01:22.913218+0000","last_fullsized":"2023-10-06T05:01:22.913218+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920770+0000","last_change":"2023-10-06T05:01:21.901636+0000","last_active":"2023-10-06T05:01:22.920770+0000","last_peered":"2023-10-06T05:01:22.920770+0000","last_clean":"2023-10-06T05:01:22.920770+0000","last_became_active":"2023-10-06T05:01:21.901348+0000","last_became_peered":"2023-10-06T05:01:21.901348+0000","last_unstale":"2023-10-06T05:01:22.920770+0000","last_undegraded":"2023-10-06T05:01:22.920770+0000","last_fullsized":"2023-10-06T05:01:22.920770+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:07.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919815+0000","last_change":"2023-10-06T05:01:22.918705+0000","last_active":"2023-10-06T05:01:22.919815+0000","last_peered":"2023-10-06T05:01:22.919815+0000","last_clean":"2023-10-06T05:01:22.919815+0000","last_became_active":"2023-10-06T05:01:22.918437+0000","last_became_peered":"2023-10-06T05:01:22.918437+0000","last_unstale":"2023-10-06T05:01:22.919815+0000","last_undegraded":"2023-10-06T05:01:22.919815+0000","last_fullsized":"2023-10-06T05:01:22.919815+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920786+0000","last_change":"2023-10-06T05:01:22.919693+0000","last_active":"2023-10-06T05:01:22.920786+0000","last_peered":"2023-10-06T05:01:22.920786+0000","last_clean":"2023-10-06T05:01:22.920786+0000","last_became_active":"2023-10-06T05:01:22.919445+0000","last_became_peered":"2023-10-06T05:01:22.919445+0000","last_unstale":"2023-10-06T05:01:22.920786+0000","last_undegraded":"2023-10-06T05:01:22.920786+0000","last_fullsized":"2023-10-06T05:01:22.920786+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:00.412186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916231+0000","last_change":"2023-10-06T05:01:21.901798+0000","last_active":"2023-10-06T05:01:22.916231+0000","last_peered":"2023-10-06T05:01:22.916231+0000","last_clean":"2023-10-06T05:01:22.916231+0000","last_became_active":"2023-10-06T05:01:21.901536+0000","last_became_peered":"2023-10-06T05:01:21.901536+0000","last_unstale":"2023-10-06T05:01:22.916231+0000","last_undegraded":"2023-10-06T05:01:22.916231+0000","last_fullsized":"2023-10-06T05:01:22.916231+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909492+0000","last_change":"2023-10-06T05:01:22.908931+0000","last_active":"2023-10-06T05:01:22.909492+0000","last_peered":"2023-10-06T05:01:22.909492+0000","last_clean":"2023-10-06T05:01:22.909492+0000","last_became_active":"2023-10-06T05:01:22.908678+0000","last_became_peered":"2023-10-06T05:01:22.908678+0000","last_unstale":"2023-10-06T05:01:22.909492+0000","last_undegraded":"2023-10-06T05:01:22.909492+0000","last_fullsized":"2023-10-06T05:01:22.909492+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:24.372776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920281+0000","last_change":"2023-10-06T05:01:22.910581+0000","last_active":"2023-10-06T05:01:22.920281+0000","last_peered":"2023-10-06T05:01:22.920281+0000","last_clean":"2023-10-06T05:01:22.920281+0000","last_became_active":"2023-10-06T05:01:22.909963+0000","last_became_peered":"2023-10-06T05:01:22.909963+0000","last_unstale":"2023-10-06T05:01:22.920281+0000","last_undegraded":"2023-10-06T05:01:22.920281+0000","last_fullsized":"2023-10-06T05:01:22.920281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:41.453873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.893893+0000","last_change":"2023-10-06T05:01:21.900893+0000","last_active":"2023-10-06T05:01:22.893893+0000","last_peered":"2023-10-06T05:01:22.893893+0000","last_clean":"2023-10-06T05:01:22.893893+0000","last_became_active":"2023-10-06T05:01:21.900591+0000","last_became_peered":"2023-10-06T05:01:21.900591+0000","last_unstale":"2023-10-06T05:01:22.893893+0000","last_undegraded":"2023-10-06T05:01:22.893893+0000","last_fullsized":"2023-10-06T05:01:22.893893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917896+0000","last_change":"2023-10-06T05:01:22.917013+0000","last_active":"2023-10-06T05:01:22.917896+0000","last_peered":"2023-10-06T05:01:22.917896+0000","last_clean":"2023-10-06T05:01:22.917896+0000","last_became_active":"2023-10-06T05:01:22.915500+0000","last_became_peered":"2023-10-06T05:01:22.915500+0000","last_unstale":"2023-10-06T05:01:22.917896+0000","last_undegraded":"2023-10-06T05:01:22.917896+0000","last_fullsized":"2023-10-06T05:01:22.917896+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912743+0000","last_change":"2023-10-06T05:01:22.907709+0000","last_active":"2023-10-06T05:01:22.912743+0000","last_peered":"2023-10-06T05:01:22.912743+0000","last_clean":"2023-10-06T05:01:22.912743+0000","last_became_active":"2023-10-06T05:01:22.907411+0000","last_became_peered":"2023-10-06T05:01:22.907411+0000","last_unstale":"2023-10-06T05:01:22.912743+0000","last_undegraded":"2023-10-06T05:01:22.912743+0000","last_fullsized":"2023-10-06T05:01:22.912743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:19.960229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.901055+0000","last_change":"2023-10-06T05:01:21.907168+0000","last_active":"2023-10-06T05:01:22.901055+0000","last_peered":"2023-10-06T05:01:22.901055+0000","last_clean":"2023-10-06T05:01:22.901055+0000","last_became_active":"2023-10-06T05:01:21.906798+0000","last_became_peered":"2023-10-06T05:01:21.906798+0000","last_unstale":"2023-10-06T05:01:22.901055+0000","last_undegraded":"2023-10-06T05:01:22.901055+0000","last_fullsized":"2023-10-06T05:01:22.901055+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919507+0000","last_change":"2023-10-06T05:01:22.917117+0000","last_active":"2023-10-06T05:01:22.919507+0000","last_peered":"2023-10-06T05:01:22.919507+0000","last_clean":"2023-10-06T05:01:22.919507+0000","last_became_active":"2023-10-06T05:01:22.916829+0000","last_became_peered":"2023-10-06T05:01:22.916829+0000","last_unstale":"2023-10-06T05:01:22.919507+0000","last_undegraded":"2023-10-06T05:01:22.919507+0000","last_fullsized":"2023-10-06T05:01:22.919507+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.896584+0000","last_change":"2023-10-06T05:01:21.897514+0000","last_active":"2023-10-06T05:01:22.896584+0000","last_peered":"2023-10-06T05:01:22.896584+0000","last_clean":"2023-10-06T05:01:22.896584+0000","last_became_active":"2023-10-06T05:01:21.897226+0000","last_became_peered":"2023-10-06T05:01:21.897226+0000","last_unstale":"2023-10-06T05:01:22.896584+0000","last_undegraded":"2023-10-06T05:01:22.896584+0000","last_fullsized":"2023-10-06T05:01:22.896584+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:44.413476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917870+0000","last_change":"2023-10-06T05:01:22.916955+0000","last_active":"2023-10-06T05:01:22.917870+0000","last_peered":"2023-10-06T05:01:22.917870+0000","last_clean":"2023-10-06T05:01:22.917870+0000","last_became_active":"2023-10-06T05:01:22.916640+0000","last_became_peered":"2023-10-06T05:01:22.916640+0000","last_unstale":"2023-10-06T05:01:22.917870+0000","last_undegraded":"2023-10-06T05:01:22.917870+0000","last_fullsized":"2023-10-06T05:01:22.917870+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:09.844106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891290+0000","last_change":"2023-10-06T05:01:21.906767+0000","last_active":"2023-10-06T05:01:22.891290+0000","last_peered":"2023-10-06T05:01:22.891290+0000","last_clean":"2023-10-06T05:01:22.891290+0000","last_became_active":"2023-10-06T05:01:21.906476+0000","last_became_peered":"2023-10-06T05:01:21.906476+0000","last_unstale":"2023-10-06T05:01:22.891290+0000","last_undegraded":"2023-10-06T05:01:22.891290+0000","last_fullsized":"2023-10-06T05:01:22.891290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900665+0000","last_change":"2023-10-06T05:01:21.910269+0000","last_active":"2023-10-06T05:01:22.900665+0000","last_peered":"2023-10-06T05:01:22.900665+0000","last_clean":"2023-10-06T05:01:22.900665+0000","last_became_active":"2023-10-06T05:01:21.909971+0000","last_became_peered":"2023-10-06T05:01:21.909971+0000","last_unstale":"2023-10-06T05:01:22.900665+0000","last_undegraded":"2023-10-06T05:01:22.900665+0000","last_fullsized":"2023-10-06T05:01:22.900665+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.894440+0000","last_change":"2023-10-06T05:01:21.902093+0000","last_active":"2023-10-06T05:01:22.894440+0000","last_peered":"2023-10-06T05:01:22.894440+0000","last_clean":"2023-10-06T05:01:22.894440+0000","last_became_active":"2023-10-06T05:01:21.900927+0000","last_became_peered":"2023-10-06T05:01:21.900927+0000","last_unstale":"2023-10-06T05:01:22.894440+0000","last_undegraded":"2023-10-06T05:01:22.894440+0000","last_fullsized":"2023-10-06T05:01:22.894440+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:16.639594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918752+0000","last_change":"2023-10-06T05:01:22.917989+0000","last_active":"2023-10-06T05:01:22.918752+0000","last_peered":"2023-10-06T05:01:22.918752+0000","last_clean":"2023-10-06T05:01:22.918752+0000","last_became_active":"2023-10-06T05:01:22.917665+0000","last_became_peered":"2023-10-06T05:01:22.917665+0000","last_unstale":"2023-10-06T05:01:22.918752+0000","last_undegraded":"2023-10-06T05:01:22.918752+0000","last_fullsized":"2023-10-06T05:01:22.918752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:10.275597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900723+0000","last_change":"2023-10-06T05:01:21.908132+0000","last_active":"2023-10-06T05:01:22.900723+0000","last_peered":"2023-10-06T05:01:22.900723+0000","last_clean":"2023-10-06T05:01:22.900723+0000","last_became_active":"2023-10-06T05:01:21.907727+0000","last_became_peered":"2023-10-06T05:01:21.907727+0000","last_unstale":"2023-10-06T05:01:22.900723+0000","last_undegraded":"2023-10-06T05:01:22.900723+0000","last_fullsized":"2023-10-06T05:01:22.900723+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:19.166059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898029+0000","last_change":"2023-10-06T05:01:21.908600+0000","last_active":"2023-10-06T05:01:22.898029+0000","last_peered":"2023-10-06T05:01:22.898029+0000","last_clean":"2023-10-06T05:01:22.898029+0000","last_became_active":"2023-10-06T05:01:21.908261+0000","last_became_peered":"2023-10-06T05:01:21.908261+0000","last_unstale":"2023-10-06T05:01:22.898029+0000","last_undegraded":"2023-10-06T05:01:22.898029+0000","last_fullsized":"2023-10-06T05:01:22.898029+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:23.366677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910946+0000","last_change":"2023-10-06T05:01:22.906979+0000","last_active":"2023-10-06T05:01:22.910946+0000","last_peered":"2023-10-06T05:01:22.910946+0000","last_clean":"2023-10-06T05:01:22.910946+0000","last_became_active":"2023-10-06T05:01:22.906652+0000","last_became_peered":"2023-10-06T05:01:22.906652+0000","last_unstale":"2023-10-06T05:01:22.910946+0000","last_undegraded":"2023-10-06T05:01:22.910946+0000","last_fullsized":"2023-10-06T05:01:22.910946+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:42.125134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:01:17.629295+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:01:17.629295+0000","last_peered":"2023-10-06T05:01:17.629295+0000","last_clean":"2023-10-06T05:01:17.629295+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:01:17.629295+0000","last_undegraded":"2023-10-06T05:01:17.629295+0000","last_fullsized":"2023-10-06T05:01:17.629295+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912616+0000","last_change":"2023-10-06T05:01:22.911622+0000","last_active":"2023-10-06T05:01:22.912616+0000","last_peered":"2023-10-06T05:01:22.912616+0000","last_clean":"2023-10-06T05:01:22.912616+0000","last_became_active":"2023-10-06T05:01:22.911388+0000","last_became_peered":"2023-10-06T05:01:22.911388+0000","last_unstale":"2023-10-06T05:01:22.912616+0000","last_undegraded":"2023-10-06T05:01:22.912616+0000","last_fullsized":"2023-10-06T05:01:22.912616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.004036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920959+0000","last_change":"2023-10-06T05:01:21.907272+0000","last_active":"2023-10-06T05:01:22.920959+0000","last_peered":"2023-10-06T05:01:22.920959+0000","last_clean":"2023-10-06T05:01:22.920959+0000","last_became_active":"2023-10-06T05:01:21.906971+0000","last_became_peered":"2023-10-06T05:01:21.906971+0000","last_unstale":"2023-10-06T05:01:22.920959+0000","last_undegraded":"2023-10-06T05:01:22.920959+0000","last_fullsized":"2023-10-06T05:01:22.920959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891246+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:01:22.891246+0000","last_peered":"2023-10-06T05:01:22.891246+0000","last_clean":"2023-10-06T05:01:22.891246+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:01:22.891246+0000","last_undegraded":"2023-10-06T05:01:22.891246+0000","last_fullsized":"2023-10-06T05:01:22.891246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.902904+0000","last_change":"2023-10-06T05:01:21.905784+0000","last_active":"2023-10-06T05:01:22.902904+0000","last_peered":"2023-10-06T05:01:22.902904+0000","last_clean":"2023-10-06T05:01:22.902904+0000","last_became_active":"2023-10-06T05:01:21.905522+0000","last_became_peered":"2023-10-06T05:01:21.905522+0000","last_unstale":"2023-10-06T05:01:22.902904+0000","last_undegraded":"2023-10-06T05:01:22.902904+0000","last_fullsized":"2023-10-06T05:01:22.902904+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:31.320712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":107,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897470+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:01:22.897470+0000","last_peered":"2023-10-06T05:01:22.897470+0000","last_clean":"2023-10-06T05:01:22.897470+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:01:22.897470+0000","last_undegraded":"2023-10-06T05:01:22.897470+0000","last_fullsized":"2023-10-06T05:01:22.897470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":254,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898054+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:01:22.898054+0000","last_peered":"2023-10-06T05:01:22.898054+0000","last_clean":"2023-10-06T05:01:22.898054+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:01:22.898054+0000","last_undegraded":"2023-10-06T05:01:22.898054+0000","last_fullsized":"2023-10-06T05:01:22.898054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915904+0000","last_change":"2023-10-06T05:01:22.914069+0000","last_active":"2023-10-06T05:01:22.915904+0000","last_peered":"2023-10-06T05:01:22.915904+0000","last_clean":"2023-10-06T05:01:22.915904+0000","last_became_active":"2023-10-06T05:01:22.913789+0000","last_became_peered":"2023-10-06T05:01:22.913789+0000","last_unstale":"2023-10-06T05:01:22.915904+0000","last_undegraded":"2023-10-06T05:01:22.915904+0000","last_fullsized":"2023-10-06T05:01:22.915904+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:37.179875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900942+0000","last_change":"2023-10-06T05:01:21.907735+0000","last_active":"2023-10-06T05:01:22.900942+0000","last_peered":"2023-10-06T05:01:22.900942+0000","last_clean":"2023-10-06T05:01:22.900942+0000","last_became_active":"2023-10-06T05:01:21.907455+0000","last_became_peered":"2023-10-06T05:01:21.907455+0000","last_unstale":"2023-10-06T05:01:22.900942+0000","last_undegraded":"2023-10-06T05:01:22.900942+0000","last_fullsized":"2023-10-06T05:01:22.900942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:50.355734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897481+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:01:22.897481+0000","last_peered":"2023-10-06T05:01:22.897481+0000","last_clean":"2023-10-06T05:01:22.897481+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:01:22.897481+0000","last_undegraded":"2023-10-06T05:01:22.897481+0000","last_fullsized":"2023-10-06T05:01:22.897481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.925929+0000","last_change":"2023-10-06T05:01:21.900909+0000","last_active":"2023-10-06T05:01:22.925929+0000","last_peered":"2023-10-06T05:01:22.925929+0000","last_clean":"2023-10-06T05:01:22.925929+0000","last_became_active":"2023-10-06T05:01:21.900571+0000","last_became_peered":"2023-10-06T05:01:21.900571+0000","last_unstale":"2023-10-06T05:01:22.925929+0000","last_undegraded":"2023-10-06T05:01:22.925929+0000","last_fullsized":"2023-10-06T05:01:22.925929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898113+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:01:22.898113+0000","last_peered":"2023-10-06T05:01:22.898113+0000","last_clean":"2023-10-06T05:01:22.898113+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:01:22.898113+0000","last_undegraded":"2023-10-06T05:01:22.898113+0000","last_fullsized":"2023-10-06T05:01:22.898113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912543+0000","last_change":"2023-10-06T05:01:22.910384+0000","last_active":"2023-10-06T05:01:22.912543+0000","last_peered":"2023-10-06T05:01:22.912543+0000","last_clean":"2023-10-06T05:01:22.912543+0000","last_became_active":"2023-10-06T05:01:22.909894+0000","last_became_peered":"2023-10-06T05:01:22.909894+0000","last_unstale":"2023-10-06T05:01:22.912543+0000","last_undegraded":"2023-10-06T05:01:22.912543+0000","last_fullsized":"2023-10-06T05:01:22.912543+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:03.895804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909184+0000","last_change":"2023-10-06T05:01:21.898111+0000","last_active":"2023-10-06T05:01:22.909184+0000","last_peered":"2023-10-06T05:01:22.909184+0000","last_clean":"2023-10-06T05:01:22.909184+0000","last_became_active":"2023-10-06T05:01:21.897816+0000","last_became_peered":"2023-10-06T05:01:21.897816+0000","last_unstale":"2023-10-06T05:01:22.909184+0000","last_undegraded":"2023-10-06T05:01:22.909184+0000","last_fullsized":"2023-10-06T05:01:22.909184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:05.782643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900952+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:01:22.900952+0000","last_peered":"2023-10-06T05:01:22.900952+0000","last_clean":"2023-10-06T05:01:22.900952+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:01:22.900952+0000","last_undegraded":"2023-10-06T05:01:22.900952+0000","last_fullsized":"2023-10-06T05:01:22.900952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921053+0000","last_change":"2023-10-06T05:01:22.920569+0000","last_active":"2023-10-06T05:01:22.921053+0000","last_peered":"2023-10-06T05:01:22.921053+0000","last_clean":"2023-10-06T05:01:22.921053+0000","last_became_active":"2023-10-06T05:01:22.920240+0000","last_became_peered":"2023-10-06T05:01:22.920240+0000","last_unstale":"2023-10-06T05:01:22.921053+0000","last_undegraded":"2023-10-06T05:01:22.921053+0000","last_fullsized":"2023-10-06T05:01:22.921053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:01:17.785309+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:01:17.785309+0000","last_peered":"2023-10-06T05:01:17.785309+0000","last_clean":"2023-10-06T05:01:17.785309+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:01:17.785309+0000","last_undegraded":"2023-10-06T05:01:17.785309+0000","last_fullsized":"2023-10-06T05:01:17.785309+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909478+0000","last_change":"2023-10-06T05:01:21.905554+0000","last_active":"2023-10-06T05:01:22.909478+0000","last_peered":"2023-10-06T05:01:22.909478+0000","last_clean":"2023-10-06T05:01:22.909478+0000","last_became_active":"2023-10-06T05:01:21.905222+0000","last_became_peered":"2023-10-06T05:01:21.905222+0000","last_unstale":"2023-10-06T05:01:22.909478+0000","last_undegraded":"2023-10-06T05:01:22.909478+0000","last_fullsized":"2023-10-06T05:01:22.909478+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:01:17.785294+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:01:17.785294+0000","last_peered":"2023-10-06T05:01:17.785294+0000","last_clean":"2023-10-06T05:01:17.785294+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:01:17.785294+0000","last_undegraded":"2023-10-06T05:01:17.785294+0000","last_fullsized":"2023-10-06T05:01:17.785294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":50,"acting":50,"num_store_stats":6},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":94,"acting":94,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.73899999999999999,"5min":0.86599999999999999,"15min":7.1779999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.73499999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.311},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":7.1879999999999997},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.073},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":1.1379999999999999},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.111},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.0880000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.874},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.214},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302952,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068888,"statfs":{"total":96636764160,"available":96326541312,"internally_reserved":0,"allocated":1605632,"data_stored":727237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58799999999999997,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.73399999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63400000000000001,"15min":0.57499999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.65500000000000003,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.77900000000000003,"5min":1.234,"15min":1.234},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.63400000000000001,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.628},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70099999999999996,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.755,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.755},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61199999999999999,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.68100000000000005,"5min":3.669,"15min":3.669},"last":0.503},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.79300000000000004,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35599999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.437,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":0.375},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.378,"15min":0.378},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.60499999999999998,"5min":0.68600000000000005,"15min":0.69399999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.622,"15min":0.70399999999999996},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.56000000000000005,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.47199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.77300000000000002,"5min":0.877,"15min":1.0780000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.998,"15min":0.998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.54400000000000004,"5min":0.496,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":1.022,"15min":1.0429999999999999},"last":0.83399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":755982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.59199999999999997,"5min":0.628,"15min":0.628},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42999999999999999,"15min":0.38800000000000001},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46100000000000002,"15min":0.443},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.98299999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.85799999999999998,"15min":0.92300000000000004},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.91700000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.93400000000000005,"15min":1.0109999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.52200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.97599999999999998,"15min":1.0329999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":0.995},"last":0.71499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":753880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.20300000000000001,"15min":0.186},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.048},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.39300000000000002},"min":{"1min":0.32300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.216},"max":{"1min":0.52400000000000002,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.219},"max":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.36499999999999999},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.90200000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.2649999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.88100000000000001,"15min":0.93100000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":0.88300000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.82499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307104,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064736,"statfs":{"total":96636764160,"available":96322289664,"internally_reserved":0,"allocated":1662976,"data_stored":754356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56399999999999995,"5min":0.68200000000000005,"15min":2.008},"last":0.22},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.53000000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":6.4370000000000003},"last":0.28299999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.224,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.67000000000000004,"15min":1.7649999999999999},"last":0.312},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":2.1970000000000001},"last":0.29999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42399999999999999,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.255},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":5.069},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.441,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":2.246},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59499999999999997,"15min":0.76100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":14.646000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":14.606999999999999},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.81399999999999995,"5min":0.94699999999999995,"15min":2.2959999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":6.4009999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.73299999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.90100000000000002,"15min":5.0309999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":0.87,"5min":1.012,"15min":2.2749999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.75800000000000001},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.92600000000000005,"15min":6.367},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":5},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307048,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064792,"statfs":{"total":96636764160,"available":96322347008,"internally_reserved":0,"allocated":1605632,"data_stored":727237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.71199999999999997,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.73299999999999998},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.76200000000000001},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.76700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.70999999999999996,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.64100000000000001},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.498},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.80000000000000004,"15min":5.415},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.64800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":4.202},"last":0.435},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.622,"15min":0.70399999999999996},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.754,"5min":1.044,"15min":4.2130000000000001},"last":0.62}]},{"osd":5,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.497},"min":{"1min":0.27800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69799999999999995,"5min":0.73399999999999999,"15min":1.1279999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.76900000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78700000000000003,"5min":0.86199999999999999,"15min":0.872},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.49399999999999999,"5min":0.375,"15min":0.313},"max":{"1min":0.76300000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307092,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064748,"statfs":{"total":96636764160,"available":96322301952,"internally_reserved":0,"allocated":1650688,"data_stored":752415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.20499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":4.9039999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.75800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57499999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":6.7949999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.26200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.86599999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.623},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.86599999999999999},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53800000000000003,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":7,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:01:27.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:01:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.183+0000 7fb605342700 1 Processor -- start 2023-10-06T05:01:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.183+0000 7fb605342700 1 -- start start 2023-10-06T05:01:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.183+0000 7fb605342700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb60015c8f0 0x7fb60015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.183+0000 7fb605342700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb6000d0870 0x7fb6000b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.183+0000 7fb605342700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000cf6c0 con 0x7fb60015ccc0 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.183+0000 7fb605342700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000cfb50 con 0x7fb6000d0870 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.183+0000 7fb605342700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60015e050 con 0x7fb60015c8f0 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb6000d0870 0x7fb6000b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb6000d0870 0x7fb6000b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50462/0 (socket says 172.21.15.3:50462) 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/2654700499 learned_addr learned my addr 172.21.15.3:0/2654700499 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5feffd700 1 --2- 172.21.15.3:0/2654700499 >> v2:172.21.15.153:3300/0 conn(0x7fb60015c8f0 0x7fb60015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/2654700499 >> v2:172.21.15.153:3300/0 conn(0x7fb60015c8f0 msgr2=0x7fb60015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 --2- 172.21.15.3:0/2654700499 >> v2:172.21.15.153:3300/0 conn(0x7fb60015c8f0 0x7fb60015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/2654700499 >> v1:172.21.15.153:6789/0 conn(0x7fb60015ccc0 legacy=0x7fb6000d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/2654700499 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000ceab0 con 0x7fb6000d0870 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5feffd700 1 --2- 172.21.15.3:0/2654700499 >> v2:172.21.15.153:3300/0 conn(0x7fb60015c8f0 0x7fb60015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5ff7fe700 1 --2- 172.21.15.3:0/2654700499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6000d0870 0x7fb6000b3ba0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009f20 tx=0x7fb5f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749fb56cf002f861 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5fe7fc700 1 -- 172.21.15.3:0/2654700499 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800e070 con 0x7fb6000d0870 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5fe7fc700 1 -- 172.21.15.3:0/2654700499 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f800ba00 con 0x7fb6000d0870 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.184+0000 7fb5fe7fc700 1 -- 172.21.15.3:0/2654700499 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80092e0 con 0x7fb6000d0870 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 -- 172.21.15.3:0/2654700499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6000d0870 msgr2=0x7fb6000b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 --2- 172.21.15.3:0/2654700499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6000d0870 0x7fb6000b3ba0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009f20 tx=0x7fb5f8009840 comp rx=0 tx=0).stop 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 -- 172.21.15.3:0/2654700499 shutdown_connections 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 --2- 172.21.15.3:0/2654700499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6000d0870 0x7fb6000b3ba0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 --2- 172.21.15.3:0/2654700499 >> v2:172.21.15.153:3300/0 conn(0x7fb60015c8f0 0x7fb60015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 -- 172.21.15.3:0/2654700499 >> 172.21.15.3:0/2654700499 conn(0x7fb600070310 msgr2=0x7fb600070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 -- 172.21.15.3:0/2654700499 shutdown_connections 2023-10-06T05:01:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.185+0000 7fb605342700 1 -- 172.21.15.3:0/2654700499 wait complete. 2023-10-06T05:01:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.186+0000 7fb605342700 1 Processor -- start 2023-10-06T05:01:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.186+0000 7fb605342700 1 -- start start 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb605342700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6000d0870 0x7fb6001571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb605342700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60015c8f0 0x7fb60016aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb605342700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb60015ccc0 0x7fb60014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb605342700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600169a30 con 0x7fb60015ccc0 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb605342700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60016d4b0 con 0x7fb60015c8f0 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb605342700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60016d790 con 0x7fb6000d0870 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb60015ccc0 0x7fb60014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb60015ccc0 0x7fb60014d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50470/0 (socket says 172.21.15.3:50470) 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/3891956931 learned_addr learned my addr 172.21.15.3:0/3891956931 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5feffd700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6000d0870 0x7fb6001571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5f7fff700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60015c8f0 0x7fb60016aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6000d0870 msgr2=0x7fb6001571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6000d0870 0x7fb6001571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60015c8f0 msgr2=0x7fb60016aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60015c8f0 0x7fb60016aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 -- 172.21.15.3:0/3891956931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009ae0 con 0x7fb60015ccc0 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5f7fff700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60015c8f0 0x7fb60016aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.187+0000 7fb5ff7fe700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb60015ccc0 0x7fb60014d8d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8003000 tx=0x7fb5f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.188+0000 7fb5f6ffd700 1 -- 172.21.15.3:0/3891956931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80032f0 con 0x7fb60015ccc0 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.188+0000 7fb5f6ffd700 1 -- 172.21.15.3:0/3891956931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f8012db0 con 0x7fb60015ccc0 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.188+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60016da70 con 0x7fb60015ccc0 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.188+0000 7fb5f6ffd700 1 -- 172.21.15.3:0/3891956931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f801d650 con 0x7fb60015ccc0 2023-10-06T05:01:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.188+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6001576f0 con 0x7fb60015ccc0 2023-10-06T05:01:27.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.188+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb600169c50 con 0x7fb60015ccc0 2023-10-06T05:01:27.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.189+0000 7fb5f6ffd700 1 -- 172.21.15.3:0/3891956931 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5f801da80 con 0x7fb60015ccc0 2023-10-06T05:01:27.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.189+0000 7fb5f6ffd700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb5e0066f30 0x7fb5e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.189+0000 7fb5f6ffd700 1 -- 172.21.15.3:0/3891956931 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5f8085e80 con 0x7fb60015ccc0 2023-10-06T05:01:27.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.190+0000 7fb5feffd700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb5e0066f30 0x7fb5e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.190+0000 7fb5feffd700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb5e0066f30 0x7fb5e00693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b1c50 tx=0x7fb5f0006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.194+0000 7fb5f6ffd700 1 -- 172.21.15.3:0/3891956931 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb600169c50 con 0x7fb60015ccc0 2023-10-06T05:01:27.202 INFO:tasks.ceph.mds.b.smithi153.stderr:2023-10-06T05:01:27.200+0000 7fedbe4b5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:01:27.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.383+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6001578e0 con 0x7fb5e0066f30 2023-10-06T05:01:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.392+0000 7fb5f6ffd700 1 -- 172.21.15.3:0/3891956931 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308531 (secure 0 0 0) 0x7fb6001578e0 con 0x7fb5e0066f30 2023-10-06T05:01:27.397 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:27.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb5e0066f30 msgr2=0x7fb5e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb5e0066f30 0x7fb5e00693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b1c50 tx=0x7fb5f0006f90 comp rx=0 tx=0).stop 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb60015ccc0 msgr2=0x7fb60014d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb60015ccc0 0x7fb60014d8d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8003000 tx=0x7fb5f8017000 comp rx=0 tx=0).stop 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 shutdown_connections 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb5e0066f30 0x7fb5e00693e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb60015ccc0 0x7fb60014d8d0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6000d0870 0x7fb6001571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 --2- 172.21.15.3:0/3891956931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60015c8f0 0x7fb60016aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 >> 172.21.15.3:0/3891956931 conn(0x7fb600070310 msgr2=0x7fb6000afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 shutdown_connections 2023-10-06T05:01:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.395+0000 7fb605342700 1 -- 172.21.15.3:0/3891956931 wait complete. 2023-10-06T05:01:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:01:27.409 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2023-10-06T05:01:25.558118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449676,"kb_used_data":14028,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752525044,"statfs":{"total":773094113280,"available":770585645056,"internally_reserved":0,"allocated":14364672,"data_stored":7156765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.679338"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919337+0000","last_change":"2023-10-06T05:01:22.917546+0000","last_active":"2023-10-06T05:01:22.919337+0000","last_peered":"2023-10-06T05:01:22.919337+0000","last_clean":"2023-10-06T05:01:22.919337+0000","last_became_active":"2023-10-06T05:01:22.916374+0000","last_became_peered":"2023-10-06T05:01:22.916374+0000","last_unstale":"2023-10-06T05:01:22.919337+0000","last_undegraded":"2023-10-06T05:01:22.919337+0000","last_fullsized":"2023-10-06T05:01:22.919337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917323+0000","last_change":"2023-10-06T05:01:22.915942+0000","last_active":"2023-10-06T05:01:22.917323+0000","last_peered":"2023-10-06T05:01:22.917323+0000","last_clean":"2023-10-06T05:01:22.917323+0000","last_became_active":"2023-10-06T05:01:22.915602+0000","last_became_peered":"2023-10-06T05:01:22.915602+0000","last_unstale":"2023-10-06T05:01:22.917323+0000","last_undegraded":"2023-10-06T05:01:22.917323+0000","last_fullsized":"2023-10-06T05:01:22.917323+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:39.759900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917901+0000","last_change":"2023-10-06T05:01:22.916859+0000","last_active":"2023-10-06T05:01:22.917901+0000","last_peered":"2023-10-06T05:01:22.917901+0000","last_clean":"2023-10-06T05:01:22.917901+0000","last_became_active":"2023-10-06T05:01:22.916504+0000","last_became_peered":"2023-10-06T05:01:22.916504+0000","last_unstale":"2023-10-06T05:01:22.917901+0000","last_undegraded":"2023-10-06T05:01:22.917901+0000","last_fullsized":"2023-10-06T05:01:22.917901+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:36.075945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912802+0000","last_change":"2023-10-06T05:01:22.908378+0000","last_active":"2023-10-06T05:01:22.912802+0000","last_peered":"2023-10-06T05:01:22.912802+0000","last_clean":"2023-10-06T05:01:22.912802+0000","last_became_active":"2023-10-06T05:01:22.907429+0000","last_became_peered":"2023-10-06T05:01:22.907429+0000","last_unstale":"2023-10-06T05:01:22.912802+0000","last_undegraded":"2023-10-06T05:01:22.912802+0000","last_fullsized":"2023-10-06T05:01:22.912802+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:32.443484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912753+0000","last_change":"2023-10-06T05:01:22.907760+0000","last_active":"2023-10-06T05:01:22.912753+0000","last_peered":"2023-10-06T05:01:22.912753+0000","last_clean":"2023-10-06T05:01:22.912753+0000","last_became_active":"2023-10-06T05:01:22.907466+0000","last_became_peered":"2023-10-06T05:01:22.907466+0000","last_unstale":"2023-10-06T05:01:22.912753+0000","last_undegraded":"2023-10-06T05:01:22.912753+0000","last_fullsized":"2023-10-06T05:01:22.912753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:21.629837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908240+0000","last_change":"2023-10-06T05:01:22.907593+0000","last_active":"2023-10-06T05:01:22.908240+0000","last_peered":"2023-10-06T05:01:22.908240+0000","last_clean":"2023-10-06T05:01:22.908240+0000","last_became_active":"2023-10-06T05:01:22.907290+0000","last_became_peered":"2023-10-06T05:01:22.907290+0000","last_unstale":"2023-10-06T05:01:22.908240+0000","last_undegraded":"2023-10-06T05:01:22.908240+0000","last_fullsized":"2023-10-06T05:01:22.908240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910976+0000","last_change":"2023-10-06T05:01:22.906539+0000","last_active":"2023-10-06T05:01:22.910976+0000","last_peered":"2023-10-06T05:01:22.910976+0000","last_clean":"2023-10-06T05:01:22.910976+0000","last_became_active":"2023-10-06T05:01:22.905906+0000","last_became_peered":"2023-10-06T05:01:22.905906+0000","last_unstale":"2023-10-06T05:01:22.910976+0000","last_undegraded":"2023-10-06T05:01:22.910976+0000","last_fullsized":"2023-10-06T05:01:22.910976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:33.022370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908499+0000","last_change":"2023-10-06T05:01:22.906059+0000","last_active":"2023-10-06T05:01:22.908499+0000","last_peered":"2023-10-06T05:01:22.908499+0000","last_clean":"2023-10-06T05:01:22.908499+0000","last_became_active":"2023-10-06T05:01:22.905782+0000","last_became_peered":"2023-10-06T05:01:22.905782+0000","last_unstale":"2023-10-06T05:01:22.908499+0000","last_undegraded":"2023-10-06T05:01:22.908499+0000","last_fullsized":"2023-10-06T05:01:22.908499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:31.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916007+0000","last_change":"2023-10-06T05:01:22.914642+0000","last_active":"2023-10-06T05:01:22.916007+0000","last_peered":"2023-10-06T05:01:22.916007+0000","last_clean":"2023-10-06T05:01:22.916007+0000","last_became_active":"2023-10-06T05:01:22.913781+0000","last_became_peered":"2023-10-06T05:01:22.913781+0000","last_unstale":"2023-10-06T05:01:22.916007+0000","last_undegraded":"2023-10-06T05:01:22.916007+0000","last_fullsized":"2023-10-06T05:01:22.916007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:39.375404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909396+0000","last_change":"2023-10-06T05:01:22.906295+0000","last_active":"2023-10-06T05:01:22.909396+0000","last_peered":"2023-10-06T05:01:22.909396+0000","last_clean":"2023-10-06T05:01:22.909396+0000","last_became_active":"2023-10-06T05:01:22.905241+0000","last_became_peered":"2023-10-06T05:01:22.905241+0000","last_unstale":"2023-10-06T05:01:22.909396+0000","last_undegraded":"2023-10-06T05:01:22.909396+0000","last_fullsized":"2023-10-06T05:01:22.909396+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:56.323569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921375+0000","last_change":"2023-10-06T05:01:22.920640+0000","last_active":"2023-10-06T05:01:22.921375+0000","last_peered":"2023-10-06T05:01:22.921375+0000","last_clean":"2023-10-06T05:01:22.921375+0000","last_became_active":"2023-10-06T05:01:22.920435+0000","last_became_peered":"2023-10-06T05:01:22.920435+0000","last_unstale":"2023-10-06T05:01:22.921375+0000","last_undegraded":"2023-10-06T05:01:22.921375+0000","last_fullsized":"2023-10-06T05:01:22.921375+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:21.298643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921735+0000","last_change":"2023-10-06T05:01:22.921275+0000","last_active":"2023-10-06T05:01:22.921735+0000","last_peered":"2023-10-06T05:01:22.921735+0000","last_clean":"2023-10-06T05:01:22.921735+0000","last_became_active":"2023-10-06T05:01:22.920953+0000","last_became_peered":"2023-10-06T05:01:22.920953+0000","last_unstale":"2023-10-06T05:01:22.921735+0000","last_undegraded":"2023-10-06T05:01:22.921735+0000","last_fullsized":"2023-10-06T05:01:22.921735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:22.455454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912183+0000","last_change":"2023-10-06T05:01:22.910088+0000","last_active":"2023-10-06T05:01:22.912183+0000","last_peered":"2023-10-06T05:01:22.912183+0000","last_clean":"2023-10-06T05:01:22.912183+0000","last_became_active":"2023-10-06T05:01:22.909830+0000","last_became_peered":"2023-10-06T05:01:22.909830+0000","last_unstale":"2023-10-06T05:01:22.912183+0000","last_undegraded":"2023-10-06T05:01:22.912183+0000","last_fullsized":"2023-10-06T05:01:22.912183+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:14.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920682+0000","last_change":"2023-10-06T05:01:22.915966+0000","last_active":"2023-10-06T05:01:22.920682+0000","last_peered":"2023-10-06T05:01:22.920682+0000","last_clean":"2023-10-06T05:01:22.920682+0000","last_became_active":"2023-10-06T05:01:22.915477+0000","last_became_peered":"2023-10-06T05:01:22.915477+0000","last_unstale":"2023-10-06T05:01:22.920682+0000","last_undegraded":"2023-10-06T05:01:22.920682+0000","last_fullsized":"2023-10-06T05:01:22.920682+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:24.762820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908898+0000","last_change":"2023-10-06T05:01:22.905656+0000","last_active":"2023-10-06T05:01:22.908898+0000","last_peered":"2023-10-06T05:01:22.908898+0000","last_clean":"2023-10-06T05:01:22.908898+0000","last_became_active":"2023-10-06T05:01:22.905402+0000","last_became_peered":"2023-10-06T05:01:22.905402+0000","last_unstale":"2023-10-06T05:01:22.908898+0000","last_undegraded":"2023-10-06T05:01:22.908898+0000","last_fullsized":"2023-10-06T05:01:22.908898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.573131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912788+0000","last_change":"2023-10-06T05:01:22.908184+0000","last_active":"2023-10-06T05:01:22.912788+0000","last_peered":"2023-10-06T05:01:22.912788+0000","last_clean":"2023-10-06T05:01:22.912788+0000","last_became_active":"2023-10-06T05:01:22.904908+0000","last_became_peered":"2023-10-06T05:01:22.904908+0000","last_unstale":"2023-10-06T05:01:22.912788+0000","last_undegraded":"2023-10-06T05:01:22.912788+0000","last_fullsized":"2023-10-06T05:01:22.912788+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:42.892786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912280+0000","last_change":"2023-10-06T05:01:22.908024+0000","last_active":"2023-10-06T05:01:22.912280+0000","last_peered":"2023-10-06T05:01:22.912280+0000","last_clean":"2023-10-06T05:01:22.912280+0000","last_became_active":"2023-10-06T05:01:22.907701+0000","last_became_peered":"2023-10-06T05:01:22.907701+0000","last_unstale":"2023-10-06T05:01:22.912280+0000","last_undegraded":"2023-10-06T05:01:22.912280+0000","last_fullsized":"2023-10-06T05:01:22.912280+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910490+0000","last_change":"2023-10-06T05:01:22.902838+0000","last_active":"2023-10-06T05:01:22.910490+0000","last_peered":"2023-10-06T05:01:22.910490+0000","last_clean":"2023-10-06T05:01:22.910490+0000","last_became_active":"2023-10-06T05:01:22.902549+0000","last_became_peered":"2023-10-06T05:01:22.902549+0000","last_unstale":"2023-10-06T05:01:22.910490+0000","last_undegraded":"2023-10-06T05:01:22.910490+0000","last_fullsized":"2023-10-06T05:01:22.910490+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:26.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912308+0000","last_change":"2023-10-06T05:01:22.908586+0000","last_active":"2023-10-06T05:01:22.912308+0000","last_peered":"2023-10-06T05:01:22.912308+0000","last_clean":"2023-10-06T05:01:22.912308+0000","last_became_active":"2023-10-06T05:01:22.908289+0000","last_became_peered":"2023-10-06T05:01:22.908289+0000","last_unstale":"2023-10-06T05:01:22.912308+0000","last_undegraded":"2023-10-06T05:01:22.912308+0000","last_fullsized":"2023-10-06T05:01:22.912308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:38.888355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914573+0000","last_change":"2023-10-06T05:01:22.910843+0000","last_active":"2023-10-06T05:01:22.914573+0000","last_peered":"2023-10-06T05:01:22.914573+0000","last_clean":"2023-10-06T05:01:22.914573+0000","last_became_active":"2023-10-06T05:01:22.910470+0000","last_became_peered":"2023-10-06T05:01:22.910470+0000","last_unstale":"2023-10-06T05:01:22.914573+0000","last_undegraded":"2023-10-06T05:01:22.914573+0000","last_fullsized":"2023-10-06T05:01:22.914573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912812+0000","last_change":"2023-10-06T05:01:22.908293+0000","last_active":"2023-10-06T05:01:22.912812+0000","last_peered":"2023-10-06T05:01:22.912812+0000","last_clean":"2023-10-06T05:01:22.912812+0000","last_became_active":"2023-10-06T05:01:22.907986+0000","last_became_peered":"2023-10-06T05:01:22.907986+0000","last_unstale":"2023-10-06T05:01:22.912812+0000","last_undegraded":"2023-10-06T05:01:22.912812+0000","last_fullsized":"2023-10-06T05:01:22.912812+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:58.242153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910601+0000","last_change":"2023-10-06T05:01:22.905470+0000","last_active":"2023-10-06T05:01:22.910601+0000","last_peered":"2023-10-06T05:01:22.910601+0000","last_clean":"2023-10-06T05:01:22.910601+0000","last_became_active":"2023-10-06T05:01:22.905162+0000","last_became_peered":"2023-10-06T05:01:22.905162+0000","last_unstale":"2023-10-06T05:01:22.910601+0000","last_undegraded":"2023-10-06T05:01:22.910601+0000","last_fullsized":"2023-10-06T05:01:22.910601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:53.259645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912590+0000","last_change":"2023-10-06T05:01:22.911945+0000","last_active":"2023-10-06T05:01:22.912590+0000","last_peered":"2023-10-06T05:01:22.912590+0000","last_clean":"2023-10-06T05:01:22.912590+0000","last_became_active":"2023-10-06T05:01:22.911675+0000","last_became_peered":"2023-10-06T05:01:22.911675+0000","last_unstale":"2023-10-06T05:01:22.912590+0000","last_undegraded":"2023-10-06T05:01:22.912590+0000","last_fullsized":"2023-10-06T05:01:22.912590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:32.223585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914827+0000","last_change":"2023-10-06T05:01:22.913460+0000","last_active":"2023-10-06T05:01:22.914827+0000","last_peered":"2023-10-06T05:01:22.914827+0000","last_clean":"2023-10-06T05:01:22.914827+0000","last_became_active":"2023-10-06T05:01:22.913178+0000","last_became_peered":"2023-10-06T05:01:22.913178+0000","last_unstale":"2023-10-06T05:01:22.914827+0000","last_undegraded":"2023-10-06T05:01:22.914827+0000","last_fullsized":"2023-10-06T05:01:22.914827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:16.124524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909358+0000","last_change":"2023-10-06T05:01:22.906868+0000","last_active":"2023-10-06T05:01:22.909358+0000","last_peered":"2023-10-06T05:01:22.909358+0000","last_clean":"2023-10-06T05:01:22.909358+0000","last_became_active":"2023-10-06T05:01:22.906368+0000","last_became_peered":"2023-10-06T05:01:22.906368+0000","last_unstale":"2023-10-06T05:01:22.909358+0000","last_undegraded":"2023-10-06T05:01:22.909358+0000","last_fullsized":"2023-10-06T05:01:22.909358+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:45.514209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910892+0000","last_change":"2023-10-06T05:01:21.905167+0000","last_active":"2023-10-06T05:01:22.910892+0000","last_peered":"2023-10-06T05:01:22.910892+0000","last_clean":"2023-10-06T05:01:22.910892+0000","last_became_active":"2023-10-06T05:01:21.904856+0000","last_became_peered":"2023-10-06T05:01:21.904856+0000","last_unstale":"2023-10-06T05:01:22.910892+0000","last_undegraded":"2023-10-06T05:01:22.910892+0000","last_fullsized":"2023-10-06T05:01:22.910892+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916718+0000","last_change":"2023-10-06T05:01:22.915682+0000","last_active":"2023-10-06T05:01:22.916718+0000","last_peered":"2023-10-06T05:01:22.916718+0000","last_clean":"2023-10-06T05:01:22.916718+0000","last_became_active":"2023-10-06T05:01:22.915425+0000","last_became_peered":"2023-10-06T05:01:22.915425+0000","last_unstale":"2023-10-06T05:01:22.916718+0000","last_undegraded":"2023-10-06T05:01:22.916718+0000","last_fullsized":"2023-10-06T05:01:22.916718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913713+0000","last_change":"2023-10-06T05:01:21.909299+0000","last_active":"2023-10-06T05:01:22.913713+0000","last_peered":"2023-10-06T05:01:22.913713+0000","last_clean":"2023-10-06T05:01:22.913713+0000","last_became_active":"2023-10-06T05:01:21.908989+0000","last_became_peered":"2023-10-06T05:01:21.908989+0000","last_unstale":"2023-10-06T05:01:22.913713+0000","last_undegraded":"2023-10-06T05:01:22.913713+0000","last_fullsized":"2023-10-06T05:01:22.913713+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914828+0000","last_change":"2023-10-06T05:01:21.909301+0000","last_active":"2023-10-06T05:01:22.914828+0000","last_peered":"2023-10-06T05:01:22.914828+0000","last_clean":"2023-10-06T05:01:22.914828+0000","last_became_active":"2023-10-06T05:01:21.908984+0000","last_became_peered":"2023-10-06T05:01:21.908984+0000","last_unstale":"2023-10-06T05:01:22.914828+0000","last_undegraded":"2023-10-06T05:01:22.914828+0000","last_fullsized":"2023-10-06T05:01:22.914828+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:26.376744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920398+0000","last_change":"2023-10-06T05:01:22.910823+0000","last_active":"2023-10-06T05:01:22.920398+0000","last_peered":"2023-10-06T05:01:22.920398+0000","last_clean":"2023-10-06T05:01:22.920398+0000","last_became_active":"2023-10-06T05:01:22.910158+0000","last_became_peered":"2023-10-06T05:01:22.910158+0000","last_unstale":"2023-10-06T05:01:22.920398+0000","last_undegraded":"2023-10-06T05:01:22.920398+0000","last_fullsized":"2023-10-06T05:01:22.920398+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:33.254913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917199+0000","last_change":"2023-10-06T05:01:21.898248+0000","last_active":"2023-10-06T05:01:22.917199+0000","last_peered":"2023-10-06T05:01:22.917199+0000","last_clean":"2023-10-06T05:01:22.917199+0000","last_became_active":"2023-10-06T05:01:21.897180+0000","last_became_peered":"2023-10-06T05:01:21.897180+0000","last_unstale":"2023-10-06T05:01:22.917199+0000","last_undegraded":"2023-10-06T05:01:22.917199+0000","last_fullsized":"2023-10-06T05:01:22.917199+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:59.743528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909059+0000","last_change":"2023-10-06T05:01:22.905942+0000","last_active":"2023-10-06T05:01:22.909059+0000","last_peered":"2023-10-06T05:01:22.909059+0000","last_clean":"2023-10-06T05:01:22.909059+0000","last_became_active":"2023-10-06T05:01:22.905653+0000","last_became_peered":"2023-10-06T05:01:22.905653+0000","last_unstale":"2023-10-06T05:01:22.909059+0000","last_undegraded":"2023-10-06T05:01:22.909059+0000","last_fullsized":"2023-10-06T05:01:22.909059+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:12.300608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916407+0000","last_change":"2023-10-06T05:01:21.906758+0000","last_active":"2023-10-06T05:01:22.916407+0000","last_peered":"2023-10-06T05:01:22.916407+0000","last_clean":"2023-10-06T05:01:22.916407+0000","last_became_active":"2023-10-06T05:01:21.906463+0000","last_became_peered":"2023-10-06T05:01:21.906463+0000","last_unstale":"2023-10-06T05:01:22.916407+0000","last_undegraded":"2023-10-06T05:01:22.916407+0000","last_fullsized":"2023-10-06T05:01:22.916407+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:42.205607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908789+0000","last_change":"2023-10-06T05:01:22.905020+0000","last_active":"2023-10-06T05:01:22.908789+0000","last_peered":"2023-10-06T05:01:22.908789+0000","last_clean":"2023-10-06T05:01:22.908789+0000","last_became_active":"2023-10-06T05:01:22.904762+0000","last_became_peered":"2023-10-06T05:01:22.904762+0000","last_unstale":"2023-10-06T05:01:22.908789+0000","last_undegraded":"2023-10-06T05:01:22.908789+0000","last_fullsized":"2023-10-06T05:01:22.908789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900865+0000","last_change":"2023-10-06T05:01:21.907467+0000","last_active":"2023-10-06T05:01:22.900865+0000","last_peered":"2023-10-06T05:01:22.900865+0000","last_clean":"2023-10-06T05:01:22.900865+0000","last_became_active":"2023-10-06T05:01:21.907169+0000","last_became_peered":"2023-10-06T05:01:21.907169+0000","last_unstale":"2023-10-06T05:01:22.900865+0000","last_undegraded":"2023-10-06T05:01:22.900865+0000","last_fullsized":"2023-10-06T05:01:22.900865+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.309675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918810+0000","last_change":"2023-10-06T05:01:22.916673+0000","last_active":"2023-10-06T05:01:22.918810+0000","last_peered":"2023-10-06T05:01:22.918810+0000","last_clean":"2023-10-06T05:01:22.918810+0000","last_became_active":"2023-10-06T05:01:22.916393+0000","last_became_peered":"2023-10-06T05:01:22.916393+0000","last_unstale":"2023-10-06T05:01:22.918810+0000","last_undegraded":"2023-10-06T05:01:22.918810+0000","last_fullsized":"2023-10-06T05:01:22.918810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:00.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.890628+0000","last_change":"2023-10-06T05:01:21.904853+0000","last_active":"2023-10-06T05:01:22.890628+0000","last_peered":"2023-10-06T05:01:22.890628+0000","last_clean":"2023-10-06T05:01:22.890628+0000","last_became_active":"2023-10-06T05:01:21.904448+0000","last_became_peered":"2023-10-06T05:01:21.904448+0000","last_unstale":"2023-10-06T05:01:22.890628+0000","last_undegraded":"2023-10-06T05:01:22.890628+0000","last_fullsized":"2023-10-06T05:01:22.890628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.567032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915557+0000","last_change":"2023-10-06T05:01:22.909027+0000","last_active":"2023-10-06T05:01:22.915557+0000","last_peered":"2023-10-06T05:01:22.915557+0000","last_clean":"2023-10-06T05:01:22.915557+0000","last_became_active":"2023-10-06T05:01:22.908783+0000","last_became_peered":"2023-10-06T05:01:22.908783+0000","last_unstale":"2023-10-06T05:01:22.915557+0000","last_undegraded":"2023-10-06T05:01:22.915557+0000","last_fullsized":"2023-10-06T05:01:22.915557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:48.927255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914458+0000","last_change":"2023-10-06T05:01:21.901342+0000","last_active":"2023-10-06T05:01:22.914458+0000","last_peered":"2023-10-06T05:01:22.914458+0000","last_clean":"2023-10-06T05:01:22.914458+0000","last_became_active":"2023-10-06T05:01:21.901020+0000","last_became_peered":"2023-10-06T05:01:21.901020+0000","last_unstale":"2023-10-06T05:01:22.914458+0000","last_undegraded":"2023-10-06T05:01:22.914458+0000","last_fullsized":"2023-10-06T05:01:22.914458+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:26.399481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920662+0000","last_change":"2023-10-06T05:01:22.915799+0000","last_active":"2023-10-06T05:01:22.920662+0000","last_peered":"2023-10-06T05:01:22.920662+0000","last_clean":"2023-10-06T05:01:22.920662+0000","last_became_active":"2023-10-06T05:01:22.915224+0000","last_became_peered":"2023-10-06T05:01:22.915224+0000","last_unstale":"2023-10-06T05:01:22.920662+0000","last_undegraded":"2023-10-06T05:01:22.920662+0000","last_fullsized":"2023-10-06T05:01:22.920662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:24.443047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909917+0000","last_change":"2023-10-06T05:01:21.903666+0000","last_active":"2023-10-06T05:01:22.909917+0000","last_peered":"2023-10-06T05:01:22.909917+0000","last_clean":"2023-10-06T05:01:22.909917+0000","last_became_active":"2023-10-06T05:01:21.903416+0000","last_became_peered":"2023-10-06T05:01:21.903416+0000","last_unstale":"2023-10-06T05:01:22.909917+0000","last_undegraded":"2023-10-06T05:01:22.909917+0000","last_fullsized":"2023-10-06T05:01:22.909917+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:38.753801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913218+0000","last_change":"2023-10-06T05:01:22.908994+0000","last_active":"2023-10-06T05:01:22.913218+0000","last_peered":"2023-10-06T05:01:22.913218+0000","last_clean":"2023-10-06T05:01:22.913218+0000","last_became_active":"2023-10-06T05:01:22.908627+0000","last_became_peered":"2023-10-06T05:01:22.908627+0000","last_unstale":"2023-10-06T05:01:22.913218+0000","last_undegraded":"2023-10-06T05:01:22.913218+0000","last_fullsized":"2023-10-06T05:01:22.913218+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920770+0000","last_change":"2023-10-06T05:01:21.901636+0000","last_active":"2023-10-06T05:01:22.920770+0000","last_peered":"2023-10-06T05:01:22.920770+0000","last_clean":"2023-10-06T05:01:22.920770+0000","last_became_active":"2023-10-06T05:01:21.901348+0000","last_became_peered":"2023-10-06T05:01:21.901348+0000","last_unstale":"2023-10-06T05:01:22.920770+0000","last_undegraded":"2023-10-06T05:01:22.920770+0000","last_fullsized":"2023-10-06T05:01:22.920770+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:07.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919815+0000","last_change":"2023-10-06T05:01:22.918705+0000","last_active":"2023-10-06T05:01:22.919815+0000","last_peered":"2023-10-06T05:01:22.919815+0000","last_clean":"2023-10-06T05:01:22.919815+0000","last_became_active":"2023-10-06T05:01:22.918437+0000","last_became_peered":"2023-10-06T05:01:22.918437+0000","last_unstale":"2023-10-06T05:01:22.919815+0000","last_undegraded":"2023-10-06T05:01:22.919815+0000","last_fullsized":"2023-10-06T05:01:22.919815+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920786+0000","last_change":"2023-10-06T05:01:22.919693+0000","last_active":"2023-10-06T05:01:22.920786+0000","last_peered":"2023-10-06T05:01:22.920786+0000","last_clean":"2023-10-06T05:01:22.920786+0000","last_became_active":"2023-10-06T05:01:22.919445+0000","last_became_peered":"2023-10-06T05:01:22.919445+0000","last_unstale":"2023-10-06T05:01:22.920786+0000","last_undegraded":"2023-10-06T05:01:22.920786+0000","last_fullsized":"2023-10-06T05:01:22.920786+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:00.412186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916231+0000","last_change":"2023-10-06T05:01:21.901798+0000","last_active":"2023-10-06T05:01:22.916231+0000","last_peered":"2023-10-06T05:01:22.916231+0000","last_clean":"2023-10-06T05:01:22.916231+0000","last_became_active":"2023-10-06T05:01:21.901536+0000","last_became_peered":"2023-10-06T05:01:21.901536+0000","last_unstale":"2023-10-06T05:01:22.916231+0000","last_undegraded":"2023-10-06T05:01:22.916231+0000","last_fullsized":"2023-10-06T05:01:22.916231+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909492+0000","last_change":"2023-10-06T05:01:22.908931+0000","last_active":"2023-10-06T05:01:22.909492+0000","last_peered":"2023-10-06T05:01:22.909492+0000","last_clean":"2023-10-06T05:01:22.909492+0000","last_became_active":"2023-10-06T05:01:22.908678+0000","last_became_peered":"2023-10-06T05:01:22.908678+0000","last_unstale":"2023-10-06T05:01:22.909492+0000","last_undegraded":"2023-10-06T05:01:22.909492+0000","last_fullsized":"2023-10-06T05:01:22.909492+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:24.372776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920281+0000","last_change":"2023-10-06T05:01:22.910581+0000","last_active":"2023-10-06T05:01:22.920281+0000","last_peered":"2023-10-06T05:01:22.920281+0000","last_clean":"2023-10-06T05:01:22.920281+0000","last_became_active":"2023-10-06T05:01:22.909963+0000","last_became_peered":"2023-10-06T05:01:22.909963+0000","last_unstale":"2023-10-06T05:01:22.920281+0000","last_undegraded":"2023-10-06T05:01:22.920281+0000","last_fullsized":"2023-10-06T05:01:22.920281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:41.453873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.893893+0000","last_change":"2023-10-06T05:01:21.900893+0000","last_active":"2023-10-06T05:01:22.893893+0000","last_peered":"2023-10-06T05:01:22.893893+0000","last_clean":"2023-10-06T05:01:22.893893+0000","last_became_active":"2023-10-06T05:01:21.900591+0000","last_became_peered":"2023-10-06T05:01:21.900591+0000","last_unstale":"2023-10-06T05:01:22.893893+0000","last_undegraded":"2023-10-06T05:01:22.893893+0000","last_fullsized":"2023-10-06T05:01:22.893893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917896+0000","last_change":"2023-10-06T05:01:22.917013+0000","last_active":"2023-10-06T05:01:22.917896+0000","last_peered":"2023-10-06T05:01:22.917896+0000","last_clean":"2023-10-06T05:01:22.917896+0000","last_became_active":"2023-10-06T05:01:22.915500+0000","last_became_peered":"2023-10-06T05:01:22.915500+0000","last_unstale":"2023-10-06T05:01:22.917896+0000","last_undegraded":"2023-10-06T05:01:22.917896+0000","last_fullsized":"2023-10-06T05:01:22.917896+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912743+0000","last_change":"2023-10-06T05:01:22.907709+0000","last_active":"2023-10-06T05:01:22.912743+0000","last_peered":"2023-10-06T05:01:22.912743+0000","last_clean":"2023-10-06T05:01:22.912743+0000","last_became_active":"2023-10-06T05:01:22.907411+0000","last_became_peered":"2023-10-06T05:01:22.907411+0000","last_unstale":"2023-10-06T05:01:22.912743+0000","last_undegraded":"2023-10-06T05:01:22.912743+0000","last_fullsized":"2023-10-06T05:01:22.912743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:19.960229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.901055+0000","last_change":"2023-10-06T05:01:21.907168+0000","last_active":"2023-10-06T05:01:22.901055+0000","last_peered":"2023-10-06T05:01:22.901055+0000","last_clean":"2023-10-06T05:01:22.901055+0000","last_became_active":"2023-10-06T05:01:21.906798+0000","last_became_peered":"2023-10-06T05:01:21.906798+0000","last_unstale":"2023-10-06T05:01:22.901055+0000","last_undegraded":"2023-10-06T05:01:22.901055+0000","last_fullsized":"2023-10-06T05:01:22.901055+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919507+0000","last_change":"2023-10-06T05:01:22.917117+0000","last_active":"2023-10-06T05:01:22.919507+0000","last_peered":"2023-10-06T05:01:22.919507+0000","last_clean":"2023-10-06T05:01:22.919507+0000","last_became_active":"2023-10-06T05:01:22.916829+0000","last_became_peered":"2023-10-06T05:01:22.916829+0000","last_unstale":"2023-10-06T05:01:22.919507+0000","last_undegraded":"2023-10-06T05:01:22.919507+0000","last_fullsized":"2023-10-06T05:01:22.919507+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.896584+0000","last_change":"2023-10-06T05:01:21.897514+0000","last_active":"2023-10-06T05:01:22.896584+0000","last_peered":"2023-10-06T05:01:22.896584+0000","last_clean":"2023-10-06T05:01:22.896584+0000","last_became_active":"2023-10-06T05:01:21.897226+0000","last_became_peered":"2023-10-06T05:01:21.897226+0000","last_unstale":"2023-10-06T05:01:22.896584+0000","last_undegraded":"2023-10-06T05:01:22.896584+0000","last_fullsized":"2023-10-06T05:01:22.896584+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:44.413476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917870+0000","last_change":"2023-10-06T05:01:22.916955+0000","last_active":"2023-10-06T05:01:22.917870+0000","last_peered":"2023-10-06T05:01:22.917870+0000","last_clean":"2023-10-06T05:01:22.917870+0000","last_became_active":"2023-10-06T05:01:22.916640+0000","last_became_peered":"2023-10-06T05:01:22.916640+0000","last_unstale":"2023-10-06T05:01:22.917870+0000","last_undegraded":"2023-10-06T05:01:22.917870+0000","last_fullsized":"2023-10-06T05:01:22.917870+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:09.844106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891290+0000","last_change":"2023-10-06T05:01:21.906767+0000","last_active":"2023-10-06T05:01:22.891290+0000","last_peered":"2023-10-06T05:01:22.891290+0000","last_clean":"2023-10-06T05:01:22.891290+0000","last_became_active":"2023-10-06T05:01:21.906476+0000","last_became_peered":"2023-10-06T05:01:21.906476+0000","last_unstale":"2023-10-06T05:01:22.891290+0000","last_undegraded":"2023-10-06T05:01:22.891290+0000","last_fullsized":"2023-10-06T05:01:22.891290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900665+0000","last_change":"2023-10-06T05:01:21.910269+0000","last_active":"2023-10-06T05:01:22.900665+0000","last_peered":"2023-10-06T05:01:22.900665+0000","last_clean":"2023-10-06T05:01:22.900665+0000","last_became_active":"2023-10-06T05:01:21.909971+0000","last_became_peered":"2023-10-06T05:01:21.909971+0000","last_unstale":"2023-10-06T05:01:22.900665+0000","last_undegraded":"2023-10-06T05:01:22.900665+0000","last_fullsized":"2023-10-06T05:01:22.900665+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.894440+0000","last_change":"2023-10-06T05:01:21.902093+0000","last_active":"2023-10-06T05:01:22.894440+0000","last_peered":"2023-10-06T05:01:22.894440+0000","last_clean":"2023-10-06T05:01:22.894440+0000","last_became_active":"2023-10-06T05:01:21.900927+0000","last_became_peered":"2023-10-06T05:01:21.900927+0000","last_unstale":"2023-10-06T05:01:22.894440+0000","last_undegraded":"2023-10-06T05:01:22.894440+0000","last_fullsized":"2023-10-06T05:01:22.894440+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:16.639594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918752+0000","last_change":"2023-10-06T05:01:22.917989+0000","last_active":"2023-10-06T05:01:22.918752+0000","last_peered":"2023-10-06T05:01:22.918752+0000","last_clean":"2023-10-06T05:01:22.918752+0000","last_became_active":"2023-10-06T05:01:22.917665+0000","last_became_peered":"2023-10-06T05:01:22.917665+0000","last_unstale":"2023-10-06T05:01:22.918752+0000","last_undegraded":"2023-10-06T05:01:22.918752+0000","last_fullsized":"2023-10-06T05:01:22.918752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:10.275597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900723+0000","last_change":"2023-10-06T05:01:21.908132+0000","last_active":"2023-10-06T05:01:22.900723+0000","last_peered":"2023-10-06T05:01:22.900723+0000","last_clean":"2023-10-06T05:01:22.900723+0000","last_became_active":"2023-10-06T05:01:21.907727+0000","last_became_peered":"2023-10-06T05:01:21.907727+0000","last_unstale":"2023-10-06T05:01:22.900723+0000","last_undegraded":"2023-10-06T05:01:22.900723+0000","last_fullsized":"2023-10-06T05:01:22.900723+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:19.166059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898029+0000","last_change":"2023-10-06T05:01:21.908600+0000","last_active":"2023-10-06T05:01:22.898029+0000","last_peered":"2023-10-06T05:01:22.898029+0000","last_clean":"2023-10-06T05:01:22.898029+0000","last_became_active":"2023-10-06T05:01:21.908261+0000","last_became_peered":"2023-10-06T05:01:21.908261+0000","last_unstale":"2023-10-06T05:01:22.898029+0000","last_undegraded":"2023-10-06T05:01:22.898029+0000","last_fullsized":"2023-10-06T05:01:22.898029+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:23.366677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910946+0000","last_change":"2023-10-06T05:01:22.906979+0000","last_active":"2023-10-06T05:01:22.910946+0000","last_peered":"2023-10-06T05:01:22.910946+0000","last_clean":"2023-10-06T05:01:22.910946+0000","last_became_active":"2023-10-06T05:01:22.906652+0000","last_became_peered":"2023-10-06T05:01:22.906652+0000","last_unstale":"2023-10-06T05:01:22.910946+0000","last_undegraded":"2023-10-06T05:01:22.910946+0000","last_fullsized":"2023-10-06T05:01:22.910946+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:42.125134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:01:17.629295+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:01:17.629295+0000","last_peered":"2023-10-06T05:01:17.629295+0000","last_clean":"2023-10-06T05:01:17.629295+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:01:17.629295+0000","last_undegraded":"2023-10-06T05:01:17.629295+0000","last_fullsized":"2023-10-06T05:01:17.629295+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912616+0000","last_change":"2023-10-06T05:01:22.911622+0000","last_active":"2023-10-06T05:01:22.912616+0000","last_peered":"2023-10-06T05:01:22.912616+0000","last_clean":"2023-10-06T05:01:22.912616+0000","last_became_active":"2023-10-06T05:01:22.911388+0000","last_became_peered":"2023-10-06T05:01:22.911388+0000","last_unstale":"2023-10-06T05:01:22.912616+0000","last_undegraded":"2023-10-06T05:01:22.912616+0000","last_fullsized":"2023-10-06T05:01:22.912616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.004036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920959+0000","last_change":"2023-10-06T05:01:21.907272+0000","last_active":"2023-10-06T05:01:22.920959+0000","last_peered":"2023-10-06T05:01:22.920959+0000","last_clean":"2023-10-06T05:01:22.920959+0000","last_became_active":"2023-10-06T05:01:21.906971+0000","last_became_peered":"2023-10-06T05:01:21.906971+0000","last_unstale":"2023-10-06T05:01:22.920959+0000","last_undegraded":"2023-10-06T05:01:22.920959+0000","last_fullsized":"2023-10-06T05:01:22.920959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891246+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:01:22.891246+0000","last_peered":"2023-10-06T05:01:22.891246+0000","last_clean":"2023-10-06T05:01:22.891246+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:01:22.891246+0000","last_undegraded":"2023-10-06T05:01:22.891246+0000","last_fullsized":"2023-10-06T05:01:22.891246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.902904+0000","last_change":"2023-10-06T05:01:21.905784+0000","last_active":"2023-10-06T05:01:22.902904+0000","last_peered":"2023-10-06T05:01:22.902904+0000","last_clean":"2023-10-06T05:01:22.902904+0000","last_became_active":"2023-10-06T05:01:21.905522+0000","last_became_peered":"2023-10-06T05:01:21.905522+0000","last_unstale":"2023-10-06T05:01:22.902904+0000","last_undegraded":"2023-10-06T05:01:22.902904+0000","last_fullsized":"2023-10-06T05:01:22.902904+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:31.320712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":107,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897470+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:01:22.897470+0000","last_peered":"2023-10-06T05:01:22.897470+0000","last_clean":"2023-10-06T05:01:22.897470+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:01:22.897470+0000","last_undegraded":"2023-10-06T05:01:22.897470+0000","last_fullsized":"2023-10-06T05:01:22.897470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":254,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898054+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:01:22.898054+0000","last_peered":"2023-10-06T05:01:22.898054+0000","last_clean":"2023-10-06T05:01:22.898054+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:01:22.898054+0000","last_undegraded":"2023-10-06T05:01:22.898054+0000","last_fullsized":"2023-10-06T05:01:22.898054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915904+0000","last_change":"2023-10-06T05:01:22.914069+0000","last_active":"2023-10-06T05:01:22.915904+0000","last_peered":"2023-10-06T05:01:22.915904+0000","last_clean":"2023-10-06T05:01:22.915904+0000","last_became_active":"2023-10-06T05:01:22.913789+0000","last_became_peered":"2023-10-06T05:01:22.913789+0000","last_unstale":"2023-10-06T05:01:22.915904+0000","last_undegraded":"2023-10-06T05:01:22.915904+0000","last_fullsized":"2023-10-06T05:01:22.915904+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:37.179875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900942+0000","last_change":"2023-10-06T05:01:21.907735+0000","last_active":"2023-10-06T05:01:22.900942+0000","last_peered":"2023-10-06T05:01:22.900942+0000","last_clean":"2023-10-06T05:01:22.900942+0000","last_became_active":"2023-10-06T05:01:21.907455+0000","last_became_peered":"2023-10-06T05:01:21.907455+0000","last_unstale":"2023-10-06T05:01:22.900942+0000","last_undegraded":"2023-10-06T05:01:22.900942+0000","last_fullsized":"2023-10-06T05:01:22.900942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:50.355734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897481+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:01:22.897481+0000","last_peered":"2023-10-06T05:01:22.897481+0000","last_clean":"2023-10-06T05:01:22.897481+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:01:22.897481+0000","last_undegraded":"2023-10-06T05:01:22.897481+0000","last_fullsized":"2023-10-06T05:01:22.897481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.925929+0000","last_change":"2023-10-06T05:01:21.900909+0000","last_active":"2023-10-06T05:01:22.925929+0000","last_peered":"2023-10-06T05:01:22.925929+0000","last_clean":"2023-10-06T05:01:22.925929+0000","last_became_active":"2023-10-06T05:01:21.900571+0000","last_became_peered":"2023-10-06T05:01:21.900571+0000","last_unstale":"2023-10-06T05:01:22.925929+0000","last_undegraded":"2023-10-06T05:01:22.925929+0000","last_fullsized":"2023-10-06T05:01:22.925929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898113+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:01:22.898113+0000","last_peered":"2023-10-06T05:01:22.898113+0000","last_clean":"2023-10-06T05:01:22.898113+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:01:22.898113+0000","last_undegraded":"2023-10-06T05:01:22.898113+0000","last_fullsized":"2023-10-06T05:01:22.898113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912543+0000","last_change":"2023-10-06T05:01:22.910384+0000","last_active":"2023-10-06T05:01:22.912543+0000","last_peered":"2023-10-06T05:01:22.912543+0000","last_clean":"2023-10-06T05:01:22.912543+0000","last_became_active":"2023-10-06T05:01:22.909894+0000","last_became_peered":"2023-10-06T05:01:22.909894+0000","last_unstale":"2023-10-06T05:01:22.912543+0000","last_undegraded":"2023-10-06T05:01:22.912543+0000","last_fullsized":"2023-10-06T05:01:22.912543+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:03.895804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909184+0000","last_change":"2023-10-06T05:01:21.898111+0000","last_active":"2023-10-06T05:01:22.909184+0000","last_peered":"2023-10-06T05:01:22.909184+0000","last_clean":"2023-10-06T05:01:22.909184+0000","last_became_active":"2023-10-06T05:01:21.897816+0000","last_became_peered":"2023-10-06T05:01:21.897816+0000","last_unstale":"2023-10-06T05:01:22.909184+0000","last_undegraded":"2023-10-06T05:01:22.909184+0000","last_fullsized":"2023-10-06T05:01:22.909184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:05.782643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900952+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:01:22.900952+0000","last_peered":"2023-10-06T05:01:22.900952+0000","last_clean":"2023-10-06T05:01:22.900952+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:01:22.900952+0000","last_undegraded":"2023-10-06T05:01:22.900952+0000","last_fullsized":"2023-10-06T05:01:22.900952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921053+0000","last_change":"2023-10-06T05:01:22.920569+0000","last_active":"2023-10-06T05:01:22.921053+0000","last_peered":"2023-10-06T05:01:22.921053+0000","last_clean":"2023-10-06T05:01:22.921053+0000","last_became_active":"2023-10-06T05:01:22.920240+0000","last_became_peered":"2023-10-06T05:01:22.920240+0000","last_unstale":"2023-10-06T05:01:22.921053+0000","last_undegraded":"2023-10-06T05:01:22.921053+0000","last_fullsized":"2023-10-06T05:01:22.921053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:20.871431+0000","last_change":"2023-10-06T05:01:20.871431+0000","last_active":"2023-10-06T05:01:20.871431+0000","last_peered":"2023-10-06T05:01:20.871431+0000","last_clean":"2023-10-06T05:01:20.871431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:20.871431+0000","last_undegraded":"2023-10-06T05:01:20.871431+0000","last_fullsized":"2023-10-06T05:01:20.871431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:01:17.785309+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:01:17.785309+0000","last_peered":"2023-10-06T05:01:17.785309+0000","last_clean":"2023-10-06T05:01:17.785309+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:01:17.785309+0000","last_undegraded":"2023-10-06T05:01:17.785309+0000","last_fullsized":"2023-10-06T05:01:17.785309+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:01:21.877530+0000","last_change":"2023-10-06T05:01:21.877530+0000","last_active":"2023-10-06T05:01:21.877530+0000","last_peered":"2023-10-06T05:01:21.877530+0000","last_clean":"2023-10-06T05:01:21.877530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:01:21.877530+0000","last_undegraded":"2023-10-06T05:01:21.877530+0000","last_fullsized":"2023-10-06T05:01:21.877530+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909478+0000","last_change":"2023-10-06T05:01:21.905554+0000","last_active":"2023-10-06T05:01:22.909478+0000","last_peered":"2023-10-06T05:01:22.909478+0000","last_clean":"2023-10-06T05:01:22.909478+0000","last_became_active":"2023-10-06T05:01:21.905222+0000","last_became_peered":"2023-10-06T05:01:21.905222+0000","last_unstale":"2023-10-06T05:01:22.909478+0000","last_undegraded":"2023-10-06T05:01:22.909478+0000","last_fullsized":"2023-10-06T05:01:22.909478+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:01:17.785294+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:01:17.785294+0000","last_peered":"2023-10-06T05:01:17.785294+0000","last_clean":"2023-10-06T05:01:17.785294+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:01:17.785294+0000","last_undegraded":"2023-10-06T05:01:17.785294+0000","last_fullsized":"2023-10-06T05:01:17.785294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":50,"acting":50,"num_store_stats":6},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":94,"acting":94,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.73899999999999999,"5min":0.86599999999999999,"15min":7.1779999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.73499999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.311},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":7.1879999999999997},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.073},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":1.1379999999999999},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.111},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.0880000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.874},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.214},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302952,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068888,"statfs":{"total":96636764160,"available":96326541312,"internally_reserved":0,"allocated":1605632,"data_stored":727237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58799999999999997,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.73399999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63400000000000001,"15min":0.57499999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.65500000000000003,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.77900000000000003,"5min":1.234,"15min":1.234},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.63400000000000001,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.628},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70099999999999996,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.755,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.755},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61199999999999999,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.68100000000000005,"5min":3.669,"15min":3.669},"last":0.503},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.79300000000000004,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35599999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.437,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":0.375},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.378,"15min":0.378},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.60499999999999998,"5min":0.68600000000000005,"15min":0.69399999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.622,"15min":0.70399999999999996},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.56000000000000005,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.47199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.77300000000000002,"5min":0.877,"15min":1.0780000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.998,"15min":0.998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.54400000000000004,"5min":0.496,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":1.022,"15min":1.0429999999999999},"last":0.83399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":755982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.59199999999999997,"5min":0.628,"15min":0.628},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42999999999999999,"15min":0.38800000000000001},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46100000000000002,"15min":0.443},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.98299999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.85799999999999998,"15min":0.92300000000000004},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.91700000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.93400000000000005,"15min":1.0109999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.52200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.97599999999999998,"15min":1.0329999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":0.995},"last":0.71499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":753880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.20300000000000001,"15min":0.186},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.048},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.39300000000000002},"min":{"1min":0.32300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.216},"max":{"1min":0.52400000000000002,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.219},"max":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.36499999999999999},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.90200000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.2649999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.88100000000000001,"15min":0.93100000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":0.88300000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.82499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307104,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064736,"statfs":{"total":96636764160,"available":96322289664,"internally_reserved":0,"allocated":1662976,"data_stored":754356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56399999999999995,"5min":0.68200000000000005,"15min":2.008},"last":0.22},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.53000000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":6.4370000000000003},"last":0.28299999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.224,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.67000000000000004,"15min":1.7649999999999999},"last":0.312},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":2.1970000000000001},"last":0.29999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42399999999999999,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.255},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":5.069},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.441,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":2.246},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59499999999999997,"15min":0.76100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":14.646000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":14.606999999999999},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.81399999999999995,"5min":0.94699999999999995,"15min":2.2959999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":6.4009999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.73299999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.90100000000000002,"15min":5.0309999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":0.87,"5min":1.012,"15min":2.2749999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.75800000000000001},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.92600000000000005,"15min":6.367},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":5},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307048,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064792,"statfs":{"total":96636764160,"available":96322347008,"internally_reserved":0,"allocated":1605632,"data_stored":727237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.71199999999999997,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.73299999999999998},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.76200000000000001},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.76700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.70999999999999996,"5min":0.99299999999999999,"15min":5.4900000000000002},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.64100000000000001},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":4.0739999999999998},"last":0.498},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.80000000000000004,"15min":5.415},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.64800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":4.202},"last":0.435},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.622,"15min":0.70399999999999996},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.754,"5min":1.044,"15min":4.2130000000000001},"last":0.62}]},{"osd":5,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.497},"min":{"1min":0.27800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69799999999999995,"5min":0.73399999999999999,"15min":1.1279999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.76900000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78700000000000003,"5min":0.86199999999999999,"15min":0.872},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.49399999999999999,"5min":0.375,"15min":0.313},"max":{"1min":0.76300000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307092,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064748,"statfs":{"total":96636764160,"available":96322301952,"internally_reserved":0,"allocated":1650688,"data_stored":752415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.20499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":4.9039999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.75800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57499999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":6.7949999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.26200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.86599999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.623},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.86599999999999999},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53800000000000003,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":7,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:01:27.417 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:01:27.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.567+0000 7fa1ed5bc700 1 Processor -- start 2023-10-06T05:01:27.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.567+0000 7fa1ed5bc700 1 -- start start 2023-10-06T05:01:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1ed5bc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 0x7fa1e80cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1ed5bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e80cc5c0 con 0x7fa1e81598d0 2023-10-06T05:01:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1ed5bc700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e80cb9b0 con 0x7fa1e80b2930 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1ed5bc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80cbc90 con 0x7fa1e8159500 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1e67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1e81598d0 0x7fa1e80b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40198/0 (socket says 172.21.15.3:40198) 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1e67fc700 1 -- 172.21.15.3:0/3544035614 learned_addr learned my addr 172.21.15.3:0/3544035614 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 0x7fa1e80cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240766474 0 0) 0x7fa1e80cc5c0 con 0x7fa1e81598d0 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.568+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d4003570 con 0x7fa1e81598d0 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 303995058 0 0) 0x7fa1d4003570 con 0x7fa1e81598d0 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 msgr2=0x7fa1e80cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 0x7fa1e80cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 >> v1:172.21.15.153:6789/0 conn(0x7fa1e80b2930 legacy=0x7fa1e80cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80cbf70 con 0x7fa1e81598d0 2023-10-06T05:01:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fa1d0001470 con 0x7fa1e81598d0 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 >> v1:172.21.15.3:6789/0 conn(0x7fa1e81598d0 legacy=0x7fa1e80b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4003a90 0x7fa1d4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4004810 0x7fa1d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 0x7fa1d40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80cbc90 con 0x7fa1d4003a90 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d400bb60 con 0x7fa1d4004810 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d400bd20 con 0x7fa1e8159500 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4003a90 0x7fa1d4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4004810 0x7fa1d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 0x7fa1d40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e6ffd700 1 -- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 msgr2=0x7fa1d40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 0x7fa1d40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e6ffd700 1 -- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4004810 msgr2=0x7fa1d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d4004810 0x7fa1d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e6ffd700 1 -- 172.21.15.3:0/3544035614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4003570 con 0x7fa1d4003a90 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1e8159500 0x7fa1d40051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.569+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4003a90 0x7fa1d4003ea0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009c00 tx=0x7fa1e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6947337fe0ae74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000e710 con 0x7fa1d4003a90 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e000e870 con 0x7fa1d4003a90 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1e5ffb700 1 -- 172.21.15.3:0/3544035614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000ec30 con 0x7fa1d4003a90 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/3544035614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4003a90 msgr2=0x7fa1d4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1ed5bc700 1 --2- 172.21.15.3:0/3544035614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d4003a90 0x7fa1d4003ea0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009c00 tx=0x7fa1e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1e77fe700 1 -- 172.21.15.3:0/3544035614 reap_dead start 2023-10-06T05:01:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/3544035614 shutdown_connections 2023-10-06T05:01:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/3544035614 >> 172.21.15.3:0/3544035614 conn(0x7fa1e8070310 msgr2=0x7fa1e8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/3544035614 shutdown_connections 2023-10-06T05:01:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.570+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/3544035614 wait complete. 2023-10-06T05:01:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.571+0000 7fa1ed5bc700 1 Processor -- start 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.571+0000 7fa1ed5bc700 1 -- start start 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1ed5bc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d4003a90 0x7fa1e815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1ed5bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e80c5950 0x7fa1e80c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1ed5bc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8155510 0x7fa1e8155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1ed5bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80b3490 con 0x7fa1e80c5950 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1ed5bc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8155eb0 con 0x7fa1e8155510 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1ed5bc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80c63d0 con 0x7fa1d4003a90 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1e67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e80c5950 0x7fa1e80c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1e67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e80c5950 0x7fa1e80c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50496/0 (socket says 172.21.15.3:50496) 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1e67fc700 1 -- 172.21.15.3:0/1927274784 learned_addr learned my addr 172.21.15.3:0/1927274784 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.572+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d4003a90 0x7fa1e815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8155510 0x7fa1e8155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e67fc700 1 -- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d4003a90 msgr2=0x7fa1e815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d4003a90 0x7fa1e815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e67fc700 1 -- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8155510 msgr2=0x7fa1e8155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8155510 0x7fa1e8155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e67fc700 1 -- 172.21.15.3:0/1927274784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80c66b0 con 0x7fa1e80c5950 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e77fe700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8155510 0x7fa1e8155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1e67fc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e80c5950 0x7fa1e80c5dc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007370 tx=0x7fa1d0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1deffd700 1 -- 172.21.15.3:0/1927274784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0018030 con 0x7fa1e80c5950 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1deffd700 1 -- 172.21.15.3:0/1927274784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d000f8d0 con 0x7fa1e80c5950 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e80c6990 con 0x7fa1e80c5950 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1deffd700 1 -- 172.21.15.3:0/1927274784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000fc90 con 0x7fa1e80c5950 2023-10-06T05:01:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.573+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e816d360 con 0x7fa1e80c5950 2023-10-06T05:01:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.574+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e8071680 con 0x7fa1e80c5950 2023-10-06T05:01:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.574+0000 7fa1deffd700 1 -- 172.21.15.3:0/1927274784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1d001b070 con 0x7fa1e80c5950 2023-10-06T05:01:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.575+0000 7fa1deffd700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.575+0000 7fa1deffd700 1 -- 172.21.15.3:0/1927274784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa1d0019070 con 0x7fa1e80c5950 2023-10-06T05:01:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.575+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.576+0000 7fa1e6ffd700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00091e0 tx=0x7fa1e001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.579+0000 7fa1deffd700 1 -- 172.21.15.3:0/1927274784 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1d0051790 con 0x7fa1e80c5950 2023-10-06T05:01:27.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.768+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1e80cc250 con 0x7fa1c8066f30 2023-10-06T05:01:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.777+0000 7fa1deffd700 1 -- 172.21.15.3:0/1927274784 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312378 (secure 0 0 0) 0x7fa1e80cc250 con 0x7fa1c8066f30 2023-10-06T05:01:27.783 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 msgr2=0x7fa1c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00091e0 tx=0x7fa1e001a000 comp rx=0 tx=0).stop 2023-10-06T05:01:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e80c5950 msgr2=0x7fa1e80c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e80c5950 0x7fa1e80c5dc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007370 tx=0x7fa1d0001e50 comp rx=0 tx=0).stop 2023-10-06T05:01:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 shutdown_connections 2023-10-06T05:01:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c8066f30 0x7fa1c80693e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8155510 0x7fa1e8155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d4003a90 0x7fa1e815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 --2- 172.21.15.3:0/1927274784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e80c5950 0x7fa1e80c5dc0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 >> 172.21.15.3:0/1927274784 conn(0x7fa1e8070310 msgr2=0x7fa1e80cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:27.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 shutdown_connections 2023-10-06T05:01:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.781+0000 7fa1ed5bc700 1 -- 172.21.15.3:0/1927274784 wait complete. 2023-10-06T05:01:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:01:27.795 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2023-10-06T05:01:27.558620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":193,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449788,"kb_used_data":14140,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524932,"statfs":{"total":773094113280,"available":770585530368,"internally_reserved":0,"allocated":14479360,"data_stored":7211433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001463"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919337+0000","last_change":"2023-10-06T05:01:22.917546+0000","last_active":"2023-10-06T05:01:22.919337+0000","last_peered":"2023-10-06T05:01:22.919337+0000","last_clean":"2023-10-06T05:01:22.919337+0000","last_became_active":"2023-10-06T05:01:22.916374+0000","last_became_peered":"2023-10-06T05:01:22.916374+0000","last_unstale":"2023-10-06T05:01:22.919337+0000","last_undegraded":"2023-10-06T05:01:22.919337+0000","last_fullsized":"2023-10-06T05:01:22.919337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917323+0000","last_change":"2023-10-06T05:01:22.915942+0000","last_active":"2023-10-06T05:01:22.917323+0000","last_peered":"2023-10-06T05:01:22.917323+0000","last_clean":"2023-10-06T05:01:22.917323+0000","last_became_active":"2023-10-06T05:01:22.915602+0000","last_became_peered":"2023-10-06T05:01:22.915602+0000","last_unstale":"2023-10-06T05:01:22.917323+0000","last_undegraded":"2023-10-06T05:01:22.917323+0000","last_fullsized":"2023-10-06T05:01:22.917323+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:39.759900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917901+0000","last_change":"2023-10-06T05:01:22.916859+0000","last_active":"2023-10-06T05:01:22.917901+0000","last_peered":"2023-10-06T05:01:22.917901+0000","last_clean":"2023-10-06T05:01:22.917901+0000","last_became_active":"2023-10-06T05:01:22.916504+0000","last_became_peered":"2023-10-06T05:01:22.916504+0000","last_unstale":"2023-10-06T05:01:22.917901+0000","last_undegraded":"2023-10-06T05:01:22.917901+0000","last_fullsized":"2023-10-06T05:01:22.917901+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:36.075945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915308+0000","last_change":"2023-10-06T05:01:22.914389+0000","last_active":"2023-10-06T05:01:22.915308+0000","last_peered":"2023-10-06T05:01:22.915308+0000","last_clean":"2023-10-06T05:01:22.915308+0000","last_became_active":"2023-10-06T05:01:22.914080+0000","last_became_peered":"2023-10-06T05:01:22.914080+0000","last_unstale":"2023-10-06T05:01:22.915308+0000","last_undegraded":"2023-10-06T05:01:22.915308+0000","last_fullsized":"2023-10-06T05:01:22.915308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:56.228351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912802+0000","last_change":"2023-10-06T05:01:22.908378+0000","last_active":"2023-10-06T05:01:22.912802+0000","last_peered":"2023-10-06T05:01:22.912802+0000","last_clean":"2023-10-06T05:01:22.912802+0000","last_became_active":"2023-10-06T05:01:22.907429+0000","last_became_peered":"2023-10-06T05:01:22.907429+0000","last_unstale":"2023-10-06T05:01:22.912802+0000","last_undegraded":"2023-10-06T05:01:22.912802+0000","last_fullsized":"2023-10-06T05:01:22.912802+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:32.443484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912753+0000","last_change":"2023-10-06T05:01:22.907760+0000","last_active":"2023-10-06T05:01:22.912753+0000","last_peered":"2023-10-06T05:01:22.912753+0000","last_clean":"2023-10-06T05:01:22.912753+0000","last_became_active":"2023-10-06T05:01:22.907466+0000","last_became_peered":"2023-10-06T05:01:22.907466+0000","last_unstale":"2023-10-06T05:01:22.912753+0000","last_undegraded":"2023-10-06T05:01:22.912753+0000","last_fullsized":"2023-10-06T05:01:22.912753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:21.629837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908240+0000","last_change":"2023-10-06T05:01:22.907593+0000","last_active":"2023-10-06T05:01:22.908240+0000","last_peered":"2023-10-06T05:01:22.908240+0000","last_clean":"2023-10-06T05:01:22.908240+0000","last_became_active":"2023-10-06T05:01:22.907290+0000","last_became_peered":"2023-10-06T05:01:22.907290+0000","last_unstale":"2023-10-06T05:01:22.908240+0000","last_undegraded":"2023-10-06T05:01:22.908240+0000","last_fullsized":"2023-10-06T05:01:22.908240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921703+0000","last_change":"2023-10-06T05:01:22.917866+0000","last_active":"2023-10-06T05:01:22.921703+0000","last_peered":"2023-10-06T05:01:22.921703+0000","last_clean":"2023-10-06T05:01:22.921703+0000","last_became_active":"2023-10-06T05:01:22.917142+0000","last_became_peered":"2023-10-06T05:01:22.917142+0000","last_unstale":"2023-10-06T05:01:22.921703+0000","last_undegraded":"2023-10-06T05:01:22.921703+0000","last_fullsized":"2023-10-06T05:01:22.921703+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910976+0000","last_change":"2023-10-06T05:01:22.906539+0000","last_active":"2023-10-06T05:01:22.910976+0000","last_peered":"2023-10-06T05:01:22.910976+0000","last_clean":"2023-10-06T05:01:22.910976+0000","last_became_active":"2023-10-06T05:01:22.905906+0000","last_became_peered":"2023-10-06T05:01:22.905906+0000","last_unstale":"2023-10-06T05:01:22.910976+0000","last_undegraded":"2023-10-06T05:01:22.910976+0000","last_fullsized":"2023-10-06T05:01:22.910976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:33.022370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908499+0000","last_change":"2023-10-06T05:01:22.906059+0000","last_active":"2023-10-06T05:01:22.908499+0000","last_peered":"2023-10-06T05:01:22.908499+0000","last_clean":"2023-10-06T05:01:22.908499+0000","last_became_active":"2023-10-06T05:01:22.905782+0000","last_became_peered":"2023-10-06T05:01:22.905782+0000","last_unstale":"2023-10-06T05:01:22.908499+0000","last_undegraded":"2023-10-06T05:01:22.908499+0000","last_fullsized":"2023-10-06T05:01:22.908499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:31.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916007+0000","last_change":"2023-10-06T05:01:22.914642+0000","last_active":"2023-10-06T05:01:22.916007+0000","last_peered":"2023-10-06T05:01:22.916007+0000","last_clean":"2023-10-06T05:01:22.916007+0000","last_became_active":"2023-10-06T05:01:22.913781+0000","last_became_peered":"2023-10-06T05:01:22.913781+0000","last_unstale":"2023-10-06T05:01:22.916007+0000","last_undegraded":"2023-10-06T05:01:22.916007+0000","last_fullsized":"2023-10-06T05:01:22.916007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:39.375404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915290+0000","last_change":"2023-10-06T05:01:22.913641+0000","last_active":"2023-10-06T05:01:22.915290+0000","last_peered":"2023-10-06T05:01:22.915290+0000","last_clean":"2023-10-06T05:01:22.915290+0000","last_became_active":"2023-10-06T05:01:22.913356+0000","last_became_peered":"2023-10-06T05:01:22.913356+0000","last_unstale":"2023-10-06T05:01:22.915290+0000","last_undegraded":"2023-10-06T05:01:22.915290+0000","last_fullsized":"2023-10-06T05:01:22.915290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:50.229096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909396+0000","last_change":"2023-10-06T05:01:22.906295+0000","last_active":"2023-10-06T05:01:22.909396+0000","last_peered":"2023-10-06T05:01:22.909396+0000","last_clean":"2023-10-06T05:01:22.909396+0000","last_became_active":"2023-10-06T05:01:22.905241+0000","last_became_peered":"2023-10-06T05:01:22.905241+0000","last_unstale":"2023-10-06T05:01:22.909396+0000","last_undegraded":"2023-10-06T05:01:22.909396+0000","last_fullsized":"2023-10-06T05:01:22.909396+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:56.323569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915262+0000","last_change":"2023-10-06T05:01:22.913298+0000","last_active":"2023-10-06T05:01:22.915262+0000","last_peered":"2023-10-06T05:01:22.915262+0000","last_clean":"2023-10-06T05:01:22.915262+0000","last_became_active":"2023-10-06T05:01:22.913012+0000","last_became_peered":"2023-10-06T05:01:22.913012+0000","last_unstale":"2023-10-06T05:01:22.915262+0000","last_undegraded":"2023-10-06T05:01:22.915262+0000","last_fullsized":"2023-10-06T05:01:22.915262+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:31.315774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921375+0000","last_change":"2023-10-06T05:01:22.920640+0000","last_active":"2023-10-06T05:01:22.921375+0000","last_peered":"2023-10-06T05:01:22.921375+0000","last_clean":"2023-10-06T05:01:22.921375+0000","last_became_active":"2023-10-06T05:01:22.920435+0000","last_became_peered":"2023-10-06T05:01:22.920435+0000","last_unstale":"2023-10-06T05:01:22.921375+0000","last_undegraded":"2023-10-06T05:01:22.921375+0000","last_fullsized":"2023-10-06T05:01:22.921375+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:21.298643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921735+0000","last_change":"2023-10-06T05:01:22.921275+0000","last_active":"2023-10-06T05:01:22.921735+0000","last_peered":"2023-10-06T05:01:22.921735+0000","last_clean":"2023-10-06T05:01:22.921735+0000","last_became_active":"2023-10-06T05:01:22.920953+0000","last_became_peered":"2023-10-06T05:01:22.920953+0000","last_unstale":"2023-10-06T05:01:22.921735+0000","last_undegraded":"2023-10-06T05:01:22.921735+0000","last_fullsized":"2023-10-06T05:01:22.921735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:22.455454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921292+0000","last_change":"2023-10-06T05:01:22.919416+0000","last_active":"2023-10-06T05:01:22.921292+0000","last_peered":"2023-10-06T05:01:22.921292+0000","last_clean":"2023-10-06T05:01:22.921292+0000","last_became_active":"2023-10-06T05:01:22.917490+0000","last_became_peered":"2023-10-06T05:01:22.917490+0000","last_unstale":"2023-10-06T05:01:22.921292+0000","last_undegraded":"2023-10-06T05:01:22.921292+0000","last_fullsized":"2023-10-06T05:01:22.921292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:14.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912183+0000","last_change":"2023-10-06T05:01:22.910088+0000","last_active":"2023-10-06T05:01:22.912183+0000","last_peered":"2023-10-06T05:01:22.912183+0000","last_clean":"2023-10-06T05:01:22.912183+0000","last_became_active":"2023-10-06T05:01:22.909830+0000","last_became_peered":"2023-10-06T05:01:22.909830+0000","last_unstale":"2023-10-06T05:01:22.912183+0000","last_undegraded":"2023-10-06T05:01:22.912183+0000","last_fullsized":"2023-10-06T05:01:22.912183+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:14.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920682+0000","last_change":"2023-10-06T05:01:22.915966+0000","last_active":"2023-10-06T05:01:22.920682+0000","last_peered":"2023-10-06T05:01:22.920682+0000","last_clean":"2023-10-06T05:01:22.920682+0000","last_became_active":"2023-10-06T05:01:22.915477+0000","last_became_peered":"2023-10-06T05:01:22.915477+0000","last_unstale":"2023-10-06T05:01:22.920682+0000","last_undegraded":"2023-10-06T05:01:22.920682+0000","last_fullsized":"2023-10-06T05:01:22.920682+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:24.762820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908898+0000","last_change":"2023-10-06T05:01:22.905656+0000","last_active":"2023-10-06T05:01:22.908898+0000","last_peered":"2023-10-06T05:01:22.908898+0000","last_clean":"2023-10-06T05:01:22.908898+0000","last_became_active":"2023-10-06T05:01:22.905402+0000","last_became_peered":"2023-10-06T05:01:22.905402+0000","last_unstale":"2023-10-06T05:01:22.908898+0000","last_undegraded":"2023-10-06T05:01:22.908898+0000","last_fullsized":"2023-10-06T05:01:22.908898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.573131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912788+0000","last_change":"2023-10-06T05:01:22.908184+0000","last_active":"2023-10-06T05:01:22.912788+0000","last_peered":"2023-10-06T05:01:22.912788+0000","last_clean":"2023-10-06T05:01:22.912788+0000","last_became_active":"2023-10-06T05:01:22.904908+0000","last_became_peered":"2023-10-06T05:01:22.904908+0000","last_unstale":"2023-10-06T05:01:22.912788+0000","last_undegraded":"2023-10-06T05:01:22.912788+0000","last_fullsized":"2023-10-06T05:01:22.912788+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:42.892786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912280+0000","last_change":"2023-10-06T05:01:22.908024+0000","last_active":"2023-10-06T05:01:22.912280+0000","last_peered":"2023-10-06T05:01:22.912280+0000","last_clean":"2023-10-06T05:01:22.912280+0000","last_became_active":"2023-10-06T05:01:22.907701+0000","last_became_peered":"2023-10-06T05:01:22.907701+0000","last_unstale":"2023-10-06T05:01:22.912280+0000","last_undegraded":"2023-10-06T05:01:22.912280+0000","last_fullsized":"2023-10-06T05:01:22.912280+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912827+0000","last_change":"2023-10-06T05:01:22.911724+0000","last_active":"2023-10-06T05:01:22.912827+0000","last_peered":"2023-10-06T05:01:22.912827+0000","last_clean":"2023-10-06T05:01:22.912827+0000","last_became_active":"2023-10-06T05:01:22.911247+0000","last_became_peered":"2023-10-06T05:01:22.911247+0000","last_unstale":"2023-10-06T05:01:22.912827+0000","last_undegraded":"2023-10-06T05:01:22.912827+0000","last_fullsized":"2023-10-06T05:01:22.912827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910490+0000","last_change":"2023-10-06T05:01:22.902838+0000","last_active":"2023-10-06T05:01:22.910490+0000","last_peered":"2023-10-06T05:01:22.910490+0000","last_clean":"2023-10-06T05:01:22.910490+0000","last_became_active":"2023-10-06T05:01:22.902549+0000","last_became_peered":"2023-10-06T05:01:22.902549+0000","last_unstale":"2023-10-06T05:01:22.910490+0000","last_undegraded":"2023-10-06T05:01:22.910490+0000","last_fullsized":"2023-10-06T05:01:22.910490+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:26.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912308+0000","last_change":"2023-10-06T05:01:22.908586+0000","last_active":"2023-10-06T05:01:22.912308+0000","last_peered":"2023-10-06T05:01:22.912308+0000","last_clean":"2023-10-06T05:01:22.912308+0000","last_became_active":"2023-10-06T05:01:22.908289+0000","last_became_peered":"2023-10-06T05:01:22.908289+0000","last_unstale":"2023-10-06T05:01:22.912308+0000","last_undegraded":"2023-10-06T05:01:22.912308+0000","last_fullsized":"2023-10-06T05:01:22.912308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:38.888355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914573+0000","last_change":"2023-10-06T05:01:22.910843+0000","last_active":"2023-10-06T05:01:22.914573+0000","last_peered":"2023-10-06T05:01:22.914573+0000","last_clean":"2023-10-06T05:01:22.914573+0000","last_became_active":"2023-10-06T05:01:22.910470+0000","last_became_peered":"2023-10-06T05:01:22.910470+0000","last_unstale":"2023-10-06T05:01:22.914573+0000","last_undegraded":"2023-10-06T05:01:22.914573+0000","last_fullsized":"2023-10-06T05:01:22.914573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912812+0000","last_change":"2023-10-06T05:01:22.908293+0000","last_active":"2023-10-06T05:01:22.912812+0000","last_peered":"2023-10-06T05:01:22.912812+0000","last_clean":"2023-10-06T05:01:22.912812+0000","last_became_active":"2023-10-06T05:01:22.907986+0000","last_became_peered":"2023-10-06T05:01:22.907986+0000","last_unstale":"2023-10-06T05:01:22.912812+0000","last_undegraded":"2023-10-06T05:01:22.912812+0000","last_fullsized":"2023-10-06T05:01:22.912812+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:58.242153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910601+0000","last_change":"2023-10-06T05:01:22.905470+0000","last_active":"2023-10-06T05:01:22.910601+0000","last_peered":"2023-10-06T05:01:22.910601+0000","last_clean":"2023-10-06T05:01:22.910601+0000","last_became_active":"2023-10-06T05:01:22.905162+0000","last_became_peered":"2023-10-06T05:01:22.905162+0000","last_unstale":"2023-10-06T05:01:22.910601+0000","last_undegraded":"2023-10-06T05:01:22.910601+0000","last_fullsized":"2023-10-06T05:01:22.910601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:53.259645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912975+0000","last_change":"2023-10-06T05:01:22.911783+0000","last_active":"2023-10-06T05:01:22.912975+0000","last_peered":"2023-10-06T05:01:22.912975+0000","last_clean":"2023-10-06T05:01:22.912975+0000","last_became_active":"2023-10-06T05:01:22.911488+0000","last_became_peered":"2023-10-06T05:01:22.911488+0000","last_unstale":"2023-10-06T05:01:22.912975+0000","last_undegraded":"2023-10-06T05:01:22.912975+0000","last_fullsized":"2023-10-06T05:01:22.912975+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912590+0000","last_change":"2023-10-06T05:01:22.911945+0000","last_active":"2023-10-06T05:01:22.912590+0000","last_peered":"2023-10-06T05:01:22.912590+0000","last_clean":"2023-10-06T05:01:22.912590+0000","last_became_active":"2023-10-06T05:01:22.911675+0000","last_became_peered":"2023-10-06T05:01:22.911675+0000","last_unstale":"2023-10-06T05:01:22.912590+0000","last_undegraded":"2023-10-06T05:01:22.912590+0000","last_fullsized":"2023-10-06T05:01:22.912590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:32.223585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914827+0000","last_change":"2023-10-06T05:01:22.913460+0000","last_active":"2023-10-06T05:01:22.914827+0000","last_peered":"2023-10-06T05:01:22.914827+0000","last_clean":"2023-10-06T05:01:22.914827+0000","last_became_active":"2023-10-06T05:01:22.913178+0000","last_became_peered":"2023-10-06T05:01:22.913178+0000","last_unstale":"2023-10-06T05:01:22.914827+0000","last_undegraded":"2023-10-06T05:01:22.914827+0000","last_fullsized":"2023-10-06T05:01:22.914827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:16.124524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.923028+0000","last_change":"2023-10-06T05:01:22.918920+0000","last_active":"2023-10-06T05:01:22.923028+0000","last_peered":"2023-10-06T05:01:22.923028+0000","last_clean":"2023-10-06T05:01:22.923028+0000","last_became_active":"2023-10-06T05:01:22.918545+0000","last_became_peered":"2023-10-06T05:01:22.918545+0000","last_unstale":"2023-10-06T05:01:22.923028+0000","last_undegraded":"2023-10-06T05:01:22.923028+0000","last_fullsized":"2023-10-06T05:01:22.923028+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909358+0000","last_change":"2023-10-06T05:01:22.906868+0000","last_active":"2023-10-06T05:01:22.909358+0000","last_peered":"2023-10-06T05:01:22.909358+0000","last_clean":"2023-10-06T05:01:22.909358+0000","last_became_active":"2023-10-06T05:01:22.906368+0000","last_became_peered":"2023-10-06T05:01:22.906368+0000","last_unstale":"2023-10-06T05:01:22.909358+0000","last_undegraded":"2023-10-06T05:01:22.909358+0000","last_fullsized":"2023-10-06T05:01:22.909358+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:45.514209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910892+0000","last_change":"2023-10-06T05:01:21.905167+0000","last_active":"2023-10-06T05:01:22.910892+0000","last_peered":"2023-10-06T05:01:22.910892+0000","last_clean":"2023-10-06T05:01:22.910892+0000","last_became_active":"2023-10-06T05:01:21.904856+0000","last_became_peered":"2023-10-06T05:01:21.904856+0000","last_unstale":"2023-10-06T05:01:22.910892+0000","last_undegraded":"2023-10-06T05:01:22.910892+0000","last_fullsized":"2023-10-06T05:01:22.910892+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916718+0000","last_change":"2023-10-06T05:01:22.915682+0000","last_active":"2023-10-06T05:01:22.916718+0000","last_peered":"2023-10-06T05:01:22.916718+0000","last_clean":"2023-10-06T05:01:22.916718+0000","last_became_active":"2023-10-06T05:01:22.915425+0000","last_became_peered":"2023-10-06T05:01:22.915425+0000","last_unstale":"2023-10-06T05:01:22.916718+0000","last_undegraded":"2023-10-06T05:01:22.916718+0000","last_fullsized":"2023-10-06T05:01:22.916718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913713+0000","last_change":"2023-10-06T05:01:21.909299+0000","last_active":"2023-10-06T05:01:22.913713+0000","last_peered":"2023-10-06T05:01:22.913713+0000","last_clean":"2023-10-06T05:01:22.913713+0000","last_became_active":"2023-10-06T05:01:21.908989+0000","last_became_peered":"2023-10-06T05:01:21.908989+0000","last_unstale":"2023-10-06T05:01:22.913713+0000","last_undegraded":"2023-10-06T05:01:22.913713+0000","last_fullsized":"2023-10-06T05:01:22.913713+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921765+0000","last_change":"2023-10-06T05:01:22.918183+0000","last_active":"2023-10-06T05:01:22.921765+0000","last_peered":"2023-10-06T05:01:22.921765+0000","last_clean":"2023-10-06T05:01:22.921765+0000","last_became_active":"2023-10-06T05:01:22.917637+0000","last_became_peered":"2023-10-06T05:01:22.917637+0000","last_unstale":"2023-10-06T05:01:22.921765+0000","last_undegraded":"2023-10-06T05:01:22.921765+0000","last_fullsized":"2023-10-06T05:01:22.921765+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914828+0000","last_change":"2023-10-06T05:01:21.909301+0000","last_active":"2023-10-06T05:01:22.914828+0000","last_peered":"2023-10-06T05:01:22.914828+0000","last_clean":"2023-10-06T05:01:22.914828+0000","last_became_active":"2023-10-06T05:01:21.908984+0000","last_became_peered":"2023-10-06T05:01:21.908984+0000","last_unstale":"2023-10-06T05:01:22.914828+0000","last_undegraded":"2023-10-06T05:01:22.914828+0000","last_fullsized":"2023-10-06T05:01:22.914828+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:26.376744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920398+0000","last_change":"2023-10-06T05:01:22.910823+0000","last_active":"2023-10-06T05:01:22.920398+0000","last_peered":"2023-10-06T05:01:22.920398+0000","last_clean":"2023-10-06T05:01:22.920398+0000","last_became_active":"2023-10-06T05:01:22.910158+0000","last_became_peered":"2023-10-06T05:01:22.910158+0000","last_unstale":"2023-10-06T05:01:22.920398+0000","last_undegraded":"2023-10-06T05:01:22.920398+0000","last_fullsized":"2023-10-06T05:01:22.920398+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:33.254913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917199+0000","last_change":"2023-10-06T05:01:21.898248+0000","last_active":"2023-10-06T05:01:22.917199+0000","last_peered":"2023-10-06T05:01:22.917199+0000","last_clean":"2023-10-06T05:01:22.917199+0000","last_became_active":"2023-10-06T05:01:21.897180+0000","last_became_peered":"2023-10-06T05:01:21.897180+0000","last_unstale":"2023-10-06T05:01:22.917199+0000","last_undegraded":"2023-10-06T05:01:22.917199+0000","last_fullsized":"2023-10-06T05:01:22.917199+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:59.743528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909059+0000","last_change":"2023-10-06T05:01:22.905942+0000","last_active":"2023-10-06T05:01:22.909059+0000","last_peered":"2023-10-06T05:01:22.909059+0000","last_clean":"2023-10-06T05:01:22.909059+0000","last_became_active":"2023-10-06T05:01:22.905653+0000","last_became_peered":"2023-10-06T05:01:22.905653+0000","last_unstale":"2023-10-06T05:01:22.909059+0000","last_undegraded":"2023-10-06T05:01:22.909059+0000","last_fullsized":"2023-10-06T05:01:22.909059+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:12.300608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921227+0000","last_change":"2023-10-06T05:01:21.902635+0000","last_active":"2023-10-06T05:01:22.921227+0000","last_peered":"2023-10-06T05:01:22.921227+0000","last_clean":"2023-10-06T05:01:22.921227+0000","last_became_active":"2023-10-06T05:01:21.901883+0000","last_became_peered":"2023-10-06T05:01:21.901883+0000","last_unstale":"2023-10-06T05:01:22.921227+0000","last_undegraded":"2023-10-06T05:01:22.921227+0000","last_fullsized":"2023-10-06T05:01:22.921227+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920542+0000","last_change":"2023-10-06T05:01:22.917447+0000","last_active":"2023-10-06T05:01:22.920542+0000","last_peered":"2023-10-06T05:01:22.920542+0000","last_clean":"2023-10-06T05:01:22.920542+0000","last_became_active":"2023-10-06T05:01:22.917130+0000","last_became_peered":"2023-10-06T05:01:22.917130+0000","last_unstale":"2023-10-06T05:01:22.920542+0000","last_undegraded":"2023-10-06T05:01:22.920542+0000","last_fullsized":"2023-10-06T05:01:22.920542+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:20.172157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916407+0000","last_change":"2023-10-06T05:01:21.906758+0000","last_active":"2023-10-06T05:01:22.916407+0000","last_peered":"2023-10-06T05:01:22.916407+0000","last_clean":"2023-10-06T05:01:22.916407+0000","last_became_active":"2023-10-06T05:01:21.906463+0000","last_became_peered":"2023-10-06T05:01:21.906463+0000","last_unstale":"2023-10-06T05:01:22.916407+0000","last_undegraded":"2023-10-06T05:01:22.916407+0000","last_fullsized":"2023-10-06T05:01:22.916407+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:42.205607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908789+0000","last_change":"2023-10-06T05:01:22.905020+0000","last_active":"2023-10-06T05:01:22.908789+0000","last_peered":"2023-10-06T05:01:22.908789+0000","last_clean":"2023-10-06T05:01:22.908789+0000","last_became_active":"2023-10-06T05:01:22.904762+0000","last_became_peered":"2023-10-06T05:01:22.904762+0000","last_unstale":"2023-10-06T05:01:22.908789+0000","last_undegraded":"2023-10-06T05:01:22.908789+0000","last_fullsized":"2023-10-06T05:01:22.908789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900865+0000","last_change":"2023-10-06T05:01:21.907467+0000","last_active":"2023-10-06T05:01:22.900865+0000","last_peered":"2023-10-06T05:01:22.900865+0000","last_clean":"2023-10-06T05:01:22.900865+0000","last_became_active":"2023-10-06T05:01:21.907169+0000","last_became_peered":"2023-10-06T05:01:21.907169+0000","last_unstale":"2023-10-06T05:01:22.900865+0000","last_undegraded":"2023-10-06T05:01:22.900865+0000","last_fullsized":"2023-10-06T05:01:22.900865+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.309675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918810+0000","last_change":"2023-10-06T05:01:22.916673+0000","last_active":"2023-10-06T05:01:22.918810+0000","last_peered":"2023-10-06T05:01:22.918810+0000","last_clean":"2023-10-06T05:01:22.918810+0000","last_became_active":"2023-10-06T05:01:22.916393+0000","last_became_peered":"2023-10-06T05:01:22.916393+0000","last_unstale":"2023-10-06T05:01:22.918810+0000","last_undegraded":"2023-10-06T05:01:22.918810+0000","last_fullsized":"2023-10-06T05:01:22.918810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:00.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.890628+0000","last_change":"2023-10-06T05:01:21.904853+0000","last_active":"2023-10-06T05:01:22.890628+0000","last_peered":"2023-10-06T05:01:22.890628+0000","last_clean":"2023-10-06T05:01:22.890628+0000","last_became_active":"2023-10-06T05:01:21.904448+0000","last_became_peered":"2023-10-06T05:01:21.904448+0000","last_unstale":"2023-10-06T05:01:22.890628+0000","last_undegraded":"2023-10-06T05:01:22.890628+0000","last_fullsized":"2023-10-06T05:01:22.890628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.567032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915557+0000","last_change":"2023-10-06T05:01:22.909027+0000","last_active":"2023-10-06T05:01:22.915557+0000","last_peered":"2023-10-06T05:01:22.915557+0000","last_clean":"2023-10-06T05:01:22.915557+0000","last_became_active":"2023-10-06T05:01:22.908783+0000","last_became_peered":"2023-10-06T05:01:22.908783+0000","last_unstale":"2023-10-06T05:01:22.915557+0000","last_undegraded":"2023-10-06T05:01:22.915557+0000","last_fullsized":"2023-10-06T05:01:22.915557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:48.927255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914458+0000","last_change":"2023-10-06T05:01:21.901342+0000","last_active":"2023-10-06T05:01:22.914458+0000","last_peered":"2023-10-06T05:01:22.914458+0000","last_clean":"2023-10-06T05:01:22.914458+0000","last_became_active":"2023-10-06T05:01:21.901020+0000","last_became_peered":"2023-10-06T05:01:21.901020+0000","last_unstale":"2023-10-06T05:01:22.914458+0000","last_undegraded":"2023-10-06T05:01:22.914458+0000","last_fullsized":"2023-10-06T05:01:22.914458+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:26.399481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920662+0000","last_change":"2023-10-06T05:01:22.915799+0000","last_active":"2023-10-06T05:01:22.920662+0000","last_peered":"2023-10-06T05:01:22.920662+0000","last_clean":"2023-10-06T05:01:22.920662+0000","last_became_active":"2023-10-06T05:01:22.915224+0000","last_became_peered":"2023-10-06T05:01:22.915224+0000","last_unstale":"2023-10-06T05:01:22.920662+0000","last_undegraded":"2023-10-06T05:01:22.920662+0000","last_fullsized":"2023-10-06T05:01:22.920662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:24.443047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909917+0000","last_change":"2023-10-06T05:01:21.903666+0000","last_active":"2023-10-06T05:01:22.909917+0000","last_peered":"2023-10-06T05:01:22.909917+0000","last_clean":"2023-10-06T05:01:22.909917+0000","last_became_active":"2023-10-06T05:01:21.903416+0000","last_became_peered":"2023-10-06T05:01:21.903416+0000","last_unstale":"2023-10-06T05:01:22.909917+0000","last_undegraded":"2023-10-06T05:01:22.909917+0000","last_fullsized":"2023-10-06T05:01:22.909917+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:38.753801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913218+0000","last_change":"2023-10-06T05:01:22.908994+0000","last_active":"2023-10-06T05:01:22.913218+0000","last_peered":"2023-10-06T05:01:22.913218+0000","last_clean":"2023-10-06T05:01:22.913218+0000","last_became_active":"2023-10-06T05:01:22.908627+0000","last_became_peered":"2023-10-06T05:01:22.908627+0000","last_unstale":"2023-10-06T05:01:22.913218+0000","last_undegraded":"2023-10-06T05:01:22.913218+0000","last_fullsized":"2023-10-06T05:01:22.913218+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920770+0000","last_change":"2023-10-06T05:01:21.901636+0000","last_active":"2023-10-06T05:01:22.920770+0000","last_peered":"2023-10-06T05:01:22.920770+0000","last_clean":"2023-10-06T05:01:22.920770+0000","last_became_active":"2023-10-06T05:01:21.901348+0000","last_became_peered":"2023-10-06T05:01:21.901348+0000","last_unstale":"2023-10-06T05:01:22.920770+0000","last_undegraded":"2023-10-06T05:01:22.920770+0000","last_fullsized":"2023-10-06T05:01:22.920770+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:07.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919815+0000","last_change":"2023-10-06T05:01:22.918705+0000","last_active":"2023-10-06T05:01:22.919815+0000","last_peered":"2023-10-06T05:01:22.919815+0000","last_clean":"2023-10-06T05:01:22.919815+0000","last_became_active":"2023-10-06T05:01:22.918437+0000","last_became_peered":"2023-10-06T05:01:22.918437+0000","last_unstale":"2023-10-06T05:01:22.919815+0000","last_undegraded":"2023-10-06T05:01:22.919815+0000","last_fullsized":"2023-10-06T05:01:22.919815+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900294+0000","last_change":"2023-10-06T05:01:21.903733+0000","last_active":"2023-10-06T05:01:22.900294+0000","last_peered":"2023-10-06T05:01:22.900294+0000","last_clean":"2023-10-06T05:01:22.900294+0000","last_became_active":"2023-10-06T05:01:21.903433+0000","last_became_peered":"2023-10-06T05:01:21.903433+0000","last_unstale":"2023-10-06T05:01:22.900294+0000","last_undegraded":"2023-10-06T05:01:22.900294+0000","last_fullsized":"2023-10-06T05:01:22.900294+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:39.894106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920786+0000","last_change":"2023-10-06T05:01:22.919693+0000","last_active":"2023-10-06T05:01:22.920786+0000","last_peered":"2023-10-06T05:01:22.920786+0000","last_clean":"2023-10-06T05:01:22.920786+0000","last_became_active":"2023-10-06T05:01:22.919445+0000","last_became_peered":"2023-10-06T05:01:22.919445+0000","last_unstale":"2023-10-06T05:01:22.920786+0000","last_undegraded":"2023-10-06T05:01:22.920786+0000","last_fullsized":"2023-10-06T05:01:22.920786+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:00.412186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916231+0000","last_change":"2023-10-06T05:01:21.901798+0000","last_active":"2023-10-06T05:01:22.916231+0000","last_peered":"2023-10-06T05:01:22.916231+0000","last_clean":"2023-10-06T05:01:22.916231+0000","last_became_active":"2023-10-06T05:01:21.901536+0000","last_became_peered":"2023-10-06T05:01:21.901536+0000","last_unstale":"2023-10-06T05:01:22.916231+0000","last_undegraded":"2023-10-06T05:01:22.916231+0000","last_fullsized":"2023-10-06T05:01:22.916231+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909492+0000","last_change":"2023-10-06T05:01:22.908931+0000","last_active":"2023-10-06T05:01:22.909492+0000","last_peered":"2023-10-06T05:01:22.909492+0000","last_clean":"2023-10-06T05:01:22.909492+0000","last_became_active":"2023-10-06T05:01:22.908678+0000","last_became_peered":"2023-10-06T05:01:22.908678+0000","last_unstale":"2023-10-06T05:01:22.909492+0000","last_undegraded":"2023-10-06T05:01:22.909492+0000","last_fullsized":"2023-10-06T05:01:22.909492+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:24.372776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.899848+0000","last_change":"2023-10-06T05:01:21.894870+0000","last_active":"2023-10-06T05:01:22.899848+0000","last_peered":"2023-10-06T05:01:22.899848+0000","last_clean":"2023-10-06T05:01:22.899848+0000","last_became_active":"2023-10-06T05:01:21.893907+0000","last_became_peered":"2023-10-06T05:01:21.893907+0000","last_unstale":"2023-10-06T05:01:22.899848+0000","last_undegraded":"2023-10-06T05:01:22.899848+0000","last_fullsized":"2023-10-06T05:01:22.899848+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920281+0000","last_change":"2023-10-06T05:01:22.910581+0000","last_active":"2023-10-06T05:01:22.920281+0000","last_peered":"2023-10-06T05:01:22.920281+0000","last_clean":"2023-10-06T05:01:22.920281+0000","last_became_active":"2023-10-06T05:01:22.909963+0000","last_became_peered":"2023-10-06T05:01:22.909963+0000","last_unstale":"2023-10-06T05:01:22.920281+0000","last_undegraded":"2023-10-06T05:01:22.920281+0000","last_fullsized":"2023-10-06T05:01:22.920281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:41.453873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.893893+0000","last_change":"2023-10-06T05:01:21.900893+0000","last_active":"2023-10-06T05:01:22.893893+0000","last_peered":"2023-10-06T05:01:22.893893+0000","last_clean":"2023-10-06T05:01:22.893893+0000","last_became_active":"2023-10-06T05:01:21.900591+0000","last_became_peered":"2023-10-06T05:01:21.900591+0000","last_unstale":"2023-10-06T05:01:22.893893+0000","last_undegraded":"2023-10-06T05:01:22.893893+0000","last_fullsized":"2023-10-06T05:01:22.893893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917896+0000","last_change":"2023-10-06T05:01:22.917013+0000","last_active":"2023-10-06T05:01:22.917896+0000","last_peered":"2023-10-06T05:01:22.917896+0000","last_clean":"2023-10-06T05:01:22.917896+0000","last_became_active":"2023-10-06T05:01:22.915500+0000","last_became_peered":"2023-10-06T05:01:22.915500+0000","last_unstale":"2023-10-06T05:01:22.917896+0000","last_undegraded":"2023-10-06T05:01:22.917896+0000","last_fullsized":"2023-10-06T05:01:22.917896+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914695+0000","last_change":"2023-10-06T05:01:21.897131+0000","last_active":"2023-10-06T05:01:22.914695+0000","last_peered":"2023-10-06T05:01:22.914695+0000","last_clean":"2023-10-06T05:01:22.914695+0000","last_became_active":"2023-10-06T05:01:21.896847+0000","last_became_peered":"2023-10-06T05:01:21.896847+0000","last_unstale":"2023-10-06T05:01:22.914695+0000","last_undegraded":"2023-10-06T05:01:22.914695+0000","last_fullsized":"2023-10-06T05:01:22.914695+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912743+0000","last_change":"2023-10-06T05:01:22.907709+0000","last_active":"2023-10-06T05:01:22.912743+0000","last_peered":"2023-10-06T05:01:22.912743+0000","last_clean":"2023-10-06T05:01:22.912743+0000","last_became_active":"2023-10-06T05:01:22.907411+0000","last_became_peered":"2023-10-06T05:01:22.907411+0000","last_unstale":"2023-10-06T05:01:22.912743+0000","last_undegraded":"2023-10-06T05:01:22.912743+0000","last_fullsized":"2023-10-06T05:01:22.912743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:19.960229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921946+0000","last_change":"2023-10-06T05:01:21.900714+0000","last_active":"2023-10-06T05:01:22.921946+0000","last_peered":"2023-10-06T05:01:22.921946+0000","last_clean":"2023-10-06T05:01:22.921946+0000","last_became_active":"2023-10-06T05:01:21.900456+0000","last_became_peered":"2023-10-06T05:01:21.900456+0000","last_unstale":"2023-10-06T05:01:22.921946+0000","last_undegraded":"2023-10-06T05:01:22.921946+0000","last_fullsized":"2023-10-06T05:01:22.921946+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:10.205802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916048+0000","last_change":"2023-10-06T05:01:22.913467+0000","last_active":"2023-10-06T05:01:22.916048+0000","last_peered":"2023-10-06T05:01:22.916048+0000","last_clean":"2023-10-06T05:01:22.916048+0000","last_became_active":"2023-10-06T05:01:22.913133+0000","last_became_peered":"2023-10-06T05:01:22.913133+0000","last_unstale":"2023-10-06T05:01:22.916048+0000","last_undegraded":"2023-10-06T05:01:22.916048+0000","last_fullsized":"2023-10-06T05:01:22.916048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:41.139543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.901055+0000","last_change":"2023-10-06T05:01:21.907168+0000","last_active":"2023-10-06T05:01:22.901055+0000","last_peered":"2023-10-06T05:01:22.901055+0000","last_clean":"2023-10-06T05:01:22.901055+0000","last_became_active":"2023-10-06T05:01:21.906798+0000","last_became_peered":"2023-10-06T05:01:21.906798+0000","last_unstale":"2023-10-06T05:01:22.901055+0000","last_undegraded":"2023-10-06T05:01:22.901055+0000","last_fullsized":"2023-10-06T05:01:22.901055+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919507+0000","last_change":"2023-10-06T05:01:22.917117+0000","last_active":"2023-10-06T05:01:22.919507+0000","last_peered":"2023-10-06T05:01:22.919507+0000","last_clean":"2023-10-06T05:01:22.919507+0000","last_became_active":"2023-10-06T05:01:22.916829+0000","last_became_peered":"2023-10-06T05:01:22.916829+0000","last_unstale":"2023-10-06T05:01:22.919507+0000","last_undegraded":"2023-10-06T05:01:22.919507+0000","last_fullsized":"2023-10-06T05:01:22.919507+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.896584+0000","last_change":"2023-10-06T05:01:21.897514+0000","last_active":"2023-10-06T05:01:22.896584+0000","last_peered":"2023-10-06T05:01:22.896584+0000","last_clean":"2023-10-06T05:01:22.896584+0000","last_became_active":"2023-10-06T05:01:21.897226+0000","last_became_peered":"2023-10-06T05:01:21.897226+0000","last_unstale":"2023-10-06T05:01:22.896584+0000","last_undegraded":"2023-10-06T05:01:22.896584+0000","last_fullsized":"2023-10-06T05:01:22.896584+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:44.413476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917870+0000","last_change":"2023-10-06T05:01:22.916955+0000","last_active":"2023-10-06T05:01:22.917870+0000","last_peered":"2023-10-06T05:01:22.917870+0000","last_clean":"2023-10-06T05:01:22.917870+0000","last_became_active":"2023-10-06T05:01:22.916640+0000","last_became_peered":"2023-10-06T05:01:22.916640+0000","last_unstale":"2023-10-06T05:01:22.917870+0000","last_undegraded":"2023-10-06T05:01:22.917870+0000","last_fullsized":"2023-10-06T05:01:22.917870+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:09.844106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891290+0000","last_change":"2023-10-06T05:01:21.906767+0000","last_active":"2023-10-06T05:01:22.891290+0000","last_peered":"2023-10-06T05:01:22.891290+0000","last_clean":"2023-10-06T05:01:22.891290+0000","last_became_active":"2023-10-06T05:01:21.906476+0000","last_became_peered":"2023-10-06T05:01:21.906476+0000","last_unstale":"2023-10-06T05:01:22.891290+0000","last_undegraded":"2023-10-06T05:01:22.891290+0000","last_fullsized":"2023-10-06T05:01:22.891290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921664+0000","last_change":"2023-10-06T05:01:22.917699+0000","last_active":"2023-10-06T05:01:22.921664+0000","last_peered":"2023-10-06T05:01:22.921664+0000","last_clean":"2023-10-06T05:01:22.921664+0000","last_became_active":"2023-10-06T05:01:22.916786+0000","last_became_peered":"2023-10-06T05:01:22.916786+0000","last_unstale":"2023-10-06T05:01:22.921664+0000","last_undegraded":"2023-10-06T05:01:22.921664+0000","last_fullsized":"2023-10-06T05:01:22.921664+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900665+0000","last_change":"2023-10-06T05:01:21.910269+0000","last_active":"2023-10-06T05:01:22.900665+0000","last_peered":"2023-10-06T05:01:22.900665+0000","last_clean":"2023-10-06T05:01:22.900665+0000","last_became_active":"2023-10-06T05:01:21.909971+0000","last_became_peered":"2023-10-06T05:01:21.909971+0000","last_unstale":"2023-10-06T05:01:22.900665+0000","last_undegraded":"2023-10-06T05:01:22.900665+0000","last_fullsized":"2023-10-06T05:01:22.900665+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920563+0000","last_change":"2023-10-06T05:01:22.919007+0000","last_active":"2023-10-06T05:01:22.920563+0000","last_peered":"2023-10-06T05:01:22.920563+0000","last_clean":"2023-10-06T05:01:22.920563+0000","last_became_active":"2023-10-06T05:01:22.911208+0000","last_became_peered":"2023-10-06T05:01:22.911208+0000","last_unstale":"2023-10-06T05:01:22.920563+0000","last_undegraded":"2023-10-06T05:01:22.920563+0000","last_fullsized":"2023-10-06T05:01:22.920563+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:08.817032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.894440+0000","last_change":"2023-10-06T05:01:21.902093+0000","last_active":"2023-10-06T05:01:22.894440+0000","last_peered":"2023-10-06T05:01:22.894440+0000","last_clean":"2023-10-06T05:01:22.894440+0000","last_became_active":"2023-10-06T05:01:21.900927+0000","last_became_peered":"2023-10-06T05:01:21.900927+0000","last_unstale":"2023-10-06T05:01:22.894440+0000","last_undegraded":"2023-10-06T05:01:22.894440+0000","last_fullsized":"2023-10-06T05:01:22.894440+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:16.639594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918752+0000","last_change":"2023-10-06T05:01:22.917989+0000","last_active":"2023-10-06T05:01:22.918752+0000","last_peered":"2023-10-06T05:01:22.918752+0000","last_clean":"2023-10-06T05:01:22.918752+0000","last_became_active":"2023-10-06T05:01:22.917665+0000","last_became_peered":"2023-10-06T05:01:22.917665+0000","last_unstale":"2023-10-06T05:01:22.918752+0000","last_undegraded":"2023-10-06T05:01:22.918752+0000","last_fullsized":"2023-10-06T05:01:22.918752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:10.275597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900723+0000","last_change":"2023-10-06T05:01:21.908132+0000","last_active":"2023-10-06T05:01:22.900723+0000","last_peered":"2023-10-06T05:01:22.900723+0000","last_clean":"2023-10-06T05:01:22.900723+0000","last_became_active":"2023-10-06T05:01:21.907727+0000","last_became_peered":"2023-10-06T05:01:21.907727+0000","last_unstale":"2023-10-06T05:01:22.900723+0000","last_undegraded":"2023-10-06T05:01:22.900723+0000","last_fullsized":"2023-10-06T05:01:22.900723+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:19.166059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912985+0000","last_change":"2023-10-06T05:01:22.911853+0000","last_active":"2023-10-06T05:01:22.912985+0000","last_peered":"2023-10-06T05:01:22.912985+0000","last_clean":"2023-10-06T05:01:22.912985+0000","last_became_active":"2023-10-06T05:01:22.911562+0000","last_became_peered":"2023-10-06T05:01:22.911562+0000","last_unstale":"2023-10-06T05:01:22.912985+0000","last_undegraded":"2023-10-06T05:01:22.912985+0000","last_fullsized":"2023-10-06T05:01:22.912985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898029+0000","last_change":"2023-10-06T05:01:21.908600+0000","last_active":"2023-10-06T05:01:22.898029+0000","last_peered":"2023-10-06T05:01:22.898029+0000","last_clean":"2023-10-06T05:01:22.898029+0000","last_became_active":"2023-10-06T05:01:21.908261+0000","last_became_peered":"2023-10-06T05:01:21.908261+0000","last_unstale":"2023-10-06T05:01:22.898029+0000","last_undegraded":"2023-10-06T05:01:22.898029+0000","last_fullsized":"2023-10-06T05:01:22.898029+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:23.366677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910946+0000","last_change":"2023-10-06T05:01:22.906979+0000","last_active":"2023-10-06T05:01:22.910946+0000","last_peered":"2023-10-06T05:01:22.910946+0000","last_clean":"2023-10-06T05:01:22.910946+0000","last_became_active":"2023-10-06T05:01:22.906652+0000","last_became_peered":"2023-10-06T05:01:22.906652+0000","last_unstale":"2023-10-06T05:01:22.910946+0000","last_undegraded":"2023-10-06T05:01:22.910946+0000","last_fullsized":"2023-10-06T05:01:22.910946+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:42.125134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.899072+0000","last_change":"2023-10-06T05:01:21.897963+0000","last_active":"2023-10-06T05:01:22.899072+0000","last_peered":"2023-10-06T05:01:22.899072+0000","last_clean":"2023-10-06T05:01:22.899072+0000","last_became_active":"2023-10-06T05:01:21.897684+0000","last_became_peered":"2023-10-06T05:01:21.897684+0000","last_unstale":"2023-10-06T05:01:22.899072+0000","last_undegraded":"2023-10-06T05:01:22.899072+0000","last_fullsized":"2023-10-06T05:01:22.899072+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:11.842466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.904603+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:01:22.904603+0000","last_peered":"2023-10-06T05:01:22.904603+0000","last_clean":"2023-10-06T05:01:22.904603+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:01:22.904603+0000","last_undegraded":"2023-10-06T05:01:22.904603+0000","last_fullsized":"2023-10-06T05:01:22.904603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912616+0000","last_change":"2023-10-06T05:01:22.911622+0000","last_active":"2023-10-06T05:01:22.912616+0000","last_peered":"2023-10-06T05:01:22.912616+0000","last_clean":"2023-10-06T05:01:22.912616+0000","last_became_active":"2023-10-06T05:01:22.911388+0000","last_became_peered":"2023-10-06T05:01:22.911388+0000","last_unstale":"2023-10-06T05:01:22.912616+0000","last_undegraded":"2023-10-06T05:01:22.912616+0000","last_fullsized":"2023-10-06T05:01:22.912616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.004036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920959+0000","last_change":"2023-10-06T05:01:21.907272+0000","last_active":"2023-10-06T05:01:22.920959+0000","last_peered":"2023-10-06T05:01:22.920959+0000","last_clean":"2023-10-06T05:01:22.920959+0000","last_became_active":"2023-10-06T05:01:21.906971+0000","last_became_peered":"2023-10-06T05:01:21.906971+0000","last_unstale":"2023-10-06T05:01:22.920959+0000","last_undegraded":"2023-10-06T05:01:22.920959+0000","last_fullsized":"2023-10-06T05:01:22.920959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891246+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:01:22.891246+0000","last_peered":"2023-10-06T05:01:22.891246+0000","last_clean":"2023-10-06T05:01:22.891246+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:01:22.891246+0000","last_undegraded":"2023-10-06T05:01:22.891246+0000","last_fullsized":"2023-10-06T05:01:22.891246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912780+0000","last_change":"2023-10-06T05:01:22.911349+0000","last_active":"2023-10-06T05:01:22.912780+0000","last_peered":"2023-10-06T05:01:22.912780+0000","last_clean":"2023-10-06T05:01:22.912780+0000","last_became_active":"2023-10-06T05:01:22.906441+0000","last_became_peered":"2023-10-06T05:01:22.906441+0000","last_unstale":"2023-10-06T05:01:22.912780+0000","last_undegraded":"2023-10-06T05:01:22.912780+0000","last_fullsized":"2023-10-06T05:01:22.912780+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.902904+0000","last_change":"2023-10-06T05:01:21.905784+0000","last_active":"2023-10-06T05:01:22.902904+0000","last_peered":"2023-10-06T05:01:22.902904+0000","last_clean":"2023-10-06T05:01:22.902904+0000","last_became_active":"2023-10-06T05:01:21.905522+0000","last_became_peered":"2023-10-06T05:01:21.905522+0000","last_unstale":"2023-10-06T05:01:22.902904+0000","last_undegraded":"2023-10-06T05:01:22.902904+0000","last_fullsized":"2023-10-06T05:01:22.902904+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:31.320712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":107,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897470+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:01:22.897470+0000","last_peered":"2023-10-06T05:01:22.897470+0000","last_clean":"2023-10-06T05:01:22.897470+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:01:22.897470+0000","last_undegraded":"2023-10-06T05:01:22.897470+0000","last_fullsized":"2023-10-06T05:01:22.897470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":254,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898054+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:01:22.898054+0000","last_peered":"2023-10-06T05:01:22.898054+0000","last_clean":"2023-10-06T05:01:22.898054+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:01:22.898054+0000","last_undegraded":"2023-10-06T05:01:22.898054+0000","last_fullsized":"2023-10-06T05:01:22.898054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915904+0000","last_change":"2023-10-06T05:01:22.914069+0000","last_active":"2023-10-06T05:01:22.915904+0000","last_peered":"2023-10-06T05:01:22.915904+0000","last_clean":"2023-10-06T05:01:22.915904+0000","last_became_active":"2023-10-06T05:01:22.913789+0000","last_became_peered":"2023-10-06T05:01:22.913789+0000","last_unstale":"2023-10-06T05:01:22.915904+0000","last_undegraded":"2023-10-06T05:01:22.915904+0000","last_fullsized":"2023-10-06T05:01:22.915904+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:37.179875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900942+0000","last_change":"2023-10-06T05:01:21.907735+0000","last_active":"2023-10-06T05:01:22.900942+0000","last_peered":"2023-10-06T05:01:22.900942+0000","last_clean":"2023-10-06T05:01:22.900942+0000","last_became_active":"2023-10-06T05:01:21.907455+0000","last_became_peered":"2023-10-06T05:01:21.907455+0000","last_unstale":"2023-10-06T05:01:22.900942+0000","last_undegraded":"2023-10-06T05:01:22.900942+0000","last_fullsized":"2023-10-06T05:01:22.900942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:50.355734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897481+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:01:22.897481+0000","last_peered":"2023-10-06T05:01:22.897481+0000","last_clean":"2023-10-06T05:01:22.897481+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:01:22.897481+0000","last_undegraded":"2023-10-06T05:01:22.897481+0000","last_fullsized":"2023-10-06T05:01:22.897481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912546+0000","last_change":"2023-10-06T05:01:22.904787+0000","last_active":"2023-10-06T05:01:22.912546+0000","last_peered":"2023-10-06T05:01:22.912546+0000","last_clean":"2023-10-06T05:01:22.912546+0000","last_became_active":"2023-10-06T05:01:22.904208+0000","last_became_peered":"2023-10-06T05:01:22.904208+0000","last_unstale":"2023-10-06T05:01:22.912546+0000","last_undegraded":"2023-10-06T05:01:22.912546+0000","last_fullsized":"2023-10-06T05:01:22.912546+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.925929+0000","last_change":"2023-10-06T05:01:21.900909+0000","last_active":"2023-10-06T05:01:22.925929+0000","last_peered":"2023-10-06T05:01:22.925929+0000","last_clean":"2023-10-06T05:01:22.925929+0000","last_became_active":"2023-10-06T05:01:21.900571+0000","last_became_peered":"2023-10-06T05:01:21.900571+0000","last_unstale":"2023-10-06T05:01:22.925929+0000","last_undegraded":"2023-10-06T05:01:22.925929+0000","last_fullsized":"2023-10-06T05:01:22.925929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898113+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:01:22.898113+0000","last_peered":"2023-10-06T05:01:22.898113+0000","last_clean":"2023-10-06T05:01:22.898113+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:01:22.898113+0000","last_undegraded":"2023-10-06T05:01:22.898113+0000","last_fullsized":"2023-10-06T05:01:22.898113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912543+0000","last_change":"2023-10-06T05:01:22.910384+0000","last_active":"2023-10-06T05:01:22.912543+0000","last_peered":"2023-10-06T05:01:22.912543+0000","last_clean":"2023-10-06T05:01:22.912543+0000","last_became_active":"2023-10-06T05:01:22.909894+0000","last_became_peered":"2023-10-06T05:01:22.909894+0000","last_unstale":"2023-10-06T05:01:22.912543+0000","last_undegraded":"2023-10-06T05:01:22.912543+0000","last_fullsized":"2023-10-06T05:01:22.912543+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:03.895804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909184+0000","last_change":"2023-10-06T05:01:21.898111+0000","last_active":"2023-10-06T05:01:22.909184+0000","last_peered":"2023-10-06T05:01:22.909184+0000","last_clean":"2023-10-06T05:01:22.909184+0000","last_became_active":"2023-10-06T05:01:21.897816+0000","last_became_peered":"2023-10-06T05:01:21.897816+0000","last_unstale":"2023-10-06T05:01:22.909184+0000","last_undegraded":"2023-10-06T05:01:22.909184+0000","last_fullsized":"2023-10-06T05:01:22.909184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:05.782643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900952+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:01:22.900952+0000","last_peered":"2023-10-06T05:01:22.900952+0000","last_clean":"2023-10-06T05:01:22.900952+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:01:22.900952+0000","last_undegraded":"2023-10-06T05:01:22.900952+0000","last_fullsized":"2023-10-06T05:01:22.900952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921053+0000","last_change":"2023-10-06T05:01:22.920569+0000","last_active":"2023-10-06T05:01:22.921053+0000","last_peered":"2023-10-06T05:01:22.921053+0000","last_clean":"2023-10-06T05:01:22.921053+0000","last_became_active":"2023-10-06T05:01:22.920240+0000","last_became_peered":"2023-10-06T05:01:22.920240+0000","last_unstale":"2023-10-06T05:01:22.921053+0000","last_undegraded":"2023-10-06T05:01:22.921053+0000","last_fullsized":"2023-10-06T05:01:22.921053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.906141+0000","last_change":"2023-10-06T05:01:21.901057+0000","last_active":"2023-10-06T05:01:22.906141+0000","last_peered":"2023-10-06T05:01:22.906141+0000","last_clean":"2023-10-06T05:01:22.906141+0000","last_became_active":"2023-10-06T05:01:21.900783+0000","last_became_peered":"2023-10-06T05:01:21.900783+0000","last_unstale":"2023-10-06T05:01:22.906141+0000","last_undegraded":"2023-10-06T05:01:22.906141+0000","last_fullsized":"2023-10-06T05:01:22.906141+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:13.427002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900563+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:01:22.900563+0000","last_peered":"2023-10-06T05:01:22.900563+0000","last_clean":"2023-10-06T05:01:22.900563+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:01:22.900563+0000","last_undegraded":"2023-10-06T05:01:22.900563+0000","last_fullsized":"2023-10-06T05:01:22.900563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918799+0000","last_change":"2023-10-06T05:01:22.915800+0000","last_active":"2023-10-06T05:01:22.918799+0000","last_peered":"2023-10-06T05:01:22.918799+0000","last_clean":"2023-10-06T05:01:22.918799+0000","last_became_active":"2023-10-06T05:01:22.915546+0000","last_became_peered":"2023-10-06T05:01:22.915546+0000","last_unstale":"2023-10-06T05:01:22.918799+0000","last_undegraded":"2023-10-06T05:01:22.918799+0000","last_fullsized":"2023-10-06T05:01:22.918799+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909478+0000","last_change":"2023-10-06T05:01:21.905554+0000","last_active":"2023-10-06T05:01:22.909478+0000","last_peered":"2023-10-06T05:01:22.909478+0000","last_clean":"2023-10-06T05:01:22.909478+0000","last_became_active":"2023-10-06T05:01:21.905222+0000","last_became_peered":"2023-10-06T05:01:21.905222+0000","last_unstale":"2023-10-06T05:01:22.909478+0000","last_undegraded":"2023-10-06T05:01:22.909478+0000","last_fullsized":"2023-10-06T05:01:22.909478+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910017+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:01:22.910017+0000","last_peered":"2023-10-06T05:01:22.910017+0000","last_clean":"2023-10-06T05:01:22.910017+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:01:22.910017+0000","last_undegraded":"2023-10-06T05:01:22.910017+0000","last_fullsized":"2023-10-06T05:01:22.910017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.73899999999999999,"5min":0.86599999999999999,"15min":7.1779999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.73499999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.311},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":7.1879999999999997},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.073},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":1.1379999999999999},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.111},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.0880000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.874},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.214},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1658880,"data_stored":753838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58799999999999997,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.73399999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63400000000000001,"15min":0.57499999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.65500000000000003,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.312},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.77900000000000003,"5min":1.234,"15min":1.234},"last":0.37}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.63400000000000001,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70099999999999996,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.35199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.755,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61199999999999999,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.68100000000000005,"5min":3.669,"15min":3.669},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.79300000000000004,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35599999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.437,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":0.375},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.378,"15min":0.378},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.60499999999999998,"5min":0.68600000000000005,"15min":0.69399999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.622,"15min":0.70399999999999996},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.56000000000000005,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.47199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.77300000000000002,"5min":0.877,"15min":1.0780000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.998,"15min":0.998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.54400000000000004,"5min":0.496,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":1.022,"15min":1.0429999999999999},"last":0.83399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":755982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.59199999999999997,"5min":0.628,"15min":0.628},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42999999999999999,"15min":0.38800000000000001},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46100000000000002,"15min":0.443},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.98299999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.85799999999999998,"15min":0.92300000000000004},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.91700000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.93400000000000005,"15min":1.0109999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.52200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.97599999999999998,"15min":1.0329999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":0.995},"last":0.71499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":753880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.20300000000000001,"15min":0.186},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.048},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.39300000000000002},"min":{"1min":0.32300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.216},"max":{"1min":0.52400000000000002,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.219},"max":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.36499999999999999},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.90200000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.2649999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.88100000000000001,"15min":0.93100000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":0.88300000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.82499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307104,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064736,"statfs":{"total":96636764160,"available":96322289664,"internally_reserved":0,"allocated":1662976,"data_stored":754356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56399999999999995,"5min":0.68200000000000005,"15min":2.008},"last":0.22},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.53000000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":6.4370000000000003},"last":0.28299999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.224,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.67000000000000004,"15min":1.7649999999999999},"last":0.312},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":2.1970000000000001},"last":0.29999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42399999999999999,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.255},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":5.069},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.441,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":2.246},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59499999999999997,"15min":0.76100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":14.646000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":14.606999999999999},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.81399999999999995,"5min":0.94699999999999995,"15min":2.2959999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":6.4009999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.73299999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.90100000000000002,"15min":5.0309999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":0.87,"5min":1.012,"15min":2.2749999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.75800000000000001},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.92600000000000005,"15min":6.367},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":5},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307108,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064732,"statfs":{"total":96636764160,"available":96322285568,"internally_reserved":0,"allocated":1667072,"data_stored":755304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.70899999999999996},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.73799999999999999},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.74099999999999999},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":5.4900000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.628},"min":{"1min":0.433,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.68200000000000005,"5min":0.75900000000000001,"15min":4.0739999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53200000000000003,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":5.415},"last":0.39200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.83199999999999996,"15min":4.202},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.73199999999999998,"5min":0.81000000000000005,"15min":4.2130000000000001},"last":0.623}]},{"osd":5,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46700000000000003,"15min":0.49399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.73399999999999999,"15min":1.1279999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.73399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51400000000000001,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.73199999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.79500000000000004,"5min":0.86099999999999999,"15min":1.1000000000000001},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307092,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064748,"statfs":{"total":96636764160,"available":96322301952,"internally_reserved":0,"allocated":1650688,"data_stored":752415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.20499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":4.9039999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.75800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57499999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":6.7949999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.26200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.86599999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.623},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.86599999999999999},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53800000000000003,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":7,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:01:27.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T05:01:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.953+0000 7f3db6dfa700 1 Processor -- start 2023-10-06T05:01:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.953+0000 7f3db6dfa700 1 -- start start 2023-10-06T05:01:27.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.953+0000 7f3db6dfa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3db00b27b0 0x7f3db015a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.953+0000 7f3db6dfa700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db015d240 0x7f3db00ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.953+0000 7f3db6dfa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db00cd5f0 con 0x7f3db00cd150 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.953+0000 7f3db6dfa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db015aba0 con 0x7f3db00b27b0 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.953+0000 7f3db6dfa700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0150b30 con 0x7f3db015d240 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db4b96700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3db00b27b0 0x7f3db015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db5397700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3db00cd150 0x7f3db00aeef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40200/0 (socket says 172.21.15.3:40200) 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db5397700 1 -- 172.21.15.3:0/3228833026 learned_addr learned my addr 172.21.15.3:0/3228833026 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3daffff700 1 --2- 172.21.15.3:0/3228833026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db015d240 0x7f3db00ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db4b96700 1 -- 172.21.15.3:0/3228833026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db015d240 msgr2=0x7f3db00ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:27.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3daf7fe700 1 -- 172.21.15.3:0/3228833026 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217271525 0 0) 0x7f3db00cd5f0 con 0x7f3db00cd150 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db4b96700 1 --2- 172.21.15.3:0/3228833026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db015d240 0x7f3db00ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db4b96700 1 -- 172.21.15.3:0/3228833026 >> v1:172.21.15.3:6789/0 conn(0x7f3db00cd150 legacy=0x7f3db00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db4b96700 1 -- 172.21.15.3:0/3228833026 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db015d6d0 con 0x7f3db00b27b0 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3db4b96700 1 --2- 172.21.15.3:0/3228833026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db00b27b0 0x7f3db015a660 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009ee0 tx=0x7f3da8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe3f1091c3fa39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3daf7fe700 1 -- 172.21.15.3:0/3228833026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da800e070 con 0x7f3db00b27b0 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3daf7fe700 1 -- 172.21.15.3:0/3228833026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da800bc10 con 0x7f3db00b27b0 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.954+0000 7f3daf7fe700 1 -- 172.21.15.3:0/3228833026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8012430 con 0x7f3db00b27b0 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 -- 172.21.15.3:0/3228833026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db00b27b0 msgr2=0x7f3db015a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/3228833026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db00b27b0 0x7f3db015a660 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009ee0 tx=0x7f3da8009780 comp rx=0 tx=0).stop 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 -- 172.21.15.3:0/3228833026 shutdown_connections 2023-10-06T05:01:27.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/3228833026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db00b27b0 0x7f3db015a660 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/3228833026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db015d240 0x7f3db00ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 -- 172.21.15.3:0/3228833026 >> 172.21.15.3:0/3228833026 conn(0x7f3db0070310 msgr2=0x7f3db0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 -- 172.21.15.3:0/3228833026 shutdown_connections 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.955+0000 7f3db6dfa700 1 -- 172.21.15.3:0/3228833026 wait complete. 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.956+0000 7f3db6dfa700 1 Processor -- start 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.956+0000 7f3db6dfa700 1 -- start start 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db6dfa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db00b27b0 0x7f3db014f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db6dfa700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db00cd150 0x7f3db00c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db6dfa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db015d240 0x7f3db00c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db6dfa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00b5d40 con 0x7f3db015d240 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db6dfa700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00b3220 con 0x7f3db00b27b0 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db6dfa700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00b3d60 con 0x7f3db00cd150 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db015d240 0x7f3db00c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db015d240 0x7f3db00c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50504/0 (socket says 172.21.15.3:50504) 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 -- 172.21.15.3:0/1268334484 learned_addr learned my addr 172.21.15.3:0/1268334484 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3daffff700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db00cd150 0x7f3db00c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db4b96700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db00b27b0 0x7f3db014f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 -- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db00cd150 msgr2=0x7f3db00c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db00cd150 0x7f3db00c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 -- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db00b27b0 msgr2=0x7f3db014f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db00b27b0 0x7f3db014f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:27.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 -- 172.21.15.3:0/1268334484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8009a20 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db4b96700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db00b27b0 0x7f3db014f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.957+0000 7f3db5397700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db015d240 0x7f3db00c6ae0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c7e0 tx=0x7f3d98007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.958+0000 7f3dacff9700 1 -- 172.21.15.3:0/1268334484 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98005c00 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.958+0000 7f3dacff9700 1 -- 172.21.15.3:0/1268334484 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d98005d80 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.958+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db00b3430 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.958+0000 7f3dacff9700 1 -- 172.21.15.3:0/1268334484 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98007320 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.958+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db00b39f0 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.959+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db00b3f10 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.959+0000 7f3dacff9700 1 -- 172.21.15.3:0/1268334484 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d9801b3b0 con 0x7f3db015d240 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.959+0000 7f3dacff9700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d90066f30 0x7f3d900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:27.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.959+0000 7f3dacff9700 1 -- 172.21.15.3:0/1268334484 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d98083c40 con 0x7f3db015d240 2023-10-06T05:01:27.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.960+0000 7f3db4b96700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d90066f30 0x7f3d900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:27.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.960+0000 7f3db4b96700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d90066f30 0x7f3d900693e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097c0 tx=0x7f3da8019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:27.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:27.964+0000 7f3dacff9700 1 -- 172.21.15.3:0/1268334484 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3db00b3f10 con 0x7f3db015d240 2023-10-06T05:01:28.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.238+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3db00b3ed0 con 0x7f3db015d240 2023-10-06T05:01:28.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.238+0000 7f3dacff9700 1 -- 172.21.15.3:0/1268334484 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1440 (secure 0 0 0) 0x7f3d98017020 con 0x7f3db015d240 2023-10-06T05:01:28.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:28.240 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":568,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1696567930,"num_in_osds":8,"osd_in_since":1696567922,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2508582912,"bytes_avail":770585530368,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":89,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"b","status":"up:active","gid":8409}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-06T05:01:09.554508+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T05:01:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d90066f30 msgr2=0x7f3d900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d90066f30 0x7f3d900693e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097c0 tx=0x7f3da8019000 comp rx=0 tx=0).stop 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db015d240 msgr2=0x7f3db00c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db015d240 0x7f3db00c6ae0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c7e0 tx=0x7f3d98007d90 comp rx=0 tx=0).stop 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 shutdown_connections 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d90066f30 0x7f3d900693e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db00b27b0 0x7f3db014f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3db015d240 0x7f3db00c6ae0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 --2- 172.21.15.3:0/1268334484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3db00cd150 0x7f3db00c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 >> 172.21.15.3:0/1268334484 conn(0x7f3db0070310 msgr2=0x7f3db00af350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:28.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 shutdown_connections 2023-10-06T05:01:28.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.240+0000 7f3db6dfa700 1 -- 172.21.15.3:0/1268334484 wait complete. 2023-10-06T05:01:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T05:01:28.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:01:28.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.371+0000 7f2b65e9a700 1 Processor -- start 2023-10-06T05:01:28.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.371+0000 7f2b65e9a700 1 -- start start 2023-10-06T05:01:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.372+0000 7f2b65e9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015cd00 0x7f2b6015d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.372+0000 7f2b65e9a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2b6006aad0 0x7f2b600b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.372+0000 7f2b65e9a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b600c93f0 con 0x7f2b6015c930 2023-10-06T05:01:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.372+0000 7f2b65e9a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b600caf50 con 0x7f2b6006aad0 2023-10-06T05:01:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.372+0000 7f2b65e9a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6015e0d0 con 0x7f2b6015cd00 2023-10-06T05:01:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.372+0000 7f2b5ffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2b6006aad0 0x7f2b600b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.372+0000 7f2b5effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015cd00 0x7f2b6015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2b6015c930 0x7f2b6006c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:42078/0 (socket says 172.21.15.3:42078) 2023-10-06T05:01:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5f7fe700 1 -- 172.21.15.3:0/1378007333 learned_addr learned my addr 172.21.15.3:0/1378007333 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5effd700 1 -- 172.21.15.3:0/1378007333 >> v2:172.21.15.153:3300/0 conn(0x7f2b6006aad0 msgr2=0x7f2b600b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5effd700 1 --2- 172.21.15.3:0/1378007333 >> v2:172.21.15.153:3300/0 conn(0x7f2b6006aad0 0x7f2b600b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5effd700 1 -- 172.21.15.3:0/1378007333 >> v1:172.21.15.153:6789/0 conn(0x7f2b6015c930 legacy=0x7f2b6006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5e7fc700 1 -- 172.21.15.3:0/1378007333 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179409285 0 0) 0x7f2b600c93f0 con 0x7f2b6015c930 2023-10-06T05:01:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5effd700 1 -- 172.21.15.3:0/1378007333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6015d830 con 0x7f2b6015cd00 2023-10-06T05:01:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5ffff700 1 --2- 172.21.15.3:0/1378007333 >> v2:172.21.15.153:3300/0 conn(0x7f2b6006aad0 0x7f2b600b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5effd700 1 --2- 172.21.15.3:0/1378007333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015cd00 0x7f2b6015d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098e0 tx=0x7f2b540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c4cc727181c3d9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5e7fc700 1 -- 172.21.15.3:0/1378007333 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400b640 con 0x7f2b6015cd00 2023-10-06T05:01:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5e7fc700 1 -- 172.21.15.3:0/1378007333 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5400b7a0 con 0x7f2b6015cd00 2023-10-06T05:01:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.373+0000 7f2b5e7fc700 1 -- 172.21.15.3:0/1378007333 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400bb40 con 0x7f2b6015cd00 2023-10-06T05:01:28.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 -- 172.21.15.3:0/1378007333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015cd00 msgr2=0x7f2b6015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/1378007333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015cd00 0x7f2b6015d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098e0 tx=0x7f2b540091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 -- 172.21.15.3:0/1378007333 shutdown_connections 2023-10-06T05:01:28.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/1378007333 >> v2:172.21.15.153:3300/0 conn(0x7f2b6006aad0 0x7f2b600b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/1378007333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015cd00 0x7f2b6015d150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 -- 172.21.15.3:0/1378007333 >> 172.21.15.3:0/1378007333 conn(0x7f2b60066fb0 msgr2=0x7f2b600673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 -- 172.21.15.3:0/1378007333 shutdown_connections 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.374+0000 7f2b65e9a700 1 -- 172.21.15.3:0/1378007333 wait complete. 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.375+0000 7f2b65e9a700 1 Processor -- start 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.375+0000 7f2b65e9a700 1 -- start start 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b65e9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b6006aad0 0x7f2b60153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b65e9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015c930 0x7f2b601432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b65e9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b601503f0 0x7f2b600c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b65e9a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60153e60 con 0x7f2b601503f0 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b65e9a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601541f0 con 0x7f2b6006aad0 2023-10-06T05:01:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b65e9a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60154440 con 0x7f2b6015c930 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b601503f0 0x7f2b600c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b601503f0 0x7f2b600c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50520/0 (socket says 172.21.15.3:50520) 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5ffff700 1 -- 172.21.15.3:0/130044817 learned_addr learned my addr 172.21.15.3:0/130044817 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5f7fe700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b6006aad0 0x7f2b60153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5effd700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015c930 0x7f2b601432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5ffff700 1 -- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015c930 msgr2=0x7f2b601432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5ffff700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015c930 0x7f2b601432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5ffff700 1 -- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b6006aad0 msgr2=0x7f2b60153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.376+0000 7f2b5ffff700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b6006aad0 0x7f2b60153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.377+0000 7f2b5ffff700 1 -- 172.21.15.3:0/130044817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009a20 con 0x7f2b601503f0 2023-10-06T05:01:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.377+0000 7f2b5ffff700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b601503f0 0x7f2b600c5a80 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009fc0 tx=0x7f2b5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.377+0000 7f2b477fe700 1 -- 172.21.15.3:0/130044817 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800cbd0 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.377+0000 7f2b477fe700 1 -- 172.21.15.3:0/130044817 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5800f040 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.377+0000 7f2b477fe700 1 -- 172.21.15.3:0/130044817 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58014480 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.377+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b54009420 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.377+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b60154a00 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.378+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b60071680 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.378+0000 7f2b477fe700 1 -- 172.21.15.3:0/130044817 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b58003020 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.379+0000 7f2b477fe700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b4c066f30 0x7f2b4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.379+0000 7f2b477fe700 1 -- 172.21.15.3:0/130044817 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2b58084fb0 con 0x7f2b601503f0 2023-10-06T05:01:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.379+0000 7f2b5f7fe700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b4c066f30 0x7f2b4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.379+0000 7f2b5f7fe700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b4c066f30 0x7f2b4c0693e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2b50007fd0 tx=0x7f2b50007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.383+0000 7f2b477fe700 1 -- 172.21.15.3:0/130044817 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b580520c0 con 0x7f2b601503f0 2023-10-06T05:01:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.573+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b6015db10 con 0x7f2b4c066f30 2023-10-06T05:01:28.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.583+0000 7f2b477fe700 1 -- 172.21.15.3:0/130044817 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312378 (secure 0 0 0) 0x7f2b6015db10 con 0x7f2b4c066f30 2023-10-06T05:01:28.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:28.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b4c066f30 msgr2=0x7f2b4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b4c066f30 0x7f2b4c0693e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2b50007fd0 tx=0x7f2b50007d30 comp rx=0 tx=0).stop 2023-10-06T05:01:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b601503f0 msgr2=0x7f2b600c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b601503f0 0x7f2b600c5a80 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009fc0 tx=0x7f2b5800b040 comp rx=0 tx=0).stop 2023-10-06T05:01:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 shutdown_connections 2023-10-06T05:01:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2b4c066f30 0x7f2b4c0693e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b6006aad0 0x7f2b60153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b601503f0 0x7f2b600c5a80 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 --2- 172.21.15.3:0/130044817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b6015c930 0x7f2b601432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:28.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 >> 172.21.15.3:0/130044817 conn(0x7f2b60066fb0 msgr2=0x7f2b60067380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:28.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 shutdown_connections 2023-10-06T05:01:28.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:28.586+0000 7f2b65e9a700 1 -- 172.21.15.3:0/130044817 wait complete. 2023-10-06T05:01:28.592 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:01:28.599 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2023-10-06T05:01:27.558620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":193,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449788,"kb_used_data":14140,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524932,"statfs":{"total":773094113280,"available":770585530368,"internally_reserved":0,"allocated":14479360,"data_stored":7211433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001463"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919337+0000","last_change":"2023-10-06T05:01:22.917546+0000","last_active":"2023-10-06T05:01:22.919337+0000","last_peered":"2023-10-06T05:01:22.919337+0000","last_clean":"2023-10-06T05:01:22.919337+0000","last_became_active":"2023-10-06T05:01:22.916374+0000","last_became_peered":"2023-10-06T05:01:22.916374+0000","last_unstale":"2023-10-06T05:01:22.919337+0000","last_undegraded":"2023-10-06T05:01:22.919337+0000","last_fullsized":"2023-10-06T05:01:22.919337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917323+0000","last_change":"2023-10-06T05:01:22.915942+0000","last_active":"2023-10-06T05:01:22.917323+0000","last_peered":"2023-10-06T05:01:22.917323+0000","last_clean":"2023-10-06T05:01:22.917323+0000","last_became_active":"2023-10-06T05:01:22.915602+0000","last_became_peered":"2023-10-06T05:01:22.915602+0000","last_unstale":"2023-10-06T05:01:22.917323+0000","last_undegraded":"2023-10-06T05:01:22.917323+0000","last_fullsized":"2023-10-06T05:01:22.917323+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:39.759900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917901+0000","last_change":"2023-10-06T05:01:22.916859+0000","last_active":"2023-10-06T05:01:22.917901+0000","last_peered":"2023-10-06T05:01:22.917901+0000","last_clean":"2023-10-06T05:01:22.917901+0000","last_became_active":"2023-10-06T05:01:22.916504+0000","last_became_peered":"2023-10-06T05:01:22.916504+0000","last_unstale":"2023-10-06T05:01:22.917901+0000","last_undegraded":"2023-10-06T05:01:22.917901+0000","last_fullsized":"2023-10-06T05:01:22.917901+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:36.075945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915308+0000","last_change":"2023-10-06T05:01:22.914389+0000","last_active":"2023-10-06T05:01:22.915308+0000","last_peered":"2023-10-06T05:01:22.915308+0000","last_clean":"2023-10-06T05:01:22.915308+0000","last_became_active":"2023-10-06T05:01:22.914080+0000","last_became_peered":"2023-10-06T05:01:22.914080+0000","last_unstale":"2023-10-06T05:01:22.915308+0000","last_undegraded":"2023-10-06T05:01:22.915308+0000","last_fullsized":"2023-10-06T05:01:22.915308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:56.228351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912802+0000","last_change":"2023-10-06T05:01:22.908378+0000","last_active":"2023-10-06T05:01:22.912802+0000","last_peered":"2023-10-06T05:01:22.912802+0000","last_clean":"2023-10-06T05:01:22.912802+0000","last_became_active":"2023-10-06T05:01:22.907429+0000","last_became_peered":"2023-10-06T05:01:22.907429+0000","last_unstale":"2023-10-06T05:01:22.912802+0000","last_undegraded":"2023-10-06T05:01:22.912802+0000","last_fullsized":"2023-10-06T05:01:22.912802+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:32.443484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912753+0000","last_change":"2023-10-06T05:01:22.907760+0000","last_active":"2023-10-06T05:01:22.912753+0000","last_peered":"2023-10-06T05:01:22.912753+0000","last_clean":"2023-10-06T05:01:22.912753+0000","last_became_active":"2023-10-06T05:01:22.907466+0000","last_became_peered":"2023-10-06T05:01:22.907466+0000","last_unstale":"2023-10-06T05:01:22.912753+0000","last_undegraded":"2023-10-06T05:01:22.912753+0000","last_fullsized":"2023-10-06T05:01:22.912753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:21.629837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908240+0000","last_change":"2023-10-06T05:01:22.907593+0000","last_active":"2023-10-06T05:01:22.908240+0000","last_peered":"2023-10-06T05:01:22.908240+0000","last_clean":"2023-10-06T05:01:22.908240+0000","last_became_active":"2023-10-06T05:01:22.907290+0000","last_became_peered":"2023-10-06T05:01:22.907290+0000","last_unstale":"2023-10-06T05:01:22.908240+0000","last_undegraded":"2023-10-06T05:01:22.908240+0000","last_fullsized":"2023-10-06T05:01:22.908240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921703+0000","last_change":"2023-10-06T05:01:22.917866+0000","last_active":"2023-10-06T05:01:22.921703+0000","last_peered":"2023-10-06T05:01:22.921703+0000","last_clean":"2023-10-06T05:01:22.921703+0000","last_became_active":"2023-10-06T05:01:22.917142+0000","last_became_peered":"2023-10-06T05:01:22.917142+0000","last_unstale":"2023-10-06T05:01:22.921703+0000","last_undegraded":"2023-10-06T05:01:22.921703+0000","last_fullsized":"2023-10-06T05:01:22.921703+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910976+0000","last_change":"2023-10-06T05:01:22.906539+0000","last_active":"2023-10-06T05:01:22.910976+0000","last_peered":"2023-10-06T05:01:22.910976+0000","last_clean":"2023-10-06T05:01:22.910976+0000","last_became_active":"2023-10-06T05:01:22.905906+0000","last_became_peered":"2023-10-06T05:01:22.905906+0000","last_unstale":"2023-10-06T05:01:22.910976+0000","last_undegraded":"2023-10-06T05:01:22.910976+0000","last_fullsized":"2023-10-06T05:01:22.910976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:33.022370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908499+0000","last_change":"2023-10-06T05:01:22.906059+0000","last_active":"2023-10-06T05:01:22.908499+0000","last_peered":"2023-10-06T05:01:22.908499+0000","last_clean":"2023-10-06T05:01:22.908499+0000","last_became_active":"2023-10-06T05:01:22.905782+0000","last_became_peered":"2023-10-06T05:01:22.905782+0000","last_unstale":"2023-10-06T05:01:22.908499+0000","last_undegraded":"2023-10-06T05:01:22.908499+0000","last_fullsized":"2023-10-06T05:01:22.908499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:31.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916007+0000","last_change":"2023-10-06T05:01:22.914642+0000","last_active":"2023-10-06T05:01:22.916007+0000","last_peered":"2023-10-06T05:01:22.916007+0000","last_clean":"2023-10-06T05:01:22.916007+0000","last_became_active":"2023-10-06T05:01:22.913781+0000","last_became_peered":"2023-10-06T05:01:22.913781+0000","last_unstale":"2023-10-06T05:01:22.916007+0000","last_undegraded":"2023-10-06T05:01:22.916007+0000","last_fullsized":"2023-10-06T05:01:22.916007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:39.375404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915290+0000","last_change":"2023-10-06T05:01:22.913641+0000","last_active":"2023-10-06T05:01:22.915290+0000","last_peered":"2023-10-06T05:01:22.915290+0000","last_clean":"2023-10-06T05:01:22.915290+0000","last_became_active":"2023-10-06T05:01:22.913356+0000","last_became_peered":"2023-10-06T05:01:22.913356+0000","last_unstale":"2023-10-06T05:01:22.915290+0000","last_undegraded":"2023-10-06T05:01:22.915290+0000","last_fullsized":"2023-10-06T05:01:22.915290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:50.229096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909396+0000","last_change":"2023-10-06T05:01:22.906295+0000","last_active":"2023-10-06T05:01:22.909396+0000","last_peered":"2023-10-06T05:01:22.909396+0000","last_clean":"2023-10-06T05:01:22.909396+0000","last_became_active":"2023-10-06T05:01:22.905241+0000","last_became_peered":"2023-10-06T05:01:22.905241+0000","last_unstale":"2023-10-06T05:01:22.909396+0000","last_undegraded":"2023-10-06T05:01:22.909396+0000","last_fullsized":"2023-10-06T05:01:22.909396+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:56.323569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915262+0000","last_change":"2023-10-06T05:01:22.913298+0000","last_active":"2023-10-06T05:01:22.915262+0000","last_peered":"2023-10-06T05:01:22.915262+0000","last_clean":"2023-10-06T05:01:22.915262+0000","last_became_active":"2023-10-06T05:01:22.913012+0000","last_became_peered":"2023-10-06T05:01:22.913012+0000","last_unstale":"2023-10-06T05:01:22.915262+0000","last_undegraded":"2023-10-06T05:01:22.915262+0000","last_fullsized":"2023-10-06T05:01:22.915262+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:31.315774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921375+0000","last_change":"2023-10-06T05:01:22.920640+0000","last_active":"2023-10-06T05:01:22.921375+0000","last_peered":"2023-10-06T05:01:22.921375+0000","last_clean":"2023-10-06T05:01:22.921375+0000","last_became_active":"2023-10-06T05:01:22.920435+0000","last_became_peered":"2023-10-06T05:01:22.920435+0000","last_unstale":"2023-10-06T05:01:22.921375+0000","last_undegraded":"2023-10-06T05:01:22.921375+0000","last_fullsized":"2023-10-06T05:01:22.921375+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:21.298643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921735+0000","last_change":"2023-10-06T05:01:22.921275+0000","last_active":"2023-10-06T05:01:22.921735+0000","last_peered":"2023-10-06T05:01:22.921735+0000","last_clean":"2023-10-06T05:01:22.921735+0000","last_became_active":"2023-10-06T05:01:22.920953+0000","last_became_peered":"2023-10-06T05:01:22.920953+0000","last_unstale":"2023-10-06T05:01:22.921735+0000","last_undegraded":"2023-10-06T05:01:22.921735+0000","last_fullsized":"2023-10-06T05:01:22.921735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:22.455454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921292+0000","last_change":"2023-10-06T05:01:22.919416+0000","last_active":"2023-10-06T05:01:22.921292+0000","last_peered":"2023-10-06T05:01:22.921292+0000","last_clean":"2023-10-06T05:01:22.921292+0000","last_became_active":"2023-10-06T05:01:22.917490+0000","last_became_peered":"2023-10-06T05:01:22.917490+0000","last_unstale":"2023-10-06T05:01:22.921292+0000","last_undegraded":"2023-10-06T05:01:22.921292+0000","last_fullsized":"2023-10-06T05:01:22.921292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:14.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912183+0000","last_change":"2023-10-06T05:01:22.910088+0000","last_active":"2023-10-06T05:01:22.912183+0000","last_peered":"2023-10-06T05:01:22.912183+0000","last_clean":"2023-10-06T05:01:22.912183+0000","last_became_active":"2023-10-06T05:01:22.909830+0000","last_became_peered":"2023-10-06T05:01:22.909830+0000","last_unstale":"2023-10-06T05:01:22.912183+0000","last_undegraded":"2023-10-06T05:01:22.912183+0000","last_fullsized":"2023-10-06T05:01:22.912183+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:14.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920682+0000","last_change":"2023-10-06T05:01:22.915966+0000","last_active":"2023-10-06T05:01:22.920682+0000","last_peered":"2023-10-06T05:01:22.920682+0000","last_clean":"2023-10-06T05:01:22.920682+0000","last_became_active":"2023-10-06T05:01:22.915477+0000","last_became_peered":"2023-10-06T05:01:22.915477+0000","last_unstale":"2023-10-06T05:01:22.920682+0000","last_undegraded":"2023-10-06T05:01:22.920682+0000","last_fullsized":"2023-10-06T05:01:22.920682+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:24.762820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908898+0000","last_change":"2023-10-06T05:01:22.905656+0000","last_active":"2023-10-06T05:01:22.908898+0000","last_peered":"2023-10-06T05:01:22.908898+0000","last_clean":"2023-10-06T05:01:22.908898+0000","last_became_active":"2023-10-06T05:01:22.905402+0000","last_became_peered":"2023-10-06T05:01:22.905402+0000","last_unstale":"2023-10-06T05:01:22.908898+0000","last_undegraded":"2023-10-06T05:01:22.908898+0000","last_fullsized":"2023-10-06T05:01:22.908898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.573131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912788+0000","last_change":"2023-10-06T05:01:22.908184+0000","last_active":"2023-10-06T05:01:22.912788+0000","last_peered":"2023-10-06T05:01:22.912788+0000","last_clean":"2023-10-06T05:01:22.912788+0000","last_became_active":"2023-10-06T05:01:22.904908+0000","last_became_peered":"2023-10-06T05:01:22.904908+0000","last_unstale":"2023-10-06T05:01:22.912788+0000","last_undegraded":"2023-10-06T05:01:22.912788+0000","last_fullsized":"2023-10-06T05:01:22.912788+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:42.892786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912280+0000","last_change":"2023-10-06T05:01:22.908024+0000","last_active":"2023-10-06T05:01:22.912280+0000","last_peered":"2023-10-06T05:01:22.912280+0000","last_clean":"2023-10-06T05:01:22.912280+0000","last_became_active":"2023-10-06T05:01:22.907701+0000","last_became_peered":"2023-10-06T05:01:22.907701+0000","last_unstale":"2023-10-06T05:01:22.912280+0000","last_undegraded":"2023-10-06T05:01:22.912280+0000","last_fullsized":"2023-10-06T05:01:22.912280+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912827+0000","last_change":"2023-10-06T05:01:22.911724+0000","last_active":"2023-10-06T05:01:22.912827+0000","last_peered":"2023-10-06T05:01:22.912827+0000","last_clean":"2023-10-06T05:01:22.912827+0000","last_became_active":"2023-10-06T05:01:22.911247+0000","last_became_peered":"2023-10-06T05:01:22.911247+0000","last_unstale":"2023-10-06T05:01:22.912827+0000","last_undegraded":"2023-10-06T05:01:22.912827+0000","last_fullsized":"2023-10-06T05:01:22.912827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910490+0000","last_change":"2023-10-06T05:01:22.902838+0000","last_active":"2023-10-06T05:01:22.910490+0000","last_peered":"2023-10-06T05:01:22.910490+0000","last_clean":"2023-10-06T05:01:22.910490+0000","last_became_active":"2023-10-06T05:01:22.902549+0000","last_became_peered":"2023-10-06T05:01:22.902549+0000","last_unstale":"2023-10-06T05:01:22.910490+0000","last_undegraded":"2023-10-06T05:01:22.910490+0000","last_fullsized":"2023-10-06T05:01:22.910490+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:26.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912308+0000","last_change":"2023-10-06T05:01:22.908586+0000","last_active":"2023-10-06T05:01:22.912308+0000","last_peered":"2023-10-06T05:01:22.912308+0000","last_clean":"2023-10-06T05:01:22.912308+0000","last_became_active":"2023-10-06T05:01:22.908289+0000","last_became_peered":"2023-10-06T05:01:22.908289+0000","last_unstale":"2023-10-06T05:01:22.912308+0000","last_undegraded":"2023-10-06T05:01:22.912308+0000","last_fullsized":"2023-10-06T05:01:22.912308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:38.888355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914573+0000","last_change":"2023-10-06T05:01:22.910843+0000","last_active":"2023-10-06T05:01:22.914573+0000","last_peered":"2023-10-06T05:01:22.914573+0000","last_clean":"2023-10-06T05:01:22.914573+0000","last_became_active":"2023-10-06T05:01:22.910470+0000","last_became_peered":"2023-10-06T05:01:22.910470+0000","last_unstale":"2023-10-06T05:01:22.914573+0000","last_undegraded":"2023-10-06T05:01:22.914573+0000","last_fullsized":"2023-10-06T05:01:22.914573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912812+0000","last_change":"2023-10-06T05:01:22.908293+0000","last_active":"2023-10-06T05:01:22.912812+0000","last_peered":"2023-10-06T05:01:22.912812+0000","last_clean":"2023-10-06T05:01:22.912812+0000","last_became_active":"2023-10-06T05:01:22.907986+0000","last_became_peered":"2023-10-06T05:01:22.907986+0000","last_unstale":"2023-10-06T05:01:22.912812+0000","last_undegraded":"2023-10-06T05:01:22.912812+0000","last_fullsized":"2023-10-06T05:01:22.912812+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:58.242153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910601+0000","last_change":"2023-10-06T05:01:22.905470+0000","last_active":"2023-10-06T05:01:22.910601+0000","last_peered":"2023-10-06T05:01:22.910601+0000","last_clean":"2023-10-06T05:01:22.910601+0000","last_became_active":"2023-10-06T05:01:22.905162+0000","last_became_peered":"2023-10-06T05:01:22.905162+0000","last_unstale":"2023-10-06T05:01:22.910601+0000","last_undegraded":"2023-10-06T05:01:22.910601+0000","last_fullsized":"2023-10-06T05:01:22.910601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:53.259645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912975+0000","last_change":"2023-10-06T05:01:22.911783+0000","last_active":"2023-10-06T05:01:22.912975+0000","last_peered":"2023-10-06T05:01:22.912975+0000","last_clean":"2023-10-06T05:01:22.912975+0000","last_became_active":"2023-10-06T05:01:22.911488+0000","last_became_peered":"2023-10-06T05:01:22.911488+0000","last_unstale":"2023-10-06T05:01:22.912975+0000","last_undegraded":"2023-10-06T05:01:22.912975+0000","last_fullsized":"2023-10-06T05:01:22.912975+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912590+0000","last_change":"2023-10-06T05:01:22.911945+0000","last_active":"2023-10-06T05:01:22.912590+0000","last_peered":"2023-10-06T05:01:22.912590+0000","last_clean":"2023-10-06T05:01:22.912590+0000","last_became_active":"2023-10-06T05:01:22.911675+0000","last_became_peered":"2023-10-06T05:01:22.911675+0000","last_unstale":"2023-10-06T05:01:22.912590+0000","last_undegraded":"2023-10-06T05:01:22.912590+0000","last_fullsized":"2023-10-06T05:01:22.912590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:32.223585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914827+0000","last_change":"2023-10-06T05:01:22.913460+0000","last_active":"2023-10-06T05:01:22.914827+0000","last_peered":"2023-10-06T05:01:22.914827+0000","last_clean":"2023-10-06T05:01:22.914827+0000","last_became_active":"2023-10-06T05:01:22.913178+0000","last_became_peered":"2023-10-06T05:01:22.913178+0000","last_unstale":"2023-10-06T05:01:22.914827+0000","last_undegraded":"2023-10-06T05:01:22.914827+0000","last_fullsized":"2023-10-06T05:01:22.914827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:16.124524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.923028+0000","last_change":"2023-10-06T05:01:22.918920+0000","last_active":"2023-10-06T05:01:22.923028+0000","last_peered":"2023-10-06T05:01:22.923028+0000","last_clean":"2023-10-06T05:01:22.923028+0000","last_became_active":"2023-10-06T05:01:22.918545+0000","last_became_peered":"2023-10-06T05:01:22.918545+0000","last_unstale":"2023-10-06T05:01:22.923028+0000","last_undegraded":"2023-10-06T05:01:22.923028+0000","last_fullsized":"2023-10-06T05:01:22.923028+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909358+0000","last_change":"2023-10-06T05:01:22.906868+0000","last_active":"2023-10-06T05:01:22.909358+0000","last_peered":"2023-10-06T05:01:22.909358+0000","last_clean":"2023-10-06T05:01:22.909358+0000","last_became_active":"2023-10-06T05:01:22.906368+0000","last_became_peered":"2023-10-06T05:01:22.906368+0000","last_unstale":"2023-10-06T05:01:22.909358+0000","last_undegraded":"2023-10-06T05:01:22.909358+0000","last_fullsized":"2023-10-06T05:01:22.909358+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:45.514209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910892+0000","last_change":"2023-10-06T05:01:21.905167+0000","last_active":"2023-10-06T05:01:22.910892+0000","last_peered":"2023-10-06T05:01:22.910892+0000","last_clean":"2023-10-06T05:01:22.910892+0000","last_became_active":"2023-10-06T05:01:21.904856+0000","last_became_peered":"2023-10-06T05:01:21.904856+0000","last_unstale":"2023-10-06T05:01:22.910892+0000","last_undegraded":"2023-10-06T05:01:22.910892+0000","last_fullsized":"2023-10-06T05:01:22.910892+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916718+0000","last_change":"2023-10-06T05:01:22.915682+0000","last_active":"2023-10-06T05:01:22.916718+0000","last_peered":"2023-10-06T05:01:22.916718+0000","last_clean":"2023-10-06T05:01:22.916718+0000","last_became_active":"2023-10-06T05:01:22.915425+0000","last_became_peered":"2023-10-06T05:01:22.915425+0000","last_unstale":"2023-10-06T05:01:22.916718+0000","last_undegraded":"2023-10-06T05:01:22.916718+0000","last_fullsized":"2023-10-06T05:01:22.916718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913713+0000","last_change":"2023-10-06T05:01:21.909299+0000","last_active":"2023-10-06T05:01:22.913713+0000","last_peered":"2023-10-06T05:01:22.913713+0000","last_clean":"2023-10-06T05:01:22.913713+0000","last_became_active":"2023-10-06T05:01:21.908989+0000","last_became_peered":"2023-10-06T05:01:21.908989+0000","last_unstale":"2023-10-06T05:01:22.913713+0000","last_undegraded":"2023-10-06T05:01:22.913713+0000","last_fullsized":"2023-10-06T05:01:22.913713+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921765+0000","last_change":"2023-10-06T05:01:22.918183+0000","last_active":"2023-10-06T05:01:22.921765+0000","last_peered":"2023-10-06T05:01:22.921765+0000","last_clean":"2023-10-06T05:01:22.921765+0000","last_became_active":"2023-10-06T05:01:22.917637+0000","last_became_peered":"2023-10-06T05:01:22.917637+0000","last_unstale":"2023-10-06T05:01:22.921765+0000","last_undegraded":"2023-10-06T05:01:22.921765+0000","last_fullsized":"2023-10-06T05:01:22.921765+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914828+0000","last_change":"2023-10-06T05:01:21.909301+0000","last_active":"2023-10-06T05:01:22.914828+0000","last_peered":"2023-10-06T05:01:22.914828+0000","last_clean":"2023-10-06T05:01:22.914828+0000","last_became_active":"2023-10-06T05:01:21.908984+0000","last_became_peered":"2023-10-06T05:01:21.908984+0000","last_unstale":"2023-10-06T05:01:22.914828+0000","last_undegraded":"2023-10-06T05:01:22.914828+0000","last_fullsized":"2023-10-06T05:01:22.914828+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:26.376744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920398+0000","last_change":"2023-10-06T05:01:22.910823+0000","last_active":"2023-10-06T05:01:22.920398+0000","last_peered":"2023-10-06T05:01:22.920398+0000","last_clean":"2023-10-06T05:01:22.920398+0000","last_became_active":"2023-10-06T05:01:22.910158+0000","last_became_peered":"2023-10-06T05:01:22.910158+0000","last_unstale":"2023-10-06T05:01:22.920398+0000","last_undegraded":"2023-10-06T05:01:22.920398+0000","last_fullsized":"2023-10-06T05:01:22.920398+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:33.254913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917199+0000","last_change":"2023-10-06T05:01:21.898248+0000","last_active":"2023-10-06T05:01:22.917199+0000","last_peered":"2023-10-06T05:01:22.917199+0000","last_clean":"2023-10-06T05:01:22.917199+0000","last_became_active":"2023-10-06T05:01:21.897180+0000","last_became_peered":"2023-10-06T05:01:21.897180+0000","last_unstale":"2023-10-06T05:01:22.917199+0000","last_undegraded":"2023-10-06T05:01:22.917199+0000","last_fullsized":"2023-10-06T05:01:22.917199+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:59.743528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909059+0000","last_change":"2023-10-06T05:01:22.905942+0000","last_active":"2023-10-06T05:01:22.909059+0000","last_peered":"2023-10-06T05:01:22.909059+0000","last_clean":"2023-10-06T05:01:22.909059+0000","last_became_active":"2023-10-06T05:01:22.905653+0000","last_became_peered":"2023-10-06T05:01:22.905653+0000","last_unstale":"2023-10-06T05:01:22.909059+0000","last_undegraded":"2023-10-06T05:01:22.909059+0000","last_fullsized":"2023-10-06T05:01:22.909059+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:12.300608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921227+0000","last_change":"2023-10-06T05:01:21.902635+0000","last_active":"2023-10-06T05:01:22.921227+0000","last_peered":"2023-10-06T05:01:22.921227+0000","last_clean":"2023-10-06T05:01:22.921227+0000","last_became_active":"2023-10-06T05:01:21.901883+0000","last_became_peered":"2023-10-06T05:01:21.901883+0000","last_unstale":"2023-10-06T05:01:22.921227+0000","last_undegraded":"2023-10-06T05:01:22.921227+0000","last_fullsized":"2023-10-06T05:01:22.921227+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920542+0000","last_change":"2023-10-06T05:01:22.917447+0000","last_active":"2023-10-06T05:01:22.920542+0000","last_peered":"2023-10-06T05:01:22.920542+0000","last_clean":"2023-10-06T05:01:22.920542+0000","last_became_active":"2023-10-06T05:01:22.917130+0000","last_became_peered":"2023-10-06T05:01:22.917130+0000","last_unstale":"2023-10-06T05:01:22.920542+0000","last_undegraded":"2023-10-06T05:01:22.920542+0000","last_fullsized":"2023-10-06T05:01:22.920542+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:20.172157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916407+0000","last_change":"2023-10-06T05:01:21.906758+0000","last_active":"2023-10-06T05:01:22.916407+0000","last_peered":"2023-10-06T05:01:22.916407+0000","last_clean":"2023-10-06T05:01:22.916407+0000","last_became_active":"2023-10-06T05:01:21.906463+0000","last_became_peered":"2023-10-06T05:01:21.906463+0000","last_unstale":"2023-10-06T05:01:22.916407+0000","last_undegraded":"2023-10-06T05:01:22.916407+0000","last_fullsized":"2023-10-06T05:01:22.916407+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:42.205607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908789+0000","last_change":"2023-10-06T05:01:22.905020+0000","last_active":"2023-10-06T05:01:22.908789+0000","last_peered":"2023-10-06T05:01:22.908789+0000","last_clean":"2023-10-06T05:01:22.908789+0000","last_became_active":"2023-10-06T05:01:22.904762+0000","last_became_peered":"2023-10-06T05:01:22.904762+0000","last_unstale":"2023-10-06T05:01:22.908789+0000","last_undegraded":"2023-10-06T05:01:22.908789+0000","last_fullsized":"2023-10-06T05:01:22.908789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900865+0000","last_change":"2023-10-06T05:01:21.907467+0000","last_active":"2023-10-06T05:01:22.900865+0000","last_peered":"2023-10-06T05:01:22.900865+0000","last_clean":"2023-10-06T05:01:22.900865+0000","last_became_active":"2023-10-06T05:01:21.907169+0000","last_became_peered":"2023-10-06T05:01:21.907169+0000","last_unstale":"2023-10-06T05:01:22.900865+0000","last_undegraded":"2023-10-06T05:01:22.900865+0000","last_fullsized":"2023-10-06T05:01:22.900865+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.309675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918810+0000","last_change":"2023-10-06T05:01:22.916673+0000","last_active":"2023-10-06T05:01:22.918810+0000","last_peered":"2023-10-06T05:01:22.918810+0000","last_clean":"2023-10-06T05:01:22.918810+0000","last_became_active":"2023-10-06T05:01:22.916393+0000","last_became_peered":"2023-10-06T05:01:22.916393+0000","last_unstale":"2023-10-06T05:01:22.918810+0000","last_undegraded":"2023-10-06T05:01:22.918810+0000","last_fullsized":"2023-10-06T05:01:22.918810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:00.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.890628+0000","last_change":"2023-10-06T05:01:21.904853+0000","last_active":"2023-10-06T05:01:22.890628+0000","last_peered":"2023-10-06T05:01:22.890628+0000","last_clean":"2023-10-06T05:01:22.890628+0000","last_became_active":"2023-10-06T05:01:21.904448+0000","last_became_peered":"2023-10-06T05:01:21.904448+0000","last_unstale":"2023-10-06T05:01:22.890628+0000","last_undegraded":"2023-10-06T05:01:22.890628+0000","last_fullsized":"2023-10-06T05:01:22.890628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.567032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915557+0000","last_change":"2023-10-06T05:01:22.909027+0000","last_active":"2023-10-06T05:01:22.915557+0000","last_peered":"2023-10-06T05:01:22.915557+0000","last_clean":"2023-10-06T05:01:22.915557+0000","last_became_active":"2023-10-06T05:01:22.908783+0000","last_became_peered":"2023-10-06T05:01:22.908783+0000","last_unstale":"2023-10-06T05:01:22.915557+0000","last_undegraded":"2023-10-06T05:01:22.915557+0000","last_fullsized":"2023-10-06T05:01:22.915557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:48.927255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914458+0000","last_change":"2023-10-06T05:01:21.901342+0000","last_active":"2023-10-06T05:01:22.914458+0000","last_peered":"2023-10-06T05:01:22.914458+0000","last_clean":"2023-10-06T05:01:22.914458+0000","last_became_active":"2023-10-06T05:01:21.901020+0000","last_became_peered":"2023-10-06T05:01:21.901020+0000","last_unstale":"2023-10-06T05:01:22.914458+0000","last_undegraded":"2023-10-06T05:01:22.914458+0000","last_fullsized":"2023-10-06T05:01:22.914458+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:26.399481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920662+0000","last_change":"2023-10-06T05:01:22.915799+0000","last_active":"2023-10-06T05:01:22.920662+0000","last_peered":"2023-10-06T05:01:22.920662+0000","last_clean":"2023-10-06T05:01:22.920662+0000","last_became_active":"2023-10-06T05:01:22.915224+0000","last_became_peered":"2023-10-06T05:01:22.915224+0000","last_unstale":"2023-10-06T05:01:22.920662+0000","last_undegraded":"2023-10-06T05:01:22.920662+0000","last_fullsized":"2023-10-06T05:01:22.920662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:24.443047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909917+0000","last_change":"2023-10-06T05:01:21.903666+0000","last_active":"2023-10-06T05:01:22.909917+0000","last_peered":"2023-10-06T05:01:22.909917+0000","last_clean":"2023-10-06T05:01:22.909917+0000","last_became_active":"2023-10-06T05:01:21.903416+0000","last_became_peered":"2023-10-06T05:01:21.903416+0000","last_unstale":"2023-10-06T05:01:22.909917+0000","last_undegraded":"2023-10-06T05:01:22.909917+0000","last_fullsized":"2023-10-06T05:01:22.909917+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:38.753801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913218+0000","last_change":"2023-10-06T05:01:22.908994+0000","last_active":"2023-10-06T05:01:22.913218+0000","last_peered":"2023-10-06T05:01:22.913218+0000","last_clean":"2023-10-06T05:01:22.913218+0000","last_became_active":"2023-10-06T05:01:22.908627+0000","last_became_peered":"2023-10-06T05:01:22.908627+0000","last_unstale":"2023-10-06T05:01:22.913218+0000","last_undegraded":"2023-10-06T05:01:22.913218+0000","last_fullsized":"2023-10-06T05:01:22.913218+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920770+0000","last_change":"2023-10-06T05:01:21.901636+0000","last_active":"2023-10-06T05:01:22.920770+0000","last_peered":"2023-10-06T05:01:22.920770+0000","last_clean":"2023-10-06T05:01:22.920770+0000","last_became_active":"2023-10-06T05:01:21.901348+0000","last_became_peered":"2023-10-06T05:01:21.901348+0000","last_unstale":"2023-10-06T05:01:22.920770+0000","last_undegraded":"2023-10-06T05:01:22.920770+0000","last_fullsized":"2023-10-06T05:01:22.920770+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:07.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919815+0000","last_change":"2023-10-06T05:01:22.918705+0000","last_active":"2023-10-06T05:01:22.919815+0000","last_peered":"2023-10-06T05:01:22.919815+0000","last_clean":"2023-10-06T05:01:22.919815+0000","last_became_active":"2023-10-06T05:01:22.918437+0000","last_became_peered":"2023-10-06T05:01:22.918437+0000","last_unstale":"2023-10-06T05:01:22.919815+0000","last_undegraded":"2023-10-06T05:01:22.919815+0000","last_fullsized":"2023-10-06T05:01:22.919815+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900294+0000","last_change":"2023-10-06T05:01:21.903733+0000","last_active":"2023-10-06T05:01:22.900294+0000","last_peered":"2023-10-06T05:01:22.900294+0000","last_clean":"2023-10-06T05:01:22.900294+0000","last_became_active":"2023-10-06T05:01:21.903433+0000","last_became_peered":"2023-10-06T05:01:21.903433+0000","last_unstale":"2023-10-06T05:01:22.900294+0000","last_undegraded":"2023-10-06T05:01:22.900294+0000","last_fullsized":"2023-10-06T05:01:22.900294+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:39.894106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920786+0000","last_change":"2023-10-06T05:01:22.919693+0000","last_active":"2023-10-06T05:01:22.920786+0000","last_peered":"2023-10-06T05:01:22.920786+0000","last_clean":"2023-10-06T05:01:22.920786+0000","last_became_active":"2023-10-06T05:01:22.919445+0000","last_became_peered":"2023-10-06T05:01:22.919445+0000","last_unstale":"2023-10-06T05:01:22.920786+0000","last_undegraded":"2023-10-06T05:01:22.920786+0000","last_fullsized":"2023-10-06T05:01:22.920786+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:00.412186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916231+0000","last_change":"2023-10-06T05:01:21.901798+0000","last_active":"2023-10-06T05:01:22.916231+0000","last_peered":"2023-10-06T05:01:22.916231+0000","last_clean":"2023-10-06T05:01:22.916231+0000","last_became_active":"2023-10-06T05:01:21.901536+0000","last_became_peered":"2023-10-06T05:01:21.901536+0000","last_unstale":"2023-10-06T05:01:22.916231+0000","last_undegraded":"2023-10-06T05:01:22.916231+0000","last_fullsized":"2023-10-06T05:01:22.916231+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909492+0000","last_change":"2023-10-06T05:01:22.908931+0000","last_active":"2023-10-06T05:01:22.909492+0000","last_peered":"2023-10-06T05:01:22.909492+0000","last_clean":"2023-10-06T05:01:22.909492+0000","last_became_active":"2023-10-06T05:01:22.908678+0000","last_became_peered":"2023-10-06T05:01:22.908678+0000","last_unstale":"2023-10-06T05:01:22.909492+0000","last_undegraded":"2023-10-06T05:01:22.909492+0000","last_fullsized":"2023-10-06T05:01:22.909492+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:24.372776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.899848+0000","last_change":"2023-10-06T05:01:21.894870+0000","last_active":"2023-10-06T05:01:22.899848+0000","last_peered":"2023-10-06T05:01:22.899848+0000","last_clean":"2023-10-06T05:01:22.899848+0000","last_became_active":"2023-10-06T05:01:21.893907+0000","last_became_peered":"2023-10-06T05:01:21.893907+0000","last_unstale":"2023-10-06T05:01:22.899848+0000","last_undegraded":"2023-10-06T05:01:22.899848+0000","last_fullsized":"2023-10-06T05:01:22.899848+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920281+0000","last_change":"2023-10-06T05:01:22.910581+0000","last_active":"2023-10-06T05:01:22.920281+0000","last_peered":"2023-10-06T05:01:22.920281+0000","last_clean":"2023-10-06T05:01:22.920281+0000","last_became_active":"2023-10-06T05:01:22.909963+0000","last_became_peered":"2023-10-06T05:01:22.909963+0000","last_unstale":"2023-10-06T05:01:22.920281+0000","last_undegraded":"2023-10-06T05:01:22.920281+0000","last_fullsized":"2023-10-06T05:01:22.920281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:41.453873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.893893+0000","last_change":"2023-10-06T05:01:21.900893+0000","last_active":"2023-10-06T05:01:22.893893+0000","last_peered":"2023-10-06T05:01:22.893893+0000","last_clean":"2023-10-06T05:01:22.893893+0000","last_became_active":"2023-10-06T05:01:21.900591+0000","last_became_peered":"2023-10-06T05:01:21.900591+0000","last_unstale":"2023-10-06T05:01:22.893893+0000","last_undegraded":"2023-10-06T05:01:22.893893+0000","last_fullsized":"2023-10-06T05:01:22.893893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917896+0000","last_change":"2023-10-06T05:01:22.917013+0000","last_active":"2023-10-06T05:01:22.917896+0000","last_peered":"2023-10-06T05:01:22.917896+0000","last_clean":"2023-10-06T05:01:22.917896+0000","last_became_active":"2023-10-06T05:01:22.915500+0000","last_became_peered":"2023-10-06T05:01:22.915500+0000","last_unstale":"2023-10-06T05:01:22.917896+0000","last_undegraded":"2023-10-06T05:01:22.917896+0000","last_fullsized":"2023-10-06T05:01:22.917896+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914695+0000","last_change":"2023-10-06T05:01:21.897131+0000","last_active":"2023-10-06T05:01:22.914695+0000","last_peered":"2023-10-06T05:01:22.914695+0000","last_clean":"2023-10-06T05:01:22.914695+0000","last_became_active":"2023-10-06T05:01:21.896847+0000","last_became_peered":"2023-10-06T05:01:21.896847+0000","last_unstale":"2023-10-06T05:01:22.914695+0000","last_undegraded":"2023-10-06T05:01:22.914695+0000","last_fullsized":"2023-10-06T05:01:22.914695+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912743+0000","last_change":"2023-10-06T05:01:22.907709+0000","last_active":"2023-10-06T05:01:22.912743+0000","last_peered":"2023-10-06T05:01:22.912743+0000","last_clean":"2023-10-06T05:01:22.912743+0000","last_became_active":"2023-10-06T05:01:22.907411+0000","last_became_peered":"2023-10-06T05:01:22.907411+0000","last_unstale":"2023-10-06T05:01:22.912743+0000","last_undegraded":"2023-10-06T05:01:22.912743+0000","last_fullsized":"2023-10-06T05:01:22.912743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:19.960229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921946+0000","last_change":"2023-10-06T05:01:21.900714+0000","last_active":"2023-10-06T05:01:22.921946+0000","last_peered":"2023-10-06T05:01:22.921946+0000","last_clean":"2023-10-06T05:01:22.921946+0000","last_became_active":"2023-10-06T05:01:21.900456+0000","last_became_peered":"2023-10-06T05:01:21.900456+0000","last_unstale":"2023-10-06T05:01:22.921946+0000","last_undegraded":"2023-10-06T05:01:22.921946+0000","last_fullsized":"2023-10-06T05:01:22.921946+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:10.205802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916048+0000","last_change":"2023-10-06T05:01:22.913467+0000","last_active":"2023-10-06T05:01:22.916048+0000","last_peered":"2023-10-06T05:01:22.916048+0000","last_clean":"2023-10-06T05:01:22.916048+0000","last_became_active":"2023-10-06T05:01:22.913133+0000","last_became_peered":"2023-10-06T05:01:22.913133+0000","last_unstale":"2023-10-06T05:01:22.916048+0000","last_undegraded":"2023-10-06T05:01:22.916048+0000","last_fullsized":"2023-10-06T05:01:22.916048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:41.139543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.901055+0000","last_change":"2023-10-06T05:01:21.907168+0000","last_active":"2023-10-06T05:01:22.901055+0000","last_peered":"2023-10-06T05:01:22.901055+0000","last_clean":"2023-10-06T05:01:22.901055+0000","last_became_active":"2023-10-06T05:01:21.906798+0000","last_became_peered":"2023-10-06T05:01:21.906798+0000","last_unstale":"2023-10-06T05:01:22.901055+0000","last_undegraded":"2023-10-06T05:01:22.901055+0000","last_fullsized":"2023-10-06T05:01:22.901055+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919507+0000","last_change":"2023-10-06T05:01:22.917117+0000","last_active":"2023-10-06T05:01:22.919507+0000","last_peered":"2023-10-06T05:01:22.919507+0000","last_clean":"2023-10-06T05:01:22.919507+0000","last_became_active":"2023-10-06T05:01:22.916829+0000","last_became_peered":"2023-10-06T05:01:22.916829+0000","last_unstale":"2023-10-06T05:01:22.919507+0000","last_undegraded":"2023-10-06T05:01:22.919507+0000","last_fullsized":"2023-10-06T05:01:22.919507+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.896584+0000","last_change":"2023-10-06T05:01:21.897514+0000","last_active":"2023-10-06T05:01:22.896584+0000","last_peered":"2023-10-06T05:01:22.896584+0000","last_clean":"2023-10-06T05:01:22.896584+0000","last_became_active":"2023-10-06T05:01:21.897226+0000","last_became_peered":"2023-10-06T05:01:21.897226+0000","last_unstale":"2023-10-06T05:01:22.896584+0000","last_undegraded":"2023-10-06T05:01:22.896584+0000","last_fullsized":"2023-10-06T05:01:22.896584+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:44.413476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917870+0000","last_change":"2023-10-06T05:01:22.916955+0000","last_active":"2023-10-06T05:01:22.917870+0000","last_peered":"2023-10-06T05:01:22.917870+0000","last_clean":"2023-10-06T05:01:22.917870+0000","last_became_active":"2023-10-06T05:01:22.916640+0000","last_became_peered":"2023-10-06T05:01:22.916640+0000","last_unstale":"2023-10-06T05:01:22.917870+0000","last_undegraded":"2023-10-06T05:01:22.917870+0000","last_fullsized":"2023-10-06T05:01:22.917870+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:09.844106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891290+0000","last_change":"2023-10-06T05:01:21.906767+0000","last_active":"2023-10-06T05:01:22.891290+0000","last_peered":"2023-10-06T05:01:22.891290+0000","last_clean":"2023-10-06T05:01:22.891290+0000","last_became_active":"2023-10-06T05:01:21.906476+0000","last_became_peered":"2023-10-06T05:01:21.906476+0000","last_unstale":"2023-10-06T05:01:22.891290+0000","last_undegraded":"2023-10-06T05:01:22.891290+0000","last_fullsized":"2023-10-06T05:01:22.891290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921664+0000","last_change":"2023-10-06T05:01:22.917699+0000","last_active":"2023-10-06T05:01:22.921664+0000","last_peered":"2023-10-06T05:01:22.921664+0000","last_clean":"2023-10-06T05:01:22.921664+0000","last_became_active":"2023-10-06T05:01:22.916786+0000","last_became_peered":"2023-10-06T05:01:22.916786+0000","last_unstale":"2023-10-06T05:01:22.921664+0000","last_undegraded":"2023-10-06T05:01:22.921664+0000","last_fullsized":"2023-10-06T05:01:22.921664+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900665+0000","last_change":"2023-10-06T05:01:21.910269+0000","last_active":"2023-10-06T05:01:22.900665+0000","last_peered":"2023-10-06T05:01:22.900665+0000","last_clean":"2023-10-06T05:01:22.900665+0000","last_became_active":"2023-10-06T05:01:21.909971+0000","last_became_peered":"2023-10-06T05:01:21.909971+0000","last_unstale":"2023-10-06T05:01:22.900665+0000","last_undegraded":"2023-10-06T05:01:22.900665+0000","last_fullsized":"2023-10-06T05:01:22.900665+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920563+0000","last_change":"2023-10-06T05:01:22.919007+0000","last_active":"2023-10-06T05:01:22.920563+0000","last_peered":"2023-10-06T05:01:22.920563+0000","last_clean":"2023-10-06T05:01:22.920563+0000","last_became_active":"2023-10-06T05:01:22.911208+0000","last_became_peered":"2023-10-06T05:01:22.911208+0000","last_unstale":"2023-10-06T05:01:22.920563+0000","last_undegraded":"2023-10-06T05:01:22.920563+0000","last_fullsized":"2023-10-06T05:01:22.920563+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:08.817032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.894440+0000","last_change":"2023-10-06T05:01:21.902093+0000","last_active":"2023-10-06T05:01:22.894440+0000","last_peered":"2023-10-06T05:01:22.894440+0000","last_clean":"2023-10-06T05:01:22.894440+0000","last_became_active":"2023-10-06T05:01:21.900927+0000","last_became_peered":"2023-10-06T05:01:21.900927+0000","last_unstale":"2023-10-06T05:01:22.894440+0000","last_undegraded":"2023-10-06T05:01:22.894440+0000","last_fullsized":"2023-10-06T05:01:22.894440+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:16.639594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918752+0000","last_change":"2023-10-06T05:01:22.917989+0000","last_active":"2023-10-06T05:01:22.918752+0000","last_peered":"2023-10-06T05:01:22.918752+0000","last_clean":"2023-10-06T05:01:22.918752+0000","last_became_active":"2023-10-06T05:01:22.917665+0000","last_became_peered":"2023-10-06T05:01:22.917665+0000","last_unstale":"2023-10-06T05:01:22.918752+0000","last_undegraded":"2023-10-06T05:01:22.918752+0000","last_fullsized":"2023-10-06T05:01:22.918752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:10.275597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900723+0000","last_change":"2023-10-06T05:01:21.908132+0000","last_active":"2023-10-06T05:01:22.900723+0000","last_peered":"2023-10-06T05:01:22.900723+0000","last_clean":"2023-10-06T05:01:22.900723+0000","last_became_active":"2023-10-06T05:01:21.907727+0000","last_became_peered":"2023-10-06T05:01:21.907727+0000","last_unstale":"2023-10-06T05:01:22.900723+0000","last_undegraded":"2023-10-06T05:01:22.900723+0000","last_fullsized":"2023-10-06T05:01:22.900723+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:19.166059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912985+0000","last_change":"2023-10-06T05:01:22.911853+0000","last_active":"2023-10-06T05:01:22.912985+0000","last_peered":"2023-10-06T05:01:22.912985+0000","last_clean":"2023-10-06T05:01:22.912985+0000","last_became_active":"2023-10-06T05:01:22.911562+0000","last_became_peered":"2023-10-06T05:01:22.911562+0000","last_unstale":"2023-10-06T05:01:22.912985+0000","last_undegraded":"2023-10-06T05:01:22.912985+0000","last_fullsized":"2023-10-06T05:01:22.912985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898029+0000","last_change":"2023-10-06T05:01:21.908600+0000","last_active":"2023-10-06T05:01:22.898029+0000","last_peered":"2023-10-06T05:01:22.898029+0000","last_clean":"2023-10-06T05:01:22.898029+0000","last_became_active":"2023-10-06T05:01:21.908261+0000","last_became_peered":"2023-10-06T05:01:21.908261+0000","last_unstale":"2023-10-06T05:01:22.898029+0000","last_undegraded":"2023-10-06T05:01:22.898029+0000","last_fullsized":"2023-10-06T05:01:22.898029+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:23.366677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910946+0000","last_change":"2023-10-06T05:01:22.906979+0000","last_active":"2023-10-06T05:01:22.910946+0000","last_peered":"2023-10-06T05:01:22.910946+0000","last_clean":"2023-10-06T05:01:22.910946+0000","last_became_active":"2023-10-06T05:01:22.906652+0000","last_became_peered":"2023-10-06T05:01:22.906652+0000","last_unstale":"2023-10-06T05:01:22.910946+0000","last_undegraded":"2023-10-06T05:01:22.910946+0000","last_fullsized":"2023-10-06T05:01:22.910946+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:42.125134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.899072+0000","last_change":"2023-10-06T05:01:21.897963+0000","last_active":"2023-10-06T05:01:22.899072+0000","last_peered":"2023-10-06T05:01:22.899072+0000","last_clean":"2023-10-06T05:01:22.899072+0000","last_became_active":"2023-10-06T05:01:21.897684+0000","last_became_peered":"2023-10-06T05:01:21.897684+0000","last_unstale":"2023-10-06T05:01:22.899072+0000","last_undegraded":"2023-10-06T05:01:22.899072+0000","last_fullsized":"2023-10-06T05:01:22.899072+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:11.842466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.904603+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:01:22.904603+0000","last_peered":"2023-10-06T05:01:22.904603+0000","last_clean":"2023-10-06T05:01:22.904603+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:01:22.904603+0000","last_undegraded":"2023-10-06T05:01:22.904603+0000","last_fullsized":"2023-10-06T05:01:22.904603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912616+0000","last_change":"2023-10-06T05:01:22.911622+0000","last_active":"2023-10-06T05:01:22.912616+0000","last_peered":"2023-10-06T05:01:22.912616+0000","last_clean":"2023-10-06T05:01:22.912616+0000","last_became_active":"2023-10-06T05:01:22.911388+0000","last_became_peered":"2023-10-06T05:01:22.911388+0000","last_unstale":"2023-10-06T05:01:22.912616+0000","last_undegraded":"2023-10-06T05:01:22.912616+0000","last_fullsized":"2023-10-06T05:01:22.912616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.004036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920959+0000","last_change":"2023-10-06T05:01:21.907272+0000","last_active":"2023-10-06T05:01:22.920959+0000","last_peered":"2023-10-06T05:01:22.920959+0000","last_clean":"2023-10-06T05:01:22.920959+0000","last_became_active":"2023-10-06T05:01:21.906971+0000","last_became_peered":"2023-10-06T05:01:21.906971+0000","last_unstale":"2023-10-06T05:01:22.920959+0000","last_undegraded":"2023-10-06T05:01:22.920959+0000","last_fullsized":"2023-10-06T05:01:22.920959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891246+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:01:22.891246+0000","last_peered":"2023-10-06T05:01:22.891246+0000","last_clean":"2023-10-06T05:01:22.891246+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:01:22.891246+0000","last_undegraded":"2023-10-06T05:01:22.891246+0000","last_fullsized":"2023-10-06T05:01:22.891246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912780+0000","last_change":"2023-10-06T05:01:22.911349+0000","last_active":"2023-10-06T05:01:22.912780+0000","last_peered":"2023-10-06T05:01:22.912780+0000","last_clean":"2023-10-06T05:01:22.912780+0000","last_became_active":"2023-10-06T05:01:22.906441+0000","last_became_peered":"2023-10-06T05:01:22.906441+0000","last_unstale":"2023-10-06T05:01:22.912780+0000","last_undegraded":"2023-10-06T05:01:22.912780+0000","last_fullsized":"2023-10-06T05:01:22.912780+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.902904+0000","last_change":"2023-10-06T05:01:21.905784+0000","last_active":"2023-10-06T05:01:22.902904+0000","last_peered":"2023-10-06T05:01:22.902904+0000","last_clean":"2023-10-06T05:01:22.902904+0000","last_became_active":"2023-10-06T05:01:21.905522+0000","last_became_peered":"2023-10-06T05:01:21.905522+0000","last_unstale":"2023-10-06T05:01:22.902904+0000","last_undegraded":"2023-10-06T05:01:22.902904+0000","last_fullsized":"2023-10-06T05:01:22.902904+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:31.320712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":107,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897470+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:01:22.897470+0000","last_peered":"2023-10-06T05:01:22.897470+0000","last_clean":"2023-10-06T05:01:22.897470+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:01:22.897470+0000","last_undegraded":"2023-10-06T05:01:22.897470+0000","last_fullsized":"2023-10-06T05:01:22.897470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":254,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898054+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:01:22.898054+0000","last_peered":"2023-10-06T05:01:22.898054+0000","last_clean":"2023-10-06T05:01:22.898054+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:01:22.898054+0000","last_undegraded":"2023-10-06T05:01:22.898054+0000","last_fullsized":"2023-10-06T05:01:22.898054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915904+0000","last_change":"2023-10-06T05:01:22.914069+0000","last_active":"2023-10-06T05:01:22.915904+0000","last_peered":"2023-10-06T05:01:22.915904+0000","last_clean":"2023-10-06T05:01:22.915904+0000","last_became_active":"2023-10-06T05:01:22.913789+0000","last_became_peered":"2023-10-06T05:01:22.913789+0000","last_unstale":"2023-10-06T05:01:22.915904+0000","last_undegraded":"2023-10-06T05:01:22.915904+0000","last_fullsized":"2023-10-06T05:01:22.915904+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:37.179875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900942+0000","last_change":"2023-10-06T05:01:21.907735+0000","last_active":"2023-10-06T05:01:22.900942+0000","last_peered":"2023-10-06T05:01:22.900942+0000","last_clean":"2023-10-06T05:01:22.900942+0000","last_became_active":"2023-10-06T05:01:21.907455+0000","last_became_peered":"2023-10-06T05:01:21.907455+0000","last_unstale":"2023-10-06T05:01:22.900942+0000","last_undegraded":"2023-10-06T05:01:22.900942+0000","last_fullsized":"2023-10-06T05:01:22.900942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:50.355734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897481+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:01:22.897481+0000","last_peered":"2023-10-06T05:01:22.897481+0000","last_clean":"2023-10-06T05:01:22.897481+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:01:22.897481+0000","last_undegraded":"2023-10-06T05:01:22.897481+0000","last_fullsized":"2023-10-06T05:01:22.897481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912546+0000","last_change":"2023-10-06T05:01:22.904787+0000","last_active":"2023-10-06T05:01:22.912546+0000","last_peered":"2023-10-06T05:01:22.912546+0000","last_clean":"2023-10-06T05:01:22.912546+0000","last_became_active":"2023-10-06T05:01:22.904208+0000","last_became_peered":"2023-10-06T05:01:22.904208+0000","last_unstale":"2023-10-06T05:01:22.912546+0000","last_undegraded":"2023-10-06T05:01:22.912546+0000","last_fullsized":"2023-10-06T05:01:22.912546+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.925929+0000","last_change":"2023-10-06T05:01:21.900909+0000","last_active":"2023-10-06T05:01:22.925929+0000","last_peered":"2023-10-06T05:01:22.925929+0000","last_clean":"2023-10-06T05:01:22.925929+0000","last_became_active":"2023-10-06T05:01:21.900571+0000","last_became_peered":"2023-10-06T05:01:21.900571+0000","last_unstale":"2023-10-06T05:01:22.925929+0000","last_undegraded":"2023-10-06T05:01:22.925929+0000","last_fullsized":"2023-10-06T05:01:22.925929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898113+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:01:22.898113+0000","last_peered":"2023-10-06T05:01:22.898113+0000","last_clean":"2023-10-06T05:01:22.898113+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:01:22.898113+0000","last_undegraded":"2023-10-06T05:01:22.898113+0000","last_fullsized":"2023-10-06T05:01:22.898113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912543+0000","last_change":"2023-10-06T05:01:22.910384+0000","last_active":"2023-10-06T05:01:22.912543+0000","last_peered":"2023-10-06T05:01:22.912543+0000","last_clean":"2023-10-06T05:01:22.912543+0000","last_became_active":"2023-10-06T05:01:22.909894+0000","last_became_peered":"2023-10-06T05:01:22.909894+0000","last_unstale":"2023-10-06T05:01:22.912543+0000","last_undegraded":"2023-10-06T05:01:22.912543+0000","last_fullsized":"2023-10-06T05:01:22.912543+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:03.895804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909184+0000","last_change":"2023-10-06T05:01:21.898111+0000","last_active":"2023-10-06T05:01:22.909184+0000","last_peered":"2023-10-06T05:01:22.909184+0000","last_clean":"2023-10-06T05:01:22.909184+0000","last_became_active":"2023-10-06T05:01:21.897816+0000","last_became_peered":"2023-10-06T05:01:21.897816+0000","last_unstale":"2023-10-06T05:01:22.909184+0000","last_undegraded":"2023-10-06T05:01:22.909184+0000","last_fullsized":"2023-10-06T05:01:22.909184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:05.782643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900952+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:01:22.900952+0000","last_peered":"2023-10-06T05:01:22.900952+0000","last_clean":"2023-10-06T05:01:22.900952+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:01:22.900952+0000","last_undegraded":"2023-10-06T05:01:22.900952+0000","last_fullsized":"2023-10-06T05:01:22.900952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921053+0000","last_change":"2023-10-06T05:01:22.920569+0000","last_active":"2023-10-06T05:01:22.921053+0000","last_peered":"2023-10-06T05:01:22.921053+0000","last_clean":"2023-10-06T05:01:22.921053+0000","last_became_active":"2023-10-06T05:01:22.920240+0000","last_became_peered":"2023-10-06T05:01:22.920240+0000","last_unstale":"2023-10-06T05:01:22.921053+0000","last_undegraded":"2023-10-06T05:01:22.921053+0000","last_fullsized":"2023-10-06T05:01:22.921053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.906141+0000","last_change":"2023-10-06T05:01:21.901057+0000","last_active":"2023-10-06T05:01:22.906141+0000","last_peered":"2023-10-06T05:01:22.906141+0000","last_clean":"2023-10-06T05:01:22.906141+0000","last_became_active":"2023-10-06T05:01:21.900783+0000","last_became_peered":"2023-10-06T05:01:21.900783+0000","last_unstale":"2023-10-06T05:01:22.906141+0000","last_undegraded":"2023-10-06T05:01:22.906141+0000","last_fullsized":"2023-10-06T05:01:22.906141+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:13.427002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900563+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:01:22.900563+0000","last_peered":"2023-10-06T05:01:22.900563+0000","last_clean":"2023-10-06T05:01:22.900563+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:01:22.900563+0000","last_undegraded":"2023-10-06T05:01:22.900563+0000","last_fullsized":"2023-10-06T05:01:22.900563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918799+0000","last_change":"2023-10-06T05:01:22.915800+0000","last_active":"2023-10-06T05:01:22.918799+0000","last_peered":"2023-10-06T05:01:22.918799+0000","last_clean":"2023-10-06T05:01:22.918799+0000","last_became_active":"2023-10-06T05:01:22.915546+0000","last_became_peered":"2023-10-06T05:01:22.915546+0000","last_unstale":"2023-10-06T05:01:22.918799+0000","last_undegraded":"2023-10-06T05:01:22.918799+0000","last_fullsized":"2023-10-06T05:01:22.918799+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909478+0000","last_change":"2023-10-06T05:01:21.905554+0000","last_active":"2023-10-06T05:01:22.909478+0000","last_peered":"2023-10-06T05:01:22.909478+0000","last_clean":"2023-10-06T05:01:22.909478+0000","last_became_active":"2023-10-06T05:01:21.905222+0000","last_became_peered":"2023-10-06T05:01:21.905222+0000","last_unstale":"2023-10-06T05:01:22.909478+0000","last_undegraded":"2023-10-06T05:01:22.909478+0000","last_fullsized":"2023-10-06T05:01:22.909478+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910017+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:01:22.910017+0000","last_peered":"2023-10-06T05:01:22.910017+0000","last_clean":"2023-10-06T05:01:22.910017+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:01:22.910017+0000","last_undegraded":"2023-10-06T05:01:22.910017+0000","last_fullsized":"2023-10-06T05:01:22.910017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.73899999999999999,"5min":0.86599999999999999,"15min":7.1779999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.73499999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.311},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":7.1879999999999997},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.073},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":1.1379999999999999},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":0.82899999999999996,"15min":1.1240000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.111},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.0880000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.27300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.874},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.214},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1658880,"data_stored":753838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58799999999999997,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.33800000000000002,"15min":0.20699999999999999},"max":{"1min":0.73399999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63400000000000001,"15min":0.57499999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.65500000000000003,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.312},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.77900000000000003,"5min":1.234,"15min":1.234},"last":0.37}]},{"osd":2,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.63400000000000001,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70099999999999996,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.35199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.755,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61199999999999999,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":4.3949999999999996,"15min":4.3949999999999996},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.68100000000000005,"5min":3.669,"15min":3.669},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.79300000000000004,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72099999999999997,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35599999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.437,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":0.375},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.378,"15min":0.378},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.60499999999999998,"5min":0.68600000000000005,"15min":0.69399999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.622,"15min":0.70399999999999996},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.56000000000000005,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.47199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.77300000000000002,"5min":0.877,"15min":1.0780000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.998,"15min":0.998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.54400000000000004,"5min":0.496,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":1.022,"15min":1.0429999999999999},"last":0.83399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":755982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.59199999999999997,"5min":0.628,"15min":0.628},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42999999999999999,"15min":0.38800000000000001},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46100000000000002,"15min":0.443},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.98299999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.85799999999999998,"15min":0.92300000000000004},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.91700000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.93400000000000005,"15min":1.0109999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.52200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.97599999999999998,"15min":1.0329999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":0.995},"last":0.71499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":753880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.20300000000000001,"15min":0.186},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.048},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.39300000000000002},"min":{"1min":0.32300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.216},"max":{"1min":0.52400000000000002,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.219},"max":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.36499999999999999},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.90200000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.2649999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.88100000000000001,"15min":0.93100000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":0.88300000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.82499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307104,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064736,"statfs":{"total":96636764160,"available":96322289664,"internally_reserved":0,"allocated":1662976,"data_stored":754356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56399999999999995,"5min":0.68200000000000005,"15min":2.008},"last":0.22},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.53000000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":6.4370000000000003},"last":0.28299999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.224,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.67000000000000004,"15min":1.7649999999999999},"last":0.312},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":2.1970000000000001},"last":0.29999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42399999999999999,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.255},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":5.069},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.441,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":2.246},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59499999999999997,"15min":0.76100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":14.646000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":14.606999999999999},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.81399999999999995,"5min":0.94699999999999995,"15min":2.2959999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":6.4009999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.73299999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.90100000000000002,"15min":5.0309999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":0.87,"5min":1.012,"15min":2.2749999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.75800000000000001},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.92600000000000005,"15min":6.367},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":5},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307108,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064732,"statfs":{"total":96636764160,"available":96322285568,"internally_reserved":0,"allocated":1667072,"data_stored":755304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.70899999999999996},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.73799999999999999},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.74099999999999999},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":5.4900000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.628},"min":{"1min":0.433,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.68200000000000005,"5min":0.75900000000000001,"15min":4.0739999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53200000000000003,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":5.415},"last":0.39200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.83199999999999996,"15min":4.202},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.73199999999999998,"5min":0.81000000000000005,"15min":4.2130000000000001},"last":0.623}]},{"osd":5,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46700000000000003,"15min":0.49399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.73399999999999999,"15min":1.1279999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.73399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51400000000000001,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.73199999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.79500000000000004,"5min":0.86099999999999999,"15min":1.1000000000000001},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307092,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064748,"statfs":{"total":96636764160,"available":96322301952,"internally_reserved":0,"allocated":1650688,"data_stored":752415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.64800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.20499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":4.9039999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.82099999999999995,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.75800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57499999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":6.7949999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.26200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.86599999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.623},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.86599999999999999},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53800000000000003,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":7,"last update":"Fri Oct 6 05:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:01:31.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:01:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.743+0000 7fb4d6954700 1 Processor -- start 2023-10-06T05:01:31.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.743+0000 7fb4d6954700 1 -- start start 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.743+0000 7fb4d6954700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4d0157700 0x7fb4d00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.743+0000 7fb4d6954700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d00b2e50 con 0x7fb4d0157ad0 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.743+0000 7fb4d6954700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d0158290 con 0x7fb4d00b1b10 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.743+0000 7fb4d6954700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d00b1fc0 con 0x7fb4d0157700 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4d0157700 0x7fb4d00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cf7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb4d0157ad0 0x7fb4d00b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40212/0 (socket says 172.21.15.3:40212) 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4d0157700 0x7fb4d00cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50534/0 (socket says 172.21.15.3:50534) 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cf7fe700 1 -- 172.21.15.3:0/1064984112 learned_addr learned my addr 172.21.15.3:0/1064984112 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cffff700 1 -- 172.21.15.3:0/1064984112 >> v1:172.21.15.153:6789/0 conn(0x7fb4d00b1b10 legacy=0x7fb4d0067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cffff700 1 -- 172.21.15.3:0/1064984112 >> v1:172.21.15.3:6789/0 conn(0x7fb4d0157ad0 legacy=0x7fb4d00b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:31.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cffff700 1 -- 172.21.15.3:0/1064984112 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00b22a0 con 0x7fb4d0157700 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4ceffd700 1 -- 172.21.15.3:0/1064984112 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589836141 0 0) 0x7fb4d00b2e50 con 0x7fb4d0157ad0 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4cffff700 1 --2- 172.21.15.3:0/1064984112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157700 0x7fb4d00cc270 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00098e0 tx=0x7fb4c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d73f4b9d9c04ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4ceffd700 1 -- 172.21.15.3:0/1064984112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0009e80 con 0x7fb4d0157700 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4ceffd700 1 -- 172.21.15.3:0/1064984112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c000b860 con 0x7fb4d0157700 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.744+0000 7fb4ceffd700 1 -- 172.21.15.3:0/1064984112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000bbe0 con 0x7fb4d0157700 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.745+0000 7fb4d6954700 1 -- 172.21.15.3:0/1064984112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157700 msgr2=0x7fb4d00cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.745+0000 7fb4d6954700 1 --2- 172.21.15.3:0/1064984112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157700 0x7fb4d00cc270 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00098e0 tx=0x7fb4c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.745+0000 7fb4d6954700 1 -- 172.21.15.3:0/1064984112 shutdown_connections 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.745+0000 7fb4d6954700 1 --2- 172.21.15.3:0/1064984112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157700 0x7fb4d00cc270 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.745+0000 7fb4d6954700 1 -- 172.21.15.3:0/1064984112 >> 172.21.15.3:0/1064984112 conn(0x7fb4d00ce2c0 msgr2=0x7fb4d00ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:31.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.745+0000 7fb4d6954700 1 -- 172.21.15.3:0/1064984112 shutdown_connections 2023-10-06T05:01:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.745+0000 7fb4d6954700 1 -- 172.21.15.3:0/1064984112 wait complete. 2023-10-06T05:01:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.746+0000 7fb4d6954700 1 Processor -- start 2023-10-06T05:01:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.746+0000 7fb4d6954700 1 -- start start 2023-10-06T05:01:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d6954700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d00b1b10 0x7fb4d014a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d6954700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4d0157700 0x7fb4d014d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d6954700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157ad0 0x7fb4d0147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d6954700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0152450 con 0x7fb4d0157ad0 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d6954700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00adcf0 con 0x7fb4d00b1b10 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d6954700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00adfd0 con 0x7fb4d0157700 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157ad0 0x7fb4d0147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157ad0 0x7fb4d0147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50536/0 (socket says 172.21.15.3:50536) 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 -- 172.21.15.3:0/2748571742 learned_addr learned my addr 172.21.15.3:0/2748571742 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4cffff700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d00b1b10 0x7fb4d014a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4cf7fe700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4d0157700 0x7fb4d014d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 -- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4d0157700 msgr2=0x7fb4d014d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4d0157700 0x7fb4d014d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 -- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d00b1b10 msgr2=0x7fb4d014a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d00b1b10 0x7fb4d014a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4d4ef1700 1 -- 172.21.15.3:0/2748571742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0009420 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.747+0000 7fb4cffff700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d00b1b10 0x7fb4d014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.748+0000 7fb4d4ef1700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157ad0 0x7fb4d0147e90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c910 tx=0x7fb4c80016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.748+0000 7fb4b7fff700 1 -- 172.21.15.3:0/2748571742 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800a7e0 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.748+0000 7fb4b7fff700 1 -- 172.21.15.3:0/2748571742 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c800a940 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.748+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d0152710 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.748+0000 7fb4b7fff700 1 -- 172.21.15.3:0/2748571742 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800ad00 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.748+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d0152c30 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.749+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d00644c0 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.749+0000 7fb4b7fff700 1 -- 172.21.15.3:0/2748571742 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4c8012630 con 0x7fb4d0157ad0 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.750+0000 7fb4b7fff700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.750+0000 7fb4b7fff700 1 -- 172.21.15.3:0/2748571742 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb4c80848f0 con 0x7fb4d0157ad0 2023-10-06T05:01:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.750+0000 7fb4cffff700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.750+0000 7fb4cffff700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb4bc066f30 0x7fb4bc0693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00093f0 tx=0x7fb4c0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.754+0000 7fb4b7fff700 1 -- 172.21.15.3:0/2748571742 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c8051a00 con 0x7fb4d0157ad0 2023-10-06T05:01:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.942+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4d0152900 con 0x7fb4bc066f30 2023-10-06T05:01:31.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.951+0000 7fb4b7fff700 1 -- 172.21.15.3:0/2748571742 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311721 (secure 0 0 0) 0x7fb4d0152900 con 0x7fb4bc066f30 2023-10-06T05:01:31.957 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb4bc066f30 msgr2=0x7fb4bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb4bc066f30 0x7fb4bc0693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00093f0 tx=0x7fb4c0017000 comp rx=0 tx=0).stop 2023-10-06T05:01:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157ad0 msgr2=0x7fb4d0147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157ad0 0x7fb4d0147e90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c910 tx=0x7fb4c80016f0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 shutdown_connections 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d00b1b10 0x7fb4d014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4d0157ad0 0x7fb4d0147e90 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 --2- 172.21.15.3:0/2748571742 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4d0157700 0x7fb4d014d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 >> 172.21.15.3:0/2748571742 conn(0x7fb4d00ce2c0 msgr2=0x7fb4d00ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 shutdown_connections 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:31.955+0000 7fb4d6954700 1 -- 172.21.15.3:0/2748571742 wait complete. 2023-10-06T05:01:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:01:31.968 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2023-10-06T05:01:31.559531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449788,"kb_used_data":14140,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524932,"statfs":{"total":773094113280,"available":770585530368,"internally_reserved":0,"allocated":14479360,"data_stored":7211433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.671210"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919337+0000","last_change":"2023-10-06T05:01:22.917546+0000","last_active":"2023-10-06T05:01:22.919337+0000","last_peered":"2023-10-06T05:01:22.919337+0000","last_clean":"2023-10-06T05:01:22.919337+0000","last_became_active":"2023-10-06T05:01:22.916374+0000","last_became_peered":"2023-10-06T05:01:22.916374+0000","last_unstale":"2023-10-06T05:01:22.919337+0000","last_undegraded":"2023-10-06T05:01:22.919337+0000","last_fullsized":"2023-10-06T05:01:22.919337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917323+0000","last_change":"2023-10-06T05:01:22.915942+0000","last_active":"2023-10-06T05:01:22.917323+0000","last_peered":"2023-10-06T05:01:22.917323+0000","last_clean":"2023-10-06T05:01:22.917323+0000","last_became_active":"2023-10-06T05:01:22.915602+0000","last_became_peered":"2023-10-06T05:01:22.915602+0000","last_unstale":"2023-10-06T05:01:22.917323+0000","last_undegraded":"2023-10-06T05:01:22.917323+0000","last_fullsized":"2023-10-06T05:01:22.917323+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:39.759900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917901+0000","last_change":"2023-10-06T05:01:22.916859+0000","last_active":"2023-10-06T05:01:22.917901+0000","last_peered":"2023-10-06T05:01:22.917901+0000","last_clean":"2023-10-06T05:01:22.917901+0000","last_became_active":"2023-10-06T05:01:22.916504+0000","last_became_peered":"2023-10-06T05:01:22.916504+0000","last_unstale":"2023-10-06T05:01:22.917901+0000","last_undegraded":"2023-10-06T05:01:22.917901+0000","last_fullsized":"2023-10-06T05:01:22.917901+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:36.075945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915308+0000","last_change":"2023-10-06T05:01:22.914389+0000","last_active":"2023-10-06T05:01:22.915308+0000","last_peered":"2023-10-06T05:01:22.915308+0000","last_clean":"2023-10-06T05:01:22.915308+0000","last_became_active":"2023-10-06T05:01:22.914080+0000","last_became_peered":"2023-10-06T05:01:22.914080+0000","last_unstale":"2023-10-06T05:01:22.915308+0000","last_undegraded":"2023-10-06T05:01:22.915308+0000","last_fullsized":"2023-10-06T05:01:22.915308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:56.228351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912802+0000","last_change":"2023-10-06T05:01:22.908378+0000","last_active":"2023-10-06T05:01:22.912802+0000","last_peered":"2023-10-06T05:01:22.912802+0000","last_clean":"2023-10-06T05:01:22.912802+0000","last_became_active":"2023-10-06T05:01:22.907429+0000","last_became_peered":"2023-10-06T05:01:22.907429+0000","last_unstale":"2023-10-06T05:01:22.912802+0000","last_undegraded":"2023-10-06T05:01:22.912802+0000","last_fullsized":"2023-10-06T05:01:22.912802+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:32.443484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912753+0000","last_change":"2023-10-06T05:01:22.907760+0000","last_active":"2023-10-06T05:01:22.912753+0000","last_peered":"2023-10-06T05:01:22.912753+0000","last_clean":"2023-10-06T05:01:22.912753+0000","last_became_active":"2023-10-06T05:01:22.907466+0000","last_became_peered":"2023-10-06T05:01:22.907466+0000","last_unstale":"2023-10-06T05:01:22.912753+0000","last_undegraded":"2023-10-06T05:01:22.912753+0000","last_fullsized":"2023-10-06T05:01:22.912753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:21.629837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908240+0000","last_change":"2023-10-06T05:01:22.907593+0000","last_active":"2023-10-06T05:01:22.908240+0000","last_peered":"2023-10-06T05:01:22.908240+0000","last_clean":"2023-10-06T05:01:22.908240+0000","last_became_active":"2023-10-06T05:01:22.907290+0000","last_became_peered":"2023-10-06T05:01:22.907290+0000","last_unstale":"2023-10-06T05:01:22.908240+0000","last_undegraded":"2023-10-06T05:01:22.908240+0000","last_fullsized":"2023-10-06T05:01:22.908240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921703+0000","last_change":"2023-10-06T05:01:22.917866+0000","last_active":"2023-10-06T05:01:22.921703+0000","last_peered":"2023-10-06T05:01:22.921703+0000","last_clean":"2023-10-06T05:01:22.921703+0000","last_became_active":"2023-10-06T05:01:22.917142+0000","last_became_peered":"2023-10-06T05:01:22.917142+0000","last_unstale":"2023-10-06T05:01:22.921703+0000","last_undegraded":"2023-10-06T05:01:22.921703+0000","last_fullsized":"2023-10-06T05:01:22.921703+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:39.089086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910976+0000","last_change":"2023-10-06T05:01:22.906539+0000","last_active":"2023-10-06T05:01:22.910976+0000","last_peered":"2023-10-06T05:01:22.910976+0000","last_clean":"2023-10-06T05:01:22.910976+0000","last_became_active":"2023-10-06T05:01:22.905906+0000","last_became_peered":"2023-10-06T05:01:22.905906+0000","last_unstale":"2023-10-06T05:01:22.910976+0000","last_undegraded":"2023-10-06T05:01:22.910976+0000","last_fullsized":"2023-10-06T05:01:22.910976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:33.022370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908499+0000","last_change":"2023-10-06T05:01:22.906059+0000","last_active":"2023-10-06T05:01:22.908499+0000","last_peered":"2023-10-06T05:01:22.908499+0000","last_clean":"2023-10-06T05:01:22.908499+0000","last_became_active":"2023-10-06T05:01:22.905782+0000","last_became_peered":"2023-10-06T05:01:22.905782+0000","last_unstale":"2023-10-06T05:01:22.908499+0000","last_undegraded":"2023-10-06T05:01:22.908499+0000","last_fullsized":"2023-10-06T05:01:22.908499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:31.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916007+0000","last_change":"2023-10-06T05:01:22.914642+0000","last_active":"2023-10-06T05:01:22.916007+0000","last_peered":"2023-10-06T05:01:22.916007+0000","last_clean":"2023-10-06T05:01:22.916007+0000","last_became_active":"2023-10-06T05:01:22.913781+0000","last_became_peered":"2023-10-06T05:01:22.913781+0000","last_unstale":"2023-10-06T05:01:22.916007+0000","last_undegraded":"2023-10-06T05:01:22.916007+0000","last_fullsized":"2023-10-06T05:01:22.916007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:39.375404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915290+0000","last_change":"2023-10-06T05:01:22.913641+0000","last_active":"2023-10-06T05:01:22.915290+0000","last_peered":"2023-10-06T05:01:22.915290+0000","last_clean":"2023-10-06T05:01:22.915290+0000","last_became_active":"2023-10-06T05:01:22.913356+0000","last_became_peered":"2023-10-06T05:01:22.913356+0000","last_unstale":"2023-10-06T05:01:22.915290+0000","last_undegraded":"2023-10-06T05:01:22.915290+0000","last_fullsized":"2023-10-06T05:01:22.915290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:50.229096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909396+0000","last_change":"2023-10-06T05:01:22.906295+0000","last_active":"2023-10-06T05:01:22.909396+0000","last_peered":"2023-10-06T05:01:22.909396+0000","last_clean":"2023-10-06T05:01:22.909396+0000","last_became_active":"2023-10-06T05:01:22.905241+0000","last_became_peered":"2023-10-06T05:01:22.905241+0000","last_unstale":"2023-10-06T05:01:22.909396+0000","last_undegraded":"2023-10-06T05:01:22.909396+0000","last_fullsized":"2023-10-06T05:01:22.909396+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:56.323569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915262+0000","last_change":"2023-10-06T05:01:22.913298+0000","last_active":"2023-10-06T05:01:22.915262+0000","last_peered":"2023-10-06T05:01:22.915262+0000","last_clean":"2023-10-06T05:01:22.915262+0000","last_became_active":"2023-10-06T05:01:22.913012+0000","last_became_peered":"2023-10-06T05:01:22.913012+0000","last_unstale":"2023-10-06T05:01:22.915262+0000","last_undegraded":"2023-10-06T05:01:22.915262+0000","last_fullsized":"2023-10-06T05:01:22.915262+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:31.315774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921375+0000","last_change":"2023-10-06T05:01:22.920640+0000","last_active":"2023-10-06T05:01:22.921375+0000","last_peered":"2023-10-06T05:01:22.921375+0000","last_clean":"2023-10-06T05:01:22.921375+0000","last_became_active":"2023-10-06T05:01:22.920435+0000","last_became_peered":"2023-10-06T05:01:22.920435+0000","last_unstale":"2023-10-06T05:01:22.921375+0000","last_undegraded":"2023-10-06T05:01:22.921375+0000","last_fullsized":"2023-10-06T05:01:22.921375+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:21.298643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921735+0000","last_change":"2023-10-06T05:01:22.921275+0000","last_active":"2023-10-06T05:01:22.921735+0000","last_peered":"2023-10-06T05:01:22.921735+0000","last_clean":"2023-10-06T05:01:22.921735+0000","last_became_active":"2023-10-06T05:01:22.920953+0000","last_became_peered":"2023-10-06T05:01:22.920953+0000","last_unstale":"2023-10-06T05:01:22.921735+0000","last_undegraded":"2023-10-06T05:01:22.921735+0000","last_fullsized":"2023-10-06T05:01:22.921735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:22.455454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921292+0000","last_change":"2023-10-06T05:01:22.919416+0000","last_active":"2023-10-06T05:01:22.921292+0000","last_peered":"2023-10-06T05:01:22.921292+0000","last_clean":"2023-10-06T05:01:22.921292+0000","last_became_active":"2023-10-06T05:01:22.917490+0000","last_became_peered":"2023-10-06T05:01:22.917490+0000","last_unstale":"2023-10-06T05:01:22.921292+0000","last_undegraded":"2023-10-06T05:01:22.921292+0000","last_fullsized":"2023-10-06T05:01:22.921292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:14.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912183+0000","last_change":"2023-10-06T05:01:22.910088+0000","last_active":"2023-10-06T05:01:22.912183+0000","last_peered":"2023-10-06T05:01:22.912183+0000","last_clean":"2023-10-06T05:01:22.912183+0000","last_became_active":"2023-10-06T05:01:22.909830+0000","last_became_peered":"2023-10-06T05:01:22.909830+0000","last_unstale":"2023-10-06T05:01:22.912183+0000","last_undegraded":"2023-10-06T05:01:22.912183+0000","last_fullsized":"2023-10-06T05:01:22.912183+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:14.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920682+0000","last_change":"2023-10-06T05:01:22.915966+0000","last_active":"2023-10-06T05:01:22.920682+0000","last_peered":"2023-10-06T05:01:22.920682+0000","last_clean":"2023-10-06T05:01:22.920682+0000","last_became_active":"2023-10-06T05:01:22.915477+0000","last_became_peered":"2023-10-06T05:01:22.915477+0000","last_unstale":"2023-10-06T05:01:22.920682+0000","last_undegraded":"2023-10-06T05:01:22.920682+0000","last_fullsized":"2023-10-06T05:01:22.920682+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:24.762820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908898+0000","last_change":"2023-10-06T05:01:22.905656+0000","last_active":"2023-10-06T05:01:22.908898+0000","last_peered":"2023-10-06T05:01:22.908898+0000","last_clean":"2023-10-06T05:01:22.908898+0000","last_became_active":"2023-10-06T05:01:22.905402+0000","last_became_peered":"2023-10-06T05:01:22.905402+0000","last_unstale":"2023-10-06T05:01:22.908898+0000","last_undegraded":"2023-10-06T05:01:22.908898+0000","last_fullsized":"2023-10-06T05:01:22.908898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.573131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912788+0000","last_change":"2023-10-06T05:01:22.908184+0000","last_active":"2023-10-06T05:01:22.912788+0000","last_peered":"2023-10-06T05:01:22.912788+0000","last_clean":"2023-10-06T05:01:22.912788+0000","last_became_active":"2023-10-06T05:01:22.904908+0000","last_became_peered":"2023-10-06T05:01:22.904908+0000","last_unstale":"2023-10-06T05:01:22.912788+0000","last_undegraded":"2023-10-06T05:01:22.912788+0000","last_fullsized":"2023-10-06T05:01:22.912788+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:42.892786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912280+0000","last_change":"2023-10-06T05:01:22.908024+0000","last_active":"2023-10-06T05:01:22.912280+0000","last_peered":"2023-10-06T05:01:22.912280+0000","last_clean":"2023-10-06T05:01:22.912280+0000","last_became_active":"2023-10-06T05:01:22.907701+0000","last_became_peered":"2023-10-06T05:01:22.907701+0000","last_unstale":"2023-10-06T05:01:22.912280+0000","last_undegraded":"2023-10-06T05:01:22.912280+0000","last_fullsized":"2023-10-06T05:01:22.912280+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912827+0000","last_change":"2023-10-06T05:01:22.911724+0000","last_active":"2023-10-06T05:01:22.912827+0000","last_peered":"2023-10-06T05:01:22.912827+0000","last_clean":"2023-10-06T05:01:22.912827+0000","last_became_active":"2023-10-06T05:01:22.911247+0000","last_became_peered":"2023-10-06T05:01:22.911247+0000","last_unstale":"2023-10-06T05:01:22.912827+0000","last_undegraded":"2023-10-06T05:01:22.912827+0000","last_fullsized":"2023-10-06T05:01:22.912827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910490+0000","last_change":"2023-10-06T05:01:22.902838+0000","last_active":"2023-10-06T05:01:22.910490+0000","last_peered":"2023-10-06T05:01:22.910490+0000","last_clean":"2023-10-06T05:01:22.910490+0000","last_became_active":"2023-10-06T05:01:22.902549+0000","last_became_peered":"2023-10-06T05:01:22.902549+0000","last_unstale":"2023-10-06T05:01:22.910490+0000","last_undegraded":"2023-10-06T05:01:22.910490+0000","last_fullsized":"2023-10-06T05:01:22.910490+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:26.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912308+0000","last_change":"2023-10-06T05:01:22.908586+0000","last_active":"2023-10-06T05:01:22.912308+0000","last_peered":"2023-10-06T05:01:22.912308+0000","last_clean":"2023-10-06T05:01:22.912308+0000","last_became_active":"2023-10-06T05:01:22.908289+0000","last_became_peered":"2023-10-06T05:01:22.908289+0000","last_unstale":"2023-10-06T05:01:22.912308+0000","last_undegraded":"2023-10-06T05:01:22.912308+0000","last_fullsized":"2023-10-06T05:01:22.912308+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:38.888355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914573+0000","last_change":"2023-10-06T05:01:22.910843+0000","last_active":"2023-10-06T05:01:22.914573+0000","last_peered":"2023-10-06T05:01:22.914573+0000","last_clean":"2023-10-06T05:01:22.914573+0000","last_became_active":"2023-10-06T05:01:22.910470+0000","last_became_peered":"2023-10-06T05:01:22.910470+0000","last_unstale":"2023-10-06T05:01:22.914573+0000","last_undegraded":"2023-10-06T05:01:22.914573+0000","last_fullsized":"2023-10-06T05:01:22.914573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912812+0000","last_change":"2023-10-06T05:01:22.908293+0000","last_active":"2023-10-06T05:01:22.912812+0000","last_peered":"2023-10-06T05:01:22.912812+0000","last_clean":"2023-10-06T05:01:22.912812+0000","last_became_active":"2023-10-06T05:01:22.907986+0000","last_became_peered":"2023-10-06T05:01:22.907986+0000","last_unstale":"2023-10-06T05:01:22.912812+0000","last_undegraded":"2023-10-06T05:01:22.912812+0000","last_fullsized":"2023-10-06T05:01:22.912812+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:58.242153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910601+0000","last_change":"2023-10-06T05:01:22.905470+0000","last_active":"2023-10-06T05:01:22.910601+0000","last_peered":"2023-10-06T05:01:22.910601+0000","last_clean":"2023-10-06T05:01:22.910601+0000","last_became_active":"2023-10-06T05:01:22.905162+0000","last_became_peered":"2023-10-06T05:01:22.905162+0000","last_unstale":"2023-10-06T05:01:22.910601+0000","last_undegraded":"2023-10-06T05:01:22.910601+0000","last_fullsized":"2023-10-06T05:01:22.910601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:53.259645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912975+0000","last_change":"2023-10-06T05:01:22.911783+0000","last_active":"2023-10-06T05:01:22.912975+0000","last_peered":"2023-10-06T05:01:22.912975+0000","last_clean":"2023-10-06T05:01:22.912975+0000","last_became_active":"2023-10-06T05:01:22.911488+0000","last_became_peered":"2023-10-06T05:01:22.911488+0000","last_unstale":"2023-10-06T05:01:22.912975+0000","last_undegraded":"2023-10-06T05:01:22.912975+0000","last_fullsized":"2023-10-06T05:01:22.912975+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912590+0000","last_change":"2023-10-06T05:01:22.911945+0000","last_active":"2023-10-06T05:01:22.912590+0000","last_peered":"2023-10-06T05:01:22.912590+0000","last_clean":"2023-10-06T05:01:22.912590+0000","last_became_active":"2023-10-06T05:01:22.911675+0000","last_became_peered":"2023-10-06T05:01:22.911675+0000","last_unstale":"2023-10-06T05:01:22.912590+0000","last_undegraded":"2023-10-06T05:01:22.912590+0000","last_fullsized":"2023-10-06T05:01:22.912590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:32.223585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914827+0000","last_change":"2023-10-06T05:01:22.913460+0000","last_active":"2023-10-06T05:01:22.914827+0000","last_peered":"2023-10-06T05:01:22.914827+0000","last_clean":"2023-10-06T05:01:22.914827+0000","last_became_active":"2023-10-06T05:01:22.913178+0000","last_became_peered":"2023-10-06T05:01:22.913178+0000","last_unstale":"2023-10-06T05:01:22.914827+0000","last_undegraded":"2023-10-06T05:01:22.914827+0000","last_fullsized":"2023-10-06T05:01:22.914827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:16.124524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.923028+0000","last_change":"2023-10-06T05:01:22.918920+0000","last_active":"2023-10-06T05:01:22.923028+0000","last_peered":"2023-10-06T05:01:22.923028+0000","last_clean":"2023-10-06T05:01:22.923028+0000","last_became_active":"2023-10-06T05:01:22.918545+0000","last_became_peered":"2023-10-06T05:01:22.918545+0000","last_unstale":"2023-10-06T05:01:22.923028+0000","last_undegraded":"2023-10-06T05:01:22.923028+0000","last_fullsized":"2023-10-06T05:01:22.923028+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909358+0000","last_change":"2023-10-06T05:01:22.906868+0000","last_active":"2023-10-06T05:01:22.909358+0000","last_peered":"2023-10-06T05:01:22.909358+0000","last_clean":"2023-10-06T05:01:22.909358+0000","last_became_active":"2023-10-06T05:01:22.906368+0000","last_became_peered":"2023-10-06T05:01:22.906368+0000","last_unstale":"2023-10-06T05:01:22.909358+0000","last_undegraded":"2023-10-06T05:01:22.909358+0000","last_fullsized":"2023-10-06T05:01:22.909358+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:45.514209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910892+0000","last_change":"2023-10-06T05:01:21.905167+0000","last_active":"2023-10-06T05:01:22.910892+0000","last_peered":"2023-10-06T05:01:22.910892+0000","last_clean":"2023-10-06T05:01:22.910892+0000","last_became_active":"2023-10-06T05:01:21.904856+0000","last_became_peered":"2023-10-06T05:01:21.904856+0000","last_unstale":"2023-10-06T05:01:22.910892+0000","last_undegraded":"2023-10-06T05:01:22.910892+0000","last_fullsized":"2023-10-06T05:01:22.910892+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916718+0000","last_change":"2023-10-06T05:01:22.915682+0000","last_active":"2023-10-06T05:01:22.916718+0000","last_peered":"2023-10-06T05:01:22.916718+0000","last_clean":"2023-10-06T05:01:22.916718+0000","last_became_active":"2023-10-06T05:01:22.915425+0000","last_became_peered":"2023-10-06T05:01:22.915425+0000","last_unstale":"2023-10-06T05:01:22.916718+0000","last_undegraded":"2023-10-06T05:01:22.916718+0000","last_fullsized":"2023-10-06T05:01:22.916718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913713+0000","last_change":"2023-10-06T05:01:21.909299+0000","last_active":"2023-10-06T05:01:22.913713+0000","last_peered":"2023-10-06T05:01:22.913713+0000","last_clean":"2023-10-06T05:01:22.913713+0000","last_became_active":"2023-10-06T05:01:21.908989+0000","last_became_peered":"2023-10-06T05:01:21.908989+0000","last_unstale":"2023-10-06T05:01:22.913713+0000","last_undegraded":"2023-10-06T05:01:22.913713+0000","last_fullsized":"2023-10-06T05:01:22.913713+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921765+0000","last_change":"2023-10-06T05:01:22.918183+0000","last_active":"2023-10-06T05:01:22.921765+0000","last_peered":"2023-10-06T05:01:22.921765+0000","last_clean":"2023-10-06T05:01:22.921765+0000","last_became_active":"2023-10-06T05:01:22.917637+0000","last_became_peered":"2023-10-06T05:01:22.917637+0000","last_unstale":"2023-10-06T05:01:22.921765+0000","last_undegraded":"2023-10-06T05:01:22.921765+0000","last_fullsized":"2023-10-06T05:01:22.921765+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914828+0000","last_change":"2023-10-06T05:01:21.909301+0000","last_active":"2023-10-06T05:01:22.914828+0000","last_peered":"2023-10-06T05:01:22.914828+0000","last_clean":"2023-10-06T05:01:22.914828+0000","last_became_active":"2023-10-06T05:01:21.908984+0000","last_became_peered":"2023-10-06T05:01:21.908984+0000","last_unstale":"2023-10-06T05:01:22.914828+0000","last_undegraded":"2023-10-06T05:01:22.914828+0000","last_fullsized":"2023-10-06T05:01:22.914828+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:26.376744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920398+0000","last_change":"2023-10-06T05:01:22.910823+0000","last_active":"2023-10-06T05:01:22.920398+0000","last_peered":"2023-10-06T05:01:22.920398+0000","last_clean":"2023-10-06T05:01:22.920398+0000","last_became_active":"2023-10-06T05:01:22.910158+0000","last_became_peered":"2023-10-06T05:01:22.910158+0000","last_unstale":"2023-10-06T05:01:22.920398+0000","last_undegraded":"2023-10-06T05:01:22.920398+0000","last_fullsized":"2023-10-06T05:01:22.920398+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:33.254913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917199+0000","last_change":"2023-10-06T05:01:21.898248+0000","last_active":"2023-10-06T05:01:22.917199+0000","last_peered":"2023-10-06T05:01:22.917199+0000","last_clean":"2023-10-06T05:01:22.917199+0000","last_became_active":"2023-10-06T05:01:21.897180+0000","last_became_peered":"2023-10-06T05:01:21.897180+0000","last_unstale":"2023-10-06T05:01:22.917199+0000","last_undegraded":"2023-10-06T05:01:22.917199+0000","last_fullsized":"2023-10-06T05:01:22.917199+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:59.743528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909059+0000","last_change":"2023-10-06T05:01:22.905942+0000","last_active":"2023-10-06T05:01:22.909059+0000","last_peered":"2023-10-06T05:01:22.909059+0000","last_clean":"2023-10-06T05:01:22.909059+0000","last_became_active":"2023-10-06T05:01:22.905653+0000","last_became_peered":"2023-10-06T05:01:22.905653+0000","last_unstale":"2023-10-06T05:01:22.909059+0000","last_undegraded":"2023-10-06T05:01:22.909059+0000","last_fullsized":"2023-10-06T05:01:22.909059+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:12.300608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921227+0000","last_change":"2023-10-06T05:01:21.902635+0000","last_active":"2023-10-06T05:01:22.921227+0000","last_peered":"2023-10-06T05:01:22.921227+0000","last_clean":"2023-10-06T05:01:22.921227+0000","last_became_active":"2023-10-06T05:01:21.901883+0000","last_became_peered":"2023-10-06T05:01:21.901883+0000","last_unstale":"2023-10-06T05:01:22.921227+0000","last_undegraded":"2023-10-06T05:01:22.921227+0000","last_fullsized":"2023-10-06T05:01:22.921227+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920542+0000","last_change":"2023-10-06T05:01:22.917447+0000","last_active":"2023-10-06T05:01:22.920542+0000","last_peered":"2023-10-06T05:01:22.920542+0000","last_clean":"2023-10-06T05:01:22.920542+0000","last_became_active":"2023-10-06T05:01:22.917130+0000","last_became_peered":"2023-10-06T05:01:22.917130+0000","last_unstale":"2023-10-06T05:01:22.920542+0000","last_undegraded":"2023-10-06T05:01:22.920542+0000","last_fullsized":"2023-10-06T05:01:22.920542+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:20.172157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916407+0000","last_change":"2023-10-06T05:01:21.906758+0000","last_active":"2023-10-06T05:01:22.916407+0000","last_peered":"2023-10-06T05:01:22.916407+0000","last_clean":"2023-10-06T05:01:22.916407+0000","last_became_active":"2023-10-06T05:01:21.906463+0000","last_became_peered":"2023-10-06T05:01:21.906463+0000","last_unstale":"2023-10-06T05:01:22.916407+0000","last_undegraded":"2023-10-06T05:01:22.916407+0000","last_fullsized":"2023-10-06T05:01:22.916407+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:42.205607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.908789+0000","last_change":"2023-10-06T05:01:22.905020+0000","last_active":"2023-10-06T05:01:22.908789+0000","last_peered":"2023-10-06T05:01:22.908789+0000","last_clean":"2023-10-06T05:01:22.908789+0000","last_became_active":"2023-10-06T05:01:22.904762+0000","last_became_peered":"2023-10-06T05:01:22.904762+0000","last_unstale":"2023-10-06T05:01:22.908789+0000","last_undegraded":"2023-10-06T05:01:22.908789+0000","last_fullsized":"2023-10-06T05:01:22.908789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900865+0000","last_change":"2023-10-06T05:01:21.907467+0000","last_active":"2023-10-06T05:01:22.900865+0000","last_peered":"2023-10-06T05:01:22.900865+0000","last_clean":"2023-10-06T05:01:22.900865+0000","last_became_active":"2023-10-06T05:01:21.907169+0000","last_became_peered":"2023-10-06T05:01:21.907169+0000","last_unstale":"2023-10-06T05:01:22.900865+0000","last_undegraded":"2023-10-06T05:01:22.900865+0000","last_fullsized":"2023-10-06T05:01:22.900865+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:30.309675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918810+0000","last_change":"2023-10-06T05:01:22.916673+0000","last_active":"2023-10-06T05:01:22.918810+0000","last_peered":"2023-10-06T05:01:22.918810+0000","last_clean":"2023-10-06T05:01:22.918810+0000","last_became_active":"2023-10-06T05:01:22.916393+0000","last_became_peered":"2023-10-06T05:01:22.916393+0000","last_unstale":"2023-10-06T05:01:22.918810+0000","last_undegraded":"2023-10-06T05:01:22.918810+0000","last_fullsized":"2023-10-06T05:01:22.918810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:00.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.890628+0000","last_change":"2023-10-06T05:01:21.904853+0000","last_active":"2023-10-06T05:01:22.890628+0000","last_peered":"2023-10-06T05:01:22.890628+0000","last_clean":"2023-10-06T05:01:22.890628+0000","last_became_active":"2023-10-06T05:01:21.904448+0000","last_became_peered":"2023-10-06T05:01:21.904448+0000","last_unstale":"2023-10-06T05:01:22.890628+0000","last_undegraded":"2023-10-06T05:01:22.890628+0000","last_fullsized":"2023-10-06T05:01:22.890628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.567032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915557+0000","last_change":"2023-10-06T05:01:22.909027+0000","last_active":"2023-10-06T05:01:22.915557+0000","last_peered":"2023-10-06T05:01:22.915557+0000","last_clean":"2023-10-06T05:01:22.915557+0000","last_became_active":"2023-10-06T05:01:22.908783+0000","last_became_peered":"2023-10-06T05:01:22.908783+0000","last_unstale":"2023-10-06T05:01:22.915557+0000","last_undegraded":"2023-10-06T05:01:22.915557+0000","last_fullsized":"2023-10-06T05:01:22.915557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:48.927255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914458+0000","last_change":"2023-10-06T05:01:21.901342+0000","last_active":"2023-10-06T05:01:22.914458+0000","last_peered":"2023-10-06T05:01:22.914458+0000","last_clean":"2023-10-06T05:01:22.914458+0000","last_became_active":"2023-10-06T05:01:21.901020+0000","last_became_peered":"2023-10-06T05:01:21.901020+0000","last_unstale":"2023-10-06T05:01:22.914458+0000","last_undegraded":"2023-10-06T05:01:22.914458+0000","last_fullsized":"2023-10-06T05:01:22.914458+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:26.399481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920662+0000","last_change":"2023-10-06T05:01:22.915799+0000","last_active":"2023-10-06T05:01:22.920662+0000","last_peered":"2023-10-06T05:01:22.920662+0000","last_clean":"2023-10-06T05:01:22.920662+0000","last_became_active":"2023-10-06T05:01:22.915224+0000","last_became_peered":"2023-10-06T05:01:22.915224+0000","last_unstale":"2023-10-06T05:01:22.920662+0000","last_undegraded":"2023-10-06T05:01:22.920662+0000","last_fullsized":"2023-10-06T05:01:22.920662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:24.443047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909917+0000","last_change":"2023-10-06T05:01:21.903666+0000","last_active":"2023-10-06T05:01:22.909917+0000","last_peered":"2023-10-06T05:01:22.909917+0000","last_clean":"2023-10-06T05:01:22.909917+0000","last_became_active":"2023-10-06T05:01:21.903416+0000","last_became_peered":"2023-10-06T05:01:21.903416+0000","last_unstale":"2023-10-06T05:01:22.909917+0000","last_undegraded":"2023-10-06T05:01:22.909917+0000","last_fullsized":"2023-10-06T05:01:22.909917+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:38.753801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.913218+0000","last_change":"2023-10-06T05:01:22.908994+0000","last_active":"2023-10-06T05:01:22.913218+0000","last_peered":"2023-10-06T05:01:22.913218+0000","last_clean":"2023-10-06T05:01:22.913218+0000","last_became_active":"2023-10-06T05:01:22.908627+0000","last_became_peered":"2023-10-06T05:01:22.908627+0000","last_unstale":"2023-10-06T05:01:22.913218+0000","last_undegraded":"2023-10-06T05:01:22.913218+0000","last_fullsized":"2023-10-06T05:01:22.913218+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:30.369642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920770+0000","last_change":"2023-10-06T05:01:21.901636+0000","last_active":"2023-10-06T05:01:22.920770+0000","last_peered":"2023-10-06T05:01:22.920770+0000","last_clean":"2023-10-06T05:01:22.920770+0000","last_became_active":"2023-10-06T05:01:21.901348+0000","last_became_peered":"2023-10-06T05:01:21.901348+0000","last_unstale":"2023-10-06T05:01:22.920770+0000","last_undegraded":"2023-10-06T05:01:22.920770+0000","last_fullsized":"2023-10-06T05:01:22.920770+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:07.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919815+0000","last_change":"2023-10-06T05:01:22.918705+0000","last_active":"2023-10-06T05:01:22.919815+0000","last_peered":"2023-10-06T05:01:22.919815+0000","last_clean":"2023-10-06T05:01:22.919815+0000","last_became_active":"2023-10-06T05:01:22.918437+0000","last_became_peered":"2023-10-06T05:01:22.918437+0000","last_unstale":"2023-10-06T05:01:22.919815+0000","last_undegraded":"2023-10-06T05:01:22.919815+0000","last_fullsized":"2023-10-06T05:01:22.919815+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900294+0000","last_change":"2023-10-06T05:01:21.903733+0000","last_active":"2023-10-06T05:01:22.900294+0000","last_peered":"2023-10-06T05:01:22.900294+0000","last_clean":"2023-10-06T05:01:22.900294+0000","last_became_active":"2023-10-06T05:01:21.903433+0000","last_became_peered":"2023-10-06T05:01:21.903433+0000","last_unstale":"2023-10-06T05:01:22.900294+0000","last_undegraded":"2023-10-06T05:01:22.900294+0000","last_fullsized":"2023-10-06T05:01:22.900294+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:39.894106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920786+0000","last_change":"2023-10-06T05:01:22.919693+0000","last_active":"2023-10-06T05:01:22.920786+0000","last_peered":"2023-10-06T05:01:22.920786+0000","last_clean":"2023-10-06T05:01:22.920786+0000","last_became_active":"2023-10-06T05:01:22.919445+0000","last_became_peered":"2023-10-06T05:01:22.919445+0000","last_unstale":"2023-10-06T05:01:22.920786+0000","last_undegraded":"2023-10-06T05:01:22.920786+0000","last_fullsized":"2023-10-06T05:01:22.920786+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:00.412186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916231+0000","last_change":"2023-10-06T05:01:21.901798+0000","last_active":"2023-10-06T05:01:22.916231+0000","last_peered":"2023-10-06T05:01:22.916231+0000","last_clean":"2023-10-06T05:01:22.916231+0000","last_became_active":"2023-10-06T05:01:21.901536+0000","last_became_peered":"2023-10-06T05:01:21.901536+0000","last_unstale":"2023-10-06T05:01:22.916231+0000","last_undegraded":"2023-10-06T05:01:22.916231+0000","last_fullsized":"2023-10-06T05:01:22.916231+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909492+0000","last_change":"2023-10-06T05:01:22.908931+0000","last_active":"2023-10-06T05:01:22.909492+0000","last_peered":"2023-10-06T05:01:22.909492+0000","last_clean":"2023-10-06T05:01:22.909492+0000","last_became_active":"2023-10-06T05:01:22.908678+0000","last_became_peered":"2023-10-06T05:01:22.908678+0000","last_unstale":"2023-10-06T05:01:22.909492+0000","last_undegraded":"2023-10-06T05:01:22.909492+0000","last_fullsized":"2023-10-06T05:01:22.909492+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:24.372776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.899848+0000","last_change":"2023-10-06T05:01:21.894870+0000","last_active":"2023-10-06T05:01:22.899848+0000","last_peered":"2023-10-06T05:01:22.899848+0000","last_clean":"2023-10-06T05:01:22.899848+0000","last_became_active":"2023-10-06T05:01:21.893907+0000","last_became_peered":"2023-10-06T05:01:21.893907+0000","last_unstale":"2023-10-06T05:01:22.899848+0000","last_undegraded":"2023-10-06T05:01:22.899848+0000","last_fullsized":"2023-10-06T05:01:22.899848+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920281+0000","last_change":"2023-10-06T05:01:22.910581+0000","last_active":"2023-10-06T05:01:22.920281+0000","last_peered":"2023-10-06T05:01:22.920281+0000","last_clean":"2023-10-06T05:01:22.920281+0000","last_became_active":"2023-10-06T05:01:22.909963+0000","last_became_peered":"2023-10-06T05:01:22.909963+0000","last_unstale":"2023-10-06T05:01:22.920281+0000","last_undegraded":"2023-10-06T05:01:22.920281+0000","last_fullsized":"2023-10-06T05:01:22.920281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:41.453873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.893893+0000","last_change":"2023-10-06T05:01:21.900893+0000","last_active":"2023-10-06T05:01:22.893893+0000","last_peered":"2023-10-06T05:01:22.893893+0000","last_clean":"2023-10-06T05:01:22.893893+0000","last_became_active":"2023-10-06T05:01:21.900591+0000","last_became_peered":"2023-10-06T05:01:21.900591+0000","last_unstale":"2023-10-06T05:01:22.893893+0000","last_undegraded":"2023-10-06T05:01:22.893893+0000","last_fullsized":"2023-10-06T05:01:22.893893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:52.694935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917896+0000","last_change":"2023-10-06T05:01:22.917013+0000","last_active":"2023-10-06T05:01:22.917896+0000","last_peered":"2023-10-06T05:01:22.917896+0000","last_clean":"2023-10-06T05:01:22.917896+0000","last_became_active":"2023-10-06T05:01:22.915500+0000","last_became_peered":"2023-10-06T05:01:22.915500+0000","last_unstale":"2023-10-06T05:01:22.917896+0000","last_undegraded":"2023-10-06T05:01:22.917896+0000","last_fullsized":"2023-10-06T05:01:22.917896+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:07.724399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.914695+0000","last_change":"2023-10-06T05:01:21.897131+0000","last_active":"2023-10-06T05:01:22.914695+0000","last_peered":"2023-10-06T05:01:22.914695+0000","last_clean":"2023-10-06T05:01:22.914695+0000","last_became_active":"2023-10-06T05:01:21.896847+0000","last_became_peered":"2023-10-06T05:01:21.896847+0000","last_unstale":"2023-10-06T05:01:22.914695+0000","last_undegraded":"2023-10-06T05:01:22.914695+0000","last_fullsized":"2023-10-06T05:01:22.914695+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912743+0000","last_change":"2023-10-06T05:01:22.907709+0000","last_active":"2023-10-06T05:01:22.912743+0000","last_peered":"2023-10-06T05:01:22.912743+0000","last_clean":"2023-10-06T05:01:22.912743+0000","last_became_active":"2023-10-06T05:01:22.907411+0000","last_became_peered":"2023-10-06T05:01:22.907411+0000","last_unstale":"2023-10-06T05:01:22.912743+0000","last_undegraded":"2023-10-06T05:01:22.912743+0000","last_fullsized":"2023-10-06T05:01:22.912743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:19.960229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921946+0000","last_change":"2023-10-06T05:01:21.900714+0000","last_active":"2023-10-06T05:01:22.921946+0000","last_peered":"2023-10-06T05:01:22.921946+0000","last_clean":"2023-10-06T05:01:22.921946+0000","last_became_active":"2023-10-06T05:01:21.900456+0000","last_became_peered":"2023-10-06T05:01:21.900456+0000","last_unstale":"2023-10-06T05:01:22.921946+0000","last_undegraded":"2023-10-06T05:01:22.921946+0000","last_fullsized":"2023-10-06T05:01:22.921946+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:10.205802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.916048+0000","last_change":"2023-10-06T05:01:22.913467+0000","last_active":"2023-10-06T05:01:22.916048+0000","last_peered":"2023-10-06T05:01:22.916048+0000","last_clean":"2023-10-06T05:01:22.916048+0000","last_became_active":"2023-10-06T05:01:22.913133+0000","last_became_peered":"2023-10-06T05:01:22.913133+0000","last_unstale":"2023-10-06T05:01:22.916048+0000","last_undegraded":"2023-10-06T05:01:22.916048+0000","last_fullsized":"2023-10-06T05:01:22.916048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:41.139543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"132'1","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-06T05:01:24.444545+0000","last_change":"2023-10-06T05:01:21.907168+0000","last_active":"2023-10-06T05:01:24.444545+0000","last_peered":"2023-10-06T05:01:24.444545+0000","last_clean":"2023-10-06T05:01:24.444545+0000","last_became_active":"2023-10-06T05:01:21.906798+0000","last_became_peered":"2023-10-06T05:01:21.906798+0000","last_unstale":"2023-10-06T05:01:24.444545+0000","last_undegraded":"2023-10-06T05:01:24.444545+0000","last_fullsized":"2023-10-06T05:01:24.444545+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:59.730017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.919507+0000","last_change":"2023-10-06T05:01:22.917117+0000","last_active":"2023-10-06T05:01:22.919507+0000","last_peered":"2023-10-06T05:01:22.919507+0000","last_clean":"2023-10-06T05:01:22.919507+0000","last_became_active":"2023-10-06T05:01:22.916829+0000","last_became_peered":"2023-10-06T05:01:22.916829+0000","last_unstale":"2023-10-06T05:01:22.919507+0000","last_undegraded":"2023-10-06T05:01:22.919507+0000","last_fullsized":"2023-10-06T05:01:22.919507+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:22.941337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.896584+0000","last_change":"2023-10-06T05:01:21.897514+0000","last_active":"2023-10-06T05:01:22.896584+0000","last_peered":"2023-10-06T05:01:22.896584+0000","last_clean":"2023-10-06T05:01:22.896584+0000","last_became_active":"2023-10-06T05:01:21.897226+0000","last_became_peered":"2023-10-06T05:01:21.897226+0000","last_unstale":"2023-10-06T05:01:22.896584+0000","last_undegraded":"2023-10-06T05:01:22.896584+0000","last_fullsized":"2023-10-06T05:01:22.896584+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:44.413476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.917870+0000","last_change":"2023-10-06T05:01:22.916955+0000","last_active":"2023-10-06T05:01:22.917870+0000","last_peered":"2023-10-06T05:01:22.917870+0000","last_clean":"2023-10-06T05:01:22.917870+0000","last_became_active":"2023-10-06T05:01:22.916640+0000","last_became_peered":"2023-10-06T05:01:22.916640+0000","last_unstale":"2023-10-06T05:01:22.917870+0000","last_undegraded":"2023-10-06T05:01:22.917870+0000","last_fullsized":"2023-10-06T05:01:22.917870+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:09.844106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891290+0000","last_change":"2023-10-06T05:01:21.906767+0000","last_active":"2023-10-06T05:01:22.891290+0000","last_peered":"2023-10-06T05:01:22.891290+0000","last_clean":"2023-10-06T05:01:22.891290+0000","last_became_active":"2023-10-06T05:01:21.906476+0000","last_became_peered":"2023-10-06T05:01:21.906476+0000","last_unstale":"2023-10-06T05:01:22.891290+0000","last_undegraded":"2023-10-06T05:01:22.891290+0000","last_fullsized":"2023-10-06T05:01:22.891290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921664+0000","last_change":"2023-10-06T05:01:22.917699+0000","last_active":"2023-10-06T05:01:22.921664+0000","last_peered":"2023-10-06T05:01:22.921664+0000","last_clean":"2023-10-06T05:01:22.921664+0000","last_became_active":"2023-10-06T05:01:22.916786+0000","last_became_peered":"2023-10-06T05:01:22.916786+0000","last_unstale":"2023-10-06T05:01:22.921664+0000","last_undegraded":"2023-10-06T05:01:22.921664+0000","last_fullsized":"2023-10-06T05:01:22.921664+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:44.900882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900665+0000","last_change":"2023-10-06T05:01:21.910269+0000","last_active":"2023-10-06T05:01:22.900665+0000","last_peered":"2023-10-06T05:01:22.900665+0000","last_clean":"2023-10-06T05:01:22.900665+0000","last_became_active":"2023-10-06T05:01:21.909971+0000","last_became_peered":"2023-10-06T05:01:21.909971+0000","last_unstale":"2023-10-06T05:01:22.900665+0000","last_undegraded":"2023-10-06T05:01:22.900665+0000","last_fullsized":"2023-10-06T05:01:22.900665+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:08.494393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920563+0000","last_change":"2023-10-06T05:01:22.919007+0000","last_active":"2023-10-06T05:01:22.920563+0000","last_peered":"2023-10-06T05:01:22.920563+0000","last_clean":"2023-10-06T05:01:22.920563+0000","last_became_active":"2023-10-06T05:01:22.911208+0000","last_became_peered":"2023-10-06T05:01:22.911208+0000","last_unstale":"2023-10-06T05:01:22.920563+0000","last_undegraded":"2023-10-06T05:01:22.920563+0000","last_fullsized":"2023-10-06T05:01:22.920563+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:08.817032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.894440+0000","last_change":"2023-10-06T05:01:21.902093+0000","last_active":"2023-10-06T05:01:22.894440+0000","last_peered":"2023-10-06T05:01:22.894440+0000","last_clean":"2023-10-06T05:01:22.894440+0000","last_became_active":"2023-10-06T05:01:21.900927+0000","last_became_peered":"2023-10-06T05:01:21.900927+0000","last_unstale":"2023-10-06T05:01:22.894440+0000","last_undegraded":"2023-10-06T05:01:22.894440+0000","last_fullsized":"2023-10-06T05:01:22.894440+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:16.639594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918752+0000","last_change":"2023-10-06T05:01:22.917989+0000","last_active":"2023-10-06T05:01:22.918752+0000","last_peered":"2023-10-06T05:01:22.918752+0000","last_clean":"2023-10-06T05:01:22.918752+0000","last_became_active":"2023-10-06T05:01:22.917665+0000","last_became_peered":"2023-10-06T05:01:22.917665+0000","last_unstale":"2023-10-06T05:01:22.918752+0000","last_undegraded":"2023-10-06T05:01:22.918752+0000","last_fullsized":"2023-10-06T05:01:22.918752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:10.275597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900723+0000","last_change":"2023-10-06T05:01:21.908132+0000","last_active":"2023-10-06T05:01:22.900723+0000","last_peered":"2023-10-06T05:01:22.900723+0000","last_clean":"2023-10-06T05:01:22.900723+0000","last_became_active":"2023-10-06T05:01:21.907727+0000","last_became_peered":"2023-10-06T05:01:21.907727+0000","last_unstale":"2023-10-06T05:01:22.900723+0000","last_undegraded":"2023-10-06T05:01:22.900723+0000","last_fullsized":"2023-10-06T05:01:22.900723+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:19.166059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912985+0000","last_change":"2023-10-06T05:01:22.911853+0000","last_active":"2023-10-06T05:01:22.912985+0000","last_peered":"2023-10-06T05:01:22.912985+0000","last_clean":"2023-10-06T05:01:22.912985+0000","last_became_active":"2023-10-06T05:01:22.911562+0000","last_became_peered":"2023-10-06T05:01:22.911562+0000","last_unstale":"2023-10-06T05:01:22.912985+0000","last_undegraded":"2023-10-06T05:01:22.912985+0000","last_fullsized":"2023-10-06T05:01:22.912985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:03.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898029+0000","last_change":"2023-10-06T05:01:21.908600+0000","last_active":"2023-10-06T05:01:22.898029+0000","last_peered":"2023-10-06T05:01:22.898029+0000","last_clean":"2023-10-06T05:01:22.898029+0000","last_became_active":"2023-10-06T05:01:21.908261+0000","last_became_peered":"2023-10-06T05:01:21.908261+0000","last_unstale":"2023-10-06T05:01:22.898029+0000","last_undegraded":"2023-10-06T05:01:22.898029+0000","last_fullsized":"2023-10-06T05:01:22.898029+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:23.366677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910946+0000","last_change":"2023-10-06T05:01:22.906979+0000","last_active":"2023-10-06T05:01:22.910946+0000","last_peered":"2023-10-06T05:01:22.910946+0000","last_clean":"2023-10-06T05:01:22.910946+0000","last_became_active":"2023-10-06T05:01:22.906652+0000","last_became_peered":"2023-10-06T05:01:22.906652+0000","last_unstale":"2023-10-06T05:01:22.910946+0000","last_undegraded":"2023-10-06T05:01:22.910946+0000","last_fullsized":"2023-10-06T05:01:22.910946+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:42.125134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.899072+0000","last_change":"2023-10-06T05:01:21.897963+0000","last_active":"2023-10-06T05:01:22.899072+0000","last_peered":"2023-10-06T05:01:22.899072+0000","last_clean":"2023-10-06T05:01:22.899072+0000","last_became_active":"2023-10-06T05:01:21.897684+0000","last_became_peered":"2023-10-06T05:01:21.897684+0000","last_unstale":"2023-10-06T05:01:22.899072+0000","last_undegraded":"2023-10-06T05:01:22.899072+0000","last_fullsized":"2023-10-06T05:01:22.899072+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:11.842466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.904603+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:01:22.904603+0000","last_peered":"2023-10-06T05:01:22.904603+0000","last_clean":"2023-10-06T05:01:22.904603+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:01:22.904603+0000","last_undegraded":"2023-10-06T05:01:22.904603+0000","last_fullsized":"2023-10-06T05:01:22.904603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912616+0000","last_change":"2023-10-06T05:01:22.911622+0000","last_active":"2023-10-06T05:01:22.912616+0000","last_peered":"2023-10-06T05:01:22.912616+0000","last_clean":"2023-10-06T05:01:22.912616+0000","last_became_active":"2023-10-06T05:01:22.911388+0000","last_became_peered":"2023-10-06T05:01:22.911388+0000","last_unstale":"2023-10-06T05:01:22.912616+0000","last_undegraded":"2023-10-06T05:01:22.912616+0000","last_fullsized":"2023-10-06T05:01:22.912616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:23.004036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.920959+0000","last_change":"2023-10-06T05:01:21.907272+0000","last_active":"2023-10-06T05:01:22.920959+0000","last_peered":"2023-10-06T05:01:22.920959+0000","last_clean":"2023-10-06T05:01:22.920959+0000","last_became_active":"2023-10-06T05:01:21.906971+0000","last_became_peered":"2023-10-06T05:01:21.906971+0000","last_unstale":"2023-10-06T05:01:22.920959+0000","last_undegraded":"2023-10-06T05:01:22.920959+0000","last_fullsized":"2023-10-06T05:01:22.920959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:31.899599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.891246+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:01:22.891246+0000","last_peered":"2023-10-06T05:01:22.891246+0000","last_clean":"2023-10-06T05:01:22.891246+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:01:22.891246+0000","last_undegraded":"2023-10-06T05:01:22.891246+0000","last_fullsized":"2023-10-06T05:01:22.891246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912780+0000","last_change":"2023-10-06T05:01:22.911349+0000","last_active":"2023-10-06T05:01:22.912780+0000","last_peered":"2023-10-06T05:01:22.912780+0000","last_clean":"2023-10-06T05:01:22.912780+0000","last_became_active":"2023-10-06T05:01:22.906441+0000","last_became_peered":"2023-10-06T05:01:22.906441+0000","last_unstale":"2023-10-06T05:01:22.912780+0000","last_undegraded":"2023-10-06T05:01:22.912780+0000","last_fullsized":"2023-10-06T05:01:22.912780+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:29.391201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.902904+0000","last_change":"2023-10-06T05:01:21.905784+0000","last_active":"2023-10-06T05:01:22.902904+0000","last_peered":"2023-10-06T05:01:22.902904+0000","last_clean":"2023-10-06T05:01:22.902904+0000","last_became_active":"2023-10-06T05:01:21.905522+0000","last_became_peered":"2023-10-06T05:01:21.905522+0000","last_unstale":"2023-10-06T05:01:22.902904+0000","last_undegraded":"2023-10-06T05:01:22.902904+0000","last_fullsized":"2023-10-06T05:01:22.902904+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:31.320712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":107,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897470+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:01:22.897470+0000","last_peered":"2023-10-06T05:01:22.897470+0000","last_clean":"2023-10-06T05:01:22.897470+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:01:22.897470+0000","last_undegraded":"2023-10-06T05:01:22.897470+0000","last_fullsized":"2023-10-06T05:01:22.897470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":254,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898054+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:01:22.898054+0000","last_peered":"2023-10-06T05:01:22.898054+0000","last_clean":"2023-10-06T05:01:22.898054+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:01:22.898054+0000","last_undegraded":"2023-10-06T05:01:22.898054+0000","last_fullsized":"2023-10-06T05:01:22.898054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.915904+0000","last_change":"2023-10-06T05:01:22.914069+0000","last_active":"2023-10-06T05:01:22.915904+0000","last_peered":"2023-10-06T05:01:22.915904+0000","last_clean":"2023-10-06T05:01:22.915904+0000","last_became_active":"2023-10-06T05:01:22.913789+0000","last_became_peered":"2023-10-06T05:01:22.913789+0000","last_unstale":"2023-10-06T05:01:22.915904+0000","last_undegraded":"2023-10-06T05:01:22.915904+0000","last_fullsized":"2023-10-06T05:01:22.915904+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:37.179875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900942+0000","last_change":"2023-10-06T05:01:21.907735+0000","last_active":"2023-10-06T05:01:22.900942+0000","last_peered":"2023-10-06T05:01:22.900942+0000","last_clean":"2023-10-06T05:01:22.900942+0000","last_became_active":"2023-10-06T05:01:21.907455+0000","last_became_peered":"2023-10-06T05:01:21.907455+0000","last_unstale":"2023-10-06T05:01:22.900942+0000","last_undegraded":"2023-10-06T05:01:22.900942+0000","last_fullsized":"2023-10-06T05:01:22.900942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:50.355734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.897481+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:01:22.897481+0000","last_peered":"2023-10-06T05:01:22.897481+0000","last_clean":"2023-10-06T05:01:22.897481+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:01:22.897481+0000","last_undegraded":"2023-10-06T05:01:22.897481+0000","last_fullsized":"2023-10-06T05:01:22.897481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912546+0000","last_change":"2023-10-06T05:01:22.904787+0000","last_active":"2023-10-06T05:01:22.912546+0000","last_peered":"2023-10-06T05:01:22.912546+0000","last_clean":"2023-10-06T05:01:22.912546+0000","last_became_active":"2023-10-06T05:01:22.904208+0000","last_became_peered":"2023-10-06T05:01:22.904208+0000","last_unstale":"2023-10-06T05:01:22.912546+0000","last_undegraded":"2023-10-06T05:01:22.912546+0000","last_fullsized":"2023-10-06T05:01:22.912546+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:10.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.925929+0000","last_change":"2023-10-06T05:01:21.900909+0000","last_active":"2023-10-06T05:01:22.925929+0000","last_peered":"2023-10-06T05:01:22.925929+0000","last_clean":"2023-10-06T05:01:22.925929+0000","last_became_active":"2023-10-06T05:01:21.900571+0000","last_became_peered":"2023-10-06T05:01:21.900571+0000","last_unstale":"2023-10-06T05:01:22.925929+0000","last_undegraded":"2023-10-06T05:01:22.925929+0000","last_fullsized":"2023-10-06T05:01:22.925929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:55.134979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.898113+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:01:22.898113+0000","last_peered":"2023-10-06T05:01:22.898113+0000","last_clean":"2023-10-06T05:01:22.898113+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:01:22.898113+0000","last_undegraded":"2023-10-06T05:01:22.898113+0000","last_fullsized":"2023-10-06T05:01:22.898113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.912543+0000","last_change":"2023-10-06T05:01:22.910384+0000","last_active":"2023-10-06T05:01:22.912543+0000","last_peered":"2023-10-06T05:01:22.912543+0000","last_clean":"2023-10-06T05:01:22.912543+0000","last_became_active":"2023-10-06T05:01:22.909894+0000","last_became_peered":"2023-10-06T05:01:22.909894+0000","last_unstale":"2023-10-06T05:01:22.912543+0000","last_undegraded":"2023-10-06T05:01:22.912543+0000","last_fullsized":"2023-10-06T05:01:22.912543+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:03.895804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909184+0000","last_change":"2023-10-06T05:01:21.898111+0000","last_active":"2023-10-06T05:01:22.909184+0000","last_peered":"2023-10-06T05:01:22.909184+0000","last_clean":"2023-10-06T05:01:22.909184+0000","last_became_active":"2023-10-06T05:01:21.897816+0000","last_became_peered":"2023-10-06T05:01:21.897816+0000","last_unstale":"2023-10-06T05:01:22.909184+0000","last_undegraded":"2023-10-06T05:01:22.909184+0000","last_fullsized":"2023-10-06T05:01:22.909184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:05.782643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900952+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:01:22.900952+0000","last_peered":"2023-10-06T05:01:22.900952+0000","last_clean":"2023-10-06T05:01:22.900952+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:01:22.900952+0000","last_undegraded":"2023-10-06T05:01:22.900952+0000","last_fullsized":"2023-10-06T05:01:22.900952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.921053+0000","last_change":"2023-10-06T05:01:22.920569+0000","last_active":"2023-10-06T05:01:22.921053+0000","last_peered":"2023-10-06T05:01:22.921053+0000","last_clean":"2023-10-06T05:01:22.921053+0000","last_became_active":"2023-10-06T05:01:22.920240+0000","last_became_peered":"2023-10-06T05:01:22.920240+0000","last_unstale":"2023-10-06T05:01:22.921053+0000","last_undegraded":"2023-10-06T05:01:22.921053+0000","last_fullsized":"2023-10-06T05:01:22.921053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:17.199908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.906141+0000","last_change":"2023-10-06T05:01:21.901057+0000","last_active":"2023-10-06T05:01:22.906141+0000","last_peered":"2023-10-06T05:01:22.906141+0000","last_clean":"2023-10-06T05:01:22.906141+0000","last_became_active":"2023-10-06T05:01:21.900783+0000","last_became_peered":"2023-10-06T05:01:21.900783+0000","last_unstale":"2023-10-06T05:01:22.906141+0000","last_undegraded":"2023-10-06T05:01:22.906141+0000","last_fullsized":"2023-10-06T05:01:22.906141+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:13.427002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.900563+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:01:22.900563+0000","last_peered":"2023-10-06T05:01:22.900563+0000","last_clean":"2023-10-06T05:01:22.900563+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:01:22.900563+0000","last_undegraded":"2023-10-06T05:01:22.900563+0000","last_fullsized":"2023-10-06T05:01:22.900563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.918799+0000","last_change":"2023-10-06T05:01:22.915800+0000","last_active":"2023-10-06T05:01:22.918799+0000","last_peered":"2023-10-06T05:01:22.918799+0000","last_clean":"2023-10-06T05:01:22.918799+0000","last_became_active":"2023-10-06T05:01:22.915546+0000","last_became_peered":"2023-10-06T05:01:22.915546+0000","last_unstale":"2023-10-06T05:01:22.918799+0000","last_undegraded":"2023-10-06T05:01:22.918799+0000","last_fullsized":"2023-10-06T05:01:22.918799+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:21.877530+0000","last_clean_scrub_stamp":"2023-10-06T05:01:21.877530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.909478+0000","last_change":"2023-10-06T05:01:21.905554+0000","last_active":"2023-10-06T05:01:22.909478+0000","last_peered":"2023-10-06T05:01:22.909478+0000","last_clean":"2023-10-06T05:01:22.909478+0000","last_became_active":"2023-10-06T05:01:21.905222+0000","last_became_peered":"2023-10-06T05:01:21.905222+0000","last_unstale":"2023-10-06T05:01:22.909478+0000","last_undegraded":"2023-10-06T05:01:22.909478+0000","last_fullsized":"2023-10-06T05:01:22.909478+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:01:20.871431+0000","last_clean_scrub_stamp":"2023-10-06T05:01:20.871431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:45.381801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-06T05:01:22.910017+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:01:22.910017+0000","last_peered":"2023-10-06T05:01:22.910017+0000","last_clean":"2023-10-06T05:01:22.910017+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:01:22.910017+0000","last_undegraded":"2023-10-06T05:01:22.910017+0000","last_fullsized":"2023-10-06T05:01:22.910017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.68300000000000005},"min":{"1min":0.378,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":7.1779999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.71599999999999997},"min":{"1min":0.42499999999999999,"5min":0.38,"15min":0.311},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":7.1879999999999997},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.78400000000000003,"15min":1.073},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71899999999999997,"5min":0.80800000000000005,"15min":1.1379999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.56000000000000005},"min":{"1min":0.40100000000000002,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.77600000000000002,"15min":1.1240000000000001},"last":0.502},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.89300000000000002,"15min":1.111},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.79400000000000004,"15min":1.0880000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.38200000000000001},"max":{"1min":0.67300000000000004,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.72999999999999998,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.53800000000000003},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.66800000000000004,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.81499999999999995,"15min":0.874},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.61799999999999999},"min":{"1min":0.39000000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":1.214},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1658880,"data_stored":753838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58199999999999996,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.20699999999999999},"max":{"1min":0.70899999999999996,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.46400000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":2.8140000000000001},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":1.093,"5min":1.093,"15min":4.3559999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.76700000000000002,"5min":0.83099999999999996,"15min":1.234},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.68799999999999994,"5min":0.77200000000000002,"15min":5.7080000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":2.8650000000000002},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.35099999999999998},"max":{"1min":1.238,"5min":1.238,"15min":4.2569999999999997},"last":0.439},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":1.175,"5min":1.175,"15min":4.3949999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47399999999999998,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.60099999999999998,"5min":0.78200000000000003,"15min":3.669},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":1.147,"5min":1.147,"15min":2.9900000000000002},"last":0.505}]},{"osd":5,"last update":"Fri Oct 6 05:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.50600000000000001,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.27500000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":4.4370000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.76800000000000002,"5min":0.85899999999999999,"15min":2.9510000000000001},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 05:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":2.9049999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.46800000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.57299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307676,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064164,"statfs":{"total":96636764160,"available":96321703936,"internally_reserved":0,"allocated":2248704,"data_stored":1342853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35599999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.437,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.378,"15min":0.378},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.58199999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.60499999999999998,"5min":0.68600000000000005,"15min":0.69399999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.622,"15min":0.70399999999999996},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.56000000000000005,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.47199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.77300000000000002,"5min":0.877,"15min":1.0780000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.998,"15min":0.998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"max":{"1min":0.81999999999999995,"5min":0.94599999999999995,"15min":1.0049999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.98999999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.54400000000000004,"5min":0.496,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":1.022,"15min":1.0429999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":755982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.59199999999999997,"5min":0.628,"15min":0.628},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42999999999999999,"15min":0.38800000000000001},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46100000000000002,"15min":0.443},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.86299999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.98299999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.85799999999999998,"15min":0.92300000000000004},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.91700000000000004},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.93400000000000005,"15min":1.0109999999999999},"last":0.96199999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.52200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.97599999999999998,"15min":1.0329999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":0.995},"last":0.77500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":753880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.20300000000000001,"15min":0.186},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.048},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.39300000000000002},"min":{"1min":0.32300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.216},"max":{"1min":0.52400000000000002,"5min":0.68200000000000005,"15min":1.1819999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.219},"max":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.38600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.312},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":1.2070000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.88600000000000001,"15min":1.246},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.36499999999999999},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.90200000000000002},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.2649999999999999},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.88100000000000001,"15min":0.93100000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":0.88300000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.82499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307104,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064736,"statfs":{"total":96636764160,"available":96322289664,"internally_reserved":0,"allocated":1662976,"data_stored":754356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38800000000000001,"15min":0.39900000000000002},"min":{"1min":0.22,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":2.008},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.51500000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26200000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":6.4370000000000003},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.39900000000000002,"15min":0.371},"min":{"1min":0.252,"5min":0.224,"15min":0.219},"max":{"1min":3.1539999999999999,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.38},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.43099999999999999,"15min":0.44900000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.601,"5min":1.601,"15min":2.1970000000000001},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.51000000000000001},"min":{"1min":0.30299999999999999,"5min":0.30099999999999999,"15min":0.255},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":5.069},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.442,"15min":0.47099999999999997},"min":{"1min":0.33200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64100000000000001,"5min":0.68400000000000005,"15min":2.246},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61199999999999999,"15min":0.75},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":14.646000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.625,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.109,"5min":3.109,"15min":14.606999999999999},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":1.506,"5min":1.506,"15min":2.2959999999999998},"last":0.623},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.65100000000000002,"15min":0.73699999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34899999999999998},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":6.4009999999999998},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":0.80900000000000005,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":16.773,"5min":16.773,"15min":16.773},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.77500000000000002,"15min":0.70899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40100000000000002},"max":{"1min":11.436,"5min":11.436,"15min":11.436},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64000000000000001,"15min":0.746},"min":{"1min":0.45900000000000002,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":1.579,"5min":1.579,"15min":6.367},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67600000000000005,"15min":0.72099999999999997},"min":{"1min":0.502,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":3.069,"5min":3.069,"15min":5},"last":0.874}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307108,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064732,"statfs":{"total":96636764160,"available":96322285568,"internally_reserved":0,"allocated":1667072,"data_stored":755304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80300000000000005,"15min":0.95799999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.70899999999999996},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.73799999999999999},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.79400000000000004,"15min":5.5510000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.74099999999999999},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":5.4900000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.628},"min":{"1min":0.433,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.68200000000000005,"5min":0.75900000000000001,"15min":4.0739999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53200000000000003,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":5.415},"last":0.39200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.83199999999999996,"15min":4.202},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.73199999999999998,"5min":0.81000000000000005,"15min":4.2130000000000001},"last":0.623}]},{"osd":5,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46700000000000003,"15min":0.49399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.73399999999999999,"15min":1.1279999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.73399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51400000000000001,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.73199999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.99099999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.79500000000000004,"5min":0.86099999999999999,"15min":1.1000000000000001},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307092,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064748,"statfs":{"total":96636764160,"available":96322301952,"internally_reserved":0,"allocated":1650688,"data_stored":752415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53700000000000003,"15min":0.63200000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":0.71399999999999997,"5min":0.78500000000000003,"15min":4.9039999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56200000000000006,"15min":0.60299999999999998},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.75800000000000001,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56899999999999995,"15min":0.66300000000000003},"min":{"1min":0.311,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.873,"15min":6.7949999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55200000000000005,"15min":0.63200000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.35999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56200000000000006,"15min":0.69899999999999995},"min":{"1min":0.35399999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67300000000000004,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.435},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55600000000000005,"15min":0.624},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":0.66600000000000004,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58399999999999996,"15min":0.82799999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.30199999999999999},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.748,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.505}]},{"osd":4,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47799999999999998,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.71899999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.55800000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80700000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.747,"5min":0.873,"15min":0.873},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:01:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.60299999999999998},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86699999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.63},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.628},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83499999999999996,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.60599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:01:31.976 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:01:31.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:01:32.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.133+0000 7f0510fe6700 1 Processor -- start 2023-10-06T05:01:32.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.133+0000 7f0510fe6700 1 -- start start 2023-10-06T05:01:32.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f0510fe6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c0cba50 0x7f050c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f0510fe6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f050c15d540 0x7f050c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f0510fe6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f050c15a290 con 0x7f050c150690 2023-10-06T05:01:32.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f0510fe6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050c0d0020 con 0x7f050c15d540 2023-10-06T05:01:32.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f0510fe6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c159f40 con 0x7f050c0cba50 2023-10-06T05:01:32.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f050c15d540 0x7f050c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f050c150690 0x7f050c15a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40226/0 (socket says 172.21.15.3:40226) 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050a59c700 1 -- 172.21.15.3:0/3252606126 learned_addr learned my addr 172.21.15.3:0/3252606126 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f0509d9b700 1 --2- 172.21.15.3:0/3252606126 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c0cba50 0x7f050c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050ad9d700 1 -- 172.21.15.3:0/3252606126 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c0cba50 msgr2=0x7f050c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050ad9d700 1 --2- 172.21.15.3:0/3252606126 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c0cba50 0x7f050c150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050ad9d700 1 -- 172.21.15.3:0/3252606126 >> v1:172.21.15.3:6789/0 conn(0x7f050c150690 legacy=0x7f050c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050ad9d700 1 -- 172.21.15.3:0/3252606126 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c15dcb0 con 0x7f050c15d540 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.134+0000 7f050959a700 1 -- 172.21.15.3:0/3252606126 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283404227 0 0) 0x7f050c15a290 con 0x7f050c150690 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f050ad9d700 1 --2- 172.21.15.3:0/3252606126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c15d540 0x7f050c0af190 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0098e0 tx=0x7f04fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747b070ca02d2ed6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f050959a700 1 -- 172.21.15.3:0/3252606126 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc009e80 con 0x7f050c15d540 2023-10-06T05:01:32.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f050959a700 1 -- 172.21.15.3:0/3252606126 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc00b860 con 0x7f050c15d540 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f050959a700 1 -- 172.21.15.3:0/3252606126 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00bbe0 con 0x7f050c15d540 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 -- 172.21.15.3:0/3252606126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c15d540 msgr2=0x7f050c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3252606126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c15d540 0x7f050c0af190 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0098e0 tx=0x7f04fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 -- 172.21.15.3:0/3252606126 shutdown_connections 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3252606126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c15d540 0x7f050c0af190 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3252606126 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c0cba50 0x7f050c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 -- 172.21.15.3:0/3252606126 >> 172.21.15.3:0/3252606126 conn(0x7f050c070310 msgr2=0x7f050c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 -- 172.21.15.3:0/3252606126 shutdown_connections 2023-10-06T05:01:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.135+0000 7f0510fe6700 1 -- 172.21.15.3:0/3252606126 wait complete. 2023-10-06T05:01:32.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.136+0000 7f0510fe6700 1 Processor -- start 2023-10-06T05:01:32.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.136+0000 7f0510fe6700 1 -- start start 2023-10-06T05:01:32.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f0510fe6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c0cba50 0x7f050c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f0510fe6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f050c150690 0x7f050c169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f0510fe6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c15d540 0x7f050c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f0510fe6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c150510 con 0x7f050c0cba50 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f0510fe6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c1530d0 con 0x7f050c150690 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f0510fe6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c16a340 con 0x7f050c15d540 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f050a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c0cba50 0x7f050c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f050a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c0cba50 0x7f050c14ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50548/0 (socket says 172.21.15.3:50548) 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f050a59c700 1 -- 172.21.15.3:0/3838299644 learned_addr learned my addr 172.21.15.3:0/3838299644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f050ad9d700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c15d540 0x7f050c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.137+0000 7f0509d9b700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f050c150690 0x7f050c169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f050a59c700 1 -- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c15d540 msgr2=0x7f050c0c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f050a59c700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c15d540 0x7f050c0c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f050a59c700 1 -- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f050c150690 msgr2=0x7f050c169e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f050a59c700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f050c150690 0x7f050c169e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f050a59c700 1 -- 172.21.15.3:0/3838299644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc009420 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f0509d9b700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f050c150690 0x7f050c169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f050a59c700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c0cba50 0x7f050c14ffd0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f050400cd80 tx=0x7f0504001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f05027fc700 1 -- 172.21.15.3:0/3838299644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050400ab10 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f05027fc700 1 -- 172.21.15.3:0/3838299644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050400ac70 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c151ec0 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f05027fc700 1 -- 172.21.15.3:0/3838299644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05040070f0 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.138+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c152480 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.139+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c04c380 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.139+0000 7f05027fc700 1 -- 172.21.15.3:0/3838299644 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05040074a0 con 0x7f050c0cba50 2023-10-06T05:01:32.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.140+0000 7f05027fc700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f04ec066f30 0x7f04ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.140+0000 7f05027fc700 1 -- 172.21.15.3:0/3838299644 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0504084c50 con 0x7f050c0cba50 2023-10-06T05:01:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.140+0000 7f0509d9b700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f04ec066f30 0x7f04ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.141+0000 7f0509d9b700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f04ec066f30 0x7f04ec0693e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f050c071290 tx=0x7f04f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:32.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.144+0000 7f05027fc700 1 -- 172.21.15.3:0/3838299644 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0504051d40 con 0x7f050c0cba50 2023-10-06T05:01:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.417+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f050c0b44a0 con 0x7f050c0cba50 2023-10-06T05:01:32.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.421+0000 7f05027fc700 1 -- 172.21.15.3:0/3838299644 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0504055360 con 0x7f050c0cba50 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f04ec066f30 msgr2=0x7f04ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f04ec066f30 0x7f04ec0693e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f050c071290 tx=0x7f04f4008040 comp rx=0 tx=0).stop 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c0cba50 msgr2=0x7f050c14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c0cba50 0x7f050c14ffd0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f050400cd80 tx=0x7f0504001ae0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 shutdown_connections 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f04ec066f30 0x7f04ec0693e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f050c15d540 0x7f050c0c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f050c0cba50 0x7f050c14ffd0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 --2- 172.21.15.3:0/3838299644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f050c150690 0x7f050c169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 >> 172.21.15.3:0/3838299644 conn(0x7f050c070310 msgr2=0x7f050c0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:32.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 shutdown_connections 2023-10-06T05:01:32.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.423+0000 7f0510fe6700 1 -- 172.21.15.3:0/3838299644 wait complete. 2023-10-06T05:01:32.426 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-10-06T05:01:32.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.554+0000 7f81d105f700 1 Processor -- start 2023-10-06T05:01:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.554+0000 7f81d105f700 1 -- start start 2023-10-06T05:01:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81d105f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 0x7f81cc0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81d105f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc0b0ce0 con 0x7f81cc15cb50 2023-10-06T05:01:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81d105f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc0cb820 con 0x7f81cc0b2740 2023-10-06T05:01:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81d105f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0b0760 con 0x7f81cc0cd950 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81cad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f81cc15cb50 0x7f81cc0b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42874/0 (socket says 172.21.15.3:42874) 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81cad9d700 1 -- 172.21.15.3:0/806133191 learned_addr learned my addr 172.21.15.3:0/806133191 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81ca59c700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 0x7f81cc0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371523056 0 0) 0x7f81cc0b0ce0 con 0x7f81cc15cb50 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b8003570 con 0x7f81cc15cb50 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3098594283 0 0) 0x7f81b8003570 con 0x7f81cc15cb50 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 msgr2=0x7f81cc0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 0x7f81cc0b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 >> v1:172.21.15.153:6789/0 conn(0x7f81cc0b2740 legacy=0x7f81cc0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0b0a00 con 0x7f81cc15cb50 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f81bc0015f0 con 0x7f81cc15cb50 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 >> v1:172.21.15.3:6789/0 conn(0x7f81cc15cb50 legacy=0x7f81cc0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b8003a90 0x7f81b8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 0x7f81b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0b0760 con 0x7f81b8004810 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b800bb40 con 0x7f81b8003a90 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b800bd00 con 0x7f81cc0cd950 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.555+0000 7f81ca59c700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81cad9d700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b8003a90 0x7f81b8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81cb59e700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 0x7f81b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81ca59c700 1 -- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 msgr2=0x7f81b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81ca59c700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 0x7f81b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81ca59c700 1 -- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b8003a90 msgr2=0x7f81b8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81ca59c700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b8003a90 0x7f81b8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81ca59c700 1 -- 172.21.15.3:0/806133191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b8003570 con 0x7f81b8004810 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81cb59e700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0cd950 0x7f81b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:32.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81ca59c700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81b8004c80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cdd40 tx=0x7f81b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decfac9d2d5ff311 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b400e710 con 0x7f81b8004810 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b400e870 con 0x7f81b8004810 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81c9d9b700 1 -- 172.21.15.3:0/806133191 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b400ec30 con 0x7f81b8004810 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81d105f700 1 -- 172.21.15.3:0/806133191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 msgr2=0x7f81b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.556+0000 7f81d105f700 1 --2- 172.21.15.3:0/806133191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81b8004c80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cdd40 tx=0x7f81b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.557+0000 7f81cb59e700 1 -- 172.21.15.3:0/806133191 reap_dead start 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.557+0000 7f81d105f700 1 -- 172.21.15.3:0/806133191 shutdown_connections 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.557+0000 7f81d105f700 1 -- 172.21.15.3:0/806133191 >> 172.21.15.3:0/806133191 conn(0x7f81cc070310 msgr2=0x7f81cc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.557+0000 7f81d105f700 1 -- 172.21.15.3:0/806133191 shutdown_connections 2023-10-06T05:01:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.557+0000 7f81d105f700 1 -- 172.21.15.3:0/806133191 wait complete. 2023-10-06T05:01:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.558+0000 7f81d105f700 1 Processor -- start 2023-10-06T05:01:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.558+0000 7f81d105f700 1 -- start start 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81cc15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81cc0d06b0 0x7f81cc0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0a6f00 0x7f81cc0a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0d1060 con 0x7f81b8004810 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0ce010 con 0x7f81cc0d06b0 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0a7c30 con 0x7f81cc0a6f00 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81cc15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81cc15ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42704/0 (socket says 172.21.15.3:42704) 2023-10-06T05:01:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 -- 172.21.15.3:0/394246716 learned_addr learned my addr 172.21.15.3:0/394246716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 -- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0a6f00 msgr2=0x7f81cc0a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0a6f00 0x7f81cc0a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 -- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81cc0d06b0 msgr2=0x7f81cc0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81cc0d06b0 0x7f81cc0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 -- 172.21.15.3:0/394246716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0a7f10 con 0x7f81b8004810 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81cad9d700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81cc15ae60 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00f9d0 tx=0x7f81bc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81c2ffd700 1 -- 172.21.15.3:0/394246716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0072e0 con 0x7f81b8004810 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81c2ffd700 1 -- 172.21.15.3:0/394246716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81bc007440 con 0x7f81b8004810 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc0a81f0 con 0x7f81b8004810 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81c2ffd700 1 -- 172.21.15.3:0/394246716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0057f0 con 0x7f81b8004810 2023-10-06T05:01:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.559+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc15b3a0 con 0x7f81b8004810 2023-10-06T05:01:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.560+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc071680 con 0x7f81b8004810 2023-10-06T05:01:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.561+0000 7f81c2ffd700 1 -- 172.21.15.3:0/394246716 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f81bc005bc0 con 0x7f81b8004810 2023-10-06T05:01:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.561+0000 7f81c2ffd700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f81ac066f30 0x7f81ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.561+0000 7f81c2ffd700 1 -- 172.21.15.3:0/394246716 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f81bc084970 con 0x7f81b8004810 2023-10-06T05:01:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.561+0000 7f81ca59c700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f81ac066f30 0x7f81ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.562+0000 7f81ca59c700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f81ac066f30 0x7f81ac0693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cd2c0 tx=0x7f81b401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.566+0000 7f81c2ffd700 1 -- 172.21.15.3:0/394246716 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81bc051a80 con 0x7f81b8004810 2023-10-06T05:01:32.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.804+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81cc0cb1d0 con 0x7f81b8004810 2023-10-06T05:01:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.804+0000 7f81c2ffd700 1 -- 172.21.15.3:0/394246716 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f81bc0550a0 con 0x7f81b8004810 2023-10-06T05:01:32.806 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:32.806 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:01:25.910940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:01:32.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f81ac066f30 msgr2=0x7f81ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f81ac066f30 0x7f81ac0693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cd2c0 tx=0x7f81b401a000 comp rx=0 tx=0).stop 2023-10-06T05:01:32.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 msgr2=0x7f81cc15ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:32.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81cc15ae60 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00f9d0 tx=0x7f81bc009040 comp rx=0 tx=0).stop 2023-10-06T05:01:32.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 shutdown_connections 2023-10-06T05:01:32.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f81ac066f30 0x7f81ac0693e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81cc0a6f00 0x7f81cc0a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81b8004810 0x7f81cc15ae60 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 --2- 172.21.15.3:0/394246716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81cc0d06b0 0x7f81cc0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 >> 172.21.15.3:0/394246716 conn(0x7f81cc070310 msgr2=0x7f81cc15bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 shutdown_connections 2023-10-06T05:01:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:32.806+0000 7f81d105f700 1 -- 172.21.15.3:0/394246716 wait complete. 2023-10-06T05:01:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 89 2023-10-06T05:01:33.222 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:01:22.299089+0000', 'modified': '2023-10-06T05:01:25.910940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8409}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8409': {'gid': 8409, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6837/2550905871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6835', 'nonce': 2550905871}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 2550905871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:01:33.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:01:33.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:01:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.343+0000 7f26fb373700 1 Processor -- start 2023-10-06T05:01:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.344+0000 7f26fb373700 1 -- start start 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.344+0000 7f26fb373700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.344+0000 7f26fb373700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26f41598d0 0x7f26f40cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.344+0000 7f26fb373700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f40cc5c0 con 0x7f26f40b2220 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.344+0000 7f26fb373700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f40cb7b0 con 0x7f26f41598d0 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.344+0000 7f26fb373700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40cda60 con 0x7f26f4159500 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.344+0000 7f26f890e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26f41598d0 0x7f26f40cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f890e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26f41598d0 0x7f26f40cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42714/0 (socket says 172.21.15.3:42714) 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f890e700 1 -- 172.21.15.3:0/3511507651 learned_addr learned my addr 172.21.15.3:0/3511507651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f910f700 1 --2- 172.21.15.3:0/3511507651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f890e700 1 -- 172.21.15.3:0/3511507651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 msgr2=0x7f26f40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f890e700 1 --2- 172.21.15.3:0/3511507651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f890e700 1 -- 172.21.15.3:0/3511507651 >> v1:172.21.15.153:6789/0 conn(0x7f26f40b2220 legacy=0x7f26f40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f890e700 1 -- 172.21.15.3:0/3511507651 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40cb9b0 con 0x7f26f41598d0 2023-10-06T05:01:33.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f890e700 1 --2- 172.21.15.3:0/3511507651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f41598d0 0x7f26f40cb270 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f26e80099b0 tx=0x7f26e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1642a21a8839543 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f3fff700 1 -- 172.21.15.3:0/3511507651 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8009ea0 con 0x7f26f41598d0 2023-10-06T05:01:33.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f3fff700 1 -- 172.21.15.3:0/3511507651 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800b860 con 0x7f26f41598d0 2023-10-06T05:01:33.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.345+0000 7f26f3fff700 1 -- 172.21.15.3:0/3511507651 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800bc00 con 0x7f26f41598d0 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 -- 172.21.15.3:0/3511507651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f41598d0 msgr2=0x7f26f40cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 --2- 172.21.15.3:0/3511507651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f41598d0 0x7f26f40cb270 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f26e80099b0 tx=0x7f26e8009250 comp rx=0 tx=0).stop 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 -- 172.21.15.3:0/3511507651 shutdown_connections 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 --2- 172.21.15.3:0/3511507651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 --2- 172.21.15.3:0/3511507651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f41598d0 0x7f26f40cb270 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 -- 172.21.15.3:0/3511507651 >> 172.21.15.3:0/3511507651 conn(0x7f26f4070310 msgr2=0x7f26f4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 -- 172.21.15.3:0/3511507651 shutdown_connections 2023-10-06T05:01:33.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.346+0000 7f26fb373700 1 -- 172.21.15.3:0/3511507651 wait complete. 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.347+0000 7f26fb373700 1 Processor -- start 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.347+0000 7f26fb373700 1 -- start start 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26fb373700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f40b2220 0x7f26f41558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26fb373700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26fb373700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f41598d0 0x7f26f41584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26fb373700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f416b0c0 con 0x7f26f40b2220 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26fb373700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4169eb0 con 0x7f26f41598d0 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26fb373700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f416ad10 con 0x7f26f4159500 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f40b2220 0x7f26f41558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f40b2220 0x7f26f41558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42728/0 (socket says 172.21.15.3:42728) 2023-10-06T05:01:33.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 -- 172.21.15.3:0/2006810891 learned_addr learned my addr 172.21.15.3:0/2006810891 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f890e700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f9910700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f41598d0 0x7f26f41584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 -- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 msgr2=0x7f26f40b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 -- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f41598d0 msgr2=0x7f26f41584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f41598d0 0x7f26f41584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 -- 172.21.15.3:0/2006810891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80094f0 con 0x7f26f40b2220 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f9910700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f41598d0 0x7f26f41584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.348+0000 7f26f910f700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f40b2220 0x7f26f41558a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009ef0 tx=0x7f26e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.349+0000 7f26f17fa700 1 -- 172.21.15.3:0/2006810891 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e400cc20 con 0x7f26f40b2220 2023-10-06T05:01:33.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.349+0000 7f26f17fa700 1 -- 172.21.15.3:0/2006810891 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e400f040 con 0x7f26f40b2220 2023-10-06T05:01:33.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.349+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f416a100 con 0x7f26f40b2220 2023-10-06T05:01:33.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.349+0000 7f26f17fa700 1 -- 172.21.15.3:0/2006810891 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4014480 con 0x7f26f40b2220 2023-10-06T05:01:33.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.349+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f416a740 con 0x7f26f40b2220 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.350+0000 7f26f17fa700 1 -- 172.21.15.3:0/2006810891 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26e40147f0 con 0x7f26f40b2220 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.350+0000 7f26f17fa700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26e0066f30 0x7f26e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.351+0000 7f26f17fa700 1 -- 172.21.15.3:0/2006810891 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26e4085080 con 0x7f26f40b2220 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.351+0000 7f26f890e700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26e0066f30 0x7f26e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.351+0000 7f26f890e700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26e0066f30 0x7f26e00693e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f26e80099b0 tx=0x7f26e8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.352+0000 7f270466bb80 1 Processor -- start 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.352+0000 7f270466bb80 1 -- start start 2023-10-06T05:01:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbf9d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf98c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f270466bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf98cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f270466bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf90cc80 con 0x55dcbf98dbd0 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf8e3a70 con 0x55dcbf9d3a90 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf8e1270 con 0x55dcbf98f9a0 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f26f9910700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf98cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f26f9910700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf98cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42734/0 (socket says 172.21.15.3:42734) 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f26f9910700 1 -- 172.21.15.3:0/2998470646 learned_addr learned my addr 172.21.15.3:0/2998470646 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f26f890e700 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf98c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f26f910f700 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbf9d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f26f9910700 1 -- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 msgr2=0x55dcbf98c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.353+0000 7f26f9910700 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf98c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26f9910700 1 -- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 msgr2=0x55dcbf9d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26f9910700 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbf9d3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26f9910700 1 -- 172.21.15.3:0/2998470646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcbf777270 con 0x55dcbf98dbd0 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26f910f700 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbf9d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26f890e700 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf98c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26f9910700 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf98cac0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0016d0 tx=0x7f26ec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b12325a43074b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26da7fc700 1 -- 172.21.15.3:0/2998470646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0011d0 con 0x55dcbf98dbd0 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26da7fc700 1 -- 172.21.15.3:0/2998470646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec00e600 con 0x55dcbf98dbd0 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f26da7fc700 1 -- 172.21.15.3:0/2998470646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00e980 con 0x55dcbf98dbd0 2023-10-06T05:01:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 -- 172.21.15.3:0/2998470646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 msgr2=0x55dcbf98cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf98cac0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0016d0 tx=0x7f26ec009040 comp rx=0 tx=0).stop 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 -- 172.21.15.3:0/2998470646 shutdown_connections 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbf9d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf98cac0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 --2- 172.21.15.3:0/2998470646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf98c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 -- 172.21.15.3:0/2998470646 >> 172.21.15.3:0/2998470646 conn(0x55dcbf98e070 msgr2=0x55dcbf98e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 -- 172.21.15.3:0/2998470646 shutdown_connections 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.354+0000 7f270466bb80 1 -- 172.21.15.3:0/2998470646 wait complete. 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.355+0000 7f270466bb80 1 -- start start 2023-10-06T05:01:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.355+0000 7f270466bb80 1 Processor -- start 2023-10-06T05:01:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f270466bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f270466bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf8dd060 con 0x55dcbf98dbd0 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf888890 con 0x55dcbf98f9a0 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf8a3ea0 con 0x55dcbf9d3a90 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f270466bb80 10 client.0 _ll_register_callbacks cb 0x55dcbf779f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42738/0 (socket says 172.21.15.3:42738) 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 -- 172.21.15.3:0/2692904802 learned_addr learned my addr 172.21.15.3:0/2692904802 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f890e700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f9910700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 -- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 msgr2=0x55dcbf9b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 -- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 msgr2=0x55dcbf9b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f910f700 1 -- 172.21.15.3:0/2692904802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec00f040 con 0x55dcbf98dbd0 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f9910700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.356+0000 7f26f890e700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26f910f700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9b9250 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f26e4055de0 tx=0x7f26e405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc00415939872287 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26d37fe700 1 -- 172.21.15.3:0/2692904802 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4059100 con 0x55dcbf98dbd0 2023-10-06T05:01:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26d37fe700 1 -- 172.21.15.3:0/2692904802 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26e405cdf0 con 0x55dcbf98dbd0 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcbf981690 con 0x55dcbf98dbd0 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26d37fe700 1 -- 172.21.15.3:0/2692904802 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e4083170 con 0x55dcbf98dbd0 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 10 client.8583 fetch_fsmap learned FSMap version 89 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55dcbf719fe0 con 0x55dcbf98dbd0 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26d37fe700 1 -- 172.21.15.3:0/2692904802 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dcbf981690 con 0x55dcbf98dbd0 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26d37fe700 1 -- 172.21.15.3:0/2692904802 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 89) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f26e4055340 con 0x55dcbf98dbd0 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 10 client.8583 fetch_fsmap finished waiting for FSMap version 89 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 10 client.8583 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 20 client.8583 populate_metadata read hostname 'smithi003' 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9cffb0 0x55dcbf9d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 4 client.8583 mds_command: new command op to 8409 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 --> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcbf95e5c0 con 0x55dcbf9cffb0 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.357+0000 7f26f890e700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9cffb0 0x55dcbf9d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.358+0000 7f26f890e700 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9cffb0 0x55dcbf9d0380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73bdad3584ae8e7 server_cookie=fa1b0e37241c553a in_seq=0 out_seq=0 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.358+0000 7f26d37fe700 10 client.8583 ms_handle_connect on v2:172.21.15.153:6835/2550905871 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.358+0000 7f26d37fe700 1 -- 172.21.15.3:0/2692904802 <== mds.0 v2:172.21.15.153:6835/2550905871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55dcbf95e5c0 con 0x55dcbf9cffb0 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.358+0000 7f26d37fe700 10 client.8583 handle_command_reply: tid=0 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 client.8583 shutdown 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 msgr2=0x55dcbf9b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9b9250 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f26e4055de0 tx=0x7f26e405b040 comp rx=0 tx=0).stop 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 shutdown_connections 2023-10-06T05:01:33.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9cffb0 0x55dcbf9d0380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f26d37fe700 0 client.8583 ms_handle_reset on v2:172.21.15.153:6835/2550905871 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9b9250 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 --2- 172.21.15.3:0/2692904802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 >> 172.21.15.3:0/2692904802 conn(0x55dcbf98e070 msgr2=0x55dcbf9d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 shutdown_connections 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 1 -- 172.21.15.3:0/2692904802 wait complete. 2023-10-06T05:01:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.359+0000 7f270466bb80 20 client.8583 trim_cache size 0 max 16384 2023-10-06T05:01:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.376+0000 7f270466bb80 1 Processor -- start 2023-10-06T05:01:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.376+0000 7f270466bb80 1 -- start start 2023-10-06T05:01:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f270466bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f270466bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf75fe30 con 0x55dcbf98dbd0 2023-10-06T05:01:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf9b7810 con 0x55dcbf98f9a0 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf985190 con 0x55dcbf9d3a90 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42754/0 (socket says 172.21.15.3:42754) 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 -- 172.21.15.3:0/476059170 learned_addr learned my addr 172.21.15.3:0/476059170 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f9910700 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f890e700 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 -- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 msgr2=0x55dcbf9897f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9897f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 -- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 msgr2=0x55dcbf9d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 -- 172.21.15.3:0/476059170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcbf8ab780 con 0x55dcbf98dbd0 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.377+0000 7f26f910f700 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9d2580 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000f80 tx=0x7f26e406d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186f83d020b0267d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f26daffd700 1 -- 172.21.15.3:0/476059170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4083170 con 0x55dcbf98dbd0 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f26daffd700 1 -- 172.21.15.3:0/476059170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e406c070 con 0x55dcbf98dbd0 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f26daffd700 1 -- 172.21.15.3:0/476059170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4059100 con 0x55dcbf98dbd0 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 -- 172.21.15.3:0/476059170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 msgr2=0x55dcbf9d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9d2580 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000f80 tx=0x7f26e406d000 comp rx=0 tx=0).stop 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 -- 172.21.15.3:0/476059170 shutdown_connections 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf9d3a90 0x55dcbf9897f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9d2580 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 --2- 172.21.15.3:0/476059170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf98f9a0 0x55dcbf9d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 -- 172.21.15.3:0/476059170 >> 172.21.15.3:0/476059170 conn(0x55dcbf98e070 msgr2=0x55dcbf9d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 -- 172.21.15.3:0/476059170 shutdown_connections 2023-10-06T05:01:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.378+0000 7f270466bb80 1 -- 172.21.15.3:0/476059170 wait complete. 2023-10-06T05:01:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.379+0000 7f270466bb80 1 -- start start 2023-10-06T05:01:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.379+0000 7f270466bb80 1 Processor -- start 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.379+0000 7f270466bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.379+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf9ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbfa382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbf9d1120 con 0x55dcbf98dbd0 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbfa3c490 con 0x55dcbf9d3a90 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbfa3b880 con 0x55dcbf98f9a0 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 10 client.0 _ll_register_callbacks cb 0x55dcbf779f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9ce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42764/0 (socket says 172.21.15.3:42764) 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 -- 172.21.15.3:0/44108147 learned_addr learned my addr 172.21.15.3:0/44108147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f9910700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbfa382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f890e700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf9ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 -- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 msgr2=0x55dcbf9ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf9ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 -- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 msgr2=0x55dcbfa382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbfa382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 -- 172.21.15.3:0/44108147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcbf719fe0 con 0x55dcbf98dbd0 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f890e700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf9ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26f910f700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9ce400 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f26e4052870 tx=0x7f26e4080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59ea915d0663be9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-06T05:01:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26d37fe700 1 -- 172.21.15.3:0/44108147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4083170 con 0x55dcbf98dbd0 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26d37fe700 1 -- 172.21.15.3:0/44108147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26e405cdf0 con 0x55dcbf98dbd0 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcbf87fe00 con 0x55dcbf98dbd0 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26d37fe700 1 -- 172.21.15.3:0/44108147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e406c070 con 0x55dcbf98dbd0 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 10 client.8589 fetch_fsmap learned FSMap version 89 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55dcbf892470 con 0x55dcbf98dbd0 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.380+0000 7f26d37fe700 1 -- 172.21.15.3:0/44108147 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e4075080 con 0x55dcbf98dbd0 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f26d37fe700 1 -- 172.21.15.3:0/44108147 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 89) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f26e407db80 con 0x55dcbf98dbd0 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f270466bb80 10 client.8589 fetch_fsmap finished waiting for FSMap version 89 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f270466bb80 10 client.8589 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f270466bb80 20 client.8589 populate_metadata read hostname 'smithi003' 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f270466bb80 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9b40f0 0x55dcbfa38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f270466bb80 4 client.8589 mds_command: new command op to 8409 tid=0[{"prefix": "status"}] 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 --> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dcbf87fb20 con 0x55dcbf9b40f0 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f26f890e700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9b40f0 0x55dcbfa38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f26d37fe700 10 client.8589 ms_handle_connect on v2:172.21.15.153:6835/2550905871 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f26f890e700 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9b40f0 0x55dcbfa38800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2b35dee10f6011 server_cookie=f9163cc7618b79ce in_seq=0 out_seq=0 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f26d37fe700 1 -- 172.21.15.3:0/44108147 <== mds.0 v2:172.21.15.153:6835/2550905871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55dcbf87fb20 con 0x55dcbf9b40f0 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f26d37fe700 10 client.8589 handle_command_reply: tid=0 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.381+0000 7f270466bb80 1 client.8589 shutdown 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 msgr2=0x55dcbf9ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9ce400 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f26e4052870 tx=0x7f26e4080000 comp rx=0 tx=0).stop 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 shutdown_connections 2023-10-06T05:01:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:6835/2550905871,v1:172.21.15.153:6837/2550905871] conn(0x55dcbf9b40f0 0x55dcbfa38800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcbf9d3a90 0x55dcbfa382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dcbf98dbd0 0x55dcbf9ce400 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26d37fe700 0 client.8589 ms_handle_reset on v2:172.21.15.153:6835/2550905871 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 --2- 172.21.15.3:0/44108147 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcbf98f9a0 0x55dcbf9ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 >> 172.21.15.3:0/44108147 conn(0x55dcbf98e070 msgr2=0x55dcbf9897f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 shutdown_connections 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 1 -- 172.21.15.3:0/44108147 wait complete. 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f270466bb80 20 client.8589 trim_cache size 0 max 16384 2023-10-06T05:01:33.391 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "fa7cd5af-16f2-4a73-ab30-42f10cc13e22", 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 8409, 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.494963628000001, 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 89, 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 132, 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 132, 2023-10-06T05:01:33.392 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.563486151999999 2023-10-06T05:01:33.393 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:01:33.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26e0066f30 msgr2=0x7f26e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26e0066f30 0x7f26e00693e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f26e80099b0 tx=0x7f26e8017000 comp rx=0 tx=0).stop 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f40b2220 msgr2=0x7f26f41558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f40b2220 0x7f26f41558a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009ef0 tx=0x7f26e400b040 comp rx=0 tx=0).stop 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 shutdown_connections 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f26e0066f30 0x7f26e00693e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26f41598d0 0x7f26f41584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26f40b2220 0x7f26f41558a0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 --2- 172.21.15.3:0/2006810891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26f4159500 0x7f26f40b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 >> 172.21.15.3:0/2006810891 conn(0x7f26f4070310 msgr2=0x7f26f40cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 shutdown_connections 2023-10-06T05:01:33.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.382+0000 7f26fb373700 1 -- 172.21.15.3:0/2006810891 wait complete. 2023-10-06T05:01:33.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:01:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.514+0000 7f9afea51700 1 Processor -- start 2023-10-06T05:01:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.514+0000 7f9afea51700 1 -- start start 2023-10-06T05:01:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.514+0000 7f9afea51700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9af815cb50 0x7f9af80b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.514+0000 7f9afea51700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80b0ce0 con 0x7f9af80cd950 2023-10-06T05:01:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.514+0000 7f9afea51700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80cb820 con 0x7f9af80b0590 2023-10-06T05:01:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.514+0000 7f9afea51700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80b0a00 con 0x7f9af815cb50 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9af815cb50 0x7f9af80b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9afcfee700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9af80cd950 0x7f9af80cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42880/0 (socket says 172.21.15.3:42880) 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9afcfee700 1 -- 172.21.15.3:0/2513536781 learned_addr learned my addr 172.21.15.3:0/2513536781 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af7fff700 1 -- 172.21.15.3:0/2513536781 >> v1:172.21.15.153:6789/0 conn(0x7f9af80b0590 legacy=0x7f9af80b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af7fff700 1 -- 172.21.15.3:0/2513536781 >> v1:172.21.15.3:6789/0 conn(0x7f9af80cd950 legacy=0x7f9af80cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af7fff700 1 -- 172.21.15.3:0/2513536781 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80cb1d0 con 0x7f9af815cb50 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af7fff700 1 --2- 172.21.15.3:0/2513536781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af815cb50 0x7f9af80b0050 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80098d0 tx=0x7f9ae8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c537359342631046 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af6ffd700 1 -- 172.21.15.3:0/2513536781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8009e70 con 0x7f9af815cb50 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af6ffd700 1 -- 172.21.15.3:0/2513536781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8005c10 con 0x7f9af815cb50 2023-10-06T05:01:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.515+0000 7f9af6ffd700 1 -- 172.21.15.3:0/2513536781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae800b650 con 0x7f9af815cb50 2023-10-06T05:01:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.516+0000 7f9afea51700 1 -- 172.21.15.3:0/2513536781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af815cb50 msgr2=0x7f9af80b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.516+0000 7f9afea51700 1 --2- 172.21.15.3:0/2513536781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af815cb50 0x7f9af80b0050 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80098d0 tx=0x7f9ae8009120 comp rx=0 tx=0).stop 2023-10-06T05:01:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.516+0000 7f9afea51700 1 -- 172.21.15.3:0/2513536781 shutdown_connections 2023-10-06T05:01:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.516+0000 7f9afea51700 1 --2- 172.21.15.3:0/2513536781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af815cb50 0x7f9af80b0050 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.516+0000 7f9afea51700 1 -- 172.21.15.3:0/2513536781 >> 172.21.15.3:0/2513536781 conn(0x7f9af8070310 msgr2=0x7f9af8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.516+0000 7f9afea51700 1 -- 172.21.15.3:0/2513536781 shutdown_connections 2023-10-06T05:01:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.516+0000 7f9afea51700 1 -- 172.21.15.3:0/2513536781 wait complete. 2023-10-06T05:01:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.517+0000 7f9afea51700 1 Processor -- start 2023-10-06T05:01:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.517+0000 7f9afea51700 1 -- start start 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9afea51700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af80b0590 0x7f9af8156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9afea51700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af80cd950 0x7f9af80cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9afea51700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9af815cb50 0x7f9af81525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9afea51700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af816a190 con 0x7f9af80cd950 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9afea51700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80cfba0 con 0x7f9af80b0590 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9afea51700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8169b10 con 0x7f9af815cb50 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af80cd950 0x7f9af80cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af80cd950 0x7f9af80cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42788/0 (socket says 172.21.15.3:42788) 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 -- 172.21.15.3:0/2333900230 learned_addr learned my addr 172.21.15.3:0/2333900230 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9afcfee700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9af815cb50 0x7f9af81525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af7fff700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af80b0590 0x7f9af8156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 -- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9af815cb50 msgr2=0x7f9af81525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9af815cb50 0x7f9af81525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 -- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af80b0590 msgr2=0x7f9af8156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af80b0590 0x7f9af8156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 -- 172.21.15.3:0/2333900230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80093c0 con 0x7f9af80cd950 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.518+0000 7f9af77fe700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af80cd950 0x7f9af80cf660 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c7c0 tx=0x7f9aec007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.519+0000 7f9adffff700 1 -- 172.21.15.3:0/2333900230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec011590 con 0x7f9af80cd950 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.519+0000 7f9adffff700 1 -- 172.21.15.3:0/2333900230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec0116f0 con 0x7f9af80cd950 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.519+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af80cfdd0 con 0x7f9af80cd950 2023-10-06T05:01:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.519+0000 7f9adffff700 1 -- 172.21.15.3:0/2333900230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec011ab0 con 0x7f9af80cd950 2023-10-06T05:01:33.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.519+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af8152b20 con 0x7f9af80cd950 2023-10-06T05:01:33.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.520+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af816a2d0 con 0x7f9af80cd950 2023-10-06T05:01:33.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.520+0000 7f9adffff700 1 -- 172.21.15.3:0/2333900230 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9aec01b3b0 con 0x7f9af80cd950 2023-10-06T05:01:33.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.520+0000 7f9adffff700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ae4066f30 0x7f9ae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:33.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.521+0000 7f9adffff700 1 -- 172.21.15.3:0/2333900230 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9aec083a60 con 0x7f9af80cd950 2023-10-06T05:01:33.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.521+0000 7f9af7fff700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ae4066f30 0x7f9ae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:33.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.521+0000 7f9af7fff700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ae4066f30 0x7f9ae40693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009160 tx=0x7f9ae8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:33.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.525+0000 7f9adffff700 1 -- 172.21.15.3:0/2333900230 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9af816a2d0 con 0x7f9af80cd950 2023-10-06T05:01:33.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.767+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9af816a2d0 con 0x7f9af80cd950 2023-10-06T05:01:33.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.767+0000 7f9adffff700 1 -- 172.21.15.3:0/2333900230 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9af816a2d0 con 0x7f9af80cd950 2023-10-06T05:01:33.769 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:01:33.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.769+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ae4066f30 msgr2=0x7f9ae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.769+0000 7f9afea51700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ae4066f30 0x7f9ae40693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009160 tx=0x7f9ae8016000 comp rx=0 tx=0).stop 2023-10-06T05:01:33.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.769+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af80cd950 msgr2=0x7f9af80cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:33.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.769+0000 7f9afea51700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af80cd950 0x7f9af80cf660 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c7c0 tx=0x7f9aec007f50 comp rx=0 tx=0).stop 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.769+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 shutdown_connections 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.770+0000 7f9afea51700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f9ae4066f30 0x7f9ae40693e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.770+0000 7f9afea51700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af80b0590 0x7f9af8156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.770+0000 7f9afea51700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9af815cb50 0x7f9af81525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.770+0000 7f9afea51700 1 --2- 172.21.15.3:0/2333900230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9af80cd950 0x7f9af80cf660 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.770+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 >> 172.21.15.3:0/2333900230 conn(0x7f9af8070310 msgr2=0x7f9af80b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.770+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 shutdown_connections 2023-10-06T05:01:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:33.770+0000 7f9afea51700 1 -- 172.21.15.3:0/2333900230 wait complete. 2023-10-06T05:01:33.783 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:01:33.783 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:01:33.783 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T05:01:33.783 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:01:33.784 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:33.784 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:01:33.784 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:01:33.784 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:01:33.784 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:01:33.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:33.784 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T05:01:33.799 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:01:33.799 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:01:33.799 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:01:33.800 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41937sec preferred_lft 41937sec 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:33.801 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T05:01:33.802 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:33.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:33.803 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:01:33.863 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T05:01:33.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:33.864 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T05:01:33.921 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T05:01:33.921 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:01:33.921 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:01:33.921 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:01:33.921 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:01:33.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:33.922 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T05:01:33.922 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T05:01:33.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:01:33.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:01:33.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:01:33.922 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T05:01:34.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:34.023 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T05:01:34.038 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:01:34.038 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:01:34.039 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:01:34.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:34.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T05:01:34.103 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:01:34.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:01:34.103 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:34.158 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:01:34.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:01:34.159 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:34.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:01:34.279 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:01:34.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:01:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:01:34.344 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:01:34.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:34.345 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:01:34.400 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:01:34.401 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:01:34.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:01:34.468 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:01:34.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:01:34.525 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T05:01:34.524+0000 7f245de3a540 -1 init, newargv = 0x5615bc590da0 newargc=15 2023-10-06T05:01:34.525 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[130157]: starting ceph client 2023-10-06T05:01:34.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[130157]: starting fuse 2023-10-06T05:01:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:01:34.536 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:01:34.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:34.536 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:01:34.592 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T05:01:34.592 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:01:34.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:01:34.592 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T05:01:34.593 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:01:34.594 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:34.594 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T05:01:34.594 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T05:01:34.594 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T05:01:34.594 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T05:01:34.594 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:34.594 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:34.595 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T05:01:34.596 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:01:34.716 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.130157.asok 2023-10-06T05:01:34.898 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.130157.asok 2023-10-06T05:01:34.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:34.898 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.130157.asok status 2023-10-06T05:01:35.053 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:01:35.053 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T05:01:35.053 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:01:35.053 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:01:35.053 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "130157", 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 8604, 2023-10-06T05:01:35.054 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 8604 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 562316581 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:35.055 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 562316581 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.8604 192.168.0.1:0/562316581", 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/562316581", 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T05:01:35.056 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 89, 2023-10-06T05:01:35.057 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 132, 2023-10-06T05:01:35.057 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:01:35.057 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T05:01:35.057 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T05:01:35.057 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:01:35.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:35.063 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T05:01:35.064 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:35.079 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T05:01:35.080 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:35.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:35.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:35.145 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:01:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.303+0000 7f31b7876700 1 Processor -- start 2023-10-06T05:01:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.303+0000 7f31b7876700 1 -- start start 2023-10-06T05:01:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.303+0000 7f31b7876700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b01598d0 0x7f31b00b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.303+0000 7f31b7876700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f31b00cb440 0x7f31b00b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.303+0000 7f31b7876700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b00cc5c0 con 0x7f31b0159500 2023-10-06T05:01:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.303+0000 7f31b7876700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b00cda60 con 0x7f31b00cb440 2023-10-06T05:01:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.303+0000 7f31b7876700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00b2ca0 con 0x7f31b01598d0 2023-10-06T05:01:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b4e11700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b01598d0 0x7f31b00b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5e13700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f31b00cb440 0x7f31b00b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5612700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f31b0159500 0x7f31b00cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:39094/0 (socket says 172.21.15.3:39094) 2023-10-06T05:01:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5612700 1 -- 172.21.15.3:0/3895035610 learned_addr learned my addr 172.21.15.3:0/3895035610 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31affff700 1 -- 172.21.15.3:0/3895035610 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401938821 0 0) 0x7f31b00cc5c0 con 0x7f31b0159500 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31affff700 1 -- 172.21.15.3:0/3895035610 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3198003570 con 0x7f31b0159500 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5e13700 1 -- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b01598d0 msgr2=0x7f31b00b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5e13700 1 --2- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b01598d0 0x7f31b00b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5e13700 1 -- 172.21.15.3:0/3895035610 >> v1:172.21.15.153:6789/0 conn(0x7f31b0159500 legacy=0x7f31b00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5e13700 1 -- 172.21.15.3:0/3895035610 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00cb9b0 con 0x7f31b00cb440 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b4e11700 1 --2- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b01598d0 0x7f31b00b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:01:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.304+0000 7f31b5e13700 1 --2- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b00cb440 0x7f31b00b2760 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f31a80098e0 tx=0x7f31a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f82d4ea4ec64fbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31affff700 1 -- 172.21.15.3:0/3895035610 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8011630 con 0x7f31b00cb440 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31affff700 1 -- 172.21.15.3:0/3895035610 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8011790 con 0x7f31b00cb440 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31affff700 1 -- 172.21.15.3:0/3895035610 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8011b30 con 0x7f31b00cb440 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 -- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b00cb440 msgr2=0x7f31b00b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 --2- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b00cb440 0x7f31b00b2760 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f31a80098e0 tx=0x7f31a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 -- 172.21.15.3:0/3895035610 shutdown_connections 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 --2- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b00cb440 0x7f31b00b2760 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 --2- 172.21.15.3:0/3895035610 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b01598d0 0x7f31b00b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 -- 172.21.15.3:0/3895035610 >> 172.21.15.3:0/3895035610 conn(0x7f31b0070310 msgr2=0x7f31b0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 -- 172.21.15.3:0/3895035610 shutdown_connections 2023-10-06T05:01:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.305+0000 7f31b7876700 1 -- 172.21.15.3:0/3895035610 wait complete. 2023-10-06T05:01:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.306+0000 7f31b7876700 1 Processor -- start 2023-10-06T05:01:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.306+0000 7f31b7876700 1 -- start start 2023-10-06T05:01:35.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b7876700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b00cb440 0x7f31b0152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b7876700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b0159500 0x7f31b00b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b7876700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b01598d0 0x7f31b015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b7876700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00b5df0 con 0x7f31b0159500 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b7876700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b015a490 con 0x7f31b01598d0 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b7876700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00b5b10 con 0x7f31b00cb440 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b4e11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b0159500 0x7f31b00b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b4e11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b0159500 0x7f31b00b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42796/0 (socket says 172.21.15.3:42796) 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.307+0000 7f31b4e11700 1 -- 172.21.15.3:0/3744179062 learned_addr learned my addr 172.21.15.3:0/3744179062 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b5612700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b00cb440 0x7f31b0152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b5e13700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b01598d0 0x7f31b015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b4e11700 1 -- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b00cb440 msgr2=0x7f31b0152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b4e11700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b00cb440 0x7f31b0152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b4e11700 1 -- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b01598d0 msgr2=0x7f31b015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b4e11700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b01598d0 0x7f31b015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b4e11700 1 -- 172.21.15.3:0/3744179062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4009420 con 0x7f31b0159500 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b4e11700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b0159500 0x7f31b00b50e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f31a400ec80 tx=0x7f31a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31ad7fa700 1 -- 172.21.15.3:0/3744179062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400c850 con 0x7f31b0159500 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31ad7fa700 1 -- 172.21.15.3:0/3744179062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a400c9b0 con 0x7f31b0159500 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a8009420 con 0x7f31b0159500 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31ad7fa700 1 -- 172.21.15.3:0/3744179062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400cd30 con 0x7f31b0159500 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.308+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b015a120 con 0x7f31b0159500 2023-10-06T05:01:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.309+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b015a640 con 0x7f31b0159500 2023-10-06T05:01:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.310+0000 7f31ad7fa700 1 -- 172.21.15.3:0/3744179062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31a4013680 con 0x7f31b0159500 2023-10-06T05:01:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.310+0000 7f31ad7fa700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f319c066f30 0x7f319c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.310+0000 7f31ad7fa700 1 -- 172.21.15.3:0/3744179062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f31a4084880 con 0x7f31b0159500 2023-10-06T05:01:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.310+0000 7f31b5612700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f319c066f30 0x7f319c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.311+0000 7f31b5612700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f319c066f30 0x7f319c0693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f31a0005c50 tx=0x7f31a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.314+0000 7f31ad7fa700 1 -- 172.21.15.3:0/3744179062 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31b015a640 con 0x7f31b0159500 2023-10-06T05:01:35.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.552+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31b015a600 con 0x7f31b0159500 2023-10-06T05:01:35.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.553+0000 7f31ad7fa700 1 -- 172.21.15.3:0/3744179062 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31a4055420 con 0x7f31b0159500 2023-10-06T05:01:35.554 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:01:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f319c066f30 msgr2=0x7f319c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f319c066f30 0x7f319c0693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f31a0005c50 tx=0x7f31a0009040 comp rx=0 tx=0).stop 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b0159500 msgr2=0x7f31b00b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b0159500 0x7f31b00b50e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f31a400ec80 tx=0x7f31a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 shutdown_connections 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f319c066f30 0x7f319c0693e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31b01598d0 0x7f31b015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31b00cb440 0x7f31b0152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 --2- 172.21.15.3:0/3744179062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b0159500 0x7f31b00b50e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 >> 172.21.15.3:0/3744179062 conn(0x7f31b0070310 msgr2=0x7f31b015db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 shutdown_connections 2023-10-06T05:01:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:35.555+0000 7f31b7876700 1 -- 172.21.15.3:0/3744179062 wait complete. 2023-10-06T05:01:35.569 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:01:35.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:01:35.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-10-06T05:01:35.569 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:01:35.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:01:35.570 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:01:35.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:01:35.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:01:35.570 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:01:35.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:35.571 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-10-06T05:01:35.589 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-10-06T05:01:35.590 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:35.591 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:35.591 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-10-06T05:01:35.591 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:35.591 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:35.591 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:01:35.591 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:35.591 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 41937sec preferred_lft 41937sec 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link noprefixroute 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:35.592 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::1c52:6fff:fee6:7783/64 scope link 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f2:b7:68:76:2e:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f0b7:68ff:fe76:2ef0/64 scope link 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether ca:7e:4a:3f:1c:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:01:35.593 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::c87e:4aff:fe3f:1cc7/64 scope link 2023-10-06T05:01:35.594 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:35.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:35.594 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:01:35.668 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-10-06T05:01:35.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:35.670 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-10-06T05:01:35.728 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-10-06T05:01:35.729 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:01:35.729 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:01:35.729 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:01:35.729 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:01:35.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:35.729 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-10-06T05:01:35.729 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-10-06T05:01:35.729 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:01:35.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:01:35.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:01:35.730 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-10-06T05:01:35.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:35.833 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T05:01:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:01:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:01:35.848 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:01:35.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:35.849 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-10-06T05:01:35.912 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T05:01:35.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:01:35.913 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:01:35.967 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:01:35.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:01:35.968 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:01:36.022 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T05:01:36.087 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:01:36.087 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:01:36.150 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:01:36.151 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:01:36.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:36.152 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T05:01:36.206 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:01:36.206 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T05:01:36.248 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2023-10-06T05:01:36.271 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:01:36.272 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:01:36.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:2023-10-06T05:01:36.327+0000 7f490af9f540 -1 init, newargv = 0x5600227b8670 newargc=15 2023-10-06T05:01:36.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[103188]: starting ceph client 2023-10-06T05:01:36.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[103188]: starting fuse 2023-10-06T05:01:36.338 INFO:teuthology.orchestra.run.smithi153.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:01:36.340 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:01:36.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:36.340 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2023-10-06T05:01:36.395 INFO:teuthology.orchestra.run.smithi153.stdout:46 2023-10-06T05:01:36.396 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> import re 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2023-10-06T05:01:36.396 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2023-10-06T05:01:36.397 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2023-10-06T05:01:36.398 DEBUG:teuthology.orchestra.run.smithi153:> return f 2023-10-06T05:01:36.398 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:01:36.398 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:01:36.398 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.1")) 2023-10-06T05:01:36.398 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T05:01:36.517 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.1.103188.asok 2023-10-06T05:01:36.699 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.103188.asok 2023-10-06T05:01:36.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:36.700 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.103188.asok status 2023-10-06T05:01:36.851 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-10-06T05:01:36.851 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2023-10-06T05:01:36.851 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:01:36.851 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:01:36.851 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "1", 2023-10-06T05:01:36.852 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2023-10-06T05:01:36.852 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T05:01:36.852 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "103188", 2023-10-06T05:01:36.852 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2023-10-06T05:01:36.852 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:01:36.852 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2023-10-06T05:01:36.852 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2023-10-06T05:01:36.853 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 6983, 2023-10-06T05:01:36.853 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2023-10-06T05:01:36.853 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2023-10-06T05:01:36.853 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2023-10-06T05:01:36.853 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 6983 2023-10-06T05:01:36.854 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:01:36.854 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T05:01:36.854 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T05:01:36.854 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:01:36.854 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 3343385700 2023-10-06T05:01:36.854 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-10-06T05:01:36.855 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:01:36.855 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2023-10-06T05:01:36.855 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2023-10-06T05:01:36.855 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:01:36.855 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 3343385700 2023-10-06T05:01:36.855 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.6983 192.168.0.1:0/3343385700", 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/3343385700", 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 89, 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 132, 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2023-10-06T05:01:36.856 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2023-10-06T05:01:36.857 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-10-06T05:01:36.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:36.861 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2023-10-06T05:01:36.861 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T05:01:36.876 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2023-10-06T05:01:36.876 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T05:01:36.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:36.877 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:01:36.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:01:36.945 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:01:36.996 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-06T05:01:36.996 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-06T05:01:36.996 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:01:36.996 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:01:36.997 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:01:36.998 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-06T05:01:36.999 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-06T05:01:37.000 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-06T05:01:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.002 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:01:37.003 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T05:01:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:01:37.005 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.006 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-06T05:01:37.007 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T05:01:37.008 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-10-06T05:01:37.009 INFO:teuthology.orchestra.run.smithi003.stdout: client inject release failure = true 2023-10-06T05:01:37.025 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-10-06T05:01:37.025 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-10-06T05:01:37.025 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:01:37.025 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-10-06T05:01:37.025 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.025 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.026 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:01:37.027 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-10-06T05:01:37.028 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:01:37.029 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:01:37.030 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-10-06T05:01:37.031 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:01:37.032 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:01:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-10-06T05:01:37.034 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:01:37.035 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:01:37.036 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:01:37.037 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2023-10-06T05:01:37.038 INFO:teuthology.orchestra.run.smithi153.stdout: client inject release failure = true 2023-10-06T05:01:37.039 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T05:01:37.039 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:01:37.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:37.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[130157]: fuse finished with error 0 and tester_r 0 2023-10-06T05:01:37.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:01:37.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:37.088 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:01:37.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:01:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.340+0000 7f616f4d6700 1 Processor -- start 2023-10-06T05:01:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.340+0000 7f616f4d6700 1 -- start start 2023-10-06T05:01:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616f4d6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61681594f0 0x7f61680b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616f4d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61681598c0 0x7f61680b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616f4d6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61680cec80 0x7f61680cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616f4d6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61680b4310 con 0x7f61681598c0 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616f4d6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61680d0030 con 0x7f61681594f0 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616f4d6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680cfa70 con 0x7f61680cec80 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616ca71700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61681598c0 0x7f61680b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616ca71700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61681598c0 0x7f61680b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42802/0 (socket says 172.21.15.3:42802) 2023-10-06T05:01:37.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616ca71700 1 -- 172.21.15.3:0/185394251 learned_addr learned my addr 172.21.15.3:0/185394251 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616d272700 1 --2- 172.21.15.3:0/185394251 >> v2:172.21.15.153:3300/0 conn(0x7f61681594f0 0x7f61680b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.341+0000 7f616da73700 1 --2- 172.21.15.3:0/185394251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61680cec80 0x7f61680cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f616ca71700 1 -- 172.21.15.3:0/185394251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61680cec80 msgr2=0x7f61680cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f616ca71700 1 --2- 172.21.15.3:0/185394251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61680cec80 0x7f61680cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f616ca71700 1 -- 172.21.15.3:0/185394251 >> v2:172.21.15.153:3300/0 conn(0x7f61681594f0 msgr2=0x7f61680b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f616ca71700 1 --2- 172.21.15.3:0/185394251 >> v2:172.21.15.153:3300/0 conn(0x7f61681594f0 0x7f61680b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f616ca71700 1 -- 172.21.15.3:0/185394251 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61680cfd50 con 0x7f61681598c0 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f616ca71700 1 --2- 172.21.15.3:0/185394251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61681598c0 0x7f61680b25a0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f61680d16f0 tx=0x7f615c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc1af5542b5ddc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:37.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f6167fff700 1 -- 172.21.15.3:0/185394251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00b640 con 0x7f61681598c0 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f6167fff700 1 -- 172.21.15.3:0/185394251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c00b7a0 con 0x7f61681598c0 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.342+0000 7f6167fff700 1 -- 172.21.15.3:0/185394251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00bb60 con 0x7f61681598c0 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 -- 172.21.15.3:0/185394251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61681598c0 msgr2=0x7f61680b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 --2- 172.21.15.3:0/185394251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61681598c0 0x7f61680b25a0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f61680d16f0 tx=0x7f615c009250 comp rx=0 tx=0).stop 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 -- 172.21.15.3:0/185394251 shutdown_connections 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 --2- 172.21.15.3:0/185394251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61680cec80 0x7f61680cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 --2- 172.21.15.3:0/185394251 >> v2:172.21.15.153:3300/0 conn(0x7f61681594f0 0x7f61680b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 --2- 172.21.15.3:0/185394251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61681598c0 0x7f61680b25a0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 -- 172.21.15.3:0/185394251 >> 172.21.15.3:0/185394251 conn(0x7f616806c2f0 msgr2=0x7f616806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:37.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 -- 172.21.15.3:0/185394251 shutdown_connections 2023-10-06T05:01:37.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.343+0000 7f616f4d6700 1 -- 172.21.15.3:0/185394251 wait complete. 2023-10-06T05:01:37.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.344+0000 7f616f4d6700 1 Processor -- start 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.344+0000 7f616f4d6700 1 -- start start 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616f4d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61680cec80 0x7f616816cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616f4d6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61681594f0 0x7f616815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616f4d6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61681598c0 0x7f61680b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616f4d6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6168139ac0 con 0x7f61680cec80 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616f4d6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6168150b90 con 0x7f61681594f0 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616f4d6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616814c620 con 0x7f61681598c0 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61680cec80 0x7f616816cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:37.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61680cec80 0x7f616816cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42808/0 (socket says 172.21.15.3:42808) 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 -- 172.21.15.3:0/437890475 learned_addr learned my addr 172.21.15.3:0/437890475 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616ca71700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61681594f0 0x7f616815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616da73700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61681598c0 0x7f61680b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 -- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61681598c0 msgr2=0x7f61680b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61681598c0 0x7f61680b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 -- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61681594f0 msgr2=0x7f616815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61681594f0 0x7f616815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 -- 172.21.15.3:0/437890475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c0094f0 con 0x7f61680cec80 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616d272700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61680cec80 0x7f616816cc40 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6158009ef0 tx=0x7f615800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f61657fa700 1 -- 172.21.15.3:0/437890475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800cb70 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f61657fa700 1 -- 172.21.15.3:0/437890475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800f040 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f61657fa700 1 -- 172.21.15.3:0/437890475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158014480 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.345+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6168150df0 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.346+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616816d320 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.346+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6168151140 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.347+0000 7f61657fa700 1 -- 172.21.15.3:0/437890475 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61580147f0 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.347+0000 7f61657fa700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6154066f30 0x7f61540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.347+0000 7f61657fa700 1 -- 172.21.15.3:0/437890475 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6158085060 con 0x7f61680cec80 2023-10-06T05:01:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.347+0000 7f616ca71700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6154066f30 0x7f61540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:37.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.348+0000 7f616ca71700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6154066f30 0x7f61540693e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f61680d16f0 tx=0x7f615c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:37.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.351+0000 7f61657fa700 1 -- 172.21.15.3:0/437890475 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61580520d0 con 0x7f61680cec80 2023-10-06T05:01:37.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.590+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f616804c380 con 0x7f61680cec80 2023-10-06T05:01:37.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.590+0000 7f61657fa700 1 -- 172.21.15.3:0/437890475 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) v1 ==== 53+0+73 (secure 0 0 0) 0x7f61580556f0 con 0x7f61680cec80 2023-10-06T05:01:37.592 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:01:37.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6154066f30 msgr2=0x7f61540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:37.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6154066f30 0x7f61540693e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f61680d16f0 tx=0x7f615c017000 comp rx=0 tx=0).stop 2023-10-06T05:01:37.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61680cec80 msgr2=0x7f616816cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:37.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61680cec80 0x7f616816cc40 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6158009ef0 tx=0x7f615800b040 comp rx=0 tx=0).stop 2023-10-06T05:01:37.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 shutdown_connections 2023-10-06T05:01:37.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6154066f30 0x7f61540693e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61681598c0 0x7f61680b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61680cec80 0x7f616816cc40 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 --2- 172.21.15.3:0/437890475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61681594f0 0x7f616815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:37.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 >> 172.21.15.3:0/437890475 conn(0x7f616806c2f0 msgr2=0x7f61680b5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:37.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 shutdown_connections 2023-10-06T05:01:37.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:37.592+0000 7f616f4d6700 1 -- 172.21.15.3:0/437890475 wait complete. 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:01:37.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:01:37.607 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:01:37.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:37.607 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:37.623 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41933sec preferred_lft 41933sec 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:37.624 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:37.625 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:37.625 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-10-06T05:01:37.625 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:01:37.625 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:37.625 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-10-06T05:01:37.625 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:01:37.625 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:01:37.626 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:01:37.626 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:37.626 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::fcde:a0ff:fe91:fd35/64 scope link 2023-10-06T05:01:37.626 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:37.626 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:01:37.626 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 56:50:b0:b9:c8:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:01:37.627 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5450:b0ff:feb9:c871/64 scope link 2023-10-06T05:01:37.627 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:37.627 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:01:37.627 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:07:6b:2a:5c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:01:37.628 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ac07:6bff:fe2a:5cc1/64 scope link 2023-10-06T05:01:37.628 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:01:37.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:37.628 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:01:37.687 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-06T05:01:37.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:37.689 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-10-06T05:01:37.746 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-10-06T05:01:37.746 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:01:37.746 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:01:37.746 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:01:37.746 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:01:37.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:37.747 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-10-06T05:01:37.747 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-10-06T05:01:37.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:01:37.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:01:37.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:01:37.747 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-10-06T05:01:37.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:37.849 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T05:01:37.864 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:01:37.864 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:01:37.864 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:01:37.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:01:37.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-10-06T05:01:37.930 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:01:37.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:01:37.930 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:37.985 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:01:37.986 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:01:37.986 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:38.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:01:38.106 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:01:38.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:01:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:01:38.171 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:01:38.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:38.172 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:01:38.227 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:01:38.227 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:01:38.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-10-06T05:01:38.294 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:01:38.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:01:38.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2023-10-06T05:01:38.351+0000 7fcb0085f540 -1 init, newargv = 0x561173e20f80 newargc=15 2023-10-06T05:01:38.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[130672]: starting ceph client 2023-10-06T05:01:38.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[130672]: starting fuse 2023-10-06T05:01:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:01:38.362 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:01:38.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:38.363 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-10-06T05:01:38.418 INFO:teuthology.orchestra.run.smithi003.stdout:46 2023-10-06T05:01:38.418 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T05:01:38.419 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:01:38.420 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T05:01:38.421 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T05:01:38.421 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T05:01:38.421 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:01:38.421 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.421 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T05:01:38.421 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:01:38.542 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.130672.asok 2023-10-06T05:01:38.723 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.130672.asok 2023-10-06T05:01:38.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:38.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.130672.asok status 2023-10-06T05:01:38.878 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:01:38.878 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-10-06T05:01:38.878 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:01:38.879 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:01:38.879 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2023-10-06T05:01:38.879 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-10-06T05:01:38.879 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:01:38.879 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "130672", 2023-10-06T05:01:38.880 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-10-06T05:01:38.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:38.880 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-10-06T05:01:38.880 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-10-06T05:01:38.880 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 8625, 2023-10-06T05:01:38.881 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T05:01:38.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T05:01:38.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T05:01:38.881 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 8625 2023-10-06T05:01:38.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:38.881 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:01:38.882 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:01:38.882 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:01:38.882 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3882866003 2023-10-06T05:01:38.882 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:38.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:38.882 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:01:38.882 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3882866003 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.8625 192.168.0.1:0/3882866003", 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3882866003", 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 89, 2023-10-06T05:01:38.883 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 132, 2023-10-06T05:01:38.884 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:01:38.884 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-10-06T05:01:38.884 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-10-06T05:01:38.884 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:01:38.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:38.888 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-10-06T05:01:38.888 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:38.904 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-10-06T05:01:38.904 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:38.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:38.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:01:38.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:01:38.969 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-10-06T05:01:38.970 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2023-10-06T05:01:38.971 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-10-06T05:01:39.092 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.130672.asok 2023-10-06T05:01:39.274 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.130672.asok 2023-10-06T05:01:39.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:39.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.130672.asok mds_sessions 2023-10-06T05:01:39.429 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:01:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 8625, 2023-10-06T05:01:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 8625 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3882866003 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.8625 v1:192.168.0.1:0/3882866003", 2023-10-06T05:01:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/3882866003", 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6835", 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2550905871 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-06T05:01:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.153:6837", 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2550905871 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 0, 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2023-10-06T05:02:39.353679+0000", 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2023-10-06T05:01:39.353679+0000", 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 1, 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2023-10-06T05:01:39.432 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-06T05:01:39.433 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-06T05:01:39.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 89 2023-10-06T05:01:39.433 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:01:39.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:01:39.439 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch file1) 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> import time 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> fd = os.open("/home/ubuntu/cephtest/mnt.1/file1", os.O_RDWR | os.O_CREAT, 0o644) 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> try: 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> while True: 2023-10-06T05:01:39.459 DEBUG:teuthology.orchestra.run.smithi153:> os.write(fd, b'"'"'content'"'"') 2023-10-06T05:01:39.460 DEBUG:teuthology.orchestra.run.smithi153:> time.sleep(1) 2023-10-06T05:01:39.460 DEBUG:teuthology.orchestra.run.smithi153:> if not False: 2023-10-06T05:01:39.460 DEBUG:teuthology.orchestra.run.smithi153:> break 2023-10-06T05:01:39.460 DEBUG:teuthology.orchestra.run.smithi153:> except IOError as e: 2023-10-06T05:01:39.460 DEBUG:teuthology.orchestra.run.smithi153:> pass 2023-10-06T05:01:39.460 DEBUG:teuthology.orchestra.run.smithi153:> os.close(fd) 2023-10-06T05:01:39.460 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-10-06T05:01:39.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:01:39.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.617+0000 7f7133baa700 1 Processor -- start 2023-10-06T05:01:39.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.618+0000 7f7133baa700 1 -- start start 2023-10-06T05:01:39.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.618+0000 7f7133baa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f712c157240 0x7f712c15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:39.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.618+0000 7f7133baa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c15d930 con 0x7f712c068130 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.618+0000 7f7133baa700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c0b0ad0 con 0x7f712c159d00 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.618+0000 7f7133baa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c15e300 con 0x7f712c157240 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7131946700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f712c157240 0x7f712c15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7132147700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f712c068130 0x7f712c06d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42890/0 (socket says 172.21.15.3:42890) 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7132147700 1 -- 172.21.15.3:0/1409408164 learned_addr learned my addr 172.21.15.3:0/1409408164 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7131946700 1 -- 172.21.15.3:0/1409408164 >> v1:172.21.15.153:6789/0 conn(0x7f712c159d00 legacy=0x7f712c067a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7131946700 1 -- 172.21.15.3:0/1409408164 >> v1:172.21.15.3:6789/0 conn(0x7f712c068130 legacy=0x7f712c06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7131946700 1 -- 172.21.15.3:0/1409408164 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c15a8c0 con 0x7f712c157240 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7130944700 1 -- 172.21.15.3:0/1409408164 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871906404 0 0) 0x7f712c15d930 con 0x7f712c068130 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7131946700 1 --2- 172.21.15.3:0/1409408164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c157240 0x7f712c15ddc0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f711c009e60 tx=0x7f711c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffbd7d15e3f692d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:39.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7130944700 1 -- 172.21.15.3:0/1409408164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c00e070 con 0x7f712c157240 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7130944700 1 -- 172.21.15.3:0/1409408164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c00bbf0 con 0x7f712c157240 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.619+0000 7f7130944700 1 -- 172.21.15.3:0/1409408164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c012430 con 0x7f712c157240 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.620+0000 7f7133baa700 1 -- 172.21.15.3:0/1409408164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c157240 msgr2=0x7f712c15ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.620+0000 7f7133baa700 1 --2- 172.21.15.3:0/1409408164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c157240 0x7f712c15ddc0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f711c009e60 tx=0x7f711c009780 comp rx=0 tx=0).stop 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.620+0000 7f7133baa700 1 -- 172.21.15.3:0/1409408164 shutdown_connections 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.620+0000 7f7133baa700 1 --2- 172.21.15.3:0/1409408164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c157240 0x7f712c15ddc0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.620+0000 7f7133baa700 1 -- 172.21.15.3:0/1409408164 >> 172.21.15.3:0/1409408164 conn(0x7f712c0ce3a0 msgr2=0x7f712c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.620+0000 7f7133baa700 1 -- 172.21.15.3:0/1409408164 shutdown_connections 2023-10-06T05:01:39.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.620+0000 7f7133baa700 1 -- 172.21.15.3:0/1409408164 wait complete. 2023-10-06T05:01:39.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.621+0000 7f7133baa700 1 Processor -- start 2023-10-06T05:01:39.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.621+0000 7f7133baa700 1 -- start start 2023-10-06T05:01:39.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7133baa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c068130 0x7f712c0c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:39.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7133baa700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f712c157240 0x7f712c16a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:39.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7133baa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c159d00 0x7f712c0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7133baa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0c2fd0 con 0x7f712c068130 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7133baa700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c16ad20 con 0x7f712c159d00 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7133baa700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0c8da0 con 0x7f712c157240 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c068130 0x7f712c0c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c068130 0x7f712c0c1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42824/0 (socket says 172.21.15.3:42824) 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 -- 172.21.15.3:0/1281434138 learned_addr learned my addr 172.21.15.3:0/1281434138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7132147700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c159d00 0x7f712c0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131145700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f712c157240 0x7f712c16a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 -- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f712c157240 msgr2=0x7f712c16a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f712c157240 0x7f712c16a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:39.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 -- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c159d00 msgr2=0x7f712c0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c159d00 0x7f712c0c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 -- 172.21.15.3:0/1281434138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c009a20 con 0x7f712c068130 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7132147700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c159d00 0x7f712c0c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131145700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f712c157240 0x7f712c16a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7131946700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c068130 0x7f712c0c1de0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f711c00b6c0 tx=0x7f711c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7129ffb700 1 -- 172.21.15.3:0/1281434138 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c012e80 con 0x7f712c068130 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.622+0000 7f7129ffb700 1 -- 172.21.15.3:0/1281434138 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c01e640 con 0x7f712c068130 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.623+0000 7f7129ffb700 1 -- 172.21.15.3:0/1281434138 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c01e9e0 con 0x7f712c068130 2023-10-06T05:01:39.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.623+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c0c23c0 con 0x7f712c068130 2023-10-06T05:01:39.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.623+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c0c2980 con 0x7f712c068130 2023-10-06T05:01:39.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.624+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c04c380 con 0x7f712c068130 2023-10-06T05:01:39.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.625+0000 7f7129ffb700 1 -- 172.21.15.3:0/1281434138 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f711c024070 con 0x7f712c068130 2023-10-06T05:01:39.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.625+0000 7f7129ffb700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:39.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.625+0000 7f7131145700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:39.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.625+0000 7f7129ffb700 1 -- 172.21.15.3:0/1281434138 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f711c057f10 con 0x7f712c068130 2023-10-06T05:01:39.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.626+0000 7f7131145700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7120006fd0 tx=0x7f7120008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.630+0000 7f7129ffb700 1 -- 172.21.15.3:0/1281434138 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f711c054250 con 0x7f712c068130 2023-10-06T05:01:39.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.868+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f712c068630 con 0x7f712c068130 2023-10-06T05:01:39.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.868+0000 7f7129ffb700 1 -- 172.21.15.3:0/1281434138 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f711c057870 con 0x7f712c068130 2023-10-06T05:01:39.870 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:39.870 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:01:25.910940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:01:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7118066f30 msgr2=0x7f71180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7120006fd0 tx=0x7f7120008040 comp rx=0 tx=0).stop 2023-10-06T05:01:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c068130 msgr2=0x7f712c0c1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c068130 0x7f712c0c1de0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f711c00b6c0 tx=0x7f711c018000 comp rx=0 tx=0).stop 2023-10-06T05:01:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 shutdown_connections 2023-10-06T05:01:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c159d00 0x7f712c0c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712c068130 0x7f712c0c1de0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 --2- 172.21.15.3:0/1281434138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f712c157240 0x7f712c16a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 >> 172.21.15.3:0/1281434138 conn(0x7f712c0ce3a0 msgr2=0x7f712c157610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 shutdown_connections 2023-10-06T05:01:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:39.870+0000 7f7133baa700 1 -- 172.21.15.3:0/1281434138 wait complete. 2023-10-06T05:01:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 89 2023-10-06T05:01:39.884 INFO:tasks.ceph_test_case:waiting 70s for health warning matching MDS_CLIENT_LATE_RELEASE 2023-10-06T05:01:39.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:01:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.000+0000 7f39efd7a700 1 Processor -- start 2023-10-06T05:01:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.000+0000 7f39efd7a700 1 -- start start 2023-10-06T05:01:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39efd7a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39e81594d0 0x7f39e80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39efd7a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f39e815da50 0x7f39e815c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39efd7a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e815be00 con 0x7f39e80ceb10 2023-10-06T05:01:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39efd7a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e815af10 con 0x7f39e81594d0 2023-10-06T05:01:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39efd7a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e815b1f0 con 0x7f39e815da50 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39edb16700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39e81594d0 0x7f39e80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39ed315700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f39e80ceb10 0x7f39e815d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42900/0 (socket says 172.21.15.3:42900) 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.001+0000 7f39ed315700 1 -- 172.21.15.3:0/3374180559 learned_addr learned my addr 172.21.15.3:0/3374180559 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39ee317700 1 --2- 172.21.15.3:0/3374180559 >> v2:172.21.15.153:3300/0 conn(0x7f39e815da50 0x7f39e815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39edb16700 1 -- 172.21.15.3:0/3374180559 >> v2:172.21.15.153:3300/0 conn(0x7f39e815da50 msgr2=0x7f39e815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39edb16700 1 --2- 172.21.15.3:0/3374180559 >> v2:172.21.15.153:3300/0 conn(0x7f39e815da50 0x7f39e815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39edb16700 1 -- 172.21.15.3:0/3374180559 >> v1:172.21.15.3:6789/0 conn(0x7f39e80ceb10 legacy=0x7f39e815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39ecb14700 1 -- 172.21.15.3:0/3374180559 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596736466 0 0) 0x7f39e815be00 con 0x7f39e80ceb10 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39edb16700 1 -- 172.21.15.3:0/3374180559 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e815b4d0 con 0x7f39e81594d0 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39edb16700 1 --2- 172.21.15.3:0/3374180559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e81594d0 0x7f39e80cd870 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f39e8089630 tx=0x7f39e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186cae76acb1ae1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39ecb14700 1 -- 172.21.15.3:0/3374180559 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000b6b0 con 0x7f39e81594d0 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39ecb14700 1 -- 172.21.15.3:0/3374180559 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e000b810 con 0x7f39e81594d0 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39ecb14700 1 -- 172.21.15.3:0/3374180559 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000bbb0 con 0x7f39e81594d0 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39efd7a700 1 -- 172.21.15.3:0/3374180559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e81594d0 msgr2=0x7f39e80cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.002+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3374180559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e81594d0 0x7f39e80cd870 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f39e8089630 tx=0x7f39e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.003+0000 7f39efd7a700 1 -- 172.21.15.3:0/3374180559 shutdown_connections 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.003+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3374180559 >> v2:172.21.15.153:3300/0 conn(0x7f39e815da50 0x7f39e815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.003+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3374180559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e81594d0 0x7f39e80cd870 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.003+0000 7f39efd7a700 1 -- 172.21.15.3:0/3374180559 >> 172.21.15.3:0/3374180559 conn(0x7f39e806c2f0 msgr2=0x7f39e806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.003+0000 7f39efd7a700 1 -- 172.21.15.3:0/3374180559 shutdown_connections 2023-10-06T05:01:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.003+0000 7f39efd7a700 1 -- 172.21.15.3:0/3374180559 wait complete. 2023-10-06T05:01:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.004+0000 7f39efd7a700 1 Processor -- start 2023-10-06T05:01:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.004+0000 7f39efd7a700 1 -- start start 2023-10-06T05:01:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.004+0000 7f39efd7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e80ceb10 0x7f39e816a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.004+0000 7f39efd7a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39e81594d0 0x7f39e814d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.004+0000 7f39efd7a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39e815da50 0x7f39e80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39efd7a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80c9d50 con 0x7f39e80ceb10 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39efd7a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e816a0c0 con 0x7f39e815da50 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39efd7a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e814d670 con 0x7f39e81594d0 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e80ceb10 0x7f39e816a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e80ceb10 0x7f39e816a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42836/0 (socket says 172.21.15.3:42836) 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 -- 172.21.15.3:0/3017195269 learned_addr learned my addr 172.21.15.3:0/3017195269 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39ee317700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39e815da50 0x7f39e80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39ed315700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39e81594d0 0x7f39e814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 -- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39e81594d0 msgr2=0x7f39e814d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39e81594d0 0x7f39e814d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 -- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39e815da50 msgr2=0x7f39e80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39e815da50 0x7f39e80c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 -- 172.21.15.3:0/3017195269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0009420 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39ee317700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39e815da50 0x7f39e80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39ed315700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39e81594d0 0x7f39e814d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39edb16700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e80ceb10 0x7f39e816a550 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f39e000b0e0 tx=0x7f39e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39e5ffb700 1 -- 172.21.15.3:0/3017195269 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0011420 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39e5ffb700 1 -- 172.21.15.3:0/3017195269 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0011580 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39e5ffb700 1 -- 172.21.15.3:0/3017195269 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0011940 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.005+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e80c9a70 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.006+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e8169d50 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.006+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e816a3a0 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.007+0000 7f39e5ffb700 1 -- 172.21.15.3:0/3017195269 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f39e0011d30 con 0x7f39e80ceb10 2023-10-06T05:01:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.007+0000 7f39e5ffb700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39d4066f30 0x7f39d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.007+0000 7f39e5ffb700 1 -- 172.21.15.3:0/3017195269 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f39e0086060 con 0x7f39e80ceb10 2023-10-06T05:01:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.008+0000 7f39ed315700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39d4066f30 0x7f39d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.008+0000 7f39ed315700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39d4066f30 0x7f39d40693e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f39e80b64b0 tx=0x7f39dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.012+0000 7f39e5ffb700 1 -- 172.21.15.3:0/3017195269 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39e0053170 con 0x7f39e80ceb10 2023-10-06T05:01:40.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.246+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f39e804c380 con 0x7f39e80ceb10 2023-10-06T05:01:40.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.246+0000 7f39e5ffb700 1 -- 172.21.15.3:0/3017195269 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f39e0056790 con 0x7f39e80ceb10 2023-10-06T05:01:40.248 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:40.248 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:01:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39d4066f30 msgr2=0x7f39d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39d4066f30 0x7f39d40693e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f39e80b64b0 tx=0x7f39dc009040 comp rx=0 tx=0).stop 2023-10-06T05:01:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e80ceb10 msgr2=0x7f39e816a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e80ceb10 0x7f39e816a550 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f39e000b0e0 tx=0x7f39e0017000 comp rx=0 tx=0).stop 2023-10-06T05:01:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 shutdown_connections 2023-10-06T05:01:40.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f39d4066f30 0x7f39d40693e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39e815da50 0x7f39e80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39e80ceb10 0x7f39e816a550 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 --2- 172.21.15.3:0/3017195269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39e81594d0 0x7f39e814d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 >> 172.21.15.3:0/3017195269 conn(0x7f39e806c2f0 msgr2=0x7f39e80c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.248+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 shutdown_connections 2023-10-06T05:01:40.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:40.249+0000 7f39efd7a700 1 -- 172.21.15.3:0/3017195269 wait complete. 2023-10-06T05:01:40.262 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:01:40.262 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:01:45.263 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:01:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.394+0000 7f2a34941700 1 Processor -- start 2023-10-06T05:01:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.394+0000 7f2a34941700 1 -- start start 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a34941700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 0x7f2a300b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a34941700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2a300b06d0 0x7f2a300aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a34941700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a300b0bb0 con 0x7f2a300b1da0 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a34941700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a300b0e90 con 0x7f2a300b06d0 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a34941700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300b1170 con 0x7f2a30159b20 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a2f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2a300b1da0 0x7f2a300b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55782/0 (socket says 172.21.15.3:55782) 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a2f7fe700 1 -- 172.21.15.3:0/3997888803 learned_addr learned my addr 172.21.15.3:0/3997888803 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a2effd700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 0x7f2a300b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a2ffff700 1 --2- 172.21.15.3:0/3997888803 >> v2:172.21.15.153:3300/0 conn(0x7f2a300b06d0 0x7f2a300aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390624684 0 0) 0x7f2a300b0bb0 con 0x7f2a300b1da0 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.395+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a18003570 con 0x7f2a300b1da0 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 451676474 0 0) 0x7f2a18003570 con 0x7f2a300b1da0 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 msgr2=0x7f2a300b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 0x7f2a300b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 >> v2:172.21.15.153:3300/0 conn(0x7f2a300b06d0 msgr2=0x7f2a300aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 --2- 172.21.15.3:0/3997888803 >> v2:172.21.15.153:3300/0 conn(0x7f2a300b06d0 0x7f2a300aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300b1450 con 0x7f2a300b1da0 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f2a200015f0 con 0x7f2a300b1da0 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 >> v1:172.21.15.3:6789/0 conn(0x7f2a300b1da0 legacy=0x7f2a300b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 0x7f2a18003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a18004d50 0x7f2a180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300b0e90 con 0x7f2a18003fe0 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300b1170 con 0x7f2a18004d50 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1800ba90 con 0x7f2a30159b20 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2effd700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2f7fe700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 0x7f2a18003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2ffff700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a18004d50 0x7f2a180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2effd700 1 -- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 msgr2=0x7f2a18003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2effd700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a30159b20 0x7f2a18003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2effd700 1 -- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a18004d50 msgr2=0x7f2a180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2effd700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a18004d50 0x7f2a180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2effd700 1 -- 172.21.15.3:0/3997888803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18003570 con 0x7f2a18003fe0 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2effd700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a18004810 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f2a30159f10 tx=0x7f2a240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34568f3cec52579e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.396+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400e6b0 con 0x7f2a18003fe0 2023-10-06T05:01:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400e810 con 0x7f2a18003fe0 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a2e7fc700 1 -- 172.21.15.3:0/3997888803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400ebd0 con 0x7f2a18003fe0 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a34941700 1 -- 172.21.15.3:0/3997888803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 msgr2=0x7f2a18004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a34941700 1 --2- 172.21.15.3:0/3997888803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a18004810 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f2a30159f10 tx=0x7f2a240091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a2ffff700 1 -- 172.21.15.3:0/3997888803 reap_dead start 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a34941700 1 -- 172.21.15.3:0/3997888803 shutdown_connections 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a34941700 1 -- 172.21.15.3:0/3997888803 >> 172.21.15.3:0/3997888803 conn(0x7f2a30099780 msgr2=0x7f2a3006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a34941700 1 -- 172.21.15.3:0/3997888803 shutdown_connections 2023-10-06T05:01:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.397+0000 7f2a34941700 1 -- 172.21.15.3:0/3997888803 wait complete. 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.398+0000 7f2a34941700 1 Processor -- start 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.398+0000 7f2a34941700 1 -- start start 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a34941700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a3015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a34941700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a3015e030 0x7f2a300cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a34941700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a3015b800 0x7f2a300b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a34941700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015d680 con 0x7f2a18003fe0 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a34941700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015ccb0 con 0x7f2a3015b800 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a34941700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300cc770 con 0x7f2a3015e030 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a2f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a3015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a2f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a3015d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39312/0 (socket says 172.21.15.3:39312) 2023-10-06T05:01:45.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a2f7fe700 1 -- 172.21.15.3:0/137107091 learned_addr learned my addr 172.21.15.3:0/137107091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a2ffff700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a3015b800 0x7f2a300b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.399+0000 7f2a2effd700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a3015e030 0x7f2a300cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a2f7fe700 1 -- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a3015e030 msgr2=0x7f2a300cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a2f7fe700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a3015e030 0x7f2a300cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a2f7fe700 1 -- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a3015b800 msgr2=0x7f2a300b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a2f7fe700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a3015b800 0x7f2a300b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a2f7fe700 1 -- 172.21.15.3:0/137107091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3015c0a0 con 0x7f2a18003fe0 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a2effd700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a3015e030 0x7f2a300cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a2f7fe700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a3015d140 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000fc30 tx=0x7f2a20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a177fe700 1 -- 172.21.15.3:0/137107091 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a200072e0 con 0x7f2a18003fe0 2023-10-06T05:01:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a177fe700 1 -- 172.21.15.3:0/137107091 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20007440 con 0x7f2a18003fe0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3015c380 con 0x7f2a18003fe0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a177fe700 1 -- 172.21.15.3:0/137107091 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a200057f0 con 0x7f2a18003fe0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.400+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3015c940 con 0x7f2a18003fe0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.401+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a30088660 con 0x7f2a18003fe0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.401+0000 7f2a177fe700 1 -- 172.21.15.3:0/137107091 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a20005b80 con 0x7f2a18003fe0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.402+0000 7f2a177fe700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a1c066f30 0x7f2a1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.402+0000 7f2a177fe700 1 -- 172.21.15.3:0/137107091 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2a20084940 con 0x7f2a18003fe0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.402+0000 7f2a2effd700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a1c066f30 0x7f2a1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.403+0000 7f2a2effd700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a1c066f30 0x7f2a1c0693e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2a30158a70 tx=0x7f2a2401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.406+0000 7f2a177fe700 1 -- 172.21.15.3:0/137107091 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a20051a50 con 0x7f2a18003fe0 2023-10-06T05:01:45.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.642+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2a300afae0 con 0x7f2a18003fe0 2023-10-06T05:01:45.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.643+0000 7f2a177fe700 1 -- 172.21.15.3:0/137107091 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2a20055070 con 0x7f2a18003fe0 2023-10-06T05:01:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:45.645 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:01:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a1c066f30 msgr2=0x7f2a1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a1c066f30 0x7f2a1c0693e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2a30158a70 tx=0x7f2a2401a000 comp rx=0 tx=0).stop 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 msgr2=0x7f2a3015d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a3015d140 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000fc30 tx=0x7f2a20009040 comp rx=0 tx=0).stop 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 shutdown_connections 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2a1c066f30 0x7f2a1c0693e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a3015b800 0x7f2a300b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18003fe0 0x7f2a3015d140 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 --2- 172.21.15.3:0/137107091 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a3015e030 0x7f2a300cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 >> 172.21.15.3:0/137107091 conn(0x7f2a30099780 msgr2=0x7f2a30158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 shutdown_connections 2023-10-06T05:01:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:45.645+0000 7f2a34941700 1 -- 172.21.15.3:0/137107091 wait complete. 2023-10-06T05:01:45.658 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:01:45.658 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:01:50.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:01:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.788+0000 7fd60a3a1700 1 Processor -- start 2023-10-06T05:01:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.788+0000 7fd60a3a1700 1 -- start start 2023-10-06T05:01:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.789+0000 7fd60a3a1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd60415da50 0x7fd60415c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.789+0000 7fd60a3a1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60415be00 con 0x7fd6040ceb10 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.789+0000 7fd60a3a1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60415af10 con 0x7fd6041594d0 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.789+0000 7fd60a3a1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60415b1f0 con 0x7fd60415da50 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.789+0000 7fd6037fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd6040ceb10 0x7fd60415d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55788/0 (socket says 172.21.15.3:55788) 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.789+0000 7fd6037fe700 1 -- 172.21.15.3:0/1936581328 learned_addr learned my addr 172.21.15.3:0/1936581328 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd60893e700 1 --2- 172.21.15.3:0/1936581328 >> v2:172.21.15.153:3300/0 conn(0x7fd60415da50 0x7fd60415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742233144 0 0) 0x7fd60415be00 con 0x7fd6040ceb10 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ec003570 con 0x7fd6040ceb10 2023-10-06T05:01:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87190692 0 0) 0x7fd5ec003570 con 0x7fd6040ceb10 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 >> v2:172.21.15.153:3300/0 conn(0x7fd60415da50 msgr2=0x7fd60415c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1936581328 >> v2:172.21.15.153:3300/0 conn(0x7fd60415da50 0x7fd60415c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 >> v1:172.21.15.153:6789/0 conn(0x7fd6041594d0 legacy=0x7fd6040cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60415b4d0 con 0x7fd6040ceb10 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fd5f8001470 con 0x7fd6040ceb10 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 >> v1:172.21.15.3:6789/0 conn(0x7fd6040ceb10 legacy=0x7fd60415d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5ec003a60 0x7fd5ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd5ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5ec0051c0 0x7fd5ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60415b1f0 con 0x7fd5ec004810 2023-10-06T05:01:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec00c160 con 0x7fd5ec0051c0 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec00c320 con 0x7fd5ec003a60 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd6037fe700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd5ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd603fff700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5ec003a60 0x7fd5ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd60893e700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5ec0051c0 0x7fd5ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd6037fe700 1 -- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5ec003a60 msgr2=0x7fd5ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd6037fe700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5ec003a60 0x7fd5ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd6037fe700 1 -- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5ec0051c0 msgr2=0x7fd5ec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd6037fe700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5ec0051c0 0x7fd5ec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd6037fe700 1 -- 172.21.15.3:0/1936581328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec003570 con 0x7fd5ec004810 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd60893e700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5ec0051c0 0x7fd5ec00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.790+0000 7fd60893e700 1 -- 172.21.15.3:0/1936581328 reap_dead start 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd6037fe700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd5ec004c80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c3c0 tx=0x7fd5f8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c3aeaf7af6ad44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800cc90 con 0x7fd5ec004810 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f800cdf0 con 0x7fd5ec004810 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd602ffd700 1 -- 172.21.15.3:0/1936581328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8005b70 con 0x7fd5ec004810 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd60a3a1700 1 -- 172.21.15.3:0/1936581328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 msgr2=0x7fd5ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd5ec004c80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c3c0 tx=0x7fd5f8001e50 comp rx=0 tx=0).stop 2023-10-06T05:01:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd60a3a1700 1 -- 172.21.15.3:0/1936581328 shutdown_connections 2023-10-06T05:01:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/1936581328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd5ec004c80 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd60a3a1700 1 -- 172.21.15.3:0/1936581328 >> 172.21.15.3:0/1936581328 conn(0x7fd60406c2f0 msgr2=0x7fd60406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd60a3a1700 1 -- 172.21.15.3:0/1936581328 shutdown_connections 2023-10-06T05:01:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.791+0000 7fd60a3a1700 1 -- 172.21.15.3:0/1936581328 wait complete. 2023-10-06T05:01:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.793+0000 7fd60a3a1700 1 Processor -- start 2023-10-06T05:01:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.793+0000 7fd60a3a1700 1 -- start start 2023-10-06T05:01:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.793+0000 7fd60a3a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd60416dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60a3a1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6041459a0 0x7fd6040b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60a3a1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd604144c50 0x7fd6041450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60a3a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604145630 con 0x7fd5ec004810 2023-10-06T05:01:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60a3a1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040b4360 con 0x7fd604144c50 2023-10-06T05:01:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60a3a1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60416e510 con 0x7fd6041459a0 2023-10-06T05:01:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd60416dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd60416dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39336/0 (socket says 172.21.15.3:39336) 2023-10-06T05:01:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 -- 172.21.15.3:0/2031404711 learned_addr learned my addr 172.21.15.3:0/2031404711 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 -- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6041459a0 msgr2=0x7fd6040b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6041459a0 0x7fd6040b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 -- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd604144c50 msgr2=0x7fd6041450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60893e700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd604144c50 0x7fd6041450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd604144c50 0x7fd6041450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 -- 172.21.15.3:0/2031404711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6040c9710 con 0x7fd5ec004810 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd603fff700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd60416dfd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4002980 tx=0x7fd5f4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd5ebfff700 1 -- 172.21.15.3:0/2031404711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400f850 con 0x7fd5ec004810 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd5ebfff700 1 -- 172.21.15.3:0/2031404711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f400f9b0 con 0x7fd5ec004810 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6040b3750 con 0x7fd5ec004810 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd5ebfff700 1 -- 172.21.15.3:0/2031404711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400fd50 con 0x7fd5ec004810 2023-10-06T05:01:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.794+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6040b3d10 con 0x7fd5ec004810 2023-10-06T05:01:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.795+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60404c380 con 0x7fd5ec004810 2023-10-06T05:01:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.796+0000 7fd5ebfff700 1 -- 172.21.15.3:0/2031404711 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5f4013600 con 0x7fd5ec004810 2023-10-06T05:01:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.796+0000 7fd5ebfff700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5f0066f30 0x7fd5f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.796+0000 7fd5ebfff700 1 -- 172.21.15.3:0/2031404711 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd5f4084ba0 con 0x7fd5ec004810 2023-10-06T05:01:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.796+0000 7fd6037fe700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5f0066f30 0x7fd5f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.797+0000 7fd6037fe700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5f0066f30 0x7fd5f00693e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd604158ef0 tx=0x7fd5f8015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:50.801+0000 7fd5ebfff700 1 -- 172.21.15.3:0/2031404711 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5f4051cb0 con 0x7fd5ec004810 2023-10-06T05:01:51.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.037+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd60415b7b0 con 0x7fd5ec004810 2023-10-06T05:01:51.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.038+0000 7fd5ebfff700 1 -- 172.21.15.3:0/2031404711 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd5f40552d0 con 0x7fd5ec004810 2023-10-06T05:01:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:51.041 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:01:51.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5f0066f30 msgr2=0x7fd5f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:51.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5f0066f30 0x7fd5f00693e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd604158ef0 tx=0x7fd5f8015000 comp rx=0 tx=0).stop 2023-10-06T05:01:51.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 msgr2=0x7fd60416dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:51.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd60416dfd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4002980 tx=0x7fd5f4007d30 comp rx=0 tx=0).stop 2023-10-06T05:01:51.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 shutdown_connections 2023-10-06T05:01:51.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd5f0066f30 0x7fd5f00693e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:51.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd604144c50 0x7fd6041450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:51.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ec004810 0x7fd60416dfd0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:51.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 --2- 172.21.15.3:0/2031404711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6041459a0 0x7fd6040b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:51.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 >> 172.21.15.3:0/2031404711 conn(0x7fd60406c2f0 msgr2=0x7fd604153b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:51.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 shutdown_connections 2023-10-06T05:01:51.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:51.040+0000 7fd60a3a1700 1 -- 172.21.15.3:0/2031404711 wait complete. 2023-10-06T05:01:51.055 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:01:51.055 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:01:55.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:55.799 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:56.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:01:56.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.185+0000 7f088528a700 1 Processor -- start 2023-10-06T05:01:56.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.186+0000 7f088528a700 1 -- start start 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.186+0000 7f088528a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08800caa60 0x7f08800703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.186+0000 7f088528a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f088015dc60 0x7f088006a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.186+0000 7f088528a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088006a290 con 0x7f08800cae30 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.186+0000 7f088528a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088006d880 con 0x7f088015dc60 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.186+0000 7f088528a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088006cfe0 con 0x7f08800caa60 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f088015dc60 0x7f088006a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f08800cae30 0x7f088015d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36282/0 (socket says 172.21.15.3:36282) 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087e7fc700 1 -- 172.21.15.3:0/1707299314 learned_addr learned my addr 172.21.15.3:0/1707299314 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:01:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087effd700 1 --2- 172.21.15.3:0/1707299314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08800caa60 0x7f08800703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087f7fe700 1 -- 172.21.15.3:0/1707299314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08800caa60 msgr2=0x7f08800703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087f7fe700 1 --2- 172.21.15.3:0/1707299314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08800caa60 0x7f08800703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087f7fe700 1 -- 172.21.15.3:0/1707299314 >> v1:172.21.15.3:6789/0 conn(0x7f08800cae30 legacy=0x7f088015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087dffb700 1 -- 172.21.15.3:0/1707299314 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654320697 0 0) 0x7f088006a290 con 0x7f08800cae30 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087f7fe700 1 -- 172.21.15.3:0/1707299314 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088006d2c0 con 0x7f088015dc60 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087f7fe700 1 --2- 172.21.15.3:0/1707299314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f088015dc60 0x7f088006a720 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f08780098e0 tx=0x7f08780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b69fc5ba8c76af6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087dffb700 1 -- 172.21.15.3:0/1707299314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800b640 con 0x7f088015dc60 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087dffb700 1 -- 172.21.15.3:0/1707299314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800b7a0 con 0x7f088015dc60 2023-10-06T05:01:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.187+0000 7f087dffb700 1 -- 172.21.15.3:0/1707299314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800bb40 con 0x7f088015dc60 2023-10-06T05:01:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 -- 172.21.15.3:0/1707299314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f088015dc60 msgr2=0x7f088006a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 --2- 172.21.15.3:0/1707299314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f088015dc60 0x7f088006a720 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f08780098e0 tx=0x7f08780091a0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 -- 172.21.15.3:0/1707299314 shutdown_connections 2023-10-06T05:01:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 --2- 172.21.15.3:0/1707299314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f088015dc60 0x7f088006a720 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 --2- 172.21.15.3:0/1707299314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08800caa60 0x7f08800703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 -- 172.21.15.3:0/1707299314 >> 172.21.15.3:0/1707299314 conn(0x7f08800c94a0 msgr2=0x7f08800c98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 -- 172.21.15.3:0/1707299314 shutdown_connections 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.188+0000 7f088528a700 1 -- 172.21.15.3:0/1707299314 wait complete. 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.189+0000 7f088528a700 1 Processor -- start 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.189+0000 7f088528a700 1 -- start start 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f088528a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08800caa60 0x7f088014c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f088528a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08800cae30 0x7f08800adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f088528a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088015dc60 0x7f08800abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f088528a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800aef70 con 0x7f08800caa60 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f088528a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800ae6d0 con 0x7f08800cae30 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f088528a700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800ae9b0 con 0x7f088015dc60 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f087effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08800caa60 0x7f088014c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:56.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f087effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08800caa60 0x7f088014c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50836/0 (socket says 172.21.15.3:50836) 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f087effd700 1 -- 172.21.15.3:0/1941860740 learned_addr learned my addr 172.21.15.3:0/1941860740 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f087e7fc700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08800cae30 0x7f08800adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.190+0000 7f087f7fe700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088015dc60 0x7f08800abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f087effd700 1 -- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088015dc60 msgr2=0x7f08800abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f087effd700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088015dc60 0x7f08800abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f087effd700 1 -- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08800cae30 msgr2=0x7f08800adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f087effd700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08800cae30 0x7f08800adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f087effd700 1 -- 172.21.15.3:0/1941860740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878009420 con 0x7f08800caa60 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f087effd700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08800caa60 0x7f088014c4f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0870009ef0 tx=0x7f087000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f0876ffd700 1 -- 172.21.15.3:0/1941860740 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000cc50 con 0x7f08800caa60 2023-10-06T05:01:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f0876ffd700 1 -- 172.21.15.3:0/1941860740 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000f040 con 0x7f08800caa60 2023-10-06T05:01:56.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08800aec90 con 0x7f08800caa60 2023-10-06T05:01:56.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f0876ffd700 1 -- 172.21.15.3:0/1941860740 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870014480 con 0x7f08800caa60 2023-10-06T05:01:56.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.191+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088014ca30 con 0x7f08800caa60 2023-10-06T05:01:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.192+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08800c6040 con 0x7f08800caa60 2023-10-06T05:01:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.192+0000 7f0876ffd700 1 -- 172.21.15.3:0/1941860740 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08700147f0 con 0x7f08800caa60 2023-10-06T05:01:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.193+0000 7f0876ffd700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:01:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.193+0000 7f0876ffd700 1 -- 172.21.15.3:0/1941860740 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f08700850d0 con 0x7f08800caa60 2023-10-06T05:01:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.193+0000 7f087e7fc700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:01:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.194+0000 7f087e7fc700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0860066f30 0x7f08600693e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f08800c2520 tx=0x7f0868009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:01:56.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.198+0000 7f0876ffd700 1 -- 172.21.15.3:0/1941860740 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08700521e0 con 0x7f08800caa60 2023-10-06T05:01:56.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.434+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f088004c380 con 0x7f08800caa60 2023-10-06T05:01:56.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.434+0000 7f0876ffd700 1 -- 172.21.15.3:0/1941860740 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0870055800 con 0x7f08800caa60 2023-10-06T05:01:56.437 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:01:56.437 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0860066f30 msgr2=0x7f08600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0860066f30 0x7f08600693e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f08800c2520 tx=0x7f0868009040 comp rx=0 tx=0).stop 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08800caa60 msgr2=0x7f088014c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08800caa60 0x7f088014c4f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0870009ef0 tx=0x7f087000b040 comp rx=0 tx=0).stop 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 shutdown_connections 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088015dc60 0x7f08800abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08800caa60 0x7f088014c4f0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 --2- 172.21.15.3:0/1941860740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08800cae30 0x7f08800adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:01:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 >> 172.21.15.3:0/1941860740 conn(0x7f08800c94a0 msgr2=0x7f08800c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:01:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 shutdown_connections 2023-10-06T05:01:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:01:56.436+0000 7f088528a700 1 -- 172.21.15.3:0/1941860740 wait complete. 2023-10-06T05:01:56.450 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:01:56.451 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:01.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.580+0000 7f0ebeaf5700 1 Processor -- start 2023-10-06T05:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.580+0000 7f0ebeaf5700 1 -- start start 2023-10-06T05:02:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebeaf5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0eb80b1da0 0x7f0eb80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebeaf5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb80aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebeaf5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb80b0bb0 con 0x7f0eb8159b20 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebeaf5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb80b0e90 con 0x7f0eb80b1da0 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebeaf5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80b1170 con 0x7f0eb80b06d0 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebdaf3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0eb80b1da0 0x7f0eb80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebdaf3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0eb80b1da0 0x7f0eb80b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50850/0 (socket says 172.21.15.3:50850) 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.581+0000 7f0ebdaf3700 1 -- 172.21.15.3:0/2087272417 learned_addr learned my addr 172.21.15.3:0/2087272417 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebe2f4700 1 --2- 172.21.15.3:0/2087272417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb80aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebdaf3700 1 -- 172.21.15.3:0/2087272417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 msgr2=0x7f0eb80aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebdaf3700 1 --2- 172.21.15.3:0/2087272417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb80aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebdaf3700 1 -- 172.21.15.3:0/2087272417 >> v1:172.21.15.153:6789/0 conn(0x7f0eb8159b20 legacy=0x7f0eb80b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebdaf3700 1 -- 172.21.15.3:0/2087272417 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80b1450 con 0x7f0eb80b1da0 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebe2f4700 1 --2- 172.21.15.3:0/2087272417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb80aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebdaf3700 1 --2- 172.21.15.3:0/2087272417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb80b1da0 0x7f0eb80b2170 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009f70 tx=0x7f0ea8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f9db3cf56a514c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebcaf1700 1 -- 172.21.15.3:0/2087272417 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800e070 con 0x7f0eb80b1da0 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebcaf1700 1 -- 172.21.15.3:0/2087272417 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea800bb20 con 0x7f0eb80b1da0 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.582+0000 7f0ebcaf1700 1 -- 172.21.15.3:0/2087272417 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80092e0 con 0x7f0eb80b1da0 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/2087272417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb80b1da0 msgr2=0x7f0eb80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/2087272417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb80b1da0 0x7f0eb80b2170 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009f70 tx=0x7f0ea8009840 comp rx=0 tx=0).stop 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/2087272417 shutdown_connections 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/2087272417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb80aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/2087272417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb80b1da0 0x7f0eb80b2170 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/2087272417 >> 172.21.15.3:0/2087272417 conn(0x7f0eb8099780 msgr2=0x7f0eb806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/2087272417 shutdown_connections 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.583+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/2087272417 wait complete. 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.584+0000 7f0ebeaf5700 1 Processor -- start 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.584+0000 7f0ebeaf5700 1 -- start start 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebeaf5700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb809d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebeaf5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb80b1da0 0x7f0eb815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebeaf5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb8159b20 0x7f0eb815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebeaf5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81557b0 con 0x7f0eb8159b20 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebeaf5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815e8c0 con 0x7f0eb80b1da0 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebeaf5700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb816ae70 con 0x7f0eb80b06d0 2023-10-06T05:02:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb8159b20 0x7f0eb815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb8159b20 0x7f0eb815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50860/0 (socket says 172.21.15.3:50860) 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 -- 172.21.15.3:0/4009037361 learned_addr learned my addr 172.21.15.3:0/4009037361 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebdaf3700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb809d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebd2f2700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb80b1da0 0x7f0eb815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 -- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 msgr2=0x7f0eb809d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb809d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 -- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb80b1da0 msgr2=0x7f0eb815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb80b1da0 0x7f0eb815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 -- 172.21.15.3:0/4009037361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8009ae0 con 0x7f0eb8159b20 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebe2f4700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb8159b20 0x7f0eb815d910 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c880 tx=0x7f0eb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0eb5ffb700 1 -- 172.21.15.3:0/4009037361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0011600 con 0x7f0eb8159b20 2023-10-06T05:02:01.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0eb5ffb700 1 -- 172.21.15.3:0/4009037361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb0011780 con 0x7f0eb8159b20 2023-10-06T05:02:01.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb816ab90 con 0x7f0eb8159b20 2023-10-06T05:02:01.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.585+0000 7f0eb5ffb700 1 -- 172.21.15.3:0/4009037361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0011b40 con 0x7f0eb8159b20 2023-10-06T05:02:01.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.586+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb815e550 con 0x7f0eb8159b20 2023-10-06T05:02:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.586+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb815ea70 con 0x7f0eb8159b20 2023-10-06T05:02:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.587+0000 7f0eb5ffb700 1 -- 172.21.15.3:0/4009037361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0eb0022350 con 0x7f0eb8159b20 2023-10-06T05:02:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.587+0000 7f0eb5ffb700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ea4066f30 0x7f0ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.587+0000 7f0eb5ffb700 1 -- 172.21.15.3:0/4009037361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0eb0084680 con 0x7f0eb8159b20 2023-10-06T05:02:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.588+0000 7f0ebdaf3700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ea4066f30 0x7f0ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.588+0000 7f0ebdaf3700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ea4066f30 0x7f0ea40693e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009880 tx=0x7f0ea8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.592+0000 7f0eb5ffb700 1 -- 172.21.15.3:0/4009037361 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eb815ea70 con 0x7f0eb8159b20 2023-10-06T05:02:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.829+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0eb80b4180 con 0x7f0eb8159b20 2023-10-06T05:02:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.830+0000 7f0eb5ffb700 1 -- 172.21.15.3:0/4009037361 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0eb0082e60 con 0x7f0eb8159b20 2023-10-06T05:02:01.833 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:01.833 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:01.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ea4066f30 msgr2=0x7f0ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ea4066f30 0x7f0ea40693e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009880 tx=0x7f0ea8017000 comp rx=0 tx=0).stop 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb8159b20 msgr2=0x7f0eb815d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb8159b20 0x7f0eb815d910 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c880 tx=0x7f0eb0008040 comp rx=0 tx=0).stop 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 shutdown_connections 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0ea4066f30 0x7f0ea40693e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb8159b20 0x7f0eb815d910 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb80b06d0 0x7f0eb809d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 --2- 172.21.15.3:0/4009037361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb80b1da0 0x7f0eb815ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 >> 172.21.15.3:0/4009037361 conn(0x7f0eb8099780 msgr2=0x7f0eb80b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 shutdown_connections 2023-10-06T05:02:01.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:01.832+0000 7f0ebeaf5700 1 -- 172.21.15.3:0/4009037361 wait complete. 2023-10-06T05:02:01.846 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:01.846 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:06.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.203+0000 7fa8eabe6700 1 Processor -- start 2023-10-06T05:02:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.203+0000 7fa8eabe6700 1 -- start start 2023-10-06T05:02:07.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8eabe6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa8e4159690 0x7fa8e406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:07.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8eabe6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e406b860 0x7fa8e406c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:07.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8eabe6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e40b09e0 con 0x7fa8e4159a60 2023-10-06T05:02:07.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8eabe6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e406a900 con 0x7fa8e4159690 2023-10-06T05:02:07.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8eabe6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e406abe0 con 0x7fa8e406b860 2023-10-06T05:02:07.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e8982700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa8e4159690 0x7fa8e406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e406b860 0x7fa8e406c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e8982700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa8e4159690 0x7fa8e406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.3:46954/0 (socket says 172.21.15.3:46954) 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e8982700 1 -- 172.21.15.3:0/2359564808 learned_addr learned my addr 172.21.15.3:0/2359564808 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e9183700 1 --1- 172.21.15.3:0/2359564808 >> v1:172.21.15.153:6789/0 conn(0x7fa8e4159a60 0x7fa8e40a98c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:43918/0 (socket says 172.21.15.3:43918) 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e8982700 1 -- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e406b860 msgr2=0x7fa8e406c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e8982700 1 --2- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e406b860 0x7fa8e406c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e8982700 1 -- 172.21.15.3:0/2359564808 >> v1:172.21.15.153:6789/0 conn(0x7fa8e4159a60 legacy=0x7fa8e40a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e8982700 1 -- 172.21.15.3:0/2359564808 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e40b0c30 con 0x7fa8e4159690 2023-10-06T05:02:07.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e37fe700 1 -- 172.21.15.3:0/2359564808 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997630682 0 0) 0x7fa8e40b09e0 con 0x7fa8e4159a60 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.204+0000 7fa8e3fff700 1 --2- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e406b860 0x7fa8e406c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8e8982700 1 --2- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e4159690 0x7fa8e406b070 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0099b0 tx=0x7fa8cc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574ae4a80ea970b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8e37fe700 1 -- 172.21.15.3:0/2359564808 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc009ea0 con 0x7fa8e4159690 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8e37fe700 1 -- 172.21.15.3:0/2359564808 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc00b860 con 0x7fa8e4159690 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8e37fe700 1 -- 172.21.15.3:0/2359564808 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00bc20 con 0x7fa8e4159690 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8eabe6700 1 -- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e4159690 msgr2=0x7fa8e406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e4159690 0x7fa8e406b070 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0099b0 tx=0x7fa8cc009250 comp rx=0 tx=0).stop 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8eabe6700 1 -- 172.21.15.3:0/2359564808 shutdown_connections 2023-10-06T05:02:07.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e4159690 0x7fa8e406b070 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/2359564808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e406b860 0x7fa8e406c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8eabe6700 1 -- 172.21.15.3:0/2359564808 >> 172.21.15.3:0/2359564808 conn(0x7fa8e406de90 msgr2=0x7fa8e40aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:07.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.205+0000 7fa8eabe6700 1 -- 172.21.15.3:0/2359564808 shutdown_connections 2023-10-06T05:02:07.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.206+0000 7fa8eabe6700 1 -- 172.21.15.3:0/2359564808 wait complete. 2023-10-06T05:02:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 Processor -- start 2023-10-06T05:02:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 -- start start 2023-10-06T05:02:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e406b860 0x7fa8e414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e4159690 0x7fa8e41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e4159a60 0x7fa8e415f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4169c30 con 0x7fa8e4159690 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4157e60 con 0x7fa8e406b860 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.207+0000 7fa8eabe6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4158140 con 0x7fa8e4159a60 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e4159690 0x7fa8e41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e4159690 0x7fa8e41574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38406/0 (socket says 172.21.15.3:38406) 2023-10-06T05:02:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 -- 172.21.15.3:0/561839541 learned_addr learned my addr 172.21.15.3:0/561839541 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e9183700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e4159a60 0x7fa8e415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e8982700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e406b860 0x7fa8e414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 -- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e4159a60 msgr2=0x7fa8e415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e4159a60 0x7fa8e415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 -- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e406b860 msgr2=0x7fa8e414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e406b860 0x7fa8e414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 -- 172.21.15.3:0/561839541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009420 con 0x7fa8e4159690 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e3fff700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e4159690 0x7fa8e41574a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400ec80 tx=0x7fa8d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e0ff9700 1 -- 172.21.15.3:0/561839541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400c920 con 0x7fa8e4159690 2023-10-06T05:02:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e0ff9700 1 -- 172.21.15.3:0/561839541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d400ca80 con 0x7fa8e4159690 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8e0ff9700 1 -- 172.21.15.3:0/561839541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400ce00 con 0x7fa8e4159690 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc0094f0 con 0x7fa8e4159690 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.208+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e414f6e0 con 0x7fa8e4159690 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.209+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e4169de0 con 0x7fa8e4159690 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.210+0000 7fa8e0ff9700 1 -- 172.21.15.3:0/561839541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8d4013680 con 0x7fa8e4159690 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.210+0000 7fa8e0ff9700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8c4066f30 0x7fa8c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.210+0000 7fa8e0ff9700 1 -- 172.21.15.3:0/561839541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa8d4084a60 con 0x7fa8e4159690 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.210+0000 7fa8e8982700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8c4066f30 0x7fa8c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.211+0000 7fa8e8982700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8c4066f30 0x7fa8c40693e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0099b0 tx=0x7fa8cc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.215+0000 7fa8e0ff9700 1 -- 172.21.15.3:0/561839541 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8e4169de0 con 0x7fa8e4159690 2023-10-06T05:02:07.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.452+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa8e404c380 con 0x7fa8e4159690 2023-10-06T05:02:07.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.453+0000 7fa8e0ff9700 1 -- 172.21.15.3:0/561839541 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa8d40555c0 con 0x7fa8e4159690 2023-10-06T05:02:07.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:07.456 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8c4066f30 msgr2=0x7fa8c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8c4066f30 0x7fa8c40693e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0099b0 tx=0x7fa8cc017000 comp rx=0 tx=0).stop 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e4159690 msgr2=0x7fa8e41574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e4159690 0x7fa8e41574a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400ec80 tx=0x7fa8d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 shutdown_connections 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa8c4066f30 0x7fa8c40693e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e406b860 0x7fa8e414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8e4159a60 0x7fa8e415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 --2- 172.21.15.3:0/561839541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e4159690 0x7fa8e41574a0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 >> 172.21.15.3:0/561839541 conn(0x7fa8e406de90 msgr2=0x7fa8e41553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 shutdown_connections 2023-10-06T05:02:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:07.455+0000 7fa8eabe6700 1 -- 172.21.15.3:0/561839541 wait complete. 2023-10-06T05:02:07.469 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:07.469 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:12.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.601+0000 7f5519d87700 1 Processor -- start 2023-10-06T05:02:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.601+0000 7f5519d87700 1 -- start start 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5519d87700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f55140b1da0 0x7f55140b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5519d87700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b06d0 0x7f55140aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5519d87700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55140b0bb0 con 0x7f5514159b20 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5519d87700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55140b0e90 con 0x7f55140b1da0 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5519d87700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140b1170 con 0x7f55140b06d0 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5513fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5514159b20 0x7f55140b26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49992/0 (socket says 172.21.15.3:49992) 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5513fff700 1 -- 172.21.15.3:0/578598390 learned_addr learned my addr 172.21.15.3:0/578598390 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5518d85700 1 --2- 172.21.15.3:0/578598390 >> v2:172.21.15.153:3300/0 conn(0x7f55140b1da0 0x7f55140b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f5519586700 1 --2- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b06d0 0x7f55140aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.602+0000 7f55137fe700 1 -- 172.21.15.3:0/578598390 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277182649 0 0) 0x7f55140b0bb0 con 0x7f5514159b20 2023-10-06T05:02:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f55137fe700 1 -- 172.21.15.3:0/578598390 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54fc003570 con 0x7f5514159b20 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f5518d85700 1 -- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b06d0 msgr2=0x7f55140aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f5518d85700 1 --2- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b06d0 0x7f55140aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f5518d85700 1 -- 172.21.15.3:0/578598390 >> v1:172.21.15.3:6789/0 conn(0x7f5514159b20 legacy=0x7f55140b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f55137fe700 1 -- 172.21.15.3:0/578598390 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1336725694 0 0) 0x7f54fc003570 con 0x7f5514159b20 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f5518d85700 1 -- 172.21.15.3:0/578598390 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140b1450 con 0x7f55140b1da0 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f5519586700 1 --2- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b06d0 0x7f55140aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f5518d85700 1 --2- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55140b1da0 0x7f55140b2170 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f55040098d0 tx=0x7f5504009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dcf362f26a15f0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f55137fe700 1 -- 172.21.15.3:0/578598390 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508001470 con 0x7f55140b1da0 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f55137fe700 1 -- 172.21.15.3:0/578598390 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504011420 con 0x7f55140b1da0 2023-10-06T05:02:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.603+0000 7f55137fe700 1 -- 172.21.15.3:0/578598390 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55040117c0 con 0x7f55140b1da0 2023-10-06T05:02:12.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 -- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55140b1da0 msgr2=0x7f55140b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:12.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 --2- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55140b1da0 0x7f55140b2170 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f55040098d0 tx=0x7f5504009120 comp rx=0 tx=0).stop 2023-10-06T05:02:12.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 -- 172.21.15.3:0/578598390 shutdown_connections 2023-10-06T05:02:12.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 --2- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b06d0 0x7f55140aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 --2- 172.21.15.3:0/578598390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55140b1da0 0x7f55140b2170 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 -- 172.21.15.3:0/578598390 >> 172.21.15.3:0/578598390 conn(0x7f5514099780 msgr2=0x7f551406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 -- 172.21.15.3:0/578598390 shutdown_connections 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.604+0000 7f5519d87700 1 -- 172.21.15.3:0/578598390 wait complete. 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.605+0000 7f5519d87700 1 Processor -- start 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.605+0000 7f5519d87700 1 -- start start 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5519d87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55140b06d0 0x7f55140cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5519d87700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b1da0 0x7f55140b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5519d87700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5514159b20 0x7f55140b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5519d87700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551415dbb0 con 0x7f55140b06d0 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5519d87700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551415d0d0 con 0x7f5514159b20 2023-10-06T05:02:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5519d87700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140cc850 con 0x7f55140b1da0 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55140b06d0 0x7f55140cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55140b06d0 0x7f55140cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50072/0 (socket says 172.21.15.3:50072) 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 -- 172.21.15.3:0/685053699 learned_addr learned my addr 172.21.15.3:0/685053699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5513fff700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b1da0 0x7f55140b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5519586700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5514159b20 0x7f55140b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 -- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b1da0 msgr2=0x7f55140b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b1da0 0x7f55140b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 -- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5514159b20 msgr2=0x7f55140b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5514159b20 0x7f55140b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 -- 172.21.15.3:0/685053699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c009a20 con 0x7f55140b06d0 2023-10-06T05:02:12.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.606+0000 7f5518d85700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55140b06d0 0x7f55140cc310 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5514088ee0 tx=0x7f5504016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.607+0000 7f5510ff9700 1 -- 172.21.15.3:0/685053699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550401cd20 con 0x7f55140b06d0 2023-10-06T05:02:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.607+0000 7f5510ff9700 1 -- 172.21.15.3:0/685053699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550401a450 con 0x7f55140b06d0 2023-10-06T05:02:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.607+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55040093c0 con 0x7f55140b06d0 2023-10-06T05:02:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.607+0000 7f5510ff9700 1 -- 172.21.15.3:0/685053699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550401cea0 con 0x7f55140b06d0 2023-10-06T05:02:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.607+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551415c7a0 con 0x7f55140b06d0 2023-10-06T05:02:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.608+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5514088660 con 0x7f55140b06d0 2023-10-06T05:02:12.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.608+0000 7f5510ff9700 1 -- 172.21.15.3:0/685053699 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f550401aa20 con 0x7f55140b06d0 2023-10-06T05:02:12.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.609+0000 7f5510ff9700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5500066f30 0x7f55000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:12.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.609+0000 7f5510ff9700 1 -- 172.21.15.3:0/685053699 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f55040860a0 con 0x7f55140b06d0 2023-10-06T05:02:12.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.609+0000 7f5513fff700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5500066f30 0x7f55000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:12.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.609+0000 7f5513fff700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5500066f30 0x7f55000693e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5508001ed0 tx=0x7f5508001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:12.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.613+0000 7f5510ff9700 1 -- 172.21.15.3:0/685053699 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55040531b0 con 0x7f55140b06d0 2023-10-06T05:02:12.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.849+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f551404bb40 con 0x7f55140b06d0 2023-10-06T05:02:12.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.849+0000 7f5510ff9700 1 -- 172.21.15.3:0/685053699 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5504056790 con 0x7f55140b06d0 2023-10-06T05:02:12.851 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:12.851 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5500066f30 msgr2=0x7f55000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5500066f30 0x7f55000693e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5508001ed0 tx=0x7f5508001520 comp rx=0 tx=0).stop 2023-10-06T05:02:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55140b06d0 msgr2=0x7f55140cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55140b06d0 0x7f55140cc310 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5514088ee0 tx=0x7f5504016000 comp rx=0 tx=0).stop 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 shutdown_connections 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5500066f30 0x7f55000693e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5514159b20 0x7f55140b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55140b06d0 0x7f55140cc310 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 --2- 172.21.15.3:0/685053699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55140b1da0 0x7f55140b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 >> 172.21.15.3:0/685053699 conn(0x7f5514099780 msgr2=0x7f55140cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 shutdown_connections 2023-10-06T05:02:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:12.851+0000 7f5519d87700 1 -- 172.21.15.3:0/685053699 wait complete. 2023-10-06T05:02:12.865 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:12.865 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:17.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:17.999+0000 7f1bb9b6e700 1 Processor -- start 2023-10-06T05:02:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:17.999+0000 7f1bb9b6e700 1 -- start start 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:17.999+0000 7f1bb9b6e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb413c750 0x7f1bb40682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb9b6e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1bb40c3170 0x7f1bb40c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb9b6e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb40c3b20 con 0x7f1bb415d3c0 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb9b6e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb4069fd0 con 0x7f1bb40c3170 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb9b6e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40693c0 con 0x7f1bb413c750 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1bb415d3c0 0x7f1bb415d800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50008/0 (socket says 172.21.15.3:50008) 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb3fff700 1 -- 172.21.15.3:0/2113047829 learned_addr learned my addr 172.21.15.3:0/2113047829 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb37fe700 1 --2- 172.21.15.3:0/2113047829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb413c750 0x7f1bb40682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb2ffd700 1 --2- 172.21.15.3:0/2113047829 >> v2:172.21.15.153:3300/0 conn(0x7f1bb40c3170 0x7f1bb40c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb27fc700 1 -- 172.21.15.3:0/2113047829 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478237152 0 0) 0x7f1bb40c3b20 con 0x7f1bb415d3c0 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb27fc700 1 -- 172.21.15.3:0/2113047829 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b9c003570 con 0x7f1bb415d3c0 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb37fe700 1 -- 172.21.15.3:0/2113047829 >> v2:172.21.15.153:3300/0 conn(0x7f1bb40c3170 msgr2=0x7f1bb40c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb37fe700 1 --2- 172.21.15.3:0/2113047829 >> v2:172.21.15.153:3300/0 conn(0x7f1bb40c3170 0x7f1bb40c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb37fe700 1 -- 172.21.15.3:0/2113047829 >> v1:172.21.15.3:6789/0 conn(0x7f1bb415d3c0 legacy=0x7f1bb415d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb37fe700 1 -- 172.21.15.3:0/2113047829 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40696a0 con 0x7f1bb413c750 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.000+0000 7f1bb37fe700 1 --2- 172.21.15.3:0/2113047829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb413c750 0x7f1bb40682f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40098d0 tx=0x7f1ba4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8303f787e6d23599 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb27fc700 1 -- 172.21.15.3:0/2113047829 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4005c10 con 0x7f1bb413c750 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb27fc700 1 -- 172.21.15.3:0/2113047829 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4005d70 con 0x7f1bb413c750 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb27fc700 1 -- 172.21.15.3:0/2113047829 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4006240 con 0x7f1bb413c750 2023-10-06T05:02:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2113047829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb413c750 msgr2=0x7f1bb40682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2113047829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb413c750 0x7f1bb40682f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40098d0 tx=0x7f1ba4009120 comp rx=0 tx=0).stop 2023-10-06T05:02:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2113047829 shutdown_connections 2023-10-06T05:02:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2113047829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb413c750 0x7f1bb40682f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2113047829 >> v2:172.21.15.153:3300/0 conn(0x7f1bb40c3170 0x7f1bb40c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2113047829 >> 172.21.15.3:0/2113047829 conn(0x7f1bb40ce3d0 msgr2=0x7f1bb40ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2113047829 shutdown_connections 2023-10-06T05:02:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.001+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2113047829 wait complete. 2023-10-06T05:02:18.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.002+0000 7f1bb9b6e700 1 Processor -- start 2023-10-06T05:02:18.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb9b6e700 1 -- start start 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb9b6e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bb40c3170 0x7f1bb416a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb9b6e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bb413c750 0x7f1bb41633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb9b6e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb415d3c0 0x7f1bb4163900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb9b6e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb415daa0 con 0x7f1bb413c750 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb9b6e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40b2780 con 0x7f1bb40c3170 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb9b6e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40b2a60 con 0x7f1bb415d3c0 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bb413c750 0x7f1bb41633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bb413c750 0x7f1bb41633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50076/0 (socket says 172.21.15.3:50076) 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.003+0000 7f1bb2ffd700 1 -- 172.21.15.3:0/2911608197 learned_addr learned my addr 172.21.15.3:0/2911608197 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb3fff700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb415d3c0 0x7f1bb4163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb37fe700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bb40c3170 0x7f1bb416a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb2ffd700 1 -- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb415d3c0 msgr2=0x7f1bb4163900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb2ffd700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb415d3c0 0x7f1bb4163900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb2ffd700 1 -- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bb40c3170 msgr2=0x7f1bb416a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb2ffd700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bb40c3170 0x7f1bb416a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb2ffd700 1 -- 172.21.15.3:0/2911608197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba40093c0 con 0x7f1bb413c750 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb3fff700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb415d3c0 0x7f1bb4163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb37fe700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bb40c3170 0x7f1bb416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb2ffd700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bb413c750 0x7f1bb41633c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800e930 tx=0x7f1ba80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1b9b7fe700 1 -- 172.21.15.3:0/2911608197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800eed0 con 0x7f1bb413c750 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1b9b7fe700 1 -- 172.21.15.3:0/2911608197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba800c850 con 0x7f1bb413c750 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb40b2d40 con 0x7f1bb413c750 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1b9b7fe700 1 -- 172.21.15.3:0/2911608197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800cbf0 con 0x7f1bb413c750 2023-10-06T05:02:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.004+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb40b3300 con 0x7f1bb413c750 2023-10-06T05:02:18.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.005+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb40b3520 con 0x7f1bb413c750 2023-10-06T05:02:18.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.006+0000 7f1b9b7fe700 1 -- 172.21.15.3:0/2911608197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ba8013470 con 0x7f1bb413c750 2023-10-06T05:02:18.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.006+0000 7f1b9b7fe700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ba0067000 0x7f1ba00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:18.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.006+0000 7f1b9b7fe700 1 -- 172.21.15.3:0/2911608197 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f1ba8084830 con 0x7f1bb413c750 2023-10-06T05:02:18.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.007+0000 7f1bb37fe700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ba0067000 0x7f1ba00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:18.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.007+0000 7f1bb37fe700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ba0067000 0x7f1ba00694b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40b1230 tx=0x7f1ba4016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:18.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.011+0000 7f1b9b7fe700 1 -- 172.21.15.3:0/2911608197 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ba8051980 con 0x7f1bb413c750 2023-10-06T05:02:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.245+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1bb404c380 con 0x7f1bb413c750 2023-10-06T05:02:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.245+0000 7f1b9b7fe700 1 -- 172.21.15.3:0/2911608197 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1ba8054fa0 con 0x7f1bb413c750 2023-10-06T05:02:18.247 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:18.247 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ba0067000 msgr2=0x7f1ba00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ba0067000 0x7f1ba00694b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40b1230 tx=0x7f1ba4016000 comp rx=0 tx=0).stop 2023-10-06T05:02:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bb413c750 msgr2=0x7f1bb41633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bb413c750 0x7f1bb41633c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800e930 tx=0x7f1ba80091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 shutdown_connections 2023-10-06T05:02:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1ba0067000 0x7f1ba00694b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bb40c3170 0x7f1bb416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bb415d3c0 0x7f1bb4163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 --2- 172.21.15.3:0/2911608197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bb413c750 0x7f1bb41633c0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 >> 172.21.15.3:0/2911608197 conn(0x7f1bb40ce3d0 msgr2=0x7f1bb40c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 shutdown_connections 2023-10-06T05:02:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:18.247+0000 7f1bb9b6e700 1 -- 172.21.15.3:0/2911608197 wait complete. 2023-10-06T05:02:18.261 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:18.261 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:23.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.393+0000 7f421f59e700 1 Processor -- start 2023-10-06T05:02:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.393+0000 7f421f59e700 1 -- start start 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f42200b06d0 0x7f42200b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200b0bb0 con 0x7f4220159b20 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421f59e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200b0e90 con 0x7f42200b1da0 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200b1170 con 0x7f42200b06d0 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4220159b20 0x7f42200aa5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34434/0 (socket says 172.21.15.3:34434) 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421dd9b700 1 -- 172.21.15.3:0/3885413630 learned_addr learned my addr 172.21.15.3:0/3885413630 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421ed9d700 1 --2- 172.21.15.3:0/3885413630 >> v2:172.21.15.153:3300/0 conn(0x7f42200b06d0 0x7f42200b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981884081 0 0) 0x7f42200b0bb0 con 0x7f4220159b20 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.394+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f420c003570 con 0x7f4220159b20 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2725895148 0 0) 0x7f420c003570 con 0x7f4220159b20 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 >> v2:172.21.15.153:3300/0 conn(0x7f42200b06d0 msgr2=0x7f42200b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 --2- 172.21.15.3:0/3885413630 >> v2:172.21.15.153:3300/0 conn(0x7f42200b06d0 0x7f42200b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 >> v1:172.21.15.153:6789/0 conn(0x7f42200b1da0 legacy=0x7f42200b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200b1450 con 0x7f4220159b20 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f4208001470 con 0x7f4220159b20 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 >> v1:172.21.15.3:6789/0 conn(0x7f4220159b20 legacy=0x7f42200aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f420c003a60 0x7f420c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f420c004810 0x7f420c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f420c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42200b1170 con 0x7f420c0051c0 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c00c050 con 0x7f420c003a60 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c00c210 con 0x7f420c004810 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421ed9d700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f420c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421e59c700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f420c003a60 0x7f420c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421dd9b700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f420c004810 0x7f420c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421ed9d700 1 -- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f420c004810 msgr2=0x7f420c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421ed9d700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f420c004810 0x7f420c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421ed9d700 1 -- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f420c003a60 msgr2=0x7f420c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421ed9d700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f420c003a60 0x7f420c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421ed9d700 1 -- 172.21.15.3:0/3885413630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c003570 con 0x7f420c0051c0 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421dd9b700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f420c004810 0x7f420c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.395+0000 7f421ed9d700 1 -- 172.21.15.3:0/3885413630 reap_dead start 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421ed9d700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f420c00ba60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f4218009c00 tx=0x7f42180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e8f77ab9fef1bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421800e710 con 0x7f420c0051c0 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421800e890 con 0x7f420c0051c0 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421d59a700 1 -- 172.21.15.3:0/3885413630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421800ec50 con 0x7f420c0051c0 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421f59e700 1 -- 172.21.15.3:0/3885413630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 msgr2=0x7f420c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421f59e700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f420c00ba60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f4218009c00 tx=0x7f42180091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421f59e700 1 -- 172.21.15.3:0/3885413630 shutdown_connections 2023-10-06T05:02:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421f59e700 1 --2- 172.21.15.3:0/3885413630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f420c00ba60 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421f59e700 1 -- 172.21.15.3:0/3885413630 >> 172.21.15.3:0/3885413630 conn(0x7f4220099780 msgr2=0x7f422006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421f59e700 1 -- 172.21.15.3:0/3885413630 shutdown_connections 2023-10-06T05:02:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.396+0000 7f421f59e700 1 -- 172.21.15.3:0/3885413630 wait complete. 2023-10-06T05:02:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.397+0000 7f421f59e700 1 Processor -- start 2023-10-06T05:02:23.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.397+0000 7f421f59e700 1 -- start start 2023-10-06T05:02:23.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f4220153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220169a50 0x7f42200b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42200b3c30 0x7f422009cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422009a500 con 0x7f420c0051c0 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42200b4080 con 0x7f4220169a50 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422009da50 con 0x7f42200b3c30 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f4220153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f4220153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54142/0 (socket says 172.21.15.3:54142) 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421e59c700 1 -- 172.21.15.3:0/1052927673 learned_addr learned my addr 172.21.15.3:0/1052927673 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421ed9d700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42200b3c30 0x7f422009cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.398+0000 7f421dd9b700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220169a50 0x7f42200b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421e59c700 1 -- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42200b3c30 msgr2=0x7f422009cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421e59c700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42200b3c30 0x7f422009cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421e59c700 1 -- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220169a50 msgr2=0x7f42200b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:23.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421e59c700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220169a50 0x7f42200b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421e59c700 1 -- 172.21.15.3:0/1052927673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422009dd30 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421e59c700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f4220153070 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f421000c1e0 tx=0x7f4210001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f42167fc700 1 -- 172.21.15.3:0/1052927673 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421000ccc0 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f42167fc700 1 -- 172.21.15.3:0/1052927673 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421000ce20 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f42167fc700 1 -- 172.21.15.3:0/1052927673 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210015680 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422009e010 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.399+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4220169ec0 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.400+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42200b3400 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.400+0000 7f42167fc700 1 -- 172.21.15.3:0/1052927673 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4210015a30 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.401+0000 7f42167fc700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4200066f30 0x7f42000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.401+0000 7f42167fc700 1 -- 172.21.15.3:0/1052927673 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f4210085420 con 0x7f420c0051c0 2023-10-06T05:02:23.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.401+0000 7f421dd9b700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4200066f30 0x7f42000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:23.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.402+0000 7f421dd9b700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4200066f30 0x7f42000693e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4208000e10 tx=0x7f4208007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:23.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.405+0000 7f42167fc700 1 -- 172.21.15.3:0/1052927673 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42100524f0 con 0x7f420c0051c0 2023-10-06T05:02:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.641+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4220088660 con 0x7f420c0051c0 2023-10-06T05:02:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.641+0000 7f42167fc700 1 -- 172.21.15.3:0/1052927673 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4210055b10 con 0x7f420c0051c0 2023-10-06T05:02:23.643 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4200066f30 msgr2=0x7f42000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4200066f30 0x7f42000693e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4208000e10 tx=0x7f4208007040 comp rx=0 tx=0).stop 2023-10-06T05:02:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 msgr2=0x7f4220153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f4220153070 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f421000c1e0 tx=0x7f4210001b10 comp rx=0 tx=0).stop 2023-10-06T05:02:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 shutdown_connections 2023-10-06T05:02:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4200066f30 0x7f42000693e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f42200b3c30 0x7f422009cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c0051c0 0x7f4220153070 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 --2- 172.21.15.3:0/1052927673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220169a50 0x7f42200b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 >> 172.21.15.3:0/1052927673 conn(0x7f4220099780 msgr2=0x7f422006d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 shutdown_connections 2023-10-06T05:02:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:23.643+0000 7f421f59e700 1 -- 172.21.15.3:0/1052927673 wait complete. 2023-10-06T05:02:23.656 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:23.657 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:25.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:25.872 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:28.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.789+0000 7fd01210f700 1 Processor -- start 2023-10-06T05:02:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.789+0000 7fd01210f700 1 -- start start 2023-10-06T05:02:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd01210f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd00c153e90 0x7fd00c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd01210f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c0cb7c0 con 0x7fd00c153ac0 2023-10-06T05:02:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd01210f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c0cbaa0 con 0x7fd00c0d0580 2023-10-06T05:02:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd01210f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c0cbd80 con 0x7fd00c153e90 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd00c153ac0 0x7fd00c15dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34450/0 (socket says 172.21.15.3:34450) 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00b7fe700 1 -- 172.21.15.3:0/30031326 learned_addr learned my addr 172.21.15.3:0/30031326 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00affd700 1 --2- 172.21.15.3:0/30031326 >> v2:172.21.15.153:3300/0 conn(0x7fd00c153e90 0x7fd00c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670047532 0 0) 0x7fd00c0cb7c0 con 0x7fd00c153ac0 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcff4003570 con 0x7fd00c153ac0 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622260541 0 0) 0x7fd00c0cbaa0 con 0x7fd00c0d0580 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c0cb7c0 con 0x7fd00c0d0580 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1751875833 0 0) 0x7fcff4003570 con 0x7fd00c153ac0 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 >> v2:172.21.15.153:3300/0 conn(0x7fd00c153e90 msgr2=0x7fd00c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 --2- 172.21.15.3:0/30031326 >> v2:172.21.15.153:3300/0 conn(0x7fd00c153e90 0x7fd00c0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 >> v1:172.21.15.153:6789/0 conn(0x7fd00c0d0580 legacy=0x7fd00c0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.790+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c15d450 con 0x7fd00c153ac0 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fcffc0015f0 con 0x7fd00c153ac0 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 >> v1:172.21.15.3:6789/0 conn(0x7fd00c153ac0 legacy=0x7fd00c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcff4003c40 0x7fcff4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4004810 0x7fcff4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff4005210 0x7fcff400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0cbd80 con 0x7fcff4005210 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff400c130 con 0x7fcff4004810 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff400c2c0 con 0x7fcff4003c40 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff4005210 0x7fcff400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00b7fe700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcff4003c40 0x7fcff4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00affd700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4004810 0x7fcff4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 -- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcff4003c40 msgr2=0x7fcff4004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcff4003c40 0x7fcff4004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 -- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4004810 msgr2=0x7fcff4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4004810 0x7fcff4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 -- 172.21.15.3:0/30031326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4003570 con 0x7fcff4005210 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 -- 172.21.15.3:0/30031326 reap_dead start 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00bfff700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff4005210 0x7fcff400ba60 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd004002150 tx=0x7fd0040017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8c81c8fd080f12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.791+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00400c940 con 0x7fcff4005210 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00400caa0 con 0x7fcff4005210 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd00a7fc700 1 -- 172.21.15.3:0/30031326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00400cc10 con 0x7fcff4005210 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd01210f700 1 -- 172.21.15.3:0/30031326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff4005210 msgr2=0x7fcff400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd01210f700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff4005210 0x7fcff400ba60 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd004002150 tx=0x7fd0040017a0 comp rx=0 tx=0).stop 2023-10-06T05:02:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd01210f700 1 -- 172.21.15.3:0/30031326 shutdown_connections 2023-10-06T05:02:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd01210f700 1 --2- 172.21.15.3:0/30031326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff4005210 0x7fcff400ba60 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd01210f700 1 -- 172.21.15.3:0/30031326 >> 172.21.15.3:0/30031326 conn(0x7fd00c06c2f0 msgr2=0x7fd00c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd01210f700 1 -- 172.21.15.3:0/30031326 shutdown_connections 2023-10-06T05:02:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.792+0000 7fd01210f700 1 -- 172.21.15.3:0/30031326 wait complete. 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.793+0000 7fd01210f700 1 Processor -- start 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.793+0000 7fd01210f700 1 -- start start 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd01210f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4005210 0x7fd00c1690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd01210f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd00c0b36e0 0x7fd00c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd01210f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00c0b1730 0x7fd00c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd01210f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c13fb90 con 0x7fd00c0b1730 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd01210f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0c78f0 con 0x7fcff4005210 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd01210f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0c7bd0 con 0x7fd00c0b36e0 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd00bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00c0b1730 0x7fd00c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd00bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00c0b1730 0x7fd00c0b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54152/0 (socket says 172.21.15.3:54152) 2023-10-06T05:02:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd00bfff700 1 -- 172.21.15.3:0/2533385541 learned_addr learned my addr 172.21.15.3:0/2533385541 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd00affd700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd00c0b36e0 0x7fd00c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.794+0000 7fd00b7fe700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4005210 0x7fd00c1690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd00bfff700 1 -- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd00c0b36e0 msgr2=0x7fd00c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd00bfff700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd00c0b36e0 0x7fd00c0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd00bfff700 1 -- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4005210 msgr2=0x7fd00c1690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd00bfff700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4005210 0x7fd00c1690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd00bfff700 1 -- 172.21.15.3:0/2533385541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0c7eb0 con 0x7fd00c0b1730 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd00bfff700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00c0b1730 0x7fd00c0b1bb0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0ce100 tx=0x7fd004015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fcff37fe700 1 -- 172.21.15.3:0/2533385541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00400c940 con 0x7fd00c0b1730 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fcff37fe700 1 -- 172.21.15.3:0/2533385541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00400cc10 con 0x7fd00c0b1730 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c0c8190 con 0x7fd00c0b1730 2023-10-06T05:02:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fcff37fe700 1 -- 172.21.15.3:0/2533385541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00401e400 con 0x7fd00c0b1730 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.795+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c0b5040 con 0x7fd00c0b1730 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.796+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c0c83b0 con 0x7fd00c0b1730 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.796+0000 7fcff37fe700 1 -- 172.21.15.3:0/2533385541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd004019070 con 0x7fd00c0b1730 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.797+0000 7fcff37fe700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcff8066f30 0x7fcff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.797+0000 7fcff37fe700 1 -- 172.21.15.3:0/2533385541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fd0040857c0 con 0x7fd00c0b1730 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.797+0000 7fd00b7fe700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcff8066f30 0x7fcff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.798+0000 7fd00b7fe700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcff8066f30 0x7fcff80693e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0cfbb0 tx=0x7fcffc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:28.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:28.801+0000 7fcff37fe700 1 -- 172.21.15.3:0/2533385541 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd004052890 con 0x7fd00c0b1730 2023-10-06T05:02:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.036+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd00c04c380 con 0x7fd00c0b1730 2023-10-06T05:02:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.036+0000 7fcff37fe700 1 -- 172.21.15.3:0/2533385541 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd004017080 con 0x7fd00c0b1730 2023-10-06T05:02:29.038 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:29.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcff8066f30 msgr2=0x7fcff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:29.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcff8066f30 0x7fcff80693e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0cfbb0 tx=0x7fcffc009040 comp rx=0 tx=0).stop 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00c0b1730 msgr2=0x7fd00c0b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00c0b1730 0x7fd00c0b1bb0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0ce100 tx=0x7fd004015000 comp rx=0 tx=0).stop 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 shutdown_connections 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fcff8066f30 0x7fcff80693e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff4005210 0x7fd00c1690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.038+0000 7fd01210f700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00c0b1730 0x7fd00c0b1bb0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.039+0000 7fd01210f700 1 --2- 172.21.15.3:0/2533385541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd00c0b36e0 0x7fd00c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.039+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 >> 172.21.15.3:0/2533385541 conn(0x7fd00c06c2f0 msgr2=0x7fd00c159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.039+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 shutdown_connections 2023-10-06T05:02:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:29.039+0000 7fd01210f700 1 -- 172.21.15.3:0/2533385541 wait complete. 2023-10-06T05:02:29.052 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:29.052 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:34.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.180+0000 7f84b7962700 1 Processor -- start 2023-10-06T05:02:34.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.180+0000 7f84b7962700 1 -- start start 2023-10-06T05:02:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b7962700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84b01596d0 0x7f84b00ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b7962700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f84b00b18c0 0x7f84b0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b7962700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b00af950 con 0x7f84b00ae560 2023-10-06T05:02:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b7962700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b00b0df0 con 0x7f84b01596d0 2023-10-06T05:02:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b7962700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b00ad3b0 con 0x7f84b00b18c0 2023-10-06T05:02:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b56fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84b01596d0 0x7f84b00ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b4efd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f84b00ae560 0x7f84b00b11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53144/0 (socket says 172.21.15.3:53144) 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b56fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84b01596d0 0x7f84b00ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60452/0 (socket says 172.21.15.3:60452) 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b4efd700 1 -- 172.21.15.3:0/245143479 learned_addr learned my addr 172.21.15.3:0/245143479 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b5eff700 1 --2- 172.21.15.3:0/245143479 >> v2:172.21.15.153:3300/0 conn(0x7f84b00b18c0 0x7f84b0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b56fe700 1 -- 172.21.15.3:0/245143479 >> v2:172.21.15.153:3300/0 conn(0x7f84b00b18c0 msgr2=0x7f84b0159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b56fe700 1 --2- 172.21.15.3:0/245143479 >> v2:172.21.15.153:3300/0 conn(0x7f84b00b18c0 0x7f84b0159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b56fe700 1 -- 172.21.15.3:0/245143479 >> v1:172.21.15.3:6789/0 conn(0x7f84b00ae560 legacy=0x7f84b00b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84affff700 1 -- 172.21.15.3:0/245143479 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60467531 0 0) 0x7f84b00af950 con 0x7f84b00ae560 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.181+0000 7f84b56fe700 1 -- 172.21.15.3:0/245143479 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b00af020 con 0x7f84b01596d0 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b56fe700 1 --2- 172.21.15.3:0/245143479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b00ae020 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f84b00b66a0 tx=0x7f84a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4480ae99268c1017 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84affff700 1 -- 172.21.15.3:0/245143479 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000b6b0 con 0x7f84b01596d0 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84affff700 1 -- 172.21.15.3:0/245143479 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a000b810 con 0x7f84b01596d0 2023-10-06T05:02:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84affff700 1 -- 172.21.15.3:0/245143479 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000bbb0 con 0x7f84b01596d0 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 -- 172.21.15.3:0/245143479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 msgr2=0x7f84b00ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 --2- 172.21.15.3:0/245143479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b00ae020 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f84b00b66a0 tx=0x7f84a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 -- 172.21.15.3:0/245143479 shutdown_connections 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 --2- 172.21.15.3:0/245143479 >> v2:172.21.15.153:3300/0 conn(0x7f84b00b18c0 0x7f84b0159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 --2- 172.21.15.3:0/245143479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b00ae020 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 -- 172.21.15.3:0/245143479 >> 172.21.15.3:0/245143479 conn(0x7f84b0099910 msgr2=0x7f84b006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 -- 172.21.15.3:0/245143479 shutdown_connections 2023-10-06T05:02:34.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.182+0000 7f84b7962700 1 -- 172.21.15.3:0/245143479 wait complete. 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 Processor -- start 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 -- start start 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84b00ae560 0x7f84b014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84b00b18c0 0x7f84b0169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b0169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b015bba0 con 0x7f84b01596d0 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b015c720 con 0x7f84b00b18c0 2023-10-06T05:02:34.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.184+0000 7f84b7962700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b015bdf0 con 0x7f84b00ae560 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b0169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b0169dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60458/0 (socket says 172.21.15.3:60458) 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 -- 172.21.15.3:0/1360126953 learned_addr learned my addr 172.21.15.3:0/1360126953 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b4efd700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84b00b18c0 0x7f84b0169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b56fe700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84b00ae560 0x7f84b014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 -- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84b00ae560 msgr2=0x7f84b014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84b00ae560 0x7f84b014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 -- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84b00b18c0 msgr2=0x7f84b0169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84b00b18c0 0x7f84b0169880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 -- 172.21.15.3:0/1360126953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a0009420 con 0x7f84b01596d0 2023-10-06T05:02:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b5eff700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b0169dc0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f84a8007f30 tx=0x7f84a80098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84ad7fa700 1 -- 172.21.15.3:0/1360126953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800ecf0 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84ad7fa700 1 -- 172.21.15.3:0/1360126953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a800ee50 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b015c0d0 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84ad7fa700 1 -- 172.21.15.3:0/1360126953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8014670 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.185+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b014f610 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.186+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b015c8d0 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.187+0000 7f84ad7fa700 1 -- 172.21.15.3:0/1360126953 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84a80149e0 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.187+0000 7f84ad7fa700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f849c066f30 0x7f849c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.187+0000 7f84ad7fa700 1 -- 172.21.15.3:0/1360126953 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f84a8084e50 con 0x7f84b01596d0 2023-10-06T05:02:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.187+0000 7f84b56fe700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f849c066f30 0x7f849c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:34.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.188+0000 7f84b56fe700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f849c066f30 0x7f849c0693e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f84a00093f0 tx=0x7f84a0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.191+0000 7f84ad7fa700 1 -- 172.21.15.3:0/1360126953 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84b015c8d0 con 0x7f84b01596d0 2023-10-06T05:02:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.428+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f84b015c890 con 0x7f84b01596d0 2023-10-06T05:02:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.429+0000 7f84ad7fa700 1 -- 172.21.15.3:0/1360126953 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f84a8051f20 con 0x7f84b01596d0 2023-10-06T05:02:34.431 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:34.431 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f849c066f30 msgr2=0x7f849c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f849c066f30 0x7f849c0693e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f84a00093f0 tx=0x7f84a0017000 comp rx=0 tx=0).stop 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 msgr2=0x7f84b0169dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b0169dc0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f84a8007f30 tx=0x7f84a80098e0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 shutdown_connections 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f849c066f30 0x7f849c0693e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b01596d0 0x7f84b0169dc0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84b00ae560 0x7f84b014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 --2- 172.21.15.3:0/1360126953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84b00b18c0 0x7f84b0169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 >> 172.21.15.3:0/1360126953 conn(0x7f84b0099910 msgr2=0x7f84b006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 shutdown_connections 2023-10-06T05:02:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:34.431+0000 7f84b7962700 1 -- 172.21.15.3:0/1360126953 wait complete. 2023-10-06T05:02:34.444 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:34.444 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:39.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.574+0000 7fc9f133e700 1 Processor -- start 2023-10-06T05:02:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.574+0000 7fc9f133e700 1 -- start start 2023-10-06T05:02:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.574+0000 7fc9f133e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec0c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.574+0000 7fc9f133e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9ec0ae6c0 0x7fc9ec0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.574+0000 7fc9f133e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ec06ba10 con 0x7fc9ec06a920 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.574+0000 7fc9f133e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec06ae00 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.574+0000 7fc9f133e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec06b0e0 con 0x7fc9ec1565f0 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9ec0ae6c0 0x7fc9ec0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9ea7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc9ec06a920 0x7fc9ec0adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53148/0 (socket says 172.21.15.3:53148) 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9ec0ae6c0 0x7fc9ec0c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60462/0 (socket says 172.21.15.3:60462) 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9ea7fc700 1 -- 172.21.15.3:0/1675577702 learned_addr learned my addr 172.21.15.3:0/1675577702 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eaffd700 1 --2- 172.21.15.3:0/1675577702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec0c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eb7fe700 1 -- 172.21.15.3:0/1675577702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 msgr2=0x7fc9ec0c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eb7fe700 1 --2- 172.21.15.3:0/1675577702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec0c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eb7fe700 1 -- 172.21.15.3:0/1675577702 >> v1:172.21.15.3:6789/0 conn(0x7fc9ec06a920 legacy=0x7fc9ec0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eb7fe700 1 -- 172.21.15.3:0/1675577702 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec06b3c0 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9e9ffb700 1 -- 172.21.15.3:0/1675577702 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640775061 0 0) 0x7fc9ec06ba10 con 0x7fc9ec06a920 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eaffd700 1 --2- 172.21.15.3:0/1675577702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec0c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9eb7fe700 1 --2- 172.21.15.3:0/1675577702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c9ee0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40098e0 tx=0x7fc9e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd23c760fb621ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9e9ffb700 1 -- 172.21.15.3:0/1675577702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4009e80 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9e9ffb700 1 -- 172.21.15.3:0/1675577702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e400b860 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.575+0000 7fc9e9ffb700 1 -- 172.21.15.3:0/1675577702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e400bbe0 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 -- 172.21.15.3:0/1675577702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 msgr2=0x7fc9ec0c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 --2- 172.21.15.3:0/1675577702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c9ee0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40098e0 tx=0x7fc9e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 -- 172.21.15.3:0/1675577702 shutdown_connections 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 --2- 172.21.15.3:0/1675577702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c9ee0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 --2- 172.21.15.3:0/1675577702 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec0c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 -- 172.21.15.3:0/1675577702 >> 172.21.15.3:0/1675577702 conn(0x7fc9ec06cd80 msgr2=0x7fc9ec0cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 -- 172.21.15.3:0/1675577702 shutdown_connections 2023-10-06T05:02:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.576+0000 7fc9f133e700 1 -- 172.21.15.3:0/1675577702 wait complete. 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.577+0000 7fc9f133e700 1 Processor -- start 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.577+0000 7fc9f133e700 1 -- start start 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9f133e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9ec06a920 0x7fc9ec06e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9f133e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9f133e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9f133e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c47d0 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9f133e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0b0550 con 0x7fc9ec06a920 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9f133e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0b0830 con 0x7fc9ec1565f0 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9ea7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9ea7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60472/0 (socket says 172.21.15.3:60472) 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9ea7fc700 1 -- 172.21.15.3:0/2225434211 learned_addr learned my addr 172.21.15.3:0/2225434211 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9eaffd700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9ec06a920 0x7fc9ec06e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9eb7fe700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9ea7fc700 1 -- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 msgr2=0x7fc9ec150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.578+0000 7fc9ea7fc700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9ea7fc700 1 -- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9ec06a920 msgr2=0x7fc9ec06e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9ea7fc700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9ec06a920 0x7fc9ec06e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9ea7fc700 1 -- 172.21.15.3:0/2225434211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4009420 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9ea7fc700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c4290 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c810 tx=0x7fc9d4007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9e2ffd700 1 -- 172.21.15.3:0/2225434211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4005c00 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9e2ffd700 1 -- 172.21.15.3:0/2225434211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4005d60 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec16a030 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9e2ffd700 1 -- 172.21.15.3:0/2225434211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4007300 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.579+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec151250 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.580+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec16a420 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.580+0000 7fc9e2ffd700 1 -- 172.21.15.3:0/2225434211 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9d401b3b0 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.581+0000 7fc9e2ffd700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9cc066f30 0x7fc9cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.581+0000 7fc9e2ffd700 1 -- 172.21.15.3:0/2225434211 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc9d4083c10 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.581+0000 7fc9eaffd700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9cc066f30 0x7fc9cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.582+0000 7fc9eaffd700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9cc066f30 0x7fc9cc0693e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00f600 tx=0x7fc9dc0077b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:39.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.585+0000 7fc9e2ffd700 1 -- 172.21.15.3:0/2225434211 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9d4050ce0 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.827+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc9ec0b09a0 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.827+0000 7fc9e2ffd700 1 -- 172.21.15.3:0/2225434211 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc9d4017080 con 0x7fc9ec0ae6c0 2023-10-06T05:02:39.829 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:39.830 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:02:39.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9cc066f30 msgr2=0x7fc9cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:39.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9cc066f30 0x7fc9cc0693e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00f600 tx=0x7fc9dc0077b0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 msgr2=0x7fc9ec0c4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c4290 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c810 tx=0x7fc9d4007db0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 shutdown_connections 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc9cc066f30 0x7fc9cc0693e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9ec06a920 0x7fc9ec06e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc9ec1565f0 0x7fc9ec150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 --2- 172.21.15.3:0/2225434211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ec0ae6c0 0x7fc9ec0c4290 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 >> 172.21.15.3:0/2225434211 conn(0x7fc9ec06cd80 msgr2=0x7fc9ec0cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:39.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 shutdown_connections 2023-10-06T05:02:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:39.830+0000 7fc9f133e700 1 -- 172.21.15.3:0/2225434211 wait complete. 2023-10-06T05:02:39.843 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-06T05:02:39.843 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-10-06T05:02:44.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:02:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.975+0000 7f699f508700 1 Processor -- start 2023-10-06T05:02:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.975+0000 7f699f508700 1 -- start start 2023-10-06T05:02:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699f508700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f69981596d0 0x7f69980ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699f508700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f69980b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699f508700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980b1b50 con 0x7f69980b16f0 2023-10-06T05:02:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699f508700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69980af950 con 0x7f69981596d0 2023-10-06T05:02:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699f508700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980ad3b0 con 0x7f69980ae560 2023-10-06T05:02:44.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699caa3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f69980b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:44.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699d2a4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f69981596d0 0x7f69980ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:44.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699caa3700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f69980b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:54620/0 (socket says 172.21.15.3:54620) 2023-10-06T05:02:44.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699caa3700 1 -- 172.21.15.3:0/1509751748 learned_addr learned my addr 172.21.15.3:0/1509751748 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:44.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699daa5700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f69980b16f0 0x7f69980afde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:38758/0 (socket says 172.21.15.3:38758) 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699caa3700 1 -- 172.21.15.3:0/1509751748 >> v2:172.21.15.153:3300/0 conn(0x7f69981596d0 msgr2=0x7f69980ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699caa3700 1 --2- 172.21.15.3:0/1509751748 >> v2:172.21.15.153:3300/0 conn(0x7f69981596d0 0x7f69980ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.976+0000 7f699caa3700 1 -- 172.21.15.3:0/1509751748 >> v1:172.21.15.153:6789/0 conn(0x7f69980b16f0 legacy=0x7f69980afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.977+0000 7f699caa3700 1 -- 172.21.15.3:0/1509751748 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980af020 con 0x7f69980ae560 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.977+0000 7f699d2a4700 1 --2- 172.21.15.3:0/1509751748 >> v2:172.21.15.153:3300/0 conn(0x7f69981596d0 0x7f69980ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.977+0000 7f699caa3700 1 --2- 172.21.15.3:0/1509751748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f69980b11b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f69880099b0 tx=0x7f6988009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad5cfe88b357341 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.977+0000 7f698ffff700 1 -- 172.21.15.3:0/1509751748 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988009ea0 con 0x7f69980ae560 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.977+0000 7f698ffff700 1 -- 172.21.15.3:0/1509751748 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698800b860 con 0x7f69980ae560 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.977+0000 7f698ffff700 1 -- 172.21.15.3:0/1509751748 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698800bc00 con 0x7f69980ae560 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 -- 172.21.15.3:0/1509751748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 msgr2=0x7f69980b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 --2- 172.21.15.3:0/1509751748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f69980b11b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f69880099b0 tx=0x7f6988009250 comp rx=0 tx=0).stop 2023-10-06T05:02:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 -- 172.21.15.3:0/1509751748 shutdown_connections 2023-10-06T05:02:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 --2- 172.21.15.3:0/1509751748 >> v2:172.21.15.153:3300/0 conn(0x7f69981596d0 0x7f69980ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 --2- 172.21.15.3:0/1509751748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f69980b11b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 -- 172.21.15.3:0/1509751748 >> 172.21.15.3:0/1509751748 conn(0x7f6998099910 msgr2=0x7f699806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 -- 172.21.15.3:0/1509751748 shutdown_connections 2023-10-06T05:02:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.978+0000 7f699f508700 1 -- 172.21.15.3:0/1509751748 wait complete. 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.979+0000 7f699f508700 1 Processor -- start 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.979+0000 7f699f508700 1 -- start start 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699f508700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f6998154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699f508700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69980b16f0 0x7f699806e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699f508700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69981596d0 0x7f699806ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699f508700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980d0a40 con 0x7f69981596d0 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699f508700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69981546d0 con 0x7f69980b16f0 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699f508700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980cfe30 con 0x7f69980ae560 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69981596d0 0x7f699806ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69981596d0 0x7f699806ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37568/0 (socket says 172.21.15.3:37568) 2023-10-06T05:02:44.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 -- 172.21.15.3:0/1358242839 learned_addr learned my addr 172.21.15.3:0/1358242839 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699caa3700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69980b16f0 0x7f699806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699d2a4700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f6998154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 -- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 msgr2=0x7f6998154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f6998154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 -- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69980b16f0 msgr2=0x7f699806e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69980b16f0 0x7f699806e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 -- 172.21.15.3:0/1358242839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6994009420 con 0x7f69981596d0 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699caa3700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69980b16f0 0x7f699806e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.980+0000 7f699daa5700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69981596d0 0x7f699806ec70 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f699000cd00 tx=0x7f6990001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.981+0000 7f698d7fa700 1 -- 172.21.15.3:0/1358242839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990005df0 con 0x7f69981596d0 2023-10-06T05:02:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.981+0000 7f698d7fa700 1 -- 172.21.15.3:0/1358242839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699000a960 con 0x7f69981596d0 2023-10-06T05:02:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.981+0000 7f698d7fa700 1 -- 172.21.15.3:0/1358242839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699000ad00 con 0x7f69981596d0 2023-10-06T05:02:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.981+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69880094f0 con 0x7f69981596d0 2023-10-06T05:02:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.981+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69980d03f0 con 0x7f69981596d0 2023-10-06T05:02:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.982+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69980d0bb0 con 0x7f69981596d0 2023-10-06T05:02:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.982+0000 7f698d7fa700 1 -- 172.21.15.3:0/1358242839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6990007300 con 0x7f69981596d0 2023-10-06T05:02:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.982+0000 7f698d7fa700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f697c066f30 0x7f697c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.983+0000 7f698d7fa700 1 -- 172.21.15.3:0/1358242839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f69900846a0 con 0x7f69981596d0 2023-10-06T05:02:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.983+0000 7f699d2a4700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f697c066f30 0x7f697c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.983+0000 7f699d2a4700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f697c066f30 0x7f697c0693e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f69940093f0 tx=0x7f6994008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:44.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:44.987+0000 7f698d7fa700 1 -- 172.21.15.3:0/1358242839 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69900517f0 con 0x7f69981596d0 2023-10-06T05:02:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.227+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f69980b1b50 con 0x7f69981596d0 2023-10-06T05:02:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.227+0000 7f698d7fa700 1 -- 172.21.15.3:0/1358242839 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+199 (secure 0 0 0) 0x7f69980b1b50 con 0x7f69981596d0 2023-10-06T05:02:45.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:45.229 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_LATE_RELEASE":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to capability release","count":1},"muted":false}},"mutes":[]} 2023-10-06T05:02:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f697c066f30 msgr2=0x7f697c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f697c066f30 0x7f697c0693e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f69940093f0 tx=0x7f6994008040 comp rx=0 tx=0).stop 2023-10-06T05:02:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69981596d0 msgr2=0x7f699806ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69981596d0 0x7f699806ec70 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f699000cd00 tx=0x7f6990001b10 comp rx=0 tx=0).stop 2023-10-06T05:02:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 shutdown_connections 2023-10-06T05:02:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f697c066f30 0x7f697c0693e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69981596d0 0x7f699806ec70 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69980ae560 0x7f6998154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 --2- 172.21.15.3:0/1358242839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69980b16f0 0x7f699806e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 >> 172.21.15.3:0/1358242839 conn(0x7f6998099910 msgr2=0x7f6998155f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 shutdown_connections 2023-10-06T05:02:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.229+0000 7f699f508700 1 -- 172.21.15.3:0/1358242839 wait complete. 2023-10-06T05:02:45.242 DEBUG:tasks.ceph_test_case:wait_until_true: success in 60s and 0 retries 2023-10-06T05:02:45.343 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T05:02:45.343 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:02:45.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:45.344 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 down 2023-10-06T05:02:45.376 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:02:45.377 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-10-06T05:02:45.377 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:02:45.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:02:45.449 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:02:45.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:02:45.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:45.449 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:02:45.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:02:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.663+0000 7f97055c6700 1 Processor -- start 2023-10-06T05:02:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.664+0000 7f97055c6700 1 -- start start 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.664+0000 7f97055c6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97000cf860 0x7f970015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.664+0000 7f97055c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97000b37f0 0x7f97000b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.664+0000 7f97055c6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97000ce3c0 con 0x7f9700159760 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.664+0000 7f97055c6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970015e3b0 con 0x7f97000b37f0 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.664+0000 7f97055c6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000cd890 con 0x7f97000cf860 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.664+0000 7f96ff7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97000b37f0 0x7f97000b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96feffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9700159760 0x7f970015d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60668/0 (socket says 172.21.15.3:60668) 2023-10-06T05:02:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96feffd700 1 -- 172.21.15.3:0/960387489 learned_addr learned my addr 172.21.15.3:0/960387489 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96fe7fc700 1 --2- 172.21.15.3:0/960387489 >> v2:172.21.15.153:3300/0 conn(0x7f97000cf860 0x7f970015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96ff7fe700 1 -- 172.21.15.3:0/960387489 >> v2:172.21.15.153:3300/0 conn(0x7f97000cf860 msgr2=0x7f970015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96ff7fe700 1 --2- 172.21.15.3:0/960387489 >> v2:172.21.15.153:3300/0 conn(0x7f97000cf860 0x7f970015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96ff7fe700 1 -- 172.21.15.3:0/960387489 >> v1:172.21.15.3:6789/0 conn(0x7f9700159760 legacy=0x7f970015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96ff7fe700 1 -- 172.21.15.3:0/960387489 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000cda90 con 0x7f97000b37f0 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96fe7fc700 1 --2- 172.21.15.3:0/960387489 >> v2:172.21.15.153:3300/0 conn(0x7f97000cf860 0x7f970015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96ff7fe700 1 --2- 172.21.15.3:0/960387489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f97000b3c10 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9700089f40 tx=0x7f96f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b080f2ed0413b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96fdffb700 1 -- 172.21.15.3:0/960387489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000e070 con 0x7f97000b37f0 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96fdffb700 1 -- 172.21.15.3:0/960387489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f000bbf0 con 0x7f97000b37f0 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.665+0000 7f96fdffb700 1 -- 172.21.15.3:0/960387489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0012430 con 0x7f97000b37f0 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 -- 172.21.15.3:0/960387489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 msgr2=0x7f97000b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 --2- 172.21.15.3:0/960387489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f97000b3c10 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9700089f40 tx=0x7f96f0009780 comp rx=0 tx=0).stop 2023-10-06T05:02:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 -- 172.21.15.3:0/960387489 shutdown_connections 2023-10-06T05:02:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 --2- 172.21.15.3:0/960387489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f97000b3c10 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 --2- 172.21.15.3:0/960387489 >> v2:172.21.15.153:3300/0 conn(0x7f97000cf860 0x7f970015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 -- 172.21.15.3:0/960387489 >> 172.21.15.3:0/960387489 conn(0x7f9700087fe0 msgr2=0x7f97000998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 -- 172.21.15.3:0/960387489 shutdown_connections 2023-10-06T05:02:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.666+0000 7f97055c6700 1 -- 172.21.15.3:0/960387489 wait complete. 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.667+0000 7f97055c6700 1 Processor -- start 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.667+0000 7f97055c6700 1 -- start start 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f97055c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f9700157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f97055c6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97000cf860 0x7f9700158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f97055c6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9700159760 0x7f9700158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f97055c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97001583e0 con 0x7f97000b37f0 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f97055c6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970009a8f0 con 0x7f97000cf860 2023-10-06T05:02:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f97055c6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970016db60 con 0x7f9700159760 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f9700157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f9700157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37582/0 (socket says 172.21.15.3:37582) 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 -- 172.21.15.3:0/3701927018 learned_addr learned my addr 172.21.15.3:0/3701927018 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 -- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9700159760 msgr2=0x7f9700158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96fe7fc700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97000cf860 0x7f9700158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96ff7fe700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9700159760 0x7f9700158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9700159760 0x7f9700158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 -- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97000cf860 msgr2=0x7f9700158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97000cf860 0x7f9700158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 -- 172.21.15.3:0/3701927018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0009a20 con 0x7f97000b37f0 2023-10-06T05:02:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96ff7fe700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9700159760 0x7f9700158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96fe7fc700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97000cf860 0x7f9700158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.668+0000 7f96feffd700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f9700157ea0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c590 tx=0x7f96f8007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.669+0000 7f96f6ffd700 1 -- 172.21.15.3:0/3701927018 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80072e0 con 0x7f97000b37f0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.669+0000 7f96f6ffd700 1 -- 172.21.15.3:0/3701927018 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8007460 con 0x7f97000b37f0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.669+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970009a2e0 con 0x7f97000b37f0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.669+0000 7f96f6ffd700 1 -- 172.21.15.3:0/3701927018 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80057f0 con 0x7f97000b37f0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.669+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970015cf80 con 0x7f97000b37f0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.670+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970004c380 con 0x7f97000b37f0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.670+0000 7f96f6ffd700 1 -- 172.21.15.3:0/3701927018 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96f8005be0 con 0x7f97000b37f0 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.670+0000 7f96f6ffd700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96e0066f30 0x7f96e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.671+0000 7f96f6ffd700 1 -- 172.21.15.3:0/3701927018 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f96f8083a80 con 0x7f97000b37f0 2023-10-06T05:02:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.671+0000 7f96fe7fc700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96e0066f30 0x7f96e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.671+0000 7f96fe7fc700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96e0066f30 0x7f96e00693e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f970016cee0 tx=0x7f96e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.675+0000 7f96f6ffd700 1 -- 172.21.15.3:0/3701927018 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96f804b5a0 con 0x7f97000b37f0 2023-10-06T05:02:45.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.911+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97000cdd70 con 0x7f97000b37f0 2023-10-06T05:02:45.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.911+0000 7f96f6ffd700 1 -- 172.21.15.3:0/3701927018 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f96f8017020 con 0x7f97000b37f0 2023-10-06T05:02:45.913 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:45.913 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:02:43.749485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":23,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96e0066f30 msgr2=0x7f96e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96e0066f30 0x7f96e00693e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f970016cee0 tx=0x7f96e8008040 comp rx=0 tx=0).stop 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 msgr2=0x7f9700157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f9700157ea0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c590 tx=0x7f96f8007e40 comp rx=0 tx=0).stop 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 shutdown_connections 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f96e0066f30 0x7f96e00693e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9700159760 0x7f9700158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97000b37f0 0x7f9700157ea0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 --2- 172.21.15.3:0/3701927018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97000cf860 0x7f9700158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 >> 172.21.15.3:0/3701927018 conn(0x7f9700087fe0 msgr2=0x7f97000afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 shutdown_connections 2023-10-06T05:02:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:45.913+0000 7f97055c6700 1 -- 172.21.15.3:0/3701927018 wait complete. 2023-10-06T05:02:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 90 2023-10-06T05:02:45.927 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session evict 8625 2023-10-06T05:02:46.836 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:02:48.015 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:02:48.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:02:48.058 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:02:48.108 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:02:48.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:02:48.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:48.109 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:02:48.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:02:48.128 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:02:48.128 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:02:48.129 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2023-10-06T05:02:48.129 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:02:48.129 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T05:02:48.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi153.stderr:ceph-fuse[103188]: fuse finished with error 0 and tester_r 0 2023-10-06T05:02:48.170 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:02:48.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:48.170 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:02:48.227 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:02:48.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:48.227 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:02:48.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:02:48.246 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:02:48.246 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:02:48.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:48.246 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:02:48.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:02:48.282 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:02:48.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:02:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.417+0000 7f1755193700 1 Processor -- start 2023-10-06T05:02:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.417+0000 7f1755193700 1 -- start start 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f1755193700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1750159c70 0x7f17500cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f1755193700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f175006d400 0x7f17500c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f1755193700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175006e5d0 con 0x7f17500c8f40 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f1755193700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17500cde20 con 0x7f175006d400 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f1755193700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175006d9c0 con 0x7f1750159c70 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f174ed9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1750159c70 0x7f17500cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f174e59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f175006d400 0x7f17500c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f174f59e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f17500c8f40 0x7f175006ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:38770/0 (socket says 172.21.15.3:38770) 2023-10-06T05:02:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.418+0000 7f174f59e700 1 -- 172.21.15.3:0/3588023595 learned_addr learned my addr 172.21.15.3:0/3588023595 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174dd9b700 1 -- 172.21.15.3:0/3588023595 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716228539 0 0) 0x7f175006e5d0 con 0x7f17500c8f40 2023-10-06T05:02:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174dd9b700 1 -- 172.21.15.3:0/3588023595 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f173c003570 con 0x7f17500c8f40 2023-10-06T05:02:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174ed9d700 1 -- 172.21.15.3:0/3588023595 >> v2:172.21.15.153:3300/0 conn(0x7f175006d400 msgr2=0x7f17500c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174ed9d700 1 --2- 172.21.15.3:0/3588023595 >> v2:172.21.15.153:3300/0 conn(0x7f175006d400 0x7f17500c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174ed9d700 1 -- 172.21.15.3:0/3588023595 >> v1:172.21.15.153:6789/0 conn(0x7f17500c8f40 legacy=0x7f175006ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174ed9d700 1 -- 172.21.15.3:0/3588023595 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175006dca0 con 0x7f1750159c70 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174e59c700 1 --2- 172.21.15.3:0/3588023595 >> v2:172.21.15.153:3300/0 conn(0x7f175006d400 0x7f17500c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174ed9d700 1 --2- 172.21.15.3:0/3588023595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1750159c70 0x7f17500cd8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1740009eb0 tx=0x7f1740009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b41e60e19c6494e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174dd9b700 1 -- 172.21.15.3:0/3588023595 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174000e070 con 0x7f1750159c70 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174dd9b700 1 -- 172.21.15.3:0/3588023595 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174000bb80 con 0x7f1750159c70 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.419+0000 7f174dd9b700 1 -- 172.21.15.3:0/3588023595 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740012430 con 0x7f1750159c70 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 -- 172.21.15.3:0/3588023595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1750159c70 msgr2=0x7f17500cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 --2- 172.21.15.3:0/3588023595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1750159c70 0x7f17500cd8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1740009eb0 tx=0x7f1740009780 comp rx=0 tx=0).stop 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 -- 172.21.15.3:0/3588023595 shutdown_connections 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 --2- 172.21.15.3:0/3588023595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1750159c70 0x7f17500cd8e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 --2- 172.21.15.3:0/3588023595 >> v2:172.21.15.153:3300/0 conn(0x7f175006d400 0x7f17500c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 -- 172.21.15.3:0/3588023595 >> 172.21.15.3:0/3588023595 conn(0x7f175006cad0 msgr2=0x7f17500ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 -- 172.21.15.3:0/3588023595 shutdown_connections 2023-10-06T05:02:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.420+0000 7f1755193700 1 -- 172.21.15.3:0/3588023595 wait complete. 2023-10-06T05:02:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.421+0000 7f1755193700 1 Processor -- start 2023-10-06T05:02:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.421+0000 7f1755193700 1 -- start start 2023-10-06T05:02:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f1755193700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175006d400 0x7f17500b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f1755193700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17500c8f40 0x7f175014c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f1755193700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1750159c70 0x7f1750155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f1755193700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17500c6460 con 0x7f175006d400 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f1755193700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17500b29e0 con 0x7f1750159c70 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f1755193700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17500c5850 con 0x7f17500c8f40 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175006d400 0x7f17500b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175006d400 0x7f17500b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37584/0 (socket says 172.21.15.3:37584) 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 -- 172.21.15.3:0/2469944497 learned_addr learned my addr 172.21.15.3:0/2469944497 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174f59e700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1750159c70 0x7f1750155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174e59c700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17500c8f40 0x7f175014c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 -- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17500c8f40 msgr2=0x7f175014c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17500c8f40 0x7f175014c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 -- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1750159c70 msgr2=0x7f1750155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1750159c70 0x7f1750155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.422+0000 7f174ed9d700 1 -- 172.21.15.3:0/2469944497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738009420 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.423+0000 7f174ed9d700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175006d400 0x7f17500b3650 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f174000b610 tx=0x7f1740019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.423+0000 7f1746ffd700 1 -- 172.21.15.3:0/2469944497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740012c00 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.423+0000 7f1746ffd700 1 -- 172.21.15.3:0/2469944497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740012d80 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.423+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1740009a20 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.423+0000 7f1746ffd700 1 -- 172.21.15.3:0/2469944497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17400276c0 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.423+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17500c5e10 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.424+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175004c380 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.424+0000 7f1746ffd700 1 -- 172.21.15.3:0/2469944497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1740024070 con 0x7f175006d400 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.425+0000 7f1746ffd700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1730066f30 0x7f17300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.425+0000 7f1746ffd700 1 -- 172.21.15.3:0/2469944497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1740086cc0 con 0x7f175006d400 2023-10-06T05:02:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.425+0000 7f174e59c700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1730066f30 0x7f17300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.425+0000 7f174e59c700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1730066f30 0x7f17300693e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f17500b4320 tx=0x7f17380091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:48.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.429+0000 7f1746ffd700 1 -- 172.21.15.3:0/2469944497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1740053d70 con 0x7f175006d400 2023-10-06T05:02:48.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.666+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f175006df80 con 0x7f175006d400 2023-10-06T05:02:48.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.667+0000 7f1746ffd700 1 -- 172.21.15.3:0/2469944497 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f174002c090 con 0x7f175006d400 2023-10-06T05:02:48.669 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:48.669 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:02:47.821136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":24,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1730066f30 msgr2=0x7f17300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1730066f30 0x7f17300693e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f17500b4320 tx=0x7f17380091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175006d400 msgr2=0x7f17500b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175006d400 0x7f17500b3650 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f174000b610 tx=0x7f1740019000 comp rx=0 tx=0).stop 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 shutdown_connections 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1730066f30 0x7f17300693e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1750159c70 0x7f1750155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175006d400 0x7f17500b3650 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 --2- 172.21.15.3:0/2469944497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17500c8f40 0x7f175014c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 >> 172.21.15.3:0/2469944497 conn(0x7f175006cad0 msgr2=0x7f17500c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 shutdown_connections 2023-10-06T05:02:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.669+0000 7f1755193700 1 -- 172.21.15.3:0/2469944497 wait complete. 2023-10-06T05:02:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 91 2023-10-06T05:02:48.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:02:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.800+0000 7ff7d1aa6700 1 Processor -- start 2023-10-06T05:02:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.800+0000 7ff7d1aa6700 1 -- start start 2023-10-06T05:02:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d1aa6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7cc0b1da0 0x7ff7cc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d1aa6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc159b20 0x7ff7cc0b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d1aa6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc0b2bf0 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d1aa6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7cc0b0bb0 con 0x7ff7cc0b1da0 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d1aa6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0b0e90 con 0x7ff7cc159b20 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d12a5700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff7cc0b06d0 0x7ff7cc0aa5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60678/0 (socket says 172.21.15.3:60678) 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d12a5700 1 -- 172.21.15.3:0/1139293676 learned_addr learned my addr 172.21.15.3:0/1139293676 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7cbfff700 1 --2- 172.21.15.3:0/1139293676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc159b20 0x7ff7cc0b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.801+0000 7ff7d0aa4700 1 --2- 172.21.15.3:0/1139293676 >> v2:172.21.15.153:3300/0 conn(0x7ff7cc0b1da0 0x7ff7cc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cb7fe700 1 -- 172.21.15.3:0/1139293676 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011502928 0 0) 0x7ff7cc0b2bf0 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cb7fe700 1 -- 172.21.15.3:0/1139293676 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b8003570 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cbfff700 1 -- 172.21.15.3:0/1139293676 >> v2:172.21.15.153:3300/0 conn(0x7ff7cc0b1da0 msgr2=0x7ff7cc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cbfff700 1 --2- 172.21.15.3:0/1139293676 >> v2:172.21.15.153:3300/0 conn(0x7ff7cc0b1da0 0x7ff7cc0b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cbfff700 1 -- 172.21.15.3:0/1139293676 >> v1:172.21.15.3:6789/0 conn(0x7ff7cc0b06d0 legacy=0x7ff7cc0aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cbfff700 1 -- 172.21.15.3:0/1139293676 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0b1170 con 0x7ff7cc159b20 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cbfff700 1 --2- 172.21.15.3:0/1139293676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc159b20 0x7ff7cc0b26b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0098e0 tx=0x7ff7bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8386f6ea17d790 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cb7fe700 1 -- 172.21.15.3:0/1139293676 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc011630 con 0x7ff7cc159b20 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cb7fe700 1 -- 172.21.15.3:0/1139293676 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc011790 con 0x7ff7cc159b20 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.802+0000 7ff7cb7fe700 1 -- 172.21.15.3:0/1139293676 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc011b30 con 0x7ff7cc159b20 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/1139293676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc159b20 msgr2=0x7ff7cc0b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/1139293676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc159b20 0x7ff7cc0b26b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0098e0 tx=0x7ff7bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/1139293676 shutdown_connections 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/1139293676 >> v2:172.21.15.153:3300/0 conn(0x7ff7cc0b1da0 0x7ff7cc0b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/1139293676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc159b20 0x7ff7cc0b26b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/1139293676 >> 172.21.15.3:0/1139293676 conn(0x7ff7cc099780 msgr2=0x7ff7cc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/1139293676 shutdown_connections 2023-10-06T05:02:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.803+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/1139293676 wait complete. 2023-10-06T05:02:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.804+0000 7ff7d1aa6700 1 Processor -- start 2023-10-06T05:02:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.804+0000 7ff7d1aa6700 1 -- start start 2023-10-06T05:02:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d1aa6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7cc0b06d0 0x7ff7cc150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d1aa6700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc0b1da0 0x7ff7cc09c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d1aa6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7cc159b20 0x7ff7cc14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d1aa6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc09dcc0 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d1aa6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc14fab0 con 0x7ff7cc159b20 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d1aa6700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc09cc20 con 0x7ff7cc0b1da0 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7cc0b06d0 0x7ff7cc150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7cc0b06d0 0x7ff7cc150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37598/0 (socket says 172.21.15.3:37598) 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 -- 172.21.15.3:0/3854011512 learned_addr learned my addr 172.21.15.3:0/3854011512 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7cbfff700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc0b1da0 0x7ff7cc09c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d12a5700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7cc159b20 0x7ff7cc14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 -- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc0b1da0 msgr2=0x7ff7cc09c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc0b1da0 0x7ff7cc09c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 -- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7cc159b20 msgr2=0x7ff7cc14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7cc159b20 0x7ff7cc14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.805+0000 7ff7d0aa4700 1 -- 172.21.15.3:0/3854011512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc009420 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.806+0000 7ff7d0aa4700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7cc0b06d0 0x7ff7cc150720 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009f40 tx=0x7ff7c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.806+0000 7ff7c8ff9700 1 -- 172.21.15.3:0/3854011512 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400ccd0 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.806+0000 7ff7c8ff9700 1 -- 172.21.15.3:0/3854011512 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c400f040 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.806+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc09cf00 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.806+0000 7ff7c8ff9700 1 -- 172.21.15.3:0/3854011512 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40146b0 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.806+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc09d4c0 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.807+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc088660 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.807+0000 7ff7c8ff9700 1 -- 172.21.15.3:0/3854011512 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7c4014a20 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.808+0000 7ff7c8ff9700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff7ac066f30 0x7ff7ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.808+0000 7ff7c8ff9700 1 -- 172.21.15.3:0/3854011512 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff7c4084f80 con 0x7ff7cc0b06d0 2023-10-06T05:02:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.808+0000 7ff7cbfff700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff7ac066f30 0x7ff7ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:48.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.808+0000 7ff7cbfff700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff7ac066f30 0x7ff7ac0693e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc159f10 tx=0x7ff7bc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:48.812+0000 7ff7c8ff9700 1 -- 172.21.15.3:0/3854011512 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7c4052030 con 0x7ff7cc0b06d0 2023-10-06T05:02:49.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.007+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7cc09db10 con 0x7ff7cc0b06d0 2023-10-06T05:02:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.007+0000 7ff7c8ff9700 1 -- 172.21.15.3:0/3854011512 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16990 (secure 0 0 0) 0x7ff7c4055650 con 0x7ff7cc0b06d0 2023-10-06T05:02:49.009 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:49.009 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":135,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:02:46.809978+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:01:20.044822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:01:21.270200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/3882866003":"2023-10-06T06:02:46.060249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff7ac066f30 msgr2=0x7ff7ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff7ac066f30 0x7ff7ac0693e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc159f10 tx=0x7ff7bc017000 comp rx=0 tx=0).stop 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7cc0b06d0 msgr2=0x7ff7cc150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7cc0b06d0 0x7ff7cc150720 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009f40 tx=0x7ff7c400b040 comp rx=0 tx=0).stop 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 shutdown_connections 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff7ac066f30 0x7ff7ac0693e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7cc159b20 0x7ff7cc14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7cc0b06d0 0x7ff7cc150720 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 --2- 172.21.15.3:0/3854011512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7cc0b1da0 0x7ff7cc09c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 >> 172.21.15.3:0/3854011512 conn(0x7ff7cc099780 msgr2=0x7ff7cc0cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 shutdown_connections 2023-10-06T05:02:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.010+0000 7ff7d1aa6700 1 -- 172.21.15.3:0/3854011512 wait complete. 2023-10-06T05:02:49.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:52:10.963739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:52:11.480186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:01:20.044822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:01:21.270200+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:02:49.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:02:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.144+0000 7ff426f5c700 1 Processor -- start 2023-10-06T05:02:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.144+0000 7ff426f5c700 1 -- start start 2023-10-06T05:02:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.145+0000 7ff426f5c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff4201598d0 0x7ff4200aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.145+0000 7ff426f5c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4200b4b80 0x7ff4200b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.145+0000 7ff426f5c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4200b0ab0 con 0x7ff420159ca0 2023-10-06T05:02:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.145+0000 7ff426f5c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200b08c0 con 0x7ff4201598d0 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.145+0000 7ff426f5c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200b0d90 con 0x7ff4200b4b80 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.145+0000 7ff4254f9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4200b4b80 0x7ff4200b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.145+0000 7ff424cf8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff4201598d0 0x7ff4200aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff41ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff420159ca0 0x7ff4200aac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:38774/0 (socket says 172.21.15.3:38774) 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff41ffff700 1 -- 172.21.15.3:0/1352568723 learned_addr learned my addr 172.21.15.3:0/1352568723 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff4254f9700 1 -- 172.21.15.3:0/1352568723 >> v2:172.21.15.153:3300/0 conn(0x7ff4201598d0 msgr2=0x7ff4200aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff4254f9700 1 --2- 172.21.15.3:0/1352568723 >> v2:172.21.15.153:3300/0 conn(0x7ff4201598d0 0x7ff4200aa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff4254f9700 1 -- 172.21.15.3:0/1352568723 >> v1:172.21.15.153:6789/0 conn(0x7ff420159ca0 legacy=0x7ff4200aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff4254f9700 1 -- 172.21.15.3:0/1352568723 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200b1070 con 0x7ff4200b4b80 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff424cf8700 1 --2- 172.21.15.3:0/1352568723 >> v2:172.21.15.153:3300/0 conn(0x7ff4201598d0 0x7ff4200aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff4254f9700 1 --2- 172.21.15.3:0/1352568723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4200b4b80 0x7ff4200b1f90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff418009e60 tx=0x7ff418009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37fc55954ebcfa71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff41f7fe700 1 -- 172.21.15.3:0/1352568723 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41800e070 con 0x7ff4200b4b80 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff41f7fe700 1 -- 172.21.15.3:0/1352568723 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41800bb60 con 0x7ff4200b4b80 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.146+0000 7ff41f7fe700 1 -- 172.21.15.3:0/1352568723 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418012430 con 0x7ff4200b4b80 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 -- 172.21.15.3:0/1352568723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4200b4b80 msgr2=0x7ff4200b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1352568723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4200b4b80 0x7ff4200b1f90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff418009e60 tx=0x7ff418009780 comp rx=0 tx=0).stop 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 -- 172.21.15.3:0/1352568723 shutdown_connections 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1352568723 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4200b4b80 0x7ff4200b1f90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1352568723 >> v2:172.21.15.153:3300/0 conn(0x7ff4201598d0 0x7ff4200aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 -- 172.21.15.3:0/1352568723 >> 172.21.15.3:0/1352568723 conn(0x7ff420099970 msgr2=0x7ff42006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 -- 172.21.15.3:0/1352568723 shutdown_connections 2023-10-06T05:02:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.147+0000 7ff426f5c700 1 -- 172.21.15.3:0/1352568723 wait complete. 2023-10-06T05:02:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.148+0000 7ff426f5c700 1 Processor -- start 2023-10-06T05:02:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.148+0000 7ff426f5c700 1 -- start start 2023-10-06T05:02:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff426f5c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4200b4b80 0x7ff42015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff426f5c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4201598d0 0x7ff42015f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff426f5c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff420159ca0 0x7ff42015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff426f5c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015c030 con 0x7ff4200b4b80 2023-10-06T05:02:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff426f5c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015e9e0 con 0x7ff420159ca0 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff426f5c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015f4e0 con 0x7ff4201598d0 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4200b4b80 0x7ff42015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4200b4b80 0x7ff42015baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37600/0 (socket says 172.21.15.3:37600) 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 -- 172.21.15.3:0/1001851985 learned_addr learned my addr 172.21.15.3:0/1001851985 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff4254f9700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff420159ca0 0x7ff42015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff41ffff700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4201598d0 0x7ff42015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 -- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4201598d0 msgr2=0x7ff42015f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4201598d0 0x7ff42015f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 -- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff420159ca0 msgr2=0x7ff42015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff420159ca0 0x7ff42015feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.149+0000 7ff424cf8700 1 -- 172.21.15.3:0/1001851985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4100093c0 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.150+0000 7ff424cf8700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4200b4b80 0x7ff42015baf0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff41000ecb0 tx=0x7ff410009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.150+0000 7ff41cff9700 1 -- 172.21.15.3:0/1001851985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410005a00 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.150+0000 7ff41cff9700 1 -- 172.21.15.3:0/1001851985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410005b60 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.150+0000 7ff41cff9700 1 -- 172.21.15.3:0/1001851985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41000c920 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.150+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff418009a20 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.150+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42015f170 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.151+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42015f7c0 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.151+0000 7ff41cff9700 1 -- 172.21.15.3:0/1001851985 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff41000ccd0 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.152+0000 7ff41cff9700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.152+0000 7ff41cff9700 1 -- 172.21.15.3:0/1001851985 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff410084c00 con 0x7ff4200b4b80 2023-10-06T05:02:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.152+0000 7ff41ffff700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.152+0000 7ff41ffff700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff40c066f30 0x7ff40c0693e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff42009a490 tx=0x7ff414009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.156+0000 7ff41cff9700 1 -- 172.21.15.3:0/1001851985 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff41008b080 con 0x7ff4200b4b80 2023-10-06T05:02:49.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.351+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4200b1350 con 0x7ff4200b4b80 2023-10-06T05:02:49.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.352+0000 7ff41cff9700 1 -- 172.21.15.3:0/1001851985 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v135) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff410020020 con 0x7ff4200b4b80 2023-10-06T05:02:49.354 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff40c066f30 msgr2=0x7ff40c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff40c066f30 0x7ff40c0693e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff42009a490 tx=0x7ff414009040 comp rx=0 tx=0).stop 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4200b4b80 msgr2=0x7ff42015baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4200b4b80 0x7ff42015baf0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff41000ecb0 tx=0x7ff410009120 comp rx=0 tx=0).stop 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 shutdown_connections 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff420159ca0 0x7ff42015feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4200b4b80 0x7ff42015baf0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 --2- 172.21.15.3:0/1001851985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4201598d0 0x7ff42015f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 >> 172.21.15.3:0/1001851985 conn(0x7ff420099970 msgr2=0x7ff4200d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:49.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 shutdown_connections 2023-10-06T05:02:49.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.354+0000 7ff426f5c700 1 -- 172.21.15.3:0/1001851985 wait complete. 2023-10-06T05:02:49.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:02:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.488+0000 7fc0296ce700 1 Processor -- start 2023-10-06T05:02:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.489+0000 7fc0296ce700 1 -- start start 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.489+0000 7fc0296ce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc0241573f0 0x7fc02415a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.489+0000 7fc0296ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc024157830 con 0x7fc024068280 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.489+0000 7fc0296ce700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0240692c0 con 0x7fc024157020 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.489+0000 7fc0296ce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc024069b60 con 0x7fc0241573f0 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc0227fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc024068280 0x7fc024159e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60686/0 (socket says 172.21.15.3:60686) 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc0227fc700 1 -- 172.21.15.3:0/1935475261 learned_addr learned my addr 172.21.15.3:0/1935475261 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc0237fe700 1 --2- 172.21.15.3:0/1935475261 >> v2:172.21.15.153:3300/0 conn(0x7fc0241573f0 0x7fc02415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269818272 0 0) 0x7fc024157830 con 0x7fc024068280 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc010003570 con 0x7fc024068280 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419061909 0 0) 0x7fc0240692c0 con 0x7fc024157020 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc024157830 con 0x7fc024157020 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140473997 0 0) 0x7fc010003570 con 0x7fc024068280 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 >> v2:172.21.15.153:3300/0 conn(0x7fc0241573f0 msgr2=0x7fc02415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 --2- 172.21.15.3:0/1935475261 >> v2:172.21.15.153:3300/0 conn(0x7fc0241573f0 0x7fc02415a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 >> v1:172.21.15.153:6789/0 conn(0x7fc024157020 legacy=0x7fc024067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02415abe0 con 0x7fc024068280 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fc00c001470 con 0x7fc024068280 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 >> v1:172.21.15.3:6789/0 conn(0x7fc024068280 legacy=0x7fc024159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc010003c40 0x7fc010004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010004810 0x7fc010004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc010005210 0x7fc01000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc024069b60 con 0x7fc010005210 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01000c1b0 con 0x7fc010004810 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01000c370 con 0x7fc010003c40 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc0237fe700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc010005210 0x7fc01000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.490+0000 7fc022ffd700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc010003c40 0x7fc010004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0227fc700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010004810 0x7fc010004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0237fe700 1 -- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc010003c40 msgr2=0x7fc010004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0237fe700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc010003c40 0x7fc010004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0237fe700 1 -- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010004810 msgr2=0x7fc010004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0237fe700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010004810 0x7fc010004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0237fe700 1 -- 172.21.15.3:0/1935475261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc010003570 con 0x7fc010005210 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0237fe700 1 -- 172.21.15.3:0/1935475261 reap_dead start 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc0237fe700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc010005210 0x7fc01000ba60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc0140094d0 tx=0x7fc01400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbee51f446d7a2bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400ea90 con 0x7fc010005210 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01400f040 con 0x7fc010005210 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.491+0000 7fc021ffb700 1 -- 172.21.15.3:0/1935475261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400ee20 con 0x7fc010005210 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.492+0000 7fc0296ce700 1 -- 172.21.15.3:0/1935475261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc010005210 msgr2=0x7fc01000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.492+0000 7fc0296ce700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc010005210 0x7fc01000ba60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc0140094d0 tx=0x7fc01400b040 comp rx=0 tx=0).stop 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.492+0000 7fc0296ce700 1 -- 172.21.15.3:0/1935475261 shutdown_connections 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.492+0000 7fc0296ce700 1 --2- 172.21.15.3:0/1935475261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc010005210 0x7fc01000ba60 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.492+0000 7fc0296ce700 1 -- 172.21.15.3:0/1935475261 >> 172.21.15.3:0/1935475261 conn(0x7fc0240ce3d0 msgr2=0x7fc0240ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.492+0000 7fc0296ce700 1 -- 172.21.15.3:0/1935475261 shutdown_connections 2023-10-06T05:02:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.492+0000 7fc0296ce700 1 -- 172.21.15.3:0/1935475261 wait complete. 2023-10-06T05:02:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.493+0000 7fc0296ce700 1 Processor -- start 2023-10-06T05:02:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.493+0000 7fc0296ce700 1 -- start start 2023-10-06T05:02:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0296ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010005210 0x7fc0240c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0296ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0240c75e0 0x7fc02415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0296ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0240c7a50 0x7fc02415d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0296ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02415dd40 con 0x7fc0240c7a50 2023-10-06T05:02:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0296ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240b1db0 con 0x7fc010005210 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0296ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240b2090 con 0x7fc0240c75e0 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0240c7a50 0x7fc02415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0240c7a50 0x7fc02415d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37612/0 (socket says 172.21.15.3:37612) 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 -- 172.21.15.3:0/3505483174 learned_addr learned my addr 172.21.15.3:0/3505483174 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0227fc700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0240c75e0 0x7fc02415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc022ffd700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010005210 0x7fc0240c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 -- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0240c75e0 msgr2=0x7fc02415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0240c75e0 0x7fc02415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 -- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010005210 msgr2=0x7fc0240c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010005210 0x7fc0240c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 -- 172.21.15.3:0/3505483174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0240b2370 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc022ffd700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010005210 0x7fc0240c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0237fe700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0240c7a50 0x7fc02415d800 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc0240687f0 tx=0x7fc014019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc01affd700 1 -- 172.21.15.3:0/3505483174 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01401e660 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc01affd700 1 -- 172.21.15.3:0/3505483174 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01401e7e0 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0240b2650 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.494+0000 7fc01affd700 1 -- 172.21.15.3:0/3505483174 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01401eba0 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.495+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0240b1980 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.495+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02404c380 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.496+0000 7fc01affd700 1 -- 172.21.15.3:0/3505483174 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc014003020 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.496+0000 7fc01affd700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc004066f30 0x7fc0040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.496+0000 7fc01affd700 1 -- 172.21.15.3:0/3505483174 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc014086740 con 0x7fc0240c7a50 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.496+0000 7fc022ffd700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc004066f30 0x7fc0040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.497+0000 7fc022ffd700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc004066f30 0x7fc0040693e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000c00 tx=0x7fc01c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.501+0000 7fc01affd700 1 -- 172.21.15.3:0/3505483174 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0140537f0 con 0x7fc0240c7a50 2023-10-06T05:02:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.695+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc024068930 con 0x7fc0240c7a50 2023-10-06T05:02:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.695+0000 7fc01affd700 1 -- 172.21.15.3:0/3505483174 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc014056e10 con 0x7fc0240c7a50 2023-10-06T05:02:49.697 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-06T05:02:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc004066f30 msgr2=0x7fc0040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc004066f30 0x7fc0040693e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000c00 tx=0x7fc01c009040 comp rx=0 tx=0).stop 2023-10-06T05:02:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0240c7a50 msgr2=0x7fc02415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0240c7a50 0x7fc02415d800 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc0240687f0 tx=0x7fc014019000 comp rx=0 tx=0).stop 2023-10-06T05:02:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 shutdown_connections 2023-10-06T05:02:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fc004066f30 0x7fc0040693e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc010005210 0x7fc0240c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0240c7a50 0x7fc02415d800 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 --2- 172.21.15.3:0/3505483174 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc0240c75e0 0x7fc02415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 >> 172.21.15.3:0/3505483174 conn(0x7fc0240ce3d0 msgr2=0x7fc0240cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 shutdown_connections 2023-10-06T05:02:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.697+0000 7fc0296ce700 1 -- 172.21.15.3:0/3505483174 wait complete. 2023-10-06T05:02:49.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:02:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.833+0000 7f4ac0d9b700 1 Processor -- start 2023-10-06T05:02:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.833+0000 7f4ac0d9b700 1 -- start start 2023-10-06T05:02:49.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4ac0d9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 0x7f4abc1621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4ac0d9b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4abc0cd730 con 0x7f4abc140e50 2023-10-06T05:02:49.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4ac0d9b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4abc0ccb20 con 0x7f4abc141220 2023-10-06T05:02:49.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4ac0d9b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0cce00 con 0x7f4abc068160 2023-10-06T05:02:49.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4abb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4abc140e50 0x7f4abc0cc060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60698/0 (socket says 172.21.15.3:60698) 2023-10-06T05:02:49.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4abb7fe700 1 -- 172.21.15.3:0/1435245602 learned_addr learned my addr 172.21.15.3:0/1435245602 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4abbfff700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 0x7f4abc1621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396079676 0 0) 0x7f4abc0cd730 con 0x7f4abc140e50 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa4003570 con 0x7f4abc140e50 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855513601 0 0) 0x7f4abc0ccb20 con 0x7f4abc141220 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4abc0cd730 con 0x7f4abc141220 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 209552068 0 0) 0x7f4aa4003570 con 0x7f4abc140e50 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 msgr2=0x7f4abc1621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 0x7f4abc1621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 >> v1:172.21.15.153:6789/0 conn(0x7f4abc141220 legacy=0x7f4abc067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.834+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc0cd0e0 con 0x7f4abc140e50 2023-10-06T05:02:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f4aac001570 con 0x7f4abc140e50 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 >> v1:172.21.15.3:6789/0 conn(0x7f4abc140e50 legacy=0x7f4abc0cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aa4003c20 0x7f4aa4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 0x7f4aa4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0cce00 con 0x7f4aa4004810 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa400bbf0 con 0x7f4aa4003c20 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa400bdb0 con 0x7f4abc068160 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abaffd700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abbfff700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 0x7f4aa4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abb7fe700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aa4003c20 0x7f4aa4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abaffd700 1 -- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 msgr2=0x7f4aa4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abaffd700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc068160 0x7f4aa4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abaffd700 1 -- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aa4003c20 msgr2=0x7f4aa4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abaffd700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aa4003c20 0x7f4aa4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abaffd700 1 -- 172.21.15.3:0/1435245602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4003570 con 0x7f4aa4004810 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4abaffd700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0001a10 tx=0x7f4ab0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832a496e0dced5e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000c5d0 con 0x7f4aa4004810 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab000c730 con 0x7f4aa4004810 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.835+0000 7f4aba7fc700 1 -- 172.21.15.3:0/1435245602 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000cad0 con 0x7f4aa4004810 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.836+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/1435245602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 msgr2=0x7f4aa4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.836+0000 7f4ac0d9b700 1 --2- 172.21.15.3:0/1435245602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0001a10 tx=0x7f4ab0007d30 comp rx=0 tx=0).stop 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.836+0000 7f4abbfff700 1 -- 172.21.15.3:0/1435245602 reap_dead start 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.836+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/1435245602 shutdown_connections 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.836+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/1435245602 >> 172.21.15.3:0/1435245602 conn(0x7f4abc0ce280 msgr2=0x7f4abc0ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.836+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/1435245602 shutdown_connections 2023-10-06T05:02:49.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.836+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/1435245602 wait complete. 2023-10-06T05:02:49.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.837+0000 7f4ac0d9b700 1 Processor -- start 2023-10-06T05:02:49.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.837+0000 7f4ac0d9b700 1 -- start start 2023-10-06T05:02:49.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4ac0d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4abc13df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4ac0d9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc0b0800 0x7f4abc15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4ac0d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4abc0c3c30 0x7f4abc0c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4ac0d9b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0c4610 con 0x7f4aa4004810 2023-10-06T05:02:49.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4ac0d9b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc15ab70 con 0x7f4abc0c3c30 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4ac0d9b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc15ae10 con 0x7f4abc0b0800 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4abc13df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4abc13df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37620/0 (socket says 172.21.15.3:37620) 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 -- 172.21.15.3:0/4235183941 learned_addr learned my addr 172.21.15.3:0/4235183941 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abaffd700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc0b0800 0x7f4abc15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abbfff700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4abc0c3c30 0x7f4abc0c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 -- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc0b0800 msgr2=0x7f4abc15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc0b0800 0x7f4abc15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 -- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4abc0c3c30 msgr2=0x7f4abc0c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4abc0c3c30 0x7f4abc0c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.838+0000 7f4abb7fe700 1 -- 172.21.15.3:0/4235183941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc06d660 con 0x7f4aa4004810 2023-10-06T05:02:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.839+0000 7f4abb7fe700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4abc13df50 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00f930 tx=0x7f4aac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.839+0000 7f4aa37fe700 1 -- 172.21.15.3:0/4235183941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0072e0 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.839+0000 7f4aa37fe700 1 -- 172.21.15.3:0/4235183941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac007460 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.839+0000 7f4aa37fe700 1 -- 172.21.15.3:0/4235183941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0057f0 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.839+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc06d940 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.839+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc15a670 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.840+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc06ab50 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.840+0000 7f4aa37fe700 1 -- 172.21.15.3:0/4235183941 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4aac005ba0 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.841+0000 7f4aa37fe700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4aa8066f30 0x7f4aa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.841+0000 7f4aa37fe700 1 -- 172.21.15.3:0/4235183941 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4aac084750 con 0x7f4aa4004810 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.841+0000 7f4abaffd700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4aa8066f30 0x7f4aa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.841+0000 7f4abaffd700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4aa8066f30 0x7f4aa80693e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4abc159010 tx=0x7f4ab0015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:49.845+0000 7f4aa37fe700 1 -- 172.21.15.3:0/4235183941 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4aac051800 con 0x7f4aa4004810 2023-10-06T05:02:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.038+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4abc0cd3c0 con 0x7f4aa4004810 2023-10-06T05:02:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.039+0000 7f4aa37fe700 1 -- 172.21.15.3:0/4235183941 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v135) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4aac054e20 con 0x7f4aa4004810 2023-10-06T05:02:50.040 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-06T05:02:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4aa8066f30 msgr2=0x7f4aa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4aa8066f30 0x7f4aa80693e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4abc159010 tx=0x7f4ab0015000 comp rx=0 tx=0).stop 2023-10-06T05:02:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 msgr2=0x7f4abc13df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4abc13df50 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00f930 tx=0x7f4aac009040 comp rx=0 tx=0).stop 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 shutdown_connections 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4aa8066f30 0x7f4aa80693e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4abc0c3c30 0x7f4abc0c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa4004810 0x7f4abc13df50 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 --2- 172.21.15.3:0/4235183941 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4abc0b0800 0x7f4abc15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 >> 172.21.15.3:0/4235183941 conn(0x7f4abc0ce280 msgr2=0x7f4abc0b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.040+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 shutdown_connections 2023-10-06T05:02:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.041+0000 7f4ac0d9b700 1 -- 172.21.15.3:0/4235183941 wait complete. 2023-10-06T05:02:50.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:02:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.174+0000 7f1a7c86f700 1 Processor -- start 2023-10-06T05:02:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.174+0000 7f1a7c86f700 1 -- start start 2023-10-06T05:02:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.175+0000 7f1a7c86f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a781598d0 0x7f1a780aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.175+0000 7f1a7c86f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1a780b4b80 0x7f1a780b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.175+0000 7f1a7c86f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a780b0ab0 con 0x7f1a78159ca0 2023-10-06T05:02:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.175+0000 7f1a7c86f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a780b08c0 con 0x7f1a780b4b80 2023-10-06T05:02:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.175+0000 7f1a7c86f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780b0d90 con 0x7f1a781598d0 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a781598d0 0x7f1a780aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a781598d0 0x7f1a780aa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:54736/0 (socket says 172.21.15.3:54736) 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a76d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1a780b4b80 0x7f1a780b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 -- 172.21.15.3:0/367432514 learned_addr learned my addr 172.21.15.3:0/367432514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a75d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1a78159ca0 0x7f1a780aac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:38802/0 (socket says 172.21.15.3:38802) 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 -- 172.21.15.3:0/367432514 >> v2:172.21.15.153:3300/0 conn(0x7f1a780b4b80 msgr2=0x7f1a780b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 --2- 172.21.15.3:0/367432514 >> v2:172.21.15.153:3300/0 conn(0x7f1a780b4b80 0x7f1a780b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 -- 172.21.15.3:0/367432514 >> v1:172.21.15.153:6789/0 conn(0x7f1a78159ca0 legacy=0x7f1a780aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 -- 172.21.15.3:0/367432514 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a780b1070 con 0x7f1a781598d0 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a76d9d700 1 --2- 172.21.15.3:0/367432514 >> v2:172.21.15.153:3300/0 conn(0x7f1a780b4b80 0x7f1a780b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.176+0000 7f1a7659c700 1 --2- 172.21.15.3:0/367432514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a781598d0 0x7f1a780aa720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009990 tx=0x7f1a70009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb20936862dc768e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7559a700 1 -- 172.21.15.3:0/367432514 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70009e80 con 0x7f1a781598d0 2023-10-06T05:02:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7559a700 1 -- 172.21.15.3:0/367432514 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a70005c10 con 0x7f1a781598d0 2023-10-06T05:02:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7559a700 1 -- 172.21.15.3:0/367432514 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7000b650 con 0x7f1a781598d0 2023-10-06T05:02:50.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 -- 172.21.15.3:0/367432514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a781598d0 msgr2=0x7f1a780aa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/367432514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a781598d0 0x7f1a780aa720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009990 tx=0x7f1a70009220 comp rx=0 tx=0).stop 2023-10-06T05:02:50.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 -- 172.21.15.3:0/367432514 shutdown_connections 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/367432514 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a781598d0 0x7f1a780aa720 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/367432514 >> v2:172.21.15.153:3300/0 conn(0x7f1a780b4b80 0x7f1a780b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 -- 172.21.15.3:0/367432514 >> 172.21.15.3:0/367432514 conn(0x7f1a78099970 msgr2=0x7f1a7806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 -- 172.21.15.3:0/367432514 shutdown_connections 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.177+0000 7f1a7c86f700 1 -- 172.21.15.3:0/367432514 wait complete. 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.178+0000 7f1a7c86f700 1 Processor -- start 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.178+0000 7f1a7c86f700 1 -- start start 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a7c86f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a780b4b80 0x7f1a780cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a7c86f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a781598d0 0x7f1a7809a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a7c86f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a78159ca0 0x7f1a7809bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a7c86f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7809a710 con 0x7f1a781598d0 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a7c86f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7809a9b0 con 0x7f1a780b4b80 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a7c86f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7809ac90 con 0x7f1a78159ca0 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a75d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a781598d0 0x7f1a7809a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a75d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a781598d0 0x7f1a7809a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37628/0 (socket says 172.21.15.3:37628) 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a7659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a780b4b80 0x7f1a780cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a75d9b700 1 -- 172.21.15.3:0/4210027592 learned_addr learned my addr 172.21.15.3:0/4210027592 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.179+0000 7f1a76d9d700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a78159ca0 0x7f1a7809bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a75d9b700 1 -- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a78159ca0 msgr2=0x7f1a7809bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a75d9b700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a78159ca0 0x7f1a7809bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a75d9b700 1 -- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a780b4b80 msgr2=0x7f1a780cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a75d9b700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a780b4b80 0x7f1a780cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a75d9b700 1 -- 172.21.15.3:0/4210027592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68009a20 con 0x7f1a781598d0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a75d9b700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a781598d0 0x7f1a7809a1d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1a60001eb0 tx=0x7f1a60007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a6e7fc700 1 -- 172.21.15.3:0/4210027592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60005a70 con 0x7f1a781598d0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a6e7fc700 1 -- 172.21.15.3:0/4210027592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60005bd0 con 0x7f1a781598d0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a70009480 con 0x7f1a781598d0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a6e7fc700 1 -- 172.21.15.3:0/4210027592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a600070d0 con 0x7f1a781598d0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.180+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7809b250 con 0x7f1a781598d0 2023-10-06T05:02:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.181+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a780644f0 con 0x7f1a781598d0 2023-10-06T05:02:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.181+0000 7f1a6e7fc700 1 -- 172.21.15.3:0/4210027592 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a60007480 con 0x7f1a781598d0 2023-10-06T05:02:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.182+0000 7f1a6e7fc700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a58066f30 0x7f1a580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.182+0000 7f1a6e7fc700 1 -- 172.21.15.3:0/4210027592 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a60083a20 con 0x7f1a781598d0 2023-10-06T05:02:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.182+0000 7f1a7659c700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a58066f30 0x7f1a580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.182+0000 7f1a7659c700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a58066f30 0x7f1a580693e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009260 tx=0x7f1a70016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.186+0000 7f1a6e7fc700 1 -- 172.21.15.3:0/4210027592 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a60050b90 con 0x7f1a781598d0 2023-10-06T05:02:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.378+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a7804c380 con 0x7f1a781598d0 2023-10-06T05:02:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.379+0000 7f1a6e7fc700 1 -- 172.21.15.3:0/4210027592 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v135) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a60017020 con 0x7f1a781598d0 2023-10-06T05:02:50.380 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-06T05:02:50.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.380+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a58066f30 msgr2=0x7f1a580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.380+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a58066f30 0x7f1a580693e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009260 tx=0x7f1a70016000 comp rx=0 tx=0).stop 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.380+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a781598d0 msgr2=0x7f1a7809a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.380+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a781598d0 0x7f1a7809a1d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1a60001eb0 tx=0x7f1a60007f40 comp rx=0 tx=0).stop 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 shutdown_connections 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f1a58066f30 0x7f1a580693e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a780b4b80 0x7f1a780cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a78159ca0 0x7f1a7809bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 --2- 172.21.15.3:0/4210027592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a781598d0 0x7f1a7809a1d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 >> 172.21.15.3:0/4210027592 conn(0x7f1a78099970 msgr2=0x7f1a7815c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 shutdown_connections 2023-10-06T05:02:50.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.381+0000 7f1a7c86f700 1 -- 172.21.15.3:0/4210027592 wait complete. 2023-10-06T05:02:50.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:02:50.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.511+0000 7f600f5ca700 1 Processor -- start 2023-10-06T05:02:50.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.512+0000 7f600f5ca700 1 -- start start 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.512+0000 7f600f5ca700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f600815db30 0x7f60080b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.512+0000 7f600f5ca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60080cf490 0x7f60080cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.512+0000 7f600f5ca700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60080c56d0 con 0x7f60080cdd60 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.512+0000 7f600f5ca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080c59b0 con 0x7f60080cf490 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.512+0000 7f600f5ca700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080c5c90 con 0x7f600815db30 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600db67700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60080cf490 0x7f60080cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600cb65700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f60080cdd60 0x7f60080ced80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60712/0 (socket says 172.21.15.3:60712) 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600cb65700 1 -- 172.21.15.3:0/2092415078 learned_addr learned my addr 172.21.15.3:0/2092415078 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:50.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600d366700 1 --2- 172.21.15.3:0/2092415078 >> v2:172.21.15.153:3300/0 conn(0x7f600815db30 0x7f60080b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600db67700 1 -- 172.21.15.3:0/2092415078 >> v2:172.21.15.153:3300/0 conn(0x7f600815db30 msgr2=0x7f60080b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600db67700 1 --2- 172.21.15.3:0/2092415078 >> v2:172.21.15.153:3300/0 conn(0x7f600815db30 0x7f60080b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f6007fff700 1 -- 172.21.15.3:0/2092415078 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605499261 0 0) 0x7f60080c56d0 con 0x7f60080cdd60 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600db67700 1 -- 172.21.15.3:0/2092415078 >> v1:172.21.15.3:6789/0 conn(0x7f60080cdd60 legacy=0x7f60080ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600db67700 1 -- 172.21.15.3:0/2092415078 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080c5f70 con 0x7f60080cf490 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f600db67700 1 --2- 172.21.15.3:0/2092415078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080cf490 0x7f60080cc460 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f600806d2d0 tx=0x7f5ff80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675b8cbf397e1e79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f6007fff700 1 -- 172.21.15.3:0/2092415078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800b690 con 0x7f60080cf490 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f6007fff700 1 -- 172.21.15.3:0/2092415078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff800b7f0 con 0x7f60080cf490 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.513+0000 7f6007fff700 1 -- 172.21.15.3:0/2092415078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800bb90 con 0x7f60080cf490 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 -- 172.21.15.3:0/2092415078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080cf490 msgr2=0x7f60080cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 --2- 172.21.15.3:0/2092415078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080cf490 0x7f60080cc460 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f600806d2d0 tx=0x7f5ff80091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 -- 172.21.15.3:0/2092415078 shutdown_connections 2023-10-06T05:02:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 --2- 172.21.15.3:0/2092415078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080cf490 0x7f60080cc460 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 --2- 172.21.15.3:0/2092415078 >> v2:172.21.15.153:3300/0 conn(0x7f600815db30 0x7f60080b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 -- 172.21.15.3:0/2092415078 >> 172.21.15.3:0/2092415078 conn(0x7f6008087fe0 msgr2=0x7f60080998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 -- 172.21.15.3:0/2092415078 shutdown_connections 2023-10-06T05:02:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.514+0000 7f600f5ca700 1 -- 172.21.15.3:0/2092415078 wait complete. 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.515+0000 7f600f5ca700 1 Processor -- start 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.515+0000 7f600f5ca700 1 -- start start 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600f5ca700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60080cdd60 0x7f600816dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600f5ca700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60080cf490 0x7f60080ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600f5ca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f600815db30 0x7f60080b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600f5ca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600816e000 con 0x7f600815db30 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600f5ca700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600815f360 con 0x7f60080cdd60 2023-10-06T05:02:50.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600f5ca700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600815f640 con 0x7f60080cf490 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f600815db30 0x7f60080b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f600815db30 0x7f60080b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37636/0 (socket says 172.21.15.3:37636) 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 -- 172.21.15.3:0/4104098132 learned_addr learned my addr 172.21.15.3:0/4104098132 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600cb65700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60080cf490 0x7f60080ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600d366700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60080cdd60 0x7f600816dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 -- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60080cf490 msgr2=0x7f60080ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60080cf490 0x7f60080ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 -- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60080cdd60 msgr2=0x7f600816dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60080cdd60 0x7f600816dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600db67700 1 -- 172.21.15.3:0/4104098132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8009420 con 0x7f600815db30 2023-10-06T05:02:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.516+0000 7f600cb65700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60080cf490 0x7f60080ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.517+0000 7f600db67700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f600815db30 0x7f60080b4810 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8013000 tx=0x7f5ff8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.517+0000 7f60057fa700 1 -- 172.21.15.3:0/4104098132 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800bb30 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.517+0000 7f60057fa700 1 -- 172.21.15.3:0/4104098132 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff8011630 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.517+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600815f920 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.517+0000 7f60057fa700 1 -- 172.21.15.3:0/4104098132 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80119f0 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.517+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600815fee0 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.518+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600804c380 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.518+0000 7f60057fa700 1 -- 172.21.15.3:0/4104098132 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ff80253f0 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.518+0000 7f60057fa700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ff4066f30 0x7f5ff40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.519+0000 7f60057fa700 1 -- 172.21.15.3:0/4104098132 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5ff8086340 con 0x7f600815db30 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.519+0000 7f600d366700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ff4066f30 0x7f5ff40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.519+0000 7f600d366700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ff4066f30 0x7f5ff40693e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f600000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.523+0000 7f60057fa700 1 -- 172.21.15.3:0/4104098132 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ff80533f0 con 0x7f600815db30 2023-10-06T05:02:50.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.760+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f600808aba0 con 0x7f600815db30 2023-10-06T05:02:50.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.761+0000 7f60057fa700 1 -- 172.21.15.3:0/4104098132 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f5ff8056a10 con 0x7f600815db30 2023-10-06T05:02:50.762 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:50.762 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:02:47.821136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":24,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:02:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.762+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ff4066f30 msgr2=0x7f5ff40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.762+0000 7f600f5ca700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ff4066f30 0x7f5ff40693e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f600000b040 comp rx=0 tx=0).stop 2023-10-06T05:02:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.762+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f600815db30 msgr2=0x7f60080b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.762+0000 7f600f5ca700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f600815db30 0x7f60080b4810 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8013000 tx=0x7f5ff8017000 comp rx=0 tx=0).stop 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.762+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 shutdown_connections 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.763+0000 7f600f5ca700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5ff4066f30 0x7f5ff40693e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.763+0000 7f600f5ca700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60080cdd60 0x7f600816dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.763+0000 7f600f5ca700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f600815db30 0x7f60080b4810 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.763+0000 7f600f5ca700 1 --2- 172.21.15.3:0/4104098132 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f60080cf490 0x7f60080ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.763+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 >> 172.21.15.3:0/4104098132 conn(0x7f6008087fe0 msgr2=0x7f600806dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.763+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 shutdown_connections 2023-10-06T05:02:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.763+0000 7f600f5ca700 1 -- 172.21.15.3:0/4104098132 wait complete. 2023-10-06T05:02:50.766 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 91 2023-10-06T05:02:50.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:02:50.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.894+0000 7f2de80bc700 1 Processor -- start 2023-10-06T05:02:50.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.894+0000 7f2de80bc700 1 -- start start 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.895+0000 7f2de80bc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2de0157300 0x7f2de00af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.895+0000 7f2de80bc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 0x7f2de00acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.895+0000 7f2de80bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de00ad4e0 con 0x7f2de01576d0 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.895+0000 7f2de80bc700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de00af0f0 con 0x7f2de0157300 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.895+0000 7f2de80bc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00afac0 con 0x7f2de00acb40 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.895+0000 7f2de5657700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2de01576d0 0x7f2de015b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60726/0 (socket says 172.21.15.3:60726) 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.895+0000 7f2de5657700 1 -- 172.21.15.3:0/3879446082 learned_addr learned my addr 172.21.15.3:0/3879446082 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de6659700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 0x7f2de00acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994596207 0 0) 0x7f2dd4001110 con 0x7f2de01576d0 2023-10-06T05:02:50.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de5e58700 1 --2- 172.21.15.3:0/3879446082 >> v2:172.21.15.153:3300/0 conn(0x7f2de0157300 0x7f2de00af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc8003570 con 0x7f2de01576d0 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184743439 0 0) 0x7f2dc8003570 con 0x7f2de01576d0 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 msgr2=0x7f2de00acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 0x7f2de00acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 >> v2:172.21.15.153:3300/0 conn(0x7f2de0157300 msgr2=0x7f2de00af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 --2- 172.21.15.3:0/3879446082 >> v2:172.21.15.153:3300/0 conn(0x7f2de0157300 0x7f2de00af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de015bb10 con 0x7f2de01576d0 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f2dd4001600 con 0x7f2de01576d0 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 >> v1:172.21.15.3:6789/0 conn(0x7f2de01576d0 legacy=0x7f2de015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8003a60 0x7f2dc8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 0x7f2dc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc8004d50 0x7f2dc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00af0f0 con 0x7f2dc8004d50 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00afac0 con 0x7f2dc8003a60 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc800ba60 con 0x7f2de00acb40 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de6659700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc8004d50 0x7f2dc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de5e58700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8003a60 0x7f2dc8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de5657700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 0x7f2dc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de6659700 1 -- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 msgr2=0x7f2dc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de6659700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00acb40 0x7f2dc8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de6659700 1 -- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8003a60 msgr2=0x7f2dc8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de6659700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8003a60 0x7f2dc8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.896+0000 7f2de6659700 1 -- 172.21.15.3:0/3879446082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc8003570 con 0x7f2dc8004d50 2023-10-06T05:02:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de6659700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc8004d50 0x7f2dc80051c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009c00 tx=0x7f2dd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc10129936439e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd800e6b0 con 0x7f2dc8004d50 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd800e810 con 0x7f2dc8004d50 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de4e56700 1 -- 172.21.15.3:0/3879446082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd800ebd0 con 0x7f2dc8004d50 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de80bc700 1 -- 172.21.15.3:0/3879446082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc8004d50 msgr2=0x7f2dc80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de80bc700 1 --2- 172.21.15.3:0/3879446082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc8004d50 0x7f2dc80051c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009c00 tx=0x7f2dd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de6659700 1 -- 172.21.15.3:0/3879446082 reap_dead start 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de80bc700 1 -- 172.21.15.3:0/3879446082 shutdown_connections 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de80bc700 1 -- 172.21.15.3:0/3879446082 >> 172.21.15.3:0/3879446082 conn(0x7f2de00675f0 msgr2=0x7f2de00679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de80bc700 1 -- 172.21.15.3:0/3879446082 shutdown_connections 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.897+0000 7f2de80bc700 1 -- 172.21.15.3:0/3879446082 wait complete. 2023-10-06T05:02:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.898+0000 7f2de80bc700 1 Processor -- start 2023-10-06T05:02:50.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de80bc700 1 -- start start 2023-10-06T05:02:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de80bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de014fb20 0x7f2de006cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de80bc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de014fef0 0x7f2de006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de80bc700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00cbcc0 0x7f2de00b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de80bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00b2ee0 con 0x7f2de014fb20 2023-10-06T05:02:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de80bc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00cb830 con 0x7f2de014fef0 2023-10-06T05:02:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de80bc700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00caf00 con 0x7f2de00cbcc0 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de5e58700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de014fb20 0x7f2de006cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de5e58700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de014fb20 0x7f2de006cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37652/0 (socket says 172.21.15.3:37652) 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.899+0000 7f2de5e58700 1 -- 172.21.15.3:0/3089438778 learned_addr learned my addr 172.21.15.3:0/3089438778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5657700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de014fef0 0x7f2de006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de6659700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00cbcc0 0x7f2de00b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5e58700 1 -- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00cbcc0 msgr2=0x7f2de00b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5e58700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00cbcc0 0x7f2de00b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5e58700 1 -- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de014fef0 msgr2=0x7f2de006d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5e58700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de014fef0 0x7f2de006d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5e58700 1 -- 172.21.15.3:0/3089438778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de00cb1e0 con 0x7f2de014fb20 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5657700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de014fef0 0x7f2de006d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de5e58700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de014fb20 0x7f2de006cc80 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009fd0 tx=0x7f2dd000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2dddffb700 1 -- 172.21.15.3:0/3089438778 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000e040 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2dddffb700 1 -- 172.21.15.3:0/3089438778 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000f040 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de00cb4c0 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2dddffb700 1 -- 172.21.15.3:0/3089438778 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0015430 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.900+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de00b28d0 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.901+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de004c380 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.902+0000 7f2dddffb700 1 -- 172.21.15.3:0/3089438778 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2dd0015860 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.902+0000 7f2dddffb700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dcc066f30 0x7f2dcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.902+0000 7f2dddffb700 1 -- 172.21.15.3:0/3089438778 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2dd0085350 con 0x7f2de014fb20 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.902+0000 7f2de5657700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dcc066f30 0x7f2dcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:50.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.903+0000 7f2de5657700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dcc066f30 0x7f2dcc0693e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2de00b19b0 tx=0x7f2dd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:50.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:50.907+0000 7f2dddffb700 1 -- 172.21.15.3:0/3089438778 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dd0052400 con 0x7f2de014fb20 2023-10-06T05:02:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.102+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2de015bdf0 con 0x7f2de014fb20 2023-10-06T05:02:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.103+0000 7f2dddffb700 1 -- 172.21.15.3:0/3089438778 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f2dd0055a20 con 0x7f2de014fb20 2023-10-06T05:02:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:51.105 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":135,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:02:46.809978+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:01:20.044822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:01:21.270200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/3882866003":"2023-10-06T06:02:46.060249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:02:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.105+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dcc066f30 msgr2=0x7f2dcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.105+0000 7f2de80bc700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dcc066f30 0x7f2dcc0693e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2de00b19b0 tx=0x7f2dd4009040 comp rx=0 tx=0).stop 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de014fb20 msgr2=0x7f2de006cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de014fb20 0x7f2de006cc80 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009fd0 tx=0x7f2dd000b040 comp rx=0 tx=0).stop 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 shutdown_connections 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f2dcc066f30 0x7f2dcc0693e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2de00cbcc0 0x7f2de00b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de014fb20 0x7f2de006cc80 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 --2- 172.21.15.3:0/3089438778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de014fef0 0x7f2de006d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 >> 172.21.15.3:0/3089438778 conn(0x7f2de00675f0 msgr2=0x7f2de006b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 shutdown_connections 2023-10-06T05:02:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.106+0000 7f2de80bc700 1 -- 172.21.15.3:0/3089438778 wait complete. 2023-10-06T05:02:51.120 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:02:51.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:02:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.238+0000 7fa1df794700 1 Processor -- start 2023-10-06T05:02:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.238+0000 7fa1df794700 1 -- start start 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1df794700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 0x7fa1d806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1df794700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d806cd40 con 0x7fa1d8159a40 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1df794700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d806d020 con 0x7fa1d806ef50 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1df794700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d806d300 con 0x7fa1d8159670 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1dcd2f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1d8159a40 0x7fa1d806e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60740/0 (socket says 172.21.15.3:60740) 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1dcd2f700 1 -- 172.21.15.3:0/2722409030 learned_addr learned my addr 172.21.15.3:0/2722409030 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1dd530700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 0x7fa1d806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225973404 0 0) 0x7fa1d806cd40 con 0x7fa1d8159a40 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c0003570 con 0x7fa1d8159a40 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2123147642 0 0) 0x7fa1c0003570 con 0x7fa1d8159a40 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 msgr2=0x7fa1d806c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 0x7fa1d806c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 >> v1:172.21.15.153:6789/0 conn(0x7fa1d806ef50 legacy=0x7fa1d80a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d806d5e0 con 0x7fa1d8159a40 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fa1cc001470 con 0x7fa1d8159a40 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 >> v1:172.21.15.3:6789/0 conn(0x7fa1d8159a40 legacy=0x7fa1d806e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.239+0000 7fa1d7fff700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 0x7fa1c0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1c0003fd0 0x7fa1c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c0004d50 0x7fa1c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d806d300 con 0x7fa1c0004d50 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c000bb40 con 0x7fa1c0003fd0 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c000bd00 con 0x7fa1d8159670 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1ddd31700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c0004d50 0x7fa1c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1dd530700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 0x7fa1c0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1dcd2f700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1c0003fd0 0x7fa1c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1ddd31700 1 -- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 msgr2=0x7fa1c0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1ddd31700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d8159670 0x7fa1c0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1ddd31700 1 -- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1c0003fd0 msgr2=0x7fa1c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1ddd31700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1c0003fd0 0x7fa1c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1ddd31700 1 -- 172.21.15.3:0/2722409030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c0003570 con 0x7fa1c0004d50 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1ddd31700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c0004d50 0x7fa1c00051c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0001970 tx=0x7fa1d0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0344dc332ed0be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000c5d0 con 0x7fa1c0004d50 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d000c750 con 0x7fa1c0004d50 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.240+0000 7fa1d7fff700 1 -- 172.21.15.3:0/2722409030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000cb10 con 0x7fa1c0004d50 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.241+0000 7fa1df794700 1 -- 172.21.15.3:0/2722409030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c0004d50 msgr2=0x7fa1c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.241+0000 7fa1df794700 1 --2- 172.21.15.3:0/2722409030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c0004d50 0x7fa1c00051c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0001970 tx=0x7fa1d0007d10 comp rx=0 tx=0).stop 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.241+0000 7fa1ddd31700 1 -- 172.21.15.3:0/2722409030 reap_dead start 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.241+0000 7fa1df794700 1 -- 172.21.15.3:0/2722409030 shutdown_connections 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.241+0000 7fa1df794700 1 -- 172.21.15.3:0/2722409030 >> 172.21.15.3:0/2722409030 conn(0x7fa1d80cca80 msgr2=0x7fa1d80cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.241+0000 7fa1df794700 1 -- 172.21.15.3:0/2722409030 shutdown_connections 2023-10-06T05:02:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.241+0000 7fa1df794700 1 -- 172.21.15.3:0/2722409030 wait complete. 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.242+0000 7fa1df794700 1 Processor -- start 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.242+0000 7fa1df794700 1 -- start start 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1df794700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d80c9350 0x7fa1d81581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1df794700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d80c9720 0x7fa1d8158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1df794700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d80c6f80 0x7fa1d80c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1df794700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80c88c0 con 0x7fa1d80c9720 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1df794700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d8158c70 con 0x7fa1d80c6f80 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1df794700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80c7cb0 con 0x7fa1d80c9350 2023-10-06T05:02:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d80c9720 0x7fa1d8158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d80c9720 0x7fa1d8158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37658/0 (socket says 172.21.15.3:37658) 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 -- 172.21.15.3:0/3610657392 learned_addr learned my addr 172.21.15.3:0/3610657392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dd530700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d80c9350 0x7fa1d81581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 -- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d80c9350 msgr2=0x7fa1d81581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1ddd31700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d80c6f80 0x7fa1d80c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d80c9350 0x7fa1d81581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 -- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d80c6f80 msgr2=0x7fa1d80c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d80c6f80 0x7fa1d80c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 -- 172.21.15.3:0/3610657392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d80c7f90 con 0x7fa1d80c9720 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.243+0000 7fa1dcd2f700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d80c9720 0x7fa1d8158730 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0015a0 tx=0x7fa1cc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.244+0000 7fa1d57fa700 1 -- 172.21.15.3:0/3610657392 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00f840 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.244+0000 7fa1d57fa700 1 -- 172.21.15.3:0/3610657392 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc00f9c0 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.244+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d80c8270 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.244+0000 7fa1d57fa700 1 -- 172.21.15.3:0/3610657392 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00fd80 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.244+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d80c7960 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.245+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d804c380 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.245+0000 7fa1d57fa700 1 -- 172.21.15.3:0/3610657392 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1cc013600 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.245+0000 7fa1d57fa700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.245+0000 7fa1d57fa700 1 -- 172.21.15.3:0/3610657392 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa1cc0848c0 con 0x7fa1d80c9720 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.246+0000 7fa1dd530700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.246+0000 7fa1dd530700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c4066f30 0x7fa1c40693e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80094d0 tx=0x7fa1c800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.250+0000 7fa1d57fa700 1 -- 172.21.15.3:0/3610657392 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1cc051970 con 0x7fa1d80c9720 2023-10-06T05:02:51.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.492+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1d806d8c0 con 0x7fa1d80c9720 2023-10-06T05:02:51.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.493+0000 7fa1d57fa700 1 -- 172.21.15.3:0/3610657392 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fa1cc054f90 con 0x7fa1d80c9720 2023-10-06T05:02:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:51.495 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:02:47.821136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":24,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c4066f30 msgr2=0x7fa1c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c4066f30 0x7fa1c40693e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80094d0 tx=0x7fa1c800b040 comp rx=0 tx=0).stop 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d80c9720 msgr2=0x7fa1d8158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d80c9720 0x7fa1d8158730 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0015a0 tx=0x7fa1cc001e70 comp rx=0 tx=0).stop 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 shutdown_connections 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1d80c6f80 0x7fa1d80c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1d80c9350 0x7fa1d81581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 --2- 172.21.15.3:0/3610657392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1d80c9720 0x7fa1d8158730 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 >> 172.21.15.3:0/3610657392 conn(0x7fa1d80cca80 msgr2=0x7fa1d80c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 shutdown_connections 2023-10-06T05:02:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.495+0000 7fa1df794700 1 -- 172.21.15.3:0/3610657392 wait complete. 2023-10-06T05:02:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 91 2023-10-06T05:02:51.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:02:51.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.626+0000 7f0ab58f9700 1 Processor -- start 2023-10-06T05:02:51.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.626+0000 7f0ab58f9700 1 -- start start 2023-10-06T05:02:51.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0ab58f9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0ab0141230 0x7f0ab00cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0ab58f9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab00cadc0 0x7f0ab00cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0ab58f9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab00cd8f0 con 0x7f0ab0140e60 2023-10-06T05:02:51.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0ab58f9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab00b11f0 con 0x7f0ab0141230 2023-10-06T05:02:51.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0ab58f9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00cd050 con 0x7f0ab00cadc0 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0aaeffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ab0140e60 0x7f0ab00ca6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60750/0 (socket says 172.21.15.3:60750) 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3137268803 learned_addr learned my addr 172.21.15.3:0/3137268803 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0aaf7fe700 1 --2- 172.21.15.3:0/3137268803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab00cadc0 0x7f0ab00cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0aae7fc700 1 --2- 172.21.15.3:0/3137268803 >> v2:172.21.15.153:3300/0 conn(0x7f0ab0141230 0x7f0ab00cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.627+0000 7f0aadffb700 1 -- 172.21.15.3:0/3137268803 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122090715 0 0) 0x7f0ab00cd8f0 con 0x7f0ab0140e60 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aadffb700 1 -- 172.21.15.3:0/3137268803 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a9c003570 con 0x7f0ab0140e60 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aaf7fe700 1 -- 172.21.15.3:0/3137268803 >> v2:172.21.15.153:3300/0 conn(0x7f0ab0141230 msgr2=0x7f0ab00cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aaf7fe700 1 --2- 172.21.15.3:0/3137268803 >> v2:172.21.15.153:3300/0 conn(0x7f0ab0141230 0x7f0ab00cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aadffb700 1 -- 172.21.15.3:0/3137268803 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3443733544 0 0) 0x7f0a9c003570 con 0x7f0ab0140e60 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aaf7fe700 1 -- 172.21.15.3:0/3137268803 >> v1:172.21.15.3:6789/0 conn(0x7f0ab0140e60 legacy=0x7f0ab00ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aaf7fe700 1 -- 172.21.15.3:0/3137268803 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00cd330 con 0x7f0ab00cadc0 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aae7fc700 1 --2- 172.21.15.3:0/3137268803 >> v2:172.21.15.153:3300/0 conn(0x7f0ab0141230 0x7f0ab00cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aaf7fe700 1 --2- 172.21.15.3:0/3137268803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab00cadc0 0x7f0ab00cc820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009e60 tx=0x7f0aa0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefac3122651fe00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aadffb700 1 -- 172.21.15.3:0/3137268803 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000e070 con 0x7f0ab00cadc0 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aadffb700 1 -- 172.21.15.3:0/3137268803 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa000b7a0 con 0x7f0ab00cadc0 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.628+0000 7f0aadffb700 1 -- 172.21.15.3:0/3137268803 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8001470 con 0x7f0ab00cadc0 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3137268803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab00cadc0 msgr2=0x7f0ab00cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3137268803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab00cadc0 0x7f0ab00cc820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009e60 tx=0x7f0aa0009780 comp rx=0 tx=0).stop 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3137268803 shutdown_connections 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3137268803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab00cadc0 0x7f0ab00cc820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3137268803 >> v2:172.21.15.153:3300/0 conn(0x7f0ab0141230 0x7f0ab00cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3137268803 >> 172.21.15.3:0/3137268803 conn(0x7f0ab0067aa0 msgr2=0x7f0ab0067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:51.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3137268803 shutdown_connections 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.629+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3137268803 wait complete. 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.630+0000 7f0ab58f9700 1 Processor -- start 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.630+0000 7f0ab58f9700 1 -- start start 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0ab58f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab00cadc0 0x7f0ab012c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0ab58f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab0140e60 0x7f0ab00afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0ab58f9700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab0141230 0x7f0ab0158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0ab58f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab012dc70 con 0x7f0ab0140e60 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0ab58f9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab012cdc0 con 0x7f0ab00cadc0 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0ab58f9700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00b0530 con 0x7f0ab0141230 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab0140e60 0x7f0ab00afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab0140e60 0x7f0ab00afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37666/0 (socket says 172.21.15.3:37666) 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 -- 172.21.15.3:0/3426076600 learned_addr learned my addr 172.21.15.3:0/3426076600 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aaf7fe700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab0141230 0x7f0ab0158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aaeffd700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab00cadc0 0x7f0ab012c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 -- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab0141230 msgr2=0x7f0ab0158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab0141230 0x7f0ab0158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 -- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab00cadc0 msgr2=0x7f0ab012c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab00cadc0 0x7f0ab012c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 -- 172.21.15.3:0/3426076600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98009420 con 0x7f0ab0140e60 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aaf7fe700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab0141230 0x7f0ab0158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aaeffd700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab00cadc0 0x7f0ab012c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aae7fc700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab0140e60 0x7f0ab00afff0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800ec80 tx=0x7f0a980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aa6ffd700 1 -- 172.21.15.3:0/3426076600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800c9d0 con 0x7f0ab0140e60 2023-10-06T05:02:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aa6ffd700 1 -- 172.21.15.3:0/3426076600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9800cb30 con 0x7f0ab0140e60 2023-10-06T05:02:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa0009a20 con 0x7f0ab0140e60 2023-10-06T05:02:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.631+0000 7f0aa6ffd700 1 -- 172.21.15.3:0/3426076600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98013470 con 0x7f0ab0140e60 2023-10-06T05:02:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.632+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab012d340 con 0x7f0ab0140e60 2023-10-06T05:02:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.632+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab012de20 con 0x7f0ab0140e60 2023-10-06T05:02:51.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.633+0000 7f0aa6ffd700 1 -- 172.21.15.3:0/3426076600 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a98013820 con 0x7f0ab0140e60 2023-10-06T05:02:51.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.633+0000 7f0aa6ffd700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0a90066f30 0x7f0a900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:51.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.633+0000 7f0aa6ffd700 1 -- 172.21.15.3:0/3426076600 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0a98084850 con 0x7f0ab0140e60 2023-10-06T05:02:51.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.634+0000 7f0aaeffd700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0a90066f30 0x7f0a900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:51.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.634+0000 7f0aaeffd700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0a90066f30 0x7f0a900693e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80011e0 tx=0x7f0aa8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:51.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.638+0000 7f0aa6ffd700 1 -- 172.21.15.3:0/3426076600 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ab012de20 con 0x7f0ab0140e60 2023-10-06T05:02:51.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.875+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab012de20 con 0x7f0ab0140e60 2023-10-06T05:02:51.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.876+0000 7f0aa6ffd700 1 -- 172.21.15.3:0/3426076600 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f0ab012de20 con 0x7f0ab0140e60 2023-10-06T05:02:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:51.878 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:01:22.299089+0000","modified":"2023-10-06T05:02:47.821136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8409":{"gid":8409,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":24,"addr":"172.21.15.153:6837/2550905871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2550905871},{"type":"v1","addr":"172.21.15.153:6837","nonce":2550905871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:02:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0a90066f30 msgr2=0x7f0a900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0a90066f30 0x7f0a900693e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80011e0 tx=0x7f0aa8009040 comp rx=0 tx=0).stop 2023-10-06T05:02:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab0140e60 msgr2=0x7f0ab00afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab0140e60 0x7f0ab00afff0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800ec80 tx=0x7f0a980091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 shutdown_connections 2023-10-06T05:02:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f0a90066f30 0x7f0a900693e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab00cadc0 0x7f0ab012c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ab0141230 0x7f0ab0158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 --2- 172.21.15.3:0/3426076600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab0140e60 0x7f0ab00afff0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 >> 172.21.15.3:0/3426076600 conn(0x7f0ab0067aa0 msgr2=0x7f0ab00c3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 shutdown_connections 2023-10-06T05:02:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:51.878+0000 7f0ab58f9700 1 -- 172.21.15.3:0/3426076600 wait complete. 2023-10-06T05:02:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 91 2023-10-06T05:02:51.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:02:52.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.011+0000 7f35e7dea700 1 Processor -- start 2023-10-06T05:02:52.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.011+0000 7f35e7dea700 1 -- start start 2023-10-06T05:02:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e7dea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35e01594f0 0x7f35e00b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e7dea700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e00b4310 con 0x7f35e01598c0 2023-10-06T05:02:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e7dea700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e00b29d0 con 0x7f35e00b25a0 2023-10-06T05:02:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e7dea700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e00d0030 con 0x7f35e01594f0 2023-10-06T05:02:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e5b86700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35e01594f0 0x7f35e00b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e5385700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f35e01598c0 0x7f35e00cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60762/0 (socket says 172.21.15.3:60762) 2023-10-06T05:02:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e5385700 1 -- 172.21.15.3:0/3007569779 learned_addr learned my addr 172.21.15.3:0/3007569779 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e5b86700 1 -- 172.21.15.3:0/3007569779 >> v1:172.21.15.153:6789/0 conn(0x7f35e00b25a0 legacy=0x7f35e00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e5b86700 1 -- 172.21.15.3:0/3007569779 >> v1:172.21.15.3:6789/0 conn(0x7f35e01598c0 legacy=0x7f35e00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e4b84700 1 -- 172.21.15.3:0/3007569779 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735307990 0 0) 0x7f35e00b4310 con 0x7f35e01598c0 2023-10-06T05:02:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.012+0000 7f35e5b86700 1 -- 172.21.15.3:0/3007569779 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e00cfa70 con 0x7f35e01594f0 2023-10-06T05:02:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e5b86700 1 --2- 172.21.15.3:0/3007569779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e00b2060 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009e60 tx=0x7f35d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6b9d0d287b6856 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e4b84700 1 -- 172.21.15.3:0/3007569779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000e070 con 0x7f35e01594f0 2023-10-06T05:02:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e4b84700 1 -- 172.21.15.3:0/3007569779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000bbf0 con 0x7f35e01594f0 2023-10-06T05:02:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e4b84700 1 -- 172.21.15.3:0/3007569779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0012430 con 0x7f35e01594f0 2023-10-06T05:02:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e7dea700 1 -- 172.21.15.3:0/3007569779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 msgr2=0x7f35e00b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e7dea700 1 --2- 172.21.15.3:0/3007569779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e00b2060 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009e60 tx=0x7f35d0009780 comp rx=0 tx=0).stop 2023-10-06T05:02:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e7dea700 1 -- 172.21.15.3:0/3007569779 shutdown_connections 2023-10-06T05:02:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e7dea700 1 --2- 172.21.15.3:0/3007569779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e00b2060 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e7dea700 1 -- 172.21.15.3:0/3007569779 >> 172.21.15.3:0/3007569779 conn(0x7f35e006c2f0 msgr2=0x7f35e006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:52.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e7dea700 1 -- 172.21.15.3:0/3007569779 shutdown_connections 2023-10-06T05:02:52.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.013+0000 7f35e7dea700 1 -- 172.21.15.3:0/3007569779 wait complete. 2023-10-06T05:02:52.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 Processor -- start 2023-10-06T05:02:52.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 -- start start 2023-10-06T05:02:52.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b25a0 0x7f35e016a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e0151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e01598c0 0x7f35e01512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e016a990 con 0x7f35e01594f0 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e01524d0 con 0x7f35e01598c0 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e7dea700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e015d7b0 con 0x7f35e00b25a0 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e5385700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e0151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e5385700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e0151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37674/0 (socket says 172.21.15.3:37674) 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.015+0000 7f35e5385700 1 -- 172.21.15.3:0/4204520693 learned_addr learned my addr 172.21.15.3:0/4204520693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e6387700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e01598c0 0x7f35e01512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e5b86700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b25a0 0x7f35e016a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e5385700 1 -- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b25a0 msgr2=0x7f35e016a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e5385700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b25a0 0x7f35e016a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e5385700 1 -- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e01598c0 msgr2=0x7f35e01512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e5385700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e01598c0 0x7f35e01512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e5385700 1 -- 172.21.15.3:0/4204520693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0009a20 con 0x7f35e01594f0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e5385700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e0151f90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c5b0 tx=0x7f35d4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35ddffb700 1 -- 172.21.15.3:0/4204520693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4005b30 con 0x7f35e01594f0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35ddffb700 1 -- 172.21.15.3:0/4204520693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d4005c90 con 0x7f35e01594f0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e015da90 con 0x7f35e01594f0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35ddffb700 1 -- 172.21.15.3:0/4204520693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4007300 con 0x7f35e01594f0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.016+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e015de20 con 0x7f35e01594f0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.017+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e004c380 con 0x7f35e01594f0 2023-10-06T05:02:52.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.018+0000 7f35ddffb700 1 -- 172.21.15.3:0/4204520693 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35d401b3b0 con 0x7f35e01594f0 2023-10-06T05:02:52.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.018+0000 7f35ddffb700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f35cc066f30 0x7f35cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.018+0000 7f35ddffb700 1 -- 172.21.15.3:0/4204520693 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f35d4083c20 con 0x7f35e01594f0 2023-10-06T05:02:52.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.018+0000 7f35e5b86700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f35cc066f30 0x7f35cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.019+0000 7f35e5b86700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f35cc066f30 0x7f35cc0693e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f35d00097c0 tx=0x7f35d0018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.022+0000 7f35ddffb700 1 -- 172.21.15.3:0/4204520693 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35d4050cd0 con 0x7f35e01594f0 2023-10-06T05:02:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.218+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35e00cfd50 con 0x7f35e01594f0 2023-10-06T05:02:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.219+0000 7f35ddffb700 1 -- 172.21.15.3:0/4204520693 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f35d4017090 con 0x7f35e01594f0 2023-10-06T05:02:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:52.222 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":135,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:02:46.809978+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:01:20.044822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:01:21.270200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/3882866003":"2023-10-06T06:02:46.060249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:02:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f35cc066f30 msgr2=0x7f35cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f35cc066f30 0x7f35cc0693e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f35d00097c0 tx=0x7f35d0018000 comp rx=0 tx=0).stop 2023-10-06T05:02:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 msgr2=0x7f35e0151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e0151f90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c5b0 tx=0x7f35d4001070 comp rx=0 tx=0).stop 2023-10-06T05:02:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 shutdown_connections 2023-10-06T05:02:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f35cc066f30 0x7f35cc0693e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e01598c0 0x7f35e01512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b25a0 0x7f35e016a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 --2- 172.21.15.3:0/4204520693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35e01594f0 0x7f35e0151f90 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 >> 172.21.15.3:0/4204520693 conn(0x7f35e006c2f0 msgr2=0x7f35e00aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 shutdown_connections 2023-10-06T05:02:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.221+0000 7f35e7dea700 1 -- 172.21.15.3:0/4204520693 wait complete. 2023-10-06T05:02:52.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:02:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.353+0000 7f4d808ce700 1 Processor -- start 2023-10-06T05:02:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.353+0000 7f4d808ce700 1 -- start start 2023-10-06T05:02:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d808ce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d7c142ff0 0x7f4d7c15b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d808ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c1433c0 0x7f4d7c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d808ce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d7c15da90 0x7f4d7c0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d808ce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c15dfd0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d808ce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c15e2b0 con 0x7f4d7c15da90 2023-10-06T05:02:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d808ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0ceca0 con 0x7f4d7c1433c0 2023-10-06T05:02:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d7c142ff0 0x7f4d7c15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d7c142ff0 0x7f4d7c15b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50346/0 (socket says 172.21.15.3:50346) 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/2488169866 learned_addr learned my addr 172.21.15.3:0/2488169866 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7affd700 1 --2- 172.21.15.3:0/2488169866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c1433c0 0x7f4d7c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/2488169866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c1433c0 msgr2=0x7f4d7c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7bfff700 1 --2- 172.21.15.3:0/2488169866 >> v2:172.21.15.153:3300/0 conn(0x7f4d7c15da90 0x7f4d7c0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 --2- 172.21.15.3:0/2488169866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c1433c0 0x7f4d7c15d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/2488169866 >> v2:172.21.15.153:3300/0 conn(0x7f4d7c15da90 msgr2=0x7f4d7c0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 --2- 172.21.15.3:0/2488169866 >> v2:172.21.15.153:3300/0 conn(0x7f4d7c15da90 0x7f4d7c0b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/2488169866 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0cef80 con 0x7f4d7c142ff0 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7affd700 1 --2- 172.21.15.3:0/2488169866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c1433c0 0x7f4d7c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.354+0000 7f4d7bfff700 1 --2- 172.21.15.3:0/2488169866 >> v2:172.21.15.153:3300/0 conn(0x7f4d7c15da90 0x7f4d7c0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d7b7fe700 1 --2- 172.21.15.3:0/2488169866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c15b330 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0cdef0 tx=0x7f4d6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ce1fd675e05ace server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d7a7fc700 1 -- 172.21.15.3:0/2488169866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00b860 con 0x7f4d7c142ff0 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d7a7fc700 1 -- 172.21.15.3:0/2488169866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c00b9c0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d7a7fc700 1 -- 172.21.15.3:0/2488169866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00bd60 con 0x7f4d7c142ff0 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 -- 172.21.15.3:0/2488169866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 msgr2=0x7f4d7c15b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 --2- 172.21.15.3:0/2488169866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c15b330 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0cdef0 tx=0x7f4d6c009250 comp rx=0 tx=0).stop 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 -- 172.21.15.3:0/2488169866 shutdown_connections 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 --2- 172.21.15.3:0/2488169866 >> v2:172.21.15.153:3300/0 conn(0x7f4d7c15da90 0x7f4d7c0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 --2- 172.21.15.3:0/2488169866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c15b330 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 --2- 172.21.15.3:0/2488169866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c1433c0 0x7f4d7c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 -- 172.21.15.3:0/2488169866 >> 172.21.15.3:0/2488169866 conn(0x7f4d7c06c2f0 msgr2=0x7f4d7c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 -- 172.21.15.3:0/2488169866 shutdown_connections 2023-10-06T05:02:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.355+0000 7f4d808ce700 1 -- 172.21.15.3:0/2488169866 wait complete. 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 Processor -- start 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 -- start start 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d7c15da90 0x7f4d7c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c140380 0x7f4d7c140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c141260 con 0x7f4d7c142ff0 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c141de0 con 0x7f4d7c15da90 2023-10-06T05:02:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d808ce700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c13f700 con 0x7f4d7c140380 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d7b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d7b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c142270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50362/0 (socket says 172.21.15.3:50362) 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.357+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/1591866746 learned_addr learned my addr 172.21.15.3:0/1591866746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7affd700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d7c15da90 0x7f4d7c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c140380 msgr2=0x7f4d7c140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7bfff700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c140380 0x7f4d7c140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7b7fe700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c140380 0x7f4d7c140820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d7c15da90 msgr2=0x7f4d7c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7b7fe700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d7c15da90 0x7f4d7c1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7b7fe700 1 -- 172.21.15.3:0/1591866746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0094f0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7affd700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d7c15da90 0x7f4d7c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d7b7fe700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c142270 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006fd0 tx=0x7f4d6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d637fe700 1 -- 172.21.15.3:0/1591866746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c024440 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d637fe700 1 -- 172.21.15.3:0/1591866746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c0245a0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d637fe700 1 -- 172.21.15.3:0/1591866746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c024960 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c1427b0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.358+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c1414b0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.359+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c1420c0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.359+0000 7f4d637fe700 1 -- 172.21.15.3:0/1591866746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d6c01a420 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.360+0000 7f4d637fe700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d68066f30 0x7f4d680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.360+0000 7f4d637fe700 1 -- 172.21.15.3:0/1591866746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4d6c086510 con 0x7f4d7c142ff0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.360+0000 7f4d7affd700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d68066f30 0x7f4d680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.361+0000 7f4d7affd700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d68066f30 0x7f4d680693e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b2040 tx=0x7f4d70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.364+0000 7f4d637fe700 1 -- 172.21.15.3:0/1591866746 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d6c0535c0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.603+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4d7c0644f0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.605 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-06T05:02:52.604+0000 7fc10117b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:02:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.608+0000 7f4d637fe700 1 -- 172.21.15.3:0/1591866746 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4d6c056be0 con 0x7f4d7c142ff0 2023-10-06T05:02:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d68066f30 msgr2=0x7f4d680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d68066f30 0x7f4d680693e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b2040 tx=0x7f4d70008040 comp rx=0 tx=0).stop 2023-10-06T05:02:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 msgr2=0x7f4d7c142270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c142270 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006fd0 tx=0x7f4d6c017000 comp rx=0 tx=0).stop 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 shutdown_connections 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f4d68066f30 0x7f4d680693e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d7c140380 0x7f4d7c140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7c142ff0 0x7f4d7c142270 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 --2- 172.21.15.3:0/1591866746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d7c15da90 0x7f4d7c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 >> 172.21.15.3:0/1591866746 conn(0x7f4d7c06c2f0 msgr2=0x7f4d7c0cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 shutdown_connections 2023-10-06T05:02:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.610+0000 7f4d808ce700 1 -- 172.21.15.3:0/1591866746 wait complete. 2023-10-06T05:02:52.614 INFO:tasks.ceph.mds.b.smithi153.stderr: -5461> 2023-10-06T05:01:27.200+0000 7fedbe4b5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:02:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:02:52.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:02:52.641 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-10-06T05:02:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.744+0000 7fd8578a8700 1 Processor -- start 2023-10-06T05:02:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.744+0000 7fd8578a8700 1 -- start start 2023-10-06T05:02:52.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd8578a8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8501598d0 0x7fd8500aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd8578a8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd850159ca0 0x7fd8500aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd8578a8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500b0ab0 con 0x7fd8500ab1a0 2023-10-06T05:02:52.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd8578a8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8500b08c0 con 0x7fd8501598d0 2023-10-06T05:02:52.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd8578a8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500b0d90 con 0x7fd850159ca0 2023-10-06T05:02:52.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd855644700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8501598d0 0x7fd8500aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd855644700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8501598d0 0x7fd8500aa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50368/0 (socket says 172.21.15.3:50368) 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd855644700 1 -- 172.21.15.3:0/3512876240 learned_addr learned my addr 172.21.15.3:0/3512876240 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.745+0000 7fd854e43700 1 --2- 172.21.15.3:0/3512876240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd850159ca0 0x7fd8500aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd855644700 1 -- 172.21.15.3:0/3512876240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd850159ca0 msgr2=0x7fd8500aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd855644700 1 --2- 172.21.15.3:0/3512876240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd850159ca0 0x7fd8500aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd855644700 1 -- 172.21.15.3:0/3512876240 >> v1:172.21.15.153:6789/0 conn(0x7fd8500ab1a0 legacy=0x7fd8500b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd855644700 1 -- 172.21.15.3:0/3512876240 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500b1070 con 0x7fd8501598d0 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd855644700 1 --2- 172.21.15.3:0/3512876240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8501598d0 0x7fd8500aa720 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd848009990 tx=0x7fd848009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb0e8ac0833a3c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd84ffff700 1 -- 172.21.15.3:0/3512876240 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848005c10 con 0x7fd8501598d0 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd84ffff700 1 -- 172.21.15.3:0/3512876240 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848005d70 con 0x7fd8501598d0 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd84ffff700 1 -- 172.21.15.3:0/3512876240 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84800b880 con 0x7fd8501598d0 2023-10-06T05:02:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd8578a8700 1 -- 172.21.15.3:0/3512876240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8501598d0 msgr2=0x7fd8500aa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd8578a8700 1 --2- 172.21.15.3:0/3512876240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8501598d0 0x7fd8500aa720 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd848009990 tx=0x7fd848009220 comp rx=0 tx=0).stop 2023-10-06T05:02:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd8578a8700 1 -- 172.21.15.3:0/3512876240 shutdown_connections 2023-10-06T05:02:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.746+0000 7fd8578a8700 1 --2- 172.21.15.3:0/3512876240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8501598d0 0x7fd8500aa720 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.747+0000 7fd8578a8700 1 --2- 172.21.15.3:0/3512876240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd850159ca0 0x7fd8500aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.747+0000 7fd8578a8700 1 -- 172.21.15.3:0/3512876240 >> 172.21.15.3:0/3512876240 conn(0x7fd850099970 msgr2=0x7fd85006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.747+0000 7fd8578a8700 1 -- 172.21.15.3:0/3512876240 shutdown_connections 2023-10-06T05:02:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.747+0000 7fd8578a8700 1 -- 172.21.15.3:0/3512876240 wait complete. 2023-10-06T05:02:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.748+0000 7fd8578a8700 1 Processor -- start 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.748+0000 7fd8578a8700 1 -- start start 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.748+0000 7fd8578a8700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8500ab1a0 0x7fd85009cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd8578a8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8501598d0 0x7fd85015f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd8578a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd850159ca0 0x7fd850157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd8578a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85009a260 con 0x7fd850159ca0 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd8578a8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85009d5d0 con 0x7fd8501598d0 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd8578a8700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85009d8b0 con 0x7fd8500ab1a0 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd850159ca0 0x7fd850157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd850159ca0 0x7fd850157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50382/0 (socket says 172.21.15.3:50382) 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 -- 172.21.15.3:0/1910532248 learned_addr learned my addr 172.21.15.3:0/1910532248 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855644700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8500ab1a0 0x7fd85009cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd854e43700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8501598d0 0x7fd85015f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 -- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8500ab1a0 msgr2=0x7fd85009cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8500ab1a0 0x7fd85009cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 -- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8501598d0 msgr2=0x7fd85015f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8501598d0 0x7fd85015f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 -- 172.21.15.3:0/1910532248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848009480 con 0x7fd850159ca0 2023-10-06T05:02:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd855e45700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd850159ca0 0x7fd850157c30 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c8e0 tx=0x7fd840001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd84d7fa700 1 -- 172.21.15.3:0/1910532248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000ce80 con 0x7fd850159ca0 2023-10-06T05:02:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd84d7fa700 1 -- 172.21.15.3:0/1910532248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84000a7e0 con 0x7fd850159ca0 2023-10-06T05:02:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85009db90 con 0x7fd850159ca0 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd84d7fa700 1 -- 172.21.15.3:0/1910532248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000ab80 con 0x7fd850159ca0 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.749+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8500cee00 con 0x7fd850159ca0 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.750+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85009a410 con 0x7fd850159ca0 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.751+0000 7fd84d7fa700 1 -- 172.21.15.3:0/1910532248 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd840012420 con 0x7fd850159ca0 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.751+0000 7fd84d7fa700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd83c066f30 0x7fd83c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.751+0000 7fd84d7fa700 1 -- 172.21.15.3:0/1910532248 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fd840084a00 con 0x7fd850159ca0 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.751+0000 7fd855644700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd83c066f30 0x7fd83c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.752+0000 7fd855644700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd83c066f30 0x7fd83c0693e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd848009260 tx=0x7fd848016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:52.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.756+0000 7fd84d7fa700 1 -- 172.21.15.3:0/1910532248 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd85009a410 con 0x7fd850159ca0 2023-10-06T05:02:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:52.995+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd8500644f0 con 0x7fd850159ca0 2023-10-06T05:02:53.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.608+0000 7fd84d7fa700 1 -- 172.21.15.3:0/1910532248 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd840051a50 con 0x7fd850159ca0 2023-10-06T05:02:53.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd83c066f30 msgr2=0x7fd83c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:53.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd83c066f30 0x7fd83c0693e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd848009260 tx=0x7fd848016000 comp rx=0 tx=0).stop 2023-10-06T05:02:53.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd850159ca0 msgr2=0x7fd850157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:53.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd850159ca0 0x7fd850157c30 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c8e0 tx=0x7fd840001710 comp rx=0 tx=0).stop 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 shutdown_connections 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd83c066f30 0x7fd83c0693e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd850159ca0 0x7fd850157c30 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8500ab1a0 0x7fd85009cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 --2- 172.21.15.3:0/1910532248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8501598d0 0x7fd85015f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 >> 172.21.15.3:0/1910532248 conn(0x7fd850099970 msgr2=0x7fd850158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 shutdown_connections 2023-10-06T05:02:53.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.611+0000 7fd8578a8700 1 -- 172.21.15.3:0/1910532248 wait complete. 2023-10-06T05:02:53.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:02:53.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.751+0000 7fd95759e700 1 Processor -- start 2023-10-06T05:02:53.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.751+0000 7fd95759e700 1 -- start start 2023-10-06T05:02:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.752+0000 7fd95759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd9580b4d00 0x7fd958164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.752+0000 7fd95759e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9580cd7e0 con 0x7fd9580b31b0 2023-10-06T05:02:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.752+0000 7fd95759e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9580b3730 con 0x7fd9580aa9a0 2023-10-06T05:02:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.752+0000 7fd95759e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9580aff50 con 0x7fd9580b4d00 2023-10-06T05:02:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.752+0000 7fd956d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd9580b4d00 0x7fd958164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.752+0000 7fd955d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd9580b31b0 0x7fd9580b45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37768/0 (socket says 172.21.15.3:37768) 2023-10-06T05:02:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.752+0000 7fd955d9b700 1 -- 172.21.15.3:0/4229251282 learned_addr learned my addr 172.21.15.3:0/4229251282 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd956d9d700 1 -- 172.21.15.3:0/4229251282 >> v1:172.21.15.153:6789/0 conn(0x7fd9580aa9a0 legacy=0x7fd9580ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd956d9d700 1 -- 172.21.15.3:0/4229251282 >> v1:172.21.15.3:6789/0 conn(0x7fd9580b31b0 legacy=0x7fd9580b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95559a700 1 -- 172.21.15.3:0/4229251282 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929388982 0 0) 0x7fd9580cd7e0 con 0x7fd9580b31b0 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd956d9d700 1 -- 172.21.15.3:0/4229251282 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9580b3a10 con 0x7fd9580b4d00 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd956d9d700 1 --2- 172.21.15.3:0/4229251282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd958164740 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd948009e60 tx=0x7fd948009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42210f5808a5405 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95559a700 1 -- 172.21.15.3:0/4229251282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800e070 con 0x7fd9580b4d00 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95559a700 1 -- 172.21.15.3:0/4229251282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94800bbf0 con 0x7fd9580b4d00 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95559a700 1 -- 172.21.15.3:0/4229251282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948012430 con 0x7fd9580b4d00 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95759e700 1 -- 172.21.15.3:0/4229251282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 msgr2=0x7fd958164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95759e700 1 --2- 172.21.15.3:0/4229251282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd958164740 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd948009e60 tx=0x7fd948009780 comp rx=0 tx=0).stop 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95759e700 1 -- 172.21.15.3:0/4229251282 shutdown_connections 2023-10-06T05:02:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95759e700 1 --2- 172.21.15.3:0/4229251282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd958164740 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.753+0000 7fd95759e700 1 -- 172.21.15.3:0/4229251282 >> 172.21.15.3:0/4229251282 conn(0x7fd95806c2f0 msgr2=0x7fd95806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.754+0000 7fd95759e700 1 -- 172.21.15.3:0/4229251282 shutdown_connections 2023-10-06T05:02:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.754+0000 7fd95759e700 1 -- 172.21.15.3:0/4229251282 wait complete. 2023-10-06T05:02:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 Processor -- start 2023-10-06T05:02:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 -- start start 2023-10-06T05:02:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9580aa9a0 0x7fd958158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9580b31b0 0x7fd9580ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd95806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9581594c0 con 0x7fd9580b4d00 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580abe70 con 0x7fd9580aa9a0 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd95759e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580ac110 con 0x7fd9580b31b0 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd956d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd95806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd956d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd95806d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50404/0 (socket says 172.21.15.3:50404) 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.755+0000 7fd956d9d700 1 -- 172.21.15.3:0/3343516167 learned_addr learned my addr 172.21.15.3:0/3343516167 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd955d9b700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9580b31b0 0x7fd9580ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd95659c700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9580aa9a0 0x7fd958158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd956d9d700 1 -- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9580b31b0 msgr2=0x7fd9580ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd956d9d700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9580b31b0 0x7fd9580ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd956d9d700 1 -- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9580aa9a0 msgr2=0x7fd958158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:53.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd956d9d700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9580aa9a0 0x7fd958158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd956d9d700 1 -- 172.21.15.3:0/3343516167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948009a20 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd95659c700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9580aa9a0 0x7fd958158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd955d9b700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9580b31b0 0x7fd9580ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd956d9d700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd95806d9a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd948003000 tx=0x7fd948018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd94e7fc700 1 -- 172.21.15.3:0/3343516167 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94801d640 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd94e7fc700 1 -- 172.21.15.3:0/3343516167 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94801d7a0 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9580ac3f0 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd94e7fc700 1 -- 172.21.15.3:0/3343516167 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94801db60 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.756+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9580ac9b0 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.757+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9580afa80 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.757+0000 7fd94e7fc700 1 -- 172.21.15.3:0/3343516167 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd948024070 con 0x7fd9580b4d00 2023-10-06T05:02:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.758+0000 7fd94e7fc700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd938066f30 0x7fd9380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.758+0000 7fd94e7fc700 1 -- 172.21.15.3:0/3343516167 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fd948087130 con 0x7fd9580b4d00 2023-10-06T05:02:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.758+0000 7fd95659c700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd938066f30 0x7fd9380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.759+0000 7fd95659c700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd938066f30 0x7fd9380693e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd950007fd0 tx=0x7fd950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.762+0000 7fd94e7fc700 1 -- 172.21.15.3:0/3343516167 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd948054180 con 0x7fd9580b4d00 2023-10-06T05:02:53.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:53.959+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd95804bb40 con 0x7fd9580b4d00 2023-10-06T05:02:54.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.851+0000 7fd94e7fc700 1 -- 172.21.15.3:0/3343516167 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd9480577a0 con 0x7fd9580b4d00 2023-10-06T05:02:54.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd938066f30 msgr2=0x7fd9380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd938066f30 0x7fd9380693e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd950007fd0 tx=0x7fd950009040 comp rx=0 tx=0).stop 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 msgr2=0x7fd95806d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd95806d9a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd948003000 tx=0x7fd948018000 comp rx=0 tx=0).stop 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 shutdown_connections 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd938066f30 0x7fd9380693e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9580aa9a0 0x7fd958158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9580b4d00 0x7fd95806d9a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 --2- 172.21.15.3:0/3343516167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9580b31b0 0x7fd9580ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 >> 172.21.15.3:0/3343516167 conn(0x7fd95806c2f0 msgr2=0x7fd958159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 shutdown_connections 2023-10-06T05:02:54.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:54.857+0000 7fd95759e700 1 -- 172.21.15.3:0/3343516167 wait complete. 2023-10-06T05:02:54.860 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:02:54.878 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:02:55.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.006+0000 7f6a65775700 1 Processor -- start 2023-10-06T05:02:55.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.006+0000 7f6a65775700 1 -- start start 2023-10-06T05:02:55.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a65775700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a600b1930 0x7f6a600b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a65775700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 0x7f6a600ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a65775700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a600afc20 con 0x7f6a60159580 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a65775700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a600b3c20 con 0x7f6a600b1930 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a65775700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600af010 con 0x7f6a60159950 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a5effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6a60159580 0x7f6a600b1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37782/0 (socket says 172.21.15.3:37782) 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a5effd700 1 -- 172.21.15.3:0/3910471054 learned_addr learned my addr 172.21.15.3:0/3910471054 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/3910471054 >> v2:172.21.15.153:3300/0 conn(0x7f6a600b1930 0x7f6a600b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a5f7fe700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 0x7f6a600ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302499958 0 0) 0x7f6a600afc20 con 0x7f6a60159580 2023-10-06T05:02:55.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a4c003570 con 0x7f6a60159580 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.007+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105051730 0 0) 0x7f6a4c003570 con 0x7f6a60159580 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 msgr2=0x7f6a600ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 0x7f6a600ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 >> v2:172.21.15.153:3300/0 conn(0x7f6a600b1930 msgr2=0x7f6a600b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 --2- 172.21.15.3:0/3910471054 >> v2:172.21.15.153:3300/0 conn(0x7f6a600b1930 0x7f6a600b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a600af2f0 con 0x7f6a60159580 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f6a58001470 con 0x7f6a60159580 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 >> v1:172.21.15.3:6789/0 conn(0x7f6a60159580 legacy=0x7f6a600b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 0x7f6a4c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a4c004d50 0x7f6a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600b3c20 con 0x7f6a4c003fe0 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600af010 con 0x7f6a4c004d50 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c00ba90 con 0x7f6a60159950 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5effd700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 0x7f6a4c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5f7fe700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a4c004d50 0x7f6a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5e7fc700 1 -- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 msgr2=0x7f6a4c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 0x7f6a4c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5e7fc700 1 -- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a4c004d50 msgr2=0x7f6a4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a4c004d50 0x7f6a4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5e7fc700 1 -- 172.21.15.3:0/3910471054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c003570 con 0x7f6a4c003fe0 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5effd700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60159950 0x7f6a4c003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5f7fe700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a4c004d50 0x7f6a4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a4c004810 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009c00 tx=0x7f6a480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fc0f250a9f0932 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800e6b0 con 0x7f6a4c003fe0 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4800e810 con 0x7f6a4c003fe0 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.008+0000 7f6a5dffb700 1 -- 172.21.15.3:0/3910471054 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800ebd0 con 0x7f6a4c003fe0 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.009+0000 7f6a65775700 1 -- 172.21.15.3:0/3910471054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 msgr2=0x7f6a4c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.009+0000 7f6a65775700 1 --2- 172.21.15.3:0/3910471054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a4c004810 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009c00 tx=0x7f6a480091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.009+0000 7f6a5f7fe700 1 -- 172.21.15.3:0/3910471054 reap_dead start 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.009+0000 7f6a65775700 1 -- 172.21.15.3:0/3910471054 shutdown_connections 2023-10-06T05:02:55.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.009+0000 7f6a65775700 1 -- 172.21.15.3:0/3910471054 >> 172.21.15.3:0/3910471054 conn(0x7f6a60087fe0 msgr2=0x7f6a600999b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:55.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.009+0000 7f6a65775700 1 -- 172.21.15.3:0/3910471054 shutdown_connections 2023-10-06T05:02:55.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.009+0000 7f6a65775700 1 -- 172.21.15.3:0/3910471054 wait complete. 2023-10-06T05:02:55.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.010+0000 7f6a65775700 1 Processor -- start 2023-10-06T05:02:55.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.010+0000 7f6a65775700 1 -- start start 2023-10-06T05:02:55.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a65775700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a60159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a65775700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60150620 0x7f6a6015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a65775700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a6016aeb0 0x7f6a6014a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a65775700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6016aa20 con 0x7f6a4c003fe0 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a65775700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6014a5b0 con 0x7f6a6016aeb0 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a65775700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6015a950 con 0x7f6a60150620 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a60159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a60159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50426/0 (socket says 172.21.15.3:50426) 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 -- 172.21.15.3:0/2964306193 learned_addr learned my addr 172.21.15.3:0/2964306193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5f7fe700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a6016aeb0 0x7f6a6014a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60150620 0x7f6a6015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 -- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60150620 msgr2=0x7f6a6015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60150620 0x7f6a6015a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 -- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a6016aeb0 msgr2=0x7f6a6014a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a6016aeb0 0x7f6a6014a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 -- 172.21.15.3:0/2964306193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6016a3d0 con 0x7f6a4c003fe0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60150620 0x7f6a6015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.011+0000 7f6a5effd700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a60159de0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800e980 tx=0x7f6a58000f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.012+0000 7f6a56ffd700 1 -- 172.21.15.3:0/2964306193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800ee70 con 0x7f6a4c003fe0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.012+0000 7f6a56ffd700 1 -- 172.21.15.3:0/2964306193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5800c760 con 0x7f6a4c003fe0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.012+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6016a6b0 con 0x7f6a4c003fe0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.012+0000 7f6a56ffd700 1 -- 172.21.15.3:0/2964306193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800cb00 con 0x7f6a4c003fe0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.012+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6014ad60 con 0x7f6a4c003fe0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.013+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6016a240 con 0x7f6a4c003fe0 2023-10-06T05:02:55.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.013+0000 7f6a56ffd700 1 -- 172.21.15.3:0/2964306193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a5801d460 con 0x7f6a4c003fe0 2023-10-06T05:02:55.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.013+0000 7f6a56ffd700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6a40066f30 0x7f6a400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:55.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.014+0000 7f6a56ffd700 1 -- 172.21.15.3:0/2964306193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f6a58084560 con 0x7f6a4c003fe0 2023-10-06T05:02:55.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.014+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6a40066f30 0x7f6a400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:55.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.014+0000 7f6a5e7fc700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6a40066f30 0x7f6a400693e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6a480091e0 tx=0x7f6a4801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:55.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.018+0000 7f6a56ffd700 1 -- 172.21.15.3:0/2964306193 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a58051770 con 0x7f6a4c003fe0 2023-10-06T05:02:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.213+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a6004c380 con 0x7f6a4c003fe0 2023-10-06T05:02:55.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.860+0000 7f6a56ffd700 1 -- 172.21.15.3:0/2964306193 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6a58054d90 con 0x7f6a4c003fe0 2023-10-06T05:02:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6a40066f30 msgr2=0x7f6a400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6a40066f30 0x7f6a400693e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6a480091e0 tx=0x7f6a4801a000 comp rx=0 tx=0).stop 2023-10-06T05:02:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 msgr2=0x7f6a60159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a60159de0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800e980 tx=0x7f6a58000f90 comp rx=0 tx=0).stop 2023-10-06T05:02:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 shutdown_connections 2023-10-06T05:02:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6a40066f30 0x7f6a400693e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a6016aeb0 0x7f6a6014a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a4c003fe0 0x7f6a60159de0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 --2- 172.21.15.3:0/2964306193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a60150620 0x7f6a6015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.864+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 >> 172.21.15.3:0/2964306193 conn(0x7f6a60087fe0 msgr2=0x7f6a6015fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.865+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 shutdown_connections 2023-10-06T05:02:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:55.865+0000 7f6a65775700 1 -- 172.21.15.3:0/2964306193 wait complete. 2023-10-06T05:02:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-10-06T05:02:55.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:02:55.888 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:02:55.937 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-06T05:02:55.937 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.938 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:02:55.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:02:55.940 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T05:02:55.941 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-06T05:02:55.942 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-06T05:02:55.943 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-06T05:02:55.944 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:02:55.945 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:02:55.946 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-06T05:02:55.947 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:02:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-06T05:02:55.949 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T05:02:55.950 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-06T05:02:55.951 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-06T05:02:55.952 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-06T05:02:55.952 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-06T05:02:55.952 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-06T05:02:55.952 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-06T05:02:55.952 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-06T05:02:55.952 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-06T05:02:55.952 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-10-06T05:02:55.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:55.954 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-10-06T05:02:55.954 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-10-06T05:02:55.955 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:02:55.956 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.957 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.3,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = fa7cd5af-16f2-4a73-ab30-42f10cc13e22 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-10-06T05:02:55.958 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-10-06T05:02:55.959 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-10-06T05:02:55.960 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:02:55.961 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-10-06T05:02:55.962 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:02:55.963 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.964 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-10-06T05:02:55.965 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-10-06T05:02:55.966 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-10-06T05:02:55.967 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-10-06T05:02:55.967 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-10-06T05:02:55.967 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-10-06T05:02:55.967 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-10-06T05:02:55.967 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-10-06T05:02:55.967 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-10-06T05:02:55.967 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2023-10-06T05:02:55.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:02:55.996 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.118+0000 7efef9eaf700 1 Processor -- start 2023-10-06T05:02:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.118+0000 7efef9eaf700 1 -- start start 2023-10-06T05:02:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.119+0000 7efef9eaf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efef40ac620 0x7efef4158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.119+0000 7efef9eaf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef415aa50 0x7efef40ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.119+0000 7efef9eaf700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef40676f0 con 0x7efef40ac9f0 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.119+0000 7efef9eaf700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef40af3e0 con 0x7efef40ac620 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.119+0000 7efef9eaf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef40af100 con 0x7efef415aa50 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef3fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef415aa50 0x7efef40ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef37fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efef40ac620 0x7efef4158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef2ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7efef40ac9f0 0x7efef415a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34210/0 (socket says 172.21.15.3:34210) 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef2ffd700 1 -- 172.21.15.3:0/3464584661 learned_addr learned my addr 172.21.15.3:0/3464584661 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef37fe700 1 -- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef415aa50 msgr2=0x7efef40ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef37fe700 1 --2- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef415aa50 0x7efef40ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef37fe700 1 -- 172.21.15.3:0/3464584661 >> v1:172.21.15.153:6789/0 conn(0x7efef40ac9f0 legacy=0x7efef415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef37fe700 1 -- 172.21.15.3:0/3464584661 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40670a0 con 0x7efef40ac620 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef27fc700 1 -- 172.21.15.3:0/3464584661 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096376979 0 0) 0x7efef40676f0 con 0x7efef40ac9f0 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef3fff700 1 --2- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef415aa50 0x7efef40ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef37fe700 1 --2- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef40ac620 0x7efef4158b60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efee40098e0 tx=0x7efee40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66d9e834bcd1ae6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef27fc700 1 -- 172.21.15.3:0/3464584661 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400b640 con 0x7efef40ac620 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef27fc700 1 -- 172.21.15.3:0/3464584661 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400b7a0 con 0x7efef40ac620 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.120+0000 7efef27fc700 1 -- 172.21.15.3:0/3464584661 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400bb40 con 0x7efef40ac620 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 -- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef40ac620 msgr2=0x7efef4158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 --2- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef40ac620 0x7efef4158b60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efee40098e0 tx=0x7efee40091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 -- 172.21.15.3:0/3464584661 shutdown_connections 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 --2- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef415aa50 0x7efef40ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 --2- 172.21.15.3:0/3464584661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef40ac620 0x7efef4158b60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 -- 172.21.15.3:0/3464584661 >> 172.21.15.3:0/3464584661 conn(0x7efef40c9400 msgr2=0x7efef40c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 -- 172.21.15.3:0/3464584661 shutdown_connections 2023-10-06T05:02:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.121+0000 7efef9eaf700 1 -- 172.21.15.3:0/3464584661 wait complete. 2023-10-06T05:02:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.122+0000 7efef9eaf700 1 Processor -- start 2023-10-06T05:02:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.122+0000 7efef9eaf700 1 -- start start 2023-10-06T05:02:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef9eaf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef40ac620 0x7efef40aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef9eaf700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef40ac9f0 0x7efef40c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef9eaf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef415aa50 0x7efef40b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef9eaf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40b06f0 con 0x7efef40ac620 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef9eaf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40af820 con 0x7efef415aa50 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef9eaf700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef40b1c60 con 0x7efef40ac9f0 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef40ac620 0x7efef40aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef40ac620 0x7efef40aade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50428/0 (socket says 172.21.15.3:50428) 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 -- 172.21.15.3:0/1578865256 learned_addr learned my addr 172.21.15.3:0/1578865256 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef2ffd700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef40ac9f0 0x7efef40c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef3fff700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef415aa50 0x7efef40b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 -- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef40ac9f0 msgr2=0x7efef40c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef40ac9f0 0x7efef40c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 -- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef415aa50 msgr2=0x7efef40b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef415aa50 0x7efef40b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef37fe700 1 -- 172.21.15.3:0/1578865256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec009a20 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef2ffd700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef40ac9f0 0x7efef40c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.123+0000 7efef3fff700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef415aa50 0x7efef40b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.124+0000 7efef37fe700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef40ac620 0x7efef40aade0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7efee40093f0 tx=0x7efee4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.124+0000 7efedb7fe700 1 -- 172.21.15.3:0/1578865256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4000bc0 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.124+0000 7efedb7fe700 1 -- 172.21.15.3:0/1578865256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400b910 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.124+0000 7efedb7fe700 1 -- 172.21.15.3:0/1578865256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4025670 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.124+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee4009420 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.124+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef40afd50 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.125+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef404c380 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.125+0000 7efedb7fe700 1 -- 172.21.15.3:0/1578865256 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efee4023070 con 0x7efef40ac620 2023-10-06T05:02:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.126+0000 7efedb7fe700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efee0066fb0 0x7efee0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.126+0000 7efedb7fe700 1 -- 172.21.15.3:0/1578865256 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efee40856d0 con 0x7efef40ac620 2023-10-06T05:02:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.126+0000 7efef2ffd700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efee0066fb0 0x7efee0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.126+0000 7efef2ffd700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efee0066fb0 0x7efee0069460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efef40c3570 tx=0x7efee8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:56.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.130+0000 7efedb7fe700 1 -- 172.21.15.3:0/1578865256 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efee4052a10 con 0x7efef40ac620 2023-10-06T05:02:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.324+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efef4067380 con 0x7efef40ac620 2023-10-06T05:02:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.327+0000 7efedb7fe700 1 -- 172.21.15.3:0/1578865256 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7efee4056030 con 0x7efef40ac620 2023-10-06T05:02:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.329+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efee0066fb0 msgr2=0x7efee0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.329+0000 7efef9eaf700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efee0066fb0 0x7efee0069460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efef40c3570 tx=0x7efee8009040 comp rx=0 tx=0).stop 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.329+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef40ac620 msgr2=0x7efef40aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.329+0000 7efef9eaf700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef40ac620 0x7efef40aade0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7efee40093f0 tx=0x7efee4017000 comp rx=0 tx=0).stop 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.329+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 shutdown_connections 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.329+0000 7efef9eaf700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7efee0066fb0 0x7efee0069460 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.329+0000 7efef9eaf700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efef415aa50 0x7efef40b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.330+0000 7efef9eaf700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef40ac620 0x7efef40aade0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.330+0000 7efef9eaf700 1 --2- 172.21.15.3:0/1578865256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efef40ac9f0 0x7efef40c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.330+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 >> 172.21.15.3:0/1578865256 conn(0x7efef40c9400 msgr2=0x7efef40cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.330+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 shutdown_connections 2023-10-06T05:02:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.330+0000 7efef9eaf700 1 -- 172.21.15.3:0/1578865256 wait complete. 2023-10-06T05:02:56.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2023-10-06T05:02:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.501+0000 7f83da447700 1 Processor -- start 2023-10-06T05:02:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.501+0000 7f83da447700 1 -- start start 2023-10-06T05:02:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83da447700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f83d40cd6c0 0x7f83d415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83da447700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 0x7f83d40cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83da447700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d40cd230 con 0x7f83d409d980 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83da447700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d415e4c0 con 0x7f83d40cd6c0 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83da447700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cc660 con 0x7f83d40c6da0 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f83d409d980 0x7f83d40692f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37786/0 (socket says 172.21.15.3:37786) 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d3fff700 1 -- 172.21.15.3:0/286519921 learned_addr learned my addr 172.21.15.3:0/286519921 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d37fe700 1 --2- 172.21.15.3:0/286519921 >> v2:172.21.15.153:3300/0 conn(0x7f83d40cd6c0 0x7f83d415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d89e4700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 0x7f83d40cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091865223 0 0) 0x7f83d40cd230 con 0x7f83d409d980 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83bc003570 con 0x7f83d409d980 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1554363725 0 0) 0x7f83bc003570 con 0x7f83d409d980 2023-10-06T05:02:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 msgr2=0x7f83d40cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 0x7f83d40cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 >> v2:172.21.15.153:3300/0 conn(0x7f83d40cd6c0 msgr2=0x7f83d415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 --2- 172.21.15.3:0/286519921 >> v2:172.21.15.153:3300/0 conn(0x7f83d40cd6c0 0x7f83d415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d40cc900 con 0x7f83d409d980 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f83c40015f0 con 0x7f83d409d980 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.502+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 >> v1:172.21.15.3:6789/0 conn(0x7f83d409d980 legacy=0x7f83d40692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83bc003a60 0x7f83bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83bc004810 0x7f83bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 0x7f83bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d415e4c0 con 0x7f83bc004810 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cc660 con 0x7f83bc003a60 2023-10-06T05:02:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc00ba60 con 0x7f83d40c6da0 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d37fe700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83bc004810 0x7f83bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d3fff700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83bc003a60 0x7f83bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d89e4700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 0x7f83bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d37fe700 1 -- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 msgr2=0x7f83bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d37fe700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83d40c6da0 0x7f83bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d37fe700 1 -- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83bc003a60 msgr2=0x7f83bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d37fe700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83bc003a60 0x7f83bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d37fe700 1 -- 172.21.15.3:0/286519921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc003570 con 0x7f83bc004810 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d37fe700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83bc004810 0x7f83bc004c80 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009c30 tx=0x7f83c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3c668da5f3c37f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800e710 con 0x7f83bc004810 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c800e870 con 0x7f83bc004810 2023-10-06T05:02:56.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.503+0000 7f83d2ffd700 1 -- 172.21.15.3:0/286519921 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800ec30 con 0x7f83bc004810 2023-10-06T05:02:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.504+0000 7f83da447700 1 -- 172.21.15.3:0/286519921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83bc004810 msgr2=0x7f83bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.504+0000 7f83da447700 1 --2- 172.21.15.3:0/286519921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83bc004810 0x7f83bc004c80 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009c30 tx=0x7f83c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.504+0000 7f83d89e4700 1 -- 172.21.15.3:0/286519921 reap_dead start 2023-10-06T05:02:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.504+0000 7f83da447700 1 -- 172.21.15.3:0/286519921 shutdown_connections 2023-10-06T05:02:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.504+0000 7f83da447700 1 -- 172.21.15.3:0/286519921 >> 172.21.15.3:0/286519921 conn(0x7f83d406e730 msgr2=0x7f83d406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.504+0000 7f83da447700 1 -- 172.21.15.3:0/286519921 shutdown_connections 2023-10-06T05:02:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.504+0000 7f83da447700 1 -- 172.21.15.3:0/286519921 wait complete. 2023-10-06T05:02:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.505+0000 7f83da447700 1 Processor -- start 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.505+0000 7f83da447700 1 -- start start 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83da447700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83bc004810 0x7f83d40ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83da447700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d40c9500 0x7f83d416de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83da447700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83d41586a0 0x7f83d4158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83da447700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d415c100 con 0x7f83d41586a0 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83da447700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40c9d30 con 0x7f83d40c9500 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83da447700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d4159080 con 0x7f83bc004810 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83d41586a0 0x7f83d4158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83d41586a0 0x7f83d4158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50442/0 (socket says 172.21.15.3:50442) 2023-10-06T05:02:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 -- 172.21.15.3:0/1822674353 learned_addr learned my addr 172.21.15.3:0/1822674353 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d37fe700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d40c9500 0x7f83d416de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d3fff700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83bc004810 0x7f83d40ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 -- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83bc004810 msgr2=0x7f83d40ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83bc004810 0x7f83d40ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 -- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d40c9500 msgr2=0x7f83d416de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d40c9500 0x7f83d416de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 -- 172.21.15.3:0/1822674353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4169fc0 con 0x7f83d41586a0 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d3fff700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83bc004810 0x7f83d40ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83d89e4700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83d41586a0 0x7f83d4158b40 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f83d4067490 tx=0x7f83cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.506+0000 7f83bbfff700 1 -- 172.21.15.3:0/1822674353 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc00e040 con 0x7f83d41586a0 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.507+0000 7f83bbfff700 1 -- 172.21.15.3:0/1822674353 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc00f040 con 0x7f83d41586a0 2023-10-06T05:02:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.507+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d416ab40 con 0x7f83d41586a0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.507+0000 7f83bbfff700 1 -- 172.21.15.3:0/1822674353 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc00fe20 con 0x7f83d41586a0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.507+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d416a210 con 0x7f83d41586a0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.508+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d416acd0 con 0x7f83d41586a0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.508+0000 7f83bbfff700 1 -- 172.21.15.3:0/1822674353 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83cc015640 con 0x7f83d41586a0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.508+0000 7f83bbfff700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83c0066f30 0x7f83c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.509+0000 7f83bbfff700 1 -- 172.21.15.3:0/1822674353 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f83cc084ae0 con 0x7f83d41586a0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.509+0000 7f83d3fff700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83c0066f30 0x7f83c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.509+0000 7f83d3fff700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83c0066f30 0x7f83c00693e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000ef0 tx=0x7f83c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:56.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.513+0000 7f83bbfff700 1 -- 172.21.15.3:0/1822674353 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83cc051ec0 con 0x7f83d41586a0 2023-10-06T05:02:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.785+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) v1 -- 0x7f83d404c380 con 0x7f83d41586a0 2023-10-06T05:02:56.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.860+0000 7f83bbfff700 1 -- 172.21.15.3:0/1822674353 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v526) v1 ==== 151+0+0 (secure 0 0 0) 0x7f83cc0554e0 con 0x7f83d41586a0 2023-10-06T05:02:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83c0066f30 msgr2=0x7f83c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83c0066f30 0x7f83c00693e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000ef0 tx=0x7f83c4009040 comp rx=0 tx=0).stop 2023-10-06T05:02:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83d41586a0 msgr2=0x7f83d4158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83d41586a0 0x7f83d4158b40 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f83d4067490 tx=0x7f83cc00b040 comp rx=0 tx=0).stop 2023-10-06T05:02:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 shutdown_connections 2023-10-06T05:02:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f83c0066f30 0x7f83c00693e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83d41586a0 0x7f83d4158b40 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83bc004810 0x7f83d40ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 --2- 172.21.15.3:0/1822674353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d40c9500 0x7f83d416de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 >> 172.21.15.3:0/1822674353 conn(0x7f83d406e730 msgr2=0x7f83d40b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 shutdown_connections 2023-10-06T05:02:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:56.862+0000 7f83da447700 1 -- 172.21.15.3:0/1822674353 wait complete. 2023-10-06T05:02:56.876 INFO:tasks.cephfs_test_runner:When a client has a bug (which we will simulate) preventing it from releasing caps, ... ok 2023-10-06T05:02:56.876 INFO:tasks.cephfs_test_runner: 2023-10-06T05:02:56.876 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T05:02:56.876 INFO:tasks.cephfs_test_runner:Ran 9 tests in 606.749s 2023-10-06T05:02:56.876 INFO:tasks.cephfs_test_runner: 2023-10-06T05:02:56.877 INFO:tasks.cephfs_test_runner:OK 2023-10-06T05:02:56.877 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T05:02:56.887 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-06T05:02:56.928 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-06T05:02:56.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:56.929 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-10-06T05:02:56.946 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:02:56.946 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:02:56.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:56.946 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:02:57.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:57.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:02:57.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:57.101 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-10-06T05:02:57.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:57.197 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-10-06T05:02:57.213 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:02:57.213 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:02:57.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:57.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:02:57.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:57.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:02:57.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:02:57.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-10-06T05:02:57.482 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T05:02:57.492 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T05:02:57.492 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:02:57.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.611+0000 7f7219224700 1 Processor -- start 2023-10-06T05:02:57.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.611+0000 7f7219224700 1 -- start start 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7219224700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72141597a0 0x7f72140ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7219224700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140aac70 con 0x7f72141593d0 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7219224700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140afb80 con 0x7f72140af740 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7219224700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72140aa060 con 0x7f72141597a0 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7212d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f72141593d0 0x7f72140a9880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37800/0 (socket says 172.21.15.3:37800) 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7212d9d700 1 -- 172.21.15.3:0/2901843137 learned_addr learned my addr 172.21.15.3:0/2901843137 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f721259c700 1 --2- 172.21.15.3:0/2901843137 >> v2:172.21.15.3:3300/0 conn(0x7f72141597a0 0x7f72140ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7211d9b700 1 -- 172.21.15.3:0/2901843137 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025434664 0 0) 0x7f72140aac70 con 0x7f72141593d0 2023-10-06T05:02:57.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f7211d9b700 1 -- 172.21.15.3:0/2901843137 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71fc003570 con 0x7f72141593d0 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f721259c700 1 -- 172.21.15.3:0/2901843137 >> v1:172.21.15.153:6789/0 conn(0x7f72140af740 legacy=0x7f7214068010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f721259c700 1 -- 172.21.15.3:0/2901843137 >> v1:172.21.15.3:6789/0 conn(0x7f72141593d0 legacy=0x7f72140a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f721259c700 1 -- 172.21.15.3:0/2901843137 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140aa340 con 0x7f72141597a0 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.612+0000 7f721259c700 1 --2- 172.21.15.3:0/2901843137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140ac490 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f72080098e0 tx=0x7f72080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ca2cc4c95bd72e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7211d9b700 1 -- 172.21.15.3:0/2901843137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208011630 con 0x7f72141597a0 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7211d9b700 1 -- 172.21.15.3:0/2901843137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208011790 con 0x7f72141597a0 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7211d9b700 1 -- 172.21.15.3:0/2901843137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208011b30 con 0x7f72141597a0 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7219224700 1 -- 172.21.15.3:0/2901843137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 msgr2=0x7f72140ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7219224700 1 --2- 172.21.15.3:0/2901843137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140ac490 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f72080098e0 tx=0x7f72080091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7219224700 1 -- 172.21.15.3:0/2901843137 shutdown_connections 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7219224700 1 --2- 172.21.15.3:0/2901843137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140ac490 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7219224700 1 -- 172.21.15.3:0/2901843137 >> 172.21.15.3:0/2901843137 conn(0x7f721406e870 msgr2=0x7f721406bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7219224700 1 -- 172.21.15.3:0/2901843137 shutdown_connections 2023-10-06T05:02:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.613+0000 7f7219224700 1 -- 172.21.15.3:0/2901843137 wait complete. 2023-10-06T05:02:57.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.615+0000 7f7219224700 1 Processor -- start 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.615+0000 7f7219224700 1 -- start start 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72140af740 0x7f721416af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72141593d0 0x7f72140c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7214155ed0 con 0x7f72141597a0 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72141555a0 con 0x7f72141593d0 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7214155880 con 0x7f72140af740 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50468/0 (socket says 172.21.15.3:50468) 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 -- 172.21.15.3:0/1962491757 learned_addr learned my addr 172.21.15.3:0/1962491757 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7212d9d700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72140af740 0x7f721416af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721259c700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72141593d0 0x7f72140c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 -- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72140af740 msgr2=0x7f721416af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72140af740 0x7f721416af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 -- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72141593d0 msgr2=0x7f72140c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72141593d0 0x7f72140c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 -- 172.21.15.3:0/1962491757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7208009420 con 0x7f72141597a0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f721359e700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140cae60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c930 tx=0x7f720c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f71faffd700 1 -- 172.21.15.3:0/1962491757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c012630 con 0x7f72141597a0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f71faffd700 1 -- 172.21.15.3:0/1962491757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c0127d0 con 0x7f72141597a0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7214155b60 con 0x7f72141597a0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f71faffd700 1 -- 172.21.15.3:0/1962491757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c012b90 con 0x7f72141597a0 2023-10-06T05:02:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.616+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721416e280 con 0x7f72141597a0 2023-10-06T05:02:57.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.617+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721404c380 con 0x7f72141597a0 2023-10-06T05:02:57.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.618+0000 7f71faffd700 1 -- 172.21.15.3:0/1962491757 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f720c01a3b0 con 0x7f72141597a0 2023-10-06T05:02:57.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.618+0000 7f71faffd700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.618+0000 7f71faffd700 1 -- 172.21.15.3:0/1962491757 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f720c084010 con 0x7f72141597a0 2023-10-06T05:02:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.618+0000 7f7212d9d700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.619+0000 7f7212d9d700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7204007fd0 tx=0x7f7204009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:57.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.622+0000 7f71faffd700 1 -- 172.21.15.3:0/1962491757 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f720c054e80 con 0x7f72141597a0 2023-10-06T05:02:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.816+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72140aa620 con 0x7f72141597a0 2023-10-06T05:02:57.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.817+0000 7f71faffd700 1 -- 172.21.15.3:0/1962491757 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+13804 (secure 0 0 0) 0x7f720c054a10 con 0x7f72141597a0 2023-10-06T05:02:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:57.819 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":138,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:02:55.850158+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:6837/2550905871":"2023-10-07T05:02:52.604494+0000","192.168.0.1:0/3882866003":"2023-10-06T06:02:46.060249+0000","172.21.15.153:6835/2550905871":"2023-10-07T05:02:52.604494+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 msgr2=0x7f72000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7204007fd0 tx=0x7f7204009040 comp rx=0 tx=0).stop 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 msgr2=0x7f72140cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140cae60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c930 tx=0x7f720c0026e0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 shutdown_connections 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72141597a0 0x7f72140cae60 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72140af740 0x7f721416af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 --2- 172.21.15.3:0/1962491757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72141593d0 0x7f72140c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 >> 172.21.15.3:0/1962491757 conn(0x7f721406e870 msgr2=0x7f72140cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 shutdown_connections 2023-10-06T05:02:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.819+0000 7f7219224700 1 -- 172.21.15.3:0/1962491757 wait complete. 2023-10-06T05:02:57.833 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:02:57.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:02:57.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.952+0000 7fadb5aeb700 1 Processor -- start 2023-10-06T05:02:58.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.952+0000 7fadb5aeb700 1 -- start start 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadb5aeb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fadb006d720 0x7fadb015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadb5aeb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadb00c6ba0 0x7fadb015b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadb5aeb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006daf0 0x7fadb015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadb5aeb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00c7c20 con 0x7fadb00c6ba0 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadb5aeb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00c7010 con 0x7fadb006d720 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadb5aeb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00c72f0 con 0x7fadb006daf0 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadb00c6ba0 0x7fadb015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadb00c6ba0 0x7fadb015b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50478/0 (socket says 172.21.15.3:50478) 2023-10-06T05:02:58.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 -- 172.21.15.3:0/2689019875 learned_addr learned my addr 172.21.15.3:0/2689019875 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaf7fe700 1 --2- 172.21.15.3:0/2689019875 >> v2:172.21.15.153:3300/0 conn(0x7fadb006d720 0x7fadb015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaffff700 1 --2- 172.21.15.3:0/2689019875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006daf0 0x7fadb015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 -- 172.21.15.3:0/2689019875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006daf0 msgr2=0x7fadb015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 --2- 172.21.15.3:0/2689019875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006daf0 0x7fadb015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 -- 172.21.15.3:0/2689019875 >> v2:172.21.15.153:3300/0 conn(0x7fadb006d720 msgr2=0x7fadb015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 --2- 172.21.15.3:0/2689019875 >> v2:172.21.15.153:3300/0 conn(0x7fadb006d720 0x7fadb015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 -- 172.21.15.3:0/2689019875 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00c75d0 con 0x7fadb00c6ba0 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaf7fe700 1 --2- 172.21.15.3:0/2689019875 >> v2:172.21.15.153:3300/0 conn(0x7fadb006d720 0x7fadb015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.953+0000 7fadaeffd700 1 --2- 172.21.15.3:0/2689019875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb015b520 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fadb006b430 tx=0x7fada4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c526caeb571043a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadae7fc700 1 -- 172.21.15.3:0/2689019875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada400b860 con 0x7fadb00c6ba0 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadae7fc700 1 -- 172.21.15.3:0/2689019875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada400b9c0 con 0x7fadb00c6ba0 2023-10-06T05:02:58.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadae7fc700 1 -- 172.21.15.3:0/2689019875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada400bd60 con 0x7fadb00c6ba0 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2689019875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 msgr2=0x7fadb015b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2689019875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb015b520 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fadb006b430 tx=0x7fada4009250 comp rx=0 tx=0).stop 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2689019875 shutdown_connections 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2689019875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006daf0 0x7fadb015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2689019875 >> v2:172.21.15.153:3300/0 conn(0x7fadb006d720 0x7fadb015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2689019875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb015b520 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2689019875 >> 172.21.15.3:0/2689019875 conn(0x7fadb0067aa0 msgr2=0x7fadb00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2689019875 shutdown_connections 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.954+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2689019875 wait complete. 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.955+0000 7fadb5aeb700 1 Processor -- start 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadb5aeb700 1 -- start start 2023-10-06T05:02:58.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadb5aeb700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006d720 0x7fadb00c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadb5aeb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb006daf0 0x7fadb00b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadb5aeb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb00c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadb5aeb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00c4810 con 0x7fadb00c6ba0 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadb5aeb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00c3c00 con 0x7fadb006daf0 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadb5aeb700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00c3ee0 con 0x7fadb006d720 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadaffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb00c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadaffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb00c3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50480/0 (socket says 172.21.15.3:50480) 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.956+0000 7fadaffff700 1 -- 172.21.15.3:0/2769667635 learned_addr learned my addr 172.21.15.3:0/2769667635 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaeffd700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb006daf0 0x7fadb00b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaf7fe700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006d720 0x7fadb00c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaffff700 1 -- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006d720 msgr2=0x7fadb00c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaffff700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006d720 0x7fadb00c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaffff700 1 -- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb006daf0 msgr2=0x7fadb00b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaffff700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb006daf0 0x7fadb00b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaffff700 1 -- 172.21.15.3:0/2769667635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40094f0 con 0x7fadb00c6ba0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaf7fe700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006d720 0x7fadb00c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadaffff700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb00c3180 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fada8009fa0 tx=0x7fada8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fad977fe700 1 -- 172.21.15.3:0/2769667635 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800ed20 con 0x7fadb00c6ba0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fad977fe700 1 -- 172.21.15.3:0/2769667635 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada800ee80 con 0x7fadb00c6ba0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fad977fe700 1 -- 172.21.15.3:0/2769667635 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8014670 con 0x7fadb00c6ba0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb00c41c0 con 0x7fadb00c6ba0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.957+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb00c51e0 con 0x7fadb00c6ba0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.958+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb004c380 con 0x7fadb00c6ba0 2023-10-06T05:02:58.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.958+0000 7fad977fe700 1 -- 172.21.15.3:0/2769667635 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fada80149e0 con 0x7fadb00c6ba0 2023-10-06T05:02:58.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.959+0000 7fad977fe700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad9c066f30 0x7fad9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.959+0000 7fad977fe700 1 -- 172.21.15.3:0/2769667635 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fada8084b40 con 0x7fadb00c6ba0 2023-10-06T05:02:58.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.959+0000 7fadaf7fe700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad9c066f30 0x7fad9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.960+0000 7fadaf7fe700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad9c066f30 0x7fad9c0693e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fada0006f00 tx=0x7fada0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:57.963+0000 7fad977fe700 1 -- 172.21.15.3:0/2769667635 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fada8051f20 con 0x7fadb00c6ba0 2023-10-06T05:02:58.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.152+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadb00ccd80 con 0x7fad9c066f30 2023-10-06T05:02:58.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.155+0000 7fad977fe700 1 -- 172.21.15.3:0/2769667635 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73106 (secure 0 0 0) 0x7fadb00ccd80 con 0x7fad9c066f30 2023-10-06T05:02:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:58.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad9c066f30 msgr2=0x7fad9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad9c066f30 0x7fad9c0693e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fada0006f00 tx=0x7fada0008040 comp rx=0 tx=0).stop 2023-10-06T05:02:58.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 msgr2=0x7fadb00c3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb00c3180 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fada8009fa0 tx=0x7fada8009900 comp rx=0 tx=0).stop 2023-10-06T05:02:58.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 shutdown_connections 2023-10-06T05:02:58.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fad9c066f30 0x7fad9c0693e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadb00c6ba0 0x7fadb00c3180 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fadb006d720 0x7fadb00c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 --2- 172.21.15.3:0/2769667635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb006daf0 0x7fadb00b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 >> 172.21.15.3:0/2769667635 conn(0x7fadb0067aa0 msgr2=0x7fadb015a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:58.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 shutdown_connections 2023-10-06T05:02:58.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.157+0000 7fadb5aeb700 1 -- 172.21.15.3:0/2769667635 wait complete. 2023-10-06T05:02:58.160 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:02:58.171 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2023-10-06T05:02:57.582758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2450232,"kb_used_data":14584,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524488,"statfs":{"total":773094113280,"available":770585075712,"internally_reserved":0,"allocated":14934016,"data_stored":7458438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14074,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001651"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.787823+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:02:52.787823+0000","last_peered":"2023-10-06T05:02:52.787823+0000","last_clean":"2023-10-06T05:02:52.787823+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:02:52.787823+0000","last_undegraded":"2023-10-06T05:02:52.787823+0000","last_fullsized":"2023-10-06T05:02:52.787823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":119,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.609697+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:02:52.609697+0000","last_peered":"2023-10-06T05:02:52.609697+0000","last_clean":"2023-10-06T05:02:52.609697+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:02:52.609697+0000","last_undegraded":"2023-10-06T05:02:52.609697+0000","last_fullsized":"2023-10-06T05:02:52.609697+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:53.305863+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:02:53.305863+0000","last_peered":"2023-10-06T05:02:53.305863+0000","last_clean":"2023-10-06T05:02:53.305863+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:02:53.305863+0000","last_undegraded":"2023-10-06T05:02:53.305863+0000","last_fullsized":"2023-10-06T05:02:53.305863+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":259,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883854+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:02:52.883854+0000","last_peered":"2023-10-06T05:02:52.883854+0000","last_clean":"2023-10-06T05:02:52.883854+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:02:52.883854+0000","last_undegraded":"2023-10-06T05:02:52.883854+0000","last_fullsized":"2023-10-06T05:02:52.883854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883758+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:02:52.883758+0000","last_peered":"2023-10-06T05:02:52.883758+0000","last_clean":"2023-10-06T05:02:52.883758+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:02:52.883758+0000","last_undegraded":"2023-10-06T05:02:52.883758+0000","last_fullsized":"2023-10-06T05:02:52.883758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883926+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:02:52.883926+0000","last_peered":"2023-10-06T05:02:52.883926+0000","last_clean":"2023-10-06T05:02:52.883926+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:02:52.883926+0000","last_undegraded":"2023-10-06T05:02:52.883926+0000","last_fullsized":"2023-10-06T05:02:52.883926+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:53.306463+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:02:53.306463+0000","last_peered":"2023-10-06T05:02:53.306463+0000","last_clean":"2023-10-06T05:02:53.306463+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:02:53.306463+0000","last_undegraded":"2023-10-06T05:02:53.306463+0000","last_fullsized":"2023-10-06T05:02:53.306463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.899047+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:02:52.899047+0000","last_peered":"2023-10-06T05:02:52.899047+0000","last_clean":"2023-10-06T05:02:52.899047+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:02:52.899047+0000","last_undegraded":"2023-10-06T05:02:52.899047+0000","last_fullsized":"2023-10-06T05:02:52.899047+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.899173+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:02:52.899173+0000","last_peered":"2023-10-06T05:02:52.899173+0000","last_clean":"2023-10-06T05:02:52.899173+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:02:52.899173+0000","last_undegraded":"2023-10-06T05:02:52.899173+0000","last_fullsized":"2023-10-06T05:02:52.899173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307736,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064104,"statfs":{"total":96636764160,"available":96321642496,"internally_reserved":0,"allocated":2310144,"data_stored":1374835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":7.1779999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.41299999999999998,"15min":0.311},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":7.1879999999999997},"last":0.378}]},{"osd":1,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.748,"5min":0.78400000000000003,"15min":1.073},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.1379999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":1.1240000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":1.111},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":1.0880000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.38200000000000001},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":0.81499999999999995,"15min":0.874},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.86099999999999999,"15min":1.214},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1708032,"data_stored":779445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.28799999999999998,"15min":0.20699999999999999},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.76000000000000001,"5min":1.21,"15min":2.8140000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":1.093,"15min":4.3559999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.68899999999999995,"5min":0.83099999999999996,"15min":1.234},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":5.7080000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.45000000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.74099999999999999,"5min":1.3120000000000001,"15min":2.8650000000000002},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.238,"15min":4.2569999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":0.66800000000000004,"5min":1.175,"15min":4.3949999999999996},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.51200000000000001},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.65200000000000002,"5min":0.78200000000000003,"15min":3.669},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.748,"5min":1.147,"15min":2.9900000000000002},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":1.2509999999999999,"15min":4.4370000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.46700000000000003,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":0.85899999999999999,"15min":2.9510000000000001},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77600000000000002,"5min":1.274,"15min":2.9049999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72699999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307724,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064116,"statfs":{"total":96636764160,"available":96321654784,"internally_reserved":0,"allocated":2297856,"data_stored":1368460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.35599999999999998},"min":{"1min":0.20599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":4.1879999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.378},"min":{"1min":0.254,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.54100000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28899999999999998,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.69399999999999995},"last":4.4359999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.622,"15min":0.70399999999999996},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41599999999999998,"15min":0.42299999999999999},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.379},"max":{"1min":0.876,"5min":0.877,"15min":1.0780000000000001},"last":4.9400000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":1.117,"15min":1.117},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61699999999999999,"15min":0.63100000000000001},"min":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.998},"last":4.6520000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":1.0049999999999999},"last":4.2880000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":0.84299999999999997,"5min":1.167,"15min":1.167},"last":2.1829999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.98999999999999999},"last":4.8650000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":0.82699999999999996,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":2.2269999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.496,"15min":0.441},"max":{"1min":0.90700000000000003,"5min":0.96599999999999997,"15min":1.0429999999999999},"last":4.9950000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":793350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.376,"15min":0.36499999999999999},"min":{"1min":0.26100000000000001,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.67200000000000004},"last":0.25},{"interface":"front","average":{"1min":0.432,"5min":0.40200000000000002,"15min":0.375},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.371}]},{"osd":2,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.59199999999999997,"15min":0.628},"last":0.312},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.39700000000000002},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":0.67500000000000004,"15min":0.77600000000000002},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.33700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.68899999999999995},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.36199999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63900000000000001,"5min":0.71999999999999997,"15min":0.86299999999999999},"last":0.34399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.749,"5min":0.94799999999999995,"15min":0.98299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76600000000000001,"5min":0.89000000000000001,"15min":0.92300000000000004},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.73699999999999999,"5min":0.90200000000000002,"15min":0.91700000000000004},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":0.93700000000000006,"15min":0.97299999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":0.96199999999999997,"15min":1.0109999999999999},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67400000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.0329999999999999},"last":0.439},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55900000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":0.97899999999999998,"15min":0.995},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303064,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068776,"statfs":{"total":96636764160,"available":96326426624,"internally_reserved":0,"allocated":1720320,"data_stored":785820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.56100000000000005,"5min":0.64100000000000001,"15min":1.048},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.249},"max":{"1min":0.58099999999999996,"5min":0.62,"15min":0.69699999999999995},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39900000000000002,"15min":0.39400000000000002},"min":{"1min":0.29899999999999999,"5min":0.253,"15min":0.216},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":1.1819999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.219},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.19900000000000001},"max":{"1min":0.47899999999999998,"5min":0.55200000000000005,"15min":0.57399999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.44400000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.53900000000000003,"5min":0.67400000000000004,"15min":1.2070000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.88300000000000001,"15min":1.246},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.90200000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.2649999999999999},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.93100000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.32600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.625},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.90600000000000003,"15min":0.92200000000000004},"last":0.67900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307164,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064676,"statfs":{"total":96636764160,"available":96322228224,"internally_reserved":0,"allocated":1724416,"data_stored":785827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.39600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.502,"5min":1.5589999999999999,"15min":2.008},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.58899999999999997,"5min":0.73399999999999999,"15min":6.4370000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39500000000000002,"15min":0.372},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.379},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.63100000000000001,"5min":1.601,"15min":2.1970000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41499999999999998,"15min":0.496},"min":{"1min":0.317,"5min":0.30199999999999999,"15min":0.255},"max":{"1min":0.67000000000000004,"5min":0.88400000000000001,"15min":5.069},"last":0.441},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":2.246},"last":0.315}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":0.97899999999999998,"5min":1.6719999999999999,"15min":14.646000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90000000000000002,"5min":3.109,"15min":14.606999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":1.506,"15min":2.2959999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.72199999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":4.3150000000000004,"15min":6.4009999999999998},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.80600000000000005,"15min":0.76900000000000002},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":16.773,"15min":16.773},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77200000000000002,"15min":0.70899999999999996},"min":{"1min":0.45400000000000001,"5min":0.40600000000000003,"15min":0.40100000000000002},"max":{"1min":0.91200000000000003,"5min":11.436,"15min":11.436},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.72999999999999998},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.579,"15min":6.367},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":3.069,"15min":5},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307172,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064668,"statfs":{"total":96636764160,"available":96322220032,"internally_reserved":0,"allocated":1732608,"data_stored":792679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.95799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.5510000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.47399999999999998,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":5.4900000000000002},"last":0.505}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":4.0739999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.626},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":5.415},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58099999999999996,"15min":0.63200000000000001},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":4.202},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":4.2130000000000001},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1279999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.53500000000000003},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.83299999999999996},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.72499999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76300000000000001,"15min":0.99099999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.56799999999999995},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.872},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":1.1000000000000001},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307140,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064700,"statfs":{"total":96636764160,"available":96322252800,"internally_reserved":0,"allocated":1699840,"data_stored":778022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":4.9039999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.752,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.873,"15min":6.7949999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.625},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.68100000000000005},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.79500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.30199999999999999},"max":{"1min":0.79800000000000004,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.79600000000000004},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.54100000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.74399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.62},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":0.873},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:02:58.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:02:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.290+0000 7fa57a05b700 1 Processor -- start 2023-10-06T05:02:58.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.290+0000 7fa57a05b700 1 -- start start 2023-10-06T05:02:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa57a05b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5740b28f0 0x7fa5740cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa57a05b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740cd780 con 0x7fa5740cd340 2023-10-06T05:02:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa57a05b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740cc970 con 0x7fa5740b2520 2023-10-06T05:02:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa57a05b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740cc3f0 con 0x7fa5740b28f0 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa578858700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5740b28f0 0x7fa5740cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa57985a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa5740cd340 0x7fa5740cb0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37802/0 (socket says 172.21.15.3:37802) 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa57985a700 1 -- 172.21.15.3:0/536684080 learned_addr learned my addr 172.21.15.3:0/536684080 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa578858700 1 -- 172.21.15.3:0/536684080 >> v1:172.21.15.153:6789/0 conn(0x7fa5740b2520 legacy=0x7fa5740b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa578858700 1 -- 172.21.15.3:0/536684080 >> v1:172.21.15.3:6789/0 conn(0x7fa5740cd340 legacy=0x7fa5740cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa573fff700 1 -- 172.21.15.3:0/536684080 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740663467 0 0) 0x7fa564001490 con 0x7fa5740cd340 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.291+0000 7fa578858700 1 -- 172.21.15.3:0/536684080 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740cbd40 con 0x7fa5740b28f0 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa578858700 1 --2- 172.21.15.3:0/536684080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa5740cce00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa5680098e0 tx=0x7fa5680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a741a5d0f15028 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa573fff700 1 -- 172.21.15.3:0/536684080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568009e80 con 0x7fa5740b28f0 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa573fff700 1 -- 172.21.15.3:0/536684080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56800b860 con 0x7fa5740b28f0 2023-10-06T05:02:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa573fff700 1 -- 172.21.15.3:0/536684080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800bbe0 con 0x7fa5740b28f0 2023-10-06T05:02:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa57a05b700 1 -- 172.21.15.3:0/536684080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 msgr2=0x7fa5740cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa57a05b700 1 --2- 172.21.15.3:0/536684080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa5740cce00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa5680098e0 tx=0x7fa5680091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa57a05b700 1 -- 172.21.15.3:0/536684080 shutdown_connections 2023-10-06T05:02:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa57a05b700 1 --2- 172.21.15.3:0/536684080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa5740cce00 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa57a05b700 1 -- 172.21.15.3:0/536684080 >> 172.21.15.3:0/536684080 conn(0x7fa57406e730 msgr2=0x7fa57406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa57a05b700 1 -- 172.21.15.3:0/536684080 shutdown_connections 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.292+0000 7fa57a05b700 1 -- 172.21.15.3:0/536684080 wait complete. 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.293+0000 7fa57a05b700 1 Processor -- start 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.293+0000 7fa57a05b700 1 -- start start 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa57a05b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5740b2520 0x7fa5740d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa57a05b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa574151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa57a05b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5740cd340 0x7fa57416a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa57a05b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574169ce0 con 0x7fa5740b28f0 2023-10-06T05:02:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa57a05b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57414cab0 con 0x7fa5740cd340 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa57a05b700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57414f530 con 0x7fa5740b2520 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa578858700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa574151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa578858700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa574151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50502/0 (socket says 172.21.15.3:50502) 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.294+0000 7fa578858700 1 -- 172.21.15.3:0/3070342555 learned_addr learned my addr 172.21.15.3:0/3070342555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa57985a700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5740cd340 0x7fa57416a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa579059700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5740b2520 0x7fa5740d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa578858700 1 -- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5740b2520 msgr2=0x7fa5740d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa578858700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5740b2520 0x7fa5740d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa578858700 1 -- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5740cd340 msgr2=0x7fa57416a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa578858700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5740cd340 0x7fa57416a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa578858700 1 -- 172.21.15.3:0/3070342555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568009420 con 0x7fa5740b28f0 2023-10-06T05:02:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa57985a700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5740cd340 0x7fa57416a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:02:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa578858700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa574151140 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fa574089580 tx=0x7fa568017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa5717fa700 1 -- 172.21.15.3:0/3070342555 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56801f070 con 0x7fa5740b28f0 2023-10-06T05:02:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa5717fa700 1 -- 172.21.15.3:0/3070342555 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa568011420 con 0x7fa5740b28f0 2023-10-06T05:02:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57414f810 con 0x7fa5740b28f0 2023-10-06T05:02:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa5717fa700 1 -- 172.21.15.3:0/3070342555 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800bbe0 con 0x7fa5740b28f0 2023-10-06T05:02:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.295+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5740d0920 con 0x7fa5740b28f0 2023-10-06T05:02:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.296+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57404c380 con 0x7fa5740b28f0 2023-10-06T05:02:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.296+0000 7fa5717fa700 1 -- 172.21.15.3:0/3070342555 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa568011a50 con 0x7fa5740b28f0 2023-10-06T05:02:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.297+0000 7fa5717fa700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa560066f30 0x7fa5600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.297+0000 7fa5717fa700 1 -- 172.21.15.3:0/3070342555 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa568085a90 con 0x7fa5740b28f0 2023-10-06T05:02:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.297+0000 7fa579059700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa560066f30 0x7fa5600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.297+0000 7fa579059700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa560066f30 0x7fa5600693e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa56c003860 tx=0x7fa56c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.301+0000 7fa5717fa700 1 -- 172.21.15.3:0/3070342555 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa568052dd0 con 0x7fa5740b28f0 2023-10-06T05:02:58.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.490+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5740cc020 con 0x7fa560066f30 2023-10-06T05:02:58.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.494+0000 7fa5717fa700 1 -- 172.21.15.3:0/3070342555 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73106 (secure 0 0 0) 0x7fa5740cc020 con 0x7fa560066f30 2023-10-06T05:02:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:58.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa560066f30 msgr2=0x7fa5600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa560066f30 0x7fa5600693e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa56c003860 tx=0x7fa56c009040 comp rx=0 tx=0).stop 2023-10-06T05:02:58.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 msgr2=0x7fa574151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa574151140 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fa574089580 tx=0x7fa568017000 comp rx=0 tx=0).stop 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 shutdown_connections 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fa560066f30 0x7fa5600693e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5740cd340 0x7fa57416a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa5740b2520 0x7fa5740d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 --2- 172.21.15.3:0/3070342555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5740b28f0 0x7fa574151140 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 >> 172.21.15.3:0/3070342555 conn(0x7fa57406e730 msgr2=0x7fa5741531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 shutdown_connections 2023-10-06T05:02:58.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.496+0000 7fa57a05b700 1 -- 172.21.15.3:0/3070342555 wait complete. 2023-10-06T05:02:58.499 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:02:58.509 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2023-10-06T05:02:57.582758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2450232,"kb_used_data":14584,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524488,"statfs":{"total":773094113280,"available":770585075712,"internally_reserved":0,"allocated":14934016,"data_stored":7458438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14074,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001651"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.787823+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:02:52.787823+0000","last_peered":"2023-10-06T05:02:52.787823+0000","last_clean":"2023-10-06T05:02:52.787823+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:02:52.787823+0000","last_undegraded":"2023-10-06T05:02:52.787823+0000","last_fullsized":"2023-10-06T05:02:52.787823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":119,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.609697+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:02:52.609697+0000","last_peered":"2023-10-06T05:02:52.609697+0000","last_clean":"2023-10-06T05:02:52.609697+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:02:52.609697+0000","last_undegraded":"2023-10-06T05:02:52.609697+0000","last_fullsized":"2023-10-06T05:02:52.609697+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:53.305863+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:02:53.305863+0000","last_peered":"2023-10-06T05:02:53.305863+0000","last_clean":"2023-10-06T05:02:53.305863+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:02:53.305863+0000","last_undegraded":"2023-10-06T05:02:53.305863+0000","last_fullsized":"2023-10-06T05:02:53.305863+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":259,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883854+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:02:52.883854+0000","last_peered":"2023-10-06T05:02:52.883854+0000","last_clean":"2023-10-06T05:02:52.883854+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:02:52.883854+0000","last_undegraded":"2023-10-06T05:02:52.883854+0000","last_fullsized":"2023-10-06T05:02:52.883854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883758+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:02:52.883758+0000","last_peered":"2023-10-06T05:02:52.883758+0000","last_clean":"2023-10-06T05:02:52.883758+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:02:52.883758+0000","last_undegraded":"2023-10-06T05:02:52.883758+0000","last_fullsized":"2023-10-06T05:02:52.883758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883926+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:02:52.883926+0000","last_peered":"2023-10-06T05:02:52.883926+0000","last_clean":"2023-10-06T05:02:52.883926+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:02:52.883926+0000","last_undegraded":"2023-10-06T05:02:52.883926+0000","last_fullsized":"2023-10-06T05:02:52.883926+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:53.306463+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:02:53.306463+0000","last_peered":"2023-10-06T05:02:53.306463+0000","last_clean":"2023-10-06T05:02:53.306463+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:02:53.306463+0000","last_undegraded":"2023-10-06T05:02:53.306463+0000","last_fullsized":"2023-10-06T05:02:53.306463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.899047+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:02:52.899047+0000","last_peered":"2023-10-06T05:02:52.899047+0000","last_clean":"2023-10-06T05:02:52.899047+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:02:52.899047+0000","last_undegraded":"2023-10-06T05:02:52.899047+0000","last_fullsized":"2023-10-06T05:02:52.899047+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.899173+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:02:52.899173+0000","last_peered":"2023-10-06T05:02:52.899173+0000","last_clean":"2023-10-06T05:02:52.899173+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:02:52.899173+0000","last_undegraded":"2023-10-06T05:02:52.899173+0000","last_fullsized":"2023-10-06T05:02:52.899173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307736,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064104,"statfs":{"total":96636764160,"available":96321642496,"internally_reserved":0,"allocated":2310144,"data_stored":1374835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":7.1779999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.41299999999999998,"15min":0.311},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":7.1879999999999997},"last":0.378}]},{"osd":1,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.748,"5min":0.78400000000000003,"15min":1.073},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.1379999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":1.1240000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":1.111},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":1.0880000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.38200000000000001},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":0.81499999999999995,"15min":0.874},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.86099999999999999,"15min":1.214},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1708032,"data_stored":779445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.28799999999999998,"15min":0.20699999999999999},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.76000000000000001,"5min":1.21,"15min":2.8140000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":1.093,"15min":4.3559999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.68899999999999995,"5min":0.83099999999999996,"15min":1.234},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":5.7080000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.45000000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.74099999999999999,"5min":1.3120000000000001,"15min":2.8650000000000002},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.238,"15min":4.2569999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":0.66800000000000004,"5min":1.175,"15min":4.3949999999999996},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.51200000000000001},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.65200000000000002,"5min":0.78200000000000003,"15min":3.669},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.748,"5min":1.147,"15min":2.9900000000000002},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":1.2509999999999999,"15min":4.4370000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.46700000000000003,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":0.85899999999999999,"15min":2.9510000000000001},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77600000000000002,"5min":1.274,"15min":2.9049999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72699999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307724,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064116,"statfs":{"total":96636764160,"available":96321654784,"internally_reserved":0,"allocated":2297856,"data_stored":1368460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.35599999999999998},"min":{"1min":0.20599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":4.1879999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.378},"min":{"1min":0.254,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.54100000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28899999999999998,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.69399999999999995},"last":4.4359999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.622,"15min":0.70399999999999996},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41599999999999998,"15min":0.42299999999999999},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.379},"max":{"1min":0.876,"5min":0.877,"15min":1.0780000000000001},"last":4.9400000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":1.117,"15min":1.117},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61699999999999999,"15min":0.63100000000000001},"min":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.998},"last":4.6520000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":1.0049999999999999},"last":4.2880000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":0.84299999999999997,"5min":1.167,"15min":1.167},"last":2.1829999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.98999999999999999},"last":4.8650000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":0.82699999999999996,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":2.2269999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.496,"15min":0.441},"max":{"1min":0.90700000000000003,"5min":0.96599999999999997,"15min":1.0429999999999999},"last":4.9950000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":793350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.376,"15min":0.36499999999999999},"min":{"1min":0.26100000000000001,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.67200000000000004},"last":0.25},{"interface":"front","average":{"1min":0.432,"5min":0.40200000000000002,"15min":0.375},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.371}]},{"osd":2,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.59199999999999997,"15min":0.628},"last":0.312},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.39700000000000002},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":0.67500000000000004,"15min":0.77600000000000002},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.33700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.68899999999999995},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.36199999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63900000000000001,"5min":0.71999999999999997,"15min":0.86299999999999999},"last":0.34399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.749,"5min":0.94799999999999995,"15min":0.98299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76600000000000001,"5min":0.89000000000000001,"15min":0.92300000000000004},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.73699999999999999,"5min":0.90200000000000002,"15min":0.91700000000000004},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":0.93700000000000006,"15min":0.97299999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":0.96199999999999997,"15min":1.0109999999999999},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67400000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.0329999999999999},"last":0.439},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55900000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":0.97899999999999998,"15min":0.995},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303064,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068776,"statfs":{"total":96636764160,"available":96326426624,"internally_reserved":0,"allocated":1720320,"data_stored":785820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.56100000000000005,"5min":0.64100000000000001,"15min":1.048},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.249},"max":{"1min":0.58099999999999996,"5min":0.62,"15min":0.69699999999999995},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39900000000000002,"15min":0.39400000000000002},"min":{"1min":0.29899999999999999,"5min":0.253,"15min":0.216},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":1.1819999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.219},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.19900000000000001},"max":{"1min":0.47899999999999998,"5min":0.55200000000000005,"15min":0.57399999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.44400000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.53900000000000003,"5min":0.67400000000000004,"15min":1.2070000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.88300000000000001,"15min":1.246},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.90200000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.2649999999999999},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.93100000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.32600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.625},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.90600000000000003,"15min":0.92200000000000004},"last":0.67900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307164,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064676,"statfs":{"total":96636764160,"available":96322228224,"internally_reserved":0,"allocated":1724416,"data_stored":785827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.39600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.502,"5min":1.5589999999999999,"15min":2.008},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.58899999999999997,"5min":0.73399999999999999,"15min":6.4370000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39500000000000002,"15min":0.372},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.379},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.63100000000000001,"5min":1.601,"15min":2.1970000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41499999999999998,"15min":0.496},"min":{"1min":0.317,"5min":0.30199999999999999,"15min":0.255},"max":{"1min":0.67000000000000004,"5min":0.88400000000000001,"15min":5.069},"last":0.441},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":2.246},"last":0.315}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":0.97899999999999998,"5min":1.6719999999999999,"15min":14.646000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90000000000000002,"5min":3.109,"15min":14.606999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":1.506,"15min":2.2959999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.72199999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":4.3150000000000004,"15min":6.4009999999999998},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.80600000000000005,"15min":0.76900000000000002},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":16.773,"15min":16.773},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77200000000000002,"15min":0.70899999999999996},"min":{"1min":0.45400000000000001,"5min":0.40600000000000003,"15min":0.40100000000000002},"max":{"1min":0.91200000000000003,"5min":11.436,"15min":11.436},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.72999999999999998},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.579,"15min":6.367},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":3.069,"15min":5},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307172,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064668,"statfs":{"total":96636764160,"available":96322220032,"internally_reserved":0,"allocated":1732608,"data_stored":792679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.95799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.5510000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.47399999999999998,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":5.4900000000000002},"last":0.505}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":4.0739999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.626},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":5.415},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58099999999999996,"15min":0.63200000000000001},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":4.202},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":4.2130000000000001},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1279999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.53500000000000003},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.83299999999999996},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.72499999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76300000000000001,"15min":0.99099999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.56799999999999995},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.872},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":1.1000000000000001},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307140,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064700,"statfs":{"total":96636764160,"available":96322252800,"internally_reserved":0,"allocated":1699840,"data_stored":778022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":4.9039999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.752,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.873,"15min":6.7949999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.625},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.68100000000000005},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.79500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.30199999999999999},"max":{"1min":0.79800000000000004,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.79600000000000004},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.54100000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.74399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.62},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":0.873},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:02:58.512 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:02:58.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:02:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.630+0000 7fb90f4f0700 1 Processor -- start 2023-10-06T05:02:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.630+0000 7fb90f4f0700 1 -- start start 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90f4f0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb908067250 0x7fb9080b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90f4f0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb9080b0eb0 0x7fb908158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90f4f0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90806ea40 con 0x7fb908067620 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90f4f0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9080af870 con 0x7fb908067250 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90f4f0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90806ed20 con 0x7fb9080b0eb0 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb908067250 0x7fb9080b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb908067250 0x7fb9080b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50514/0 (socket says 172.21.15.3:50514) 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 -- 172.21.15.3:0/1940697403 learned_addr learned my addr 172.21.15.3:0/1940697403 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 -- 172.21.15.3:0/1940697403 >> v2:172.21.15.153:3300/0 conn(0x7fb9080b0eb0 msgr2=0x7fb908158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 --2- 172.21.15.3:0/1940697403 >> v2:172.21.15.153:3300/0 conn(0x7fb9080b0eb0 0x7fb908158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 -- 172.21.15.3:0/1940697403 >> v1:172.21.15.153:6789/0 conn(0x7fb908067620 legacy=0x7fb908159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 -- 172.21.15.3:0/1940697403 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90806f000 con 0x7fb908067250 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb90d28c700 1 --2- 172.21.15.3:0/1940697403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067250 0x7fb9080b0970 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb9080b1da0 tx=0x7fb8f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19517f1b859a2c12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb907fff700 1 -- 172.21.15.3:0/1940697403 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800b820 con 0x7fb908067250 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.631+0000 7fb907fff700 1 -- 172.21.15.3:0/1940697403 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800b980 con 0x7fb908067250 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb907fff700 1 -- 172.21.15.3:0/1940697403 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800bd20 con 0x7fb908067250 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 -- 172.21.15.3:0/1940697403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067250 msgr2=0x7fb9080b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/1940697403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067250 0x7fb9080b0970 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb9080b1da0 tx=0x7fb8f8009250 comp rx=0 tx=0).stop 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 -- 172.21.15.3:0/1940697403 shutdown_connections 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/1940697403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067250 0x7fb9080b0970 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/1940697403 >> v2:172.21.15.153:3300/0 conn(0x7fb9080b0eb0 0x7fb908158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 -- 172.21.15.3:0/1940697403 >> 172.21.15.3:0/1940697403 conn(0x7fb90806b230 msgr2=0x7fb9080aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 -- 172.21.15.3:0/1940697403 shutdown_connections 2023-10-06T05:02:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.632+0000 7fb90f4f0700 1 -- 172.21.15.3:0/1940697403 wait complete. 2023-10-06T05:02:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.633+0000 7fb90f4f0700 1 Processor -- start 2023-10-06T05:02:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.633+0000 7fb90f4f0700 1 -- start start 2023-10-06T05:02:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90f4f0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb908067250 0x7fb90813f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90f4f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067620 0x7fb90815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90f4f0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9080b0eb0 0x7fb9081572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90f4f0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908155920 con 0x7fb908067620 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90f4f0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908154d10 con 0x7fb9080b0eb0 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90f4f0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb908154ff0 con 0x7fb908067250 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90ca8b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067620 0x7fb90815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90ca8b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067620 0x7fb90815e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50516/0 (socket says 172.21.15.3:50516) 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90ca8b700 1 -- 172.21.15.3:0/2598591105 learned_addr learned my addr 172.21.15.3:0/2598591105 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90da8d700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9080b0eb0 0x7fb9081572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.634+0000 7fb90d28c700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb908067250 0x7fb90813f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90ca8b700 1 -- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb908067250 msgr2=0x7fb90813f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90ca8b700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb908067250 0x7fb90813f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90ca8b700 1 -- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9080b0eb0 msgr2=0x7fb9081572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90ca8b700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9080b0eb0 0x7fb9081572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90ca8b700 1 -- 172.21.15.3:0/2598591105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80094f0 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90ca8b700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067620 0x7fb90815e730 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00b5e0 tx=0x7fb8fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb9057fa700 1 -- 172.21.15.3:0/2598591105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00f510 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb9057fa700 1 -- 172.21.15.3:0/2598591105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc00f670 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb9057fa700 1 -- 172.21.15.3:0/2598591105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00fa30 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9081552d0 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.635+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90813fb30 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.636+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb908155a90 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.636+0000 7fb9057fa700 1 -- 172.21.15.3:0/2598591105 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8fc01a400 con 0x7fb908067620 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.637+0000 7fb9057fa700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8f4066f30 0x7fb8f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.637+0000 7fb9057fa700 1 -- 172.21.15.3:0/2598591105 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb8fc083d70 con 0x7fb908067620 2023-10-06T05:02:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.637+0000 7fb90d28c700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8f4066f30 0x7fb8f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.637+0000 7fb90d28c700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8f4066f30 0x7fb8f40693e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb9080b1da0 tx=0x7fb8f8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.641+0000 7fb9057fa700 1 -- 172.21.15.3:0/2598591105 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8fc0511d0 con 0x7fb908067620 2023-10-06T05:02:58.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.830+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb90804c380 con 0x7fb8f4066f30 2023-10-06T05:02:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.834+0000 7fb9057fa700 1 -- 172.21.15.3:0/2598591105 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73106 (secure 0 0 0) 0x7fb90804c380 con 0x7fb8f4066f30 2023-10-06T05:02:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8f4066f30 msgr2=0x7fb8f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8f4066f30 0x7fb8f40693e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb9080b1da0 tx=0x7fb8f8016000 comp rx=0 tx=0).stop 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067620 msgr2=0x7fb90815e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067620 0x7fb90815e730 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00b5e0 tx=0x7fb8fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 shutdown_connections 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb8f4066f30 0x7fb8f40693e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9080b0eb0 0x7fb9081572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb908067250 0x7fb90813f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 --2- 172.21.15.3:0/2598591105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb908067620 0x7fb90815e730 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 >> 172.21.15.3:0/2598591105 conn(0x7fb90806b230 msgr2=0x7fb9080ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:58.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 shutdown_connections 2023-10-06T05:02:58.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.836+0000 7fb90f4f0700 1 -- 172.21.15.3:0/2598591105 wait complete. 2023-10-06T05:02:58.839 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:02:58.849 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2023-10-06T05:02:57.582758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2450232,"kb_used_data":14584,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524488,"statfs":{"total":773094113280,"available":770585075712,"internally_reserved":0,"allocated":14934016,"data_stored":7458438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14074,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001651"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.787823+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:02:52.787823+0000","last_peered":"2023-10-06T05:02:52.787823+0000","last_clean":"2023-10-06T05:02:52.787823+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:02:52.787823+0000","last_undegraded":"2023-10-06T05:02:52.787823+0000","last_fullsized":"2023-10-06T05:02:52.787823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":119,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.609697+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:02:52.609697+0000","last_peered":"2023-10-06T05:02:52.609697+0000","last_clean":"2023-10-06T05:02:52.609697+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:02:52.609697+0000","last_undegraded":"2023-10-06T05:02:52.609697+0000","last_fullsized":"2023-10-06T05:02:52.609697+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:53.305863+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:02:53.305863+0000","last_peered":"2023-10-06T05:02:53.305863+0000","last_clean":"2023-10-06T05:02:53.305863+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:02:53.305863+0000","last_undegraded":"2023-10-06T05:02:53.305863+0000","last_fullsized":"2023-10-06T05:02:53.305863+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":259,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883854+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:02:52.883854+0000","last_peered":"2023-10-06T05:02:52.883854+0000","last_clean":"2023-10-06T05:02:52.883854+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:02:52.883854+0000","last_undegraded":"2023-10-06T05:02:52.883854+0000","last_fullsized":"2023-10-06T05:02:52.883854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883758+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:02:52.883758+0000","last_peered":"2023-10-06T05:02:52.883758+0000","last_clean":"2023-10-06T05:02:52.883758+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:02:52.883758+0000","last_undegraded":"2023-10-06T05:02:52.883758+0000","last_fullsized":"2023-10-06T05:02:52.883758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.883926+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:02:52.883926+0000","last_peered":"2023-10-06T05:02:52.883926+0000","last_clean":"2023-10-06T05:02:52.883926+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:02:52.883926+0000","last_undegraded":"2023-10-06T05:02:52.883926+0000","last_fullsized":"2023-10-06T05:02:52.883926+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:53.306463+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:02:53.306463+0000","last_peered":"2023-10-06T05:02:53.306463+0000","last_clean":"2023-10-06T05:02:53.306463+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:02:53.306463+0000","last_undegraded":"2023-10-06T05:02:53.306463+0000","last_fullsized":"2023-10-06T05:02:53.306463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.899047+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:02:52.899047+0000","last_peered":"2023-10-06T05:02:52.899047+0000","last_clean":"2023-10-06T05:02:52.899047+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:02:52.899047+0000","last_undegraded":"2023-10-06T05:02:52.899047+0000","last_fullsized":"2023-10-06T05:02:52.899047+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T05:02:52.899173+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:02:52.899173+0000","last_peered":"2023-10-06T05:02:52.899173+0000","last_clean":"2023-10-06T05:02:52.899173+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:02:52.899173+0000","last_undegraded":"2023-10-06T05:02:52.899173+0000","last_fullsized":"2023-10-06T05:02:52.899173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307736,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064104,"statfs":{"total":96636764160,"available":96321642496,"internally_reserved":0,"allocated":2310144,"data_stored":1374835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":7.1779999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.41299999999999998,"15min":0.311},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":7.1879999999999997},"last":0.378}]},{"osd":1,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.748,"5min":0.78400000000000003,"15min":1.073},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.1379999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":1.1240000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":1.111},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":1.0880000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.38200000000000001},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":0.81499999999999995,"15min":0.874},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.86099999999999999,"15min":1.214},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1708032,"data_stored":779445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.28799999999999998,"15min":0.20699999999999999},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.76000000000000001,"5min":1.21,"15min":2.8140000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":1.093,"15min":4.3559999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.68899999999999995,"5min":0.83099999999999996,"15min":1.234},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":5.7080000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.45000000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.74099999999999999,"5min":1.3120000000000001,"15min":2.8650000000000002},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.238,"15min":4.2569999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":0.66800000000000004,"5min":1.175,"15min":4.3949999999999996},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.51200000000000001},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.65200000000000002,"5min":0.78200000000000003,"15min":3.669},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.748,"5min":1.147,"15min":2.9900000000000002},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":1.2509999999999999,"15min":4.4370000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.46700000000000003,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":0.85899999999999999,"15min":2.9510000000000001},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77600000000000002,"5min":1.274,"15min":2.9049999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72699999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307724,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064116,"statfs":{"total":96636764160,"available":96321654784,"internally_reserved":0,"allocated":2297856,"data_stored":1368460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.35599999999999998},"min":{"1min":0.20599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":4.1879999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.378},"min":{"1min":0.254,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.54100000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28899999999999998,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.69399999999999995},"last":4.4359999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.622,"15min":0.70399999999999996},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41599999999999998,"15min":0.42299999999999999},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.379},"max":{"1min":0.876,"5min":0.877,"15min":1.0780000000000001},"last":4.9400000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":1.117,"15min":1.117},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61699999999999999,"15min":0.63100000000000001},"min":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.998},"last":4.6520000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":1.0049999999999999},"last":4.2880000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":0.84299999999999997,"5min":1.167,"15min":1.167},"last":2.1829999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.98999999999999999},"last":4.8650000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":0.82699999999999996,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":2.2269999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.496,"15min":0.441},"max":{"1min":0.90700000000000003,"5min":0.96599999999999997,"15min":1.0429999999999999},"last":4.9950000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1740800,"data_stored":793350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.376,"15min":0.36499999999999999},"min":{"1min":0.26100000000000001,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.67200000000000004},"last":0.25},{"interface":"front","average":{"1min":0.432,"5min":0.40200000000000002,"15min":0.375},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.371}]},{"osd":2,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.59199999999999997,"15min":0.628},"last":0.312},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.39700000000000002},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":0.67500000000000004,"15min":0.77600000000000002},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.33700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.68899999999999995},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.36199999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63900000000000001,"5min":0.71999999999999997,"15min":0.86299999999999999},"last":0.34399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.749,"5min":0.94799999999999995,"15min":0.98299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76600000000000001,"5min":0.89000000000000001,"15min":0.92300000000000004},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.73699999999999999,"5min":0.90200000000000002,"15min":0.91700000000000004},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":0.93700000000000006,"15min":0.97299999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":0.96199999999999997,"15min":1.0109999999999999},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67400000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.0329999999999999},"last":0.439},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55900000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":0.97899999999999998,"15min":0.995},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303064,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068776,"statfs":{"total":96636764160,"available":96326426624,"internally_reserved":0,"allocated":1720320,"data_stored":785820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.56100000000000005,"5min":0.64100000000000001,"15min":1.048},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.249},"max":{"1min":0.58099999999999996,"5min":0.62,"15min":0.69699999999999995},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39900000000000002,"15min":0.39400000000000002},"min":{"1min":0.29899999999999999,"5min":0.253,"15min":0.216},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":1.1819999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.219},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.19900000000000001},"max":{"1min":0.47899999999999998,"5min":0.55200000000000005,"15min":0.57399999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.44400000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.53900000000000003,"5min":0.67400000000000004,"15min":1.2070000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.88300000000000001,"15min":1.246},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.90200000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.2649999999999999},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.93100000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.32600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.625},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.90600000000000003,"15min":0.92200000000000004},"last":0.67900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307164,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064676,"statfs":{"total":96636764160,"available":96322228224,"internally_reserved":0,"allocated":1724416,"data_stored":785827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.39600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.502,"5min":1.5589999999999999,"15min":2.008},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.58899999999999997,"5min":0.73399999999999999,"15min":6.4370000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39500000000000002,"15min":0.372},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.379},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.63100000000000001,"5min":1.601,"15min":2.1970000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41499999999999998,"15min":0.496},"min":{"1min":0.317,"5min":0.30199999999999999,"15min":0.255},"max":{"1min":0.67000000000000004,"5min":0.88400000000000001,"15min":5.069},"last":0.441},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":2.246},"last":0.315}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":0.97899999999999998,"5min":1.6719999999999999,"15min":14.646000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90000000000000002,"5min":3.109,"15min":14.606999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":1.506,"15min":2.2959999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.72199999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":4.3150000000000004,"15min":6.4009999999999998},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.80600000000000005,"15min":0.76900000000000002},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":16.773,"15min":16.773},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77200000000000002,"15min":0.70899999999999996},"min":{"1min":0.45400000000000001,"5min":0.40600000000000003,"15min":0.40100000000000002},"max":{"1min":0.91200000000000003,"5min":11.436,"15min":11.436},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.72999999999999998},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.579,"15min":6.367},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":3.069,"15min":5},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307172,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064668,"statfs":{"total":96636764160,"available":96322220032,"internally_reserved":0,"allocated":1732608,"data_stored":792679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.95799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.5510000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.47399999999999998,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":5.4900000000000002},"last":0.505}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":4.0739999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.626},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":5.415},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58099999999999996,"15min":0.63200000000000001},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":4.202},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":4.2130000000000001},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1279999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.53500000000000003},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.83299999999999996},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.72499999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76300000000000001,"15min":0.99099999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.56799999999999995},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.872},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":1.1000000000000001},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307140,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064700,"statfs":{"total":96636764160,"available":96322252800,"internally_reserved":0,"allocated":1699840,"data_stored":778022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":4.9039999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.752,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.873,"15min":6.7949999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.625},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.68100000000000005},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.79500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.30199999999999999},"max":{"1min":0.79800000000000004,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.79600000000000004},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.54100000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.74399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.62},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":0.873},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:02:58.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:02:58.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.969+0000 7f3d5c12d700 1 Processor -- start 2023-10-06T05:02:58.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d5c12d700 1 -- start start 2023-10-06T05:02:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d5c12d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d540cd6c0 0x7f3d5415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d5c12d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d540cd230 con 0x7f3d540c6da0 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d5c12d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5415e4c0 con 0x7f3d5409d980 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d5c12d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d540cc660 con 0x7f3d540cd6c0 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d596c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d540cd6c0 0x7f3d5415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d5a6ca700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3d540c6da0 0x7f3d540cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37804/0 (socket says 172.21.15.3:37804) 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.970+0000 7f3d5a6ca700 1 -- 172.21.15.3:0/2585169610 learned_addr learned my addr 172.21.15.3:0/2585169610 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d58ec7700 1 -- 172.21.15.3:0/2585169610 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614425625 0 0) 0x7f3d540cd230 con 0x7f3d540c6da0 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d58ec7700 1 -- 172.21.15.3:0/2585169610 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d3c003570 con 0x7f3d540c6da0 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d596c8700 1 -- 172.21.15.3:0/2585169610 >> v1:172.21.15.153:6789/0 conn(0x7f3d5409d980 legacy=0x7f3d540692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d596c8700 1 -- 172.21.15.3:0/2585169610 >> v1:172.21.15.3:6789/0 conn(0x7f3d540c6da0 legacy=0x7f3d540cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d596c8700 1 -- 172.21.15.3:0/2585169610 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d540cc900 con 0x7f3d540cd6c0 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d596c8700 1 --2- 172.21.15.3:0/2585169610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540cd6c0 0x7f3d5415df80 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3d480098e0 tx=0x7f3d480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3eb853b8745c209 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d58ec7700 1 -- 172.21.15.3:0/2585169610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800b640 con 0x7f3d540cd6c0 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d58ec7700 1 -- 172.21.15.3:0/2585169610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4800b7a0 con 0x7f3d540cd6c0 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.971+0000 7f3d58ec7700 1 -- 172.21.15.3:0/2585169610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800bb40 con 0x7f3d540cd6c0 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.972+0000 7f3d5c12d700 1 -- 172.21.15.3:0/2585169610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540cd6c0 msgr2=0x7f3d5415df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.972+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/2585169610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540cd6c0 0x7f3d5415df80 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3d480098e0 tx=0x7f3d480091a0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.972+0000 7f3d5c12d700 1 -- 172.21.15.3:0/2585169610 shutdown_connections 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.972+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/2585169610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540cd6c0 0x7f3d5415df80 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.972+0000 7f3d5c12d700 1 -- 172.21.15.3:0/2585169610 >> 172.21.15.3:0/2585169610 conn(0x7f3d5406e730 msgr2=0x7f3d5406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.972+0000 7f3d5c12d700 1 -- 172.21.15.3:0/2585169610 shutdown_connections 2023-10-06T05:02:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.972+0000 7f3d5c12d700 1 -- 172.21.15.3:0/2585169610 wait complete. 2023-10-06T05:02:58.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.973+0000 7f3d5c12d700 1 Processor -- start 2023-10-06T05:02:58.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.973+0000 7f3d5c12d700 1 -- start start 2023-10-06T05:02:58.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5c12d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d5409d980 0x7f3d5415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5c12d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540c6da0 0x7f3d540cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5c12d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d540cd6c0 0x7f3d54145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5c12d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5415a5a0 con 0x7f3d540c6da0 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5c12d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5416e3f0 con 0x7f3d540cd6c0 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5c12d700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5416d590 con 0x7f3d5409d980 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540c6da0 0x7f3d540cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540c6da0 0x7f3d540cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50542/0 (socket says 172.21.15.3:50542) 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 -- 172.21.15.3:0/1629464553 learned_addr learned my addr 172.21.15.3:0/1629464553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5a6ca700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d540cd6c0 0x7f3d54145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d59ec9700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d5409d980 0x7f3d5415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 -- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d5409d980 msgr2=0x7f3d5415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d5409d980 0x7f3d5415a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 -- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d540cd6c0 msgr2=0x7f3d54145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d540cd6c0 0x7f3d54145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 -- 172.21.15.3:0/1629464553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48009420 con 0x7f3d540c6da0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5a6ca700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d540cd6c0 0x7f3d54145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d596c8700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540c6da0 0x7f3d540cf8c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3d54089650 tx=0x7f3d48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d51ffb700 1 -- 172.21.15.3:0/1629464553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800bb20 con 0x7f3d540c6da0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d51ffb700 1 -- 172.21.15.3:0/1629464553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d48011420 con 0x7f3d540c6da0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5416d7e0 con 0x7f3d540c6da0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.974+0000 7f3d51ffb700 1 -- 172.21.15.3:0/1629464553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d480117a0 con 0x7f3d540c6da0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.975+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5416dda0 con 0x7f3d540c6da0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.976+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5404c380 con 0x7f3d540c6da0 2023-10-06T05:02:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.976+0000 7f3d51ffb700 1 -- 172.21.15.3:0/1629464553 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d48011b50 con 0x7f3d540c6da0 2023-10-06T05:02:58.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.976+0000 7f3d51ffb700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d40066f30 0x7f3d400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:02:58.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.976+0000 7f3d51ffb700 1 -- 172.21.15.3:0/1629464553 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3d48085b80 con 0x7f3d540c6da0 2023-10-06T05:02:58.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.977+0000 7f3d59ec9700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d40066f30 0x7f3d400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:02:58.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.977+0000 7f3d59ec9700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d40066f30 0x7f3d400693e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3d44002200 tx=0x7f3d44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:02:58.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:58.981+0000 7f3d51ffb700 1 -- 172.21.15.3:0/1629464553 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d48052f60 con 0x7f3d540c6da0 2023-10-06T05:02:59.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.176+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d540ccbe0 con 0x7f3d540c6da0 2023-10-06T05:02:59.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.177+0000 7f3d51ffb700 1 -- 172.21.15.3:0/1629464553 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+13804 (secure 0 0 0) 0x7f3d48056580 con 0x7f3d540c6da0 2023-10-06T05:02:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:02:59.179 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":138,"fsid":"fa7cd5af-16f2-4a73-ab30-42f10cc13e22","created":"2023-10-06T04:51:59.745443+0000","modified":"2023-10-06T05:02:55.850158+0000","last_up_change":"2023-10-06T04:52:10.464542+0000","last_in_change":"2023-10-06T04:52:02.170377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T04:52:10.963739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T04:52:11.480186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a13544f3-f143-4c31-b32e-182882aca1bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":90654},{"type":"v1","addr":"172.21.15.3:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":90654},{"type":"v1","addr":"172.21.15.3:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":90654},{"type":"v1","addr":"172.21.15.3:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":90654},{"type":"v1","addr":"172.21.15.3:6805","nonce":90654}]},"public_addr":"172.21.15.3:6801/90654","cluster_addr":"172.21.15.3:6803/90654","heartbeat_back_addr":"172.21.15.3:6807/90654","heartbeat_front_addr":"172.21.15.3:6805/90654","state":["exists","up"]},{"osd":1,"uuid":"af00dc42-0e73-4607-a038-73b65ceee91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":90655},{"type":"v1","addr":"172.21.15.3:6809","nonce":90655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":90655},{"type":"v1","addr":"172.21.15.3:6811","nonce":90655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":90655},{"type":"v1","addr":"172.21.15.3:6815","nonce":90655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":90655},{"type":"v1","addr":"172.21.15.3:6813","nonce":90655}]},"public_addr":"172.21.15.3:6809/90655","cluster_addr":"172.21.15.3:6811/90655","heartbeat_back_addr":"172.21.15.3:6815/90655","heartbeat_front_addr":"172.21.15.3:6813/90655","state":["exists","up"]},{"osd":2,"uuid":"0eb853ca-ec51-4cf7-bd6d-6dd1805a3fde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":90656},{"type":"v1","addr":"172.21.15.3:6817","nonce":90656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":90656},{"type":"v1","addr":"172.21.15.3:6819","nonce":90656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":90656},{"type":"v1","addr":"172.21.15.3:6823","nonce":90656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":90656},{"type":"v1","addr":"172.21.15.3:6821","nonce":90656}]},"public_addr":"172.21.15.3:6817/90656","cluster_addr":"172.21.15.3:6819/90656","heartbeat_back_addr":"172.21.15.3:6823/90656","heartbeat_front_addr":"172.21.15.3:6821/90656","state":["exists","up"]},{"osd":3,"uuid":"76585c50-61a9-4bff-b04a-4e0f55264fac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":90685},{"type":"v1","addr":"172.21.15.3:6825","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":90685},{"type":"v1","addr":"172.21.15.3:6827","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":90685},{"type":"v1","addr":"172.21.15.3:6831","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":90685},{"type":"v1","addr":"172.21.15.3:6829","nonce":90685}]},"public_addr":"172.21.15.3:6825/90685","cluster_addr":"172.21.15.3:6827/90685","heartbeat_back_addr":"172.21.15.3:6831/90685","heartbeat_front_addr":"172.21.15.3:6829/90685","state":["exists","up"]},{"osd":4,"uuid":"b2c6e53d-d13b-4d9a-94c4-d8543698430b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6817","nonce":90907},{"type":"v1","addr":"172.21.15.153:6819","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6821","nonce":90907},{"type":"v1","addr":"172.21.15.153:6823","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":90907},{"type":"v1","addr":"172.21.15.153:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":90907},{"type":"v1","addr":"172.21.15.153:6828","nonce":90907}]},"public_addr":"172.21.15.153:6819/90907","cluster_addr":"172.21.15.153:6823/90907","heartbeat_back_addr":"172.21.15.153:6831/90907","heartbeat_front_addr":"172.21.15.153:6828/90907","state":["exists","up"]},{"osd":5,"uuid":"12e26508-a4e6-4c5a-9a52-a4c718ecf083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":90897},{"type":"v1","addr":"172.21.15.153:6801","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":90897},{"type":"v1","addr":"172.21.15.153:6803","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":90897},{"type":"v1","addr":"172.21.15.153:6807","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":90897},{"type":"v1","addr":"172.21.15.153:6805","nonce":90897}]},"public_addr":"172.21.15.153:6801/90897","cluster_addr":"172.21.15.153:6803/90897","heartbeat_back_addr":"172.21.15.153:6807/90897","heartbeat_front_addr":"172.21.15.153:6805/90897","state":["exists","up"]},{"osd":6,"uuid":"e127d2fe-d41c-4f6a-9936-acac760c6a38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":90898},{"type":"v1","addr":"172.21.15.153:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":90898},{"type":"v1","addr":"172.21.15.153:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":90898},{"type":"v1","addr":"172.21.15.153:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":90898},{"type":"v1","addr":"172.21.15.153:6813","nonce":90898}]},"public_addr":"172.21.15.153:6809/90898","cluster_addr":"172.21.15.153:6811/90898","heartbeat_back_addr":"172.21.15.153:6815/90898","heartbeat_front_addr":"172.21.15.153:6813/90898","state":["exists","up"]},{"osd":7,"uuid":"09729f85-d60d-44ba-a198-bbfd41694af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":90928},{"type":"v1","addr":"172.21.15.153:6818","nonce":90928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":90928},{"type":"v1","addr":"172.21.15.153:6822","nonce":90928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6827","nonce":90928},{"type":"v1","addr":"172.21.15.153:6829","nonce":90928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":90928},{"type":"v1","addr":"172.21.15.153:6825","nonce":90928}]},"public_addr":"172.21.15.153:6818/90928","cluster_addr":"172.21.15.153:6822/90928","heartbeat_back_addr":"172.21.15.153:6829/90928","heartbeat_front_addr":"172.21.15.153:6825/90928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.066510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.276739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.344312+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.372564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.385717+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.062851+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.314589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:52:07.364810+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:6837/2550905871":"2023-10-07T05:02:52.604494+0000","192.168.0.1:0/3882866003":"2023-10-06T06:02:46.060249+0000","172.21.15.153:6835/2550905871":"2023-10-07T05:02:52.604494+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:02:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d40066f30 msgr2=0x7f3d400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d40066f30 0x7f3d400693e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3d44002200 tx=0x7f3d44009040 comp rx=0 tx=0).stop 2023-10-06T05:02:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540c6da0 msgr2=0x7f3d540cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540c6da0 0x7f3d540cf8c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3d54089650 tx=0x7f3d48017000 comp rx=0 tx=0).stop 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 shutdown_connections 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d40066f30 0x7f3d400693e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d540cd6c0 0x7f3d54145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d5409d980 0x7f3d5415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 --2- 172.21.15.3:0/1629464553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d540c6da0 0x7f3d540cf8c0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 >> 172.21.15.3:0/1629464553 conn(0x7f3d5406e730 msgr2=0x7f3d540692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 shutdown_connections 2023-10-06T05:02:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:02:59.179+0000 7f3d5c12d700 1 -- 172.21.15.3:0/1629464553 wait complete. 2023-10-06T05:03:00.193 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T05:03:00.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:00.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.328+0000 7fe4f6233700 1 Processor -- start 2023-10-06T05:03:00.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.328+0000 7fe4f6233700 1 -- start start 2023-10-06T05:03:00.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4f6233700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4f0154f20 0x7fe4f00b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4f6233700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f01552f0 0x7fe4f00b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4f6233700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00b4160 con 0x7fe4f00b0010 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4f6233700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f0157df0 con 0x7fe4f0154f20 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4f6233700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b3550 con 0x7fe4f01552f0 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4ef7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4f0154f20 0x7fe4f00b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4effff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe4f00b0010 0x7fe4f00b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37812/0 (socket says 172.21.15.3:37812) 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4effff700 1 -- 172.21.15.3:0/2754176315 learned_addr learned my addr 172.21.15.3:0/2754176315 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4eeffd700 1 --2- 172.21.15.3:0/2754176315 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f01552f0 0x7fe4f00b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4ee7fc700 1 -- 172.21.15.3:0/2754176315 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136006926 0 0) 0x7fe4f00b4160 con 0x7fe4f00b0010 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.329+0000 7fe4ee7fc700 1 -- 172.21.15.3:0/2754176315 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d8003570 con 0x7fe4f00b0010 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ef7fe700 1 -- 172.21.15.3:0/2754176315 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f01552f0 msgr2=0x7fe4f00b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ef7fe700 1 --2- 172.21.15.3:0/2754176315 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f01552f0 0x7fe4f00b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ef7fe700 1 -- 172.21.15.3:0/2754176315 >> v1:172.21.15.3:6789/0 conn(0x7fe4f00b0010 legacy=0x7fe4f00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ef7fe700 1 -- 172.21.15.3:0/2754176315 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00b3830 con 0x7fe4f0154f20 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ef7fe700 1 --2- 172.21.15.3:0/2754176315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f0154f20 0x7fe4f00b2a00 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00098e0 tx=0x7fe4e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cf888e8ce119e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ee7fc700 1 -- 172.21.15.3:0/2754176315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0011630 con 0x7fe4f0154f20 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ee7fc700 1 -- 172.21.15.3:0/2754176315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0011790 con 0x7fe4f0154f20 2023-10-06T05:03:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4ee7fc700 1 -- 172.21.15.3:0/2754176315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0011b30 con 0x7fe4f0154f20 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4f6233700 1 -- 172.21.15.3:0/2754176315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f0154f20 msgr2=0x7fe4f00b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4f6233700 1 --2- 172.21.15.3:0/2754176315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f0154f20 0x7fe4f00b2a00 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00098e0 tx=0x7fe4e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4f6233700 1 -- 172.21.15.3:0/2754176315 shutdown_connections 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4f6233700 1 --2- 172.21.15.3:0/2754176315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f0154f20 0x7fe4f00b2a00 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4f6233700 1 --2- 172.21.15.3:0/2754176315 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f01552f0 0x7fe4f00b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.330+0000 7fe4f6233700 1 -- 172.21.15.3:0/2754176315 >> 172.21.15.3:0/2754176315 conn(0x7fe4f0070310 msgr2=0x7fe4f0070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.331+0000 7fe4f6233700 1 -- 172.21.15.3:0/2754176315 shutdown_connections 2023-10-06T05:03:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.331+0000 7fe4f6233700 1 -- 172.21.15.3:0/2754176315 wait complete. 2023-10-06T05:03:00.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.332+0000 7fe4f6233700 1 Processor -- start 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.332+0000 7fe4f6233700 1 -- start start 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4f6233700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f00b0010 0x7fe4f0144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4f6233700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0154f20 0x7fe4f00c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4f6233700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f01552f0 0x7fe4f00c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4f6233700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00ceb30 con 0x7fe4f01552f0 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4f6233700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00cdf20 con 0x7fe4f0154f20 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4f6233700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00ce200 con 0x7fe4f00b0010 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f01552f0 0x7fe4f00c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f01552f0 0x7fe4f00c3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50552/0 (socket says 172.21.15.3:50552) 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 -- 172.21.15.3:0/1311443268 learned_addr learned my addr 172.21.15.3:0/1311443268 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4ef7fe700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f00b0010 0x7fe4f0144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4eeffd700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0154f20 0x7fe4f00c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 -- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f00b0010 msgr2=0x7fe4f0144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe4f00b0010 0x7fe4f0144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 -- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0154f20 msgr2=0x7fe4f00c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0154f20 0x7fe4f00c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 -- 172.21.15.3:0/1311443268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e0009420 con 0x7fe4f01552f0 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4effff700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f01552f0 0x7fe4f00c3600 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8001460 tx=0x7fe4e8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800ce80 con 0x7fe4f01552f0 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e800ab70 con 0x7fe4f01552f0 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f00ce4e0 con 0x7fe4f01552f0 2023-10-06T05:03:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.333+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80070f0 con 0x7fe4f01552f0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.334+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01451f0 con 0x7fe4f01552f0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.334+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4f004c380 con 0x7fe4f01552f0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.335+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4e8007520 con 0x7fe4f01552f0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.335+0000 7fe4d77fe700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe4dc066f30 0x7fe4dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.335+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe4e8084890 con 0x7fe4f01552f0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.335+0000 7fe4d77fe700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7fe4dc06d3f0 0x7fe4dc06f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.335+0000 7fe4ef7fe700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe4dc066f30 0x7fe4dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.335+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 --> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4dc06feb0 con 0x7fe4dc06d3f0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.335+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e8089240 con 0x7fe4f01552f0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.336+0000 7fe4eeffd700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7fe4dc06d3f0 0x7fe4dc06f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.336+0000 7fe4eeffd700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7fe4dc06d3f0 0x7fe4dc06f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.336+0000 7fe4ef7fe700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe4dc066f30 0x7fe4dc0693e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00091e0 tx=0x7fe4e0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.337+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== osd.0 v2:172.21.15.3:6800/90654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe4dc06feb0 con 0x7fe4dc06d3f0 2023-10-06T05:03:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.350+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 --> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe4f006a950 con 0x7fe4dc06d3f0 2023-10-06T05:03:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4d77fe700 1 -- 172.21.15.3:0/1311443268 <== osd.0 v2:172.21.15.3:6800/90654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe4f006a950 con 0x7fe4dc06d3f0 2023-10-06T05:03:00.352 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:00.353 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:00.353 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7fe4dc06d3f0 msgr2=0x7fe4dc06f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:6800/90654,v1:172.21.15.3:6801/90654] conn(0x7fe4dc06d3f0 0x7fe4dc06f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe4dc066f30 msgr2=0x7fe4dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe4dc066f30 0x7fe4dc0693e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00091e0 tx=0x7fe4e0017000 comp rx=0 tx=0).stop 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f01552f0 msgr2=0x7fe4f00c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 --2- 172.21.15.3:0/1311443268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f01552f0 0x7fe4f00c3600 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8001460 tx=0x7fe4e8001ae0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4effff700 1 -- 172.21.15.3:0/1311443268 reap_dead start 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 shutdown_connections 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 >> 172.21.15.3:0/1311443268 conn(0x7fe4f0070310 msgr2=0x7fe4f00adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 shutdown_connections 2023-10-06T05:03:00.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.351+0000 7fe4f6233700 1 -- 172.21.15.3:0/1311443268 wait complete. 2023-10-06T05:03:00.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T05:03:00.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.482+0000 7f7490293700 1 Processor -- start 2023-10-06T05:03:00.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.482+0000 7f7490293700 1 -- start start 2023-10-06T05:03:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.483+0000 7f7490293700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880cd6c0 0x7f748815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.483+0000 7f7490293700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f74880c6da0 0x7f74880cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.483+0000 7f7490293700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74880cd230 con 0x7f748809d980 2023-10-06T05:03:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.483+0000 7f7490293700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748815e4c0 con 0x7f74880c6da0 2023-10-06T05:03:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.483+0000 7f7490293700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880cc660 con 0x7f74880cd6c0 2023-10-06T05:03:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748d82e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880cd6c0 0x7f748815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e830700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f74880c6da0 0x7f74880cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e02f700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f748809d980 0x7f74880692f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34258/0 (socket says 172.21.15.3:34258) 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e02f700 1 -- 172.21.15.3:0/1885755474 learned_addr learned my addr 172.21.15.3:0/1885755474 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748d82e700 1 --2- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880cd6c0 0x7f748815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:41360/0 (socket says 172.21.15.3:41360) 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e830700 1 -- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880cd6c0 msgr2=0x7f748815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e830700 1 --2- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880cd6c0 0x7f748815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e830700 1 -- 172.21.15.3:0/1885755474 >> v1:172.21.15.153:6789/0 conn(0x7f748809d980 legacy=0x7f74880692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748d02d700 1 -- 172.21.15.3:0/1885755474 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742396298 0 0) 0x7f74880cd230 con 0x7f748809d980 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e830700 1 -- 172.21.15.3:0/1885755474 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880cc900 con 0x7f74880c6da0 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748d82e700 1 --2- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880cd6c0 0x7f748815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:03:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.484+0000 7f748e830700 1 --2- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74880c6da0 0x7f74880cab70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7478009e60 tx=0x7f7478009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a72a99e1228c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f748d02d700 1 -- 172.21.15.3:0/1885755474 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747800e070 con 0x7f74880c6da0 2023-10-06T05:03:00.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f748d02d700 1 -- 172.21.15.3:0/1885755474 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747800bb60 con 0x7f74880c6da0 2023-10-06T05:03:00.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f748d02d700 1 -- 172.21.15.3:0/1885755474 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478012430 con 0x7f74880c6da0 2023-10-06T05:03:00.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 -- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74880c6da0 msgr2=0x7f74880cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 --2- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74880c6da0 0x7f74880cab70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7478009e60 tx=0x7f7478009780 comp rx=0 tx=0).stop 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 -- 172.21.15.3:0/1885755474 shutdown_connections 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 --2- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74880c6da0 0x7f74880cab70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 --2- 172.21.15.3:0/1885755474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880cd6c0 0x7f748815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 -- 172.21.15.3:0/1885755474 >> 172.21.15.3:0/1885755474 conn(0x7f748806e730 msgr2=0x7f748806eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 -- 172.21.15.3:0/1885755474 shutdown_connections 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.485+0000 7f7490293700 1 -- 172.21.15.3:0/1885755474 wait complete. 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.486+0000 7f7490293700 1 Processor -- start 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.486+0000 7f7490293700 1 -- start start 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f7490293700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f748809d980 0x7f74880cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f7490293700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880c6da0 0x7f748815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f7490293700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74880cd6c0 0x7f7488150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f7490293700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c80a0 con 0x7f74880cd6c0 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f7490293700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748815ade0 con 0x7f748809d980 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f7490293700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880cf9a0 con 0x7f74880c6da0 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f748e830700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74880cd6c0 0x7f7488150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f748e830700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74880cd6c0 0x7f7488150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50556/0 (socket says 172.21.15.3:50556) 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f748e830700 1 -- 172.21.15.3:0/2407870945 learned_addr learned my addr 172.21.15.3:0/2407870945 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f748e02f700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f748809d980 0x7f74880cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.487+0000 7f748d82e700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880c6da0 0x7f748815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f748e830700 1 -- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880c6da0 msgr2=0x7f748815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f748e830700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880c6da0 0x7f748815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f748e830700 1 -- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f748809d980 msgr2=0x7f74880cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f748e830700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f748809d980 0x7f74880cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f748e830700 1 -- 172.21.15.3:0/2407870945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c009420 con 0x7f74880cd6c0 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f748e830700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74880cd6c0 0x7f7488150720 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7478012ad0 tx=0x7f7478019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f74867fc700 1 -- 172.21.15.3:0/2407870945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478027470 con 0x7f74880cd6c0 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f74867fc700 1 -- 172.21.15.3:0/2407870945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74780275d0 con 0x7f74880cd6c0 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7478009a20 con 0x7f74880cd6c0 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f74867fc700 1 -- 172.21.15.3:0/2407870945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478027990 con 0x7f74880cd6c0 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.488+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880cf3e0 con 0x7f74880cd6c0 2023-10-06T05:03:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.489+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880c8250 con 0x7f74880cd6c0 2023-10-06T05:03:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.489+0000 7f74867fc700 1 -- 172.21.15.3:0/2407870945 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7478024070 con 0x7f74880cd6c0 2023-10-06T05:03:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.490+0000 7f74867fc700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7474066f30 0x7f74740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.490+0000 7f74867fc700 1 -- 172.21.15.3:0/2407870945 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7478086b10 con 0x7f74880cd6c0 2023-10-06T05:03:00.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.490+0000 7f748e02f700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7474066f30 0x7f74740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.491+0000 7f748e02f700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7474066f30 0x7f74740693e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f74800019b0 tx=0x7f7480009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.494+0000 7f74867fc700 1 -- 172.21.15.3:0/2407870945 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74880c8250 con 0x7f74880cd6c0 2023-10-06T05:03:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.688+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f74880c8250 con 0x7f7474066f30 2023-10-06T05:03:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.689+0000 7f74867fc700 1 -- 172.21.15.3:0/2407870945 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f74880c8250 con 0x7f7474066f30 2023-10-06T05:03:00.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7474066f30 msgr2=0x7f74740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7474066f30 0x7f74740693e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f74800019b0 tx=0x7f7480009040 comp rx=0 tx=0).stop 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74880cd6c0 msgr2=0x7f7488150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74880cd6c0 0x7f7488150720 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7478012ad0 tx=0x7f7478019000 comp rx=0 tx=0).stop 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 shutdown_connections 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7474066f30 0x7f74740693e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f748809d980 0x7f74880cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74880cd6c0 0x7f7488150720 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 --2- 172.21.15.3:0/2407870945 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74880c6da0 0x7f748815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 >> 172.21.15.3:0/2407870945 conn(0x7f748806e730 msgr2=0x7f74880692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.691+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 shutdown_connections 2023-10-06T05:03:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.692+0000 7f7490293700 1 -- 172.21.15.3:0/2407870945 wait complete. 2023-10-06T05:03:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T05:03:00.705 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T05:03:00.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.827+0000 7f71d8dad700 1 Processor -- start 2023-10-06T05:03:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.827+0000 7f71d8dad700 1 -- start start 2023-10-06T05:03:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d8dad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71d415d7c0 0x7f71d40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d8dad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c6170 0x7f71d415db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d8dad700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40c5c90 con 0x7f71d40c65e0 2023-10-06T05:03:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d8dad700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d4159e70 con 0x7f71d415d7c0 2023-10-06T05:03:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d8dad700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40af860 con 0x7f71d40c6170 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71d415d7c0 0x7f71d40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f71d40c65e0 0x7f71d40cd070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37822/0 (socket says 172.21.15.3:37822) 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d2d9d700 1 -- 172.21.15.3:0/285299995 learned_addr learned my addr 172.21.15.3:0/285299995 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d1d9b700 1 --2- 172.21.15.3:0/285299995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c6170 0x7f71d415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d259c700 1 -- 172.21.15.3:0/285299995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c6170 msgr2=0x7f71d415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d259c700 1 --2- 172.21.15.3:0/285299995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c6170 0x7f71d415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d159a700 1 -- 172.21.15.3:0/285299995 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144795719 0 0) 0x7f71d40c5c90 con 0x7f71d40c65e0 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d259c700 1 -- 172.21.15.3:0/285299995 >> v1:172.21.15.3:6789/0 conn(0x7f71d40c65e0 legacy=0x7f71d40cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d259c700 1 -- 172.21.15.3:0/285299995 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40b5190 con 0x7f71d415d7c0 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d1d9b700 1 --2- 172.21.15.3:0/285299995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c6170 0x7f71d415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.828+0000 7f71d259c700 1 --2- 172.21.15.3:0/285299995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40b4bb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f71c40098d0 tx=0x7f71c4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322dd051f2d8e479 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d159a700 1 -- 172.21.15.3:0/285299995 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4009e70 con 0x7f71d415d7c0 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d159a700 1 -- 172.21.15.3:0/285299995 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4005c10 con 0x7f71d415d7c0 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d159a700 1 -- 172.21.15.3:0/285299995 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c400b650 con 0x7f71d415d7c0 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 -- 172.21.15.3:0/285299995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 msgr2=0x7f71d40b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 --2- 172.21.15.3:0/285299995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40b4bb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f71c40098d0 tx=0x7f71c4009120 comp rx=0 tx=0).stop 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 -- 172.21.15.3:0/285299995 shutdown_connections 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 --2- 172.21.15.3:0/285299995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40b4bb0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 --2- 172.21.15.3:0/285299995 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c6170 0x7f71d415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 -- 172.21.15.3:0/285299995 >> 172.21.15.3:0/285299995 conn(0x7f71d406e730 msgr2=0x7f71d406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 -- 172.21.15.3:0/285299995 shutdown_connections 2023-10-06T05:03:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.829+0000 7f71d8dad700 1 -- 172.21.15.3:0/285299995 wait complete. 2023-10-06T05:03:00.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.830+0000 7f71d8dad700 1 Processor -- start 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.830+0000 7f71d8dad700 1 -- start start 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d8dad700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71d40c6170 0x7f71d4157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d8dad700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c65e0 0x7f71d40c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d8dad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d8dad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40c85b0 con 0x7f71d415d7c0 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d8dad700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40c9410 con 0x7f71d40c6170 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d8dad700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4157640 con 0x7f71d40c65e0 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50578/0 (socket says 172.21.15.3:50578) 2023-10-06T05:03:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 -- 172.21.15.3:0/3959462639 learned_addr learned my addr 172.21.15.3:0/3959462639 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d259c700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71d40c6170 0x7f71d4157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d1d9b700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c65e0 0x7f71d40c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 -- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c65e0 msgr2=0x7f71d40c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c65e0 0x7f71d40c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 -- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71d40c6170 msgr2=0x7f71d4157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71d40c6170 0x7f71d4157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 -- 172.21.15.3:0/3959462639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40093c0 con 0x7f71d415d7c0 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d1d9b700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71d40c65e0 0x7f71d40c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.831+0000 7f71d2d9d700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40cd3b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00ccf0 tx=0x7f71cc001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.832+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00aad0 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.832+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00ac30 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.832+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d40c98a0 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.832+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0070f0 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.832+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d40c8ae0 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.832+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71d404c380 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.833+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71cc007520 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.833+0000 7f71ca7fc700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f71b4066f30 0x7f71b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.833+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f71cc0848f0 con 0x7f71d415d7c0 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.834+0000 7f71ca7fc700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7f71b406d3f0 0x7f71b406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.834+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 --> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71b406feb0 con 0x7f71b406d3f0 2023-10-06T05:03:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.834+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71cc089220 con 0x7f71d415d7c0 2023-10-06T05:03:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.834+0000 7f71d259c700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f71b4066f30 0x7f71b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.834+0000 7f71d1d9b700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7f71b406d3f0 0x7f71b406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.834+0000 7f71d1d9b700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7f71b406d3f0 0x7f71b406f800 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.834+0000 7f71d259c700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f71b4066f30 0x7f71b40693e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009160 tx=0x7f71c4016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.835+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== osd.1 v2:172.21.15.3:6808/90655 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f71b406feb0 con 0x7f71b406d3f0 2023-10-06T05:03:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.848+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 --> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f71d40b46d0 con 0x7f71b406d3f0 2023-10-06T05:03:00.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71ca7fc700 1 -- 172.21.15.3:0/3959462639 <== osd.1 v2:172.21.15.3:6808/90655 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f71d40b46d0 con 0x7f71b406d3f0 2023-10-06T05:03:00.850 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:00.850 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:00.850 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:00.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7f71b406d3f0 msgr2=0x7f71b406f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:6808/90655,v1:172.21.15.3:6809/90655] conn(0x7f71b406d3f0 0x7f71b406f800 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f71b4066f30 msgr2=0x7f71b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f71b4066f30 0x7f71b40693e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009160 tx=0x7f71c4016000 comp rx=0 tx=0).stop 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 msgr2=0x7f71d40cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 --2- 172.21.15.3:0/3959462639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d415d7c0 0x7f71d40cd3b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00ccf0 tx=0x7f71cc001a20 comp rx=0 tx=0).stop 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d2d9d700 1 -- 172.21.15.3:0/3959462639 reap_dead start 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 shutdown_connections 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 >> 172.21.15.3:0/3959462639 conn(0x7f71d406e730 msgr2=0x7f71d40d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 shutdown_connections 2023-10-06T05:03:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.849+0000 7f71d8dad700 1 -- 172.21.15.3:0/3959462639 wait complete. 2023-10-06T05:03:00.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T05:03:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.981+0000 7fd13340e700 1 Processor -- start 2023-10-06T05:03:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.981+0000 7fd13340e700 1 -- start start 2023-10-06T05:03:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd13340e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b2520 0x7fd12c0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd13340e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd12c0cd510 0x7fd12c0cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd13340e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c0cc970 con 0x7fd12c0b3680 2023-10-06T05:03:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd13340e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0cc3f0 con 0x7fd12c0cd510 2023-10-06T05:03:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd13340e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0cbd40 con 0x7fd12c0b2520 2023-10-06T05:03:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd131c0b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd12c0b3680 0x7fd12c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37830/0 (socket says 172.21.15.3:37830) 2023-10-06T05:03:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd132c0d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd12c0cd510 0x7fd12c0cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd131c0b700 1 -- 172.21.15.3:0/4011814360 learned_addr learned my addr 172.21.15.3:0/4011814360 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd13240c700 1 --2- 172.21.15.3:0/4011814360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b2520 0x7fd12c0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd132c0d700 1 -- 172.21.15.3:0/4011814360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b2520 msgr2=0x7fd12c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd132c0d700 1 --2- 172.21.15.3:0/4011814360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b2520 0x7fd12c0b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd132c0d700 1 -- 172.21.15.3:0/4011814360 >> v1:172.21.15.3:6789/0 conn(0x7fd12c0b3680 legacy=0x7fd12c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd132c0d700 1 -- 172.21.15.3:0/4011814360 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c0cc020 con 0x7fd12c0cd510 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.982+0000 7fd13140a700 1 -- 172.21.15.3:0/4011814360 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066154165 0 0) 0x7fd12c0cc970 con 0x7fd12c0b3680 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd132c0d700 1 --2- 172.21.15.3:0/4011814360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c0cb0c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd1240098e0 tx=0x7fd1240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fb5559af54c2a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13140a700 1 -- 172.21.15.3:0/4011814360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124009e80 con 0x7fd12c0cd510 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13140a700 1 -- 172.21.15.3:0/4011814360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12400b860 con 0x7fd12c0cd510 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13140a700 1 -- 172.21.15.3:0/4011814360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400bbe0 con 0x7fd12c0cd510 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 -- 172.21.15.3:0/4011814360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 msgr2=0x7fd12c0cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 --2- 172.21.15.3:0/4011814360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c0cb0c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd1240098e0 tx=0x7fd1240091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 -- 172.21.15.3:0/4011814360 shutdown_connections 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 --2- 172.21.15.3:0/4011814360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c0cb0c0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 --2- 172.21.15.3:0/4011814360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b2520 0x7fd12c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 -- 172.21.15.3:0/4011814360 >> 172.21.15.3:0/4011814360 conn(0x7fd12c06e730 msgr2=0x7fd12c06eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 -- 172.21.15.3:0/4011814360 shutdown_connections 2023-10-06T05:03:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.983+0000 7fd13340e700 1 -- 172.21.15.3:0/4011814360 wait complete. 2023-10-06T05:03:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.984+0000 7fd13340e700 1 Processor -- start 2023-10-06T05:03:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.984+0000 7fd13340e700 1 -- start start 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd13340e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd12c0b2520 0x7fd12c0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd13340e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b3680 0x7fd12c0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd13340e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd13340e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c14d2d0 con 0x7fd12c0cd510 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd13340e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0d04c0 con 0x7fd12c0b2520 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd13340e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0cf8b0 con 0x7fd12c0b3680 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd132c0d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd132c0d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c1509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50608/0 (socket says 172.21.15.3:50608) 2023-10-06T05:03:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd132c0d700 1 -- 172.21.15.3:0/3737440153 learned_addr learned my addr 172.21.15.3:0/3737440153 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd13240c700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd12c0b2520 0x7fd12c0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.985+0000 7fd131c0b700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b3680 0x7fd12c0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd132c0d700 1 -- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b3680 msgr2=0x7fd12c0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd132c0d700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b3680 0x7fd12c0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd132c0d700 1 -- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd12c0b2520 msgr2=0x7fd12c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd132c0d700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd12c0b2520 0x7fd12c0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd132c0d700 1 -- 172.21.15.3:0/3737440153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124009420 con 0x7fd12c0cd510 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd132c0d700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c1509c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fd124013000 tx=0x7fd124017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd12a7fc700 1 -- 172.21.15.3:0/3737440153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124011630 con 0x7fd12c0cd510 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd12a7fc700 1 -- 172.21.15.3:0/3737440153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1240117b0 con 0x7fd12c0cd510 2023-10-06T05:03:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd12a7fc700 1 -- 172.21.15.3:0/3737440153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124011b70 con 0x7fd12c0cd510 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c0cfb90 con 0x7fd12c0cd510 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.986+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c0d0150 con 0x7fd12c0cd510 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.987+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c0d0650 con 0x7fd12c0cd510 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.987+0000 7fd12a7fc700 1 -- 172.21.15.3:0/3737440153 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1240193b0 con 0x7fd12c0cd510 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.988+0000 7fd12a7fc700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.988+0000 7fd12a7fc700 1 -- 172.21.15.3:0/3737440153 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd1240861f0 con 0x7fd12c0cd510 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.988+0000 7fd13240c700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.988+0000 7fd13240c700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd118066f30 0x7fd1180693e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e4d0 tx=0x7fd11c007680 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:00.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:00.992+0000 7fd12a7fc700 1 -- 172.21.15.3:0/3737440153 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd124053650 con 0x7fd12c0cd510 2023-10-06T05:03:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.186+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd12c04c380 con 0x7fd118066f30 2023-10-06T05:03:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.187+0000 7fd12a7fc700 1 -- 172.21.15.3:0/3737440153 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd12c04c380 con 0x7fd118066f30 2023-10-06T05:03:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd118066f30 msgr2=0x7fd1180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd118066f30 0x7fd1180693e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e4d0 tx=0x7fd11c007680 comp rx=0 tx=0).stop 2023-10-06T05:03:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 msgr2=0x7fd12c1509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c1509c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fd124013000 tx=0x7fd124017000 comp rx=0 tx=0).stop 2023-10-06T05:03:01.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 shutdown_connections 2023-10-06T05:03:01.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd12c0b2520 0x7fd12c0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd12c0cd510 0x7fd12c1509c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 --2- 172.21.15.3:0/3737440153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd12c0b3680 0x7fd12c0d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 >> 172.21.15.3:0/3737440153 conn(0x7fd12c06e730 msgr2=0x7fd12c06eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 shutdown_connections 2023-10-06T05:03:01.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.189+0000 7fd13340e700 1 -- 172.21.15.3:0/3737440153 wait complete. 2023-10-06T05:03:01.193 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T05:03:01.202 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T05:03:01.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.324+0000 7f14cf438700 1 Processor -- start 2023-10-06T05:03:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cf438700 1 -- start start 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cf438700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14c80aae50 0x7f14c80b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cf438700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8067d60 0x7f14c8069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cf438700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c806a020 con 0x7f14c8069bc0 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cf438700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c8068320 con 0x7f14c80aae50 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cf438700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c8068600 con 0x7f14c8067d60 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cd1d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14c80aae50 0x7f14c80b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cd9d5700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14c8069bc0 0x7f14c815d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37846/0 (socket says 172.21.15.3:37846) 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.325+0000 7f14cd9d5700 1 -- 172.21.15.3:0/283341749 learned_addr learned my addr 172.21.15.3:0/283341749 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14cc9d3700 1 --2- 172.21.15.3:0/283341749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8067d60 0x7f14c8069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14c7fff700 1 -- 172.21.15.3:0/283341749 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453894208 0 0) 0x7f14b8001110 con 0x7f14c8069bc0 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14c7fff700 1 -- 172.21.15.3:0/283341749 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b0003570 con 0x7f14c8069bc0 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14cd1d4700 1 -- 172.21.15.3:0/283341749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8067d60 msgr2=0x7f14c8069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14cd1d4700 1 --2- 172.21.15.3:0/283341749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8067d60 0x7f14c8069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14cd1d4700 1 -- 172.21.15.3:0/283341749 >> v1:172.21.15.3:6789/0 conn(0x7f14c8069bc0 legacy=0x7f14c815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14cd1d4700 1 -- 172.21.15.3:0/283341749 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c80688e0 con 0x7f14c80aae50 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14cd1d4700 1 --2- 172.21.15.3:0/283341749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c80aae50 0x7f14c80b3d20 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009eb0 tx=0x7f14c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd763e91c636a254 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14c7fff700 1 -- 172.21.15.3:0/283341749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c000e070 con 0x7f14c80aae50 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14c7fff700 1 -- 172.21.15.3:0/283341749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c000bc30 con 0x7f14c80aae50 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.326+0000 7f14c7fff700 1 -- 172.21.15.3:0/283341749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0012430 con 0x7f14c80aae50 2023-10-06T05:03:01.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 -- 172.21.15.3:0/283341749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c80aae50 msgr2=0x7f14c80b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 --2- 172.21.15.3:0/283341749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c80aae50 0x7f14c80b3d20 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009eb0 tx=0x7f14c0009780 comp rx=0 tx=0).stop 2023-10-06T05:03:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 -- 172.21.15.3:0/283341749 shutdown_connections 2023-10-06T05:03:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 --2- 172.21.15.3:0/283341749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c80aae50 0x7f14c80b3d20 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 --2- 172.21.15.3:0/283341749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8067d60 0x7f14c8069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 -- 172.21.15.3:0/283341749 >> 172.21.15.3:0/283341749 conn(0x7f14c806e730 msgr2=0x7f14c806eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 -- 172.21.15.3:0/283341749 shutdown_connections 2023-10-06T05:03:01.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.327+0000 7f14cf438700 1 -- 172.21.15.3:0/283341749 wait complete. 2023-10-06T05:03:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.328+0000 7f14cf438700 1 Processor -- start 2023-10-06T05:03:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.328+0000 7f14cf438700 1 -- start start 2023-10-06T05:03:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cf438700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c8067d60 0x7f14c814ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cf438700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8069bc0 0x7f14c816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cf438700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c80aae50 0x7f14c815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cf438700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c80ab220 con 0x7f14c8067d60 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cf438700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c80ab500 con 0x7f14c80aae50 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cf438700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c816a8b0 con 0x7f14c8069bc0 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c8067d60 0x7f14c814ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c8067d60 0x7f14c814ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50634/0 (socket says 172.21.15.3:50634) 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 -- 172.21.15.3:0/2002073620 learned_addr learned my addr 172.21.15.3:0/2002073620 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd9d5700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c80aae50 0x7f14c815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cc9d3700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8069bc0 0x7f14c816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 -- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8069bc0 msgr2=0x7f14c816a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14c8069bc0 0x7f14c816a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 -- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c80aae50 msgr2=0x7f14c815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c80aae50 0x7f14c815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 -- 172.21.15.3:0/2002073620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0009a20 con 0x7f14c8067d60 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cd1d4700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c8067d60 0x7f14c814ac80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f14c0013000 tx=0x7f14c0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0012db0 con 0x7f14c8067d60 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0027470 con 0x7f14c8067d60 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c816ab90 con 0x7f14c8067d60 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.329+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0027830 con 0x7f14c8067d60 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.330+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c814b1c0 con 0x7f14c8067d60 2023-10-06T05:03:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.330+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14c804c380 con 0x7f14c8067d60 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.331+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14c0024070 con 0x7f14c8067d60 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.331+0000 7f14c57fa700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14b4066f30 0x7f14b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.331+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f14c0086dc0 con 0x7f14c8067d60 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.331+0000 7f14c57fa700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f14b406d3f0 0x7f14b406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.331+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 --> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14b406feb0 con 0x7f14b406d3f0 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.331+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14c0089400 con 0x7f14c8067d60 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.331+0000 7f14cc9d3700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14b4066f30 0x7f14b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.332+0000 7f14cd9d5700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f14b406d3f0 0x7f14b406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.332+0000 7f14cd9d5700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f14b406d3f0 0x7f14b406f800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.332+0000 7f14cc9d3700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14b4066f30 0x7f14b40693e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f14c80d0d20 tx=0x7f14bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.332+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== osd.2 v2:172.21.15.3:6816/90656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f14b406feb0 con 0x7f14b406d3f0 2023-10-06T05:03:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.346+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 --> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f14c80b4780 con 0x7f14b406d3f0 2023-10-06T05:03:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14c57fa700 1 -- 172.21.15.3:0/2002073620 <== osd.2 v2:172.21.15.3:6816/90656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f14c80b4780 con 0x7f14b406d3f0 2023-10-06T05:03:01.349 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:01.349 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:01.349 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f14b406d3f0 msgr2=0x7f14b406f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:6816/90656,v1:172.21.15.3:6817/90656] conn(0x7f14b406d3f0 0x7f14b406f800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14b4066f30 msgr2=0x7f14b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f14b4066f30 0x7f14b40693e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f14c80d0d20 tx=0x7f14bc008040 comp rx=0 tx=0).stop 2023-10-06T05:03:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c8067d60 msgr2=0x7f14c814ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 --2- 172.21.15.3:0/2002073620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c8067d60 0x7f14c814ac80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f14c0013000 tx=0x7f14c0019000 comp rx=0 tx=0).stop 2023-10-06T05:03:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cd9d5700 1 -- 172.21.15.3:0/2002073620 reap_dead start 2023-10-06T05:03:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 shutdown_connections 2023-10-06T05:03:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 >> 172.21.15.3:0/2002073620 conn(0x7f14c806e730 msgr2=0x7f14c815c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 shutdown_connections 2023-10-06T05:03:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.347+0000 7f14cf438700 1 -- 172.21.15.3:0/2002073620 wait complete. 2023-10-06T05:03:01.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T05:03:01.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.478+0000 7f6c4f59e700 1 Processor -- start 2023-10-06T05:03:01.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.478+0000 7f6c4f59e700 1 -- start start 2023-10-06T05:03:01.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c500b2520 0x7f6c500b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c500b3680 0x7f6c500cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c500cc970 con 0x7f6c500b28f0 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4f59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c500cc3f0 con 0x7f6c500b3680 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c500cbd40 con 0x7f6c500b2520 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c500b3680 0x7f6c500cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6c500b28f0 0x7f6c500cb0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37862/0 (socket says 172.21.15.3:37862) 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4ed9d700 1 -- 172.21.15.3:0/1061661379 learned_addr learned my addr 172.21.15.3:0/1061661379 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4e59c700 1 --2- 172.21.15.3:0/1061661379 >> v2:172.21.15.153:3300/0 conn(0x7f6c500b2520 0x7f6c500b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4dd9b700 1 -- 172.21.15.3:0/1061661379 >> v2:172.21.15.153:3300/0 conn(0x7f6c500b2520 msgr2=0x7f6c500b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4dd9b700 1 --2- 172.21.15.3:0/1061661379 >> v2:172.21.15.153:3300/0 conn(0x7f6c500b2520 0x7f6c500b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4dd9b700 1 -- 172.21.15.3:0/1061661379 >> v1:172.21.15.3:6789/0 conn(0x7f6c500b28f0 legacy=0x7f6c500cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.479+0000 7f6c4dd9b700 1 -- 172.21.15.3:0/1061661379 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c500cc020 con 0x7f6c500b3680 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4dd9b700 1 --2- 172.21.15.3:0/1061661379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500cce00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f6c50068da0 tx=0x7f6c380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a163ebdec8abfed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4d59a700 1 -- 172.21.15.3:0/1061661379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800b6b0 con 0x7f6c500b3680 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4d59a700 1 -- 172.21.15.3:0/1061661379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3800b810 con 0x7f6c500b3680 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4d59a700 1 -- 172.21.15.3:0/1061661379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800bbb0 con 0x7f6c500b3680 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1061661379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 msgr2=0x7f6c500cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1061661379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500cce00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f6c50068da0 tx=0x7f6c380091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1061661379 shutdown_connections 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1061661379 >> v2:172.21.15.153:3300/0 conn(0x7f6c500b2520 0x7f6c500b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1061661379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500cce00 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1061661379 >> 172.21.15.3:0/1061661379 conn(0x7f6c5006e730 msgr2=0x7f6c5006eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1061661379 shutdown_connections 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.480+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1061661379 wait complete. 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.481+0000 7f6c4f59e700 1 Processor -- start 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.481+0000 7f6c4f59e700 1 -- start start 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500b2520 0x7f6c5014d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c500b28f0 0x7f6c500d0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5016b0c0 con 0x7f6c500b3680 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5014ddb0 con 0x7f6c500b2520 2023-10-06T05:03:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5016a4b0 con 0x7f6c500b28f0 2023-10-06T05:03:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50644/0 (socket says 172.21.15.3:50644) 2023-10-06T05:03:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4ed9d700 1 -- 172.21.15.3:0/1638194540 learned_addr learned my addr 172.21.15.3:0/1638194540 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4e59c700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500b2520 0x7f6c5014d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.482+0000 7f6c4dd9b700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c500b28f0 0x7f6c500d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4ed9d700 1 -- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c500b28f0 msgr2=0x7f6c500d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4ed9d700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c500b28f0 0x7f6c500d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4ed9d700 1 -- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500b2520 msgr2=0x7f6c5014d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4ed9d700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500b2520 0x7f6c5014d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4ed9d700 1 -- 172.21.15.3:0/1638194540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38009420 con 0x7f6c500b3680 2023-10-06T05:03:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4ed9d700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500d08d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c590 tx=0x7f6c48007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c467fc700 1 -- 172.21.15.3:0/1638194540 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48005b30 con 0x7f6c500b3680 2023-10-06T05:03:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c467fc700 1 -- 172.21.15.3:0/1638194540 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48005cb0 con 0x7f6c500b3680 2023-10-06T05:03:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5016a790 con 0x7f6c500b3680 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c467fc700 1 -- 172.21.15.3:0/1638194540 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48007320 con 0x7f6c500b3680 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.483+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5016ad50 con 0x7f6c500b3680 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.484+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5004c380 con 0x7f6c500b3680 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.484+0000 7f6c467fc700 1 -- 172.21.15.3:0/1638194540 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c4801b3b0 con 0x7f6c500b3680 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.485+0000 7f6c467fc700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c30066f30 0x7f6c300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.485+0000 7f6c467fc700 1 -- 172.21.15.3:0/1638194540 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c48083840 con 0x7f6c500b3680 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.485+0000 7f6c4e59c700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c30066f30 0x7f6c300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.486+0000 7f6c4e59c700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c30066f30 0x7f6c300693e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009fd0 tx=0x7f6c40009880 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.489+0000 7f6c467fc700 1 -- 172.21.15.3:0/1638194540 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c48050b80 con 0x7f6c500b3680 2023-10-06T05:03:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.683+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c500cc600 con 0x7f6c30066f30 2023-10-06T05:03:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.684+0000 7f6c467fc700 1 -- 172.21.15.3:0/1638194540 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6c500cc600 con 0x7f6c30066f30 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c30066f30 msgr2=0x7f6c300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c30066f30 0x7f6c300693e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009fd0 tx=0x7f6c40009880 comp rx=0 tx=0).stop 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 msgr2=0x7f6c500d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500d08d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c590 tx=0x7f6c48007ef0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 shutdown_connections 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6c30066f30 0x7f6c300693e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500b2520 0x7f6c5014d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b3680 0x7f6c500d08d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 --2- 172.21.15.3:0/1638194540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c500b28f0 0x7f6c500d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 >> 172.21.15.3:0/1638194540 conn(0x7f6c5006e730 msgr2=0x7f6c500b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 shutdown_connections 2023-10-06T05:03:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.686+0000 7f6c4f59e700 1 -- 172.21.15.3:0/1638194540 wait complete. 2023-10-06T05:03:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T05:03:01.699 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T05:03:01.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.821+0000 7fe39979e700 1 Processor -- start 2023-10-06T05:03:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.821+0000 7fe39979e700 1 -- start start 2023-10-06T05:03:01.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe39979e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe394159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe39979e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe3940aff30 0x7fe39415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe39979e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3940b0fd0 con 0x7fe3940d02b0 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe39979e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3940b03c0 con 0x7fe3940aff30 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe39979e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940b06a0 con 0x7fe394159530 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe393fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe394159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe3937fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe3940aff30 0x7fe39415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe393fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe394159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.3:41432/0 (socket says 172.21.15.3:41432) 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe398f9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe3940d02b0 0x7fe3940d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.3:34270/0 (socket says 172.21.15.3:34270) 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.822+0000 7fe393fff700 1 -- 172.21.15.3:0/2247585084 learned_addr learned my addr 172.21.15.3:0/2247585084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe393fff700 1 -- 172.21.15.3:0/2247585084 >> v2:172.21.15.153:3300/0 conn(0x7fe3940aff30 msgr2=0x7fe39415ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe393fff700 1 --2- 172.21.15.3:0/2247585084 >> v2:172.21.15.153:3300/0 conn(0x7fe3940aff30 0x7fe39415ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe393fff700 1 -- 172.21.15.3:0/2247585084 >> v1:172.21.15.153:6789/0 conn(0x7fe3940d02b0 legacy=0x7fe3940d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe393fff700 1 -- 172.21.15.3:0/2247585084 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3940b0980 con 0x7fe394159530 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe3937fe700 1 --2- 172.21.15.3:0/2247585084 >> v2:172.21.15.153:3300/0 conn(0x7fe3940aff30 0x7fe39415ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe393fff700 1 --2- 172.21.15.3:0/2247585084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe394159900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0099b0 tx=0x7fe37c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f6e3f538069ce1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe392ffd700 1 -- 172.21.15.3:0/2247585084 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009ea0 con 0x7fe394159530 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe392ffd700 1 -- 172.21.15.3:0/2247585084 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c00b860 con 0x7fe394159530 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.823+0000 7fe392ffd700 1 -- 172.21.15.3:0/2247585084 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00bc00 con 0x7fe394159530 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 -- 172.21.15.3:0/2247585084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 msgr2=0x7fe394159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 --2- 172.21.15.3:0/2247585084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe394159900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0099b0 tx=0x7fe37c009250 comp rx=0 tx=0).stop 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 -- 172.21.15.3:0/2247585084 shutdown_connections 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 --2- 172.21.15.3:0/2247585084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe394159900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 --2- 172.21.15.3:0/2247585084 >> v2:172.21.15.153:3300/0 conn(0x7fe3940aff30 0x7fe39415ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 -- 172.21.15.3:0/2247585084 >> 172.21.15.3:0/2247585084 conn(0x7fe394068050 msgr2=0x7fe394068450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 -- 172.21.15.3:0/2247585084 shutdown_connections 2023-10-06T05:03:01.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.824+0000 7fe39979e700 1 -- 172.21.15.3:0/2247585084 wait complete. 2023-10-06T05:03:01.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.825+0000 7fe39979e700 1 Processor -- start 2023-10-06T05:03:01.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.825+0000 7fe39979e700 1 -- start start 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe39979e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3940aff30 0x7fe3940b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe39979e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3940d02b0 0x7fe3940b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe39979e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe3940b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe39979e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe394158ed0 con 0x7fe3940d02b0 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe39979e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940c9650 con 0x7fe3940aff30 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe39979e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940c8d40 con 0x7fe394159530 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3940d02b0 0x7fe3940b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3940d02b0 0x7fe3940b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50650/0 (socket says 172.21.15.3:50650) 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 -- 172.21.15.3:0/477795767 learned_addr learned my addr 172.21.15.3:0/477795767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe398f9d700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe3940b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe393fff700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3940aff30 0x7fe3940b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 -- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 msgr2=0x7fe3940b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe394159530 0x7fe3940b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 -- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3940aff30 msgr2=0x7fe3940b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3940aff30 0x7fe3940b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 -- 172.21.15.3:0/477795767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384009420 con 0x7fe3940d02b0 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe393fff700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3940aff30 0x7fe3940b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.826+0000 7fe3937fe700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3940d02b0 0x7fe3940b3ab0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe38400ec80 tx=0x7fe3840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.827+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400c9d0 con 0x7fe3940d02b0 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.827+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c0094f0 con 0x7fe3940d02b0 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.827+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38400cb30 con 0x7fe3940d02b0 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.827+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400ceb0 con 0x7fe3940d02b0 2023-10-06T05:03:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.827+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3940c9000 con 0x7fe3940d02b0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.827+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe39404c380 con 0x7fe3940d02b0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.828+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe384013680 con 0x7fe3940d02b0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.828+0000 7fe38bfff700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe374066f30 0x7fe3740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe384084c50 con 0x7fe3940d02b0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe393fff700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe374066f30 0x7fe3740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe38bfff700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7fe37406d3f0 0x7fe37406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 --> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe37406feb0 con 0x7fe37406d3f0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe384084f40 con 0x7fe3940d02b0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe398f9d700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7fe37406d3f0 0x7fe37406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe398f9d700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7fe37406d3f0 0x7fe37406f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.829+0000 7fe393fff700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe374066f30 0x7fe3740693e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0099b0 tx=0x7fe37c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.830+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== osd.3 v2:172.21.15.3:6824/90685 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe37406feb0 con 0x7fe37406d3f0 2023-10-06T05:03:01.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 --> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe3940cac30 con 0x7fe37406d3f0 2023-10-06T05:03:01.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe38bfff700 1 -- 172.21.15.3:0/477795767 <== osd.3 v2:172.21.15.3:6824/90685 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe3940cac30 con 0x7fe37406d3f0 2023-10-06T05:03:01.846 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:01.846 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:01.846 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7fe37406d3f0 msgr2=0x7fe37406f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:6824/90685,v1:172.21.15.3:6825/90685] conn(0x7fe37406d3f0 0x7fe37406f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe374066f30 msgr2=0x7fe3740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fe374066f30 0x7fe3740693e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0099b0 tx=0x7fe37c017000 comp rx=0 tx=0).stop 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3940d02b0 msgr2=0x7fe3940b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 --2- 172.21.15.3:0/477795767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3940d02b0 0x7fe3940b3ab0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe38400ec80 tx=0x7fe3840091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe398f9d700 1 -- 172.21.15.3:0/477795767 reap_dead start 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 shutdown_connections 2023-10-06T05:03:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 >> 172.21.15.3:0/477795767 conn(0x7fe394068050 msgr2=0x7fe3940ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 shutdown_connections 2023-10-06T05:03:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.844+0000 7fe39979e700 1 -- 172.21.15.3:0/477795767 wait complete. 2023-10-06T05:03:01.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T05:03:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.976+0000 7f6ad7265700 1 Processor -- start 2023-10-06T05:03:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.976+0000 7f6ad7265700 1 -- start start 2023-10-06T05:03:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad7265700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ad00cc230 0x7f6ad00c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad7265700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b07f0 0x7f6ad00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad7265700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00c3eb0 con 0x7f6ad00b0420 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad7265700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad00cd420 con 0x7f6ad00cc230 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad7265700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00b0c20 con 0x7f6ad00b07f0 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad4800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ad00cc230 0x7f6ad00c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad4800700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ad00cc230 0x7f6ad00c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50652/0 (socket says 172.21.15.3:50652) 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.977+0000 7f6ad4800700 1 -- 172.21.15.3:0/699323865 learned_addr learned my addr 172.21.15.3:0/699323865 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ad5802700 1 --2- 172.21.15.3:0/699323865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b07f0 0x7f6ad00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ad4800700 1 -- 172.21.15.3:0/699323865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b07f0 msgr2=0x7f6ad00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ad4800700 1 --2- 172.21.15.3:0/699323865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b07f0 0x7f6ad00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ad4800700 1 -- 172.21.15.3:0/699323865 >> v1:172.21.15.153:6789/0 conn(0x7f6ad00b0420 legacy=0x7f6ad00b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ad4800700 1 -- 172.21.15.3:0/699323865 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00b1990 con 0x7f6ad00cc230 2023-10-06T05:03:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ad4800700 1 --2- 172.21.15.3:0/699323865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00cc230 0x7f6ad00c4a80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009f20 tx=0x7f6acc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f60c9b4f9597b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ac7fff700 1 -- 172.21.15.3:0/699323865 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00e070 con 0x7f6ad00cc230 2023-10-06T05:03:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ac7fff700 1 -- 172.21.15.3:0/699323865 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc00ba80 con 0x7f6ad00cc230 2023-10-06T05:03:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.978+0000 7f6ac7fff700 1 -- 172.21.15.3:0/699323865 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0092e0 con 0x7f6ad00cc230 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 -- 172.21.15.3:0/699323865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00cc230 msgr2=0x7f6ad00c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 --2- 172.21.15.3:0/699323865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00cc230 0x7f6ad00c4a80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009f20 tx=0x7f6acc009840 comp rx=0 tx=0).stop 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 -- 172.21.15.3:0/699323865 shutdown_connections 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 --2- 172.21.15.3:0/699323865 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b07f0 0x7f6ad00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 --2- 172.21.15.3:0/699323865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00cc230 0x7f6ad00c4a80 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 -- 172.21.15.3:0/699323865 >> 172.21.15.3:0/699323865 conn(0x7f6ad00976f0 msgr2=0x7f6ad00677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 -- 172.21.15.3:0/699323865 shutdown_connections 2023-10-06T05:03:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.979+0000 7f6ad7265700 1 -- 172.21.15.3:0/699323865 wait complete. 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.980+0000 7f6ad7265700 1 Processor -- start 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.980+0000 7f6ad7265700 1 -- start start 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad7265700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b0420 0x7f6ad0098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad7265700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b07f0 0x7f6ad00afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad7265700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad00cc230 0x7f6ad015e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad7265700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0098e40 con 0x7f6ad00b07f0 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad7265700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0099120 con 0x7f6ad00cc230 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad7265700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0099400 con 0x7f6ad00b0420 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b07f0 0x7f6ad00afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b07f0 0x7f6ad00afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50654/0 (socket says 172.21.15.3:50654) 2023-10-06T05:03:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 -- 172.21.15.3:0/2239317925 learned_addr learned my addr 172.21.15.3:0/2239317925 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad5802700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad00cc230 0x7f6ad015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad5001700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b0420 0x7f6ad0098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 -- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b0420 msgr2=0x7f6ad0098860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b0420 0x7f6ad0098860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 -- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad00cc230 msgr2=0x7f6ad015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad00cc230 0x7f6ad015e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad4800700 1 -- 172.21.15.3:0/2239317925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc009ae0 con 0x7f6ad00b07f0 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.981+0000 7f6ad5001700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b0420 0x7f6ad0098860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.982+0000 7f6ad4800700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b07f0 0x7f6ad00afca0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009430 tx=0x7f6acc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.982+0000 7f6ac57fa700 1 -- 172.21.15.3:0/2239317925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc008a80 con 0x7f6ad00b07f0 2023-10-06T05:03:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.982+0000 7f6ac57fa700 1 -- 172.21.15.3:0/2239317925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc008be0 con 0x7f6ad00b07f0 2023-10-06T05:03:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.982+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad00996e0 con 0x7f6ad00b07f0 2023-10-06T05:03:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.982+0000 7f6ac57fa700 1 -- 172.21.15.3:0/2239317925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc026460 con 0x7f6ad00b07f0 2023-10-06T05:03:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.982+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad00b01e0 con 0x7f6ad00b07f0 2023-10-06T05:03:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.983+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad0099c00 con 0x7f6ad00b07f0 2023-10-06T05:03:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.983+0000 7f6ac57fa700 1 -- 172.21.15.3:0/2239317925 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6acc003040 con 0x7f6ad00b07f0 2023-10-06T05:03:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.984+0000 7f6ac57fa700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ab4066f30 0x7f6ab40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.984+0000 7f6ac57fa700 1 -- 172.21.15.3:0/2239317925 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6acc086290 con 0x7f6ad00b07f0 2023-10-06T05:03:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.984+0000 7f6ad5001700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ab4066f30 0x7f6ab40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.984+0000 7f6ad5001700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ab4066f30 0x7f6ab40693e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0001850 tx=0x7f6ac0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:01.988+0000 7f6ac57fa700 1 -- 172.21.15.3:0/2239317925 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ad0099c00 con 0x7f6ad00b07f0 2023-10-06T05:03:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.181+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ad0099c00 con 0x7f6ab4066f30 2023-10-06T05:03:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.182+0000 7f6ac57fa700 1 -- 172.21.15.3:0/2239317925 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6ad0099c00 con 0x7f6ab4066f30 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ab4066f30 msgr2=0x7f6ab40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ab4066f30 0x7f6ab40693e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0001850 tx=0x7f6ac0009040 comp rx=0 tx=0).stop 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b07f0 msgr2=0x7f6ad00afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b07f0 0x7f6ad00afca0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009430 tx=0x7f6acc018000 comp rx=0 tx=0).stop 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 shutdown_connections 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f6ab4066f30 0x7f6ab40693e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad00cc230 0x7f6ad015e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ad00b0420 0x7f6ad0098860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 --2- 172.21.15.3:0/2239317925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b07f0 0x7f6ad00afca0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 >> 172.21.15.3:0/2239317925 conn(0x7f6ad00976f0 msgr2=0x7f6ad01590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 shutdown_connections 2023-10-06T05:03:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.184+0000 7f6ad7265700 1 -- 172.21.15.3:0/2239317925 wait complete. 2023-10-06T05:03:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T05:03:02.197 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T05:03:02.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.319+0000 7f885ecab700 1 Processor -- start 2023-10-06T05:03:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.319+0000 7f885ecab700 1 -- start start 2023-10-06T05:03:02.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885ecab700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 0x7f8858159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885ecab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88580d09c0 0x7f885815ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885ecab700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580b0fd0 con 0x7f8858159530 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885ecab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88580b03c0 con 0x7f88580d09c0 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885ecab700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88580b06a0 con 0x7f88580aff30 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885e4aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88580d09c0 0x7f885815ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885e4aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88580d09c0 0x7f885815ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33838/0 (socket says 172.21.15.3:33838) 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885e4aa700 1 -- 172.21.15.3:0/1673467135 learned_addr learned my addr 172.21.15.3:0/1673467135 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.320+0000 7f885d4a8700 1 --2- 172.21.15.3:0/1673467135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 0x7f8858159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885e4aa700 1 -- 172.21.15.3:0/1673467135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 msgr2=0x7f8858159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885e4aa700 1 --2- 172.21.15.3:0/1673467135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 0x7f8858159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885e4aa700 1 -- 172.21.15.3:0/1673467135 >> v1:172.21.15.153:6789/0 conn(0x7f8858159530 legacy=0x7f88580d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885e4aa700 1 -- 172.21.15.3:0/1673467135 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88580b0980 con 0x7f88580d09c0 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885e4aa700 1 --2- 172.21.15.3:0/1673467135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88580d09c0 0x7f885815ddd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f88500099b0 tx=0x7f8850009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80138adf05bb629 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885cca7700 1 -- 172.21.15.3:0/1673467135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850009ea0 con 0x7f88580d09c0 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885cca7700 1 -- 172.21.15.3:0/1673467135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885000b860 con 0x7f88580d09c0 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.321+0000 7f885cca7700 1 -- 172.21.15.3:0/1673467135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885000bc00 con 0x7f88580d09c0 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 -- 172.21.15.3:0/1673467135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88580d09c0 msgr2=0x7f885815ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 --2- 172.21.15.3:0/1673467135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88580d09c0 0x7f885815ddd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f88500099b0 tx=0x7f8850009250 comp rx=0 tx=0).stop 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 -- 172.21.15.3:0/1673467135 shutdown_connections 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 --2- 172.21.15.3:0/1673467135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88580d09c0 0x7f885815ddd0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 --2- 172.21.15.3:0/1673467135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 0x7f8858159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 -- 172.21.15.3:0/1673467135 >> 172.21.15.3:0/1673467135 conn(0x7f8858068050 msgr2=0x7f8858068450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 -- 172.21.15.3:0/1673467135 shutdown_connections 2023-10-06T05:03:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.322+0000 7f885ecab700 1 -- 172.21.15.3:0/1673467135 wait complete. 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.323+0000 7f885ecab700 1 Processor -- start 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.323+0000 7f885ecab700 1 -- start start 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.323+0000 7f885ecab700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 0x7f88581582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.323+0000 7f885ecab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88580d09c0 0x7f8858157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885ecab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858159530 0x7f8858169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885ecab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885815c440 con 0x7f8858159530 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885ecab700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88580b4290 con 0x7f88580d09c0 2023-10-06T05:03:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885ecab700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885815b900 con 0x7f88580aff30 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858159530 0x7f8858169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858159530 0x7f8858169a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33848/0 (socket says 172.21.15.3:33848) 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 -- 172.21.15.3:0/3082147809 learned_addr learned my addr 172.21.15.3:0/3082147809 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885d4a8700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88580d09c0 0x7f8858157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885dca9700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 0x7f88581582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 -- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 msgr2=0x7f88581582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f88580aff30 0x7f88581582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 -- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88580d09c0 msgr2=0x7f8858157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88580d09c0 0x7f8858157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 -- 172.21.15.3:0/3082147809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500094f0 con 0x7f8858159530 2023-10-06T05:03:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885d4a8700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88580d09c0 0x7f8858157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885e4aa700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858159530 0x7f8858169a10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f8858067a80 tx=0x7f8850017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850011630 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8850011790 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850011b30 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885815bba0 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.324+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885815c160 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.325+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f885804c380 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.326+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f885002c3b0 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.326+0000 7f8855ffb700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8844066f30 0x7f88440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.326+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8850085b40 con 0x7f8858159530 2023-10-06T05:03:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.326+0000 7f885dca9700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8844066f30 0x7f88440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.326+0000 7f8855ffb700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f884406d3f0 0x7f884406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.326+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 --> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f884406feb0 con 0x7f884406d3f0 2023-10-06T05:03:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.326+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f885008a240 con 0x7f8858159530 2023-10-06T05:03:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.327+0000 7f885d4a8700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f884406d3f0 0x7f884406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.327+0000 7f885dca9700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8844066f30 0x7f88440693e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8848007fd0 tx=0x7f8848009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.327+0000 7f885d4a8700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f884406d3f0 0x7f884406f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.328+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== osd.4 v2:172.21.15.153:6817/90907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f884406feb0 con 0x7f884406d3f0 2023-10-06T05:03:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.342+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 --> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f88580afd10 con 0x7f884406d3f0 2023-10-06T05:03:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.342+0000 7f8855ffb700 1 -- 172.21.15.3:0/3082147809 <== osd.4 v2:172.21.15.153:6817/90907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f88580afd10 con 0x7f884406d3f0 2023-10-06T05:03:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:02.344 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f884406d3f0 msgr2=0x7f884406f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6817/90907,v1:172.21.15.153:6819/90907] conn(0x7f884406d3f0 0x7f884406f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8844066f30 msgr2=0x7f88440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f8844066f30 0x7f88440693e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8848007fd0 tx=0x7f8848009040 comp rx=0 tx=0).stop 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858159530 msgr2=0x7f8858169a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 --2- 172.21.15.3:0/3082147809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858159530 0x7f8858169a10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f8858067a80 tx=0x7f8850017000 comp rx=0 tx=0).stop 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885e4aa700 1 -- 172.21.15.3:0/3082147809 reap_dead start 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 shutdown_connections 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 >> 172.21.15.3:0/3082147809 conn(0x7f8858068050 msgr2=0x7f885806e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 shutdown_connections 2023-10-06T05:03:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.343+0000 7f885ecab700 1 -- 172.21.15.3:0/3082147809 wait complete. 2023-10-06T05:03:02.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T05:03:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.474+0000 7f357aae0700 1 Processor -- start 2023-10-06T05:03:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.475+0000 7f357aae0700 1 -- start start 2023-10-06T05:03:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.475+0000 7f357aae0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f357409d980 0x7f357415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.475+0000 7f357aae0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35740c6da0 0x7f35740cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.475+0000 7f357aae0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740cd230 con 0x7f35740cd6c0 2023-10-06T05:03:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.475+0000 7f357aae0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357415e4c0 con 0x7f35740c6da0 2023-10-06T05:03:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.475+0000 7f357aae0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740cc660 con 0x7f357409d980 2023-10-06T05:03:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35740c6da0 0x7f35740cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35740c6da0 0x7f35740cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33862/0 (socket says 172.21.15.3:33862) 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 -- 172.21.15.3:0/1271679936 learned_addr learned my addr 172.21.15.3:0/1271679936 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357887c700 1 --2- 172.21.15.3:0/1271679936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f357409d980 0x7f357415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 -- 172.21.15.3:0/1271679936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f357409d980 msgr2=0x7f357415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 --2- 172.21.15.3:0/1271679936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f357409d980 0x7f357415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 -- 172.21.15.3:0/1271679936 >> v1:172.21.15.153:6789/0 conn(0x7f35740cd6c0 legacy=0x7f35740692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 -- 172.21.15.3:0/1271679936 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740cc900 con 0x7f35740c6da0 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357887c700 1 --2- 172.21.15.3:0/1271679936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f357409d980 0x7f357415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f357907d700 1 --2- 172.21.15.3:0/1271679936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740c6da0 0x7f35740cab70 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f356c009f20 tx=0x7f356c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e3b784babe2eae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f35737fe700 1 -- 172.21.15.3:0/1271679936 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00e070 con 0x7f35740c6da0 2023-10-06T05:03:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f35737fe700 1 -- 172.21.15.3:0/1271679936 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c00ba80 con 0x7f35740c6da0 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.476+0000 7f35737fe700 1 -- 172.21.15.3:0/1271679936 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c0092e0 con 0x7f35740c6da0 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 -- 172.21.15.3:0/1271679936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740c6da0 msgr2=0x7f35740cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 --2- 172.21.15.3:0/1271679936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740c6da0 0x7f35740cab70 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f356c009f20 tx=0x7f356c009840 comp rx=0 tx=0).stop 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 -- 172.21.15.3:0/1271679936 shutdown_connections 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 --2- 172.21.15.3:0/1271679936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740c6da0 0x7f35740cab70 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 --2- 172.21.15.3:0/1271679936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f357409d980 0x7f357415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 -- 172.21.15.3:0/1271679936 >> 172.21.15.3:0/1271679936 conn(0x7f357406e730 msgr2=0x7f357406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 -- 172.21.15.3:0/1271679936 shutdown_connections 2023-10-06T05:03:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.477+0000 7f357aae0700 1 -- 172.21.15.3:0/1271679936 wait complete. 2023-10-06T05:03:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.478+0000 7f357aae0700 1 Processor -- start 2023-10-06T05:03:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.478+0000 7f357aae0700 1 -- start start 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357aae0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f357409d980 0x7f35740c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357aae0700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35740c6da0 0x7f3574145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357aae0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740cd6c0 0x7f3574157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357aae0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c9de0 con 0x7f35740cd6c0 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357aae0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c7880 con 0x7f357409d980 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357aae0700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35741456c0 con 0x7f35740c6da0 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740cd6c0 0x7f3574157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740cd6c0 0x7f3574157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33872/0 (socket says 172.21.15.3:33872) 2023-10-06T05:03:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 -- 172.21.15.3:0/4033312851 learned_addr learned my addr 172.21.15.3:0/4033312851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357887c700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f357409d980 0x7f35740c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f3573fff700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35740c6da0 0x7f3574145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 -- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35740c6da0 msgr2=0x7f3574145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35740c6da0 0x7f3574145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 -- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f357409d980 msgr2=0x7f35740c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f357409d980 0x7f35740c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 -- 172.21.15.3:0/4033312851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c009ae0 con 0x7f35740cd6c0 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.479+0000 7f357907d700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740cd6c0 0x7f3574157660 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f356c00b6c0 tx=0x7f356c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.480+0000 7f3570ff9700 1 -- 172.21.15.3:0/4033312851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c0032f0 con 0x7f35740cd6c0 2023-10-06T05:03:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.480+0000 7f3570ff9700 1 -- 172.21.15.3:0/4033312851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c008d50 con 0x7f35740cd6c0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.480+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35740ca000 con 0x7f35740cd6c0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.480+0000 7f3570ff9700 1 -- 172.21.15.3:0/4033312851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c026690 con 0x7f35740cd6c0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.480+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357415a060 con 0x7f35740cd6c0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.481+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740c6580 con 0x7f35740cd6c0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.481+0000 7f3570ff9700 1 -- 172.21.15.3:0/4033312851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f356c003040 con 0x7f35740cd6c0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.481+0000 7f3570ff9700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3560066f30 0x7f35600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.481+0000 7f3570ff9700 1 -- 172.21.15.3:0/4033312851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f356c0862c0 con 0x7f35740cd6c0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.481+0000 7f357887c700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3560066f30 0x7f35600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.482+0000 7f357887c700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3560066f30 0x7f35600693e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3564006ea0 tx=0x7f3564008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.486+0000 7f3570ff9700 1 -- 172.21.15.3:0/4033312851 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f356c0536a0 con 0x7f35740cd6c0 2023-10-06T05:03:02.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.679+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f357404c380 con 0x7f3560066f30 2023-10-06T05:03:02.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.680+0000 7f3570ff9700 1 -- 172.21.15.3:0/4033312851 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f357404c380 con 0x7f3560066f30 2023-10-06T05:03:02.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3560066f30 msgr2=0x7f35600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3560066f30 0x7f35600693e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3564006ea0 tx=0x7f3564008040 comp rx=0 tx=0).stop 2023-10-06T05:03:02.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740cd6c0 msgr2=0x7f3574157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740cd6c0 0x7f3574157660 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f356c00b6c0 tx=0x7f356c018000 comp rx=0 tx=0).stop 2023-10-06T05:03:02.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 shutdown_connections 2023-10-06T05:03:02.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3560066f30 0x7f35600693e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f357409d980 0x7f35740c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35740cd6c0 0x7f3574157660 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 --2- 172.21.15.3:0/4033312851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35740c6da0 0x7f3574145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 >> 172.21.15.3:0/4033312851 conn(0x7f357406e730 msgr2=0x7f35740c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 shutdown_connections 2023-10-06T05:03:02.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.682+0000 7f357aae0700 1 -- 172.21.15.3:0/4033312851 wait complete. 2023-10-06T05:03:02.686 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T05:03:02.695 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T05:03:02.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:02.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.818+0000 7f3a86724700 1 Processor -- start 2023-10-06T05:03:02.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.818+0000 7f3a86724700 1 -- start start 2023-10-06T05:03:02.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a86724700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a8015db70 0x7f3a80067d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a86724700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a800aa8c0 0x7f3a800aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a86724700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a800cc550 con 0x7f3a800b13c0 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a86724700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a800b51c0 con 0x7f3a8015db70 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a86724700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a80068340 con 0x7f3a800aa8c0 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a7ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a8015db70 0x7f3a80067d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a7f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3a800b13c0 0x7f3a8015df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45878/0 (socket says 172.21.15.3:45878) 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.819+0000 7f3a7f7fe700 1 -- 172.21.15.3:0/1092540122 learned_addr learned my addr 172.21.15.3:0/1092540122 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a84cc1700 1 --2- 172.21.15.3:0/1092540122 >> v2:172.21.15.153:3300/0 conn(0x7f3a800aa8c0 0x7f3a800aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7effd700 1 -- 172.21.15.3:0/1092540122 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110948577 0 0) 0x7f3a800cc550 con 0x7f3a800b13c0 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7effd700 1 -- 172.21.15.3:0/1092540122 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a6c003570 con 0x7f3a800b13c0 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7ffff700 1 -- 172.21.15.3:0/1092540122 >> v2:172.21.15.153:3300/0 conn(0x7f3a800aa8c0 msgr2=0x7f3a800aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7ffff700 1 --2- 172.21.15.3:0/1092540122 >> v2:172.21.15.153:3300/0 conn(0x7f3a800aa8c0 0x7f3a800aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7ffff700 1 -- 172.21.15.3:0/1092540122 >> v1:172.21.15.3:6789/0 conn(0x7f3a800b13c0 legacy=0x7f3a8015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7ffff700 1 -- 172.21.15.3:0/1092540122 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80068620 con 0x7f3a8015db70 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a84cc1700 1 --2- 172.21.15.3:0/1092540122 >> v2:172.21.15.153:3300/0 conn(0x7f3a800aa8c0 0x7f3a800aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7ffff700 1 --2- 172.21.15.3:0/1092540122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8015db70 0x7f3a80067d60 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f3a80069640 tx=0x7f3a700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8776f0c706128e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7effd700 1 -- 172.21.15.3:0/1092540122 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70011630 con 0x7f3a8015db70 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7effd700 1 -- 172.21.15.3:0/1092540122 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a70011790 con 0x7f3a8015db70 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.820+0000 7f3a7effd700 1 -- 172.21.15.3:0/1092540122 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70011b30 con 0x7f3a8015db70 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 -- 172.21.15.3:0/1092540122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8015db70 msgr2=0x7f3a80067d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 --2- 172.21.15.3:0/1092540122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8015db70 0x7f3a80067d60 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f3a80069640 tx=0x7f3a700091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 -- 172.21.15.3:0/1092540122 shutdown_connections 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 --2- 172.21.15.3:0/1092540122 >> v2:172.21.15.153:3300/0 conn(0x7f3a800aa8c0 0x7f3a800aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 --2- 172.21.15.3:0/1092540122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a8015db70 0x7f3a80067d60 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 -- 172.21.15.3:0/1092540122 >> 172.21.15.3:0/1092540122 conn(0x7f3a8006e730 msgr2=0x7f3a8006eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 -- 172.21.15.3:0/1092540122 shutdown_connections 2023-10-06T05:03:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.821+0000 7f3a86724700 1 -- 172.21.15.3:0/1092540122 wait complete. 2023-10-06T05:03:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.822+0000 7f3a86724700 1 Processor -- start 2023-10-06T05:03:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.822+0000 7f3a86724700 1 -- start start 2023-10-06T05:03:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a800aa8c0 0x7f3a80157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a800b13c0 0x7f3a80151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a8015db70 0x7f3a800ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a80155fb0 con 0x7f3a800b13c0 2023-10-06T05:03:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801583e0 con 0x7f3a800aa8c0 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801586c0 con 0x7f3a8015db70 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a800b13c0 0x7f3a80151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a800b13c0 0x7f3a80151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33890/0 (socket says 172.21.15.3:33890) 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 -- 172.21.15.3:0/3127058652 learned_addr learned my addr 172.21.15.3:0/3127058652 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7ffff700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a800aa8c0 0x7f3a80157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 -- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a8015db70 msgr2=0x7f3a800ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a8015db70 0x7f3a800ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 -- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a800aa8c0 msgr2=0x7f3a80157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a800aa8c0 0x7f3a80157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 -- 172.21.15.3:0/3127058652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a70009420 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a7f7fe700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a800b13c0 0x7f3a80151b80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c7c0 tx=0x7f3a68007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68005c00 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a68005d60 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a801589a0 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68007320 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.823+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a801580f0 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.824+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a8004c380 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.825+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a68007690 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.825+0000 7f3a77fff700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3a60066f60 0x7f3a60069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.825+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3a680848c0 con 0x7f3a800b13c0 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.825+0000 7f3a77fff700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f3a6006d420 0x7f3a6006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.825+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 --> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a6006fee0 con 0x7f3a6006d420 2023-10-06T05:03:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.825+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a68084c90 con 0x7f3a800b13c0 2023-10-06T05:03:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.825+0000 7f3a7ffff700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3a60066f60 0x7f3a60069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.826+0000 7f3a84cc1700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f3a6006d420 0x7f3a6006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.826+0000 7f3a7ffff700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3a60066f60 0x7f3a60069410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3a70003080 tx=0x7f3a70017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.826+0000 7f3a84cc1700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f3a6006d420 0x7f3a6006f830 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.827+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== osd.5 v2:172.21.15.153:6800/90897 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3a6006fee0 con 0x7f3a6006d420 2023-10-06T05:03:02.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.841+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 --> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3a800b4710 con 0x7f3a6006d420 2023-10-06T05:03:02.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.841+0000 7f3a77fff700 1 -- 172.21.15.3:0/3127058652 <== osd.5 v2:172.21.15.153:6800/90897 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3a800b4710 con 0x7f3a6006d420 2023-10-06T05:03:02.843 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:02.843 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:02.843 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:02.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.841+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f3a6006d420 msgr2=0x7f3a6006f830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.841+0000 7f3a86724700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6800/90897,v1:172.21.15.153:6801/90897] conn(0x7f3a6006d420 0x7f3a6006f830 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3a60066f60 msgr2=0x7f3a60069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3a60066f60 0x7f3a60069410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3a70003080 tx=0x7f3a70017000 comp rx=0 tx=0).stop 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a800b13c0 msgr2=0x7f3a80151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 --2- 172.21.15.3:0/3127058652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a800b13c0 0x7f3a80151b80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c7c0 tx=0x7f3a68007dd0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a84cc1700 1 -- 172.21.15.3:0/3127058652 reap_dead start 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 shutdown_connections 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 >> 172.21.15.3:0/3127058652 conn(0x7f3a8006e730 msgr2=0x7f3a8015b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 shutdown_connections 2023-10-06T05:03:02.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.842+0000 7f3a86724700 1 -- 172.21.15.3:0/3127058652 wait complete. 2023-10-06T05:03:02.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T05:03:02.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.974+0000 7f600e372700 1 Processor -- start 2023-10-06T05:03:02.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.974+0000 7f600e372700 1 -- start start 2023-10-06T05:03:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f600e372700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806d720 0x7f600815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f600e372700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60080c6ba0 0x7f600815b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f600e372700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60080c7c20 con 0x7f600806daf0 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f600e372700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080c7010 con 0x7f60080c6ba0 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f600e372700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60080c72f0 con 0x7f600806d720 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f60077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60080c6ba0 0x7f600815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f60077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60080c6ba0 0x7f600815b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33900/0 (socket says 172.21.15.3:33900) 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.975+0000 7f60077fe700 1 -- 172.21.15.3:0/2057242593 learned_addr learned my addr 172.21.15.3:0/2057242593 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:02.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f6007fff700 1 --2- 172.21.15.3:0/2057242593 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806d720 0x7f600815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f60077fe700 1 -- 172.21.15.3:0/2057242593 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806d720 msgr2=0x7f600815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f60077fe700 1 --2- 172.21.15.3:0/2057242593 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806d720 0x7f600815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f60077fe700 1 -- 172.21.15.3:0/2057242593 >> v1:172.21.15.153:6789/0 conn(0x7f600806daf0 legacy=0x7f60081579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f60077fe700 1 -- 172.21.15.3:0/2057242593 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080c75d0 con 0x7f60080c6ba0 2023-10-06T05:03:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f60077fe700 1 --2- 172.21.15.3:0/2057242593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f600815b520 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00099b0 tx=0x7f5ff0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3b8af25751a5c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f6006ffd700 1 -- 172.21.15.3:0/2057242593 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000b860 con 0x7f60080c6ba0 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f6006ffd700 1 -- 172.21.15.3:0/2057242593 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff000b9c0 con 0x7f60080c6ba0 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f6006ffd700 1 -- 172.21.15.3:0/2057242593 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000bd60 con 0x7f60080c6ba0 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f600e372700 1 -- 172.21.15.3:0/2057242593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 msgr2=0x7f600815b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.976+0000 7f600e372700 1 --2- 172.21.15.3:0/2057242593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f600815b520 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00099b0 tx=0x7f5ff0009250 comp rx=0 tx=0).stop 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.977+0000 7f600e372700 1 -- 172.21.15.3:0/2057242593 shutdown_connections 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.977+0000 7f600e372700 1 --2- 172.21.15.3:0/2057242593 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806d720 0x7f600815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.977+0000 7f600e372700 1 --2- 172.21.15.3:0/2057242593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f600815b520 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.977+0000 7f600e372700 1 -- 172.21.15.3:0/2057242593 >> 172.21.15.3:0/2057242593 conn(0x7f6008067aa0 msgr2=0x7f60080cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.977+0000 7f600e372700 1 -- 172.21.15.3:0/2057242593 shutdown_connections 2023-10-06T05:03:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.977+0000 7f600e372700 1 -- 172.21.15.3:0/2057242593 wait complete. 2023-10-06T05:03:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.978+0000 7f600e372700 1 Processor -- start 2023-10-06T05:03:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.978+0000 7f600e372700 1 -- start start 2023-10-06T05:03:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.978+0000 7f600e372700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f600806d720 0x7f6008153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.978+0000 7f600e372700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806daf0 0x7f6008153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600e372700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f6008155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600e372700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008154fe0 con 0x7f60080c6ba0 2023-10-06T05:03:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600e372700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008150730 con 0x7f600806d720 2023-10-06T05:03:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600e372700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6008152090 con 0x7f600806daf0 2023-10-06T05:03:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f6008155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f6008155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33916/0 (socket says 172.21.15.3:33916) 2023-10-06T05:03:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 -- 172.21.15.3:0/3829991293 learned_addr learned my addr 172.21.15.3:0/3829991293 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f60077fe700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806daf0 0x7f6008153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f6007fff700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f600806d720 0x7f6008153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 -- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806daf0 msgr2=0x7f6008153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806daf0 0x7f6008153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 -- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f600806d720 msgr2=0x7f6008153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f600806d720 0x7f6008153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 -- 172.21.15.3:0/3829991293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00094f0 con 0x7f60080c6ba0 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600c90f700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f6008155ff0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f600000c8e0 tx=0x7f6000001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f5ffffff700 1 -- 172.21.15.3:0/3829991293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000ce80 con 0x7f60080c6ba0 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f5ffffff700 1 -- 172.21.15.3:0/3829991293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600000a7e0 con 0x7f60080c6ba0 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6008152370 con 0x7f60080c6ba0 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f5ffffff700 1 -- 172.21.15.3:0/3829991293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000ab80 con 0x7f60080c6ba0 2023-10-06T05:03:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.979+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008152930 con 0x7f60080c6ba0 2023-10-06T05:03:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.980+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600806b270 con 0x7f60080c6ba0 2023-10-06T05:03:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.981+0000 7f5ffffff700 1 -- 172.21.15.3:0/3829991293 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6000012420 con 0x7f60080c6ba0 2023-10-06T05:03:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.981+0000 7f5ffffff700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5fe8066f30 0x7f5fe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.981+0000 7f5ffffff700 1 -- 172.21.15.3:0/3829991293 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f60000846b0 con 0x7f60080c6ba0 2023-10-06T05:03:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.981+0000 7f6007fff700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5fe8066f30 0x7f5fe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:02.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.982+0000 7f6007fff700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5fe8066f30 0x7f5fe80693e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8006fd0 tx=0x7f5ff8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:02.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:02.985+0000 7f5ffffff700 1 -- 172.21.15.3:0/3829991293 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6000051a90 con 0x7f60080c6ba0 2023-10-06T05:03:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.177+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f600804c380 con 0x7f5fe8066f30 2023-10-06T05:03:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.178+0000 7f5ffffff700 1 -- 172.21.15.3:0/3829991293 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f600804c380 con 0x7f5fe8066f30 2023-10-06T05:03:03.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5fe8066f30 msgr2=0x7f5fe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5fe8066f30 0x7f5fe80693e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8006fd0 tx=0x7f5ff8008040 comp rx=0 tx=0).stop 2023-10-06T05:03:03.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 msgr2=0x7f6008155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f6008155ff0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f600000c8e0 tx=0x7f6000001710 comp rx=0 tx=0).stop 2023-10-06T05:03:03.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 shutdown_connections 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5fe8066f30 0x7f5fe80693e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f600806d720 0x7f6008153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60080c6ba0 0x7f6008155ff0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 --2- 172.21.15.3:0/3829991293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f600806daf0 0x7f6008153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 >> 172.21.15.3:0/3829991293 conn(0x7f6008067aa0 msgr2=0x7f60080b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 shutdown_connections 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.180+0000 7f600e372700 1 -- 172.21.15.3:0/3829991293 wait complete. 2023-10-06T05:03:03.183 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T05:03:03.194 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T05:03:03.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.315+0000 7faf29a16700 1 Processor -- start 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.315+0000 7faf29a16700 1 -- start start 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf29a16700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 0x7faf24067d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf29a16700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faf240aa8c0 0x7faf240aad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf29a16700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf240cc550 con 0x7faf2415db70 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf29a16700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf240b51c0 con 0x7faf240aa8c0 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf29a16700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf24068340 con 0x7faf240b13c0 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf22ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faf2415db70 0x7faf2415df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45892/0 (socket says 172.21.15.3:45892) 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf22ffd700 1 -- 172.21.15.3:0/1437168599 learned_addr learned my addr 172.21.15.3:0/1437168599 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081113892 0 0) 0x7faf240cc550 con 0x7faf2415db70 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf10003570 con 0x7faf2415db70 2023-10-06T05:03:03.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.316+0000 7faf237fe700 1 --2- 172.21.15.3:0/1437168599 >> v2:172.21.15.153:3300/0 conn(0x7faf240aa8c0 0x7faf240aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf227fc700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 0x7faf24067d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2098214362 0 0) 0x7faf10003570 con 0x7faf2415db70 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 msgr2=0x7faf24067d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 0x7faf24067d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 >> v2:172.21.15.153:3300/0 conn(0x7faf240aa8c0 msgr2=0x7faf240aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 --2- 172.21.15.3:0/1437168599 >> v2:172.21.15.153:3300/0 conn(0x7faf240aa8c0 0x7faf240aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24068620 con 0x7faf2415db70 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7faf14001470 con 0x7faf2415db70 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 --2- 172.21.15.3:0/1437168599 >> v2:172.21.15.153:3300/0 conn(0x7faf240aa8c0 0x7faf240aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:03.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 >> v1:172.21.15.3:6789/0 conn(0x7faf2415db70 legacy=0x7faf2415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 0x7faf10003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004090 0x7faf10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf10004d50 0x7faf100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240b51c0 con 0x7faf10004d50 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf24068340 con 0x7faf10004090 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1000bbf0 con 0x7faf240b13c0 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf10004d50 0x7faf100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf227fc700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004090 0x7faf10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf22ffd700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 0x7faf10003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 -- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 msgr2=0x7faf10003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b13c0 0x7faf10003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 -- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004090 msgr2=0x7faf10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004090 0x7faf10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 -- 172.21.15.3:0/1437168599 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf10003570 con 0x7faf10004d50 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.317+0000 7faf237fe700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf10004d50 0x7faf100051c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007580 tx=0x7faf1c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf645cbbae9064f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c009ae0 con 0x7faf10004d50 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c007dd0 con 0x7faf10004d50 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf21ffb700 1 -- 172.21.15.3:0/1437168599 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c014610 con 0x7faf10004d50 2023-10-06T05:03:03.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf29a16700 1 -- 172.21.15.3:0/1437168599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf10004d50 msgr2=0x7faf100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf29a16700 1 --2- 172.21.15.3:0/1437168599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf10004d50 0x7faf100051c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007580 tx=0x7faf1c0097d0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf237fe700 1 -- 172.21.15.3:0/1437168599 reap_dead start 2023-10-06T05:03:03.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf29a16700 1 -- 172.21.15.3:0/1437168599 shutdown_connections 2023-10-06T05:03:03.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf29a16700 1 -- 172.21.15.3:0/1437168599 >> 172.21.15.3:0/1437168599 conn(0x7faf2406e730 msgr2=0x7faf2406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf29a16700 1 -- 172.21.15.3:0/1437168599 shutdown_connections 2023-10-06T05:03:03.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.318+0000 7faf29a16700 1 -- 172.21.15.3:0/1437168599 wait complete. 2023-10-06T05:03:03.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.319+0000 7faf29a16700 1 Processor -- start 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.319+0000 7faf29a16700 1 -- start start 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf29a16700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004d50 0x7faf24158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf29a16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf240aef40 0x7faf240d0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf29a16700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240af3b0 0x7faf240d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf29a16700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240adf40 con 0x7faf240aef40 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf29a16700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240aeab0 con 0x7faf10004d50 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf29a16700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240adc60 con 0x7faf240af3b0 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf227fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf240aef40 0x7faf240d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf227fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf240aef40 0x7faf240d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33922/0 (socket says 172.21.15.3:33922) 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf227fc700 1 -- 172.21.15.3:0/3688313910 learned_addr learned my addr 172.21.15.3:0/3688313910 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:03.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf22ffd700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004d50 0x7faf24158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.320+0000 7faf237fe700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240af3b0 0x7faf240d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf227fc700 1 -- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240af3b0 msgr2=0x7faf240d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf227fc700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240af3b0 0x7faf240d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf227fc700 1 -- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004d50 msgr2=0x7faf24158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf227fc700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf10004d50 0x7faf24158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf227fc700 1 -- 172.21.15.3:0/3688313910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240ae460 con 0x7faf240aef40 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf227fc700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf240aef40 0x7faf240d0610 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0109e0 tx=0x7faf0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00e6f0 con 0x7faf240aef40 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c00e850 con 0x7faf240aef40 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00ebf0 con 0x7faf240aef40 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf240ae740 con 0x7faf240aef40 2023-10-06T05:03:03.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.321+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf240acf30 con 0x7faf240aef40 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.322+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf2404c380 con 0x7faf240aef40 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.322+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf0c016480 con 0x7faf240aef40 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf1affd700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf04066f30 0x7faf040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7faf0c084f40 con 0x7faf240aef40 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf1affd700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7faf0406d3f0 0x7faf0406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 --> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf0406feb0 con 0x7faf0406d3f0 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf0c087520 con 0x7faf240aef40 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf22ffd700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf04066f30 0x7faf040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf237fe700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7faf0406d3f0 0x7faf0406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf22ffd700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf04066f30 0x7faf040693e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faf140079e0 tx=0x7faf2415a3a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.323+0000 7faf237fe700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7faf0406d3f0 0x7faf0406f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.324+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== osd.6 v2:172.21.15.153:6808/90898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7faf0406feb0 con 0x7faf0406d3f0 2023-10-06T05:03:03.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.338+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 --> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faf240b4710 con 0x7faf0406d3f0 2023-10-06T05:03:03.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.338+0000 7faf1affd700 1 -- 172.21.15.3:0/3688313910 <== osd.6 v2:172.21.15.153:6808/90898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faf240b4710 con 0x7faf0406d3f0 2023-10-06T05:03:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:03.340 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7faf0406d3f0 msgr2=0x7faf0406f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6808/90898,v1:172.21.15.153:6809/90898] conn(0x7faf0406d3f0 0x7faf0406f800 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf04066f30 msgr2=0x7faf040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7faf04066f30 0x7faf040693e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faf140079e0 tx=0x7faf2415a3a0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf240aef40 msgr2=0x7faf240d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 --2- 172.21.15.3:0/3688313910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf240aef40 0x7faf240d0610 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0109e0 tx=0x7faf0c008040 comp rx=0 tx=0).stop 2023-10-06T05:03:03.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf237fe700 1 -- 172.21.15.3:0/3688313910 reap_dead start 2023-10-06T05:03:03.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 shutdown_connections 2023-10-06T05:03:03.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 >> 172.21.15.3:0/3688313910 conn(0x7faf2406e730 msgr2=0x7faf2415c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 shutdown_connections 2023-10-06T05:03:03.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.339+0000 7faf29a16700 1 -- 172.21.15.3:0/3688313910 wait complete. 2023-10-06T05:03:03.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T05:03:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.470+0000 7fb93f59e700 1 Processor -- start 2023-10-06T05:03:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.470+0000 7fb93f59e700 1 -- start start 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93f59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb9400cd510 0x7fb9400cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400cc970 con 0x7fb9400b2520 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93f59e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400cbd40 con 0x7fb9400b28f0 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93f59e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9400cc3f0 con 0x7fb9400cd510 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb9400b2520 0x7fb9400b3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45894/0 (socket says 172.21.15.3:45894) 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93e59c700 1 -- 172.21.15.3:0/671439872 learned_addr learned my addr 172.21.15.3:0/671439872 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167219451 0 0) 0x7fb9400cc970 con 0x7fb9400b2520 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93ed9d700 1 --2- 172.21.15.3:0/671439872 >> v2:172.21.15.153:3300/0 conn(0x7fb9400cd510 0x7fb9400cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb92c003570 con 0x7fb9400b2520 2023-10-06T05:03:03.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26275428 0 0) 0x7fb92c003570 con 0x7fb9400b2520 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 >> v2:172.21.15.153:3300/0 conn(0x7fb9400cd510 msgr2=0x7fb9400cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.471+0000 7fb93d59a700 1 --2- 172.21.15.3:0/671439872 >> v2:172.21.15.153:3300/0 conn(0x7fb9400cd510 0x7fb9400cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 >> v1:172.21.15.153:6789/0 conn(0x7fb9400b28f0 legacy=0x7fb9400cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400cc020 con 0x7fb9400b2520 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7fb938001470 con 0x7fb9400b2520 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 >> v1:172.21.15.3:6789/0 conn(0x7fb9400b2520 legacy=0x7fb9400b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c003a60 0x7fb92c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb92c004810 0x7fb92c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb92c0051c0 0x7fb92c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400cc3f0 con 0x7fb92c004810 2023-10-06T05:03:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c00c160 con 0x7fb92c0051c0 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c00c320 con 0x7fb92c003a60 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93dd9b700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb92c004810 0x7fb92c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93ed9d700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb92c0051c0 0x7fb92c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93e59c700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c003a60 0x7fb92c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93dd9b700 1 -- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c003a60 msgr2=0x7fb92c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93dd9b700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c003a60 0x7fb92c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93dd9b700 1 -- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb92c0051c0 msgr2=0x7fb92c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93dd9b700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb92c0051c0 0x7fb92c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93ed9d700 1 -- 172.21.15.3:0/671439872 reap_dead start 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93dd9b700 1 -- 172.21.15.3:0/671439872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c003570 con 0x7fb92c004810 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93dd9b700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb92c004810 0x7fb92c004c80 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb9280018b0 tx=0x7fb928007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acb1e020f1305e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928012420 con 0x7fb92c004810 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9280125a0 con 0x7fb92c004810 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.472+0000 7fb93d59a700 1 -- 172.21.15.3:0/671439872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928012960 con 0x7fb92c004810 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.473+0000 7fb93f59e700 1 -- 172.21.15.3:0/671439872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb92c004810 msgr2=0x7fb92c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.473+0000 7fb93f59e700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb92c004810 0x7fb92c004c80 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb9280018b0 tx=0x7fb928007d30 comp rx=0 tx=0).stop 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.473+0000 7fb93f59e700 1 -- 172.21.15.3:0/671439872 shutdown_connections 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.473+0000 7fb93f59e700 1 --2- 172.21.15.3:0/671439872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb92c004810 0x7fb92c004c80 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.473+0000 7fb93f59e700 1 -- 172.21.15.3:0/671439872 >> 172.21.15.3:0/671439872 conn(0x7fb94006e730 msgr2=0x7fb94006eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.473+0000 7fb93f59e700 1 -- 172.21.15.3:0/671439872 shutdown_connections 2023-10-06T05:03:03.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.473+0000 7fb93f59e700 1 -- 172.21.15.3:0/671439872 wait complete. 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.474+0000 7fb93f59e700 1 Processor -- start 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.474+0000 7fb93f59e700 1 -- start start 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93f59e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c004810 0x7fb94016a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9401587d0 0x7fb9401512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9400c44a0 0x7fb9400c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94016a320 con 0x7fb9401587d0 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400c5160 con 0x7fb9400c44a0 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93f59e700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400c4e80 con 0x7fb92c004810 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9401587d0 0x7fb9401512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9401587d0 0x7fb9401512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33940/0 (socket says 172.21.15.3:33940) 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 -- 172.21.15.3:0/1181837353 learned_addr learned my addr 172.21.15.3:0/1181837353 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93ed9d700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9400c44a0 0x7fb9400c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93e59c700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c004810 0x7fb94016a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 -- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c004810 msgr2=0x7fb94016a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c004810 0x7fb94016a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 -- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9400c44a0 msgr2=0x7fb9400c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9400c44a0 0x7fb9400c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 -- 172.21.15.3:0/1181837353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb940169cd0 con 0x7fb9401587d0 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93e59c700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c004810 0x7fb94016a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.475+0000 7fb93dd9b700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9401587d0 0x7fb9401512e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c130 tx=0x7fb928015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.476+0000 7fb9367fc700 1 -- 172.21.15.3:0/1181837353 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92801f6f0 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.476+0000 7fb9367fc700 1 -- 172.21.15.3:0/1181837353 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92801f850 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.476+0000 7fb9367fc700 1 -- 172.21.15.3:0/1181837353 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92801fbf0 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.476+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb940169fb0 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.476+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9400c4b40 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.476+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94016a4d0 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.477+0000 7fb9367fc700 1 -- 172.21.15.3:0/1181837353 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb92800c420 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.477+0000 7fb9367fc700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb920066f30 0x7fb9200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.477+0000 7fb9367fc700 1 -- 172.21.15.3:0/1181837353 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb928085b10 con 0x7fb9401587d0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.477+0000 7fb93e59c700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb920066f30 0x7fb9200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.478+0000 7fb93e59c700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb920066f30 0x7fb9200693e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb9380015a0 tx=0x7fb938009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.481+0000 7fb9367fc700 1 -- 172.21.15.3:0/1181837353 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb94016a4d0 con 0x7fb9401587d0 2023-10-06T05:03:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.675+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fb94004c380 con 0x7fb920066f30 2023-10-06T05:03:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.676+0000 7fb9367fc700 1 -- 172.21.15.3:0/1181837353 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb94004c380 con 0x7fb920066f30 2023-10-06T05:03:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb920066f30 msgr2=0x7fb9200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb920066f30 0x7fb9200693e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb9380015a0 tx=0x7fb938009040 comp rx=0 tx=0).stop 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9401587d0 msgr2=0x7fb9401512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9401587d0 0x7fb9401512e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c130 tx=0x7fb928015000 comp rx=0 tx=0).stop 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 shutdown_connections 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fb920066f30 0x7fb9200693e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9400c44a0 0x7fb9400c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb92c004810 0x7fb94016a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 --2- 172.21.15.3:0/1181837353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9401587d0 0x7fb9401512e0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 >> 172.21.15.3:0/1181837353 conn(0x7fb94006e730 msgr2=0x7fb9400c8660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 shutdown_connections 2023-10-06T05:03:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.678+0000 7fb93f59e700 1 -- 172.21.15.3:0/1181837353 wait complete. 2023-10-06T05:03:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T05:03:03.691 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T05:03:03.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:03:03.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.813+0000 7f36253c1700 1 Processor -- start 2023-10-06T05:03:03.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.813+0000 7f36253c1700 1 -- start start 2023-10-06T05:03:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.814+0000 7f36253c1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 0x7f36200b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.814+0000 7f36253c1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3620068320 con 0x7f3620067d60 2023-10-06T05:03:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.814+0000 7f36253c1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3620068600 con 0x7f3620069d90 2023-10-06T05:03:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.814+0000 7f36253c1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200688e0 con 0x7f36200aae50 2023-10-06T05:03:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.814+0000 7f361e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3620067d60 0x7f3620069680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45896/0 (socket says 172.21.15.3:45896) 2023-10-06T05:03:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.814+0000 7f361e7fc700 1 -- 172.21.15.3:0/3682915352 learned_addr learned my addr 172.21.15.3:0/3682915352 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489521308 0 0) 0x7f3620068320 con 0x7f3620067d60 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361effd700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 0x7f36200b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f360c003570 con 0x7f3620067d60 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516875918 0 0) 0x7f360c003570 con 0x7f3620067d60 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 msgr2=0x7f36200b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 0x7f36200b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 >> v1:172.21.15.153:6789/0 conn(0x7f3620069d90 legacy=0x7f36200cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620068bc0 con 0x7f3620067d60 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f3608001470 con 0x7f3620067d60 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 >> v1:172.21.15.3:6789/0 conn(0x7f3620067d60 legacy=0x7f3620069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 0x7f360c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f360c003fd0 0x7f360c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360c004d50 0x7f360c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200688e0 con 0x7f360c004d50 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c00bb40 con 0x7f360c003fd0 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c00bd00 con 0x7f36200aae50 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361f7fe700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360c004d50 0x7f360c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361e7fc700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f360c003fd0 0x7f360c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361effd700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 0x7f360c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361f7fe700 1 -- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 msgr2=0x7f360c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361f7fe700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36200aae50 0x7f360c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361f7fe700 1 -- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f360c003fd0 msgr2=0x7f360c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361f7fe700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f360c003fd0 0x7f360c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.815+0000 7f361f7fe700 1 -- 172.21.15.3:0/3682915352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c003570 con 0x7f360c004d50 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f361f7fe700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360c004d50 0x7f360c0051c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f362006a190 tx=0x7f3618007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ef927db7373f17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800c5d0 con 0x7f360c004d50 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800c750 con 0x7f360c004d50 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f361dffb700 1 -- 172.21.15.3:0/3682915352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800cb10 con 0x7f360c004d50 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f36253c1700 1 -- 172.21.15.3:0/3682915352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360c004d50 msgr2=0x7f360c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f36253c1700 1 --2- 172.21.15.3:0/3682915352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360c004d50 0x7f360c0051c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f362006a190 tx=0x7f3618007d10 comp rx=0 tx=0).stop 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f361f7fe700 1 -- 172.21.15.3:0/3682915352 reap_dead start 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f36253c1700 1 -- 172.21.15.3:0/3682915352 shutdown_connections 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f36253c1700 1 -- 172.21.15.3:0/3682915352 >> 172.21.15.3:0/3682915352 conn(0x7f362006e730 msgr2=0x7f362006eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f36253c1700 1 -- 172.21.15.3:0/3682915352 shutdown_connections 2023-10-06T05:03:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.816+0000 7f36253c1700 1 -- 172.21.15.3:0/3682915352 wait complete. 2023-10-06T05:03:03.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 Processor -- start 2023-10-06T05:03:03.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 -- start start 2023-10-06T05:03:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f360c004d50 0x7f362014da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36200b23a0 0x7f36200b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362009cf40 0x7f362009d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b2d50 con 0x7f36200b23a0 2023-10-06T05:03:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362009b1d0 con 0x7f362009cf40 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f36253c1700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362009a5c0 con 0x7f360c004d50 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.818+0000 7f361e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36200b23a0 0x7f36200b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36200b23a0 0x7f36200b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33952/0 (socket says 172.21.15.3:33952) 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 -- 172.21.15.3:0/1238749797 learned_addr learned my addr 172.21.15.3:0/1238749797 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361f7fe700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362009cf40 0x7f362009d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361effd700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f360c004d50 0x7f362014da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 -- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f360c004d50 msgr2=0x7f362014da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f360c004d50 0x7f362014da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 -- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362009cf40 msgr2=0x7f362009d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362009cf40 0x7f362009d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 -- 172.21.15.3:0/1238749797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362009a8a0 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361effd700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f360c004d50 0x7f362014da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f361e7fc700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36200b23a0 0x7f36200b2810 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f36080010e0 tx=0x7f3608001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800f870 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800f9f0 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362009ab80 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800fdb0 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.819+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362009d8f0 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.820+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f362004c380 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3608013600 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f3616ffd700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3600066f30 0x7f36000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3608084fd0 con 0x7f36200b23a0 2023-10-06T05:03:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f361effd700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3600066f30 0x7f36000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f3616ffd700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f360006d3f0 0x7f360006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 --> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f360006feb0 con 0x7f360006d3f0 2023-10-06T05:03:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36080853a0 con 0x7f36200b23a0 2023-10-06T05:03:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.821+0000 7f361f7fe700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f360006d3f0 0x7f360006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.822+0000 7f361effd700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3600066f30 0x7f36000693e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f36100099f0 tx=0x7f361000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.822+0000 7f361f7fe700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f360006d3f0 0x7f360006f800 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.823+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== osd.7 v2:172.21.15.153:6816/90928 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f360006feb0 con 0x7f360006d3f0 2023-10-06T05:03:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.836+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 --> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f36200d0fc0 con 0x7f360006d3f0 2023-10-06T05:03:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f3616ffd700 1 -- 172.21.15.3:0/1238749797 <== osd.7 v2:172.21.15.153:6816/90928 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f36200d0fc0 con 0x7f360006d3f0 2023-10-06T05:03:03.838 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-06T05:03:03.839 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:03:03.839 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f360006d3f0 msgr2=0x7f360006f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6816/90928,v1:172.21.15.153:6818/90928] conn(0x7f360006d3f0 0x7f360006f800 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3600066f30 msgr2=0x7f36000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3600066f30 0x7f36000693e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f36100099f0 tx=0x7f361000b040 comp rx=0 tx=0).stop 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36200b23a0 msgr2=0x7f36200b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 --2- 172.21.15.3:0/1238749797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36200b23a0 0x7f36200b2810 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f36080010e0 tx=0x7f3608001e70 comp rx=0 tx=0).stop 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f361f7fe700 1 -- 172.21.15.3:0/1238749797 reap_dead start 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 shutdown_connections 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 >> 172.21.15.3:0/1238749797 conn(0x7f362006e730 msgr2=0x7f36200cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 shutdown_connections 2023-10-06T05:03:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.837+0000 7f36253c1700 1 -- 172.21.15.3:0/1238749797 wait complete. 2023-10-06T05:03:03.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T05:03:03.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.969+0000 7fdac1829700 1 Processor -- start 2023-10-06T05:03:03.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.970+0000 7fdac1829700 1 -- start start 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.970+0000 7fdac1829700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdabc0cc230 0x7fdabc0c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.970+0000 7fdac1829700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.970+0000 7fdac1829700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc0c3eb0 con 0x7fdabc0b0420 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.970+0000 7fdac1829700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0cd420 con 0x7fdabc0cc230 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.970+0000 7fdac1829700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0b0c20 con 0x7fdabc0b07f0 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.970+0000 7fdaba7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdabc0cc230 0x7fdabc0c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdabaffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdabc0b0420 0x7fdabc0b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45902/0 (socket says 172.21.15.3:45902) 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdabaffd700 1 -- 172.21.15.3:0/793228044 learned_addr learned my addr 172.21.15.3:0/793228044 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdabb7fe700 1 --2- 172.21.15.3:0/793228044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdab9ffb700 1 -- 172.21.15.3:0/793228044 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621310119 0 0) 0x7fdabc0c3eb0 con 0x7fdabc0b0420 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdab9ffb700 1 -- 172.21.15.3:0/793228044 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa8003570 con 0x7fdabc0b0420 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdaba7fc700 1 -- 172.21.15.3:0/793228044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 msgr2=0x7fdabc0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdaba7fc700 1 --2- 172.21.15.3:0/793228044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdaba7fc700 1 -- 172.21.15.3:0/793228044 >> v1:172.21.15.3:6789/0 conn(0x7fdabc0b0420 legacy=0x7fdabc0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdaba7fc700 1 -- 172.21.15.3:0/793228044 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0b1990 con 0x7fdabc0cc230 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdaba7fc700 1 --2- 172.21.15.3:0/793228044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc0c4a80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40098e0 tx=0x7fdaa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da00e17577a3bc92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdab9ffb700 1 -- 172.21.15.3:0/793228044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400b640 con 0x7fdabc0cc230 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdab9ffb700 1 -- 172.21.15.3:0/793228044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa400b7a0 con 0x7fdabc0cc230 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.971+0000 7fdab9ffb700 1 -- 172.21.15.3:0/793228044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400bb20 con 0x7fdabc0cc230 2023-10-06T05:03:03.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 -- 172.21.15.3:0/793228044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 msgr2=0x7fdabc0c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 --2- 172.21.15.3:0/793228044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc0c4a80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40098e0 tx=0x7fdaa40091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 -- 172.21.15.3:0/793228044 shutdown_connections 2023-10-06T05:03:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 --2- 172.21.15.3:0/793228044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 --2- 172.21.15.3:0/793228044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc0c4a80 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 -- 172.21.15.3:0/793228044 >> 172.21.15.3:0/793228044 conn(0x7fdabc0976f0 msgr2=0x7fdabc0677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 -- 172.21.15.3:0/793228044 shutdown_connections 2023-10-06T05:03:03.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.972+0000 7fdac1829700 1 -- 172.21.15.3:0/793228044 wait complete. 2023-10-06T05:03:03.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.973+0000 7fdac1829700 1 Processor -- start 2023-10-06T05:03:03.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.973+0000 7fdac1829700 1 -- start start 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdac1829700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdabc0b0420 0x7fdabc15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdac1829700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdac1829700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdac1829700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc15d2a0 con 0x7fdabc0cc230 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdac1829700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc15d730 con 0x7fdabc0b0420 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdac1829700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc14c7a0 con 0x7fdabc0b07f0 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc15e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33962/0 (socket says 172.21.15.3:33962) 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 -- 172.21.15.3:0/3208211784 learned_addr learned my addr 172.21.15.3:0/3208211784 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabaffd700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdabc0b0420 0x7fdabc15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdaba7fc700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 -- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 msgr2=0x7fdabc14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 -- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdabc0b0420 msgr2=0x7fdabc15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdabc0b0420 0x7fdabc15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 -- 172.21.15.3:0/3208211784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4009420 con 0x7fdabc0cc230 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdabb7fe700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc15e610 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009fa0 tx=0x7fdaac009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.974+0000 7fdab2ffd700 1 -- 172.21.15.3:0/3208211784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00ed20 con 0x7fdabc0cc230 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.975+0000 7fdab2ffd700 1 -- 172.21.15.3:0/3208211784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac00ee80 con 0x7fdabc0cc230 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.975+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc15c690 con 0x7fdabc0cc230 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.975+0000 7fdab2ffd700 1 -- 172.21.15.3:0/3208211784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac014670 con 0x7fdabc0cc230 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.975+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc15cc50 con 0x7fdabc0cc230 2023-10-06T05:03:03.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.976+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc15d410 con 0x7fdabc0cc230 2023-10-06T05:03:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.976+0000 7fdab2ffd700 1 -- 172.21.15.3:0/3208211784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdaac0149e0 con 0x7fdabc0cc230 2023-10-06T05:03:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.976+0000 7fdab2ffd700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fda9c066f30 0x7fda9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.976+0000 7fdab2ffd700 1 -- 172.21.15.3:0/3208211784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdaac084c00 con 0x7fdabc0cc230 2023-10-06T05:03:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.977+0000 7fdabaffd700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fda9c066f30 0x7fda9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.977+0000 7fdabaffd700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fda9c066f30 0x7fda9c0693e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000c00 tx=0x7fdab4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:03.981+0000 7fdab2ffd700 1 -- 172.21.15.3:0/3208211784 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdaac08b080 con 0x7fdabc0cc230 2023-10-06T05:03:04.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.173+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fdabc04c380 con 0x7fda9c066f30 2023-10-06T05:03:04.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.173+0000 7fdab2ffd700 1 -- 172.21.15.3:0/3208211784 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdabc04c380 con 0x7fda9c066f30 2023-10-06T05:03:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fda9c066f30 msgr2=0x7fda9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fda9c066f30 0x7fda9c0693e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000c00 tx=0x7fdab4009040 comp rx=0 tx=0).stop 2023-10-06T05:03:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 msgr2=0x7fdabc15e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc15e610 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009fa0 tx=0x7fdaac009900 comp rx=0 tx=0).stop 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 shutdown_connections 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7fda9c066f30 0x7fda9c0693e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdabc0b0420 0x7fdabc15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc0cc230 0x7fdabc15e610 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 --2- 172.21.15.3:0/3208211784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdabc0b07f0 0x7fdabc14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 >> 172.21.15.3:0/3208211784 conn(0x7fdabc0976f0 msgr2=0x7fdabc06ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 shutdown_connections 2023-10-06T05:03:04.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.175+0000 7fdac1829700 1 -- 172.21.15.3:0/3208211784 wait complete. 2023-10-06T05:03:04.179 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T05:03:04.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:03:04.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.306+0000 7f734fa8c700 1 Processor -- start 2023-10-06T05:03:04.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.306+0000 7f734fa8c700 1 -- start start 2023-10-06T05:03:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734fa8c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734fa8c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f734815d6e0 0x7f73480cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734fa8c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73480cd840 con 0x7f73480cd410 2023-10-06T05:03:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734fa8c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734815dc20 con 0x7f734815d6e0 2023-10-06T05:03:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734fa8c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734815df00 con 0x7f7348153bd0 2023-10-06T05:03:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734d027700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f734815d6e0 0x7f73480cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734d027700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f734815d6e0 0x7f73480cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33972/0 (socket says 172.21.15.3:33972) 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.307+0000 7f734d027700 1 -- 172.21.15.3:0/3367334816 learned_addr learned my addr 172.21.15.3:0/3367334816 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734d828700 1 --2- 172.21.15.3:0/3367334816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734d027700 1 -- 172.21.15.3:0/3367334816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 msgr2=0x7f734815d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734d027700 1 --2- 172.21.15.3:0/3367334816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734d027700 1 -- 172.21.15.3:0/3367334816 >> v1:172.21.15.153:6789/0 conn(0x7f73480cd410 legacy=0x7f7348067ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734d027700 1 -- 172.21.15.3:0/3367334816 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480ce7f0 con 0x7f734815d6e0 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734d828700 1 --2- 172.21.15.3:0/3367334816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734d027700 1 --2- 172.21.15.3:0/3367334816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734815d6e0 0x7f73480cced0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f733c0099b0 tx=0x7f733c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f012dcd933e1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734c826700 1 -- 172.21.15.3:0/3367334816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00b860 con 0x7f734815d6e0 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734c826700 1 -- 172.21.15.3:0/3367334816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c00b9c0 con 0x7f734815d6e0 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.308+0000 7f734c826700 1 -- 172.21.15.3:0/3367334816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00bd60 con 0x7f734815d6e0 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 -- 172.21.15.3:0/3367334816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734815d6e0 msgr2=0x7f73480cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3367334816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734815d6e0 0x7f73480cced0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f733c0099b0 tx=0x7f733c009250 comp rx=0 tx=0).stop 2023-10-06T05:03:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 -- 172.21.15.3:0/3367334816 shutdown_connections 2023-10-06T05:03:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3367334816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3367334816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734815d6e0 0x7f73480cced0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 -- 172.21.15.3:0/3367334816 >> 172.21.15.3:0/3367334816 conn(0x7f734806e730 msgr2=0x7f734806eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 -- 172.21.15.3:0/3367334816 shutdown_connections 2023-10-06T05:03:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.309+0000 7f734fa8c700 1 -- 172.21.15.3:0/3367334816 wait complete. 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.310+0000 7f734fa8c700 1 Processor -- start 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.310+0000 7f734fa8c700 1 -- start start 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734fa8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73480cd410 0x7f7348148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734fa8c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734fa8c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f734815d6e0 0x7f734816a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734fa8c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734815c510 con 0x7f73480cd410 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734fa8c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734815bb40 con 0x7f734815d6e0 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734fa8c700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734815af30 con 0x7f7348153bd0 2023-10-06T05:03:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73480cd410 0x7f7348148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73480cd410 0x7f7348148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33976/0 (socket says 172.21.15.3:33976) 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 -- 172.21.15.3:0/3205221938 learned_addr learned my addr 172.21.15.3:0/3205221938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d027700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734e029700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f734815d6e0 0x7f734816a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 -- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 msgr2=0x7f734815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 -- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f734815d6e0 msgr2=0x7f734816a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f734815d6e0 0x7f734816a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 -- 172.21.15.3:0/3205221938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0094f0 con 0x7f73480cd410 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734d828700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73480cd410 0x7f7348148e50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f733800e920 tx=0x7f7338008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f7345ffb700 1 -- 172.21.15.3:0/3205221938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338005a00 con 0x7f73480cd410 2023-10-06T05:03:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f7345ffb700 1 -- 172.21.15.3:0/3205221938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7338005b60 con 0x7f73480cd410 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734815b210 con 0x7f73480cd410 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.311+0000 7f7345ffb700 1 -- 172.21.15.3:0/3205221938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338006020 con 0x7f73480cd410 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.312+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734815b7d0 con 0x7f73480cd410 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.312+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734804c380 con 0x7f73480cd410 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.313+0000 7f7345ffb700 1 -- 172.21.15.3:0/3205221938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7338006390 con 0x7f73480cd410 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.313+0000 7f7345ffb700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7334066f30 0x7f73340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.313+0000 7f7345ffb700 1 -- 172.21.15.3:0/3205221938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f73380843a0 con 0x7f73480cd410 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.314+0000 7f734d027700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7334066f30 0x7f73340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.314+0000 7f734d027700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7334066f30 0x7f73340693e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f734815db50 tx=0x7f733c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.318+0000 7f7345ffb700 1 -- 172.21.15.3:0/3205221938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7338051780 con 0x7f73480cd410 2023-10-06T05:03:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.507+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73480cb300 con 0x7f7334066f30 2023-10-06T05:03:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.511+0000 7f7345ffb700 1 -- 172.21.15.3:0/3205221938 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73084 (secure 0 0 0) 0x7f73480cb300 con 0x7f7334066f30 2023-10-06T05:03:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7334066f30 msgr2=0x7f73340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7334066f30 0x7f73340693e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f734815db50 tx=0x7f733c017000 comp rx=0 tx=0).stop 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73480cd410 msgr2=0x7f7348148e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73480cd410 0x7f7348148e50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f733800e920 tx=0x7f7338008040 comp rx=0 tx=0).stop 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 shutdown_connections 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f7334066f30 0x7f73340693e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f734815d6e0 0x7f734816a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73480cd410 0x7f7348148e50 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 --2- 172.21.15.3:0/3205221938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7348153bd0 0x7f734815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 >> 172.21.15.3:0/3205221938 conn(0x7f734806e730 msgr2=0x7f73480c64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 shutdown_connections 2023-10-06T05:03:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:04.513+0000 7f734fa8c700 1 -- 172.21.15.3:0/3205221938 wait complete. 2023-10-06T05:03:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:03:04.527 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2023-10-06T05:03:03.583975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2450364,"kb_used_data":14716,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524356,"statfs":{"total":773094113280,"available":770584940544,"internally_reserved":0,"allocated":15069184,"data_stored":7511101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.731971"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.866557+0000","last_change":"2023-10-06T04:52:13.226712+0000","last_active":"2023-10-06T05:02:54.866557+0000","last_peered":"2023-10-06T05:02:54.866557+0000","last_clean":"2023-10-06T05:02:54.866557+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:02:54.866557+0000","last_undegraded":"2023-10-06T05:02:54.866557+0000","last_fullsized":"2023-10-06T05:02:54.866557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":120,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.854819+0000","last_change":"2023-10-06T04:52:12.525811+0000","last_active":"2023-10-06T05:02:54.854819+0000","last_peered":"2023-10-06T05:02:54.854819+0000","last_clean":"2023-10-06T05:02:54.854819+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:02:54.854819+0000","last_undegraded":"2023-10-06T05:02:54.854819+0000","last_fullsized":"2023-10-06T05:02:54.854819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:59.412738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":113,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.856173+0000","last_change":"2023-10-06T04:52:12.525238+0000","last_active":"2023-10-06T05:02:54.856173+0000","last_peered":"2023-10-06T05:02:54.856173+0000","last_clean":"2023-10-06T05:02:54.856173+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:02:54.856173+0000","last_undegraded":"2023-10-06T05:02:54.856173+0000","last_fullsized":"2023-10-06T05:02:54.856173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":260,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.865419+0000","last_change":"2023-10-06T04:52:12.525632+0000","last_active":"2023-10-06T05:02:54.865419+0000","last_peered":"2023-10-06T05:02:54.865419+0000","last_clean":"2023-10-06T05:02:54.865419+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:02:54.865419+0000","last_undegraded":"2023-10-06T05:02:54.865419+0000","last_fullsized":"2023-10-06T05:02:54.865419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.864357+0000","last_change":"2023-10-06T04:52:13.028827+0000","last_active":"2023-10-06T05:02:54.864357+0000","last_peered":"2023-10-06T05:02:54.864357+0000","last_clean":"2023-10-06T05:02:54.864357+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:02:54.864357+0000","last_undegraded":"2023-10-06T05:02:54.864357+0000","last_fullsized":"2023-10-06T05:02:54.864357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.865474+0000","last_change":"2023-10-06T04:52:13.029057+0000","last_active":"2023-10-06T05:02:54.865474+0000","last_peered":"2023-10-06T05:02:54.865474+0000","last_clean":"2023-10-06T05:02:54.865474+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:02:54.865474+0000","last_undegraded":"2023-10-06T05:02:54.865474+0000","last_fullsized":"2023-10-06T05:02:54.865474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:07.331420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.859741+0000","last_change":"2023-10-06T04:52:13.028904+0000","last_active":"2023-10-06T05:02:54.859741+0000","last_peered":"2023-10-06T05:02:54.859741+0000","last_clean":"2023-10-06T05:02:54.859741+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:02:54.859741+0000","last_undegraded":"2023-10-06T05:02:54.859741+0000","last_fullsized":"2023-10-06T05:02:54.859741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:58.964738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.862594+0000","last_change":"2023-10-06T04:52:13.028898+0000","last_active":"2023-10-06T05:02:54.862594+0000","last_peered":"2023-10-06T05:02:54.862594+0000","last_clean":"2023-10-06T05:02:54.862594+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:02:54.862594+0000","last_undegraded":"2023-10-06T05:02:54.862594+0000","last_fullsized":"2023-10-06T05:02:54.862594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:12.193371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-06T05:02:54.863541+0000","last_change":"2023-10-06T04:52:12.525693+0000","last_active":"2023-10-06T05:02:54.863541+0000","last_peered":"2023-10-06T05:02:54.863541+0000","last_clean":"2023-10-06T05:02:54.863541+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:02:54.863541+0000","last_undegraded":"2023-10-06T05:02:54.863541+0000","last_fullsized":"2023-10-06T05:02:54.863541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:52:11.511026+0000","last_clean_scrub_stamp":"2023-10-06T04:52:11.511026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:14.977289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307752,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064088,"statfs":{"total":96636764160,"available":96321626112,"internally_reserved":0,"allocated":2326528,"data_stored":1380818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":7.1779999999999999},"last":0.495},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.41299999999999998,"15min":0.311},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":7.1879999999999997},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.748,"5min":0.78400000000000003,"15min":1.073},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.1379999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":1.1240000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":1.111},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":1.0880000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.38200000000000001},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.39700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":0.81499999999999995,"15min":0.874},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.86099999999999999,"15min":1.214},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574982,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303080,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068760,"statfs":{"total":96636764160,"available":96326410240,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.28799999999999998,"15min":0.20699999999999999},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.76000000000000001,"5min":1.21,"15min":2.8140000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":1.093,"15min":4.3559999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.68899999999999995,"5min":0.83099999999999996,"15min":1.234},"last":0.504}]},{"osd":2,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":5.7080000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.45000000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.74099999999999999,"5min":1.3120000000000001,"15min":2.8650000000000002},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.238,"15min":4.2569999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":0.66800000000000004,"5min":1.175,"15min":4.3949999999999996},"last":0.42299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.51200000000000001},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.65200000000000002,"5min":0.78200000000000003,"15min":3.669},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.748,"5min":1.147,"15min":2.9900000000000002},"last":0.38900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":1.2509999999999999,"15min":4.4370000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.46700000000000003,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":0.85899999999999999,"15min":2.9510000000000001},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77600000000000002,"5min":1.274,"15min":2.9049999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72699999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.52100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307752,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064088,"statfs":{"total":96636764160,"available":96321626112,"internally_reserved":0,"allocated":2326528,"data_stored":1380818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.35599999999999998},"min":{"1min":0.20599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.378},"min":{"1min":0.254,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.54100000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.373}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28899999999999998,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.69399999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.622,"15min":0.70399999999999996},"last":0.313}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41599999999999998,"15min":0.42299999999999999},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.379},"max":{"1min":0.876,"5min":0.877,"15min":1.0780000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":1.117,"15min":1.117},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61699999999999999,"15min":0.63100000000000001},"min":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":1.0049999999999999},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":0.84299999999999997,"5min":1.167,"15min":1.167},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.98999999999999999},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":0.82699999999999996,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.496,"15min":0.441},"max":{"1min":0.90700000000000003,"5min":0.96599999999999997,"15min":1.0429999999999999},"last":0.60399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307176,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064664,"statfs":{"total":96636764160,"available":96322215936,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.376,"15min":0.36499999999999999},"min":{"1min":0.26100000000000001,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.67200000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.40200000000000002,"15min":0.375},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.59199999999999997,"15min":0.628},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.39700000000000002},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":0.67500000000000004,"15min":0.77600000000000002},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.33700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.68899999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.36199999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63900000000000001,"5min":0.71999999999999997,"15min":0.86299999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.749,"5min":0.94799999999999995,"15min":0.98299999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76600000000000001,"5min":0.89000000000000001,"15min":0.92300000000000004},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.73699999999999999,"5min":0.90200000000000002,"15min":0.91700000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":0.93700000000000006,"15min":0.97299999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":0.96199999999999997,"15min":1.0109999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67400000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.0329999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55900000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":0.97899999999999998,"15min":0.995},"last":0.53200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303080,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068760,"statfs":{"total":96636764160,"available":96326410240,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.56100000000000005,"5min":0.64100000000000001,"15min":1.048},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.249},"max":{"1min":0.58099999999999996,"5min":0.62,"15min":0.69699999999999995},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39900000000000002,"15min":0.39400000000000002},"min":{"1min":0.29899999999999999,"5min":0.253,"15min":0.216},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":1.1819999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.219},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.374}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.19900000000000001},"max":{"1min":0.47899999999999998,"5min":0.55200000000000005,"15min":0.57399999999999995},"last":0.255},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.44400000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.53900000000000003,"5min":0.67400000000000004,"15min":1.2070000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.88300000000000001,"15min":1.246},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.90200000000000002},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.2649999999999999},"last":0.504}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.93100000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.32600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.90600000000000003,"15min":0.92200000000000004},"last":0.53000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307176,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064664,"statfs":{"total":96636764160,"available":96322215936,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.39600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.502,"5min":1.5589999999999999,"15min":2.008},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.58899999999999997,"5min":0.73399999999999999,"15min":6.4370000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39500000000000002,"15min":0.372},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.63100000000000001,"5min":1.601,"15min":2.1970000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41499999999999998,"15min":0.496},"min":{"1min":0.317,"5min":0.30199999999999999,"15min":0.255},"max":{"1min":0.67000000000000004,"5min":0.88400000000000001,"15min":5.069},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":2.246},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":0.97899999999999998,"5min":1.6719999999999999,"15min":14.646000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90000000000000002,"5min":3.109,"15min":14.606999999999999},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":1.506,"15min":2.2959999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.72199999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":4.3150000000000004,"15min":6.4009999999999998},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.80600000000000005,"15min":0.76900000000000002},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":16.773,"15min":16.773},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77200000000000002,"15min":0.70899999999999996},"min":{"1min":0.45400000000000001,"5min":0.40600000000000003,"15min":0.40100000000000002},"max":{"1min":0.91200000000000003,"5min":11.436,"15min":11.436},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.72999999999999998},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.579,"15min":6.367},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":3.069,"15min":5},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307176,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064664,"statfs":{"total":96636764160,"available":96322215936,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.95799999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.34799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.5510000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.47399999999999998,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":5.4900000000000002},"last":0.41999999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":4.0739999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.626},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":5.415},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58099999999999996,"15min":0.63200000000000001},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":4.202},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":4.2130000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1279999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.53500000000000003},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.83299999999999996},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.72499999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76300000000000001,"15min":0.99099999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.56799999999999995},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.872},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":1.1000000000000001},"last":0.625}]}]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307172,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064668,"statfs":{"total":96636764160,"available":96322220032,"internally_reserved":0,"allocated":1732608,"data_stored":790450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":4.9039999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.752,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.873,"15min":6.7949999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.625},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.68100000000000005},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.626}]},{"osd":3,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.79500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.30199999999999999},"max":{"1min":0.79800000000000004,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.79600000000000004},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.754}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.54100000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.74399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":0.873},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.56299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:03:04.532 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.532 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.532 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.532 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.532 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.532 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.533 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.533 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.533 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-10-06T04:52:11.511026+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=52, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=2, tm_sec=59, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:03:04.533 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T05:03:24.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:03:24.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.665+0000 7f860431f700 1 Processor -- start 2023-10-06T05:03:24.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.665+0000 7f860431f700 1 -- start start 2023-10-06T05:03:24.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f860431f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0b0420 0x7f85fc0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:24.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f860431f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85fc0b07f0 0x7f85fc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:24.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f860431f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc0c3eb0 con 0x7f85fc0c4a80 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f860431f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc0cd420 con 0x7f85fc0b07f0 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f860431f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0b0c50 con 0x7f85fc0b0420 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f86028bc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85fc0b07f0 0x7f85fc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f86028bc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85fc0b07f0 0x7f85fc0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37056/0 (socket says 172.21.15.3:37056) 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f86028bc700 1 -- 172.21.15.3:0/4026219687 learned_addr learned my addr 172.21.15.3:0/4026219687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.666+0000 7f86020bb700 1 --2- 172.21.15.3:0/4026219687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0b0420 0x7f85fc0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86028bc700 1 -- 172.21.15.3:0/4026219687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0b0420 msgr2=0x7f85fc0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86028bc700 1 --2- 172.21.15.3:0/4026219687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0b0420 0x7f85fc0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86028bc700 1 -- 172.21.15.3:0/4026219687 >> v1:172.21.15.153:6789/0 conn(0x7f85fc0c4a80 legacy=0x7f85fc0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86028bc700 1 -- 172.21.15.3:0/4026219687 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0b1990 con 0x7f85fc0b07f0 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86028bc700 1 --2- 172.21.15.3:0/4026219687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc0cd8b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f85f40099b0 tx=0x7f85f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb811d82c56f5e64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:24.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86010b9700 1 -- 172.21.15.3:0/4026219687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4009ea0 con 0x7f85fc0b07f0 2023-10-06T05:03:24.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86010b9700 1 -- 172.21.15.3:0/4026219687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400b860 con 0x7f85fc0b07f0 2023-10-06T05:03:24.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.667+0000 7f86010b9700 1 -- 172.21.15.3:0/4026219687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400bc00 con 0x7f85fc0b07f0 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 -- 172.21.15.3:0/4026219687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 msgr2=0x7f85fc0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 --2- 172.21.15.3:0/4026219687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc0cd8b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f85f40099b0 tx=0x7f85f4009250 comp rx=0 tx=0).stop 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 -- 172.21.15.3:0/4026219687 shutdown_connections 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 --2- 172.21.15.3:0/4026219687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc0cd8b0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 --2- 172.21.15.3:0/4026219687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0b0420 0x7f85fc0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 -- 172.21.15.3:0/4026219687 >> 172.21.15.3:0/4026219687 conn(0x7f85fc0976f0 msgr2=0x7f85fc0677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 -- 172.21.15.3:0/4026219687 shutdown_connections 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.668+0000 7f860431f700 1 -- 172.21.15.3:0/4026219687 wait complete. 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.669+0000 7f860431f700 1 Processor -- start 2023-10-06T05:03:24.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.669+0000 7f860431f700 1 -- start start 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f860431f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85fc0b0420 0x7f85fc09ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f860431f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc09b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f860431f700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0c4a80 0x7f85fc169840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f860431f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc09a290 con 0x7f85fc0b07f0 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f860431f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc167480 con 0x7f85fc0b0420 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f860431f700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc157540 con 0x7f85fc0c4a80 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc09b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc09b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37062/0 (socket says 172.21.15.3:37062) 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 -- 172.21.15.3:0/1624264223 learned_addr learned my addr 172.21.15.3:0/1624264223 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86028bc700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0c4a80 0x7f85fc169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86020bb700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85fc0b0420 0x7f85fc09ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:24.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 -- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0c4a80 msgr2=0x7f85fc169840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0c4a80 0x7f85fc169840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 -- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85fc0b0420 msgr2=0x7f85fc09ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85fc0b0420 0x7f85fc09ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 -- 172.21.15.3:0/1624264223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40094f0 con 0x7f85fc0b07f0 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f86018ba700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc09b3c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c590 tx=0x7f85f000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f85fa7fc700 1 -- 172.21.15.3:0/1624264223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0011590 con 0x7f85fc0b07f0 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f85fa7fc700 1 -- 172.21.15.3:0/1624264223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f00116f0 con 0x7f85fc0b07f0 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc16aa10 con 0x7f85fc0b07f0 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.670+0000 7f85fa7fc700 1 -- 172.21.15.3:0/1624264223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0011ab0 con 0x7f85fc0b07f0 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.671+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc16afd0 con 0x7f85fc0b07f0 2023-10-06T05:03:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.671+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc04c380 con 0x7f85fc0b07f0 2023-10-06T05:03:24.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.672+0000 7f85fa7fc700 1 -- 172.21.15.3:0/1624264223 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85f001b3b0 con 0x7f85fc0b07f0 2023-10-06T05:03:24.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.672+0000 7f85fa7fc700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f85e8066f30 0x7f85e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:24.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.672+0000 7f85fa7fc700 1 -- 172.21.15.3:0/1624264223 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f85f0083950 con 0x7f85fc0b07f0 2023-10-06T05:03:24.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.673+0000 7f86020bb700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f85e8066f30 0x7f85e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:24.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.673+0000 7f86020bb700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f85e8066f30 0x7f85e80693e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0094d0 tx=0x7f85ec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:24.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.677+0000 7f85fa7fc700 1 -- 172.21.15.3:0/1624264223 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85f0050d30 con 0x7f85fc0b07f0 2023-10-06T05:03:24.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.867+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 --> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85fc0b2810 con 0x7f85e8066f30 2023-10-06T05:03:24.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.871+0000 7f85fa7fc700 1 -- 172.21.15.3:0/1624264223 <== mgr.4105 v2:172.21.15.153:6832/90310 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73153 (secure 0 0 0) 0x7f85fc0b2810 con 0x7f85e8066f30 2023-10-06T05:03:24.873 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:03:24.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f85e8066f30 msgr2=0x7f85e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f85e8066f30 0x7f85e80693e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0094d0 tx=0x7f85ec00b040 comp rx=0 tx=0).stop 2023-10-06T05:03:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 msgr2=0x7f85fc09b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc09b3c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c590 tx=0x7f85f000c950 comp rx=0 tx=0).stop 2023-10-06T05:03:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 shutdown_connections 2023-10-06T05:03:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f85e8066f30 0x7f85e80693e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85fc0b0420 0x7f85fc09ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85fc0c4a80 0x7f85fc169840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 --2- 172.21.15.3:0/1624264223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85fc0b07f0 0x7f85fc09b3c0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 >> 172.21.15.3:0/1624264223 conn(0x7f85fc0976f0 msgr2=0x7f85fc159080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 shutdown_connections 2023-10-06T05:03:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:24.872+0000 7f860431f700 1 -- 172.21.15.3:0/1624264223 wait complete. 2023-10-06T05:03:24.877 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-06T05:03:24.887 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2023-10-06T05:03:23.588220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2450364,"kb_used_data":14716,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524356,"statfs":{"total":773094113280,"available":770584940544,"internally_reserved":0,"allocated":15069184,"data_stored":7511101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002646"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:05.493491+0000","last_change":"2023-10-06T05:03:05.493464+0000","last_active":"2023-10-06T05:03:05.493491+0000","last_peered":"2023-10-06T05:03:05.493491+0000","last_clean":"2023-10-06T05:03:05.493491+0000","last_became_active":"2023-10-06T04:52:13.226447+0000","last_became_peered":"2023-10-06T04:52:13.226447+0000","last_unstale":"2023-10-06T05:03:05.493491+0000","last_undegraded":"2023-10-06T05:03:05.493491+0000","last_fullsized":"2023-10-06T05:03:05.493491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:05.493353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:05.493353+0000","last_clean_scrub_stamp":"2023-10-06T05:03:05.493353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:51.042598+0000","scrub_duration":0.002727915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":131,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:04.015596+0000","last_change":"2023-10-06T05:03:04.015557+0000","last_active":"2023-10-06T05:03:04.015596+0000","last_peered":"2023-10-06T05:03:04.015596+0000","last_clean":"2023-10-06T05:03:04.015596+0000","last_became_active":"2023-10-06T04:52:12.525435+0000","last_became_peered":"2023-10-06T04:52:12.525435+0000","last_unstale":"2023-10-06T05:03:04.015596+0000","last_undegraded":"2023-10-06T05:03:04.015596+0000","last_fullsized":"2023-10-06T05:03:04.015596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:04.015462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:04.015462+0000","last_clean_scrub_stamp":"2023-10-06T05:03:04.015462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:46.117619+0000","scrub_duration":0.002539735,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":124,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:01.658967+0000","last_change":"2023-10-06T05:03:01.658942+0000","last_active":"2023-10-06T05:03:01.658967+0000","last_peered":"2023-10-06T05:03:01.658967+0000","last_clean":"2023-10-06T05:03:01.658967+0000","last_became_active":"2023-10-06T04:52:12.524882+0000","last_became_peered":"2023-10-06T04:52:12.524882+0000","last_unstale":"2023-10-06T05:03:01.658967+0000","last_undegraded":"2023-10-06T05:03:01.658967+0000","last_fullsized":"2023-10-06T05:03:01.658967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:01.658851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:01.658851+0000","last_clean_scrub_stamp":"2023-10-06T05:03:01.658851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:32.657999+0000","scrub_duration":0.0027743300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"138'119","reported_seq":273,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:11.166652+0000","last_change":"2023-10-06T05:03:11.166620+0000","last_active":"2023-10-06T05:03:11.166652+0000","last_peered":"2023-10-06T05:03:11.166652+0000","last_clean":"2023-10-06T05:03:11.166652+0000","last_became_active":"2023-10-06T04:52:12.525356+0000","last_became_peered":"2023-10-06T04:52:12.525356+0000","last_unstale":"2023-10-06T05:03:11.166652+0000","last_undegraded":"2023-10-06T05:03:11.166652+0000","last_fullsized":"2023-10-06T05:03:11.166652+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'119","last_scrub_stamp":"2023-10-06T05:03:11.166516+0000","last_deep_scrub":"138'119","last_deep_scrub_stamp":"2023-10-06T05:03:11.166516+0000","last_clean_scrub_stamp":"2023-10-06T05:03:11.166516+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:27.641711+0000","scrub_duration":0.006839543,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:08.256765+0000","last_change":"2023-10-06T05:03:08.256739+0000","last_active":"2023-10-06T05:03:08.256765+0000","last_peered":"2023-10-06T05:03:08.256765+0000","last_clean":"2023-10-06T05:03:08.256765+0000","last_became_active":"2023-10-06T04:52:13.028557+0000","last_became_peered":"2023-10-06T04:52:13.028557+0000","last_unstale":"2023-10-06T05:03:08.256765+0000","last_undegraded":"2023-10-06T05:03:08.256765+0000","last_fullsized":"2023-10-06T05:03:08.256765+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:08.256612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:08.256612+0000","last_clean_scrub_stamp":"2023-10-06T05:03:08.256612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:58.185864+0000","scrub_duration":0.0025175269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:05.210414+0000","last_change":"2023-10-06T05:03:05.210386+0000","last_active":"2023-10-06T05:03:05.210414+0000","last_peered":"2023-10-06T05:03:05.210414+0000","last_clean":"2023-10-06T05:03:05.210414+0000","last_became_active":"2023-10-06T04:52:13.028478+0000","last_became_peered":"2023-10-06T04:52:13.028478+0000","last_unstale":"2023-10-06T05:03:05.210414+0000","last_undegraded":"2023-10-06T05:03:05.210414+0000","last_fullsized":"2023-10-06T05:03:05.210414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:05.210283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:05.210283+0000","last_clean_scrub_stamp":"2023-10-06T05:03:05.210283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:57.894022+0000","scrub_duration":0.0024599980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:04.676073+0000","last_change":"2023-10-06T05:03:04.676045+0000","last_active":"2023-10-06T05:03:04.676073+0000","last_peered":"2023-10-06T05:03:04.676073+0000","last_clean":"2023-10-06T05:03:04.676073+0000","last_became_active":"2023-10-06T04:52:13.028496+0000","last_became_peered":"2023-10-06T04:52:13.028496+0000","last_unstale":"2023-10-06T05:03:04.676073+0000","last_undegraded":"2023-10-06T05:03:04.676073+0000","last_fullsized":"2023-10-06T05:03:04.676073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:04.675935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:04.675935+0000","last_clean_scrub_stamp":"2023-10-06T05:03:04.675935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:29.376447+0000","scrub_duration":0.0022198790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":120,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:03.775843+0000","last_change":"2023-10-06T05:03:03.775816+0000","last_active":"2023-10-06T05:03:03.775843+0000","last_peered":"2023-10-06T05:03:03.775843+0000","last_clean":"2023-10-06T05:03:03.775843+0000","last_became_active":"2023-10-06T04:52:13.028518+0000","last_became_peered":"2023-10-06T04:52:13.028518+0000","last_unstale":"2023-10-06T05:03:03.775843+0000","last_undegraded":"2023-10-06T05:03:03.775843+0000","last_fullsized":"2023-10-06T05:03:03.775843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:03.775703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:03.775703+0000","last_clean_scrub_stamp":"2023-10-06T05:03:03.775703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:09.158674+0000","scrub_duration":0.0020347519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:03:06.827602+0000","last_change":"2023-10-06T05:03:06.827553+0000","last_active":"2023-10-06T05:03:06.827602+0000","last_peered":"2023-10-06T05:03:06.827602+0000","last_clean":"2023-10-06T05:03:06.827602+0000","last_became_active":"2023-10-06T04:52:12.525391+0000","last_became_peered":"2023-10-06T04:52:12.525391+0000","last_unstale":"2023-10-06T05:03:06.827602+0000","last_undegraded":"2023-10-06T05:03:06.827602+0000","last_fullsized":"2023-10-06T05:03:06.827602+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:03:06.827440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:03:06.827440+0000","last_clean_scrub_stamp":"2023-10-06T05:03:06.827440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:26.397018+0000","scrub_duration":0.00279306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307752,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064088,"statfs":{"total":96636764160,"available":96321626112,"internally_reserved":0,"allocated":2326528,"data_stored":1380818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.311,"15min":0.22800000000000001},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":7.1779999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.41299999999999998,"15min":0.311},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":7.1879999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.23999999999999999},"max":{"1min":0.748,"5min":0.78400000000000003,"15min":1.073},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.1379999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":1.1240000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":1.111},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":1.0880000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.38200000000000001},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":1.1539999999999999},"last":0.501}]},{"osd":4,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.85999999999999999},"last":0.376},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.82699999999999996,"15min":0.86899999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":0.81499999999999995,"15min":0.874},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.86099999999999999,"15min":1.214},"last":0.626}]},{"osd":6,"last update":"Fri Oct 6 05:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574986,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303080,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068760,"statfs":{"total":96636764160,"available":96326410240,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.28799999999999998,"15min":0.20699999999999999},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.38600000000000001,"15min":0.32100000000000001},"max":{"1min":0.76000000000000001,"5min":1.21,"15min":2.8140000000000001},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":1.093,"15min":4.3559999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.68899999999999995,"5min":0.83099999999999996,"15min":1.234},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":5.7080000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.45000000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.74099999999999999,"5min":1.3120000000000001,"15min":2.8650000000000002},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.238,"15min":4.2569999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":0.66800000000000004,"5min":1.175,"15min":4.3949999999999996},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.51200000000000001},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.248},"max":{"1min":0.65200000000000002,"5min":0.78200000000000003,"15min":3.669},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.26300000000000001},"max":{"1min":0.748,"5min":1.147,"15min":2.9900000000000002},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":1.2509999999999999,"15min":4.4370000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.46700000000000003,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":0.85899999999999999,"15min":2.9510000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77600000000000002,"5min":1.274,"15min":2.9049999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72699999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.57599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307752,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064088,"statfs":{"total":96636764160,"available":96321626112,"internally_reserved":0,"allocated":2326528,"data_stored":1380818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.35599999999999998},"min":{"1min":0.20599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.68400000000000005},"last":0.439},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.378},"min":{"1min":0.254,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":0.54100000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28899999999999998,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.69399999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.622,"15min":0.70399999999999996},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41599999999999998,"15min":0.42299999999999999},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.69699999999999995,"15min":0.70999999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.379},"max":{"1min":0.876,"5min":0.877,"15min":1.0780000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.39100000000000001,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":1.117,"15min":1.117},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61699999999999999,"15min":0.63100000000000001},"min":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":1.0049999999999999},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":0.84299999999999997,"5min":1.167,"15min":1.167},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.98999999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.40799999999999997},"max":{"1min":0.82699999999999996,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.496,"15min":0.441},"max":{"1min":0.90700000000000003,"5min":0.96599999999999997,"15min":1.0429999999999999},"last":0.71399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307176,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064664,"statfs":{"total":96636764160,"available":96322215936,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.376,"15min":0.36499999999999999},"min":{"1min":0.26100000000000001,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.67200000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.40200000000000002,"15min":0.375},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.59199999999999997,"15min":0.628},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.39700000000000002},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":0.67500000000000004,"15min":0.77600000000000002},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.33700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.68899999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.36199999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63900000000000001,"5min":0.71999999999999997,"15min":0.86299999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.749,"5min":0.94799999999999995,"15min":0.98299999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76600000000000001,"5min":0.89000000000000001,"15min":0.92300000000000004},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.026},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38500000000000001},"max":{"1min":0.73699999999999999,"5min":0.90200000000000002,"15min":0.91700000000000004},"last":0.94699999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":0.93700000000000006,"15min":0.97299999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":0.96199999999999997,"15min":1.0109999999999999},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67400000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.0329999999999999},"last":1.048},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55900000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":0.97899999999999998,"15min":0.995},"last":0.91400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303080,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068760,"statfs":{"total":96636764160,"available":96326410240,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.56100000000000005,"5min":0.64100000000000001,"15min":1.048},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.249},"max":{"1min":0.58099999999999996,"5min":0.62,"15min":0.69699999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39900000000000002,"15min":0.39400000000000002},"min":{"1min":0.29899999999999999,"5min":0.253,"15min":0.216},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":1.1819999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.219},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.19900000000000001},"max":{"1min":0.47899999999999998,"5min":0.55200000000000005,"15min":0.57399999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.44400000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.53900000000000003,"5min":0.67400000000000004,"15min":1.2070000000000001},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.88300000000000001,"15min":1.246},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.90200000000000002},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.2649999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.93100000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.32600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.90600000000000003,"15min":0.92200000000000004},"last":0.47099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307176,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064664,"statfs":{"total":96636764160,"available":96322215936,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.39600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.502,"5min":1.5589999999999999,"15min":2.008},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.58899999999999997,"5min":0.73399999999999999,"15min":6.4370000000000003},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39500000000000002,"15min":0.372},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.63100000000000001,"5min":1.601,"15min":2.1970000000000001},"last":0.46600000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41499999999999998,"15min":0.496},"min":{"1min":0.317,"5min":0.30199999999999999,"15min":0.255},"max":{"1min":0.67000000000000004,"5min":0.88400000000000001,"15min":5.069},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":2.246},"last":0.505}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":0.97899999999999998,"5min":1.6719999999999999,"15min":14.646000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90000000000000002,"5min":3.109,"15min":14.606999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":1.506,"15min":2.2959999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.72199999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":4.3150000000000004,"15min":6.4009999999999998},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.80600000000000005,"15min":0.76900000000000002},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":16.773,"15min":16.773},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77200000000000002,"15min":0.70899999999999996},"min":{"1min":0.45400000000000001,"5min":0.40600000000000003,"15min":0.40100000000000002},"max":{"1min":0.91200000000000003,"5min":11.436,"15min":11.436},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.72999999999999998},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.579,"15min":6.367},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":3.069,"15min":5},"last":0.747}]}]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307176,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064664,"statfs":{"total":96636764160,"available":96322215936,"internally_reserved":0,"allocated":1736704,"data_stored":791803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34699999999999998,"15min":0.29699999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.95799999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.25600000000000001},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":5.4569999999999999},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.5510000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.47399999999999998,"5min":0.36299999999999999,"15min":0.308},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":5.4900000000000002},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":4.0739999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.626},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":5.415},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58099999999999996,"15min":0.63200000000000001},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.33700000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":4.202},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":4.2130000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1279999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.53500000000000003},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.83299999999999996},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.31,"15min":0.29399999999999998},"max":{"1min":0.72499999999999998,"5min":0.80700000000000005,"15min":0.92600000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76300000000000001,"15min":0.99099999999999999},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.56799999999999995},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.872},"last":0.63},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":1.1000000000000001},"last":0.76200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307172,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064668,"statfs":{"total":96636764160,"available":96322220032,"internally_reserved":0,"allocated":1732608,"data_stored":790450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":4.9039999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.26800000000000002,"15min":0.247},"max":{"1min":0.752,"5min":0.88800000000000001,"15min":3.7679999999999998},"last":0.55200000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.31,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.873,"15min":6.7949999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.625},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.80500000000000005,"15min":4.8399999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.68100000000000005},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85099999999999998,"15min":4.9199999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":4.8170000000000002},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.79500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.30199999999999999},"max":{"1min":0.79800000000000004,"5min":0.93100000000000005,"15min":9.7210000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.79600000000000004},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":9.7539999999999996},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78800000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.54100000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.74399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":0.873},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.88700000000000001,"15min":0.92900000000000005},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:03:24.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T05:03:25.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.007+0000 7f3d94745700 1 Processor -- start 2023-10-06T05:03:25.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.007+0000 7f3d94745700 1 -- start start 2023-10-06T05:03:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d94745700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3d8c159d00 0x7f3d8c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d94745700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d8c0c7100 0x7f3d8c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d94745700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8c0cebd0 con 0x7f3d8c0cac40 2023-10-06T05:03:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d94745700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8c0cb9e0 con 0x7f3d8c0c7100 2023-10-06T05:03:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d94745700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8c0cbcc0 con 0x7f3d8c159d00 2023-10-06T05:03:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d8c0c7100 0x7f3d8c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d8c0c7100 0x7f3d8c0cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37078/0 (socket says 172.21.15.3:37078) 2023-10-06T05:03:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 -- 172.21.15.3:0/2058535681 learned_addr learned my addr 172.21.15.3:0/2058535681 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d924e1700 1 --2- 172.21.15.3:0/2058535681 >> v2:172.21.15.153:3300/0 conn(0x7f3d8c159d00 0x7f3d8c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 -- 172.21.15.3:0/2058535681 >> v2:172.21.15.153:3300/0 conn(0x7f3d8c159d00 msgr2=0x7f3d8c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 --2- 172.21.15.3:0/2058535681 >> v2:172.21.15.153:3300/0 conn(0x7f3d8c159d00 0x7f3d8c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 -- 172.21.15.3:0/2058535681 >> v1:172.21.15.153:6789/0 conn(0x7f3d8c0cac40 legacy=0x7f3d8c0c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 -- 172.21.15.3:0/2058535681 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0cbfa0 con 0x7f3d8c0c7100 2023-10-06T05:03:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d92ce2700 1 --2- 172.21.15.3:0/2058535681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c0cf000 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c088f10 tx=0x7f3d84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341dda00572ffe67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d914df700 1 -- 172.21.15.3:0/2058535681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400e070 con 0x7f3d8c0c7100 2023-10-06T05:03:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d914df700 1 -- 172.21.15.3:0/2058535681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8400ba50 con 0x7f3d8c0c7100 2023-10-06T05:03:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.008+0000 7f3d914df700 1 -- 172.21.15.3:0/2058535681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840092e0 con 0x7f3d8c0c7100 2023-10-06T05:03:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 -- 172.21.15.3:0/2058535681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 msgr2=0x7f3d8c0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 --2- 172.21.15.3:0/2058535681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c0cf000 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c088f10 tx=0x7f3d84009840 comp rx=0 tx=0).stop 2023-10-06T05:03:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 -- 172.21.15.3:0/2058535681 shutdown_connections 2023-10-06T05:03:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 --2- 172.21.15.3:0/2058535681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c0cf000 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 --2- 172.21.15.3:0/2058535681 >> v2:172.21.15.153:3300/0 conn(0x7f3d8c159d00 0x7f3d8c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 -- 172.21.15.3:0/2058535681 >> 172.21.15.3:0/2058535681 conn(0x7f3d8c087fa0 msgr2=0x7f3d8c06ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 -- 172.21.15.3:0/2058535681 shutdown_connections 2023-10-06T05:03:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.009+0000 7f3d94745700 1 -- 172.21.15.3:0/2058535681 wait complete. 2023-10-06T05:03:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.010+0000 7f3d94745700 1 Processor -- start 2023-10-06T05:03:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.010+0000 7f3d94745700 1 -- start start 2023-10-06T05:03:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d94745700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d94745700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d8c0cac40 0x7f3d8c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d94745700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c159d00 0x7f3d8c0cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d94745700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c151a50 con 0x7f3d8c0c7100 2023-10-06T05:03:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d94745700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0b5430 con 0x7f3d8c159d00 2023-10-06T05:03:25.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d94745700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0b5710 con 0x7f3d8c0cac40 2023-10-06T05:03:25.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37092/0 (socket says 172.21.15.3:37092) 2023-10-06T05:03:25.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 -- 172.21.15.3:0/2106180019 learned_addr learned my addr 172.21.15.3:0/2106180019 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:25.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d91ce0700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d8c0cac40 0x7f3d8c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d92ce2700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c159d00 0x7f3d8c0cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 -- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d8c0cac40 msgr2=0x7f3d8c0cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d8c0cac40 0x7f3d8c0cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 -- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c159d00 msgr2=0x7f3d8c0cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c159d00 0x7f3d8c0cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 -- 172.21.15.3:0/2106180019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84009ae0 con 0x7f3d8c0c7100 2023-10-06T05:03:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d92ce2700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c159d00 0x7f3d8c0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:03:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d924e1700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c151510 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00d840 tx=0x7f3d7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.011+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c014620 con 0x7f3d8c0c7100 2023-10-06T05:03:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.012+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c014780 con 0x7f3d8c0c7100 2023-10-06T05:03:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.012+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c0b59f0 con 0x7f3d8c0c7100 2023-10-06T05:03:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.012+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c014b40 con 0x7f3d8c0c7100 2023-10-06T05:03:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.012+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c0b5050 con 0x7f3d8c0c7100 2023-10-06T05:03:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.013+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c0b5e90 con 0x7f3d8c0c7100 2023-10-06T05:03:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.013+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d7c01d400 con 0x7f3d8c0c7100 2023-10-06T05:03:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.013+0000 7f3d8a7fc700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.013+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3d7c084d20 con 0x7f3d8c0c7100 2023-10-06T05:03:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.013+0000 7f3d91ce0700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.014+0000 7f3d91ce0700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c1563f0 tx=0x7f3d80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:25.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.018+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d7c052100 con 0x7f3d8c0c7100 2023-10-06T05:03:25.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.212+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3d8c0887e0 con 0x7f3d8c0c7100 2023-10-06T05:03:25.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.216+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3d7c088030 con 0x7f3d8c0c7100 2023-10-06T05:03:25.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.217+0000 7f3d8a7fc700 1 -- 172.21.15.3:0/2106180019 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v66)=0 v66) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3d7c055720 con 0x7f3d8c0c7100 2023-10-06T05:03:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 msgr2=0x7f3d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c1563f0 tx=0x7f3d80009040 comp rx=0 tx=0).stop 2023-10-06T05:03:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 msgr2=0x7f3d8c151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c151510 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00d840 tx=0x7f3d7c008040 comp rx=0 tx=0).stop 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 shutdown_connections 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f3d78066f30 0x7f3d780693e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c159d00 0x7f3d8c0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c0c7100 0x7f3d8c151510 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 --2- 172.21.15.3:0/2106180019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3d8c0cac40 0x7f3d8c0cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 >> 172.21.15.3:0/2106180019 conn(0x7f3d8c087fa0 msgr2=0x7f3d8c069330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 shutdown_connections 2023-10-06T05:03:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.219+0000 7f3d94745700 1 -- 172.21.15.3:0/2106180019 wait complete. 2023-10-06T05:03:25.233 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:03:25.234 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:03:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.350+0000 7f583b1cd700 1 Processor -- start 2023-10-06T05:03:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.350+0000 7f583b1cd700 1 -- start start 2023-10-06T05:03:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f583b1cd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f583409d980 0x7f583415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f583b1cd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 0x7f58340692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f583b1cd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340cd230 con 0x7f58340c6da0 2023-10-06T05:03:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f583b1cd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583415e4c0 con 0x7f583409d980 2023-10-06T05:03:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f583b1cd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cc660 con 0x7f58340cd6c0 2023-10-06T05:03:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f583976a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f58340c6da0 0x7f58340c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56340/0 (socket says 172.21.15.3:56340) 2023-10-06T05:03:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f583976a700 1 -- 172.21.15.3:0/1532798906 learned_addr learned my addr 172.21.15.3:0/1532798906 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-06T05:03:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f5833fff700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 0x7f58340692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f5838f69700 1 --2- 172.21.15.3:0/1532798906 >> v2:172.21.15.153:3300/0 conn(0x7f583409d980 0x7f583415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486765109 0 0) 0x7f58340cd230 con 0x7f58340c6da0 2023-10-06T05:03:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5824003570 con 0x7f58340c6da0 2023-10-06T05:03:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2763657456 0 0) 0x7f5824003570 con 0x7f58340c6da0 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 msgr2=0x7f58340692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 0x7f58340692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 >> v2:172.21.15.153:3300/0 conn(0x7f583409d980 msgr2=0x7f583415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 --2- 172.21.15.3:0/1532798906 >> v2:172.21.15.153:3300/0 conn(0x7f583409d980 0x7f583415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340cc900 con 0x7f58340c6da0 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3339196943 0 0) 0x7f582c0017f0 con 0x7f58340c6da0 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 >> v1:172.21.15.3:6789/0 conn(0x7f58340c6da0 legacy=0x7f58340c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5824003a60 0x7f5824003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5824004810 0x7f5824004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 0x7f58240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583415e4c0 con 0x7f5824004810 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cc660 con 0x7f5824003a60 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582400ba60 con 0x7f58340cd6c0 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.351+0000 7f5833fff700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5824004810 0x7f5824004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f5838f69700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5824003a60 0x7f5824003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f583976a700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 0x7f58240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f5833fff700 1 -- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 msgr2=0x7f58240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f5833fff700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58340cd6c0 0x7f58240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f5833fff700 1 -- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5824003a60 msgr2=0x7f5824003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:03:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f5833fff700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5824003a60 0x7f5824003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f5833fff700 1 -- 172.21.15.3:0/1532798906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5824003570 con 0x7f5824004810 2023-10-06T05:03:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f5833fff700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5824004810 0x7f5824004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5820009c00 tx=0x7f58200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d523e2da7978cdc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000e710 con 0x7f5824004810 2023-10-06T05:03:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f582000e870 con 0x7f5824004810 2023-10-06T05:03:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.352+0000 7f58337fe700 1 -- 172.21.15.3:0/1532798906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000ec50 con 0x7f5824004810 2023-10-06T05:03:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.353+0000 7f583b1cd700 1 -- 172.21.15.3:0/1532798906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5824004810 msgr2=0x7f5824004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.353+0000 7f583b1cd700 1 --2- 172.21.15.3:0/1532798906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5824004810 0x7f5824004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5820009c00 tx=0x7f58200091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.353+0000 7f583976a700 1 -- 172.21.15.3:0/1532798906 reap_dead start 2023-10-06T05:03:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.353+0000 7f583b1cd700 1 -- 172.21.15.3:0/1532798906 shutdown_connections 2023-10-06T05:03:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.353+0000 7f583b1cd700 1 -- 172.21.15.3:0/1532798906 >> 172.21.15.3:0/1532798906 conn(0x7f583406e730 msgr2=0x7f583406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.353+0000 7f583b1cd700 1 -- 172.21.15.3:0/1532798906 shutdown_connections 2023-10-06T05:03:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.353+0000 7f583b1cd700 1 -- 172.21.15.3:0/1532798906 wait complete. 2023-10-06T05:03:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.354+0000 7f583b1cd700 1 Processor -- start 2023-10-06T05:03:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.354+0000 7f583b1cd700 1 -- start start 2023-10-06T05:03:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5824004810 0x7f58340c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58341552f0 0x7f5834141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58340c9140 0x7f58340c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5834155c00 con 0x7f58341552f0 2023-10-06T05:03:25.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340ca850 con 0x7f58340c9140 2023-10-06T05:03:25.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340c7ce0 con 0x7f5824004810 2023-10-06T05:03:25.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58341552f0 0x7f5834141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58341552f0 0x7f5834141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37110/0 (socket says 172.21.15.3:37110) 2023-10-06T05:03:25.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 -- 172.21.15.3:0/4164158054 learned_addr learned my addr 172.21.15.3:0/4164158054 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-06T05:03:25.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5838f69700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5824004810 0x7f58340c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583976a700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58340c9140 0x7f58340c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 -- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5824004810 msgr2=0x7f58340c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5824004810 0x7f58340c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 -- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58340c9140 msgr2=0x7f58340c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58340c9140 0x7f58340c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 -- 172.21.15.3:0/4164158054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340c9f20 con 0x7f58341552f0 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5833fff700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58341552f0 0x7f5834141860 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f58200137a0 tx=0x7f58200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5830ff9700 1 -- 172.21.15.3:0/4164158054 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820022070 con 0x7f58341552f0 2023-10-06T05:03:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5830ff9700 1 -- 172.21.15.3:0/4164158054 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5820013b90 con 0x7f58341552f0 2023-10-06T05:03:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f5830ff9700 1 -- 172.21.15.3:0/4164158054 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000e710 con 0x7f58341552f0 2023-10-06T05:03:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58340ca200 con 0x7f58341552f0 2023-10-06T05:03:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.355+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5834141da0 con 0x7f58341552f0 2023-10-06T05:03:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.356+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583404c380 con 0x7f58341552f0 2023-10-06T05:03:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.357+0000 7f5830ff9700 1 -- 172.21.15.3:0/4164158054 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f582001c660 con 0x7f58341552f0 2023-10-06T05:03:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.357+0000 7f5830ff9700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:03:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.357+0000 7f5830ff9700 1 -- 172.21.15.3:0/4164158054 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5820085a00 con 0x7f58341552f0 2023-10-06T05:03:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.357+0000 7f5838f69700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:03:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.358+0000 7f5838f69700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5818066f30 0x7f58180693e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5828006ea0 tx=0x7f5828009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:03:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.361+0000 7f5830ff9700 1 -- 172.21.15.3:0/4164158054 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5820052f20 con 0x7f58341552f0 2023-10-06T05:03:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.598+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58340ccbe0 con 0x7f58341552f0 2023-10-06T05:03:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.598+0000 7f5830ff9700 1 -- 172.21.15.3:0/4164158054 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f582002bde0 con 0x7f58341552f0 2023-10-06T05:03:25.601 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-06T05:03:25.601 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7094,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2494924368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2494924368},{"type":"v1","addr":"172.21.15.153:6837","nonce":2494924368}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3103530178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103530178},{"type":"v1","addr":"172.21.15.153:6836","nonce":3103530178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/4024929308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4024929308},{"type":"v1","addr":"172.21.15.3:6834","nonce":4024929308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1899349246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":1899349246},{"type":"v1","addr":"172.21.15.3:6835","nonce":1899349246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[]} 2023-10-06T05:03:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5818066f30 msgr2=0x7f58180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5818066f30 0x7f58180693e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5828006ea0 tx=0x7f5828009120 comp rx=0 tx=0).stop 2023-10-06T05:03:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58341552f0 msgr2=0x7f5834141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:03:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58341552f0 0x7f5834141860 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f58200137a0 tx=0x7f58200091a0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 shutdown_connections 2023-10-06T05:03:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:6832/90310,v1:172.21.15.153:6833/90310] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58340c9140 0x7f58340c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5824004810 0x7f58340c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 --2- 172.21.15.3:0/4164158054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58341552f0 0x7f5834141860 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:03:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 >> 172.21.15.3:0/4164158054 conn(0x7f583406e730 msgr2=0x7f58340b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:03:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 shutdown_connections 2023-10-06T05:03:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-06T05:03:25.600+0000 7f583b1cd700 1 -- 172.21.15.3:0/4164158054 wait complete. 2023-10-06T05:03:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 93 2023-10-06T05:03:25.615 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T05:03:25.615 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T05:03:25.615 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:03:25.616 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.624 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:03:25.624 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:03:25.624 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.673 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:03:25.674 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:03:25.674 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.686 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:03:25.686 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:03:25.686 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.736 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:03:25.736 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T05:03:25.737 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T05:03:25.737 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.765 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T05:03:25.766 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T05:03:25.766 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.838 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T05:03:25.838 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T05:03:25.838 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.903 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T05:03:25.904 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T05:03:25.904 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:25.968 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T05:03:25.969 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T05:03:25.969 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.013 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T05:03:26.013 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T05:03:26.013 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.055 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T05:03:26.056 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T05:03:26.056 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:26.083 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:26.101 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T05:03:26.102 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T05:03:26.102 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.170 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T05:03:26.170 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T05:03:26.170 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T05:03:26.170 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.193 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T05:03:26.194 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T05:03:26.194 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.237 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T05:03:26.237 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T05:03:26.237 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T05:03:26.237 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.248 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T05:03:26.248 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T05:03:26.248 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.288 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T05:03:26.288 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T05:03:26.288 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:03:26.339 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T05:03:26.339 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T05:03:26.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-10-06T05:03:26.370 INFO:teuthology.orchestra.run.smithi003.stdout:1696567937.6259139 mon.a (mon.0) 136 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:03:26.371 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T05:03:26.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-10-06T05:03:26.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-10-06T05:03:26.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-10-06T05:03:26.584 INFO:teuthology.orchestra.run.smithi003.stdout:1696567937.6259139 mon.a (mon.0) 136 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:03:26.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:03:26.584 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T05:03:26.836 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:03:26.837 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T05:03:26.933 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:03:26.933 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T05:03:27.016 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:03:27.016 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T05:03:27.118 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:03:27.118 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T05:03:27.341 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:03:27.342 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T05:03:27.429 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:03:27.430 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T05:03:27.495 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:03:27.495 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T05:03:27.605 INFO:tasks.ceph:Archiving mon data... 2023-10-06T05:03:27.606 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/data/mon.a.tgz 2023-10-06T05:03:27.607 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2023-10-06T05:03:27.623 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.2EqRRRp3IA 2023-10-06T05:03:27.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2EqRRRp3IA 2023-10-06T05:03:27.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.2EqRRRp3IA 2023-10-06T05:03:27.856 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.2EqRRRp3IA is 385KB 2023-10-06T05:03:27.953 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.2EqRRRp3IA 2023-10-06T05:03:27.968 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/data/mon.b.tgz 2023-10-06T05:03:27.968 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2023-10-06T05:03:27.983 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.jeMZRGjkEr 2023-10-06T05:03:27.984 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jeMZRGjkEr 2023-10-06T05:03:28.129 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.jeMZRGjkEr 2023-10-06T05:03:28.210 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.jeMZRGjkEr is 386KB 2023-10-06T05:03:28.276 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.jeMZRGjkEr 2023-10-06T05:03:28.290 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/data/mon.c.tgz 2023-10-06T05:03:28.290 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2023-10-06T05:03:28.344 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.9YmZRYHDdf 2023-10-06T05:03:28.344 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9YmZRYHDdf 2023-10-06T05:03:28.476 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.9YmZRYHDdf 2023-10-06T05:03:28.557 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.9YmZRYHDdf is 386KB 2023-10-06T05:03:28.653 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.9YmZRYHDdf 2023-10-06T05:03:28.669 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T05:03:28.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:03:28.675 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:03:28.738 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T05:03:28.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:03:28.746 DEBUG:teuthology.orchestra.run.smithi153:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:03:28.773 INFO:teuthology.orchestra.run.smithi003.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:03:28.804 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:03:28.810 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T05:03:28.813 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/remote/smithi003/crash 2023-10-06T05:03:28.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:03:28.843 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/remote/smithi153/crash 2023-10-06T05:03:28.845 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:03:28.875 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T05:03:28.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:28.888 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:28.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:29.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:29.038 INFO:tasks.ceph:Compressing logs... 2023-10-06T05:03:29.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:03:29.105 DEBUG:teuthology.orchestra.run.smithi153:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:03:38.653 INFO:tasks.ceph:Archiving logs... 2023-10-06T05:03:38.653 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/remote/smithi003/log 2023-10-06T05:03:38.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:03:40.173 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/remote/smithi153/log 2023-10-06T05:03:40.174 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:03:41.258 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T05:03:41.268 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T05:03:41.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:03:41.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:03:41.346 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T05:03:41.357 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T05:03:41.357 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:03:41.359 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:03:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:bash: ntpq: command not found 2023-10-06T05:03:41.373 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-10-06T05:03:41.375 INFO:teuthology.orchestra.run.smithi003.stdout:210 Number of sources = 4 2023-10-06T05:03:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:03:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2023-10-06T05:03:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 +299us[ +299us] +/- 82ms 2023-10-06T05:03:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -357us[ -355us] +/- 55ms 2023-10-06T05:03:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +182us[ +182us] +/- 61ms 2023-10-06T05:03:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:03:41.377 INFO:teuthology.orchestra.run.smithi153.stdout:210 Number of sources = 4 2023-10-06T05:03:41.377 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:03:41.378 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-10-06T05:03:41.378 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 24 +312us[ +312us] +/- 82ms 2023-10-06T05:03:41.378 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -357us[ -340us] +/- 55ms 2023-10-06T05:03:41.378 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +179us[ +179us] +/- 61ms 2023-10-06T05:03:41.378 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:03:41.378 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T05:03:41.391 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T05:03:41.391 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T05:03:41.431 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:03:41.475 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:03:41.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:03:41.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:03:41.602 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:03:41.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:03:41.632 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T05:03:41.643 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T05:03:41.691 INFO:teuthology.task.internal:Duration was 1385.305674 seconds 2023-10-06T05:03:41.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T05:03:41.732 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T05:03:41.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:03:41.735 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:03:41.778 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:03:41.780 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:03:42.158 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T05:03:42.158 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2023-10-06T05:03:42.158 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:03:42.178 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2023-10-06T05:03:42.179 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:03:42.198 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T05:03:42.198 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:03:42.223 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:03:42.278 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T05:03:42.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:03:42.281 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:03:42.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T05:03:42.409 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T05:03:42.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:03:42.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:03:42.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T05:03:42.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:03:42.478 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:03:42.500 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2023-10-06T05:03:42.523 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2023-10-06T05:03:42.537 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:03:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:03:42.569 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:03:42.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:03:42.591 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T05:03:42.601 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T05:03:42.602 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/remote/smithi003 2023-10-06T05:03:42.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:03:42.644 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299/remote/smithi153 2023-10-06T05:03:42.644 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:03:42.684 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T05:03:42.684 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:03:42.687 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:03:42.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T05:03:42.746 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T05:03:42.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T05:03:42.782 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T05:03:42.782 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:03:42.785 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:03:42.797 INFO:teuthology.orchestra.run.smithi003.stdout: 262586 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:03 /home/ubuntu/cephtest 2023-10-06T05:03:42.798 INFO:teuthology.orchestra.run.smithi153.stdout: 262183 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:03 /home/ubuntu/cephtest 2023-10-06T05:03:42.800 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T05:03:42.827 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T05:03:42.916 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T05:03:42.943 DEBUG:teuthology.nuke:shortname: smithi003 2023-10-06T05:03:42.943 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:03:42.966 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:33:21.072999', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKbo8AaA0SBNBZE0ptrEHo7x74gvdsBIRnPvtAxdvksWWeYJ1dFzKIZy7Vyzh+Fj35g42ByfopqfFPStxzQBIU='} 2023-10-06T05:03:42.970 DEBUG:teuthology.nuke:shortname: smithi153 2023-10-06T05:03:42.970 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:03:42.991 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412299', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:33:21.071038', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGkRhRxkKOUwHgczTnlg0rpGLrnMsnPV1os71P+MSj+tcnfyIzqBkyl+OIJC6UJ7RsBIak0eR+BLBohXjbpT7s4='} 2023-10-06T05:03:43.022 INFO:teuthology.orchestra.console.smithi003:Power off 2023-10-06T05:03:43.022 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:03:43.046 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-10-06T05:03:43.047 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:43.062 INFO:teuthology.orchestra.console.smithi153:Power off 2023-10-06T05:03:43.062 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:03:43.072 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-06T05:03:43.087 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2023-10-06T05:03:43.087 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:43.112 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2023-10-06T05:03:47.073 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:47.114 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:47.183 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-06T05:03:47.224 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2023-10-06T05:03:51.186 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:51.225 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:51.296 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-06T05:03:51.334 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2023-10-06T05:03:55.298 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:55.336 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:03:55.409 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-06T05:03:55.445 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2023-10-06T05:03:55.510 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-10-06T05:03:55.546 INFO:teuthology.orchestra.console.smithi153:Power off completed 2023-10-06T05:03:55.651 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} duration: 1385.3056735992432 failure_reason: '"1696567937.6259139 mon.a (mon.0) 136 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T05:03:55.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T05:03:55.737 INFO:teuthology.run:FAIL